Java Introduces Virtual Threads

In this article we discuss virtual threads that were first introduced in OpenJDK 19, and are now in a second preview in OpenJDK 20. To better understand virtual threads, it is best to draw a conceptual comparison with how an operating system can map a large amount of virtual memory to a limited amount of physical RAM. Java’s concurrency model is based on threads; multiple code segments can run independent of each other with each code running sequentially in its own thread. A server application can receive multiple requests, and it assigns a thread to each request. This is called a thread-per-request style.

Problem

As the number of requests an application receives grows, it needs to scale proportionally by assigning as many threads based on the thread-per-request style of handling concurrent requests. Up to a certain limit that is largely defined by the number of available operating system threads, an application is able to follow the thread-per-request style and assign new threads as new requests arrive. But when the operating system reaches its limit of available threads, a server application is not able to assign new threads to new requests.

Solution

This is where virtual threads come in handy. Virtual threads map a limited number of physical operating system threads to a much greater number of virtual threads. In other words, the number of virtual threads is much greater than the actual number of physical operating system threads.

How Could Virtual Threads Be More than the Operating System Threads?

The way the Java runtime is able to provide more virtual threads than the available operating system threads is that a virtual thread consumes an operating system thread for only the short duration during which it needs to perform some calculation on the CPU. In other words, unlike a platform thread, a virtual thread is not tied to a single operating system thread. The ratio of virtual threads to operating system threads could be very large depending on the duration for which a virtual thread uses an operating system thread; it could be as high as 10,000:1, or even higher.

Virtual Threads Are Transparent to the Application User

To a user sending a request to an application, the code still runs in the thread-per-request paradigm, and code still has a thread assigned to it for the entire duration of its run. The difference is that after a request has completed running the code it needs to run, the operating system thread that was being used as a virtual thread to perform a calculation is released back to the operating system, perhaps to be used by a different virtual thread.

When to Use Virtual Threads

Virtual threads may not always be the best approach to running multiple user requests concurrently. Virtual threads provide the greatest benefit under the scenarios:

• The number of concurrent requests, or tasks, is quite high—perhaps a few thousand.
• The workload is not CPU-bound. In other words, the code is not CPU intensive. If the code makes ample use of the CPU, each virtual thread could take up, or use, an operating system thread to the same effect as if the OS thread were tied to the virtual thread. If the workload is very CPU-bound, using the traditional platform threads may be a better option.

Virtual Threads Are Verisimilar to Platform Threads

The virtual threads are verisimilar, though not the exact same, to platform threads.

• Virtual threads can run any code that a platform thread can run.
• Virtual threads support thread-local variables and thread interruption just as platform threads do.
• Virtual threads do support thread pooling, but there is no reason to use thread pooling with virtual threads because virtual threads are plentiful. Platform threads, however, could benefit from thread pooling by limiting concurrency within a thread pool, which defines a limited thread resource.