Virtual threads in Java have sparked significant excitement in the developer community. They promise lightweight thread management and improved scalability for services that handle millions of users. However, effectively utilising virtual threads requires a solid understanding of their strengths, limitations, and best use cases. This blog post dives deep into the nuances of virtual threads, offering key learnings and practical advice for developers.
What Are Virtual Threads?
Virtual threads, introduced as part of Project Loom, are lightweight threads that aim to simplify concurrency in Java. Unlike traditional platform threads, virtual threads are not tied directly to OS-level threads. Instead, they are managed by the Java Virtual Machine (JVM), enabling the creation of millions of threads without overwhelming system resources.
Traditional threads in frameworks like Spring MVC create multiple thread-local variables, such as request contexts and security contexts. This works well for limited thread counts but becomes memory-intensive when scaled to millions of threads. When you create a thread, it will allocate a certain amount of memory. For example, on my MacBook M3 Pro late-2023 with 36GB RAM. If I do not change any settings, each thread takes 2 MB of memory on the stack, which is the default JVM setting for MacOS. To test this, I implemented a simple code example to count how many threads I could create on my laptop. The result was slightly over 9,000 threads—assuming all the memory was used exclusively for threads.
while (true) {
Thread.ofPlatform().start( () -> {
try { // Make the thread sleep to simulate work Thread.sleep(Long.MAX_VALUE);
} catch (InterruptedException e) {
Thread.currentThread().interrupt();
}
});
threadCount++;
log.info("Thread #{} created.", threadCount);
}
This illustrates how traditional threads quickly hit scalability limits due to memory constraints (specifically stack size). Virtual threads, previewed in Java 19 and officially available starting with Java 21, address this challenge by dramatically reducing the overhead associated with thread management.
Virtual threads enable readable sequential code with understandable flow, support over 1 million connections on standard hardware, and offer a simpler alternative to asynchronous or reactive programming patterns.
Key Advantages of Virtual Threads
Scalability: Virtual threads enable applications to handle significantly higher concurrency. Their lightweight nature can greatly benefit tasks like handling HTTP requests or other I/O-bound operations.
Think of them as „thread per request“, „thread per task“, „thread per connection“
Automatic Cleanup with Scoped Values: Scoped values offer an efficient alternative to thread-local variables. They offer better sharing of context and automatic cleanup, reducing the manual effort required for resource management.
Improved Resource Utilization: Virtual threads yield control during I/O operations, allowing the CPU to be utilised more effectively by other tasks. This makes them well-suited for I/O-bound workloads.
(Continuations: The Foundation of Virtual Threads)
Virtual threads wrap tasks using a low-level construct called a Continuation. Continuations, while powerful, are not meant to be used directly by developers (as evidenced by their placement in jdk.internal.vm.Continuation). Instead, they act as the building blocks for virtual threads, enabling lightweight concurrency.
Here’s how it works:
A Continuation can yield when a task needs to block (e.g., during an I/O operation).
The same continuation resumes when the task is ready to proceed, maintaining its state seamlessly.
Example in the code:
public static void main(String[] args) {
var cont = getContinuation();
cont.run();
cont.run();
}
private static Continuation getContinuation() {
ContinuationScope scope = new ContinuationScope("demo");
return new jdk.internal.vm.Continuation(scope, () -> {
System.out.println("Let's count:");
System.out.println("1");
Continuation.yield(scope); // Stop execution
System.out.println("2");
Continuation.yield(scope); // Stop execution
System.out.println("3");
});
}
How Continuation works with memory
When you run the continuation and then yield it, the platform copies the continuation from the Stack to the Heap, freeing space in the Stack.
When you resume the continuation, it is moved back from the Heap to the Stack.
This example demonstrates the core mechanics of continuations: pausing and resuming execution as needed. Virtual threads leverage this capability to efficiently manage large numbers of concurrent tasks without the overhead of traditional threads.
If we run the code to get the maximum number of threads using virtual threads (code is below) and 256 megabytes of RAM, my laptop would handle approximately 300,000 threads. Increasing the RAM to 1 GB allows for 1.5 million threads. The highest number I achieved on my laptop was 17 million threads. This showcases the incredible scalability potential of virtual threads when compared to traditional threading models.
while (true) {
Thread.ofVirtual().start(() -> {
try {
// Make the thread sleep to simulate work
Thread.sleep(Long.MAX_VALUE);
} catch (InterruptedException e) {
Thread.currentThread().interrupt();
}
});
threadCount++; log.info("VirtualThread #{} created.", threadCount);
}
Common Pitfalls and Misconceptions
While virtual threads bring many advantages, there are several pitfalls and misconceptions to be aware of:
Thread pinning: Even with virtual threads, performing blocking operations within synchronised blocks can lead to thread pinning, where a virtual thread becomes bound to a platform thread. In the worst-case scenario, this effectively degrades performance to the same level as using platform threads. This reduces scalability and can lead to unexpected bottlenecks.
Memory Overhead from Thread Locals: Traditional thread locals, such as those used for request or security contexts, can cause memory exhaustion when scaled to millions of threads. Developers should consider alternatives like scoped values, which are more efficient, provide immutable, scoped context, support better data sharing across threads, and automatically clean up and reduce the risk of memory leaks.
Misplaced Expectations: Virtual threads optimise concurrency but do not inherently improve performance for CPU-bound tasks. For instance, heavy computations or number-crunching inside virtual threads can block the underlying platform threads, negating the benefits of virtualisation.
How to Detect Thread Pinning?
Detecting pinning in virtual threads presents a significant challenge. To address this, developers have limited options. One approach is to log thread activity using the JVM parameter -Djdk.tracePinnedThreads=full. This provides detailed information about thread behavior, allowing for potential identification of pinned threads.
Alternatively, leveraging the JFR (Java Flight Recorder) event "jdk.VirtualThreadPinned" can offer insights into pinning occurrences.
/** * Hooks event listener to JFR thread pinned event */
public RecordingStream listenToPinnedEvent() {
var rs = new RecordingStream();
rs.setMaxAge(Duration.ofSeconds(10));
rs.enable("jdk.VirtualThreadPinned").withStackTrace();
rs.onEvent("jdk.VirtualThreadPinned", event -> {
log.warn("Pinned thread {} event No.{}: {}", event.getThread().getJavaName(),
counter.incrementAndGet(), event);
});
rs.startAsync();
return rs;
}
By analyzing these logs or JFR events, developers can attempt to pinpoint instances of pinning and investigate their root causes.
Best Practices and Future of Virtual Threads
Choose the Right Use Cases: Virtual threads are ideal for I/O-bound operations, such as handling REST API requests. Avoid using them for CPU-intensive tasks, where traditional thread pools still perform better.
Minimise Pinning: Be cautious with synchronised blocks or any operation that may inadvertently cause pinning. Use tools like Semaphore to manage concurrency more effectively. Should be resolved on platform level in java 25.
Leverage Structured Concurrency: Structured concurrency, a forthcoming feature in Java, simplifies task management by allowing developers to manage multiple tasks as a unit. For example, if one task in a group fails, the framework can handle rollback or recovery.
Monitor Performance: Use tools like Java Flight Recorder to identify bottlenecks and optimise performance. Virtual threads eliminate certain bottlenecks but do not resolve all performance challenges in complex systems.
The Java ecosystem continues to evolve, and virtual threads are no exception. Upcoming improvements in Java 24 and Java 25 include:
Enhanced I/O Event Handling: Optimized performance for I/O-bound operations.
Long-Term Support (LTS) Stability: Java 25, an LTS release, is expected to incorporate significant refinements, making virtual threads more robust and production-ready.
Until these improvements are fully realised, developers should carefully evaluate their use cases and stick to best practices to maximise the benefits of virtual threads.
Final Thoughts
Virtual Threads represent a pivotal advancement in Java, potentially marking the most significant change since the introduction of lambdas. By decoupling threads from operating system resources, VTs unlock unparalleled concurrency, enabling developers to write highly responsive and efficient applications. With current changes in the platform, it is expected that reactive programming will be replaced by structured concurrency, allowing for similar performance while bringing better readability. While challenges like pinning and debugging still require attention, the potential benefits of VTs are immense.
