Motivation: Trying to migrate Virtual Threads.
Problem: Even though Virtual Threads are cheap, OS may find it suspicious for some processes to be stacked up at the same time, like searching for IP or ports on the network.
I am using the below code to limit the creation of threads. TS_NetworkIPUtils TS_NetworkPortUtils
var executor = useVirtualThread
? Executors.newVirtualThreadPerTaskExecutor()
: Executors.newFixedThreadPool(MAX_THREAD_COUNT());
Is it possible to create an executor service for creating virtual threads and having a limiting feature at the same time?
Is it possible to create an executor service for creating virtual threads and having limiting feature at the same time?
Yes.
Pile up many virtual threads, blocked while waiting for a counting semaphore to be released by any of the limited number of currently-executing tasks.
I am using below code to limit the creation of threads. TS_NetworkIPUtils
No, do not limit the number of threads. Virtual threads are by design extremely “cheap”, taking little in the way system resources (memory, CPU) by default. You likely can support millions of virtual threads at a time.
When your tasks are expensive, you may need to limit the amount of simultaneous tasks. Go ahead and stack up many threads, but block their execution to wait upon a certain number of concurrent tasks busy executing. Blocking on virtual threads is extremely cheap (their raison d’être), so no problem if they pile up.
You can use a counting semaphore to limit the number of simultaneous tasks running in virtual threads. See section Rate Limiting in tutorial Virtual Threads by Cay Horstmann.
Something like the following example. We submit 15 tasks, but limit to only 5 or less executing simultaneously. The new Semaphore ( 5 ) provides a bucket of five permits that may acquired and released. The permits are like a total of five copies of a book being borrowed from a library and eventually returned for someone else to borrow — lots of borrowers, but only five at a time.
Notice how the later tasks are waiting several seconds for a permit from our counting semaphore.
Finally, after all the submitted tasks are done, we exit our try-with-resources syntax block.
(Caveat: I just whipped out this code example, without much thought behind it. I was shooting for something simpler than the example seen in the linked Cay Horstmann tutorial.)
package work.basil.example.threading;
import java.time.Duration;
import java.time.Instant;
import java.util.SequencedCollection;
import java.util.concurrent.*;
import java.util.concurrent.locks.LockSupport;
public class LimitTasks
{
public static void main ( String[] args )
{
LimitTasks app = new LimitTasks ( );
app.demo ( );
}
private void demo ( )
{
System.out.println ( "Demo start. " + Instant.now ( ) );
SequencedCollection < String > log = new CopyOnWriteArrayList <> ( ); // System.out.println calls across threads do *NOT* necessarily appear on console in chronological order.
final Semaphore TASKS_PERMITS = new Semaphore ( 5 );
log.add ( "Permits available: " + TASKS_PERMITS.availablePermits ( ) );
try (
ExecutorService executorService = Executors.newVirtualThreadPerTaskExecutor ( ) ;
)
{
for ( int nthTask = 1 ; nthTask <= 15 ; nthTask++ )
{
executorService.submit ( new PrintIt ( nthTask , TASKS_PERMITS , log ) );
}
}
log.add ( "After try-with-resources on ExecutorService. " + Instant.now ( ) );
log.forEach ( System.out :: println );
System.out.println ( "Demo done. " + Instant.now ( ) );
}
}
class PrintIt implements Runnable
{
private final int label;
private final Semaphore tasksPermits;
private final SequencedCollection < String > log;
public PrintIt ( final int label , final Semaphore tasksPermits , final SequencedCollection < String > log )
{
this.label = label;
this.tasksPermits = tasksPermits;
this.log = log;
}
@Override
public void run ( )
{
this.log.add ( "nthTask " + label + ". Approximately " + this.tasksPermits.getQueueLength ( ) + " tasks waiting for a permit. " + Instant.now ( ) );
long startNanos = System.nanoTime ( );
try
{
this.tasksPermits.acquire ( );
long elapsed = System.nanoTime ( ) - startNanos ;
this.log.add ( "nthTask " + label + " waited " + Duration.ofNanos ( elapsed ) + " for a permit. Permits available: " + this.tasksPermits.availablePermits ( ) + ". " + Instant.now ( ) );
long low = Duration.ofSeconds ( 2 ).toNanos ( );
long high = Duration.ofSeconds ( 10 ).toNanos ( );
long random = ThreadLocalRandom.current ( ).nextLong ( low , high );
LockSupport.parkNanos ( random );
this.log.add ( "nthTask " + label + " running at " + Instant.now ( ) + ". Permits available: " + this.tasksPermits.availablePermits ( ) + "." );
} catch ( InterruptedException e )
{
throw new RuntimeException ( e );
} finally
{
this.tasksPermits.release ( );
}
}
}
When run on Java 21 on a Mac with Apple Silicon from IntelliJ IDEA 2023.3.3 (Ultimate Edition).
Demo start. 2024-02-06T04:42:12.414617Z
Permits available: 5
nthTask 3. Approximately 0 tasks waiting for a permit. 2024-02-06T04:42:12.425896Z
nthTask 4. Approximately 0 tasks waiting for a permit. 2024-02-06T04:42:12.425813Z
nthTask 7. Approximately 0 tasks waiting for a permit. 2024-02-06T04:42:12.427279Z
nthTask 1. Approximately 0 tasks waiting for a permit. 2024-02-06T04:42:12.425813Z
nthTask 2. Approximately 0 tasks waiting for a permit. 2024-02-06T04:42:12.425813Z
nthTask 9. Approximately 0 tasks waiting for a permit. 2024-02-06T04:42:12.428066Z
nthTask 10. Approximately 0 tasks waiting for a permit. 2024-02-06T04:42:12.428092Z
nthTask 8. Approximately 0 tasks waiting for a permit. 2024-02-06T04:42:12.427894Z
nthTask 5. Approximately 0 tasks waiting for a permit. 2024-02-06T04:42:12.425983Z
nthTask 6. Approximately 0 tasks waiting for a permit. 2024-02-06T04:42:12.426432Z
nthTask 11. Approximately 5 tasks waiting for a permit. 2024-02-06T04:42:12.435172Z
nthTask 12. Approximately 5 tasks waiting for a permit. 2024-02-06T04:42:12.435178Z
nthTask 13. Approximately 7 tasks waiting for a permit. 2024-02-06T04:42:12.435214Z
nthTask 14. Approximately 7 tasks waiting for a permit. 2024-02-06T04:42:12.435225Z
nthTask 15. Approximately 9 tasks waiting for a permit. 2024-02-06T04:42:12.435261Z
nthTask 3 waited PT0.0088205S for a permit. Permits available: 0. 2024-02-06T04:42:12.434720Z
nthTask 2 waited PT0.008820167S for a permit. Permits available: 1. 2024-02-06T04:42:12.434640Z
nthTask 4 waited PT0.008743834S for a permit. Permits available: 1. 2024-02-06T04:42:12.434705Z
nthTask 7 waited PT0.007267583S for a permit. Permits available: 1. 2024-02-06T04:42:12.434666Z
nthTask 1 waited PT0.008801666S for a permit. Permits available: 2. 2024-02-06T04:42:12.434619Z
nthTask 2 running at 2024-02-06T04:42:15.306355Z. Permits available: 0.
nthTask 9 waited PT2.887408834S for a permit. Permits available: 0. 2024-02-06T04:42:15.315471Z
nthTask 3 running at 2024-02-06T04:42:17.420189Z. Permits available: 0.
nthTask 10 waited PT4.9929405S for a permit. Permits available: 0. 2024-02-06T04:42:17.421040Z
nthTask 7 running at 2024-02-06T04:42:17.920249Z. Permits available: 0.
nthTask 8 waited PT5.493109666S for a permit. Permits available: 0. 2024-02-06T04:42:17.921006Z
nthTask 4 running at 2024-02-06T04:42:18.204203Z. Permits available: 0.
nthTask 5 waited PT5.779108792S for a permit. Permits available: 0. 2024-02-06T04:42:18.205082Z
nthTask 9 running at 2024-02-06T04:42:19.479390Z. Permits available: 0.
nthTask 6 waited PT7.053674S for a permit. Permits available: 0. 2024-02-06T04:42:19.480079Z
nthTask 1 running at 2024-02-06T04:42:19.800778Z. Permits available: 0.
nthTask 11 waited PT7.366630625S for a permit. Permits available: 0. 2024-02-06T04:42:19.801791Z
nthTask 10 running at 2024-02-06T04:42:20.015210Z. Permits available: 0.
nthTask 12 waited PT7.580567167S for a permit. Permits available: 0. 2024-02-06T04:42:20.015699Z
nthTask 11 running at 2024-02-06T04:42:24.640389Z. Permits available: 0.
nthTask 13 waited PT12.205975708S for a permit. Permits available: 0. 2024-02-06T04:42:24.641142Z
nthTask 8 running at 2024-02-06T04:42:26.301821Z. Permits available: 0.
nthTask 14 waited PT13.867504542S for a permit. Permits available: 0. 2024-02-06T04:42:26.302663Z
nthTask 12 running at 2024-02-06T04:42:26.407675Z. Permits available: 0.
nthTask 15 waited PT13.972995459S for a permit. Permits available: 0. 2024-02-06T04:42:26.408182Z
nthTask 5 running at 2024-02-06T04:42:27.728516Z. Permits available: 0.
nthTask 6 running at 2024-02-06T04:42:28.044373Z. Permits available: 1.
nthTask 15 running at 2024-02-06T04:42:29.974056Z. Permits available: 2.
nthTask 13 running at 2024-02-06T04:42:30.613859Z. Permits available: 3.
nthTask 14 running at 2024-02-06T04:42:33.088984Z. Permits available: 4.
After try-with-resources on ExecutorService. 2024-02-06T04:42:33.089508Z
Demo done. 2024-02-06T04:42:33.094512Z
Process finished with exit code 0
For more info on virtual threads, see the fascinating video talks by Ron Pressler, Alan Bateman, or José Paumard. And read the JEP 444.