Search code examples
javamultithreadingsynchronizationoverhead

Can you push synchronization costs onto one thread?

I have two threads: a primary thread that does the main processing of the application, and a secondary thread that receives data batches from the primary thread and processes and outputs them, either to the user, to a file, or over the network. In general, data should be processed at a much faster rate than it is produced. I would like to ensure that the main thread never waits for the secondary thread. The secondary thread can accept any amount of overhead, expanding buffers, redoing work, and so on, with the sole objective of maximizing performance of the main thread. Ideally the main thread will never synchronize at all. Is there any way to push synchronization costs onto one thread in Java?

Solution

  • This is an outline of a solution:

    1. The main thread works in isolation for some time, piling up data into a collection;

    2. when it has generated a nice batch, it:

      i. creates a new collection for itself;

      ii. sets the filled-up collection aside, available to be picked up by the reading thread;

      iii. CASes this collection into an AtomicReference.

    3. The reading thread polls this AtomicReference for updates;

    4. when it notices it has been set, it picks up the batch, CASing null into the shared reference, so that the main thread knows it can put another collection in.

    This has negligible coordination costs for the main thread (just one CAS operation per batch) assuming that the reference is always already null when it's time to share a new batch.

    The reading thread may run a busy loop polling the shared reference, sleeping a small amount of time each time it reads null. The best technique to make the thread sleep for a really short time is

    LockSupport.parkNanos(1);

    which will typically sleep for some 30 µs and the whole loop will consume about 2-3% CPU time. You could use a longer pause, of course, if you want to bring down the CPU time even more.

    Note that coordination techniques which make the thread wait in a wait set impose a very large latency on both sides, so you should stay away from them if, say, 1 ms latency is a big concern for you.