Search code examples
chronicle-queuelinkedblockingqueue

Does a ChronicleQueue block on reads after a tailer has retrieved the most recently written excerpt?


I too am beginning to look at ChronicleQueue.

From the documentation:

Reading the queue follows the same pattern as writing, except there is a possibility there is not a message when you attempt to read it

I am trying to understand what happens when there is no message. The beauty of a LinkedBlockingQueue is that it just works, with little CPU load, when one thread is putting items into a queue (say, from a stream) and another thread is consuming items (yes, removing elements as they arrive and processing them in small groups). I would like to take advantage of the mapped files for history for raw items from the stream and the performance of a ChronicleQueue, but it seems that to replicate the behavior of a LinkedBlockingQueue my code will have to keep track of excerpt indices to know what excerpts have been processed - either when they arrive (placed in queue), or through some machinations using replay in the code that actually process the content of an excerpt. That said, I suspect I am missing something fundamental in my understanding of appender/tailer behavior. Any insights would be appreciated.


Solution

  • Chronicle Queue is designed for low-latency, single-threaded environments and hence it never blocks on any operations.

    Therefore, the usual pattern of reading from the queue involves busy-spinning (or various types of pausers, the simplest being Thread.yield()), e.g.:

    while (true) {
        try (DocumentContext dc = tailer.readingDocument()) {
            // false here means there's no data to read, so we just keep waiting
            if (dc.isPresent()) {
                 Bytes bytes = dc.wire().bytes();
                // read from bytes
            }
        }
    }
    

    Unless you want to return to certain items in your queue later, you don't need to bother with indices or anything of this sort.

    NB: busy-spinning is opposed to blocking and it WILL consume 100% of CPU, and that's a deliberate trade-off for low-latency solutions.