Search code examples
chroniclechronicle-queue

Chronicle queue named appender to get the most recent roll cycle file for iteration returns earliest file

I am trying to implement a named tailer to pickup where a task was previously left off. But in between continuing from a task, I may want to check the contents of the most recent roll cycle. I then tried to test if I can get the most recent roll cycle to iterate over (named tailers pick up where left off so I think it should iterate most recent file) and this is the output:

output

The code is as follows:

    public static void main(String[] args) {
        ChronicleQueue QUEUE = SingleChronicleQueueBuilder.single("./chronicle/roll").rollCycle(RollCycles.MINUTELY).build();
        ExcerptAppender APPENDER = QUEUE.acquireAppender();
        ExcerptTailer TAILER = QUEUE.createTailer("a");

        //this reads all roll cycles starting from first and carries on to next rollcycle.
        //busy spinner that spins non-stop trying to read from queue
        while(true){
            System.out.println(TAILER.cycle());
            if (TAILER.readDocument(w -> w.read("packet").marshallable(
                    m -> {
                        System.out.println(m.read("moldUdpHeader").text());
                    }
            ))){ ; } else { TAILER.close(); break; }
            //breaks out from spinner if nothing to be read.
            //a named tailer could be used to pick up from where is left off.

        }

        //by calling TAILER.approximateExcerptsInCycle(TAILER.cycle()) inside the for-loops condition statement,
        //it winds the index back to start of the cycle, so TAILER will constantly only read first item
        ExcerptTailer tailer = QUEUE.createTailer("a");
        tailer.moveToCycle(tailer.cycle());
        System.out.println(tailer.cycle());
        long excerpts = tailer.approximateExcerptsInCycle(tailer.cycle());  //gets the number of records for earliest roll cycle available
        System.out.println(excerpts);
        for (int i=0; i< excerpts; i++){
            System.out.println(tailer.cycle());
            tailer.readDocument(w -> w.read("packet").marshallable(
                    m -> {
                        System.out.println(m.read("moldUdpHeader").text());
                    }
            ));
        }
        tailer.close();

Explanation of output:

I ran the while loop, so it went to the end of the tailer and there is nothing to further iterate. So it prints the cycle, but does not print any documents then breaks.

I then try to move to start of the cycle with .moveToCycle(TAILER.cycle());, it prints and shows that I am still on most recent (last cycle 27927733), I print the number of excerpts (6) inside the tailer and enter the for loop. Inside the for loop, it outputs values from the earliest documents from the first chronicle roll cycle file (27927664). Although the first line prints 27927733 before printing output, the output is also from the first roll cycle.

Chronicle Queue POM:

<!-- https://mvnrepository.com/artifact/net.openhft/chronicle-queue -->
        <dependency>
            <groupId>net.openhft</groupId>
            <artifactId>chronicle-queue</artifactId>
            <version>5.24ea7</version>
        </dependency>
Solution

  • I have checked out this SO question and approached the problem from a different perspective. Now, I create a new tailer (not named) move to end, get the approximate excerpts in cycle (from code above), move to the first index (approximate*) of the last available roll cycle, then iterate.

    This may not always get the first item in the roll cycle, as it is an approximate. Please still do note the weird behaviour mentioned in above question.

        tailer.toEnd();
    long lastIndex = tailer.index();
    long excerpts = tailer.approximateExcerptsInCycle(tailer.cycle());
    tailer.moveToIndex(lastIndex - excerpts);