What is the nextSequenceToken for the PutLogEvents API for AWS.CloudWatchLogs

Question

What is the nextSequenceToken mentioned here?

https://docs.aws.amazon.com/AmazonCloudWatchLogs/latest/APIReference/API_PutLogEvents.html

Unclear to me, and probably others.

Answer 1

This question (and answer) are obsolete

I'm leaving the answer here, because it was relevant in 2019 when the question was asked. See @ErmiyaEskandary answer below for more information.

---

Each batch of events sent to CloudWatch Logs must include a sequence token (from the PutLogEvents API doc):

{
   "logEvents": [ 
      { 
         "message": "string",
         "timestamp": number
      }
   ],
   "logGroupName": "string",
   "logStreamName": "string",
   "sequenceToken": "string"
}

The response returns nextSequenceToken (again from the API docs):

{
   "nextSequenceToken": "string",
   "rejectedLogEventsInfo": { 
      "expiredLogEventEndIndex": number,
      "tooNewLogEventStartIndex": number,
      "tooOldLogEventEndIndex": number
   }
}

So the short answer to your question is: If you have a single producer writing to a stream, you can save nextSequenceToken and use it to populate sequenceToken for your next PutLogEvents request.

The longer answer is that you can't use this technique if you have multiple producers, because a producer doesn't have access to the response to another producer's request. Instead, you must call DescribeLogStreams before each request. The following code is extracted from a Java logging framework that I wrote (so includes references to functions that aren't shown here, and may contain syntax errors because I elided stuff specific to the logging library):

/**
 *  This function retrieves the current information for a specific log stream.
 *  DescribeLogStreams is a paginated operation, which means that we have to
 *  be prepared for a large number of rows in the response, but since we're
 *  passing the full stream name as a prefix this should never happen.
 */
private LogStream findLogStream(String logGroupName, String logStreamName)
{
    DescribeLogStreamsRequest request = new DescribeLogStreamsRequest()
                                        .withLogGroupName(logGroupName)
                                        .withLogStreamNamePrefix(logStreamName);
    DescribeLogStreamsResult result;
    do
    {
        result = client.describeLogStreams(request);
        for (LogStream stream : result.getLogStreams())
        {
            if (stream.getLogStreamName().equals(logStreamName))
                return stream;
        }
        request.setNextToken(result.getNextToken());
    } while (result.getNextToken() != null);
    return null;
}

/**
 *  This function tries to send a batch of messages, retrieving the sequence
 *  number for each batch and handling the data race if another process has
 *  made that sequence number invalid.
 */
private List<LogMessage> attemptToSend(List<LogMessage> batch)
{
    if (batch.isEmpty())
        return batch;

    PutLogEventsRequest request = new PutLogEventsRequest()
                                  .withLogGroupName(config.logGroupName)
                                  .withLogStreamName(config.logStreamName)
                                  .withLogEvents(constructLogEvents(batch));

    for (int ii = 0 ; ii < 5 ; ii++)
    {
        LogStream stream = findLogStream();

        try
        {
            request.setSequenceToken(stream.getUploadSequenceToken());
            client.putLogEvents(request);
            return Collections.emptyList();
        }
        catch (InvalidSequenceTokenException ex)
        {
            stats.updateWriterRaceRetries();
            Utils.sleepQuietly(100);
            // continue retry loop
        }
        catch (DataAlreadyAcceptedException ex)
        {
            reportError("received DataAlreadyAcceptedException, dropping batch", ex);
            return Collections.emptyList();
        }
        catch (Exception ex)
        {
            reportError("failed to send batch", ex);
            return batch;
        }
    }

    reportError("received repeated InvalidSequenceTokenException responses -- increase batch delay?", null);
    stats.updateUnrecoveredWriterRaceRetries();
    return batch;
}

Most of the exceptions that you'll get back from the PutLogEvents request are unrecoverable, so this code ignores them. InvalidSequenceTokenException, however, indicates that there was a race between two producers, and another producer was able to write a batch between the time that this producer retrieved the stream description and tried to write its batch. This is unlikely but possible, so it makes a few retries and then rejects the batch (it's requeued for another attempt).

There's one last piece of the response that may be important to you: CloudWatch has rules about the timestamps on events in a batch (not too far in the past or future). If your batch contains events that are outside of that range, they will be dropped but the rest of the events will be added to the stream. You can see if this happens by looking at the rejectedLogEventsInfo object in the response, which will have non-zero indexes if any records were dropped (for the logging framework, that's not likely to happen, and there's no correcting for it, so I just ignore that response value).