Handling duplicate notifications from superfeedr

Question

I have a subscription to an RSS feed via SuperFeedr, and have noticed a lot of duplicate notifications lately. I've double checked my subscriptions via the web application, but I only see 1. I also looked into making sure that I am returning the correct response code. The documentation says that a 200 should be sent as the response, and the retry structure is after 5, 10, and 15 seconds, but in my logs, I see the duplicates being sent milliseconds apart, which leads me to believe that these are not retries.

The very FIRST lines of my doPost() method are:

    response.setStatus(HttpServletResponse.SC_OK);
    response.flushBuffer();

The attempt there is to let superfeedr know that I received the message, an ack essentially.

As a short term remedy, I have decided to keep a record in the datastore that represents the 'most recent message'.

I know that normally within the context of a servlet, you wouldn't synchronize on the servlet instance itself, because that would cause a queue of requests, but in this case, Superfeedr is the only service using this servlet, it isn't consumed by normal users, so that is exactly what I want, just wanted to make sure that there wouldn't be any ill side effects of this approach.

I want to allow time for a request to come in from Superfeedr, analyze the message, make sure it's not a duplicate, and if it's not, update the value in the datastore that represents the most recent message, one at a time. Here is what I have:

    synchronized(this) {
        //check datastore entry to see if the message coming in is a duplicate
        if(messageIsADuplicate(title)){
            logger.log(Level.SEVERE, "DUPLICATE MESSAGE RECEIVED");
            return;
        }
        //delete the entry in the datastore that represents the most recent message
        if(!mostRecentMessageDeletedFromDatastore()){
            logger.log(Level.SEVERE, "UNABLE TO DELETE MOST RECENT MSG");
            return;
        }
        //add an entry to the datastore that represents the most recent message from superfeedr
        if(!mostRecentMessageUpdatedInDatastore(title)){
            logger.log(Level.SEVERE, "UNABLE TO UPDATE MOST RECENT MSG");
            return;
        }
    }

So, assuming that I want one at a time access to this particular block of code, does this seem appropriate?

Answer 1

I think you're bumping into an issue were your GAE application is shutdown between notifications because it has too little activity. The consequence is that our notifications are usually timing out during the "boot" of your GAE application and the duplicates you see are retries.

You can easily inspect this by checking the HTTP headers. If they contain X-Superfeedr-Retried you can see that they were retries. (See docs for details)