I'm currently integrating PubSub using the following approach https://docs.spring.io/spring-cloud-gcp/docs/1.2.4.BUILD-SNAPSHOT/reference/html/#spring-integration
Basically I'm subscribed to a topic, I receive the message and do something with it.
I would like to, in case of any error scenario of my business rules (like not accept the message because X,Y,Z reason) send it to a dlq.
I see that just a month and a half ago Google came out with DLQ for PubSub: https://cloud.google.com/pubsub/docs/dead-letter-topics
But I'm not sure what the right way to integrate it with the spring integration approach should be.
The built-in DLQ support will work automatically -- all you have to do is turn on Dead Lettering (through add/edit subscription screen in Cloud Console), setting the "Maximum delivery attempts" field to the number of attempts. After your application nack()
s a message this number of times, Cloud Pub/Sub will redirect it to the topic you set up as the DLQ.
The built-in support works great for environmental, retryable causes of message delivery failure. However, it has a minimum of 5 retries before it will send the message to DLQ. In your case of business rules validation, you may prefer to mimic DLQs with a custom topic and Spring Integration redirection, since after failing on attempt #1, the message won't suddenly become valid, and the remaining attempts #2 - #5 are a waste of resources.
To mimic the DLQ that redirects after a single failure, you would create a new topic, let's call it validation-dlq
, wire it up as a Spring Integration channel, run custom validation checks on each message, and if a message fails, ack()
the original message on the source subscription, and publish the message to the validation-dlq
topic instead.