Search code examples
gogoogle-cloud-platformgoogle-cloud-pubsub

Google Pub Sub Subscription. Receive retry policy


I am newbie in using google pubsub, please pardon my ignorance. I am using go language We are a client of a company who uses pubsub who are publishing notification meaning, we do not own the pubsub subscription just consumers, the company who owns the pubsub will be pushing 100 messages an hour and we need to continuously looking for messages. I have the below sample code for getting message which is run in the background

func PullMyMessages() {
   ctx := context.Background()
   client, _ := pubsub.NewClient(ctx, "myAPI", option.WithCredentialsFile("abc.json"))
   sub := client.Subscription("Mysub")
   msgSlice := make(chan *pubsub.Message, 1)
   cctx, cancel := context.WithCancel(context.TODO())
   go sub.Receive(cctx, func(ctx context.Context, m *pubsub.Message) {
      msgSlice <- m
   })
   for {
      select {
      case res := <-msgSlice:
         fmt.Printf("Got message: %q\n", string(res.Data))
         res.Ack()

      case <-time.After(5 * time.minute):
         cancel()
      }
   }
}

With the above code I was able to get messages for the subscription and when there is no subscription for minutes i would exit out of the application. Question

  1. ref: https://cloud.google.com/pubsub/docs/handling-failures#pubsub_dead_letter_delivery_attempt-go , since I am only the consumer/subscriber of the subscription will be able to set a retry policy set for the message that fail to process. any sample code or pointers would be helpful on this

  2. Currently I am expecting that this application is running in background and when it does not get any message for 5 minute, it would exit out. is the understanding correct?

I would in real time remove the time factor to continuously run but would want the retry mechanism in place


Solution

  • There are several things to know on PubSub.

    • You can't change the retry policies if you are not the owner (or at list to be authorized to update) of the subscription
    • On a subscription, the messages are kept for a while (by default it's 7 days, but it could be between 10s and 30 days). That means if you don't listen the subscription for a while, the messages are not lost, but stacked in the subscription and you will received them when your subscriber is up.
    • You can run already up subscriber (remove the case with the timer), but you can also imagine to process the messages by bulk (listen 5 minutes the subscription every 10 minutes for example). Like that you can leverage serverless services and save resource, money and CO2! It's depends on your use case and realtime requirement.