Search code examples
javagoogle-cloud-pubsub

Pub/sub in java - long lived vs short live Publisher, which is the correct approach?


We're currently implementing pub/sub in our java project and I have a question about how long should Publisher object exist.

Most tutorials are relatively simple and just show pattern of:

  • call a publish method
  • in that method create publisher
  • send a message or two
  • close the publisher

so that's what we've replicated, a send() method with message as parameter, creating and closing Publisher for each message.

We've observed that performance is way below what we expected (30 messages per second).

Now we're considering if we have to change the old code to allow sending messages in batches, or if we should just keep the publisher open for the whole time service is active, or maybe keep it in some time expiring cache.

Is creating and closing a Publisher for each message a normal pattern or should the publisher be more long lived? Is it ok to keep a Publisher open for the whole time the service is working or can it lead to problem with resources?


Solution

  • Publishers should be long-lived. A lot of tutorials are going to be trivial examples where the lifetime of the program itself is relatively short, which is why they tend to close publishers after messages are published. If you create and close a Publisher for each message, then the connection needs to be recreated every time, which is going to slow down publishing. You also won't be able to take advantage of batching in the client library, which allows for more efficient sending of messages to the server.