It seems contract testing doesn't yield any benefits for message queues. Does it have any benefits?
At the provider side we instantiate an object that normally goes into kafka.
So what the provider essentially does is this:
At the consumer side, we have have the opposite of this, i.e., we recreate that object in memory and trigger the code that consumes it.
In other words, what the consumer essentially does is:
In my case on the consumer side the code which consumes the object cannot be triggered. This is because it requires starting an instance of the server itself which introduces a lot of complexity.
Should contract testing be avoided for message queues when we're sending data packets (not requests)?
It is important to keep in mind that Contract Testing =|= Functional Testing. So we're not going to retest with different messages because that would be functional testing.
A contract test asks "are these services able to communicate with each other". This question is absolutely valuable for message queues.
Is it safe to deploy a new version of your consumer? It depends on whether the producer is able to read and understand the messages the provider is sending. Contract testing improves your confidence that the consumer and provider are able to speak to each other.
You could argue that contract testing is more important with message queues, because (depending on the design and how long messages live in queues/whether they are replayed), you might be consuming messages that were produced by several different versions of the producer.
Even if the consumer and provider are in the same deployment unit (eg, a single microservice), then you're likely to be consuming messages from the previous version of the provider immediately after a deployment happens.
Speaking theoretically, contract testing a message queue is like contract testing an HTTP request/response pair where the request is implicit (you could argue the "request" is the subscription to the queue).
When testing/verifying an HTTP Pact contract, Pact provides both the transport (mock HTTP client/server), and the content (content shape and constraints).
At the moment, when doing the same test with a Message Pact, Pact only provides the content / constraints. There's no mock consumer/provider - the data transport is not tested by the Pact framework. This means that a message pact test has slightly less coverage than the corresponding HTTP pact would - it's not testing that your queue library is set up and invoked correctly (unless you do that testing yourself).
The Pact Spec V4 Plugin proposal exists to make it easier to address this gap in the future, so that a message pact test will be able to more easily cover the queue libraries too. But, it's not something that is available to use today.
it requires starting an instance of the server itself which introduces a lot of complexity.
This feels like it might be a software design issue that a bit of refactoring would be able to solve. You should be able to run message pact consumer tests without starting the whole server. This will be specific to your usecase and design, but join us in #general on https://slack.pact.io/ and we can probably help you.
It is important to keep in mind that Contract Testing =|= Functional Testing. So we're not going to retest with different messages because that would be functional testing.
Yes, this is a good observation. In general, you want to test that your consumer can understand every type of message that the provider can send, and in the case of message queues, this is usually only one type (although it is still sometimes a few).
It's a little off the topic of your question, but there's a small caveat here in that although a Pact test is not a functional test, it is ok if it has some functional coverage. All testing is risk reduction - and the closer you can get to testing the actual code you will be running, the better the risk is reduced. For this reason, I usually put any mocks as deep in the service as is practical. Of course, I would cover the behaviour of the service with its own unit tests, but I think it's good if the pact test (which doesn't cover the behaviour directly) happens to still exercise some of the behaviour in the course of contract verification. Happy to elaborate on that if you need.