I have a question geared towards using zendesk's deliveryboy/racecar wrappers in tandem for processing Kafka events.
I consume events using Racecar but send events back for reprocessing, at a delayed interval, into the same topic via DeliveryBoy using a Sidekiq scheduled job.
It worked a number of times but since, Racecar has stopped consuming those re-process events.
DeliveryBoy still confirms that it appended the message to the topic:
Sending 1 messages to b-4.sa-xxx.kafka.sa-east-1.amazonaws.com:9094 (node_id=4)
Opening connection to b-4.sa-xxx.kafka.sa-east-1.amazonaws.com:9094 with client id service_sidekiq...
Sending produce API request 1 to b-4.sa-xxx.kafka.sa-east-1.amazonaws.com:9094
Waiting for response 1 from b-4.sa-xxx.kafka.sa-east-1.amazonaws.com:9094
Received response 1 from b-4.sa-xxx.kafka.sa-east-1.amazonaws.com:9094
Successfully appended 1 messages to topic-normalized/7 on b-4.sa-xxx.kafka.sa-east-1.amazonaws.com:9094 (node_id=4)
but the consumer doesn’t pick up the event.
I know events written to the topic via other means are getting picked up by Racecar, seems to be the only ones its not consuming is the ones written by DeliveryBoy.
I’ve changed the client_id of DeliveryBoy to differ from the client_id of Racecar (write with one, listen with the other) but that didn’t help.
Any advice is appreciated.
It ended up being that I had log level set incorrectly and was unable to witness side-effects since I was writing to debug.
Everything working as expected, user error..