Search code examples
apache-kafkaapache-kafka-connect

Finding Corresponding Source connector of a kafka topic


Is any way to find out corresponding source connector from kafka topic name? eg: Consider I have topic called "etl.userDb.users". I want to know which source connector is publishing message to this topic.


Solution

  • It's not possible to find any topic producer without including metadata within the record itself or enforcing+tracking ACL/certificate policies.

    Another approach I've seen is to enforce centralized log collection across all services, then searching for Kafka producer logs using a topic name, but that is unrelated to Kafka tooling, and requires long data retention windows.

    You may be able to run tcpdump around/on the brokers and inspect packets, but that assumes that you have direct access to the brokers or the network routers, and that the producer is actually active in the time that you're collecting those packets.