Kafka Connector To read from a Topic and write to a topic

Question

I want to build a Kafka connector which needs to read from the Kafka topic and make a call to the GRPC service to get some data and write the whole data into another kafka topic.

I have written a Kafka Sink connector which reads from a topic and called a GRPC service. But not sure how to redirect this data into a Kafka topic.

Answer 1

Kafka Streams can read from topics, call external services as necessary, then forward this data to a new topic in the same cluster.

MirrorMaker2 can be used between different clusters, but using Connect transforms is generally not recommended with external services.

Or you could make your gRPC service into a Kafka producer.