Search code examples
apache-kafkaapache-kafka-connectkafka-producer-api

Send data from Rest API To kafka


I have just started learning Kafka and I am trying to build a prototype to have a producer which is a REST API and send the data to Kafka consumer. I went through quite a lot of documentation to figure out some particular procedure.

I wasn't able to understand if there is a single connector that I could just use just like the fileconnector or the JDBC connectors provided for Apache Kafka. Should I be writing a custom connector for this ?

I am pretty much confused on where to start. I am particularly looking for some structured documentation or idea on how to get this done.


Solution

  • It sounds like you're talking about the functionality that already exists in the REST Proxy. This provides a REST API for producing data into Kafka, or consuming data from Kafka.


    Edit: From your comment I understand your question to be different. If you want to pull data from a REST endpoint into Kafka you can use Kafka Connect and the kafka-connect-rest plugin. There's an example of it in use here.