How to send to data with kafkaProducer in Python?

Question

I am having trouble to send data to a remote Kafka cluster which should have appropriate rights.

The topic is already created on the cluster. I have also tried to send data in bytes but still having the same error. If you have any information it could be of great help! The error i have is on the picture provided.

Answer 1

First of all, install "pykafka" => pip install pykafka

Then launch a consumer (in a terminal for example ), run the following command :

from pykafka import KafkaClient
import threading

KAFKA_HOST = "localhost:9092" # Or the address you want

client = KafkaClient(hosts = KAFKA_HOST)
topic = client.topics["test"]

with topic.get_sync_producer() as producer:
    for i in range(10):
        message = "Test message " + str(i)
        encoded_message = message.encode("utf-8")
        producer.produce(encoded_message)

You will be able to see :

Test message 0

Test message 1

Test message 2

...

Test message 9