First I would like to say that I am a newbie to Kafka and also stackoverflow, So I am sorry if I am not asking this in the right way. I am trying to implement the producer-consumer using kafka-python. But its not working properly
I have the zookeeper installed and its up and running. I have up the kafka-server also. But when I am running the consumer and producer through pycharm, the messages are not getting received by the receiver.The consumer keeps on running but the producer stops.
consumer.py
from kafka import KafkaConsumer
consumer = KafkaConsumer('test', group_id='test-consumer-group',
bootstrap_servers=['my_ip:9092'], api_version=(0, 10, 1),
auto_offset_reset='earliest')
print("Consuming messages")
for msg in consumer:
print(msg)
producer.py
from kafka import KafkaProducer
print('above producer')
producer = KafkaProducer(bootstrap_servers=['my_ip:9092'], api_version=(0, 10, 1),
compression_type=None
)
print('after producer')
for _ in range(100):
producer.send('test', b'HELLO NITHIN chandran')
print('after sending messages')
In the place of my_ip, I have provided with my system ip address from ipconfig.
consumer.py Output -
Consuming messages
The consumer.py doesnt stop running
producer.py Output -
above producer
after producer
after sending messages
Process finished with exit code 0
The producer.py stops running and the process is finished as shown in the output.
Please help me in resolving this issue. All help are appreciated
Your code is ok, the problem is about your broker configuration. Please set it to initial configuration, just change the log.dirs to the path that you want to store Kafka data.
After changing the config file follow these steps: