kafka-python basic producing and consuming not working

Question

I am new to kafka and I'm trying to run basic example.

My kafka is running with this config: https://developer.confluent.io/quickstart/kafka-docker/

python 3.7; kafka installation as follows: pip install kafka-python (2.0.2)

I follow this doc; then I run two consoles (one for each of consume and produce)

consumer:

from kafka import KafkaConsumer
for m in KafkaConsumer('my-topic', bootstrap_servers='broker'):
    print(m)

producer:

from kafka import KafkaProducer
p = KafkaProducer(bootstrap_servers='broker')
p.send('my-topic', b'my message!')

And after p.send() I expect to that consumer gets the message. But nothing happens. What is wrong with my setup?

Edit: consoles are run container within the same docker-compose

Answer 1

broker only resolves inside the docker-compose network, if you are running the scripts in the host, you should use localhost.

And if you are running the scripts as containers in the same docker-compose, you should use broker:29092 since there is where Kafka is listening for connections from within the docker-compose network.