Kafka + Java app. in Docker on local computer

Question

I have Java Spring Boot microservice app using Postgresql and Kafka on MacOS. There are 2 modules - filmservice and eventservice. Filmservice got kafka producer, eventservice got kafka consumer. When I try to start it in Docker container - application modules can't connect to Kafka server.

filmservice(producer) got this error:

2024-09-05 14:12:10 filminator-filmservice  | 2024-09-05T11:12:10.978Z  INFO 1 --- [           main] o.a.k.c.c.internals.LegacyKafkaConsumer  : [Consumer clientId=consumer-my-group-id-1, groupId=my-group-id] Subscribed to topic(s): my-topic
2024-09-05 14:12:10 filminator-filmservice  | 2024-09-05T11:12:10.990Z  INFO 1 --- [           main] c.z.f.filmservice.FilminatorFilmService  : Started FilminatorFilmService in 1.974 seconds (process running for 2.223)
2024-09-05 14:12:11 filminator-filmservice  | 2024-09-05T11:12:11.102Z  INFO 1 --- [ntainer#0-0-C-1] org.apache.kafka.clients.NetworkClient   : [Consumer clientId=consumer-my-group-id-1, groupId=my-group-id] Node -1 disconnected.
2024-09-05 14:12:11 filminator-filmservice  | 2024-09-05T11:12:11.103Z  WARN 1 --- [ntainer#0-0-C-1] org.apache.kafka.clients.NetworkClient   : [Consumer clientId=consumer-my-group-id-1, groupId=my-group-id] Connection to node -1 (localhost/127.0.0.1:9092) could not be established. Node may not be available.
2024-09-05 14:12:11 filminator-filmservice  | 2024-09-05T11:12:11.103Z  WARN 1 --- [ntainer#0-0-C-1] org.apache.kafka.clients.NetworkClient   : [Consumer clientId=consumer-my-group-id-1, groupId=my-group-id] Bootstrap broker localhost:9092 (id: -1 rack: null) disconnected

eventservice(consumer) got this error:

2024-09-05 14:14:58 filminator-eventservice  | 2024-09-05T11:14:58.843Z  WARN 1 --- [ntainer#1-0-C-1] org.apache.kafka.clients.NetworkClient   : [Consumer clientId=consumer-my-group-id-2, groupId=my-group-id] Connection to node -1 (localhost/127.0.0.1:9092) could not be established. Node may not be available.

I try to stop application in container and start in from IDE - app successfully connected to container Kafka server.

docker-compose

services:

  zookeeper:
    image: 'bitnami/zookeeper:latest'
    ports:
      - '2182:2182'
    environment:
      - ALLOW_ANONYMOUS_LOGIN=yes

  kafka:
    image: 'bitnami/kafka:latest'
    ports:
      - '9092:9092'
    environment:
      - KAFKA_BROKER_ID=1
      - KAFKA_LISTENERS=PLAINTEXT://:9092
      - KAFKA_ADVERTISED_LISTENERS=PLAINTEXT://127.0.0.1:9092
      - KAFKA_ZOOKEEPER_CONNECT=zookeeper:2181
      - ALLOW_PLAINTEXT_LISTENER=yes
    depends_on:
      - zookeeper

  userservice:
    build: eventservice
    image: eventservice-image
    container_name: filminator-eventservice
    ports:
      - "8082:8082"
    depends_on:
      - kafka
      - filmservice
      - postgres_db
    environment:
      - SPRING_DATASOURCE_URL=jdbc:postgresql://postgres_db:5432/filminator

  filmservice:
      build: filmservice
      image: filmservice-image
      container_name: filminator-filmservice
      ports:
        - "8081:8081"
      depends_on:
        - kafka
        - postgres_db
      environment:
        - SPRING_DATASOURCE_URL=jdbc:postgresql://postgres_db:5432/filminator


  postgres_db:
      image: postgres:13.7-alpine
      volumes:
        - /var/lib/postgresql/data/
      container_name: db_postgres
      ports:
        - "6541:5432"
      environment:
        - POSTGRES_DB=filminator
        - POSTGRES_USER=postgres
        - POSTGRES_PASSWORD=iamroot

Dockerfile:

FROM openjdk:19
COPY target/*.jar app.jar
ENTRYPOINT ["java","-jar","/app.jar"]

application.properties

filmservice:

server.port=8081
spring.datasource.driver-class-name=org.postgresql.Driver
spring.sql.init.mode=always
spring.datasource.url=jdbc:postgresql://localhost:5432/filminator
spring.datasource.username=postgres
spring.datasource.password=iamroot

spring.kafka.bootstrap-servers=localhost:9092
spring.kafka.consumer.group-id=my-group-id

eventservice:

server.port=8082

other eventservice application.properties identical to filmservice

Is there way to launch my app and Kafka server in one docker container?

Answer 1

You need to add a host to your docker-compose file, see this example

  kafka:
    image: 'bitnami/kafka:latest'
    hostname: kafka
    ports:
      - '9092:9092'
    environment:
      - KAFKA_BROKER_ID=1
      - KAFKA_LISTENERS=PLAINTEXT://:9092
      - KAFKA_ADVERTISED_LISTENERS=PLAINTEXT://127.0.0.1:9092
      - KAFKA_ZOOKEEPER_CONNECT=zookeeper:2181
      - ALLOW_PLAINTEXT_LISTENER=yes
    depends_on:
      - zookeeper

Then you can connect inside the docker network like this

spring.kafka.bootstrap-servers=kafka:9092

I think the hostname defaults to the container name.