Context deadline exceeded issue for actuator/prometheus in Spring Boot with Docker (Get <service-name> context deadline exceeded)

Question

I have a problem to use prometheus and grafana defined in docker-compose yml file.

When I run docker-compose up -d , all services are up.

Next, I try to go to localhost:1221/actuator/prometheus. I get this result shown below

# HELP jvm_buffer_count_buffers An estimate of the number of buffers in the pool
# TYPE jvm_buffer_count_buffers gauge
jvm_buffer_count_buffers{id="mapped - 'non-volatile memory'",} 0.0
jvm_buffer_count_buffers{id="mapped",} 0.0
jvm_buffer_count_buffers{id="direct",} 9.0
# HELP spring_security_filterchains_requestcache_before_total  
# TYPE spring_security_filterchains_requestcache_before_total counter
spring_security_filterchains_requestcache_before_total{security_security_reached_filter_section="before",spring_security_filterchain_position="0",spring_security_filterchain_size="0",spring_security_reached_filter_name="none",} 7.0
....

I think it works.

Next, I open to http://localhost:9090/targets to reach prometheus and I get this issue shown below

Get "http://book-delivery-app-1:1221/actuator/prometheus": context deadline exceeded

Where is the problem in docker-compose.yml or prometheus.yml file?

Here is docker-compose.yml shown

version: "3.9"

services:
  database:
    container_name: database
    image: mysql:8.0.33
    restart: always
    env_file:
      - .env  # Use the .env file for environment variables
    environment:
      MYSQL_DATABASE: bookdelivery
      MYSQL_PASSWORD: ${DATABASE_PASSWORD}
      MYSQL_ROOT_PASSWORD: ${DATABASE_PASSWORD}
      MYSQL_ROOT_HOST: '%'
      MYSQL_PORT: 3307
    volumes:
      - ./db:/var/lib/mysql
    ports:
      - "3307:3306"
    networks:
      - bookDeliveryNetwork

  bookdelivery:
    image: 'bookdelivery:latest'
    build:
      context: .
      dockerfile: Dockerfile
    container_name: bookdelivery
    restart: on-failure
    env_file:
      - .env  # Use the .env file for environment variables
    ports:
      - "1221:1221"
    environment:
      - server.port=1221
      - spring.datasource.username=${DATABASE_USERNAME}
      - spring.datasource.password=${DATABASE_PASSWORD}
      - BOOK_DELIVERY_DB_IP=database
      - BOOK_DELIVERY_DB_PORT=3307
      - spring.datasource.url=jdbc:mysql://host.docker.internal:3307/bookdelivery
    depends_on:
      - database
    networks:
      - bookDeliveryNetwork

  prometheus:
    image: prom/prometheus:v2.35.0
    container_name: prometheus
    restart: unless-stopped
    ports:
      - "9090:9090"
    volumes:
      - ./data/prometheus/config:/etc/prometheus/
    command:
      - '--config.file=/etc/prometheus/prometheus.yml'
    networks:
      - bookDeliveryNetwork

  grafana:
    image: grafana/grafana-oss:8.5.2
    pull_policy: always
    container_name: grafana
    restart: unless-stopped
    ports:
      - "3000:3000"
    volumes:
      - ./data/grafana:/var/lib/grafana
    environment:
      - GF_SECURITY_ADMIN_PASSWORD=admin
      - GF_SERVER_DOMAIN=localhost
    networks:
      - bookDeliveryNetwork


networks:
  bookDeliveryNetwork:

Here is prometheus.yml shown below

global:
  scrape_interval: 120s # By default, scrape targets every 15 seconds.
  evaluation_interval: 120s # By default, scrape targets every 15 seconds.

# A scrape configuration containing exactly one endpoint to scrape:
# Here it's Prometheus itself.
scrape_configs:
  # The job name is added as a label `job=<job_name>` to any timeseries scraped from this config.
  - job_name: 'prometheus'
    # Override the global default and scrape targets from this job every 5 seconds.
    scrape_interval: 5s
    # metrics_path defaults to '/metrics'
    # scheme defaults to 'http'.
    static_configs:
      - targets: ['localhost:9090']

  - job_name: 'Spring Boot Application input'
    metrics_path: '/actuator/prometheus'
    scrape_interval: 2s
    static_configs:
      - targets: [ 'book-delivery-app-1:1221' ]
        labels:
          application: 'Book Delivery API'

Here is the repo : Link

Answer 1

Change your prometheus config to use the container name that you have used. Currently you have provided -

'book-delivery-app-1:1221'

whereas, it should be -

'bookdelivery:1221'

The file inside data/prometheus/config should be the one to be modified.