Scrapping from the default kubelet and metrics-server with Prometheus

Question

I want to scrap metrics from the default kubelet and metrics-server with Prometheus in the kube-system namespace. How can I accomplish that?

I tried to use this configuration, but it didn't work. I have: Get "https://InternalIP:443/metrics": dial tcp: lookup InternalIP on 10.96.128.2:53: no such host.

apiVersion: v1
kind: ConfigMap
metadata:
  name: prometheus
data:
  prometheus.yml: |
    global:
      scrape_interval: 1s
    scrape_configs:
      - job_name: metrics-server
        scheme: https
        static_configs:
          - targets: [metrics-server]

Please help, I've been trying a lot. Thanks :)

Answer 1

Based on Prometheus documentation, you should include the port number of the host on your configmap data, see below:

apiVersion: v1
kind: ConfigMap
metadata:
  name: prometheus
data:
  prometheus.yml: |
    global:
      scrape_interval: 1s
    scrape_configs:
      - job_name: metrics-server
        scheme: https
        static_configs:
          - targets: [metrics-server:8080]

## check the service port number of the kube state metrics server

Alternatively, [this solution}(How to scrape metrics-server to prometheus outside kubernetes cluster) will need you to add annotations on your kube state metrics deployment:

apiVersion: apps/v1
kind: Deployment
metadata:
  labels:
    app.kubernetes.io/component: exporter
    app.kubernetes.io/name: kube-state-metrics
    app.kubernetes.io/version: 2.10.0
  name: kube-state-metrics
  namespace: kube-system
  annotations: 
    prometheus.io/scrape: 'true'
    prometheus.io/port: 'port'
    prometheus.io/path: '/metrics'
    prometheus.io/scheme: 'http'

You can view yaml files of kube state metrics here