Search code examples
grafanagrafana-loki

Loki connection refused on Grafana


caller=client.go:349 component=client host=loki:3100 msg="error sending batch, will retry" status=-1 error="Post "http://loki:3100/loki/api/v1/push": dial tcp 172

Please help


Solution

  • I encountered the same problem as you. Finally I found the missing rbac. I solved it by the following method:

    1. get sa loki-promtail

    kubectl get sa -n loki

    NAME                SECRETS   AGE
    default             1         17h
    loki                1         41m
    loki-grafana        1         41m
    loki-grafana-test   1         41m
    loki-promtail       1         41m
    
    1. create ClusterRole
    apiVersion: rbac.authorization.k8s.io/v1
    kind: ClusterRole
    metadata:
      name: promtail-clusterrole
    rules:
      - apiGroups: [""]
        resources:
        - nodes
        - services
        - pods
        verbs:
        - get
        - watch
        - list
    
    1. create ClusterRoleBinding
    apiVersion: rbac.authorization.k8s.io/v1
    kind: ClusterRoleBinding
    metadata:
      name: promtail-clusterrolebinding
    subjects:
        - kind: ServiceAccount
          name: loki-promtail  # your promtail sa
          namespace: loki
    roleRef:
      kind: ClusterRole
      name: promtail-clusterrole
      apiGroup: rbac.authorization.k8s.io