Traefik shows 404 to everywhere except dashboard - Metallb

Question

I try to setup the ingress with traefik but no luck. I would not use TLS just simply the port 80. I have a service with port 8080. If I curl to that service from inside of the cluster it works well. I get the HTTP/200. But if I would connect to he path externally it doesnt working. The Traefik dashboard works well on port 8080

Im using the following setup

Traefik: 1.7.7

kind: Deployment
apiVersion: apps/v1
metadata:
  name: traefik-ingress-controller
  namespace: default
  labels:
    k8s-app: traefik-ingress-lb
spec:
  replicas: 1
  selector:
    matchLabels:
      k8s-app: traefik-ingress-lb
  template:
    metadata:
      labels:
        k8s-app: traefik-ingress-lb
        name: traefik-ingress-lb
    spec:
      serviceAccountName: traefik-ingress-controller
      terminationGracePeriodSeconds: 60
      containers:
      - image: traefik:v1.7.7
        name: traefik-ingress-lb
        ports:
        - name: http
          containerPort: 80
        - name: admin
          containerPort: 8080
        args:
        - --api
        - --web
        - --kubernetes
        - --logLevel=DEBUG

---
kind: Service
apiVersion: v1
metadata:
  name: traefik-ingress-service
  namespace: default
  annotations:
    metallb.universe.tf/address-pool: mmas-ip-space
spec:
  selector:
    k8s-app: traefik-ingress-lb
  ports:
    - protocol: TCP
      port: 80
      name: web
    - protocol: TCP
      port: 8080
      name: admin
  type: LoadBalancer

I have a service what running on port 8080 and I created an ingress rule for it. This is a test service

apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  name: webmust-ing
  namespace: default
  annotations:
    kubernetes.io/ingress.class: traefik
spec:
  rules:
    - http:
        paths:
        - path: /helloservice
          backend:
            serviceName: hellok8s-service
            servicePort: 8080

I get the 404 with curl to /helloservice or try to open from a browser

curl -v http://10.24.33.32/helloservice
curl -v http://10.24.33.32:8080/helloservice

I if curl to the service's ip address directly inside of the cluster, I get the 200/OK

curl -v http://10.100.168.2:8080

NAME                      TYPE           CLUSTER-IP       EXTERNAL-IP   PORT(S)                       AGE
hellok8s-service          ClusterIP      10.100.168.2     <none>        8080/TCP                      5d7h

cat helloservice.yaml

apiVersion: apps/v1
kind: Deployment
metadata:
  name: hellok8s-deployment
  labels:
    app: hellok8s
spec:
  selector:
    matchLabels:
      app: hellok8s
  template:
    metadata:
      labels:
        app: hellok8s
    spec:
      containers:
      - name: hellok8s
        image: docker.io/rlkamradt/hellok8s:latest
        ports:
        - containerPort: 8080
---
apiVersion: v1
kind: Service
metadata:
  name: hellok8s-service
spec:
  type: ClusterIP
  selector:
    app: hellok8s
  ports:
  - port: 8080
    targetPort: 8080

Answer 1

Finally I fixed the problem with the following change in the ingress.

annotations: 
    ingress.kubernetes.io/protocol: http 
    traefik.ingress.kubernetes.io/rule-type: PathPrefixStrip