Okay, this problem is driving me up a wall, I can't figure out why Prometheus won't serve metrics. I got to 192.168.2.13:8534/metrics but all I get is the spinner contacting the port. I go to "wget http://localhost:8055" locally inside the container and I get
"Connecting to localhost (localhost)|127.0.0.1|:8055... connected. HTTP request sent, awaiting response..."
Docker is pointing 8543 -> 8055
I call my app to start which is nothing more than a class with Main method. With the call below my class app starts and responds perfectly. I also open JVisualVM and JMX show all the JVM info and metrics perfectly. I can attach a debugger no problem.
java -javaagent:"/pro/prometheus/jmx_prometheus_javaagent-0.13.0.jar=8055:/pro/prometheus/config_prometheus.yaml" -Xdebug -Xrunjdwp:transport=dt_socket,server=y,suspend=n,address=1044 -XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath=/tmp/heapdump.hprof -Dcom.sun.management.jmxremote -Dcom.sun.management.jmxremote.local.only=false -Dcom.sun.management.jmxremote.ssl=false -Dcom.sun.management.jmxremote.authenticate=false -Dcom.sun.management.jmxremote.port=31787 -Dcom.sun.management.jmxremote.rmi.port=31787 -Djava.rmi.server.hostname=192.168.2.13 -cp $LIBS service.core2.BootStrap
How can it be that Prometheus won't server metrics but JMX works fine? my config file is really simple.
startDelaySeconds: 0
ssl: false
lowercaseOutputName: false
lowercaseOutputLabelNames: false
port: 8055
I have tried everything I can think of.
Any help would be appreciated.
Turns out in the underlying code was calling an embedded Jetty server thus the actual pace where I needed the metrics from. Now to figure out how to get metrics from embedded jetty.