My problem is that I cannot gather the container metrics (cpu, mem, ...) of containers running in the same swarm but on a different node than the node (manager) answering my request.
Is this the normal behavior?
How do I get the stats of the other running containers (I cannot request the stats from a worker node via http/tcp...)?
Currently I am using the Docker API via TCP/HTTP: https://docs.docker.com/engine/api/v1.30/#operation/ContainerStats
PS: I cannot use cAdvisor.
There are no "swarm container stats" from the docker engine, only "container stats" and containers are local to each node. If you want stats across all nodes in the cluster, you'll need to use something else to gather them. cAdvisor/Prometheus is used by most because it handles the specific task with least privilege, you could probably get classic swarm to work, or just script something running across every node. I haven't played with it, but swarm-exec may also support your use case.