Search code examples
javaspring-bootprometheusmicrometer

Micrometer/Prometheus How do I keep a gauge value from becoming NaN?


I am trying to monitor logged in users, i am getting the logged in user info by calling api, this is the code i have used,

public class MonitorService {
    private InfoCollectionService infoService;
    public MonitorService(InfoCollectionService infoService) {
        this.infoService = infoService
    }

    @Scheduled(fixedDelay = 5000)
    public void currentLoggedInUserMonitor() {
        infoService.getLoggedInUser("channel").forEach(channel -> {
            Metrics.gauge("LoggedInUsers.Inchannel_" + channel.getchannelName(), channel.getgetLoggedInUser());
        });
    }
}

And i see the values in Prometheus, the problem is after a few seconds, the value become NaN, i have read that Micrometer gauges wrap their obj input with a WeakReference(hence Garbage Collected ).I don't know how to fix it.If anybody knows how to fix this it would be great.


Solution

  • This is a shortcoming in Micrometer that I would like to fix eventually.

    You need to keep the value in a map in the meantime so it avoid the garbage collection. Notice how we then point the gauge at the map and us a lambda to pull out the value to avoid the garbage collection.

    public class MonitorService {
        private Map<String, Integer> gaugeCache = new HashMap<>();
        private InfoCollectionService infoService;
        public MonitorService(InfoCollectionService infoService) {
            this.infoService = infoService
        }
    
        @Scheduled(fixedDelay = 5000)
        public void currentLoggedInUserMonitor() {
            infoService.getLoggedInUser("channel").forEach(channel -> {
                gaugeCache.put(channel.getchannelName(), channel.getgetLoggedInUser());
                Metrics.gauge("LoggedInUsers.Inchannel_" + channel.getchannelName(), gaugeCache, g -> g.get(channel.getchannelName()));
            });
        }
    }
    

    I would also recommend using tags for the various channels:

    Metrics.gauge("loggedInUsers.inChannel", Tag.of("channel",channel.getchannelName()), gaugeCache, g -> g.get(channel.getchannelName()));