When using HPX, if I understand correctly, with hpx::async you can easily spawn a task that is scheduled by the threading subsystem to run on any available core either on the current locality or on a specific remote locality by supplying an id_type. There is also the possibility of invoking a component action that may be on any locality.
However, I'm looking for a way to invoke a plain action on any core on any locality that is not overloaded to achieve simple load balancing. If the current locality is underutilized the function should run there. Otherwise, the function should be scheduled on the next best remote locality. Does the described functionality exist in HPX?
I wrote a test program that generates load in hpx::async invocations to see if hpx::async(f) would actually run on remote localities but AFAICT it only uses the current locality.
I also thought find_locality might be what I need, but judging from the source code, it appears to just pick the first suitable locality, not the one with the least load or some other load balancing logic.
There is no built-in way to achieve what you want. You can however either use one of the existing performance counters, or create your own one that exposes the information you can use to make that decision yourself.