I have a collection of healpy maps I want to blur with healpy.sphtfunc.smoothing. Things run smoothly on my Macbook Pro but on the remote Linux machine we use for testing, I have found the tests stall due to the healpy.sphtfunc.smoothing taking in some cases 2700x longer to execute! Here's a MWE:
def test_hp_smooth_mwe():
import numpy as np
import os, platform, sys, time
import healpy as hp
print(os.name, platform.system(), platform.release())
print(hp.__version__)
print(sys.version)
resp_matrix = 10 * np.random.random(size=(5, 64, 768))
times = np.zeros((5, 64))
for i in range(resp_matrix.shape[0]):
for j in range(resp_matrix.shape[1]):
t0 = time.time()
resp_matrix[i, j, :] = hp.sphtfunc.smoothing(
resp_matrix[i, j, :], fwhm=np.deg2rad(10)
)
times[i, j] = time.time() - t0
assert False, f"mean smoothing time: {times.mean():.3e} s"
My Macbook Pro result:
posix Darwin 18.7.0
1.15.0
3.8.10 | packaged by conda-forge | (default, May 10 2021, 22:58:09)
[Clang 11.1.0 ]
AssertionError: mean smoothing time: 4.865e-04 s
Remote Linux machine result, running in the test environment:
posix Linux 5.8.0-43-generic
1.15.0
3.6.14 (default, Aug 17 2021, 16:32:35)
[GCC 10.2.1 20210110]
AssertionError: mean smoothing time: 1.360e+00 s
Remote Linux machine result, running in the Python interpreter:
posix Linux 5.8.0-43-generic
1.15.0
3.8.5 (default, Jul 28 2020, 12:59:40)
[GCC 9.3.0]
AssertionError: mean smoothing time: 3.423e-04 s
(Don't mind the strange import patterns and the assert False, that's just to get it to work easily on the remote testing machine.)
As we can see, smoothing in the remote Linux test environment takes 2800x longer than on my local machine. As far as I can tell, no other operation experiences such a drastic slowdown, just the smoothing. Note also I see this slowdown on python 3.6, 3.7, and 3.8.
The test environment uses docker---could that slow things down?
hp.sphtfunc.smoothing turns out to be very compute-intensive, briefly flaring up all 16 threads on the runner to ~60% utilization. In the test environment, this was running on python 3.6, 3.7, and 3.8 at the same time, causing a bottleneck. If I test just 3.6 on its own, for example, I get the expected mean time of ~5e-04 s.