Search code examples
pythonmatplotlibpyinstaller

Exe generated by PyInstaller fails quietly on a different machine

I'm having trouble getting an exe generated by PyInstaller on one PC (PC-Good) working on another (PC-Bad).

  • The exe is created on PC-Good, and executes as expected on PC-Good
  • Both PCs are running Windows 10
  • Issue occurs on PC-Bad upon calling most of the matplotlib.pyplot methods (such as subplots() or plot())
  • Issue persists even when matplotlib is in non-interactive mode
  • No error is reported upon failure. exe simply exits
    • even when matplotlib verbosity is changed to 'debug'
    • even when the problem statement is captured in try-exception

See below for code:

print('Start')
import matplotlib
matplotlib.use('Agg')

import matplotlib.pyplot as plt
print("Import of matplotlib & pyplot successful")

plt.set_loglevel("debug")

x = [0, 1, 2, 3, 4]
y = [4, 3, 2, 1, 0]
print('list creation successful')

try:
    fig, ax = plt.subplots(1, 1, figsize=(12, 6))
except Exception as exception:
    print(exception.__class__.__name__ + ": ", + exception.message)
finally:
    print('run subplots() successful')

plt.scatter(x, y)
print('plot creation successful')
plt.savefig('saved_plot.png')
print('code complete')

Output on PC-Good:

Start
C:\tools\miniconda3\envs\bcht\lib\site-packages\PyInstaller\loader\pyimod03_importers.py:623: MatplotlibDeprecationWarning:
The MATPLOTLIBDATA environment variable was deprecated in Matplotlib 3.1 and will be removed in 3.3.
  exec(bytecode, module.__dict__)
Import of matplotlib & pyplot successful
list creation successful
run subplots() successful
plot creation successful
DEBUG:matplotlib.font_manager:findfont: ...
(and many more matplotlib DEBUG messages)
code complete

Output on PC-Bad:

Start
C:\tools\miniconda3\envs\bcht\lib\site-packages\PyInstaller\loader\pyimod03_importers.py:623: MatplotlibDeprecationWarning:
The MATPLOTLIBDATA environment variable was deprecated in Matplotlib 3.1 and will be removed in 3.3.
Import of matplotlib & pyplot successful
list creation successful

Since I get no error output, I'm quite lost as to where to go next. Matplotlib documentation doesn't offer any additional tips on getting more granular debug messages for methods like plot or subplots. Has anyone else observed such an issue or know of a fix? Or does anyone know if there's a way to get Matplotlib to tell me more?

Solution

  • I found a workaround for this issue, which I can't call a fix yet due to the fact that neither PyInstaller nor MatPlotlib provides any debug message to further the investigation for rootcause. Nevertheless, it gets rid of the issue that I was seeing.

    TL;DR

    • Issue is due to incompatibility when using PyInstaller to generate executable with Python codes with matplotlib 3.1.3 and numpy 1.18.1. This incompatibility does not cause the python script itself to error out. This incompatibility does not produce any error message or symptoms that would point to itself, thus, making it difficult to rootcause.
    • If anyone observes a frozen executable quitting silently without any error message. It's worthwhile to investigate whether this is due to library incompatibility. If the quitting occurs during execution of mpl methods, then look into numpy immediately.

    Now the TL: It started with a suspicion that the issue was due to virtual environment or conda meddling with the freezing process (they are not) so I used another computer to install a clean python (3.7.9), and matplotlib 3.1.3 which populated all its dependencies automatically (let's call this new environment "lean env" and the Source PC in the problem statement "full env"). I replicated the bundling/freezing process on lean env and everything works fine. At this point, I noticed that a number of libraries in the lean env are newer than the full env. So I upgraded them all on the full env, including:

    numpy from 1.18.1 to 1.20.0 kiwisolver from 1.3.0 to 1.3.1

    After this, the full env is also able to generate a bundle that works perfectly fine when run on both Source PC and Target PC. So the problem is likely to be the "silent" incompatibility between matplotlib 3.1.3 and one of those two.

    To narrow things further, I reverted numpy to 1.18.1 in the full env and reproduced the old problem I saw. There it is.

    The frustrating aspect of this is that neither numpy nor matplotlib generated any discernable error (nor Pyinstaller). In addition, matplotlib itself does not have any method to produce any useful verbose debug message to further the investigation. (I did turn on the log_level of matplotlib to debug, but only got some info indicating the fonts being used.) For this reason, I will not proceed any further to investigate. The workaround is sufficient for my use. If anyone else is interested, feel free to proceed by interrogating mpl. Neither will I conduct further interop between pyinstaller, matplotlib, and numpy. All I'll say is:

    Good: matplotlib==3.1.3, numpy==1.20.0, pyinstaller==3.6

    Bad: matplotlib==3.1.3, numpy==1.18.1, pyinstaller==3.6