I am attempting to use PyInstaller to create an application for internal use within my company. The script works great from a working python environment, but loses something when translated to a package.
I know how to include and reference data files that I myself need within my package, but I am having trouble including or referencing files that should come in when imported.
I am using a pip-installable package called tk-tools, which includes some nice images for panel-like displays (looks like LEDs). The problem is that when I create a pyinstaller script, any time that one of those images is referenced, I get an error:
DEBUG:aspen_comm.display:COM23 19200
INFO:aspen_comm.display:adding pump 1 to the pump list: [1]
DEBUG:aspen_comm.display:updating interrogation list: [1]
Exception in Tkinter callback
Traceback (most recent call last):
File "tkinter\__init__.py", line 1550, in __call__
File "aspen_comm\display.py", line 206, in add
File "aspen_comm\display.py", line 121, in add
File "aspen_comm\display.py", line 271, in __init__
File "aspen_comm\display.py", line 311, in __init__
File "lib\site-packages\tk_tools\visual.py", line 277, in __init__
File "lib\site-packages\tk_tools\visual.py", line 289, in to_grey
File "lib\site-packages\tk_tools\visual.py", line 284, in _load_new
File "tkinter\__init__.py", line 3394, in __init__
File "tkinter\__init__.py", line 3350, in __init__
_tkinter.TclError: couldn't open "C:\_code\tools\python\aspen_comm\dist\aspen_comm\tk_tools\img/led-grey.png": no such file or directory
I looked within that directory in the last line - which is where my distribution is located - and found that there is no tk_tools directory present.
How to I get pyinstaller to collect the data files of imported packages?
Currently, my datas is blank. Spec file, created with pyinstaller -n aspen_comm aspen_comm/__main__.py:
# -*- mode: python -*-
block_cipher = None
a = Analysis(['aspen_comm\\__main__.py'],
pathex=['C:\\_code\\tools\\python\\aspen_comm'],
binaries=[],
datas=[],
hiddenimports=[],
hookspath=[],
runtime_hooks=[],
excludes=[],
win_no_prefer_redirects=False,
win_private_assemblies=False,
cipher=block_cipher)
pyz = PYZ(a.pure, a.zipped_data,
cipher=block_cipher)
exe = EXE(pyz,
a.scripts,
exclude_binaries=True,
name='aspen_comm',
debug=False,
strip=False,
upx=True,
console=True )
coll = COLLECT(exe,
a.binaries,
a.zipfiles,
a.datas,
strip=False,
upx=True,
name='aspen_comm')
When I look within /build/aspen_comm/out00-Analysis.toc and /build/aspen_comm/out00-PYZ.toc, I find an entry that looks like it found the tk_tools package. Additionally, there are features of the tk_tools package that work perfectly before getting to the point of finding data files, so I know that it is getting imported somewhere, I just don't know where. When I do searches for tk_tools, I can find no reference to it within the file structure.
I have also tried the --hidden-imports option with the same results.
If I 'manually' add the path to the spec file using datas = [('C:\\_virtualenv\\aspen\\Lib\\site-packages\\tk_tools\\img\\', 'tk_tools\\img\\')] and datas=datas in the Analysis, then all works as expected. This will work, but I would rather PyInstaller find the package data since it is clearly installed. I will keep looking for a solution, but - for the moment - I will probably use this non-ideal workaround.
Then you can use stringify on the subpackage, but this only works if it is your own package.
To solve this problem more permanently, I created a pip-installable package called stringify which will take a file or directory and convert it into a python string so that packages such as pyinstaller will recognize them as native python files.
Check out the project page, feedback is welcome!
The answer is a bit roundabout and deals with the way that tk_tools is packaged rather than pyinstaller.
Someone recently made me aware of a technique in which binary data - such as image data - could be stored as a base64 string:
with open(img_path, 'rb') as f:
encoded_string = base64.encode(f.read())
The encoded string actually stores the data. If the originating package simply stores the package files as strings instead of as image files and creates a python file with that data accessible as a string variable, then it is possible to simply include the binary data within the package in a form that pyinstaller will find and detect without intervention.
Consider the below functions:
def create_image_string(img_path):
"""
creates the base64 encoded string from the image path
and returns the (filename, data) as a tuple
"""
with open(img_path, 'rb') as f:
encoded_string = base64.b64encode(f.read())
file_name = os.path.basename(img_path).split('.')[0]
file_name = file_name.replace('-', '_')
return file_name, encoded_string
def archive_image_files():
"""
Reads all files in 'images' directory and saves them as
encoded strings accessible as python variables. The image
at images/my_image.png can now be found in tk_tools/images.py
with a variable name of my_image
"""
destination_path = "tk_tools"
py_file = ''
for root, dirs, files in os.walk("images"):
for name in files:
img_path = os.path.join(root, name)
file_name, file_string = create_image_string(img_path)
py_file += '{} = {}\n'.format(file_name, file_string)
py_file += '\n'
with open(os.path.join(destination_path, 'images.py'), 'w') as f:
f.write(py_file)
If archive_image_files() is placed within the setup file, then the <package_name>/images.py is automatically created anytime the setup script is run (during wheel creation and installation).
I may improve on this technique in the near future. Thank you all for your assistance,
j