Search code examples
pythonpython-multiprocessingeel

Python EEL and Multiprocessing?


im currently facing an issue with eel and multiprocessing opening multiple eel windows when an external python script executes that uses multiprocessing.

Folder Structure

Project[folder]
           |
        start.py
           |_______web[folder]
                          |_______partials[folder]
                                              |
                                           partial.py

From start.py(script that starts my app) i import partial.py as i use its functionality in part part of the application and for speed requires the use of multiprocessing.

Code snippet(s)

START.PY (putting a snippet as its way to long)

import eel
#import ... other imports(NO multiprocessing here)
sys.path.append(os.getcwd() + "/web/partials/")
import partials

#... several non relevant @eel.exposed functions are here...

APP ={#...dict with all the keys seen below...}

if __name__ == "__main__":
    try:
        # Start the application and pass all initial params below from APP dict
        eel.start(APP["start_html"], host=APP["host"], size=APP["size"], mode=APP["mode"], port=APP["port"])
    except (SystemExit, MemoryError, KeyboardInterrupt):
        # Handle bottle errors, in all cases call a function close of all remaining windows
        close_handler()
...

PARTIAL.PY

import eel
import os
import re
import socket
import ntc_templates
from mac_vendor_lookup import MacLookup, BaseMacLookup
import netmiko
from netmiko import Netmiko, NetMikoAuthenticationException, NetMikoTimeoutException
import concurrent.futures


def device_details_builder_mp():
    #... code that will run with multiprocessing from every entry in device_data below...


@eel.expose
def device_details_search(site, user, password, device_data):
    # Init list that will hold all the final data
    final_device_details = []

    with concurrent.futures.ProcessPoolExecutor() as executor_detail:
        results_details = [executor_detail.submit(device_details_builder_mp, device, site, user, password) for device in device_data]

        for result_detail in concurrent.futures.as_completed(results_details):
            final_device_details.append(result_detail.result())

Issue

So i think my issue is the way im importing partial.py into start.py, if i run start.py and use the app and trigger the feature that uses multiprocessing which gets called from the java-script side it just works.

The issue happens when i pack the app and execute it from an .exe when i use the feature that use multiprocessing it doesn't work and it opens multiple eel windows, Was reading that multiprocessing re imports everything again but i couldn't figure that out exatly, i saw this as well: https://github.com/samuelhwilliams/Eel/issues/224

Hoping someone can assist me in how to better import partial.py into start.py so it doesn't cause the issue with multiprocessing.

Thanks

Desktop

  • OS: Windows 10 1809
  • Browser: Brave
  • Version: EEL 0.12.4

Solution

  • That time when a single line does it.

    Added multiprocessing.freeze_support() in the if name == "main": and after packing, multiprocessing works fine.

    I wasnt looking in the right place, it was not an eel issue but a PyInstaller one

    https://github.com/pyinstaller/pyinstaller/wiki/Recipe-Multiprocessing#additional-code-for-pyinstaller--33-and-windows