how can i tell the python interpreter running inside the flatpak sandbox to use libraries from the main system

I'm creating a Gtk4 application with python bindings on Linux. I'm using flatpak as well. So I'm creating like an extensions/plugins systems where user can define his main module to call in a specific file and later on ill load it. Every thing works but when the user uses imports for external libraries like NumPy or pandas it will start looking inside the flatpak sandbox and this is normal. I'm wondering how can I tell the python interpreter to use the modules of the system for the imported plugins instead of looking in the modules of the app.

User's code and requirements should be independent of the app's requirements.

This is how I'm loading the modules:

extension_module = SourceFileLoader(file_name, module_path).load_module()
extension_class = getattr(dataset_module, class_name)
obj = extension_class()

This is an example of the loaded class

the absolute path of this module is /home/user/.extensions/ext1/module.py

import numpy as np

class Module1:
    def __init__(self):
        self.data = np.array([1, 2, 3, 4, 5, 6])

    def get_data(self):
        return self.data

I tried using

os.path.append('/usr/lib64/python-10.8/site-packages')

It's added, but in the sandbox environment.

I thought about looking for user imports manually like when a user import pandas ill try to look for installed python packages in the system and use the importlib or the SourceFileLoader to load it, but I don't think it's a good way to do it.

Solution

So, finally, after a day of reading the Flatpak docs, I found a way to do it.

I had to add the argument --filesystem=home. This argument will give you access to the user's directory. When you use pip to install packages, the packages will be installed in the following directory ~/.local/lib/python3.10/site-packages/. To let python interpreter search for packages in that folder, you can add it to the path like this.

import sys
sys.path.append('~/.local/lib/python3.10/site-packages/')

Note1: In my case, this is enough because the app is for learning and not serious, so it's 0 security concerns.

I am using openSUSE Tumbleweed, so I have python 3.10.11 and on the Flatpak runtime there is python 3.10.6. So some users who have an old distro or a distro like Ubuntu or Debian maybe you don't have the latest python version, and you may have compatibility issues.

A better solution is to create a simple folder in the user local directory f.g ~/.cache/myapp/packages and then you can add it to the flatpak manifest file --filesystem=~/.cache/myapp:create this way you're mapped your folder to be accessed from the sandbox and the option :create will create the folder if it doesn't exist. Then i your python script to install required packages based on the imports used in external scripts. In your python, add the folder path to the path sys.path.append('~/.cache/app/packages/').

Note2: It's not safe to import directly scripts to your code when you want to create a plugin system it's better to create a sub process that executes these scripts, this way you isolate your code from external code. You can use IPC protocol or other techniques to change data between the main process and the subprocess.