IronPython dependencies for scripts stored as strings

Question

I have a C# application that stores python script files (*.py) as strings. I load them using:

scriptEngine.CreateScriptSourceFromString(code);

But now I have multiple script files with dependencies between them (imports). In order to handle dependencies, I could save all the strings back to files in a folder and load the script that i want to execute using:

scriptEngine.CreateScriptSourceFromFile(filePath);

but this would make all the script files visible. Is there a way to achieve this in a in-memory way, so that the script files are not first saved to disk but loaded from the strings directly?

TL;DR: Example of how this might look:

myutils.py:

def SomeMethod(p):
    print ('SomeMethod(p=%s)' % p)

script1.py:

import myutils;

if __name__ == '__main__':
    myutils.SomeMethod('script1')

script2.py:

import myutils;

if __name__ == '__main__':
    myutils.SomeMethod('script2')

My application has the scripts stored as strings. something like

Dictionary<string, string> filePathToContent = new Dictionary<string, string>();

filePathToContent["myutils.py"] = "..."; // The script file content.
filePathToContent["script1.py"] = "..."; // The script file content.
filePathToContent["script2.py"] = "..."; // The script file content.

I want to call script1.py without having to first save the scripts into a folder. Note: the code is just a simplified example of what I have.

Answer 1

There are several approaches for custom import handling in IronPython and Python in general. Most concepts are defined in PEP 0302 (New Import Hooks).

Two python mechanisms that can solve the requirements are meta_path and path_hooks. Both can be implemented in Python or (in case of IronPython) C#/.NET as well. Given that the question concerns hosting IronPython from C# implementing the import infrastructure can work either way.

Using meta_path

IronPython ships with ResourceMetaPathImporter which allows you to have a ZIP-archive containing your scripts as an embedded resource. Assuming such an archive is called scripts.zip contained in the currently executing assembly, the required setup could look like:

var engine = Python.CreateEngine();
var sysScope = engine.GetSysModule();
List metaPath = sysScope.GetVariable("meta_path");
var importer = new ResourceMetaPathImporter(Assembly.GetExecutingAssembly(), "scripts.zip");
metaPath.Add(importer);
sysScope.SetVariable("meta_path", metaPath);

This approach works well if the assemblies and scripts are known and ZIP-packaging does not disturb the development process.

Using path_hooks

Path hooks contain a chain of importers that are queried for all items in sys.path in order to determine if they can handle a given path. An importer similar to zipimport.cs but responsible for embedded resources in DLLs/EXEs instead of ZIP archives. This could provide a more general approach that handles additional files by just adding a DLL to the path.

Using PlatformAdaptationLayer

A third approach works by providing a PlatformAdaptationLayer which is part of Microsoft.Scripting/IronPython. This answer shows a full working example of an platform adaption layer resolving embedded resources of a predefined assembly and package namespace.

General note: Related issue/discussion on github.