Search code examples
pythonipython-parallel

ipyparallel view.sync_import does not bind to additional name

i have the following issue with ipyparallel (using python 3.4). The view.sync_imports does neglect the extra module name to which i try to bind them with the 'as' directive.

Example: 

c = Client(profile='slurm')
v = c[:]
with v.sync_imports():

    import pandas as pd

Output: importing pandas on engine(s)

So pandas is only available under the name 'pandas'. Anyone a suggestion what i can do about this?

I also tried the %px magic, does not help

Solution

  • When you do

    import pandas as pd

    Python interprets that as two operations: import and assignment

    import pandas
pd = pandas

    sync_imports() is implemented as an import hook, which (as far as I can tell) only has access to the import part, and not the assignment part. That means that sync_imports can only support import pandas and not import pandas as pd.

    However, %px should work, as it is just executed directly:

    import pandas as pd     # local
%px import pandas as pd # remote

    Or as a cell magic with --local, if you want to do shared initialization of all engines and the local workspace:

    %%px --local
import pandas as pd
def defined_everywhere():
    """..."""