IPython ipengineapp creation with keyword arguments

Question

I am trying to write a script that will start an new engine.

Using some code from IPython source I have: [engines.py]

def make_engine():
    from IPython.parallel.apps import ipengineapp as app
    app.launch_new_instance()

if __name__ == '__main__':
    make_engine(file='./profiles/security/ipcontroller-engine.json', config='./profiles/e2.py')

if I run this with python engines.py in the command line I run into a configuration problem and my traceback is:

Traceback (most recent call last):
  File "engines.py", line 30, in <module>
    make_engine(file='./profiles/security/ipcontroller-engine.json', config='./profiles/e2.py')
  File "engines.py", line 20, in make_engine
    app.launch_new_instance(**kwargs)
  File "/Users/martin/anaconda/lib/python2.7/site-packages/IPython/config/application.py", line 562, in launch_instance
    app = cls.instance(**kwargs)
  File "/Users/martin/anaconda/lib/python2.7/site-packages/IPython/config/configurable.py", line 354, in instance
    inst = cls(*args, **kwargs)
  File "<string>", line 2, in __init__
  File "/Users/martin/anaconda/lib/python2.7/site-packages/IPython/config/application.py", line 94, in catch_config_error
    app.print_help()
  File "/Users/martin/anaconda/lib/python2.7/site-packages/IPython/config/application.py", line 346, in print_help
    self.print_options()
  File "/Users/martin/anaconda/lib/python2.7/site-packages/IPython/config/application.py", line 317, in print_options
    self.print_alias_help()
  File "/Users/martin/anaconda/lib/python2.7/site-packages/IPython/config/application.py", line 281, in print_alias_help
    cls = classdict[classname]
KeyError: 'BaseIPythonApplication'

if I do a super ugly hack like the following, it works:

def make_engine():
    from IPython.parallel.apps import ipengineapp as app
    app.launch_new_instance()

if __name__ == '__main__':
    from sys import argv
    argv = ['--file=./profiles/security/ipcontroller-engine.json', '--config=./profiles/e2.py'] #OUCH this is ugly!
    make_engine()

Why can't I pass the keyword arguments in the launch_new_instance method?

What are the right keyword arguments?

Where can I get the entry point to entering my configuration options?

Thanks,

Martin

Answer 1

The way to instantiate a new ipengine using the IPEngineApp api is:

def make_engine():
    from IPython.parallel.apps.ipengineapp import IPEngineApp
    lines1 ="a_command()"
    app1 = IPEngineApp()
    app1.url_file = './profiles/security/ipcontroller-engine.json'
    app1.cluster_id = 'e2'
    app1.startup_command = lines1
    app1.init_engine()
    app1.start()

However, this starts a new ipengine process that takes control of the script execution process, so there is no way I can start multiple engines in the same script using this method.

Thus I had to fallback on the subprocess module to spawn all additional new ipengines:

import subprocess
import os

pids = []
for num in range(1,3):
    args = ["ipengine", "--config", os.path.abspath("./profiles/e%d.py" % num), "--file",os.path.abspath( "./profiles/security/ipcontroller-engine.json") ]
    pid = subprocess.Popen(args).pid
    pids.append(pid)