Search code examples
pythonbottle

Running 2 bottle apps at the same time using multiprocess.Pool


I came into a situation where I need to run 2 applications (on a different port) at the same time. I understand this is probably not the best idea/design but please bear with me.

The main problem is that bottle.run is blocking. The obvious workaround was to use multiprocessing.Pool, but I guess it is not that obvious.

This works (running the same app on 2 different ports):

from multiprocessing import Pool
import bottle

app = bottle.Bottle()

@app.route('/')
def index():
    return 'hi'


def run_app(port):
    bottle.run(port=port)


if __name__ == '__main__':
    pool = Pool(2)

    ps = [pool.apply_async(run_app, (port,))
          for port in (8081, 8082)]

    [p.get() for p in ps]  # YES, using a list comp for side-effect

Output is

Bottle v0.12.15 server starting up (using WSGIRefServer())...
Listening on http://127.0.0.1:8081/
Hit Ctrl-C to quit.

Bottle v0.12.15 server starting up (using WSGIRefServer())...
Listening on http://127.0.0.1:8082/
Hit Ctrl-C to quit.

However, trying to use 2 different apps just does not work.

from multiprocessing import Pool
import bottle

app1 = bottle.Bottle()
app2 = bottle.Bottle()
apps = [app1, app2]


@app1.route('/')
def index_app1():
    return 'app1'


@app2.route('/')
def index_app2():
    return 'app2'


def run_app(app, port):
    bottle.run(app, port=port)


if __name__ == '__main__':
    pool = Pool(2)

    ps = [pool.apply_async(run_app, (app, port))
          for app, port in zip(apps, [8081, 8082])]

    [p.get() for p in ps]

This outputs

Traceback (most recent call last):
  File "test.py", line 29, in <module>
    [p.get() for p in ps]
  File "test.py", line 29, in <listcomp>
    [p.get() for p in ps]
  File "D:\Python37\Lib\multiprocessing\pool.py", line 657, in get
    raise self._value
  File "D:\Python37\Lib\multiprocessing\pool.py", line 431, in _handle_tasks
    put(task)
  File "D:\Python37\Lib\multiprocessing\connection.py", line 206, in send
    self._send_bytes(_ForkingPickler.dumps(obj))
  File "D:\Python37\Lib\multiprocessing\reduction.py", line 51, in dumps
    cls(buf, protocol).dump(obj)
AttributeError: Can't pickle local object 'ConfigDict.__init__.<locals>.<lambda>'

UPDATE

Changing the first example to:

.
.
.

def run_app(app, port):
    # instead of bottle.run(port=port) which uses default_app()
    app.run(port=port) # or bottle.run(app=app, port=port)
.
.
.
ps = [pool.apply_async(run_app, (app, port))
      for port in (8081, 8082)]

is also causing the same error


Solution

  • Turns out the appropriate way to tackle this is to use Bottle.mount. This works perfectly:

    import bottle
    
    parent_app = bottle.Bottle()
    child_app = bottle.Bottle()
    
    
    @parent_app.route('/')
    def index_app1():
        return 'parent_app'
    
    
    @child_app.route('/')
    def index_app2():
        return 'child_app'
    
    
    if __name__ == '__main__':
        parent_app.mount('/child_app/', child_app)
        parent_app.run()