Paho MQTT not working with Flask(Google app engine)

Question

In my Flask(Google app engine) application I need to publish a message to the Mosquitto broker, which is running locally in the terminal to the port 1884.

    import paho.mqtt.client as mqtt
    # ...
    try:
        # publish.single("Car/Command1", dict, hostname='localhost', port=1884)
        client = mqtt.Client('Flask_Publisher')
        client.connect("localhost", 1884, 60)
        client.publish("mytopic", dict)
        client.disconnect()
    except Exception as e:
        logging.warning("Exception occurred: {}".format(e))
        error = e
    # ...

I run the Mosquitto broker locally on the port 1884 in the terminal by typing the following command: sudo mosquitto -p 1884

For testing purposes I write a python script which has the role of a mqtt subscriber.

But the issue is that when I try to run Flask it gives me the following error (as caught from Exception) when it tries to publish the message: [Errno 13] Permission denied

EDIT: the full traceback

    ERROR    2018-08-30 16:47:52,665 wsgi.py:279] 
Traceback (most recent call last):
  File "/home/santoryu/Scrivania/google-cloud-sdk/platform/google_appengine/google/appengine/runtime/wsgi.py", line 267, in Handle
    result = handler(dict(self._environ), self._StartResponse)
  File "/home/santoryu/Scrivania/IOT/gae/lib/flask/app.py", line 2309, in __call__
    return self.wsgi_app(environ, start_response)
  File "/home/santoryu/Scrivania/IOT/gae/lib/flask/app.py", line 2295, in wsgi_app
    response = self.handle_exception(e)
  File "/home/santoryu/Scrivania/IOT/gae/lib/flask_cors/extension.py", line 161, in wrapped_function
    return cors_after_request(app.make_response(f(*args, **kwargs)))
  File "/home/santoryu/Scrivania/IOT/gae/lib/flask_restful/__init__.py", line 273, in error_router
    return original_handler(e)
  File "/home/santoryu/Scrivania/IOT/gae/lib/flask/app.py", line 1741, in handle_exception
    reraise(exc_type, exc_value, tb)
  File "/home/santoryu/Scrivania/IOT/gae/lib/flask/app.py", line 2292, in wsgi_app
    response = self.full_dispatch_request()
  File "/home/santoryu/Scrivania/IOT/gae/lib/flask/app.py", line 1815, in full_dispatch_request
    rv = self.handle_user_exception(e)
  File "/home/santoryu/Scrivania/IOT/gae/lib/flask_cors/extension.py", line 161, in wrapped_function
    return cors_after_request(app.make_response(f(*args, **kwargs)))
  File "/home/santoryu/Scrivania/IOT/gae/lib/flask_restful/__init__.py", line 273, in error_router
    return original_handler(e)
  File "/home/santoryu/Scrivania/IOT/gae/lib/flask/app.py", line 1718, in handle_user_exception
    reraise(exc_type, exc_value, tb)
  File "/home/santoryu/Scrivania/IOT/gae/lib/flask/app.py", line 1813, in full_dispatch_request
    rv = self.dispatch_request()
  File "/home/santoryu/Scrivania/IOT/gae/lib/flask/app.py", line 1799, in dispatch_request
    return self.view_functions[rule.endpoint](**req.view_args)
  File "/home/santoryu/Scrivania/IOT/gae/app/handlers/my_handler.py", line 44, in funzione1
    client = mqtt.Client('Flask_Publisher')
  File "/home/santoryu/Scrivania/IOT/gae/lib/paho/mqtt/client.py", line 498, in __init__
    self._sockpairR, self._sockpairW = _socketpair_compat()
  File "/home/santoryu/Scrivania/IOT/gae/lib/paho/mqtt/client.py", line 238, in _socketpair_compat
    listensock.bind(("127.0.0.1", 0))
  File "/home/santoryu/Scrivania/google-cloud-sdk/platform/google_appengine/google/appengine/dist27/socket.py", line 227, in meth
    return getattr(self._sock,name)(*args)
  File "/home/santoryu/Scrivania/google-cloud-sdk/platform/google_appengine/google/appengine/api/remote_socket/_remote_socket.py", line 679, in bind
    raise _SystemExceptionFromAppError(e)
error: [Errno 13] Permission denied

Answer 1

The issue is that the App Engine SDK provides a sandboxed version of Python's socket module. When the MQTT client library is trying to bind to a local port, it's using this sandboxed module instead of the standard library module it was expecting, and fails to bind to localhost.

You can work around this by replacing the GAE socket module with the original standard library module when running in development mode, allowing you to bind to localhost.

In your appengine_config.py file:

import os

# Only do the following if we're in development mode
if os.environ.get('SERVER_SOFTWARE', '').startswith('Development'):
    import imp
    import inspect

    # Whitelist the ssl and socket modules
    from google.appengine.tools.devappserver2.python import sandbox
    sandbox._WHITE_LIST_C_MODULES += ['_ssl', '_socket']

    # Use the standard library socket module instead
    real_os_src_path = os.path.realpath(inspect.getsourcefile(os))
    real_socket = os.path.join(os.path.dirname(real_os_src_path), 'socket.py')
    imp.load_source('socket', real_socket)