Does on_message in paho mqtt run in a new thread?

Question

Lets assume I have subscribed to some topic and the mosquitto server is continuosly publishing some message on that topic.

How does the on_message method get invoked when a new message arrives from mosquitto broker? Does it run on the main thread thereby blocking it and only processing the next message when the current one is processed or does it spawn a new thread with the on_message method every time a new message arrives?

import paho.mqtt.client as mqtt

def on_message(client, userdata, message):
    print message.payload
    return True

topics = ["topic1",]
client = mqtt.Client("testclient",protocol=mqtt.MQTTv31,clean_session=True)
client.on_message=on_message
client.connect("127.0.0.1", 1883, 60)
for tpc in topics:
    client.subscribe(tpc,0)
client.loop_forever()

Answer 1

The callbacks all run on the clients network thread.

What that is depends on how you "start" the network thread.

If you use the client.loop_forever() method then this uses the current thread to run the client's network thread and as such blocks at that call.

If you user the client.loop_start() method then this starts a new thread in the background to run the network loop and all the callbacks on.

All callbacks block the execution of the network thread, this is why you should not run long running or blocking tasks directly in the callback. If you have long running processing to do you should queue the incoming message and have another thread (or pool of threads) deal with it.

p.s. for the code you have posted you should really move the for loop and the calls to subscribe to the on_connect callback so that:

1. they only get run on a successful connection
2. the network thread is running to deal with sending the packets for the subscriptions and handling any incoming messages that might arrive before you've finished subscribing to all the topics (if the array is longer than 1)