Issue running self objects in mqtt paho callback function

I'm trying to write a script that saves mqtt data and sends it to influxDB. The issue I'm having is that the callback function of the mqtt-paho module keeps giving the error: AttributeError: 'Client' object has no attribute 'write_api'. I think this is because of the self in the internal 'Client' class of the mqtt-paho. My full script can be found below:

# Imported modules 
# standard time module
from datetime import datetime
import time
# InfluxDB specific modules 
from influxdb_client import InfluxDBClient, Point, WritePrecision
from influxdb_client.client.write_api import SYNCHRONOUS

#MQTT paho specific modules
import paho.mqtt.client as mqtt



class data_handler(): # Default namespaces are just for all the ESPs.


    def __init__(self, namespace_list=["ESP01","ESP02","ESP03","ESP04","ESP05","ESP06","ESP07","ESP08"]):
        
        # initialize influxdb client and define access token and data bucket
        token = "XXXXXXXXXX" # robotlab's token
        self.org = "Home"   
        self.bucket = "HomeSensors"
        self.flux_client = InfluxDBClient(url="http://localhost:8086", token=token)
        self.write_api = self.flux_client.write_api(write_options=SYNCHRONOUS)
        
        # Initialize and establish connection to MQTT broker 
        broker_address="XXX.XXX.XXX.XXX"
        self.mqtt_client = mqtt.Client("influx_client") #create new instance
        self.mqtt_client.on_message=data_handler.mqtt_message #attach function to callback
        self.mqtt_client.connect(broker_address) #connect to broker

        # Define list of namespaces
        self.namespace_list = namespace_list
        print(self.namespace_list)


    def mqtt_message(self, client, message):
        print("message received " ,str(message.payload.decode("utf-8")))
        print("message topic=",message.topic)
        print("message qos=",message.qos)
        print("message retain flag=",message.retain)
        
        sequence = [message.topic, message.payload.decode("utf-8")]
        self.write_api.write(self.bucket, self.org, sequence)

    def mqtt_listener(self):

        for namespace in self.namespace_list:
            self.mqtt_client.loop_start() #start the loop
            print("Subscribing to topics!")
            message = namespace+"/#"
            self.mqtt_client.subscribe(message, 0)
            time.sleep(4) # wait
            self.mqtt_client.loop_stop() #stop the loop


def main():
    influxHandler = data_handler(["ESP07"])
    influxHandler.mqtt_listener()

if  __name__ == '__main__':
    main()

The code works fine until I add self.someVariable in the callback function. What would be a good way to solve this problem? I don't really want to be making global variables hence why I chose to use a class.

Thanks in advance!

Solution

Dealing with self when there are multiple classes involved can get confusing. The paho library calls on_message as follows:

on_message(self, self._userdata, message)

So the first argument passed is the instance of Client so what you are seeing is expected (in the absence of any classes).

If the callback is a method object (which appears to be your aim) "the instance object is passed as the first argument of the function". This means your function would take four arguments and the definition be:

mqtt_message(self, client, userdata, msg)

Based upon this you might expect your application to fail earlier than it is but lets look at how you are setting the callback:

self.mqtt_client.on_message=data_handler.mqtt_message

datahandler is the class itself, not an instance of the class. This means you are effectively setting the callback to a static function (with no binding to any instance of the class - this answer might help). You need to change this to:

self.mqtt_client.on_message=self.mqtt_message

However this will not work as the method currently only takes three arguments; update the definition to:

def mqtt_message(self, client, userdata, msg)

with those changes I believe this will work (or at least you will find another issue :-) ).

An example might be a better way to explain this:

class mqtt_sim():
  def __init__(self):
    self._on_message = None
    
  @property
  def on_message(self):
     return self._on_message
      
  @on_message.setter
  def on_message(self, func):
    self._on_message = func

# This is what you are doing    
class data_handler1(): # Default namespaces are just for all the ESPs.
   def __init__(self):
      self.mqtt = mqtt_sim()      
      self.mqtt.on_message = data_handler1.mqtt_message    # xxxxx         
   def mqtt_message(self, client, message):        
      print("mqtt_message1", self, client, message) 

# This is what you should be doing      
class data_handler2(): # Default namespaces are just for all the ESPs.
   def __init__(self):
      self.mqtt = mqtt_sim()      
      self.mqtt.on_message = self.mqtt_message #attach function to callback             
   def mqtt_message(self, mqttself, client, message):        
      print("mqtt_message2", self, mqttself, client, message)    

# Lets try using both of the above      
d = data_handler1()
d.mqtt._on_message("self", "userdata", "message")

d = data_handler2()
d.mqtt._on_message("self", "userdata", "message")