Trouble getting paho-mqtt (Python 3.9) to connect to Ably

First attempt to get paho-mqtt working with Ably. I'm sort of translating the following NodeJS example (which works for me) to Python. In the Python version, I don't seem to get any CONNACK back from the server.

NodeJS Example

// Using https://www.npmjs.com/package/mqtt

var [usr, pwd] = process.env['ABLY_API_KEY'].split(':');

let mqtt = require('mqtt');
let options = {
  keepAlive: 15,
  username: usr,
  password: pwd,
  port: 8883
};
let client = mqtt.connect('mqtts:mqtt.ably.io', options);

let channel = '[product:flight-data/flight-data]flight';
client.on('connect', () => {
  console.log('connected!')
  client.subscribe(channel);
});
client.on('message', (topic, message) => {
  console.log('Message received', message.toString());
});

My Python version (with logger added for debugging output)

mport os
import paho.mqtt.client as mqtt 


import logging

def create_logger():
    logger = logging.getLogger('mqtt debug')
    logger.setLevel(logging.DEBUG)
    ch = logging.StreamHandler()
    ch.setLevel(logging.DEBUG)
    formatter = logging.Formatter('%(asctime)s - %(name)s - %(levelname)s - %(message)s')
    ch.setFormatter(formatter)
    logger.addHandler(ch)
    return logger


host = 'mqtt.ably.io'
port =  8883
channel = '[product:flight-data/flight-data]flight'


def on_connect(client, userdata, flags, rc):
    print("Connected with result code "+str(rc))

    # Subscribing in on_connect() means that if we lose the connection and
    # reconnect then subscriptions will be renewed.
    client.subscribe(channel)

def on_message(client, userdata, msg):
    print(msg.topic+" "+str(msg.payload))

api_key = os.environ.get('ABLY_API_KEY', '')
username, passwd = api_key.split(':')
print(username)


mqttc = mqtt.Client(client_id="Test", clean_session=True, userdata=None)
mqttc.enable_logger(logger=create_logger())
mqttc.on_connect = on_connect
mqttc.on_message = on_message
mqttc.username_pw_set(username, passwd)
mqttc.connect(host, port=port, keepalive=15)


mqttc.loop_forever()

I get logger output like this

2022-06-20 20:03:39,956 - mqtt debug - DEBUG - Sending CONNECT (u1, p1, wr0, wq0, wf0, c1, k15) client_id=b'Test'
2022-06-20 20:03:56,040 - mqtt debug - DEBUG - Sending CONNECT (u1, p1, wr0, wq0, wf0, c1, k15) client_id=b'Test'

And this repeats until I ctrl-C out of the program.

However, testing very similar code with another broker seems to work. Here's the working test (cutting out the imports and create_logger function which are identical to the above)

(snip)

port = 1883
host = 'broker.hivemq.com'
channel = '$SYS/#'


def on_connect(client, userdata, flags, rc):
    print("Connected with result code "+str(rc))

    # Subscribing in on_connect() means that if we lose the connection and
    # reconnect then subscriptions will be renewed.
    client.subscribe(channel)

# The callback for when a PUBLISH message is received from the server.
def on_message(client, userdata, msg):
    print(msg.topic+" "+str(msg.payload))


mqttc = mqtt.Client(client_id="Test", clean_session=True, userdata=None)
mqttc.enable_logger(logger=create_logger())
mqttc.on_connect = on_connect
mqttc.on_message = on_message
mqttc.connect(host, port=port, keepalive=15)

mqttc.loop_forever()

This time I get expected output

2022-06-20 20:02:51,380 - mqtt debug - DEBUG - Sending CONNECT (u0, p0, wr0, wq0, wf0, c1, k15) client_id=b'Test'
2022-06-20 20:02:51,496 - mqtt debug - DEBUG - Received CONNACK (0, 0)
Connected with result code 0

So, in summary, I can connect to Ably in JS but not in Python. Yet my Python code is able to connect to other brokers. I'm sure I'm overlooking something I need to do in paho-mqtt, but have not been able to figure it out.

Solution

Here's a link to a working Ably + Python MQTT gist on my github: https://gist.github.com/Ugbot/f7bd307e74a804a1529c8f63bc597aa2

Its using asyncio_mqtt rather than the paho library but its what I had on hand.

I'll try and get your version working next and host up a gist of that...

EDIT:

I got your version working, it took a couple of steps, firstly you need to be sure you've adding hte flights subscription to your account (this is the easy bit) Next you need to change it to port 1883 from 8883.

Here is my working gist: https://gist.github.com/Ugbot/fc2ebc17f7dfcf7973f6b441d90f3766