PyBluez over HCI working oddly on Raspberry Pi Compute Module with ESP32

I'm using a Raspberry PI compute module 3 in a custom motherboard connected to an ESP32 using HCI over UART to connect to a mobile phone over bluetooth. It partially works: The phone is able to pair with the device, the device can see the phone in the bluetoothctl UI, SDPtool can see the services (RFCOMM) exposed by the device.

However, when I use pybluez, certain functionality does not work: First I tried using the device as the bluetooth slave. This is my preferred configuration. btmon showed the phone trying to connect, but pybluez never gets past sock.accept(). The code is pretty much exactly what you see in the example on pybluez github, and I've used the same code successfully with a stock Raspberry Pi Zero W (with the integrated bluetooth) successfully.

The phone shows the following error in the socket.connect() command: W/System.err: java.io.IOException: read failed, socket might closed or timeout, read ret: -1 W/System.err: at android.bluetooth.BluetoothSocket.readAll(BluetoothSocket.java:698) W/System.err: at android.bluetooth.BluetoothSocket.readInt(BluetoothSocket.java:710) W/System.err: at android.bluetooth.BluetoothSocket.connect(BluetoothSocket.java:387)

Hard to find anything of value in the Pi device's logs, but I did see this in btmon:

ACL Data RX: Handle 128 flags 0x02 dlen 14 [hci0] 75.711662 L2CAP: Command Reject (0x01) ident 5 len 6 Reason: Invalid CID in request (0x0002) Destination CID: 0 Source CID: 0

I also tried using the Android hack for connecting directly using a channel ID rather than a UUID (essentially skipping the SDP lookup) - I got the channel ID using sdptool browse local, and this resulted in the same error.

After exhaustively trying different things I switched gears to trying to use the phone as the bluetooth slave instead of the device. Here I see more success, but still things aren't working as expected: discover_devices from pybluez doesn't report any devices, although the phone and bluetoothctl both show the devices as paired. However, if I hardcode in the device address I see in bluetoothctl, I am actually able to connect.

I'm convinced that something is screwed up with the bluetooth configuration, perhaps the versions of bluez or pybluez, but I can't find anyone else who has reported these sorts of issues. I'd note that I'm using the compatibility mode switch on the bluetoothd service (otherwise it doesn't work at all). Bluez version is 5.43 I believe.

Has anyone seen anything like this? Any suggestions for troubleshooting these issues (different versions of bluez, etc)?

Solution

I ultimately did solve this, but I'm not sure which change did it:

I reduced the baud rate to 115200.
I switched it to classic bluetooth in the firmware flash.
I disabled bluetooth sleep.

One of these three changes must have done the trick

ret = esp_bt_controller_enable(ESP_BT_MODE_CLASSIC_BT);
if (ret != ESP_OK) {
    ESP_LOGE(tag, "Bluetooth Controller initialize failed: %s", esp_err_to_name(ret));
    return;
}

ret = esp_bt_sleep_disable();
if (ret != ESP_OK) {
    ESP_LOGE(tag, "Bluetooth Sleep Disable Failed: %s", esp_err_to_name(ret));
    return;
}