python qt visual-studio-code raspberry-pi x11

qt.qpa.xcb: could not connect to display localhost:17.0 DIFFERENT set up

I run my Python code in VSCode on Windows 10 which ssh to the remote RPi 4. Got this error:

qt.qpa.xcb: could not connect to display :1
qt.qpa.plugin: Could not load the Qt platform plugin "xcb" in "/home/pi/mambaforge/envs/mb/lib/python3.8/site-packages/cv2/qt/plugins" even though it was found.
This application failed to start because no Qt platform plugin could be initialized. Reinstalling the application may fix this problem.

Available platform plugins are: xcb.

Stack trace (most recent call last):
#25   Object "python", at 0x558888297b, in 
#24   Object "/lib/aarch64-linux-gnu/libc.so.6", at 0x7f847bbe17, in __libc_start_main
#23   Object "python", at 0x55888840a3, in Py_BytesMain
#22   Object "python", at 0x5588883a3b, in Py_RunMain
#21   Object "python", at 0x558895f4eb, in PyRun_SimpleFileExFlags
#20   Object "python", at 0x558895f323, in 
#19   Object "python", at 0x558895bfcb, in 
#18   Object "python", at 0x558895beb7, in 
#17   Object "python", at 0x5588924057, in PyEval_EvalCode
#16   Object "python", at 0x5588924017, in PyEval_EvalCodeEx
#15   Object "python", at 0x5588923c4f, in _PyEval_EvalCodeWithName
#14   Object "python", at 0x5588881e9b, in _PyEval_EvalFrameDefault
#13   Object "python", at 0x558888fe7f, in _PyObject_MakeTpCall
#12   Object "python", at 0x558888f7af, in 
#11   Object "/home/pi/mambaforge/envs/mb/lib/python3.8/site-packages/cv2/cv2.abi3.so", at 0x7f77b8a43b, in 
#10   Object "/home/pi/mambaforge/envs/mb/lib/python3.8/site-packages/cv2/cv2.abi3.so", at 0x7f7876e87f, in 
#9    Object "/home/pi/mambaforge/envs/mb/lib/python3.8/site-packages/cv2/cv2.abi3.so", at 0x7f7878089b, in 
#8    Object "/home/pi/mambaforge/envs/mb/lib/python3.8/site-packages/cv2/cv2.abi3.so", at 0x7f78774537, in 
#7    Object "/home/pi/mambaforge/envs/mb/lib/python3.8/site-packages/cv2/../opencv_contrib_python.libs/libQt5Widgets-b1296c1e.so.5.15.0", at 0x7f7585db03, in QtOpenCVPython::QApplicationPrivate::init()
#6    Object "/home/pi/mambaforge/envs/mb/lib/python3.8/site-packages/cv2/../opencv_contrib_python.libs/libQt5Gui-a45894d1.so.5.15.0", at 0x7f74f9fa77, in QtOpenCVPython::QGuiApplicationPrivate::init()
#5    Object "/home/pi/mambaforge/envs/mb/lib/python3.8/site-packages/cv2/../opencv_contrib_python.libs/libQt5Core-9e162752.so.5.15.0", at 0x7f749710ff, in QtOpenCVPython::QCoreApplicationPrivate::init()
#4    Object "/home/pi/mambaforge/envs/mb/lib/python3.8/site-packages/cv2/../opencv_contrib_python.libs/libQt5Gui-a45894d1.so.5.15.0", at 0x7f74f9ce07, in QtOpenCVPython::QGuiApplicationPrivate::createEventDispatcher()
#3    Object "/home/pi/mambaforge/envs/mb/lib/python3.8/site-packages/cv2/../opencv_contrib_python.libs/libQt5Gui-a45894d1.so.5.15.0", at 0x7f74f9c963, in QtOpenCVPython::QGuiApplicationPrivate::createPlatformIntegration()
#2    Object "/home/pi/mambaforge/envs/mb/lib/python3.8/site-packages/cv2/../opencv_contrib_python.libs/libQt5Core-9e162752.so.5.15.0", at 0x7f7477213b, in QtOpenCVPython::QMessageLogger::fatal(char const*, ...) const
#1    Object "/lib/aarch64-linux-gnu/libc.so.6", at 0x7f847bba9f, in abort
#0    Object "/lib/aarch64-linux-gnu/libc.so.6", at 0x7f847ceeac, in gsignal
Aborted (Signal sent by tkill() 5843 1000)
Aborted

Here is my C:\Users\mywname\config

Host 192.168.1.8
  HostName 192.168.1.8
  User pi
  ForwardX11 yes
  ForwardX11Trusted yes
  EnableSSHKeysign yes
  IdentityFile C:\Users\<myusername>\.ssh\id_rsa

A bit of the code context: My Python code which is on RPi and calls OpenCV cv.image() to shows the video from a camera connected to the RPi. I want to run the code from my VSCode on Windows 10 and see the Video window show up on my Windows 10.

I do not have QT installed on my Windows 10, I do have the opencv-contrib-python 4.7.0.72 on my RPi and I wish not to change it to the opencv headless as one of the Python lib needs the opencv-contrib-python 4.7.0.72.

What I did Did some research and found I need to enable X11 Forwarding, so I installed Remote X11 and Remote X11(SSH), also set up my SSH login with private and pub key pairs and tested to run the ssh from terminal directly, it works without asking me the password.

Also found that I need to use the magic: export DISPLAY=:0

I tried, instead of showing the Video on my Windows 10, it shows the video on RPi, which makes sense. So I changed it to export DISPLAY=:1 and then I got the error.

So my question is how can I get the video window showing up on my Windows 10.

Solution

I finally figured out how to make this happen:

install the X Server on Windows VcXsrv Windows X Server
I set up SSH key based authentication, (not sure whether this is mandatory)
I installed both Remote X11 (SSH) and Remote X11 extensions on VSCode on both Windows and RPi sides.
For the X11 (SSH) extension settings, I set XAuth Permission Level is set to trusted.
Go to the Remote [SSH: ] settings tab and change the Display Command to: echo DISPLAY=$DISPLAY
I start the XLauncher(X Server) on Windows and click enter to choose all default settings til end.
Run my code and the cv2.imshow() finally shows up in my Windows~!!