I'm trying to connect a DJI Osmo Action 3 in USB webcam mode to a Linux video pipeline. The camera supports both MJPG and H264:
$ v4l2-ctl -d /dev/video1 --list-formats-ext
ioctl: VIDIOC_ENUM_FMT
Type: Video Capture
[0]: 'MJPG' (Motion-JPEG, compressed)
Size: Discrete 1280x720
Interval: Discrete 0.033s (30.000 fps)
Size: Discrete 1920x1080
Interval: Discrete 0.033s (30.000 fps)
[1]: 'H264' (H.264, compressed)
Size: Discrete 1280x720
Interval: Discrete 0.033s (30.000 fps)
Size: Discrete 1920x1080
Interval: Discrete 0.033s (30.000 fps)
The MJPG output appears to be much lower quality than the H264 output, so I want to negotiate the latter. I believe the camera is able to output H264 on Windows because the output is much higher quality and the device does not support setting the JPEG compression level:
$ v4l2-ctl --all -d /dev/video1
Driver Info:
Driver name : uvcvideo
Card type : OsmoAction3: OsmoAction3
Bus info : usb-xhci-hcd.11.auto-1
Driver version : 5.10.66
Capabilities : 0x84a00001
Video Capture
Metadata Capture
Streaming
Extended Pix Format
Device Capabilities
Device Caps : 0x04200001
Video Capture
Streaming
Extended Pix Format
Media Driver Info:
Driver name : uvcvideo
Model : OsmoAction3: OsmoAction3
Serial : 123456789ABCDEF
Bus info : usb-xhci-hcd.11.auto-1
Media version : 5.10.66
Hardware revision: 0x00000504 (1284)
Driver version : 5.10.66
Interface Info:
ID : 0x03000002
Type : V4L Video
Entity Info:
ID : 0x00000001 (1)
Name : OsmoAction3: OsmoAction3
Function : V4L2 I/O
Flags : default
Pad 0x01000007 : 0: Sink
Link 0x0200000d: from remote pad 0x100000a of entity 'Processing 2': Data, Enabled, Immutable
Priority: 2
Video input : 0 (Camera 1: ok)
Format Video Capture:
Width/Height : 1920/1080
Pixel Format : 'MJPG' (Motion-JPEG)
Field : None
Bytes per Line : 0
Size Image : 4147200
Colorspace : Default
Transfer Function : Default (maps to Rec. 709)
YCbCr/HSV Encoding: Default (maps to ITU-R 601)
Quantization : Default (maps to Full Range)
Flags :
Crop Capability Video Capture:
Bounds : Left 0, Top 0, Width 1920, Height 1080
Default : Left 0, Top 0, Width 1920, Height 1080
Pixel Aspect: 1/1
Selection Video Capture: crop_default, Left 0, Top 0, Width 1920, Height 1080, Flags:
Selection Video Capture: crop_bounds, Left 0, Top 0, Width 1920, Height 1080, Flags:
Streaming Parameters Video Capture:
Capabilities : timeperframe
Frames per second: 30.000 (30/1)
Read buffers : 0
However, pulling video via H264 doesn't seem to work correctly with the UVC driver:
$ gst-launch-1.0 -vvv v4l2src device=/dev/video1 ! video/x-h264 ! fakesink
...
ERROR: from element /GstPipeline:pipeline0/GstV4l2Src:v4l2src0: Video device did not suggest any buffer size.
...
And same with v4l2-ctl:
$ v4l2-ctl -d /dev/video1 --set-fmt-video width=1920,height=1080,pixelformat=MJPG --stream-mmap --stream-count=50
<<<<<<<<<<<<<<<<<<<<<<<<<^C
$ v4l2-ctl -d /dev/video1 --set-fmt-video width=1920,height=1080,pixelformat=H264 --stream-mmap --stream-count=50
VIDIOC_REQBUFS returned -1 (Invalid argument)
I'm running kernel 5.10.66 and I do get a pretty nasty stack trace in dmesg:
[3445509.809183] WARNING: CPU: 1 PID: 125186 at drivers/media/common/videobuf2/videobuf2-core.c:806 vb2_core_reqbufs+0x2a4/0x380
[3445509.809188] Modules linked in: bcmdhd
[3445509.809200] CPU: 1 PID: 125186 Comm: v4l2-ctl Tainted: G W 5.10.66 #1
[3445509.809204] Hardware name: RK3588 MINIPC-MIZHUO LP4x V1.0 BlueBerry Board (DT)
[3445509.809210] pstate: 20400009 (nzCv daif +PAN -UAO -TCO BTYPE=--)
[3445509.809216] pc : vb2_core_reqbufs+0x2a4/0x380
[3445509.809221] lr : vb2_core_reqbufs+0x278/0x380
[3445509.809225] sp : ffffffc017deba90
[3445509.809229] x29: ffffffc017deba90 x28: ffffffc0112b40e0
[3445509.809235] x27: ffffff80429d0400 x26: 0000000000000000
[3445509.809241] x25: ffffff80c35c4018 x24: 0000000000000000
[3445509.809248] x23: ffffff80c35c4720 x22: ffffffc017debd18
[3445509.809254] x21: 0000000000000001 x20: 0000000000000000
[3445509.809260] x19: ffffff80c35c46b8 x18: 0000000000000000
[3445509.809266] x17: 0000000000000000 x16: 0000000000000000
[3445509.809272] x15: 0000000000000000 x14: 0000000000000000
[3445509.809278] x13: 0000000000000000 x12: 0000000000000000
[3445509.809284] x11: 0000000000000000 x10: 0000000000000000
[3445509.809290] x9 : ffffffc010ba5750 x8 : ffffffc010b73428
[3445509.809296] x7 : 0000000000000000 x6 : 0000000000000000
[3445509.809302] x5 : ffffffc010c0d1cc x4 : ffffff80c35c4720
[3445509.809308] x3 : 0000000000000001 x2 : ffffffc017debae4
[3445509.809314] x1 : 0000000000000000 x0 : 0000000000000001
[3445509.809320] Call trace:
[3445509.809325] vb2_core_reqbufs+0x2a4/0x380
[3445509.809331] vb2_reqbufs+0x54/0x70
[3445509.809338] uvc_request_buffers+0x40/0x6c
[3445509.809345] uvc_ioctl_reqbufs+0x58/0x9c
[3445509.809350] v4l_reqbufs+0x58/0x6c
[3445509.809354] __video_do_ioctl+0x258/0x384
[3445509.809359] video_usercopy+0x23c/0x490
So it seems like the uvcvideo driver isn't compatible with H264 output? Is that true? Is there a webcam driver that supports H264 output in Linux?
For anyone who stumbles upon this question in the future, I figured out the answer. It turns out that libuvc and the linux driver don't seem to support H264 the way the Osmo Action 4 negotiates it. Specifically, the Osmo Action 4 negotiates it as a frame-based format that has four cc H264, not the H264 as defined in the UVC specification. This appears to be because the Osmo Action 4 advertises support for UVC 1.0 which does not include H264 yet (it was added in UVC 1.5).
This also explains why Windows supports it as it uses a different underlying driver which presumably supports this workaround to transport H264.
To address this issue, I've created a user-space UVC driver which supports this negotiation format, among others that are missing in the base libuvc and linux drivers: https://github.com/kevmo314/go-uvc
Admittedly it would be nice to upstream this however that's beyond my realm of expertise and I have had a bad experience with proposing similar patches in the past.