"Unable to start capturing: Invalid argument" on Raspberry Pi 2B

[fphammerle]

Hi,

I get "Unable to start capturing: Invalid argument" on a Raspberry Pi 2B running Raspberry Pi OS Bullseye:

$ sudo -g video ustreamer --resolution 1920x1080
-- INFO  [13606.486      main] -- Using internal blank placeholder
-- INFO  [13606.487      main] -- Listening HTTP on [127.0.0.1]:8080
-- INFO  [13606.488    stream] -- Using V4L2 device: /dev/video0
-- INFO  [13606.488    stream] -- Using desired FPS: 0
-- INFO  [13606.488      http] -- Starting HTTP eventloop ...
================================================================================
-- INFO  [13606.493    stream] -- Device fd=8 opened
-- INFO  [13606.493    stream] -- Using input channel: 0
-- INFO  [13606.493    stream] -- Using resolution: 1920x1080
-- INFO  [13606.493    stream] -- Using pixelformat: YUYV
-- INFO  [13606.493    stream] -- Querying HW FPS changing is not supported
-- INFO  [13606.493    stream] -- Using IO method: MMAP
-- INFO  [13606.523    stream] -- Requested 5 device buffers, got 5
-- ERROR [13606.524    stream] -- Unable to start capturing: Invalid argument
-- INFO  [13606.542    stream] -- Device fd=8 closed
-- INFO  [13606.542    stream] -- Sleeping 1 seconds before new stream init ...
================================================================================
-- INFO  [13607.548    stream] -- Device fd=8 opened
-- INFO  [13607.548    stream] -- Using input channel: 0
-- INFO  [13607.548    stream] -- Using resolution: 1920x1080
-- INFO  [13607.548    stream] -- Using pixelformat: YUYV
-- INFO  [13607.548    stream] -- Querying HW FPS changing is not supported
-- INFO  [13607.548    stream] -- Using IO method: MMAP
-- INFO  [13607.578    stream] -- Requested 5 device buffers, got 5
-- ERROR [13607.580    stream] -- Unable to start capturing: Invalid argument
-- INFO  [13607.597    stream] -- Device fd=8 closed
-- INFO  [13607.597    stream] -- Sleeping 1 seconds before new stream init ...
^C-- INFO  [13608.282      main] -- ===== Stopping by SIGINT =====
-- INFO  [13608.283      http] -- HTTP eventloop stopped
-- INFO  [13608.598      main] -- Bye-bye
$ journalctl --lines=3 -o cat
   someone : TTY=pts/1 ; PWD=/home/someone ; USER=someone ; GROUP=video ; COMMAND=/usr/bin/ustreamer>
unicam 3f801000.csi: Failed to start media pipeline: -22
unicam 3f801000.csi: Failed to start media pipeline: -22

$ cat /proc/cpuinfo | tail -n 4 | grep -v Serial
Hardware	: BCM2835
Revision	: a01041
Model		: Raspberry Pi 2 Model B Rev 1.1
$ cat /etc/issue
Raspbian GNU/Linux 11 \n \l
$ sudo -g video v4l2-ctl --info
Driver Info:
	Driver name      : unicam
	Card type        : unicam
	Bus info         : platform:3f801000.csi
	Driver version   : 5.10.92
	Capabilities     : 0xa5a00001
		Video Capture
		Metadata Capture
		Read/Write
		Streaming
		Extended Pix Format
		Device Capabilities
	Device Caps      : 0x25200001
		Video Capture
		Read/Write
		Streaming
		Extended Pix Format
Media Driver Info:
	Driver name      : unicam
	Model            : unicam
	Serial           : 
	Bus info         : platform:3f801000.csi
	Media version    : 5.10.92
	Hardware revision: 0x00000000 (0)
	Driver version   : 5.10.92
Interface Info:
	ID               : 0x03000005
	Type             : V4L Video
Entity Info:
	ID               : 0x00000003 (3)
	Name             : unicam-image
	Function         : V4L2 I/O
	Flags         : default
	Pad 0x01000004   : 0: Sink
	 Link 0x02000007: from remote pad 0x1000002 of entity 'ov5647 10-0036': Data, Enabled, Immutable
$ sudo -g video v4l2-ctl --list-formats-ext
ioctl: VIDIOC_ENUM_FMT
	Type: Video Capture

	[0]: 'YUYV' (YUYV 4:2:2)
		Size: Stepwise 16x16 - 16376x16376 with step 1/1
	[1]: 'UYVY' (UYVY 4:2:2)
		Size: Stepwise 16x16 - 16376x16376 with step 1/1
	[2]: 'YVYU' (YVYU 4:2:2)
		Size: Stepwise 16x16 - 16376x16376 with step 1/1
	[3]: 'VYUY' (VYUY 4:2:2)
		Size: Stepwise 16x16 - 16376x16376 with step 1/1
	[4]: 'RGBP' (16-bit RGB 5-6-5)
		Size: Stepwise 16x16 - 16376x16376 with step 1/1
	[5]: 'RGBR' (16-bit RGB 5-6-5 BE)
		Size: Stepwise 16x16 - 16376x16376 with step 1/1
	[6]: 'RGBO' (16-bit A/XRGB 1-5-5-5)
		Size: Stepwise 16x16 - 16376x16376 with step 1/1
	[7]: 'RGBQ' (16-bit A/XRGB 1-5-5-5 BE)
		Size: Stepwise 16x16 - 16376x16376 with step 1/1
	[8]: 'RGB3' (24-bit RGB 8-8-8)
		Size: Stepwise 16x16 - 16376x16376 with step 1/1
	[9]: 'BGR3' (24-bit BGR 8-8-8)
		Size: Stepwise 16x16 - 16376x16376 with step 1/1
	[10]: 'RGB4' (32-bit A/XRGB 8-8-8-8)
		Size: Stepwise 16x16 - 16376x16376 with step 1/1
	[11]: 'BA81' (8-bit Bayer BGBG/GRGR)
		Size: Stepwise 16x16 - 16376x16376 with step 1/1
	[12]: 'GBRG' (8-bit Bayer GBGB/RGRG)
		Size: Stepwise 16x16 - 16376x16376 with step 1/1
	[13]: 'GRBG' (8-bit Bayer GRGR/BGBG)
		Size: Stepwise 16x16 - 16376x16376 with step 1/1
	[14]: 'RGGB' (8-bit Bayer RGRG/GBGB)
		Size: Stepwise 16x16 - 16376x16376 with step 1/1
	[15]: 'pBAA' (10-bit Bayer BGBG/GRGR Packed)
		Size: Stepwise 16x16 - 16376x16376 with step 1/1
	[16]: 'BG10' (10-bit Bayer BGBG/GRGR)
		Size: Stepwise 16x16 - 16376x16376 with step 1/1
	[17]: 'pGAA' (10-bit Bayer GBGB/RGRG Packed)
		Size: Stepwise 16x16 - 16376x16376 with step 1/1
	[18]: 'GB10' (10-bit Bayer GBGB/RGRG)
		Size: Stepwise 16x16 - 16376x16376 with step 1/1
	[19]: 'pgAA' (10-bit Bayer GRGR/BGBG Packed)
		Size: Stepwise 16x16 - 16376x16376 with step 1/1
	[20]: 'BA10' (10-bit Bayer GRGR/BGBG)
		Size: Stepwise 16x16 - 16376x16376 with step 1/1
	[21]: 'pRAA' (10-bit Bayer RGRG/GBGB Packed)
		Size: Stepwise 16x16 - 16376x16376 with step 1/1
	[22]: 'RG10' (10-bit Bayer RGRG/GBGB)
		Size: Stepwise 16x16 - 16376x16376 with step 1/1
	[23]: 'pBCC' (12-bit Bayer BGBG/GRGR Packed)
		Size: Stepwise 16x16 - 16376x16376 with step 1/1
	[24]: 'BG12' (12-bit Bayer BGBG/GRGR)
		Size: Stepwise 16x16 - 16376x16376 with step 1/1
	[25]: 'pGCC' (12-bit Bayer GBGB/RGRG Packed)
		Size: Stepwise 16x16 - 16376x16376 with step 1/1
	[26]: 'GB12' (12-bit Bayer GBGB/RGRG)
		Size: Stepwise 16x16 - 16376x16376 with step 1/1
	[27]: 'pgCC' (12-bit Bayer GRGR/BGBG Packed)
		Size: Stepwise 16x16 - 16376x16376 with step 1/1
	[28]: 'BA12' (12-bit Bayer GRGR/BGBG)
		Size: Stepwise 16x16 - 16376x16376 with step 1/1
	[29]: 'pRCC' (12-bit Bayer RGRG/GBGB Packed)
		Size: Stepwise 16x16 - 16376x16376 with step 1/1
	[30]: 'RG12' (12-bit Bayer RGRG/GBGB)
		Size: Stepwise 16x16 - 16376x16376 with step 1/1
	[31]: 'pBEE' (14-bit Bayer BGBG/GRGR Packed)
		Size: Stepwise 16x16 - 16376x16376 with step 1/1
	[32]: 'BG14' (14-bit Bayer BGBG/GRGR)
		Size: Stepwise 16x16 - 16376x16376 with step 1/1
	[33]: 'pGEE' (14-bit Bayer GBGB/RGRG Packed)
		Size: Stepwise 16x16 - 16376x16376 with step 1/1
	[34]: 'GB14' (14-bit Bayer GBGB/RGRG)
		Size: Stepwise 16x16 - 16376x16376 with step 1/1
	[35]: 'pgEE' (14-bit Bayer GRGR/BGBG Packed)
		Size: Stepwise 16x16 - 16376x16376 with step 1/1
	[36]: 'GR14' (14-bit Bayer GRGR/BGBG)
		Size: Stepwise 16x16 - 16376x16376 with step 1/1
	[37]: 'pREE' (14-bit Bayer RGRG/GBGB Packed)
		Size: Stepwise 16x16 - 16376x16376 with step 1/1
	[38]: 'RG14' (14-bit Bayer RGRG/GBGB)
		Size: Stepwise 16x16 - 16376x16376 with step 1/1
	[39]: 'GREY' (8-bit Greyscale)
		Size: Stepwise 16x16 - 16376x16376 with step 1/1
	[40]: 'Y10P' (10-bit Greyscale (MIPI Packed))
		Size: Stepwise 16x16 - 16376x16376 with step 1/1
	[41]: 'Y10 ' (10-bit Greyscale)
		Size: Stepwise 16x16 - 16376x16376 with step 1/1
	[42]: 'Y12P' (12-bit Greyscale (MIPI Packed))
		Size: Stepwise 16x16 - 16376x16376 with step 1/1
	[43]: 'Y12 ' (12-bit Greyscale)
		Size: Stepwise 16x16 - 16376x16376 with step 1/1
	[44]: 'Y14P' (14-bit Greyscale (MIPI Packed))
		Size: Stepwise 16x16 - 16376x16376 with step 1/1
	[45]: 'Y14 ' (14-bit Greyscale)
		Size: Stepwise 16x16 - 16376x16376 with step 1/1
$ sudo -g video libcamera-vid --list-cameras 
[4:13:48.035845093] [4230]  INFO Camera camera_manager.cpp:293 libcamera v0.0.0+3384-44d59841
[4:13:48.083237034] [4231] ERROR CameraSensor camera_sensor.cpp:535 'ov5647 10-0036': Camera sensor does not support test pattern modes.
[4:13:48.166319146] [4231]  INFO RPI raspberrypi.cpp:1313 Registered camera /base/soc/i2c0mux/i2c@1/ov5647@36 to Unicam device /dev/media3 and ISP device /dev/media0
Available cameras
-----------------
0 : ov5647 [2592x1944] (/base/soc/i2c0mux/i2c@1/ov5647@36)
    Modes: 'SGBRG10_CSI2P' : 640x480 1296x972 1920x1080 2592x1944 
           'SGBRG8' : 640x480 

I installed ustreamer from "deb http://raspbian.raspberrypi.org/raspbian/ bullseye":

$ ustreamer --version
3.16
$ apt-cache show ustreamer | grep Version:
Version: 3.16-1

I tried upgrading to ustreamer v4.11 (source build), but the error persists.
According to strace, the error occurs in ioctl(8, VIDIOC_STREAMON, [V4L2_BUF_TYPE_VIDEO_CAPTURE]):

$ ./ustreamer --version
4.11
$ sudo -g video strace -f -e ioctl ./ustreamer --resolution 1920x1080 --format=uyvy
ioctl(2, TCGETS, {B38400 opost isig icanon echo ...}) = 0
-- INFO  [14867.371      main] -- Using internal blank placeholder
-- INFO  [14867.376      main] -- Listening HTTP on [127.0.0.1]:8080
strace: Process 4210 attached
strace: Process 4211 attached
-- INFO  [14867.378    stream] -- Using V4L2 device: /dev/video0
-- INFO  [14867.379    stream] -- Using desired FPS: 0
================================================================================
-- INFO  [14867.393      http] -- Starting HTTP eventloop ...
-- INFO  [14867.396    stream] -- Device fd=8 opened
[pid  4210] ioctl(8, VIDIOC_QUERYCAP, {driver="unicam", card="unicam", bus_info="platform:3f801000.csi", version=KERNEL_VERSION(5, 10, 92), capabilities=V4L2_CAP_VIDEO_CAPTURE|V4L2_CAP_EXT_PIX_FORMAT|V4L2_CAP_META_CAPTURE|V4L2_CAP_READWRITE|V4L2_CAP_STREAMING|V4L2_CAP_DEVICE_CAPS|0x20000000, device_caps=V4L2_CAP_VIDEO_CAPTURE|V4L2_CAP_EXT_PIX_FORMAT|V4L2_CAP_READWRITE|V4L2_CAP_STREAMING|0x20000000}) = 0
-- INFO  [14867.398    stream] -- Using input channel: 0
[pid  4210] ioctl(8, VIDIOC_S_INPUT, [0]) = 0
[pid  4210] ioctl(8, VIDIOC_S_FMT, {type=V4L2_BUF_TYPE_VIDEO_CAPTURE, fmt.pix={width=1920, height=1080, pixelformat=v4l2_fourcc('U', 'Y', 'V', 'Y') /* V4L2_PIX_FMT_UYVY */, field=V4L2_FIELD_ANY, bytesperline=3840, sizeimage=0, colorspace=V4L2_COLORSPACE_DEFAULT}} => {fmt.pix={width=1920, height=1080, pixelformat=v4l2_fourcc('U', 'Y', 'V', 'Y') /* V4L2_PIX_FMT_UYVY */, field=V4L2_FIELD_NONE, bytesperline=3840, sizeimage=4147200, colorspace=V4L2_COLORSPACE_SMPTE170M}}) = 0
-- INFO  [14867.399    stream] -- Using resolution: 1920x1080
-- INFO  [14867.400    stream] -- Using pixelformat: UYVY
[pid  4210] ioctl(8, VIDIOC_G_PARM, {type=V4L2_BUF_TYPE_VIDEO_CAPTURE}) = -1 ENOTTY (Inappropriate ioctl for device)
-- INFO  [14867.401    stream] -- Querying HW FPS changing is not supported
-- INFO  [14867.401    stream] -- Using IO method: MMAP
[pid  4210] ioctl(8, VIDIOC_REQBUFS, {type=V4L2_BUF_TYPE_VIDEO_CAPTURE, memory=V4L2_MEMORY_MMAP, count=5 => 5}) = 0
-- INFO  [14867.425    stream] -- Requested 5 device buffers, got 5
[pid  4210] ioctl(8, VIDIOC_QUERYBUF_TIME32, 0x76a97824) = 0
[pid  4210] ioctl(8, VIDIOC_QUERYBUF_TIME32, 0x76a97824) = 0
[pid  4210] ioctl(8, VIDIOC_QUERYBUF_TIME32, 0x76a97824) = 0
[pid  4210] ioctl(8, VIDIOC_QUERYBUF_TIME32, 0x76a97824) = 0
[pid  4210] ioctl(8, VIDIOC_QUERYBUF_TIME32, 0x76a97824) = 0
[pid  4210] ioctl(8, VIDIOC_QBUF_TIME32, 0x76a98e38) = 0
[pid  4210] ioctl(8, VIDIOC_QBUF_TIME32, 0x76a98e38) = 0
[pid  4210] ioctl(8, VIDIOC_QBUF_TIME32, 0x76a98e38) = 0
[pid  4210] ioctl(8, VIDIOC_QBUF_TIME32, 0x76a98e38) = 0
[pid  4210] ioctl(8, VIDIOC_QBUF_TIME32, 0x76a98e38) = 0
[pid  4210] ioctl(8, VIDIOC_STREAMON, [V4L2_BUF_TYPE_VIDEO_CAPTURE]) = -1 EINVAL (Invalid argument)
-- ERROR [14867.433    stream] -- Unable to start capturing: Invalid argument
-- INFO  [14867.449    stream] -- Device fd=8 closed
-- INFO  [14867.450    stream] -- Sleeping 1 seconds before new stream init ...
^Cstrace: Process 4209 detached
strace: Process 4210 detached
strace: Process 4211 detached
-- INFO  [14867.921      main] -- ===== Stopping by SIGINT =====
-- INFO  [14867.921      http] -- HTTP eventloop stopped

libcamera-vid works fine.

Any ideas how to solve this issue?

Thank you!

[fphammerle]

ustreamer works fine when I connect a TC358743 board instead of the camera.

[benjaminjacobreji]

@fphammerle

A simple fix for now is to enable Legacy Camera assuming you are on Raspbian Bullseye

sudo raspi-config

3 Interface Options > I1 Legacy Camera > Yes

sudo reboot

and voila it works (I hope xD, this is what I had to do)

My assumption is that ustreamer may be reliant on the older system and since the author is busy with the pikvm, the code hasn't been updated to the newer libcamera stack.
Hopefully someone picks it up and is able to look into it.
I might give it a shot, once I learn more about how the new system is implemented.

[fphammerle]

Thanks @benjaminjacobreji for your suggestion to revert to the legacy stack!

If possible, I would prefer keeping libcamera, as I also use libcamera-vid --post-process-file ... from time to time.
I guess I'll wait for a fix, thanks!

[noahwilliamsson]

I assume this is related to the recent (late 2021) introduction of the Media Controller API in newer Linux kernels and Raspbian Bullseye. Support for the Media Controller API was enabled for several camera modules including OV5647 (Pi Camera v1, end of life) and IMX219 (Pi Camera v2). It's not enabled for TC358743 though.
See this post bcm2835-unicam and the Media Controller API - this is now live on the forums.

On a fresh install of Raspbian Bullseye, you'll likely have camera_auto_detect=1 in /boot/config.txt. In this case, the appropriate device tree overlay will be loaded automatically for supported camera modules and the Media Controller API will be enabled.

If you have, say, an IMX219 module, you can disable support for the Media Controller API by loading the appropriate overlay and adding media-controller=0:

camera_auto_detect=0
start_x=1
dtoverlay=imx219,media-controller=0

I ran into the same issue as you with:

• Raspberry Pi Zero 2 with an IMX219 module
• Raspbian Bullseye with a 5.15.30 kernel from yesterday, obtained by running sudo rpi-update
• ustreamer built from the https://github.com/PiKVM/uStreamer/tree/m2m branch (not sure it matters but that's what I tested with)

My V4L2 settings after a fresh boot:

v4l2-ctl --list-devices

$ v4l2-ctl --list-devices
unicam (platform:3f801000.csi):
	/dev/video0
	/dev/video1
	/dev/media3

bcm2835-codec-decode (platform:bcm2835-codec):
	/dev/video10
	/dev/video11
	/dev/video12
	/dev/video18
	/dev/video31
	/dev/media2

bcm2835-isp (platform:bcm2835-isp):
	/dev/video13
	/dev/video14
	/dev/video15
	/dev/video16
	/dev/video20
	/dev/video21
	/dev/video22
	/dev/video23
	/dev/media0
	/dev/media1

v4l2-ctl --info --device /dev/video0

$ v4l2-ctl --info --device /dev/video0
Driver Info:
	Driver name      : unicam
	Card type        : unicam
	Bus info         : platform:3f801000.csi
	Driver version   : 5.15.30
	Capabilities     : 0xa5a00001
		Video Capture
		Metadata Capture
		Read/Write
		Streaming
		Extended Pix Format
		Device Capabilities
	Device Caps      : 0x25200001
		Video Capture
		Read/Write
		Streaming
		Extended Pix Format
Media Driver Info:
	Driver name      : unicam
	Model            : unicam
	Serial           : 
	Bus info         : platform:3f801000.csi
	Media version    : 5.15.30
	Hardware revision: 0x00000000 (0)
	Driver version   : 5.15.30
Interface Info:
	ID               : 0x03000006
	Type             : V4L Video
Entity Info:
	ID               : 0x00000004 (4)
	Name             : unicam-image
	Function         : V4L2 I/O
	Flags         : default
	Pad 0x01000005   : 0: Sink
	  Link 0x02000008: from remote pad 0x1000002 of entity 'imx219 10-0010': Data, Enabled, Immutable

$ v4l2-ctl --get-fmt-video --device /dev/video0
Format Video Capture:
	Width/Height      : 640/480
	Pixel Format      : 'YUYV' (YUYV 4:2:2)
	Field             : None
	Bytes per Line    : 1280
	Size Image        : 614400
	Colorspace        : sRGB
	Transfer Function : sRGB
	YCbCr/HSV Encoding: ITU-R 601
	Quantization      : Limited Range
	Flags             : 

That the Media Controller API is available can be seen in the Device Caps value 0x25200001, in which the V4L2_CAP_IO_MC (0x20000000) flag is set. You also see a /dev/media3 node along with the /dev/video0 node for the unicam device.

Below is the topology for the Image Signal Processor (ISP).

media-ctl --print-topology --device /dev/media1

$ media-ctl --print-topology --device /dev/media1
Media controller API version 5.15.30

Media device information
------------------------
driver          bcm2835-isp
model           bcm2835-isp
serial          
bus info        platform:bcm2835-isp
hw revision     0x0
driver version  5.15.30

Device topology
- entity 1: bcm2835_isp0 (4 pads, 4 links)
            type Node subtype Unknown flags 0
	pad0: Sink
		<- "bcm2835-isp0-output0":0 [ENABLED,IMMUTABLE]
	pad1: Source
		-> "bcm2835-isp0-capture1":0 [ENABLED,IMMUTABLE]
	pad2: Source
		-> "bcm2835-isp0-capture2":0 [ENABLED,IMMUTABLE]
	pad3: Source
		-> "bcm2835-isp0-capture3":0 [ENABLED,IMMUTABLE]

- entity 6: bcm2835-isp0-output0 (1 pad, 1 link)
            type Node subtype V4L flags 0
            device node name /dev/video20
	pad0: Source
		-> "bcm2835_isp0":0 [ENABLED,IMMUTABLE]

- entity 12: bcm2835-isp0-capture1 (1 pad, 1 link)
             type Node subtype V4L flags 0
             device node name /dev/video21
	pad0: Sink
		<- "bcm2835_isp0":1 [ENABLED,IMMUTABLE]

- entity 18: bcm2835-isp0-capture2 (1 pad, 1 link)
             type Node subtype V4L flags 0
             device node name /dev/video22
	pad0: Sink
		<- "bcm2835_isp0":2 [ENABLED,IMMUTABLE]

- entity 24: bcm2835-isp0-capture3 (1 pad, 1 link)
             type Node subtype V4L flags 0
             device node name /dev/video23
	pad0: Sink
		<- "bcm2835_isp0":3 [ENABLED,IMMUTABLE]

media-ctl --print-topology --device /dev/media2

$ media-ctl --print-topology --device /dev/media2
Media controller API version 5.15.30

Media device information
------------------------
driver          bcm2835-codec
model           bcm2835-codec
serial          0000
bus info        platform:bcm2835-codec
hw revision     0x1
driver version  5.15.30

Device topology
- entity 1: bcm2835-codec-decode-source (1 pad, 1 link)
            type Node subtype V4L flags 0
            device node name /dev/video10
	pad0: Source
		-> "bcm2835-codec-decode-proc":0 [ENABLED,IMMUTABLE]

- entity 3: bcm2835-codec-decode-proc (2 pads, 2 links)
            type Node subtype Unknown flags 0
	pad0: Sink
		<- "bcm2835-codec-decode-source":0 [ENABLED,IMMUTABLE]
	pad1: Source
		-> "bcm2835-codec-decode-sink":0 [ENABLED,IMMUTABLE]

- entity 6: bcm2835-codec-decode-sink (1 pad, 1 link)
            type Node subtype V4L flags 0
            device node name /dev/video10
	pad0: Sink
		<- "bcm2835-codec-decode-proc":1 [ENABLED,IMMUTABLE]

- entity 15: bcm2835-codec-encode-source (1 pad, 1 link)
             type Node subtype V4L flags 0
             device node name /dev/video11
	pad0: Source
		-> "bcm2835-codec-encode-proc":0 [ENABLED,IMMUTABLE]

- entity 17: bcm2835-codec-encode-proc (2 pads, 2 links)
             type Node subtype Unknown flags 0
	pad0: Sink
		<- "bcm2835-codec-encode-source":0 [ENABLED,IMMUTABLE]
	pad1: Source
		-> "bcm2835-codec-encode-sink":0 [ENABLED,IMMUTABLE]

- entity 20: bcm2835-codec-encode-sink (1 pad, 1 link)
             type Node subtype V4L flags 0
             device node name /dev/video11
	pad0: Sink
		<- "bcm2835-codec-encode-proc":1 [ENABLED,IMMUTABLE]

- entity 29: bcm2835-codec-isp-source (1 pad, 1 link)
             type Node subtype V4L flags 0
             device node name /dev/video12
	pad0: Source
		-> "bcm2835-codec-isp-proc":0 [ENABLED,IMMUTABLE]

- entity 31: bcm2835-codec-isp-proc (2 pads, 2 links)
             type Node subtype Unknown flags 0
	pad0: Sink
		<- "bcm2835-codec-isp-source":0 [ENABLED,IMMUTABLE]
	pad1: Source
		-> "bcm2835-codec-isp-sink":0 [ENABLED,IMMUTABLE]

- entity 34: bcm2835-codec-isp-sink (1 pad, 1 link)
             type Node subtype V4L flags 0
             device node name /dev/video12
	pad0: Sink
		<- "bcm2835-codec-isp-proc":1 [ENABLED,IMMUTABLE]

- entity 43: bcm2835-codec-image_fx-source (1 pad, 1 link)
             type Node subtype V4L flags 0
             device node name /dev/video18
	pad0: Source
		-> "bcm2835-codec-image_fx-proc":0 [ENABLED,IMMUTABLE]

- entity 45: bcm2835-codec-image_fx-proc (2 pads, 2 links)
             type Node subtype Unknown flags 0
	pad0: Sink
		<- "bcm2835-codec-image_fx-source":0 [ENABLED,IMMUTABLE]
	pad1: Source
		-> "bcm2835-codec-image_fx-sink":0 [ENABLED,IMMUTABLE]

- entity 48: bcm2835-codec-image_fx-sink (1 pad, 1 link)
             type Node subtype V4L flags 0
             device node name /dev/video18
	pad0: Sink
		<- "bcm2835-codec-image_fx-proc":1 [ENABLED,IMMUTABLE]

- entity 57: bcm2835-codec-encode_image-sour (1 pad, 1 link)
             type Node subtype V4L flags 0
             device node name /dev/video31
	pad0: Source
		-> "bcm2835-codec-encode_image-proc":0 [ENABLED,IMMUTABLE]

- entity 59: bcm2835-codec-encode_image-proc (2 pads, 2 links)
             type Node subtype Unknown flags 0
	pad0: Sink
		<- "bcm2835-codec-encode_image-sour":0 [ENABLED,IMMUTABLE]
	pad1: Source
		-> "bcm2835-codec-encode_image-sink":0 [ENABLED,IMMUTABLE]

- entity 62: bcm2835-codec-encode_image-sink (1 pad, 1 link)
             type Node subtype V4L flags 0
             device node name /dev/video31
	pad0: Sink
		<- "bcm2835-codec-encode_image-proc":1 [ENABLED,IMMUTABLE]

The topology associated with the camera sensor module.

media-ctl --print-topology --device /dev/media3

$ media-ctl --print-topology --device /dev/media3
Media controller API version 5.15.30

Media device information
------------------------
driver          unicam
model           unicam
serial          
bus info        platform:3f801000.csi
hw revision     0x0
driver version  5.15.30

Device topology
- entity 1: imx219 10-0010 (2 pads, 2 links)
            type V4L2 subdev subtype Sensor flags 0
            device node name /dev/v4l-subdev0
	pad0: Source
		[fmt:SRGGB10_1X10/3280x2464 field:none colorspace:raw xfer:none ycbcr:601 quantization:full-range
		 crop.bounds:(8,8)/3280x2464
		 crop:(8,8)/3280x2464]
		-> "unicam-image":0 [ENABLED,IMMUTABLE]
	pad1: Source
		[fmt:unknown/16384x1 field:none
		 crop.bounds:(8,8)/3280x2464
		 crop:(8,8)/3280x2464]
		-> "unicam-embedded":0 [ENABLED,IMMUTABLE]

- entity 4: unicam-image (1 pad, 1 link)
            type Node subtype V4L flags 1
            device node name /dev/video0
	pad0: Sink
		<- "imx219 10-0010":0 [ENABLED,IMMUTABLE]

- entity 10: unicam-embedded (1 pad, 1 link)
             type Node subtype V4L flags 0
             device node name /dev/video1
	pad0: Sink
		<- "imx219 10-0010":1 [ENABLED,IMMUTABLE]

If just run ./ustreamer in this setup, it reports:

-- ERROR [99.029    stream] -- Can't start capturing: Invalid argument

In the kernel log (journal -t kernel --since '3 min ago'), the following lines are logged:

Mar 23 18:30:59 raspberry kernel: unicam 3f801000.csi: Wrong width or height 640x480 (remote pad set to 3280x2464)
Mar 23 18:30:59 raspberry kernel: unicam 3f801000.csi: Failed to start media pipeline: -22

The first problem can be corrected by updating the size on the V4L2 subdev.

# Display details for the sensor V4L2 subdev
$ media-ctl --print-topology --device /dev/media3
...
Device topology
- entity 1: imx219 10-0010 (2 pads, 2 links)
            type V4L2 subdev subtype Sensor flags 0
            device node name /dev/v4l-subdev0
	pad0: Source
		[fmt:SRGGB10_1X10/3280x2464 field:none colorspace:raw xfer:none ycbcr:601 quantization:full-range
		 crop.bounds:(8,8)/3280x2464
		 crop:(8,8)/3280x2464]
		-> "unicam-image":0 [ENABLED,IMMUTABLE]
	pad1: Source
		[fmt:unknown/16384x1 field:none
		 crop.bounds:(8,8)/3280x2464
		 crop:(8,8)/3280x2464]
		-> "unicam-embedded":0 [ENABLED,IMMUTABLE]
...

# See `media-ctl -h` for the syntax (pretty much: `'<entity>:<pad> <v4l2 properties>'`)
$ media-ctl --device 3 --set-v4l2 '"imx219 10-0010":0 [fmt:SRGGB10_1X10/640x480]'

# Verify
$ media-ctl -p -d 3
...
Device topology
- entity 1: imx219 10-0010 (2 pads, 2 links)
            type V4L2 subdev subtype Sensor flags 0
            device node name /dev/v4l-subdev0
	pad0: Source
		[fmt:SRGGB10_1X10/640x480 field:none colorspace:raw xfer:none ycbcr:601 quantization:full-range
		 crop.bounds:(8,8)/3280x2464
		 crop:(1008,760)/1280x960]
		-> "unicam-image":0 [ENABLED,IMMUTABLE]
	pad1: Source
		[fmt:unknown/16384x1 field:none
		 crop.bounds:(8,8)/3280x2464
		 crop:(1008,760)/1280x960]
		-> "unicam-embedded":0 [ENABLED,IMMUTABLE]
...

Unfortunately, this is not enough and ustreamer still fails with Can't start capturing: Invalid argument and the kernel still logs Failed to start media pipeline: -22 (Invalid argument). It's not very helpful but indicates that one or more things aren't setup properly.

I've tried to disable use of Media Controller API by updating /boot/config.txt with:

start_x=1
camera_auto_detect=0
dtoverlay=imx219,media-controller=0

After a reboot, I see:

$ v4l2-ctl --list-formats --device /dev/video0
ioctl: VIDIOC_ENUM_FMT
	Type: Video Capture

	[0]: 'pRAA' (10-bit Bayer RGRG/GBGB Packed)
	[1]: 'RG10' (10-bit Bayer RGRG/GBGB)
	[2]: 'RGGB' (8-bit Bayer RGRG/GBGB)

None of these pixel formats are among the ones supported by ustreamer, so it fails after proposing YUVY but getting pRAA from the driver.

I attempted to circumvent this problem by adding definitions for these formats to the source code but all I ended up with after running ./ustreamer -s :: --log-level 3 -c m2m-video (or -c M2M-MJPEG) is a green/black weave pattern on the video stream.

[ayufan]

It is failing the same on imx519 (Arducam 16MP)

-- DEBUG [1226.812    stream] -- Starting device capturing ...
-- ERROR [1226.812    stream] -- Can't start capturing: Invalid argument
-- DEBUG [1226.812    stream] -- Releasing device buffers ...

[pid   879] ioctl(8, VIDIOC_STREAMON, [V4L2_BUF_TYPE_VIDEO_CAPTURE]) = -1 EINVAL (Invalid argument)

[bertvdijk]

Hello,

I just tried with the latest code from the master branch (ustreamer 5.3)

• fresh install of Raspbian Bullseye (2022-01-28)
• upgraded with apt update && apt dist-upgrade -y && reboot (kernel 5.15.30)
• brought up to kernel 5.15.32 (as mentioned in the README) with rpi-update
• Raspberry Pi Camera v2 (IMX219) connected via CSI (camera_auto_detect=1 in /boot/config.txt)

Using the examples in the README, I tried the examples:

# Prevent kernel message "Wrong width or height 640x480 (remote pad set to 3280x2464)"
media-ctl --device 3 --set-v4l2 '"imx219 10-0010":1 [fmt:SRGGB10_1X10/640x480]'

# From the "Usage" section but with a lower resolution
./ustreamer --host :: -m jpeg --device-timeout=5 --buffers=3 -r 640x480

# From the "Raspberry Pi Camera Example" section
./ustreamer  --format=uyvy --encoder=m2m-image --workers=3 --persistent --dv-timings  --drop-same-frames=30

Both of these commands failed with:

-- ERROR [1616.438    stream] -- Can't start capturing: Invalid argument

With dmesg -T I see:

unicam 3f801000.csi: Failed to start media pipeline: -22

@mdevaev, do you have any input on what's necessary to be able to use ustreamer for streaming video from MIPI CSI-2 sensors under Raspbian Bullseye with the modern camera stack (unicam)?

(comments in this thread suggested it's possible to workaround the problem by enabling the legacy camera stack and I did not try this. Personally I'd rather spend time on getting things to work with the modern camera stack to future proof things and ease setups under a default install)

Thanks for your time and keep up the good work with PiKVM!

[ayufan]

OK. I have been playing with this to use a new bcm2835-isp. In general libcamera does convert pRGAA into YUYV using isp and DMA transfer. It is not hard but the ustreamer is not very flexible to extend and inject. I hacked it somehow adding m2m isp. It kind of works, and breaks in number of places, but should support imx219 and imx519.

You can see my branch in here: https://github.com/ayufan-research/ustreamer. Compile and then try to run it. I have troubles with higher resolutions. It seems to work on 5.10 and 5.15. On 5.15 it does work with M2M-JPEG codec to further accelerate transfer.

This is not efficient, nor finished, and can only be seen how it might be done.

ustreamer --host :: -r 1920x1080 -m pRAA -w 1 -b 1 --debug --no-log-colors -d /dev/video0

[noahwilliamsson]

Nice work!

I just tested this with a Raspberry Pi Zero 2W and later a Zero W v1.1, both running kernel 5.15.32.

media-ctl --device 3 --set-v4l2 '"imx219 10-0010":0 [fmt:SRGGB10_1X10/640x480]'
./ustreamer --host :: -r 640x480 -m pRAA -w 1 -b 1 --debug --no-log-colors -d /dev/video0

This might be completely incidental.. but the Zero 2W appears to have fried within about 15 seconds after running these commands, before I managed to open the browser to test out the stream. I lost SSH access and smelled a very hot PCB. After powering it off, I let it cool for 10 minutes and then tried to power it up again but it immediately becomes very hot and doesn't light any LEDs. Not sure what happened here -- it had been running idle for a few hours already on a clean desk, and I only accessed it over SSH.

Anyway, I moved the SD card and the camera over to a Raspberry Pi Zero W v1.1 and re-ran the commands. If I move my hand in front of the camera, I see a dark grey silhouette of it on a darker/black background on the streaming page. Is that expected? (libcamera-still -o still.jpg produces an OK picture though.)

I also note that if I run libcamera-still -o still.jpg, then a subsequent ./ustreamer ... fails with Can't start capturing: Invalid argument and the kernel reports unicam 20801000.csi: Failed to start media pipeline: -22. It seems some settings are left in a different (unexpected) state than after a fresh boot when you've run libcamera-still.

BTW, what Raspberry Pi hardware did you test with?

[AntiHeld889]

same issue

[mdevaev]

Sorry for the long answer.

I'm using ustreamer with some usb webcams and csi-hdmi bridge so I don't know how to make it work with the regular csi camera, I don't have any. I'll try to get some, but I can't promise that it will be happen soon.

[fphammerle]

Thanks a lot for looking into this issue, @mdevaev !

[ayufan]

This might be completely incidental.. but the Zero 2W appears to have fried within about 15 seconds after running these commands, before I managed to open the browser to test out the stream. I lost SSH access and smelled a very hot PCB. After powering it off, I let it cool for 10 minutes and then tried to power it up again but it immediately becomes very hot and doesn't light any LEDs. Not sure what happened here -- it had been running idle for a few hours already on a clean desk, and I only accessed it over SSH.

I did not had such problems.

Anyway, I moved the SD card and the camera over to a Raspberry Pi Zero W v1.1 and re-ran the commands. If I move my hand in front of the camera, I see a dark grey silhouette of it on a darker/black background on the streaming page. Is that expected? (libcamera-still -o still.jpg produces an OK picture though.)

This is a problem that bcm2835-isp is not configured. The libcamera does read sensor parameters from /usr/share/libcamera/ipa/raspberrypi/imx519.json (or other depending on used sensor) and configures the /dev/video13 (you can see available with v4l2-ctl -d /dev/video13 -L or ISP params only: v4l2-ctl -d /dev/video13 -C red_balance -C colour_correction_matrix -C black_level -C green_equalisation -C gamma -C denoise -C sharpen -C defective_pixel_correction -C colour_denoise).
Once you configure this the above https://github.com/ayufan-research/ustreamer to work.

[ayufan]

In general all those DLSR type sensors (from Sony, and others) they return only 8-bit or 10-bit Bayer packed format (once you hook into this stream, the latency is simply awesome) that resembles exactly how the CMOS sensor for cameras are built. You now once have this data need to manually calculate exposure time, analogue and digital gains. This can be done by ISP. Raspberry PI does have bcm2835-isp (present in system under /dev/video12-15) which can take this raw data, apply all corrections (color, gamma, etc.) and output YUYV/YU12/NV12 which can be then used as a source for other encoders (JPEG or H264). The ISP is able to output two resolutions at the same time, so effectively able to perform a single capture, and have high-res and low-res YUYV stream being handled concurrently.

Why I'm writing about this? The ISP is software controlled. Trying to re-implement libcamera which does manage ISP is tendious task given all aspects that needs to be covered. Previously we would have UVC, or other type of sensor with integrated ISP returning YUYV, where we would configure brightness, contrast, focus, etc. It does mean that without libcamera it is fairly hard to implement automated brightness control, or auto focus. The https://github.com/ayufan-research/ustreamer does use ISP directly, and this is why you saw the black image mostly, since ISP was not configured. It can be one time off configured with usage of libcamera-still (with any type of parameters you might want).

About ustreamer. This can be supported today with using a trick:
a. comment out the VIDIOC_S_INPUT:

ustreamer/src/ustreamer/device.c

Line 421 in 7ceb8a3

if (xioctl(RUN(fd), VIDIOC_S_INPUT, &input) < 0) {

b. use v4l2-ctl to manually feed ISP with a stream of data v4l2-ctl --device /dev/video0 --out-device /dev/video13 --stream-mmap --stream-out-dmabuf (it will copy over RG10 data into ISP)
c. run ustreamer --device /dev/video14 (it will expose YUYV, or whatever you configure it to)
d. this will work, but have pretty terrible latency

Exactly above behavior is implemented in https://github.com/ayufan-research/ustreamer. The camera data, is feed into ISP, and this is feed into JPEG. The implementation in https://github.com/ayufan-research/ustreamer is pretty clumsy, since ustreamer architecture does not allow to easily create a multi-stage pipelines for data processing, or hooking libcamera. For example the https://github.com/ayufan-research/ustreamer even though it hooks ISP, it does not offer YUYV stream for H264 support and there's no easy way aroud this limitation except maybe rewriting the device be libcamera centric (which internally uses ISP).

The integrated kernel data pipelining (as done with v4l2-ctl ...) might land at some point in linux kernel: https://lore.kernel.org/linux-arm-kernel/Yd8fJd2SASGkhOjm@pendragon.ideasonboard.com/t/. This change uses media-controller routing capabilities, to hook into avaiable v4l2 devices and to route data streams between them to get sink with a desired formats. Unsure if this got merged, and if yes when it filters to Raspberry PI OS. However, this as well will have the same limitation: libcamera needs to control ISP to provide auto brightness and auto focus control.

As an exercise I tried to do it, and thus (sorry, for slightly hijacking this thread with own project) created in last few days the https://github.com/ayufan-research/camera-streamer. It did felt slightly easier after trying the first iteration in ustreamer codebase (due to lack of flexible multi-stage pipelines) to start from scratch with a project being v4l2 hardware centric. I wrote the v4l2 only pipeline, but at some point figured out above and saw that not using libcamera is in general not wise idea. Using libcamera comes with latency impact (but you get good control of exposure and focus), as I measured in my project compared to direct ISP-mode. Using direct ISP-mode I get a way better performance and latency than anything that I tested so far (using my branch for https://github.com/ayufan-research/ustreamer). Got pretty far, to the point that I started using the linked project on my Voron 0 with Arducam 16MP for MJPEG and H264 daily.

It appears that might I took a wrong approach with https://github.com/ayufan-research/ustreamer to implement ISP step instead of rewriting device to be libcamera centric. Usage of libcamera is not really that hard and it offers the same access to mmaped and dma-enabled buffer: https://github.com/ayufan-research/camera-streamer/tree/master/device/libcamera.

Ref.:

• https://github.com/ayufan-research/ustreamer
• https://github.com/ayufan-research/camera-streamer
• https://lore.kernel.org/linux-arm-kernel/Yd8fJd2SASGkhOjm@pendragon.ideasonboard.com/t/