[PATCH v4 0/5] Raspberry Pi HEVC decoder driver

From: Dave Stevenson
Date: Tue Jul 01 2025 - 12:02:10 EST

Next message: Dave Stevenson: "[PATCH v4 1/5] docs: uapi: media: Document Raspberry Pi NV12 column format"
Previous message: Jeongjun Park: "Re: [PATCH v6] mm/vmalloc: fix data race in show_numa_info()"
Next in thread: Dave Stevenson: "[PATCH v4 1/5] docs: uapi: media: Document Raspberry Pi NV12 column format"
Messages sorted by: [ date ] [ thread ] [ subject ] [ author ]

Hi All

This has been in the pipeline for a while, but I've finally cleaned
up our HEVC decoder driver to be in a shape to at least get a first
review.
John Cox has done almost all of the work under contract to Raspberry
Pi, and I'm largely just doing the process of patch curation and
sending.

This series has been updated to use the manual request completion
framework from Hans and Nicolas as a dependency
https://lore.kernel.org/linux-media/20250604-sebastianfricke-vcodec_manual_request_completion_with_state_machine-v3-0-603db4749d90@xxxxxxxxxxxxx/

Thanks
Dave

v4l2-compliance output:
v4l-utils $ ./build/utils/v4l2-compliance/v4l2-compliance
v4l2-compliance 1.31.0-5374, 64 bits, 64-bit time_t
v4l2-compliance SHA: b8c26bb16d75 2025-06-10 21:30:38

Compliance test for rpi-hevc-dec device /dev/video0:

Driver Info:
Driver name : rpi-hevc-dec
Card type : rpi-hevc-dec
Bus info : platform:rpi-hevc-dec
Driver version : 6.16.0
Capabilities : 0x84204000
Video Memory-to-Memory Multiplanar
Streaming
Extended Pix Format
Device Capabilities
Device Caps : 0x04204000
Video Memory-to-Memory Multiplanar
Streaming
Extended Pix Format
Detected Stateless Decoder
Media Driver Info:
Driver name : rpi-hevc-dec
Model : rpi-hevc-dec
Serial :
Bus info : platform:rpi-hevc-dec
Media version : 6.16.0
Hardware revision: 0x00000000 (0)
Driver version : 6.16.0
Interface Info:
ID : 0x0300000c
Type : V4L Video
Entity Info:
ID : 0x00000001 (1)
Name : rpi-hevc-dec-source
Function : V4L2 I/O
Pad 0x01000002 : 0: Source
Link 0x02000008: to remote pad 0x1000004 of entity 'rpi-hevc-dec-proc' (Video Decoder): Data, Enabled, Immutable

Required ioctls:
test MC information (see 'Media Driver Info' above): OK
test VIDIOC_QUERYCAP: OK
test invalid ioctls: OK

Allow for multiple opens:
test second /dev/video0 open: OK
test VIDIOC_QUERYCAP: OK
test VIDIOC_G/S_PRIORITY: OK
test for unlimited opens: OK

Debug ioctls:
test VIDIOC_DBG_G/S_REGISTER: OK (Not Supported)
test VIDIOC_LOG_STATUS: OK (Not Supported)

Input ioctls:
test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported)
test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported)
test VIDIOC_ENUMAUDIO: OK (Not Supported)
test VIDIOC_G/S/ENUMINPUT: OK (Not Supported)
test VIDIOC_G/S_AUDIO: OK (Not Supported)
Inputs: 0 Audio Inputs: 0 Tuners: 0

Output ioctls:
test VIDIOC_G/S_MODULATOR: OK (Not Supported)
test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
test VIDIOC_ENUMAUDOUT: OK (Not Supported)
test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported)
test VIDIOC_G/S_AUDOUT: OK (Not Supported)
Outputs: 0 Audio Outputs: 0 Modulators: 0

Input/Output configuration ioctls:
test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported)
test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported)
test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported)
test VIDIOC_G/S_EDID: OK (Not Supported)

Control ioctls:
test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK
test VIDIOC_QUERYCTRL: OK
test VIDIOC_G/S_CTRL: OK
fail: v4l2-test-controls.cpp(939): try_ext_ctrls returned an error (22)
test VIDIOC_G/S/TRY_EXT_CTRLS: FAIL
test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: OK
test VIDIOC_G/S_JPEGCOMP: OK (Not Supported)
Standard Controls: 3 Private Controls: 0
Standard Compound Controls: 5 Private Compound Controls: 0

Format ioctls:
test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK
test VIDIOC_G/S_PARM: OK (Not Supported)
test VIDIOC_G_FBUF: OK (Not Supported)
test VIDIOC_G_FMT: OK
test VIDIOC_TRY_FMT: OK
test VIDIOC_S_FMT: OK
test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported)
test Cropping: OK (Not Supported)
test Composing: OK (Not Supported)
test Scaling: OK (Not Supported)

Codec ioctls:
test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported)
test VIDIOC_G_ENC_INDEX: OK (Not Supported)
test VIDIOC_(TRY_)DECODER_CMD: OK

Buffer ioctls:
test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: OK
test CREATE_BUFS maximum buffers: OK
test VIDIOC_REMOVE_BUFS: OK
test VIDIOC_EXPBUF: OK
test Requests: OK
test blocking wait: OK

Total for rpi-hevc-dec device /dev/video0: 49, Succeeded: 48, Failed: 1, Warnings: 0

Testing is mainly with a downstream patchset to FFmpeg. I'm told FFmpeg
currently has no stateless decode support, but we will be reviewing
works that have been in progress and our downstream patches to see
whether that can be pushed onwards.
Downstream tree is at
https://github.com/jc-kynesim/rpi-ffmpeg/tree/dev/5.1.6/sandm_1

https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/9247
adds support for the new formats to GStreamer, and allows Fluster to run.
Using the JCT-VC-HEVC_V1 test suite we get 142 passes out of 147 tests.
PICSIZE_[ABCD]_Bossen_1 all fail due to the image sizes being greater than
the 4096x4096 limit of this driver.
TSUNEQBD_A_MAIN10_Technicolor_2 fails as the hardware doesn't support
having a different bit depth for luma and chroma.

Signed-off-by: Dave Stevenson <dave.stevenson@xxxxxxxxxxxxxxx>
---
Changes in v4:
- dt-bindings: Drop to just SoC compatibles and updated description (Krzysztof).
- dts: Corrected address in node name.
- NV12MT_COL128 format description added vertical alignment to a
multiple of 8 (Nicolas)
- driver: Merged in driver updates based on Nicolas' review.
- driver: Added debug module parameter to avoid log spam due to some bitstreams.
- cover-letter: Added Fluster results.
- rebased and based on the updated version of
https://lore.kernel.org/linux-media/20250604-sebastianfricke-vcodec_manual_request_completion_with_state_machine-v3-0-603db4749d90@xxxxxxxxxxxxx/
- update v4l2-compliance output
- Link to v3: https://lore.kernel.org/r/20250423-media-rpi-hevc-dec-v3-0-8fd3fad1d6fb@xxxxxxxxxxxxxxx

Changes in v3:
- Updated the dtbinding with SoC specific compatible strings (Rob).
- Reordered hevc_dec and v3d in bcm2711.dtsi to keep them in ascending
register order (Stefan).
- Reordered hevc_dec in bcm2711-rpi.dtsi to keep them in alphabetical
order (Stefan).
- Tested on top of Nicolas' revised version of Hans' patch set for
manual request completion.
https://lore.kernel.org/all/20250410-sebastianfricke-vcodec_manual_request_completion_with_state_machine-v2-0-5b99ec0450e6@xxxxxxxxxxxxx/
- Link to v2: https://lore.kernel.org/r/20250206-media-rpi-hevc-dec-v2-0-69353c8805b2@xxxxxxxxxxxxxxx

Changes in v2:
- Rebased to use Hans' manual request completion scheme.
https://lore.kernel.org/linux-media/cover.1724928939.git.hverkuil-cisco@xxxxxxxxx/
- Require all slices for a frame to be submitted in one request.
- Added the missing header file.
- Used the full macro name for pix format docs (Sakari)
- Dropped unneeded |- from dtbinding (Rob)
- Made reg and reg-names match in order (Rob)
- Removed clock-names from dtbinding (Rob)
- Driver changed to not request the clock by name
- Dropped clock-names from DTS file
- Minor fixes for compliance failures
fail: v4l2-test-formats.cpp(958): fmt_cap.g_colorspace() != col
fail: v4l2-test-buffers.cpp(901): q.create_bufs(node, 1, &fmt) != EINVAL
- v4l2-compliance output added to cover letter (Nicholas)
I believe the "fail: v4l2-test-controls.cpp(939): try_ext_ctrls
returned an error (22)" is expected as it is validating the SPS.
Hantro and Cedrus certainly both appear to return errors in the same place
- Link to v1: https://lore.kernel.org/r/20241220-media-rpi-hevc-dec-v1-0-0ebcc04ed42e@xxxxxxxxxxxxxxx

---
Dave Stevenson (4):
docs: uapi: media: Document Raspberry Pi NV12 column format
media: ioctl: Add pixel formats NV12MT_COL128 and NV12MT_10_COL128
dt-bindings: media: Add the Raspberry Pi HEVC decoder
arm: dts: bcm2711-rpi: Add HEVC decoder node

John Cox (1):
media: platform: Add Raspberry Pi HEVC decoder driver

.../bindings/media/raspberrypi,hevc-dec.yaml | 72 +
.../media/v4l/ext-ctrls-codec-stateless.rst | 6 +-
.../userspace-api/media/v4l/pixfmt-yuv-planar.rst | 42 +
MAINTAINERS | 10 +
arch/arm/boot/dts/broadcom/bcm2711-rpi.dtsi | 4 +
arch/arm/boot/dts/broadcom/bcm2711.dtsi | 9 +
drivers/media/platform/raspberrypi/Kconfig | 1 +
drivers/media/platform/raspberrypi/Makefile | 1 +
.../media/platform/raspberrypi/hevc_dec/Kconfig | 17 +
.../media/platform/raspberrypi/hevc_dec/Makefile | 5 +
.../media/platform/raspberrypi/hevc_dec/hevc_d.c | 369 +++
.../media/platform/raspberrypi/hevc_dec/hevc_d.h | 192 ++
.../platform/raspberrypi/hevc_dec/hevc_d_h265.c | 2554 ++++++++++++++++++++
.../platform/raspberrypi/hevc_dec/hevc_d_h265.h | 23 +
.../platform/raspberrypi/hevc_dec/hevc_d_hw.c | 376 +++
.../platform/raspberrypi/hevc_dec/hevc_d_hw.h | 314 +++
.../platform/raspberrypi/hevc_dec/hevc_d_video.c | 674 ++++++
.../platform/raspberrypi/hevc_dec/hevc_d_video.h | 38 +
drivers/media/v4l2-core/v4l2-ioctl.c | 2 +
include/uapi/linux/videodev2.h | 4 +
20 files changed, 4711 insertions(+), 2 deletions(-)
---
base-commit: 6669fd53a3c2be6dc632a87d9105016139088144
change-id: 20241212-media-rpi-hevc-dec-3b5be739f3bd

Best regards,
--
Dave Stevenson <dave.stevenson@xxxxxxxxxxxxxxx>

Next message: Dave Stevenson: "[PATCH v4 1/5] docs: uapi: media: Document Raspberry Pi NV12 column format"
Previous message: Jeongjun Park: "Re: [PATCH v6] mm/vmalloc: fix data race in show_numa_info()"
Next in thread: Dave Stevenson: "[PATCH v4 1/5] docs: uapi: media: Document Raspberry Pi NV12 column format"
Messages sorted by: [ date ] [ thread ] [ subject ] [ author ]