Re: [RFC PATCH] uvcvideo: Add mapping for HEVC payloads

From: Dmitry Buzdyk
Date: Wed Jul 29 2020 - 02:26:10 EST

Hi Laurent,

On Tue, Jul 28, 2020 at 03:17:21AM +0300, Laurent Pinchart wrote:
> Hi Dmitry,
>
> Sorry for the late reply.
>
> Mauro, there's a question for your below.
>
> On Tue, Jul 28, 2020 at 09:25:46AM +1000, Dmitry Buzdyk wrote:
> > Hi Laurent,
> >
> > Had you a chance to review USB descriptors from the device provided
> > below? Is any additional information needed?
> >
> > On Wed, Jul 15, 2020 at 06:00:10PM +1000, Dmitry Buzdyk wrote:
> > > On Tue, Jun 09, 2020 at 02:57:36PM +1000, Dmitry Buzdyk wrote:
> > > Hi Laurent,
> > >
> > > Please see updated information below
> > >
> > >> On Sun, Jun 07, 2020 at 04:07:19AM +0300, Laurent Pinchart wrote:
> > >>> Hi Dmitry,
> > >>>
> > >>> Thank you for the patch.
> > >>>
> > >>> On Fri, May 29, 2020 at 11:05:47AM +1000, Dmitry Buzdyk wrote:
> > >>>> Add HEVC GUID and assotiate with HEVC pixel format so that frame
> > >>>> based format descriptors recognized by the UVC video driver.
> > >>>
> > >>> The patch itself looks OK to me, but could you share a bit more
> > >>> information about which device(s) implement this ? Are they UVC 1.1
> > >>> devices ? Could you share their full USB descriptors (retrieved with
> > >>> 'lsusb -v', running as root if possible) ?
> > >>
> > >> This is a UVC1.1 camera device based on Ambarella H22 SOC.
>
> That's interesting. It would be nice to have upstream support for the
> Ambarella SoCs in the kernel :-)
>
> > >> Please note that device is still in development and yet to get actual
> > >> VID and PID.
> > >
> > > Device got its VID:PID from USB-IF:
> > >
> > > Bus 001 Device 009: ID 3301:1000
> > > Device Descriptor:
> > > bLength 18
> > > bDescriptorType 1
> > > bcdUSB 2.00
> > > bDeviceClass 239 Miscellaneous Device
> > > bDeviceSubClass 2
> > > bDeviceProtocol 1 Interface Association
> > > bMaxPacketSize0 64
> > > idVendor 0x3301
> > > idProduct 0x1000
> > > bcdDevice 0.10
> > > iManufacturer 1 Rhonda
> > > iProduct 2 Rhonda Cam
> > > iSerial 3 FMABCLE15000007
> > > bNumConfigurations 1
>
> Thank you for the descriptors.
>
> [snip]
>
> > > VideoControl Interface Descriptor:
> > > bLength 9
> > > bDescriptorType 36
> > > bDescriptorSubtype 3 (OUTPUT_TERMINAL)
> > > bTerminalID 16
> > > wTerminalType 0x0101 USB Streaming
> > > bAssocTerminal 0
> > > bSourceID 10
> > > iTerminal 0
> > > VideoControl Interface Descriptor:
> > > bLength 9
> > > bDescriptorType 36
> > > bDescriptorSubtype 3 (OUTPUT_TERMINAL)
> > > bTerminalID 17
> > > wTerminalType 0x0101 USB Streaming
> > > bAssocTerminal 0
> > > bSourceID 10
> > > iTerminal 0
>
> Two output terminals ? Does that mean the device can provide two streams
> from the same source ?
Correct. This device encode and stream two indpendent H264 or HEVC video streams.
Picture for both streams comes from single image sensor, with own ROI applied for each stream.

>
> [snip]
>
>
> > > Endpoint Descriptor:
> > > bLength 7
> > > bDescriptorType 5
> > > bEndpointAddress 0x83 EP 3 IN
> > > bmAttributes 2
> > > Transfer Type Bulk
> > > Synch Type None
> > > Usage Type Data
> > > wMaxPacketSize 0x0200 1x 512 bytes
> > > bInterval 0
>
> This is interesting too, does it provide enough bandwidth for 3000x4000
> @10fps MJPEG ?
This video mode has relatively high compression ratio thus total bitrate
for this mode does not exceed 100Mbps.

>
> [snip]
>
> > >>> Is there anything else needed to get HEVC capture working, such as
> > >>> extension unit controls, or is this patch enough ? What userspace
> > >>> software do you use to capture and decode HEVC (or capture it to disk) ?
> > >>
> > >> No other changes to Linux nor extra actions needed to start HEVC capture.
> > >> We use patched version of FFmpeg to capture, decode and display HEVC
> > >> stream from camera device. That simple patch also going to be sent to
> > >> FFmpeg upstream.
> > >
> > > Patch for FFmpeg sent to https://patchwork.ffmpeg.org/project/ffmpeg/list/?series=1760
> > >
> > >>>> Signed-off-by: Dmitry Buzdyk <dima.buzdyk@xxxxxxxxx>
>
> Reviewed-by: Laurent Pinchart <laurent.pinchart@xxxxxxxxxxxxxxxx>
>
> and taken in my tree. I'm afraid we're a bit too close to the v5.9 merge
> window for me to send a pull request now, unless Mauro would be fine
> with that. Otherwise I'll include it in the pull request for the next
> release.
>
> > >>>> ---
> > >>>> drivers/media/usb/uvc/uvc_driver.c | 5 +++++
> > >>>> drivers/media/usb/uvc/uvcvideo.h | 4 ++++
> > >>>> 2 files changed, 9 insertions(+)
> > >>>>
> > >>>> diff --git a/drivers/media/usb/uvc/uvc_driver.c b/drivers/media/usb/uvc/uvc_driver.c
> > >>>> index 431d86e1c94b..825ee3601661 100644
> > >>>> --- a/drivers/media/usb/uvc/uvc_driver.c
> > >>>> +++ b/drivers/media/usb/uvc/uvc_driver.c
> > >>>> @@ -214,6 +214,11 @@ static struct uvc_format_desc uvc_fmts[] = {
> > >>>> .guid = UVC_GUID_FORMAT_CNF4,
> > >>>> .fcc = V4L2_PIX_FMT_CNF4,
> > >>>> },
> > >>>> + {
> > >>>> + .name = "HEVC",
> > >>>> + .guid = UVC_GUID_FORMAT_HEVC,
> > >>>> + .fcc = V4L2_PIX_FMT_HEVC,
> > >>>> + },
> > >>>> };
> > >>>>
> > >>>> /* ------------------------------------------------------------------------
> > >>>> diff --git a/drivers/media/usb/uvc/uvcvideo.h b/drivers/media/usb/uvc/uvcvideo.h
> > >>>> index 6ab972c643e3..c7f043121b41 100644
> > >>>> --- a/drivers/media/usb/uvc/uvcvideo.h
> > >>>> +++ b/drivers/media/usb/uvc/uvcvideo.h
> > >>>> @@ -165,6 +165,10 @@
> > >>>> {0x32, 0x00, 0x00, 0x00, 0x02, 0x00, 0x10, 0x00, \
> > >>>> 0x80, 0x00, 0x00, 0xaa, 0x00, 0x38, 0x9b, 0x71}
> > >>>>
> > >>>> +#define UVC_GUID_FORMAT_HEVC \
> > >>>> + { 'H', 'E', 'V', 'C', 0x00, 0x00, 0x10, 0x00, \
> > >>>> + 0x80, 0x00, 0x00, 0xaa, 0x00, 0x38, 0x9b, 0x71}
> > >>>> +
> > >>>>
> > >>>> /* ------------------------------------------------------------------------
> > >>>> * Driver specific constants.
>
> --
> Regards,
>
> Laurent Pinchart

--
Dmitry Buzdyk
Rhonda Software