Re: [PATCH 2/2] media: uvcvideo: Convey full ycbcr colorspace information to v4l2
From: Laurent Pinchart
Date: Sun Aug 23 2020 - 11:08:57 EST
Hi Adam,
(CC'ing Hans Verkuil)
On Sun, Aug 23, 2020 at 05:54:24PM +0300, Laurent Pinchart wrote:
> Hi Adam,
>
> Thank you for the patch.
>
> On Sat, Aug 22, 2020 at 09:21:34PM -0400, Adam Goode wrote:
> > The Color Matching Descriptor has been present in USB cameras since
> > the original version of UVC, but it has never been fully used
> > in Linux.
> >
> > This change informs V4L2 of all of the critical colorspace parameters:
> > colorspace (called "color primaries" in UVC), transfer function
> > (called "transfer characteristics" in UVC), ycbcr encoding (called
> > "matrix coefficients" in UVC), and quantization, which is always
> > LIMITED for UVC, see section 2.26 in USB_Video_FAQ_1.5.pdf.
>
> Isn't this valid for MJPEG only though ? There's not much else we can do
> though, as UVC cameras don't report quantization information. Shouldn't
> we however reflect this in the commit message, and in the comment below,
> to state that UVC requires limited quantization range for MJPEG, and
> while it doesn't explicitly specify the quantization range for other
> formats, we can only assume it should be limited as well ?
>
> The code otherwise looks good to me.
>
> Reviewed-by: Laurent Pinchart <laurent.pinchart@xxxxxxxxxxxxxxxx>
>
> Please let me know if you'd like to address the above issue.
>
> > The quantization is the most important improvement for this patch,
> > because V4L2 will otherwise interpret MJPEG as FULL range. Web browsers
> > such as Chrome and Firefox already ignore V4L2's quantization for USB
> > devices and use the correct LIMITED value, but other programs such
> > as qv4l2 will incorrectly interpret the output of MJPEG from USB
> > cameras without this change.
> >
> > Signed-off-by: Adam Goode <agoode@xxxxxxxxxx>
> > ---
> > drivers/media/usb/uvc/uvc_driver.c | 52 +++++++++++++++++++++++++++---
> > drivers/media/usb/uvc/uvc_v4l2.c | 6 ++++
> > drivers/media/usb/uvc/uvcvideo.h | 5 ++-
> > 3 files changed, 58 insertions(+), 5 deletions(-)
> >
> > diff --git a/drivers/media/usb/uvc/uvc_driver.c b/drivers/media/usb/uvc/uvc_driver.c
> > index 431d86e1c94b..c0c81b089b7d 100644
> > --- a/drivers/media/usb/uvc/uvc_driver.c
> > +++ b/drivers/media/usb/uvc/uvc_driver.c
> > @@ -248,10 +248,10 @@ static struct uvc_format_desc *uvc_format_by_guid(const u8 guid[16])
> > return NULL;
> > }
> >
> > -static u32 uvc_colorspace(const u8 primaries)
> > +static enum v4l2_colorspace uvc_colorspace(const u8 primaries)
> > {
> > - static const u8 colorprimaries[] = {
> > - 0,
> > + static const enum v4l2_colorspace colorprimaries[] = {
> > + V4L2_COLORSPACE_DEFAULT, /* Unspecified */
> > V4L2_COLORSPACE_SRGB,
> > V4L2_COLORSPACE_470_SYSTEM_M,
> > V4L2_COLORSPACE_470_SYSTEM_BG,
> > @@ -262,7 +262,43 @@ static u32 uvc_colorspace(const u8 primaries)
> > if (primaries < ARRAY_SIZE(colorprimaries))
> > return colorprimaries[primaries];
> >
> > - return 0;
> > + return V4L2_COLORSPACE_DEFAULT; /* Reserved */
> > +}
> > +
> > +static enum v4l2_xfer_func uvc_xfer_func(const u8 transfer_characteristics)
> > +{
> > + static const enum v4l2_xfer_func xfer_funcs[] = {
> > + V4L2_XFER_FUNC_DEFAULT, /* Unspecified */
> > + V4L2_XFER_FUNC_709,
> > + V4L2_XFER_FUNC_709, /* BT.470-2 M */
> > + V4L2_XFER_FUNC_709, /* BT.470-2 B, G */
> > + V4L2_XFER_FUNC_709, /* SMPTE 170M */
> > + V4L2_XFER_FUNC_SMPTE240M,
> > + V4L2_XFER_FUNC_NONE, /* Linear (V = Lc) */
> > + V4L2_XFER_FUNC_SRGB,
> > + };
> > +
> > + if (transfer_characteristics < ARRAY_SIZE(xfer_funcs))
> > + return xfer_funcs[transfer_characteristics];
> > +
> > + return V4L2_XFER_FUNC_DEFAULT; /* Reserved */
> > +}
> > +
> > +static enum v4l2_ycbcr_encoding uvc_ycbcr_enc(const u8 matrix_coefficients)
> > +{
> > + static const enum v4l2_ycbcr_encoding ycbcr_encs[] = {
> > + V4L2_YCBCR_ENC_DEFAULT, /* Unspecified */
> > + V4L2_YCBCR_ENC_709,
> > + V4L2_YCBCR_ENC_601, /* FCC */
I may have spoken a bit too fast. Doesn't FCC differ from BT.601 ?
According to https://en.wikipedia.org/wiki/Talk%3AYCbCr, the former uses
E'Y = 0.59 E'G + 0.11 E'B + 0.30 E'R
E'PB = – 0.331 E'G + 0.500 E'B – 0.169 E'R
E'PR = – 0.421 E'G – 0.079 E'B + 0.500 E'R
while the latter uses
E'Y = 0.587 E'G + 0.114 E'B + 0.299 E'R
E'PB = – 0.331 E'G + 0.500 E'B – 0.169 E'R
E'PR = – 0.419 E'G – 0.081 E'B + 0.500 E'R
We seems to be missing FCC in the V4L2 color space definitions.
> > + V4L2_YCBCR_ENC_601, /* BT.470-2 B, G */
> > + V4L2_YCBCR_ENC_601, /* SMPTE 170M */
> > + V4L2_YCBCR_ENC_SMPTE240M,
> > + };
> > +
> > + if (matrix_coefficients < ARRAY_SIZE(ycbcr_encs))
> > + return ycbcr_encs[matrix_coefficients];
> > +
> > + return V4L2_YCBCR_ENC_DEFAULT; /* Reserved */
> > }
> >
> > /* Simplify a fraction using a simple continued fraction decomposition. The
> > @@ -704,6 +740,14 @@ static int uvc_parse_format(struct uvc_device *dev,
> > }
> >
> > format->colorspace = uvc_colorspace(buffer[3]);
> > + format->xfer_func = uvc_xfer_func(buffer[4]);
> > + format->ycbcr_enc = uvc_ycbcr_enc(buffer[5]);
> > + /* All USB YCbCr encodings use LIMITED range as of UVC 1.5.
> > + * This is true even for MJPEG, which V4L2 otherwise assumes to
> > + * be FULL.
> > + * See section 2.26 in USB_Video_FAQ_1.5.pdf.
> > + */
> > + format->quantization = V4L2_QUANTIZATION_LIM_RANGE;
> >
> > buflen -= buffer[0];
> > buffer += buffer[0];
> > diff --git a/drivers/media/usb/uvc/uvc_v4l2.c b/drivers/media/usb/uvc/uvc_v4l2.c
> > index 7f14096cb44d..79daf46b3dcd 100644
> > --- a/drivers/media/usb/uvc/uvc_v4l2.c
> > +++ b/drivers/media/usb/uvc/uvc_v4l2.c
> > @@ -279,6 +279,9 @@ static int uvc_v4l2_try_format(struct uvc_streaming *stream,
> > fmt->fmt.pix.sizeimage = probe->dwMaxVideoFrameSize;
> > fmt->fmt.pix.pixelformat = format->fcc;
> > fmt->fmt.pix.colorspace = format->colorspace;
> > + fmt->fmt.pix.xfer_func = format->xfer_func;
> > + fmt->fmt.pix.ycbcr_enc = format->ycbcr_enc;
> > + fmt->fmt.pix.quantization = format->quantization;
> >
> > if (uvc_format != NULL)
> > *uvc_format = format;
> > @@ -315,6 +318,9 @@ static int uvc_v4l2_get_format(struct uvc_streaming *stream,
> > fmt->fmt.pix.bytesperline = uvc_v4l2_get_bytesperline(format, frame);
> > fmt->fmt.pix.sizeimage = stream->ctrl.dwMaxVideoFrameSize;
> > fmt->fmt.pix.colorspace = format->colorspace;
> > + fmt->fmt.pix.xfer_func = format->xfer_func;
> > + fmt->fmt.pix.ycbcr_enc = format->ycbcr_enc;
> > + fmt->fmt.pix.quantization = format->quantization;
> >
> > done:
> > mutex_unlock(&stream->mutex);
> > diff --git a/drivers/media/usb/uvc/uvcvideo.h b/drivers/media/usb/uvc/uvcvideo.h
> > index 6ab972c643e3..6508192173dd 100644
> > --- a/drivers/media/usb/uvc/uvcvideo.h
> > +++ b/drivers/media/usb/uvc/uvcvideo.h
> > @@ -370,7 +370,10 @@ struct uvc_format {
> > u8 type;
> > u8 index;
> > u8 bpp;
> > - u8 colorspace;
> > + enum v4l2_colorspace colorspace;
> > + enum v4l2_xfer_func xfer_func;
> > + enum v4l2_ycbcr_encoding ycbcr_enc;
> > + enum v4l2_quantization quantization;
> > u32 fcc;
> > u32 flags;
> >
--
Regards,
Laurent Pinchart