[PATCH 3/3] drm/gud: Fix endianness in gud_xrgb8888_to_color() helper

From: Geert Uytterhoeven
Date: Fri Jul 08 2022 - 14:22:03 EST


DRM formats are defined to be little-endian, unless the
DRM_FORMAT_BIG_ENDIAN flag is set. Hence when converting from one
format to another, multi-byte pixel values loaded from memory must be
converted from little-endian to host-endian. Conversely, multi-byte
pixel values written to memory must be converted from host-endian to
little-endian. Currently only drm_fb_xrgb8888_to_rgb332_line() includes
endianness handling.

Fix gud_xrgb8888_to_color() on big-endian platforms by adding the
missing endianness handling.

Signed-off-by: Geert Uytterhoeven <geert@xxxxxxxxxxxxxx>
---
Compile-tested only.

Interestingly, drm_fb_xrgb8888_to_rgb332() was introduced for GUD,
and always had correct endiannes handling...
---
drivers/gpu/drm/gud/gud_pipe.c | 14 ++++++++------
1 file changed, 8 insertions(+), 6 deletions(-)

diff --git a/drivers/gpu/drm/gud/gud_pipe.c b/drivers/gpu/drm/gud/gud_pipe.c
index 4873f9799f412e04..d42592f6daab8b2a 100644
--- a/drivers/gpu/drm/gud/gud_pipe.c
+++ b/drivers/gpu/drm/gud/gud_pipe.c
@@ -105,7 +105,8 @@ static size_t gud_xrgb8888_to_color(u8 *dst, const struct drm_format_info *forma
unsigned int bits_per_pixel = 8 / block_width;
u8 r, g, b, pix, *block = dst; /* Assign to silence compiler warning */
unsigned int x, y, width;
- u32 *pix32;
+ __le32 *sbuf32;
+ u32 pix32;
size_t len;

/* Start on a byte boundary */
@@ -114,8 +115,8 @@ static size_t gud_xrgb8888_to_color(u8 *dst, const struct drm_format_info *forma
len = drm_format_info_min_pitch(format, 0, width) * drm_rect_height(rect);

for (y = rect->y1; y < rect->y2; y++) {
- pix32 = src + (y * fb->pitches[0]);
- pix32 += rect->x1;
+ sbuf32 = src + (y * fb->pitches[0]);
+ sbuf32 += rect->x1;

for (x = 0; x < width; x++) {
unsigned int pixpos = x % block_width; /* within byte from the left */
@@ -126,9 +127,10 @@ static size_t gud_xrgb8888_to_color(u8 *dst, const struct drm_format_info *forma
*block = 0;
}

- r = *pix32 >> 16;
- g = *pix32 >> 8;
- b = *pix32++;
+ pix32 = le32_to_cpu(*sbuf32++);
+ r = pix32 >> 16;
+ g = pix32 >> 8;
+ b = pix32;

switch (format->format) {
case GUD_DRM_FORMAT_XRGB1111:
--
2.25.1