[FFmpeg-devel] [PATCH] avcodec/libvpxenc: fix sRGB colorspace for non-RGB pixel formats

[Guangyu Sun via ffmpeg-devel <ffmpeg-devel@ffmpeg.org>]

When encoding VP9 with a YUV pixel format (e.g. yuv420p) and
AVCOL_SPC_RGB colorspace metadata, libvpxenc unconditionally set
VPX_CS_SRGB. This produced a spec-violating bitstream: Profile 0
(4:2:0) with sRGB colorspace, which is only valid for Profile 1/3
(4:4:4). The resulting file is undecodable.

RGB pixel formats (GBRP, GBRP10, GBRP12) already correctly set
ctx->vpx_cs to VPX_CS_SRGB in set_pix_fmt() and bypass the
colorspace switch in set_colorspace(). So reaching the AVCOL_SPC_RGB
case means the pixel format is YUV with incorrect RGB metadata.
Log a warning and fall back to VPX_CS_BT_709.

To reproduce:

  # generate a bad source
  ffmpeg -f lavfi -i testsrc=s=64x64:d=1:r=1 -pix_fmt yuv420p -colorspace rgb bad.mp4
  # transcode with default parameters
  ffmpeg -i bad.mp4 bad.webm
  # check decoding
  ffmpeg -i bad.webm -f null -
  # -> 0 frames decoded, error

Signed-off-by: Guangyu Sun <gsun@roblox.com>
---
 libavcodec/libvpxenc.c | 12 +++++++++++-
 1 file changed, 11 insertions(+), 1 deletion(-)

diff --git a/libavcodec/libvpxenc.c b/libavcodec/libvpxenc.c
index 88c058c403..be731a014f 100644
--- a/libavcodec/libvpxenc.c
+++ b/libavcodec/libvpxenc.c
@@ -902,7 +902,17 @@ static void set_colorspace(AVCodecContext *avctx)
         vpx_cs = ctx->vpx_cs;
     } else {
         switch (avctx->colorspace) {
-        case AVCOL_SPC_RGB:         vpx_cs = VPX_CS_SRGB;      break;
+        case AVCOL_SPC_RGB:
+            // RGB pixel formats (GBRP, GBRP10, GBRP12) set ctx->vpx_cs
+            // to VPX_CS_SRGB in set_pix_fmt() and take the branch above,
+            // so reaching here means the pixel format is YUV with
+            // incorrect RGB colorspace metadata.
+            av_log(avctx, AV_LOG_WARNING,
+                   "RGB colorspace is not compatible with pixel format %s, "
+                   "using BT.709 instead.\n",
+                   av_get_pix_fmt_name(avctx->pix_fmt));
+            vpx_cs = VPX_CS_BT_709;
+            break;
         case AVCOL_SPC_BT709:       vpx_cs = VPX_CS_BT_709;    break;
         case AVCOL_SPC_UNSPECIFIED: vpx_cs = VPX_CS_UNKNOWN;   break;
         case AVCOL_SPC_RESERVED:    vpx_cs = VPX_CS_RESERVED;  break;
-- 
2.52.0

_______________________________________________
ffmpeg-devel mailing list -- ffmpeg-devel@ffmpeg.org
To unsubscribe send an email to ffmpeg-devel-leave@ffmpeg.org