[FFmpeg-devel] Re: [PATCH v2] avcodec/libvpxenc: fix sRGB colorspace for non-RGB pixel formats

[James Zern via ffmpeg-devel <ffmpeg-devel@ffmpeg.org>]

On Mon, Feb 23, 2026 at 4:26 PM Guangyu Sun via ffmpeg-devel
<ffmpeg-devel@ffmpeg.org> wrote:
>
> When encoding VP9 with a YUV pixel format (e.g. yuv420p) and
> AVCOL_SPC_RGB colorspace metadata, libvpxenc unconditionally set
> VPX_CS_SRGB. This produced a spec-violating bitstream: Profile 0
> (4:2:0) with sRGB colorspace, which is only valid for Profile 1/3
> (4:4:4). The resulting file is undecodable.
>
> Fix this by setting ctx->vpx_cs to VPX_CS_SRGB in set_pix_fmt()
> for 4:4:4 YUV formats when AVCOL_SPC_RGB is set, matching the
> existing GBRP path. This covers the legitimate case of RGB data in
> YUV444 containers (e.g. H.264 High 4:4:4 with identity matrix).
>
> With this change, any AVCOL_SPC_RGB that reaches the switch in
> set_colorspace() is guaranteed to be a subsampled format where
> sRGB is invalid. Log a warning and fall back to VPX_CS_BT_709.
>
> To reproduce:
>
>   # generate a bad source
>   ffmpeg -f lavfi -i testsrc=s=64x64:d=1:r=1 \
>     -pix_fmt yuv420p -colorspace rgb bad.mp4
>   # transcode with default parameters
>   ffmpeg -i bad.mp4 bad.webm
>   # check decoding
>   ffmpeg -i bad.webm -f null -
>   # -> 0 frames decoded, error
>
> Signed-off-by: Guangyu Sun <gsun@roblox.com>
> ---
> v2:
>   - Only fall back to BT.709 if the pixel format is not 4:4:4,
>     addressing feedback regarding Profile 1/3 support.
>
>  libavcodec/libvpxenc.c | 16 +++++++++++++++-
>  1 file changed, 15 insertions(+), 1 deletion(-)
>
> [...]
>          switch (avctx->colorspace) {
> -        case AVCOL_SPC_RGB:         vpx_cs = VPX_CS_SRGB;      break;
> +        case AVCOL_SPC_RGB:
> +            // All sRGB-compatible formats (GBRP*, and YUV444* with
> +            // AVCOL_SPC_RGB) set ctx->vpx_cs in set_pix_fmt() and
> +            // take the branch above, so reaching here means a
> +            // subsampled format incompatible with RGB colorspace.
> +            av_log(avctx, AV_LOG_WARNING,
> +                   "RGB colorspace is not compatible with pixel format %s, "
> +                   "using BT.709 instead.\n",
> +                   av_get_pix_fmt_name(avctx->pix_fmt));
> +            vpx_cs = VPX_CS_BT_709;

I don't think an arbitrary fallback is the best option here. The
library should have rejected the configuration (which will be fixed),
but in practice if the colorspace is RGB and there's no conversion
being done, then this may not be what the user intended. I think it
would be better to surface this as an error.

> +            break;
>          case AVCOL_SPC_BT709:       vpx_cs = VPX_CS_BT_709;    break;
>          case AVCOL_SPC_UNSPECIFIED: vpx_cs = VPX_CS_UNKNOWN;   break;
>          case AVCOL_SPC_RESERVED:    vpx_cs = VPX_CS_RESERVED;  break;
> --
> 2.52.0
>
> _______________________________________________
> ffmpeg-devel mailing list -- ffmpeg-devel@ffmpeg.org
> To unsubscribe send an email to ffmpeg-devel-leave@ffmpeg.org
_______________________________________________
ffmpeg-devel mailing list -- ffmpeg-devel@ffmpeg.org
To unsubscribe send an email to ffmpeg-devel-leave@ffmpeg.org