Git Inbox Mirror of the ffmpeg-devel mailing list - see https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
 help / color / mirror / Atom feed
* [FFmpeg-devel] [PATCH] avcodec/libvpxenc: fix sRGB colorspace for non-RGB pixel formats
@ 2026-02-23 19:21 Guangyu Sun via ffmpeg-devel
  2026-02-23 21:25 ` [FFmpeg-devel] " James Zern via ffmpeg-devel
                   ` (2 more replies)
  0 siblings, 3 replies; 6+ messages in thread
From: Guangyu Sun via ffmpeg-devel @ 2026-02-23 19:21 UTC (permalink / raw)
  To: ffmpeg-devel; +Cc: Guangyu Sun

When encoding VP9 with a YUV pixel format (e.g. yuv420p) and
AVCOL_SPC_RGB colorspace metadata, libvpxenc unconditionally set
VPX_CS_SRGB. This produced a spec-violating bitstream: Profile 0
(4:2:0) with sRGB colorspace, which is only valid for Profile 1/3
(4:4:4). The resulting file is undecodable.

RGB pixel formats (GBRP, GBRP10, GBRP12) already correctly set
ctx->vpx_cs to VPX_CS_SRGB in set_pix_fmt() and bypass the
colorspace switch in set_colorspace(). So reaching the AVCOL_SPC_RGB
case means the pixel format is YUV with incorrect RGB metadata.
Log a warning and fall back to VPX_CS_BT_709.

To reproduce:

  # generate a bad source
  ffmpeg -f lavfi -i testsrc=s=64x64:d=1:r=1 -pix_fmt yuv420p -colorspace rgb bad.mp4
  # transcode with default parameters
  ffmpeg -i bad.mp4 bad.webm
  # check decoding
  ffmpeg -i bad.webm -f null -
  # -> 0 frames decoded, error

Signed-off-by: Guangyu Sun <gsun@roblox.com>
---
 libavcodec/libvpxenc.c | 12 +++++++++++-
 1 file changed, 11 insertions(+), 1 deletion(-)

diff --git a/libavcodec/libvpxenc.c b/libavcodec/libvpxenc.c
index 88c058c403..be731a014f 100644
--- a/libavcodec/libvpxenc.c
+++ b/libavcodec/libvpxenc.c
@@ -902,7 +902,17 @@ static void set_colorspace(AVCodecContext *avctx)
         vpx_cs = ctx->vpx_cs;
     } else {
         switch (avctx->colorspace) {
-        case AVCOL_SPC_RGB:         vpx_cs = VPX_CS_SRGB;      break;
+        case AVCOL_SPC_RGB:
+            // RGB pixel formats (GBRP, GBRP10, GBRP12) set ctx->vpx_cs
+            // to VPX_CS_SRGB in set_pix_fmt() and take the branch above,
+            // so reaching here means the pixel format is YUV with
+            // incorrect RGB colorspace metadata.
+            av_log(avctx, AV_LOG_WARNING,
+                   "RGB colorspace is not compatible with pixel format %s, "
+                   "using BT.709 instead.\n",
+                   av_get_pix_fmt_name(avctx->pix_fmt));
+            vpx_cs = VPX_CS_BT_709;
+            break;
         case AVCOL_SPC_BT709:       vpx_cs = VPX_CS_BT_709;    break;
         case AVCOL_SPC_UNSPECIFIED: vpx_cs = VPX_CS_UNKNOWN;   break;
         case AVCOL_SPC_RESERVED:    vpx_cs = VPX_CS_RESERVED;  break;
-- 
2.52.0

_______________________________________________
ffmpeg-devel mailing list -- ffmpeg-devel@ffmpeg.org
To unsubscribe send an email to ffmpeg-devel-leave@ffmpeg.org

^ permalink raw reply	[flat|nested] 6+ messages in thread

* [FFmpeg-devel] Re: [PATCH] avcodec/libvpxenc: fix sRGB colorspace for non-RGB pixel formats
  2026-02-23 19:21 [FFmpeg-devel] [PATCH] avcodec/libvpxenc: fix sRGB colorspace for non-RGB pixel formats Guangyu Sun via ffmpeg-devel
@ 2026-02-23 21:25 ` James Zern via ffmpeg-devel
  2026-02-24  0:25 ` [FFmpeg-devel] [PATCH v2] " Guangyu Sun via ffmpeg-devel
  2026-02-26 18:05 ` [FFmpeg-devel] [PATCH v3] " Guangyu Sun via ffmpeg-devel
  2 siblings, 0 replies; 6+ messages in thread
From: James Zern via ffmpeg-devel @ 2026-02-23 21:25 UTC (permalink / raw)
  To: FFmpeg development discussions and patches; +Cc: Guangyu Sun, James Zern

Hi,

On Mon, Feb 23, 2026 at 11:22 AM Guangyu Sun via ffmpeg-devel
<ffmpeg-devel@ffmpeg.org> wrote:
>
> When encoding VP9 with a YUV pixel format (e.g. yuv420p) and
> AVCOL_SPC_RGB colorspace metadata, libvpxenc unconditionally set
> VPX_CS_SRGB. This produced a spec-violating bitstream: Profile 0
> (4:2:0) with sRGB colorspace, which is only valid for Profile 1/3
> (4:4:4). The resulting file is undecodable.
>
> RGB pixel formats (GBRP, GBRP10, GBRP12) already correctly set
> ctx->vpx_cs to VPX_CS_SRGB in set_pix_fmt() and bypass the
> colorspace switch in set_colorspace(). So reaching the AVCOL_SPC_RGB
> case means the pixel format is YUV with incorrect RGB metadata.
> Log a warning and fall back to VPX_CS_BT_709.
>
> To reproduce:
>
>   # generate a bad source
>   ffmpeg -f lavfi -i testsrc=s=64x64:d=1:r=1 -pix_fmt yuv420p -colorspace rgb bad.mp4
>   # transcode with default parameters
>   ffmpeg -i bad.mp4 bad.webm
>   # check decoding
>   ffmpeg -i bad.webm -f null -
>   # -> 0 frames decoded, error
>
> Signed-off-by: Guangyu Sun <gsun@roblox.com>
> ---
>  libavcodec/libvpxenc.c | 12 +++++++++++-
>  1 file changed, 11 insertions(+), 1 deletion(-)
>
> diff --git a/libavcodec/libvpxenc.c b/libavcodec/libvpxenc.c
> index 88c058c403..be731a014f 100644
> --- a/libavcodec/libvpxenc.c
> +++ b/libavcodec/libvpxenc.c
> @@ -902,7 +902,17 @@ static void set_colorspace(AVCodecContext *avctx)
>          vpx_cs = ctx->vpx_cs;
>      } else {
>          switch (avctx->colorspace) {
> -        case AVCOL_SPC_RGB:         vpx_cs = VPX_CS_SRGB;      break;
> +        case AVCOL_SPC_RGB:
> +            // RGB pixel formats (GBRP, GBRP10, GBRP12) set ctx->vpx_cs
> +            // to VPX_CS_SRGB in set_pix_fmt() and take the branch above,
> +            // so reaching here means the pixel format is YUV with
> +            // incorrect RGB colorspace metadata.

What if the input is yuv444? In that case it could be holding rgb
data. Profile 1 and 3 assume no subsampling when decoding sRGB
content.

> +            av_log(avctx, AV_LOG_WARNING,
> +                   "RGB colorspace is not compatible with pixel format %s, "
> +                   "using BT.709 instead.\n",
> +                   av_get_pix_fmt_name(avctx->pix_fmt));
> +            vpx_cs = VPX_CS_BT_709;
> +            break;
>          case AVCOL_SPC_BT709:       vpx_cs = VPX_CS_BT_709;    break;
>          case AVCOL_SPC_UNSPECIFIED: vpx_cs = VPX_CS_UNKNOWN;   break;
>          case AVCOL_SPC_RESERVED:    vpx_cs = VPX_CS_RESERVED;  break;
> --
> 2.52.0
>
> _______________________________________________
> ffmpeg-devel mailing list -- ffmpeg-devel@ffmpeg.org
> To unsubscribe send an email to ffmpeg-devel-leave@ffmpeg.org
_______________________________________________
ffmpeg-devel mailing list -- ffmpeg-devel@ffmpeg.org
To unsubscribe send an email to ffmpeg-devel-leave@ffmpeg.org

^ permalink raw reply	[flat|nested] 6+ messages in thread

* [FFmpeg-devel] [PATCH v2] avcodec/libvpxenc: fix sRGB colorspace for non-RGB pixel formats
  2026-02-23 19:21 [FFmpeg-devel] [PATCH] avcodec/libvpxenc: fix sRGB colorspace for non-RGB pixel formats Guangyu Sun via ffmpeg-devel
  2026-02-23 21:25 ` [FFmpeg-devel] " James Zern via ffmpeg-devel
@ 2026-02-24  0:25 ` Guangyu Sun via ffmpeg-devel
  2026-02-24 18:33   ` [FFmpeg-devel] " James Zern via ffmpeg-devel
  2026-02-26 18:05 ` [FFmpeg-devel] [PATCH v3] " Guangyu Sun via ffmpeg-devel
  2 siblings, 1 reply; 6+ messages in thread
From: Guangyu Sun via ffmpeg-devel @ 2026-02-24  0:25 UTC (permalink / raw)
  To: ffmpeg-devel; +Cc: Guangyu Sun

When encoding VP9 with a YUV pixel format (e.g. yuv420p) and
AVCOL_SPC_RGB colorspace metadata, libvpxenc unconditionally set
VPX_CS_SRGB. This produced a spec-violating bitstream: Profile 0
(4:2:0) with sRGB colorspace, which is only valid for Profile 1/3
(4:4:4). The resulting file is undecodable.

Fix this by setting ctx->vpx_cs to VPX_CS_SRGB in set_pix_fmt()
for 4:4:4 YUV formats when AVCOL_SPC_RGB is set, matching the
existing GBRP path. This covers the legitimate case of RGB data in
YUV444 containers (e.g. H.264 High 4:4:4 with identity matrix).

With this change, any AVCOL_SPC_RGB that reaches the switch in
set_colorspace() is guaranteed to be a subsampled format where
sRGB is invalid. Log a warning and fall back to VPX_CS_BT_709.

To reproduce:

  # generate a bad source
  ffmpeg -f lavfi -i testsrc=s=64x64:d=1:r=1 \
    -pix_fmt yuv420p -colorspace rgb bad.mp4
  # transcode with default parameters
  ffmpeg -i bad.mp4 bad.webm
  # check decoding
  ffmpeg -i bad.webm -f null -
  # -> 0 frames decoded, error

Signed-off-by: Guangyu Sun <gsun@roblox.com>
---
v2:
  - Only fall back to BT.709 if the pixel format is not 4:4:4,
    addressing feedback regarding Profile 1/3 support.

 libavcodec/libvpxenc.c | 16 +++++++++++++++-
 1 file changed, 15 insertions(+), 1 deletion(-)

diff --git a/libavcodec/libvpxenc.c b/libavcodec/libvpxenc.c
index 082709c41f..fd80d7ca04 100644
--- a/libavcodec/libvpxenc.c
+++ b/libavcodec/libvpxenc.c
@@ -835,6 +835,8 @@ static int set_pix_fmt(AVCodecContext *avctx, vpx_codec_caps_t codec_caps,
     case AV_PIX_FMT_GBRP:
         ctx->vpx_cs = VPX_CS_SRGB;
     case AV_PIX_FMT_YUV444P:
+        if (avctx->colorspace == AVCOL_SPC_RGB)
+            ctx->vpx_cs = VPX_CS_SRGB;
         enccfg->g_profile = 1;
         *img_fmt = VPX_IMG_FMT_I444;
         return 0;
@@ -870,6 +872,8 @@ static int set_pix_fmt(AVCodecContext *avctx, vpx_codec_caps_t codec_caps,
         ctx->vpx_cs = VPX_CS_SRGB;
     case AV_PIX_FMT_YUV444P10:
     case AV_PIX_FMT_YUV444P12:
+        if (avctx->colorspace == AVCOL_SPC_RGB)
+            ctx->vpx_cs = VPX_CS_SRGB;
         if (codec_caps & VPX_CODEC_CAP_HIGHBITDEPTH) {
             enccfg->g_profile = 3;
             *img_fmt = VPX_IMG_FMT_I44416;
@@ -893,7 +897,17 @@ static void set_colorspace(AVCodecContext *avctx)
         vpx_cs = ctx->vpx_cs;
     } else {
         switch (avctx->colorspace) {
-        case AVCOL_SPC_RGB:         vpx_cs = VPX_CS_SRGB;      break;
+        case AVCOL_SPC_RGB:
+            // All sRGB-compatible formats (GBRP*, and YUV444* with
+            // AVCOL_SPC_RGB) set ctx->vpx_cs in set_pix_fmt() and
+            // take the branch above, so reaching here means a
+            // subsampled format incompatible with RGB colorspace.
+            av_log(avctx, AV_LOG_WARNING,
+                   "RGB colorspace is not compatible with pixel format %s, "
+                   "using BT.709 instead.\n",
+                   av_get_pix_fmt_name(avctx->pix_fmt));
+            vpx_cs = VPX_CS_BT_709;
+            break;
         case AVCOL_SPC_BT709:       vpx_cs = VPX_CS_BT_709;    break;
         case AVCOL_SPC_UNSPECIFIED: vpx_cs = VPX_CS_UNKNOWN;   break;
         case AVCOL_SPC_RESERVED:    vpx_cs = VPX_CS_RESERVED;  break;
-- 
2.52.0

_______________________________________________
ffmpeg-devel mailing list -- ffmpeg-devel@ffmpeg.org
To unsubscribe send an email to ffmpeg-devel-leave@ffmpeg.org

^ permalink raw reply	[flat|nested] 6+ messages in thread

* [FFmpeg-devel] Re: [PATCH v2] avcodec/libvpxenc: fix sRGB colorspace for non-RGB pixel formats
  2026-02-24  0:25 ` [FFmpeg-devel] [PATCH v2] " Guangyu Sun via ffmpeg-devel
@ 2026-02-24 18:33   ` James Zern via ffmpeg-devel
  2026-02-24 19:29     ` Guangyu Sun via ffmpeg-devel
  0 siblings, 1 reply; 6+ messages in thread
From: James Zern via ffmpeg-devel @ 2026-02-24 18:33 UTC (permalink / raw)
  To: FFmpeg development discussions and patches; +Cc: James Zern

On Mon, Feb 23, 2026 at 4:26 PM Guangyu Sun via ffmpeg-devel
<ffmpeg-devel@ffmpeg.org> wrote:
>
> When encoding VP9 with a YUV pixel format (e.g. yuv420p) and
> AVCOL_SPC_RGB colorspace metadata, libvpxenc unconditionally set
> VPX_CS_SRGB. This produced a spec-violating bitstream: Profile 0
> (4:2:0) with sRGB colorspace, which is only valid for Profile 1/3
> (4:4:4). The resulting file is undecodable.
>
> Fix this by setting ctx->vpx_cs to VPX_CS_SRGB in set_pix_fmt()
> for 4:4:4 YUV formats when AVCOL_SPC_RGB is set, matching the
> existing GBRP path. This covers the legitimate case of RGB data in
> YUV444 containers (e.g. H.264 High 4:4:4 with identity matrix).
>
> With this change, any AVCOL_SPC_RGB that reaches the switch in
> set_colorspace() is guaranteed to be a subsampled format where
> sRGB is invalid. Log a warning and fall back to VPX_CS_BT_709.
>
> To reproduce:
>
>   # generate a bad source
>   ffmpeg -f lavfi -i testsrc=s=64x64:d=1:r=1 \
>     -pix_fmt yuv420p -colorspace rgb bad.mp4
>   # transcode with default parameters
>   ffmpeg -i bad.mp4 bad.webm
>   # check decoding
>   ffmpeg -i bad.webm -f null -
>   # -> 0 frames decoded, error
>
> Signed-off-by: Guangyu Sun <gsun@roblox.com>
> ---
> v2:
>   - Only fall back to BT.709 if the pixel format is not 4:4:4,
>     addressing feedback regarding Profile 1/3 support.
>
>  libavcodec/libvpxenc.c | 16 +++++++++++++++-
>  1 file changed, 15 insertions(+), 1 deletion(-)
>
> [...]
>          switch (avctx->colorspace) {
> -        case AVCOL_SPC_RGB:         vpx_cs = VPX_CS_SRGB;      break;
> +        case AVCOL_SPC_RGB:
> +            // All sRGB-compatible formats (GBRP*, and YUV444* with
> +            // AVCOL_SPC_RGB) set ctx->vpx_cs in set_pix_fmt() and
> +            // take the branch above, so reaching here means a
> +            // subsampled format incompatible with RGB colorspace.
> +            av_log(avctx, AV_LOG_WARNING,
> +                   "RGB colorspace is not compatible with pixel format %s, "
> +                   "using BT.709 instead.\n",
> +                   av_get_pix_fmt_name(avctx->pix_fmt));
> +            vpx_cs = VPX_CS_BT_709;

I don't think an arbitrary fallback is the best option here. The
library should have rejected the configuration (which will be fixed),
but in practice if the colorspace is RGB and there's no conversion
being done, then this may not be what the user intended. I think it
would be better to surface this as an error.

> +            break;
>          case AVCOL_SPC_BT709:       vpx_cs = VPX_CS_BT_709;    break;
>          case AVCOL_SPC_UNSPECIFIED: vpx_cs = VPX_CS_UNKNOWN;   break;
>          case AVCOL_SPC_RESERVED:    vpx_cs = VPX_CS_RESERVED;  break;
> --
> 2.52.0
>
> _______________________________________________
> ffmpeg-devel mailing list -- ffmpeg-devel@ffmpeg.org
> To unsubscribe send an email to ffmpeg-devel-leave@ffmpeg.org
_______________________________________________
ffmpeg-devel mailing list -- ffmpeg-devel@ffmpeg.org
To unsubscribe send an email to ffmpeg-devel-leave@ffmpeg.org

^ permalink raw reply	[flat|nested] 6+ messages in thread

* [FFmpeg-devel] Re: [PATCH v2] avcodec/libvpxenc: fix sRGB colorspace for non-RGB pixel formats
  2026-02-24 18:33   ` [FFmpeg-devel] " James Zern via ffmpeg-devel
@ 2026-02-24 19:29     ` Guangyu Sun via ffmpeg-devel
  0 siblings, 0 replies; 6+ messages in thread
From: Guangyu Sun via ffmpeg-devel @ 2026-02-24 19:29 UTC (permalink / raw)
  To: FFmpeg development discussions and patches; +Cc: Guangyu Sun

On Tue, Feb 24, 2026 at 10:34 AM James Zern via ffmpeg-devel
<ffmpeg-devel@ffmpeg.org> wrote:
>
> On Mon, Feb 23, 2026 at 4:26 PM Guangyu Sun via ffmpeg-devel
> <ffmpeg-devel@ffmpeg.org> wrote:
> >
> > When encoding VP9 with a YUV pixel format (e.g. yuv420p) and
> > AVCOL_SPC_RGB colorspace metadata, libvpxenc unconditionally set
> > VPX_CS_SRGB. This produced a spec-violating bitstream: Profile 0
> > (4:2:0) with sRGB colorspace, which is only valid for Profile 1/3
> > (4:4:4). The resulting file is undecodable.
> >
> > Fix this by setting ctx->vpx_cs to VPX_CS_SRGB in set_pix_fmt()
> > for 4:4:4 YUV formats when AVCOL_SPC_RGB is set, matching the
> > existing GBRP path. This covers the legitimate case of RGB data in
> > YUV444 containers (e.g. H.264 High 4:4:4 with identity matrix).
> >
> > With this change, any AVCOL_SPC_RGB that reaches the switch in
> > set_colorspace() is guaranteed to be a subsampled format where
> > sRGB is invalid. Log a warning and fall back to VPX_CS_BT_709.
> >
> > To reproduce:
> >
> >   # generate a bad source
> >   ffmpeg -f lavfi -i testsrc=s=64x64:d=1:r=1 \
> >     -pix_fmt yuv420p -colorspace rgb bad.mp4
> >   # transcode with default parameters
> >   ffmpeg -i bad.mp4 bad.webm
> >   # check decoding
> >   ffmpeg -i bad.webm -f null -
> >   # -> 0 frames decoded, error
> >
> > Signed-off-by: Guangyu Sun <gsun@roblox.com>
> > ---
> > v2:
> >   - Only fall back to BT.709 if the pixel format is not 4:4:4,
> >     addressing feedback regarding Profile 1/3 support.
> >
> >  libavcodec/libvpxenc.c | 16 +++++++++++++++-
> >  1 file changed, 15 insertions(+), 1 deletion(-)
> >
> > [...]
> >          switch (avctx->colorspace) {
> > -        case AVCOL_SPC_RGB:         vpx_cs = VPX_CS_SRGB;      break;
> > +        case AVCOL_SPC_RGB:
> > +            // All sRGB-compatible formats (GBRP*, and YUV444* with
> > +            // AVCOL_SPC_RGB) set ctx->vpx_cs in set_pix_fmt() and
> > +            // take the branch above, so reaching here means a
> > +            // subsampled format incompatible with RGB colorspace.
> > +            av_log(avctx, AV_LOG_WARNING,
> > +                   "RGB colorspace is not compatible with pixel format %s, "
> > +                   "using BT.709 instead.\n",
> > +                   av_get_pix_fmt_name(avctx->pix_fmt));
> > +            vpx_cs = VPX_CS_BT_709;
>
> I don't think an arbitrary fallback is the best option here. The
> library should have rejected the configuration (which will be fixed),
> but in practice if the colorspace is RGB and there's no conversion
> being done, then this may not be what the user intended. I think it
> would be better to surface this as an error.

Thanks for the feedback. Will try to submit a patch to libvpx.
Meanwhile, do you think it is still worth returning an error here in
ffmpeg? I can send out a v3 if preferred.
>
> > +            break;
> >          case AVCOL_SPC_BT709:       vpx_cs = VPX_CS_BT_709;    break;
> >          case AVCOL_SPC_UNSPECIFIED: vpx_cs = VPX_CS_UNKNOWN;   break;
> >          case AVCOL_SPC_RESERVED:    vpx_cs = VPX_CS_RESERVED;  break;
> > --
> > 2.52.0
> >
> > _______________________________________________
> > ffmpeg-devel mailing list -- ffmpeg-devel@ffmpeg.org
> > To unsubscribe send an email to ffmpeg-devel-leave@ffmpeg.org
> _______________________________________________
> ffmpeg-devel mailing list -- ffmpeg-devel@ffmpeg.org
> To unsubscribe send an email to ffmpeg-devel-leave@ffmpeg.org
_______________________________________________
ffmpeg-devel mailing list -- ffmpeg-devel@ffmpeg.org
To unsubscribe send an email to ffmpeg-devel-leave@ffmpeg.org

^ permalink raw reply	[flat|nested] 6+ messages in thread

* [FFmpeg-devel] [PATCH v3] avcodec/libvpxenc: fix sRGB colorspace for non-RGB pixel formats
  2026-02-23 19:21 [FFmpeg-devel] [PATCH] avcodec/libvpxenc: fix sRGB colorspace for non-RGB pixel formats Guangyu Sun via ffmpeg-devel
  2026-02-23 21:25 ` [FFmpeg-devel] " James Zern via ffmpeg-devel
  2026-02-24  0:25 ` [FFmpeg-devel] [PATCH v2] " Guangyu Sun via ffmpeg-devel
@ 2026-02-26 18:05 ` Guangyu Sun via ffmpeg-devel
  2 siblings, 0 replies; 6+ messages in thread
From: Guangyu Sun via ffmpeg-devel @ 2026-02-26 18:05 UTC (permalink / raw)
  To: ffmpeg-devel; +Cc: Guangyu Sun

When encoding VP9 with a YUV pixel format (e.g. yuv420p) and
AVCOL_SPC_RGB colorspace metadata, libvpxenc unconditionally set
VPX_CS_SRGB. This produced a spec-violating bitstream: Profile 0
(4:2:0) with sRGB colorspace, which is only valid for Profile 1/3
(4:4:4). The resulting file is undecodable.

Fix this by setting ctx->vpx_cs to VPX_CS_SRGB in set_pix_fmt()
for 4:4:4 YUV formats when AVCOL_SPC_RGB is set, matching the
existing GBRP path. This covers the legitimate case of RGB data in
YUV444 containers (e.g. H.264 High 4:4:4 with identity matrix).

With this change, any AVCOL_SPC_RGB that reaches the switch in
set_colorspace() is guaranteed to be a subsampled format where
sRGB is invalid. Return an error so the user can fix their
pipeline rather than silently producing incorrect output.

To reproduce:

  ffmpeg -f lavfi -i testsrc=s=64x64:d=1:r=1 \
    -c:v libvpx-vp9 -pix_fmt yuv420p -colorspace rgb bad.webm
  ffprobe bad.webm
  # -> "vp9 (Profile 0), none(pc, gbr/...), 64x64"
  ffmpeg -i bad.webm -f null -
  # -> 0 frames decoded, error

Signed-off-by: Guangyu Sun <gsun@roblox.com>
---
v2:
  - Only fall back to BT.709 if the pixel format is not 4:4:4,
    addressing feedback regarding Profile 1/3 support.
v3:
  - Hard error via AVERROR(EINVAL) - set_colorspace() now returns
    int, and the call site in vpx_init() propagates the error

 libavcodec/libvpxenc.c | 19 +++++++++++++++----
 1 file changed, 15 insertions(+), 4 deletions(-)

diff --git a/libavcodec/libvpxenc.c b/libavcodec/libvpxenc.c
index 88c058c403..c181927a1d 100644
--- a/libavcodec/libvpxenc.c
+++ b/libavcodec/libvpxenc.c
@@ -838,6 +838,8 @@ static int set_pix_fmt(AVCodecContext *avctx, vpx_codec_caps_t codec_caps,
         ctx->vpx_cs = VPX_CS_SRGB;
     case AV_PIX_FMT_YUV444P:
     case AV_PIX_FMT_YUVA444P:
+        if (avctx->colorspace == AVCOL_SPC_RGB)
+            ctx->vpx_cs = VPX_CS_SRGB;
         enccfg->g_profile = 1;
         *img_fmt = VPX_IMG_FMT_I444;
         return 0;
@@ -879,6 +881,8 @@ static int set_pix_fmt(AVCodecContext *avctx, vpx_codec_caps_t codec_caps,
     case AV_PIX_FMT_YUVA444P10:
     case AV_PIX_FMT_YUV444P12:
     case AV_PIX_FMT_YUVA444P12:
+        if (avctx->colorspace == AVCOL_SPC_RGB)
+            ctx->vpx_cs = VPX_CS_SRGB;
         if (codec_caps & VPX_CODEC_CAP_HIGHBITDEPTH) {
             enccfg->g_profile = 3;
             *img_fmt = VPX_IMG_FMT_I44416;
@@ -893,7 +897,7 @@ static int set_pix_fmt(AVCodecContext *avctx, vpx_codec_caps_t codec_caps,
     return AVERROR_INVALIDDATA;
 }
 
-static void set_colorspace(AVCodecContext *avctx)
+static int set_colorspace(AVCodecContext *avctx)
 {
     enum vpx_color_space vpx_cs;
     VPxContext *ctx = avctx->priv_data;
@@ -902,7 +906,11 @@ static void set_colorspace(AVCodecContext *avctx)
         vpx_cs = ctx->vpx_cs;
     } else {
         switch (avctx->colorspace) {
-        case AVCOL_SPC_RGB:         vpx_cs = VPX_CS_SRGB;      break;
+        case AVCOL_SPC_RGB:
+            av_log(avctx, AV_LOG_ERROR,
+                   "RGB colorspace is not compatible with pixel format %s.\n",
+                   av_get_pix_fmt_name(avctx->pix_fmt));
+            return AVERROR(EINVAL);
         case AVCOL_SPC_BT709:       vpx_cs = VPX_CS_BT_709;    break;
         case AVCOL_SPC_UNSPECIFIED: vpx_cs = VPX_CS_UNKNOWN;   break;
         case AVCOL_SPC_RESERVED:    vpx_cs = VPX_CS_RESERVED;  break;
@@ -913,10 +921,11 @@ static void set_colorspace(AVCodecContext *avctx)
         default:
             av_log(avctx, AV_LOG_WARNING, "Unsupported colorspace (%d)\n",
                    avctx->colorspace);
-            return;
+            return 0;
         }
     }
     codecctl_int(avctx, VP9E_SET_COLOR_SPACE, vpx_cs);
+    return 0;
 }
 
 #if VPX_ENCODER_ABI_VERSION >= 11
@@ -1277,7 +1286,9 @@ static av_cold int vpx_init(AVCodecContext *avctx,
             codecctl_int(avctx, VP9E_SET_FRAME_PARALLEL_DECODING, ctx->frame_parallel);
         if (ctx->aq_mode >= 0)
             codecctl_int(avctx, VP9E_SET_AQ_MODE, ctx->aq_mode);
-        set_colorspace(avctx);
+        res = set_colorspace(avctx);
+        if (res < 0)
+            return res;
 #if VPX_ENCODER_ABI_VERSION >= 11
         set_color_range(avctx);
 #endif
-- 
2.52.0

_______________________________________________
ffmpeg-devel mailing list -- ffmpeg-devel@ffmpeg.org
To unsubscribe send an email to ffmpeg-devel-leave@ffmpeg.org

^ permalink raw reply	[flat|nested] 6+ messages in thread

end of thread, other threads:[~2026-02-26 18:06 UTC | newest]

Thread overview: 6+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2026-02-23 19:21 [FFmpeg-devel] [PATCH] avcodec/libvpxenc: fix sRGB colorspace for non-RGB pixel formats Guangyu Sun via ffmpeg-devel
2026-02-23 21:25 ` [FFmpeg-devel] " James Zern via ffmpeg-devel
2026-02-24  0:25 ` [FFmpeg-devel] [PATCH v2] " Guangyu Sun via ffmpeg-devel
2026-02-24 18:33   ` [FFmpeg-devel] " James Zern via ffmpeg-devel
2026-02-24 19:29     ` Guangyu Sun via ffmpeg-devel
2026-02-26 18:05 ` [FFmpeg-devel] [PATCH v3] " Guangyu Sun via ffmpeg-devel

Git Inbox Mirror of the ffmpeg-devel mailing list - see https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

This inbox may be cloned and mirrored by anyone:

	git clone --mirror https://master.gitmailbox.com/ffmpegdev/0 ffmpegdev/git/0.git

	# If you have public-inbox 1.1+ installed, you may
	# initialize and index your mirror using the following commands:
	public-inbox-init -V2 ffmpegdev ffmpegdev/ https://master.gitmailbox.com/ffmpegdev \
		ffmpegdev@gitmailbox.com
	public-inbox-index ffmpegdev

Example config snippet for mirrors.


AGPL code for this site: git clone https://public-inbox.org/public-inbox.git