[FFmpeg-devel] [PATCH] avcodec/vvc: add boundary checks for CTU table access

[FFmpeg-devel] [PATCH] avcodec/vvc: add boundary checks for CTU table access

[0xBat via ffmpeg-devel]

In libavcodec/vvc/ctu.c, the functions set_tb_size and set_cb_pos lack proper boundary checks before performing memset operations. This patch adds validation against PPS dimensions to prevent heap overflows.

Signed-off-by: 0xBat <monsterbat02@gmail.com>
---
 libavcodec/vvc/ctu.c | 5 +++++
 1 file changed, 5 insertions(+)

diff --git a/libavcodec/vvc/ctu.c b/libavcodec/vvc/ctu.c
index 18cbe0fe0f..ce8a7ab90e 100644
--- a/libavcodec/vvc/ctu.c
+++ b/libavcodec/vvc/ctu.c
@@ -52,6 +52,9 @@ static void set_tb_size(const VVCFrameContext *fc, const TransformBlock *tb)
 
     for (int y = y_tb; y < end; y++) {
         const int off = y * fc->ps.pps->min_tu_width + x_tb;
+        int max_off = fc->ps.pps->min_tu_width * fc->ps.pps->min_tu_height;
+        if (off + width > max_off)
+            return;
         memset(fc->tab.tb_width [is_chroma] + off, tb->tb_width,  width);
         memset(fc->tab.tb_height[is_chroma] + off, tb->tb_height, width);
     }
@@ -1185,6 +1188,8 @@ static void set_cb_pos(const VVCFrameContext *fc, const CodingUnit *cu)
             fc->tab.cb_pos_x[ch_type][x + i] = cu->x0;
             fc->tab.cb_pos_y[ch_type][x + i] = cu->y0;
         }
+        if (x + width > fc->ps.pps->min_tu_width * fc->ps.pps->min_tu_height)
+            return;
         memset(&fc->tab.cb_width[ch_type][x], cu->cb_width, width);
         memset(&fc->tab.cb_height[ch_type][x], cu->cb_height, width);
         memset(&fc->tab.cqt_depth[ch_type][x], cu->cqt_depth, width);
-- 
2.52.0.windows.1

_______________________________________________
ffmpeg-devel mailing list -- ffmpeg-devel@ffmpeg.org
To unsubscribe send an email to ffmpeg-devel-leave@ffmpeg.org

[FFmpeg-devel] [PATCH] avcodec/vvc: clamp palette predictor size

[0xBat via ffmpeg-devel]

In palette_update_predictor, the number of entries could exceed VVC_MAX_NUM_PALETTE_PREDICTOR_SIZE. This patch clamps the size before memcpy to avoid buffer overflows.

Signed-off-by: 0xBat <monsterbat02@gmail.com>
---
 libavcodec/vvc/ctu.c | 2 ++
 1 file changed, 2 insertions(+)

diff --git a/libavcodec/vvc/ctu.c b/libavcodec/vvc/ctu.c
index 18cbe0fe0f..9d56abe8fd 100644
--- a/libavcodec/vvc/ctu.c
+++ b/libavcodec/vvc/ctu.c
@@ -1932,6 +1932,8 @@ static void palette_update_predictor(VVCLocalContext *lc, const bool local_dual_
             }
         }
 
+        if (i > VVC_MAX_NUM_PALETTE_PREDICTOR_SIZE)
+            i = VVC_MAX_NUM_PALETTE_PREDICTOR_SIZE;
         memcpy(pp->entries, plt->entries, i * sizeof(pp->entries[0]));
         pp->size = i;
     }
-- 
2.52.0.windows.1

_______________________________________________
ffmpeg-devel mailing list -- ffmpeg-devel@ffmpeg.org
To unsubscribe send an email to ffmpeg-devel-leave@ffmpeg.org

[FFmpeg-devel] [PATCH] avcodec/vvc: validate reference picture list indices

[0xBat via ffmpeg-devel]

The function derive_mmvd in libavcodec/vvc/ctu.c used reference indices without validating them against the RPL size. This patch adds checks to prevent out-of-bounds memory access.

Signed-off-by: 0xBat <monsterbat02@gmail.com>
---
 libavcodec/vvc/ctu.c | 2 ++
 1 file changed, 2 insertions(+)

diff --git a/libavcodec/vvc/ctu.c b/libavcodec/vvc/ctu.c
index 18cbe0fe0f..8e4f003886 100644
--- a/libavcodec/vvc/ctu.c
+++ b/libavcodec/vvc/ctu.c
@@ -1287,6 +1287,8 @@ static void derive_mmvd(const VVCLocalContext *lc, MvField *mvf, const Mv *mmvd_
     if (mvf->pred_flag == PF_BI) {
         const RefPicList *rpl = sc->rpl;
         const int poc = lc->fc->ps.ph.poc;
+        if (mvf->ref_idx[L0] >= rpl[L0].nb_refs || mvf->ref_idx[L1] >= rpl[L1].nb_refs)
+            return;
         const int diff[] = {
             poc - rpl[L0].refs[mvf->ref_idx[L0]].poc,
             poc - rpl[L1].refs[mvf->ref_idx[L1]].poc
-- 
2.52.0.windows.1

_______________________________________________
ffmpeg-devel mailing list -- ffmpeg-devel@ffmpeg.org
To unsubscribe send an email to ffmpeg-devel-leave@ffmpeg.org

[FFmpeg-devel] Re: [PATCH] avcodec/vvc: validate reference picture list indices

[Christophe Gisquet via ffmpeg-devel]

Hi,

sorry in advance for any bad formatting in my reply from a phone.

Le mar. 6 janv. 2026, 03:13, 0xBat via ffmpeg-devel <ffmpeg-devel@ffmpeg.org>
a écrit

> ---
>  libavcodec/vvc/ctu.c | 2 ++
>  1 file changed, 2 insertions(+)
>
> diff --git a/libavcodec/vvc/ctu.c b/libavcodec/vvc/ctu.c
> index 18cbe0fe0f..8e4f003886 100644
> --- a/libavcodec/vvc/ctu.c
> +++ b/libavcodec/vvc/ctu.c
> @@ -1287,6 +1287,8 @@ static void derive_mmvd(const VVCLocalContext *lc,
> MvField *mvf, const Mv *mmvd_
>      if (mvf->pred_flag == PF_BI) {
>          const RefPicList *rpl = sc->rpl;
>          const int poc = lc->fc->ps.ph.poc;
> +        if (mvf->ref_idx[L0] >= rpl[L0].nb_refs || mvf->ref_idx[L1] >=
> rpl[L1].nb_refs)
> +            return;
>          const int diff[] = {
>              poc - rpl[L0].refs[mvf->ref_idx[L0]].poc,
>

Not a maintainer or contributor to the VVC decoder, but sounds fishy. That
a ref_idx is invalid, means that wherever it came from didn't validate
against the RPL. I don't know where mvf comes from, but if that's from the
current frame, it is extremely weird.

And by that I mean, the issue sounds like it should be caught earlier to
avoid it from propagating to other places.

Regards,
Christophe

> Christophe
_______________________________________________
ffmpeg-devel mailing list -- ffmpeg-devel@ffmpeg.org
To unsubscribe send an email to ffmpeg-devel-leave@ffmpeg.org

[FFmpeg-devel] Re: [PATCH] avcodec/vvc: validate reference picture list indices

[Frank Plowman via ffmpeg-devel]

[-- Attachment #1.1.1.1: Type: text/plain, Size: 1857 bytes --]

On 06/01/2026 07:58, Christophe Gisquet via ffmpeg-devel wrote:
> Hi,
> 
> sorry in advance for any bad formatting in my reply from a phone.
> 
> Le mar. 6 janv. 2026, 03:13, 0xBat via ffmpeg-devel <ffmpeg-devel@ffmpeg.org>
> a écrit
> 
>> ---
>>  libavcodec/vvc/ctu.c | 2 ++
>>  1 file changed, 2 insertions(+)
>>
>> diff --git a/libavcodec/vvc/ctu.c b/libavcodec/vvc/ctu.c
>> index 18cbe0fe0f..8e4f003886 100644
>> --- a/libavcodec/vvc/ctu.c
>> +++ b/libavcodec/vvc/ctu.c
>> @@ -1287,6 +1287,8 @@ static void derive_mmvd(const VVCLocalContext *lc,
>> MvField *mvf, const Mv *mmvd_
>>      if (mvf->pred_flag == PF_BI) {
>>          const RefPicList *rpl = sc->rpl;
>>          const int poc = lc->fc->ps.ph.poc;
>> +        if (mvf->ref_idx[L0] >= rpl[L0].nb_refs || mvf->ref_idx[L1] >=
>> rpl[L1].nb_refs)
>> +            return;
>>          const int diff[] = {
>>              poc - rpl[L0].refs[mvf->ref_idx[L0]].poc,
>>
> 
> Not a maintainer or contributor to the VVC decoder, but sounds fishy. That
> a ref_idx is invalid, means that wherever it came from didn't validate
> against the RPL. I don't know where mvf comes from, but if that's from the
> current frame, it is extremely weird.
> 
> And by that I mean, the issue sounds like it should be caught earlier to
> avoid it from propagating to other places.
> 
> Regards,
> Christophe
> 

Hi,

Yes I agree with Christophe.  MMVD takes its "base MVs" (what is called
mvf here) from the regular merge list, which is shared with various
other merge modes.  Consequently, if there are invalid MVs in MMVD then
I suspect the issue may be more fundamental and there may be issues with
other merge modes.

I'm guessing you have fuzzed bitstreams which led you to these issues,
would it be possible to share them?

Thanks a lot for your work,
Frank

[-- Attachment #1.1.1.2: OpenPGP public key --]
[-- Type: application/pgp-keys, Size: 1091 bytes --]

[-- Attachment #1.2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 236 bytes --]

[-- Attachment #2: Type: text/plain, Size: 163 bytes --]

_______________________________________________
ffmpeg-devel mailing list -- ffmpeg-devel@ffmpeg.org
To unsubscribe send an email to ffmpeg-devel-leave@ffmpeg.org

[FFmpeg-devel] Re: [PATCH] avcodec/vvc: add boundary checks for CTU table access

[Christophe Gisquet via ffmpeg-devel]

Hi,

Le mar. 6 janv. 2026, 03:13, 0xBat via ffmpeg-devel <ffmpeg-devel@ffmpeg.org>
a écrit :

> In libavcodec/vvc/ctu.c, the functions set_tb_size and set_cb_pos lack
> proper boundary checks before performing memset operations. This patch adds
> validation against PPS dimensions to prevent heap overflows.
>
> Signed-off-by: 0xBat <monsterbat02@gmail.com>
> ---
>  libavcodec/vvc/ctu.c | 5 +++++
>  1 file changed, 5 insertions(+)
>
> diff --git a/libavcodec/vvc/ctu.c b/libavcodec/vvc/ctu.c
> index 18cbe0fe0f..ce8a7ab90e 100644
> --- a/libavcodec/vvc/ctu.c
> +++ b/libavcodec/vvc/ctu.c
> @@ -52,6 +52,9 @@ static void set_tb_size(const VVCFrameContext *fc, const
> TransformBlock *tb)
>
>      for (int y = y_tb; y < end; y++) {
>          const int off = y * fc->ps.pps->min_tu_width + x_tb;
> +        int max_off = fc->ps.pps->min_tu_width *
> fc->ps.pps->min_tu_height;
> +        if (off + width > max_off)
> +            return;
>          memset(fc->tab.tb_width [is_chroma] + off, tb->tb_width,  width);
>          memset(fc->tab.tb_height[is_chroma] + off, tb->tb_height, width);
>      }
> @@ -1185,6 +1188,8 @@ static void set_cb_pos(const VVCFrameContext *fc,
> const CodingUnit *cu)
>              fc->tab.cb_pos_x[ch_type][x + i] = cu->x0;
>              fc->tab.cb_pos_y[ch_type][x + i] = cu->y0;
>          }
> +        if (x + width > fc->ps.pps->min_tu_width *
> fc->ps.pps->min_tu_height)
> +            return;
>          memset(&fc->tab.cb_width[ch_type][x], cu->cb_width, width);
>          memset(&fc->tab.cb_height[ch_type][x], cu->cb_height, width);
>          memset(&fc->tab.cqt_depth[ch_type][x], cu->cqt_depth, width);
>

So lines that are processed are ok if overflowing. It sounds like width
should be clipped if that's an expected behaviour, or width is incorrect to
being with?

This is a question probably more for the maintainers/contributors.

Regards,
Christophe

>
_______________________________________________
ffmpeg-devel mailing list -- ffmpeg-devel@ffmpeg.org
To unsubscribe send an email to ffmpeg-devel-leave@ffmpeg.org

[FFmpeg-devel] Re: [PATCH] avcodec/vvc: add boundary checks for CTU table access

[Christophe Gisquet via ffmpeg-devel]

Le mar. 6 janv. 2026, 09:01, Christophe Gisquet <
christophe.gisquet@gmail.com> a écrit

> So lines that are processed are ok if overflowing. It sounds like width
>

I meant previous lines. Which can be an okish optimization sometimes but
maybe not here.

>
_______________________________________________
ffmpeg-devel mailing list -- ffmpeg-devel@ffmpeg.org
To unsubscribe send an email to ffmpeg-devel-leave@ffmpeg.org