[FFmpeg-devel] [PATCH] lavc/vvc: Fix slice_idx out-of-bounds memset

[FFmpeg-devel] [PATCH] lavc/vvc: Fix slice_idx out-of-bounds memset

[post]

From: Frank Plowman <post@frankplowman.com>

If the number of CTUs reduces between one picture and the next, the
slice_idx table is reduced in size in the frame_context_for_each_tl call
on vvcdec.c:321.  When initialising the slice_idx table on vvcdec.c:325,
the old code uses fc->tab.sz.ctu_count when calculating the table size.
fc->tab.sz.ctu_count holds the old ctu count at this point however, not
being updated to hold the new ctu count until vvcdec.c:342.  This causes
an out-of-bounds write.

Patch fixes the problem by using pps->ctb_count, which was just used
when allocating the table, in place of fc->tab.sz.ctu_count when
initialising the table.

Signed-off-by: Frank Plowman <post@frankplowman.com>
---
 libavcodec/vvc/vvcdec.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/libavcodec/vvc/vvcdec.c b/libavcodec/vvc/vvcdec.c
index 1a050600eb..8163b5ecb6 100644
--- a/libavcodec/vvc/vvcdec.c
+++ b/libavcodec/vvc/vvcdec.c
@@ -322,7 +322,7 @@ static int pic_arrays_init(VVCContext *s, VVCFrameContext *fc)
     if (ret < 0)
         return ret;
 
-    memset(fc->tab.slice_idx, -1, sizeof(*fc->tab.slice_idx) * fc->tab.sz.ctu_count);
+    memset(fc->tab.slice_idx, -1, sizeof(*fc->tab.slice_idx) * ctu_count);
 
     if (fc->tab.sz.ctu_count != ctu_count) {
         ff_refstruct_pool_uninit(&fc->rpl_tab_pool);
-- 
2.43.0

_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] [PATCH] lavc/vvc: Fix slice_idx out-of-bounds memset

[Nuo Mi]

On Tue, Feb 6, 2024 at 2:00 AM <post@frankplowman.com> wrote:

> From: Frank Plowman <post@frankplowman.com>
>
> If the number of CTUs reduces between one picture and the next, the
> slice_idx table is reduced in size in the frame_context_for_each_tl call
> on vvcdec.c:321.  When initialising the slice_idx table on vvcdec.c:325,
> the old code uses fc->tab.sz.ctu_count when calculating the table size.
> fc->tab.sz.ctu_count holds the old ctu count at this point however, not
> being updated to hold the new ctu count until vvcdec.c:342.  This causes
> an out-of-bounds write.
>
> Patch fixes the problem by using pps->ctb_count, which was just used
> when allocating the table, in place of fc->tab.sz.ctu_count when
> initialising the table.
>
> Signed-off-by: Frank Plowman <post@frankplowman.com>
> ---
>  libavcodec/vvc/vvcdec.c | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
>
> diff --git a/libavcodec/vvc/vvcdec.c b/libavcodec/vvc/vvcdec.c
> index 1a050600eb..8163b5ecb6 100644
> --- a/libavcodec/vvc/vvcdec.c
> +++ b/libavcodec/vvc/vvcdec.c
> @@ -322,7 +322,7 @@ static int pic_arrays_init(VVCContext *s,
> VVCFrameContext *fc)
>      if (ret < 0)
>          return ret;
>
> -    memset(fc->tab.slice_idx, -1, sizeof(*fc->tab.slice_idx) *
> fc->tab.sz.ctu_count);
> +    memset(fc->tab.slice_idx, -1, sizeof(*fc->tab.slice_idx) * ctu_count);
>
applied,
thank you frank

>
>      if (fc->tab.sz.ctu_count != ctu_count) {
>          ff_refstruct_pool_uninit(&fc->rpl_tab_pool);
> --
> 2.43.0
>
> _______________________________________________
> ffmpeg-devel mailing list
> ffmpeg-devel@ffmpeg.org
> https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
>
> To unsubscribe, visit link above, or email
> ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".
>
_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".