[FFmpeg-devel] [PATCH] lavc/vvc: Increase IntraEdgeParams buffer size

[FFmpeg-devel] [PATCH] lavc/vvc: Increase IntraEdgeParams buffer size

[Frank Plowman]

The reference line buffers are used with indices in the range
-MAX_TB_SIZE - 3 to refw + FFMAX(1, w/h) * ref_idx + 1, which is
at most 5*MAX_TB_SIZE + 1.

Fixes buffer overflows.
http://fate.ffmpeg.org/report.cgi?slot=armv7-linux-gcc-9&time=20240124051736
---
 libavcodec/vvc/vvcdsp.c | 8 ++++----
 1 file changed, 4 insertions(+), 4 deletions(-)

diff --git a/libavcodec/vvc/vvcdsp.c b/libavcodec/vvc/vvcdsp.c
index c82ea7be30..56e71d5163 100644
--- a/libavcodec/vvc/vvcdsp.c
+++ b/libavcodec/vvc/vvcdsp.c
@@ -87,10 +87,10 @@ typedef struct IntraEdgeParams {
     uint8_t* left;
     int filter_flag;
 
-    uint16_t left_array[3 * MAX_TB_SIZE + 3];
-    uint16_t filtered_left_array[3 * MAX_TB_SIZE + 3];
-    uint16_t top_array[3 * MAX_TB_SIZE + 3];
-    uint16_t filtered_top_array[3 * MAX_TB_SIZE + 3];
+    uint16_t left_array[6 * MAX_TB_SIZE + 5];
+    uint16_t filtered_left_array[6 * MAX_TB_SIZE + 5];
+    uint16_t top_array[6 * MAX_TB_SIZE + 5];
+    uint16_t filtered_top_array[6 * MAX_TB_SIZE + 5];
 } IntraEdgeParams;
 
 #define PROF_BORDER_EXT         1
-- 
2.43.0

_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] [PATCH] lavc/vvc: Increase IntraEdgeParams buffer size

[James Almer]

On 1/29/2024 8:22 PM, Frank Plowman wrote:
> The reference line buffers are used with indices in the range
> -MAX_TB_SIZE - 3 to refw + FFMAX(1, w/h) * ref_idx + 1, which is
> at most 5*MAX_TB_SIZE + 1.
> 
> Fixes buffer overflows.
> http://fate.ffmpeg.org/report.cgi?slot=armv7-linux-gcc-9&time=20240124051736
> ---
>   libavcodec/vvc/vvcdsp.c | 8 ++++----
>   1 file changed, 4 insertions(+), 4 deletions(-)
> 
> diff --git a/libavcodec/vvc/vvcdsp.c b/libavcodec/vvc/vvcdsp.c
> index c82ea7be30..56e71d5163 100644
> --- a/libavcodec/vvc/vvcdsp.c
> +++ b/libavcodec/vvc/vvcdsp.c
> @@ -87,10 +87,10 @@ typedef struct IntraEdgeParams {
>       uint8_t* left;
>       int filter_flag;
>   
> -    uint16_t left_array[3 * MAX_TB_SIZE + 3];
> -    uint16_t filtered_left_array[3 * MAX_TB_SIZE + 3];
> -    uint16_t top_array[3 * MAX_TB_SIZE + 3];
> -    uint16_t filtered_top_array[3 * MAX_TB_SIZE + 3];
> +    uint16_t left_array[6 * MAX_TB_SIZE + 5];
> +    uint16_t filtered_left_array[6 * MAX_TB_SIZE + 5];
> +    uint16_t top_array[6 * MAX_TB_SIZE + 5];
> +    uint16_t filtered_top_array[6 * MAX_TB_SIZE + 5];
>   } IntraEdgeParams;
>   
>   #define PROF_BORDER_EXT         1

Applied, thanks.
_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".