[FFmpeg-devel] [PATCH 1/2] avcodec/dvbsubenc: Sanity check num_rects

* [FFmpeg-devel] [PATCH 1/2] avcodec/dvbsubenc: Sanity check num_rects
@ 2025-04-16  9:17 Andreas Rheinhardt
  2025-04-17  0:14 ` Michael Niedermayer
  0 siblings, 1 reply; 4+ messages in thread
From: Andreas Rheinhardt @ 2025-04-16  9:17 UTC (permalink / raw)
  To: FFmpeg development discussions and patches

[-- Attachment #1: Type: text/plain, Size: 29 bytes --]

Patches attached.

- Andreas

[-- Attachment #2: 0001-avcodec-dvbsubenc-Sanity-check-num_rects.patch --]
[-- Type: text/x-patch, Size: 1180 bytes --]

From b3a4381000d13ac344114dd5f0230c9eae7b32aa Mon Sep 17 00:00:00 2001
From: Andreas Rheinhardt <andreas.rheinhardt@outlook.com>
Date: Wed, 16 Apr 2025 10:23:01 +0200
Subject: [PATCH 1/2] avcodec/dvbsubenc: Sanity check num_rects
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

It is written as region_id which is a single byte.
Also fixes a potential (defined) overflow in the num_rects * 6
multiplication later; this has been found by 김승호 <kimsho98@naver.com>.

Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@outlook.com>
---
 libavcodec/dvbsubenc.c | 3 +++
 1 file changed, 3 insertions(+)

diff --git a/libavcodec/dvbsubenc.c b/libavcodec/dvbsubenc.c
index 822e3a5309..8eeac76855 100644
--- a/libavcodec/dvbsubenc.c
+++ b/libavcodec/dvbsubenc.c
@@ -284,6 +284,9 @@ static int dvbsub_encode(AVCodecContext *avctx, uint8_t *outbuf, int buf_size,
     if (h->num_rects && !h->rects)
         return AVERROR(EINVAL);
 
+    if (h->num_rects >= 256)
+        return AVERROR_INVALIDDATA;
+
     if (avctx->width > 0 && avctx->height > 0) {
         if (buf_size < 11)
             return AVERROR_BUFFER_TOO_SMALL;
-- 
2.45.2


[-- Attachment #3: 0002-avcodec-dvbsubenc-Check-nb_colors-before-using-it.patch --]
[-- Type: text/x-patch, Size: 1921 bytes --]

From c150cc5f4ae4961a56d035b1fdad7c2eb0eccfad Mon Sep 17 00:00:00 2001
From: Andreas Rheinhardt <andreas.rheinhardt@outlook.com>
Date: Wed, 16 Apr 2025 11:03:53 +0200
Subject: [PATCH 2/2] avcodec/dvbsubenc: Check nb_colors before using it

Avoids a potential overflow when multiplying nb_colors by 6.
Also make the nb_colors check a bit more strict.

Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@outlook.com>
---
 libavcodec/dvbsubenc.c | 11 +++++------
 1 file changed, 5 insertions(+), 6 deletions(-)

diff --git a/libavcodec/dvbsubenc.c b/libavcodec/dvbsubenc.c
index 8eeac76855..3659e3f9ed 100644
--- a/libavcodec/dvbsubenc.c
+++ b/libavcodec/dvbsubenc.c
@@ -329,24 +329,23 @@ static int dvbsub_encode(AVCodecContext *avctx, uint8_t *outbuf, int buf_size,
 
     if (h->num_rects) {
         for (clut_id = 0; clut_id < h->num_rects; clut_id++) {
-            if (buf_size < 6 + h->rects[clut_id]->nb_colors * 6)
-                return AVERROR_BUFFER_TOO_SMALL;
-
             /* CLUT segment */
 
-            if (h->rects[clut_id]->nb_colors <= 4) {
+            if (h->rects[clut_id]->nb_colors <= 4U) {
                 /* 2 bpp, some decoders do not support it correctly */
                 bpp_index = 0;
-            } else if (h->rects[clut_id]->nb_colors <= 16) {
+            } else if (h->rects[clut_id]->nb_colors <= 16U) {
                 /* 4 bpp, standard encoding */
                 bpp_index = 1;
-            } else if (h->rects[clut_id]->nb_colors <= 256) {
+            } else if (h->rects[clut_id]->nb_colors <= 256U) {
                 /* 8 bpp, standard encoding */
                 bpp_index = 2;
             } else {
                 return AVERROR(EINVAL);
             }
 
+            if (buf_size < 6 + h->rects[clut_id]->nb_colors * 6)
+                return AVERROR_BUFFER_TOO_SMALL;
 
             /* CLUT segment */
             *q++ = 0x0f; /* sync byte */
-- 
2.45.2


[-- Attachment #4: Type: text/plain, Size: 251 bytes --]

_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".

^ permalink raw reply	[flat|nested] 4+ messages in thread