From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org [79.124.17.100]) by master.gitmailbox.com (Postfix) with ESMTP id 2B80947B42 for ; Mon, 2 Oct 2023 20:25:37 +0000 (UTC) Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 1E24568CB2D; Mon, 2 Oct 2023 23:25:35 +0300 (EEST) Received: from mail-qk1-f181.google.com (mail-qk1-f181.google.com [209.85.222.181]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id D7C2768CAAA for ; Mon, 2 Oct 2023 23:25:27 +0300 (EEST) Received: by mail-qk1-f181.google.com with SMTP id af79cd13be357-77406094bc3so2420185a.1 for ; Mon, 02 Oct 2023 13:25:27 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1696278326; x=1696883126; darn=ffmpeg.org; h=content-transfer-encoding:mime-version:message-id:date:subject:cc :to:from:from:to:cc:subject:date:message-id:reply-to; bh=TFNshG8BbCdPWdHCFa8BenWDOHnck80H23DJ8DuwqLE=; b=TsLGBEmFelk5a5iR1mSBw6iZOCoIu2z03DxbirSU8ed17NgeFNUE7prYnpF3qHdRfa s37r95x8cuwXozPp0Moqaihe6HVqE48YlE1xBNoT89UrSSfMhTzQOhlvWM+kXlx2XmJJ TpxW/PVsq9nEQQVKiWYIJ32niuk44zO1XVf9VFlIuYdGWcXKvHJgAtk+C0QIgtpzBVxE Stz9ZUqeW+b9GGqSySQb7dHQA8xq+F0RU+yS4goIzQvDuvhiAMgLw2ch3ckEjb3yR8Ql oIwuNdcp/EY1j0+qTGQtPempcscTm8PyufbF3gx1ioE0CG1tElXf7omDfzne46VYkeWR s0xg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1696278326; x=1696883126; h=content-transfer-encoding:mime-version:message-id:date:subject:cc :to:from:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=TFNshG8BbCdPWdHCFa8BenWDOHnck80H23DJ8DuwqLE=; b=wUbnmS6Nju4LnTZO1M3fVeHTbmpXmJtEYk0c28jDWzxeXCHTHnkJOPSuPygXoq4bgu kN/CnWo2XBcrvO6HpULsd0pPUdZbE8B/KxMD+ycX2NBM67Q5RIzKo0Qqd6OMHb/1ZYaZ 2z7KeRNorNrXS4zsA77B86+GUV/WU1gPo1bONbWNER6ZyB0KmK59ANxpU78ihsdJCMXp kFbYyTatv9rOnl6zbjoZtbh1wPmaAHk897n75XW40xtZ4IPWaiYKanc27KWzA9amFzn0 qmsgmr3Dfr0pqLAhlDiWie5n2rjW0yIhqLIhmbNVp7Vno/umaGz2zY4kPYZbc6IZCOMD NaCg== X-Gm-Message-State: AOJu0Yyzc2YFtTw0P0Tj93jBSoZOjkK0fkbPeFy6223M0Zz5Tm8SJEJM ukTlU4JDlYxeLNcm9tOIQJsK/mcvXFradA== X-Google-Smtp-Source: AGHT+IFKFRGJwixT2yQyiiYTvX9/kdtdyX5TGbvqLnG+kl46t6MNq98PQbJPXfOF7icU9Zb5Z+jj8A== X-Received: by 2002:a05:620a:1a1f:b0:775:7921:732e with SMTP id bk31-20020a05620a1a1f00b007757921732emr13330799qkb.3.1696278326374; Mon, 02 Oct 2023 13:25:26 -0700 (PDT) Received: from gauss.local (c-68-56-149-176.hsd1.mi.comcast.net. [68.56.149.176]) by smtp.gmail.com with ESMTPSA id x18-20020a05620a099200b00767da10efb6sm9020145qkx.97.2023.10.02.13.25.25 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 02 Oct 2023 13:25:25 -0700 (PDT) From: Leo Izen To: ffmpeg-devel@ffmpeg.org Date: Mon, 2 Oct 2023 16:25:23 -0400 Message-ID: <20231002202523.148560-1-leo.izen@gmail.com> X-Mailer: git-send-email 2.42.0 MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH] avcodec/jpegxl_parser: fix various memory issues X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Michael Niedermayer , Leo Izen Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" Archived-At: List-Archive: List-Post: The spec caps the prefix alphabet size to 32768 (i.e. 1 << 15) so we need to check for that and reject alphabets that are too large. Additionally, there's no need to allocate buffers that are as large as the maximum alphabet size as these aren't stack-allocated, they're heap allocated and thus can be variable size. Added an overflow check as well, which fixes leaking the buffer, and capping the alphabet size fixes two potential overruns as well. Fixes: out of array access Fixes: 62089/clusterfuzz-testcase-minimized-ffmpeg_DEMUXER_fuzzer- 5437089094959104.fuzz Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg Found-by: Hardik Shah of Vehere (Dawn Treaders team) Co-authored-by: Michael Niedermayer Signed-off-by: Leo Izen --- libavcodec/jpegxl_parser.c | 23 +++++++++++++++++------ 1 file changed, 17 insertions(+), 6 deletions(-) diff --git a/libavcodec/jpegxl_parser.c b/libavcodec/jpegxl_parser.c index d25a1b6e1d..51af0f4ed1 100644 --- a/libavcodec/jpegxl_parser.c +++ b/libavcodec/jpegxl_parser.c @@ -46,6 +46,8 @@ #define JXL_FLAG_USE_LF_FRAME 32 #define JXL_FLAG_SKIP_ADAPTIVE_LF_SMOOTH 128 +#define MAX_PREFIX_ALPHABET_SIZE (1u << 15) + #define clog1p(x) (ff_log2(x) + !!(x)) #define unpack_signed(x) (((x) & 1 ? -(x)-1 : (x))/2) #define div_ceil(x, y) (((x) - 1) / (y) + 1) @@ -724,16 +726,17 @@ static int read_vlc_prefix(GetBitContext *gb, JXLEntropyDecoder *dec, JXLSymbolD if (ret < 0) goto end; - buf = av_calloc(1, 262148); // 32768 * 8 + 4 + buf = av_calloc(1, dist->alphabet_size * (2 * sizeof(int8_t) + sizeof(int16_t) + sizeof(uint32_t)) + + sizeof(uint32_t)); if (!buf) { ret = AVERROR(ENOMEM); goto end; } level2_lens = (int8_t *)buf; - level2_lens_s = (int8_t *)(buf + 32768); - level2_syms = (int16_t *)(buf + 65536); - level2_codecounts = (uint32_t *)(buf + 131072); + level2_lens_s = (int8_t *)(buf + dist->alphabet_size * sizeof(int8_t)); + level2_syms = (int16_t *)(buf + dist->alphabet_size * (2 * sizeof(int8_t))); + level2_codecounts = (uint32_t *)(buf + dist->alphabet_size * (2 * sizeof(int8_t) + sizeof(int16_t))); total_code = 0; for (int i = 0; i < dist->alphabet_size; i++) { @@ -742,6 +745,10 @@ static int read_vlc_prefix(GetBitContext *gb, JXLEntropyDecoder *dec, JXLSymbolD int extra = 3 + get_bits(gb, 2); if (repeat_count_prev) extra = 4 * (repeat_count_prev - 2) - repeat_count_prev + extra; + if (i + extra > dist->alphabet_size) { + ret = AVERROR_INVALIDDATA; + goto end; + } for (int j = 0; j < extra; j++) level2_lens[i + j] = prev; total_code += (32768 >> prev) * extra; @@ -772,8 +779,10 @@ static int read_vlc_prefix(GetBitContext *gb, JXLEntropyDecoder *dec, JXLSymbolD } } - if (total_code != 32768 && level2_codecounts[0] < dist->alphabet_size - 1) - return AVERROR_INVALIDDATA; + if (total_code != 32768 && level2_codecounts[0] < dist->alphabet_size - 1) { + ret = AVERROR_INVALIDDATA; + goto end; + } for (int i = 1; i < dist->alphabet_size + 1; i++) level2_codecounts[i] += level2_codecounts[i - 1]; @@ -848,6 +857,8 @@ static int read_distribution_bundle(GetBitContext *gb, JXLEntropyDecoder *dec, if (get_bits1(gb)) { int n = get_bits(gb, 4); dist->alphabet_size = 1 + (1 << n) + get_bitsz(gb, n); + if (dist->alphabet_size > MAX_PREFIX_ALPHABET_SIZE) + return AVERROR_INVALIDDATA; } else { dist->alphabet_size = 1; } -- 2.42.0 _______________________________________________ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".