From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org [79.124.17.100]) by master.gitmailbox.com (Postfix) with ESMTP id D1A5447B9B for ; Tue, 3 Oct 2023 15:00:48 +0000 (UTC) Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 0BC1D68CCA6; Tue, 3 Oct 2023 18:00:46 +0300 (EEST) Received: from mail-qv1-f42.google.com (mail-qv1-f42.google.com [209.85.219.42]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 9EDA568CAC2 for ; Tue, 3 Oct 2023 18:00:39 +0300 (EEST) Received: by mail-qv1-f42.google.com with SMTP id 6a1803df08f44-65afc3310e3so1682676d6.1 for ; Tue, 03 Oct 2023 08:00:39 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1696345238; x=1696950038; darn=ffmpeg.org; h=content-transfer-encoding:mime-version:message-id:date:subject:cc :to:from:from:to:cc:subject:date:message-id:reply-to; bh=4nS9AiRu9XVghU/nfNYwmYpOuu0frvX0kSxSjrKjGU0=; b=VwdVWOPb5344RprMEER7/sJL18hgxhFjlLO+r1kCMHtVJVfHScsDRBueei207/Vlfn eTbsunAtSHP9sDV0zE3dh+OGQRK68SXJCZXDG304GluK5KpVXv2bkGEjPldbSPciAlkL YHt9Fft5wBiInWWxN1QOGlO4+C1SjSEbPR3SVKNtSQPn8b/01xeUkGOQNPEVJg6QbkCe CspwAayehgzuV3hvnSQsVLDeOtVnlBvmVc3+6QEB0xmtGx3L86yuJyHt5n/GJcsLJl+K kGnXPsEp6ufgcY3knh0q44b7zjoR1Pf7DIYRjMVOIvV4opf6nGN7ZtxZvRgEB6mz0cuJ ty4Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1696345238; x=1696950038; h=content-transfer-encoding:mime-version:message-id:date:subject:cc :to:from:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=4nS9AiRu9XVghU/nfNYwmYpOuu0frvX0kSxSjrKjGU0=; b=hiPmEGPpRZR8KK/cGs9QqurJNMX1SY4JYQYWWv/Tt5gULsa12b9wNAugz/HJqR2XB5 13MAI2gmmrWhtRzMQj9sXezVN6uhibwYZpjDn4jzdc5ZBpqZopeOVoH66NOoENlHLmFJ tgwrlgT6aQCZPg2yjAEwoS3hZNSN5WHtIa+vevZDEy4Fbcw6nDdpqPuq7vhi/FsV/fo5 qeMrfb1YvvCpB2YyHnJpwmlQQ+lKA9b4Q646910T49F0qtxxjoI3PqJGjLGS8YjyA+HV uQ9HX5N2NoTZyZHjMbijEVxqw2fPZYg+LZMk2XgtsiZdyib6Qh1Kz1UoWcgRvGEdLcxO JbYg== X-Gm-Message-State: AOJu0YzO4QYD9RiIGvde+twxOHNolL0YF0S8PlAaXeRtrg3CULI4Ryct CQqP7Yhzq6SZPIojto4Dbl+tQcK3SwFSUw== X-Google-Smtp-Source: AGHT+IFqVn/BB1TvCUuTtTibitAasHqc9W8ydgbimJy4Wah0aCMDFligS/u14nlPRdAD788wRscKYg== X-Received: by 2002:a05:6214:5005:b0:651:675b:37f9 with SMTP id jo5-20020a056214500500b00651675b37f9mr16725939qvb.1.1696345237963; Tue, 03 Oct 2023 08:00:37 -0700 (PDT) Received: from gauss.local (c-68-56-149-176.hsd1.mi.comcast.net. [68.56.149.176]) by smtp.gmail.com with ESMTPSA id e8-20020a0ce3c8000000b0065b27fe4737sm542102qvl.15.2023.10.03.08.00.36 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 03 Oct 2023 08:00:37 -0700 (PDT) From: Leo Izen To: ffmpeg-devel@ffmpeg.org Date: Tue, 3 Oct 2023 11:00:35 -0400 Message-ID: <20231003150035.176199-1-leo.izen@gmail.com> X-Mailer: git-send-email 2.42.0 MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH v2] avcodec/jpegxl_parser: fix various memory issues X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Michael Niedermayer , Leo Izen Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" Archived-At: List-Archive: List-Post: The spec caps the prefix alphabet size to 32768 (i.e. 1 << 15) so we should check for that and reject alphabets that are too large, in order to prevent over-allocating. Additionally, there's no need to allocate buffers that are as large as the maximum alphabet size as these aren't stack-allocated, they're heap allocated and thus can be variable size. Added an overflow check as well, which fixes leaking the buffer, and capping the alphabet size fixes two potential overruns as well. Fixes: out of array access Fixes: 62089/clusterfuzz-testcase-minimized-ffmpeg_DEMUXER_fuzzer- 5437089094959104.fuzz Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg Found-by: Hardik Shah of Vehere (Dawn Treaders team) Co-authored-by: Michael Niedermayer Signed-off-by: Leo Izen --- libavcodec/jpegxl_parser.c | 23 +++++++++++++++++------ 1 file changed, 17 insertions(+), 6 deletions(-) diff --git a/libavcodec/jpegxl_parser.c b/libavcodec/jpegxl_parser.c index d25a1b6e1d..2405c1d5e5 100644 --- a/libavcodec/jpegxl_parser.c +++ b/libavcodec/jpegxl_parser.c @@ -46,6 +46,8 @@ #define JXL_FLAG_USE_LF_FRAME 32 #define JXL_FLAG_SKIP_ADAPTIVE_LF_SMOOTH 128 +#define MAX_PREFIX_ALPHABET_SIZE (1u << 15) + #define clog1p(x) (ff_log2(x) + !!(x)) #define unpack_signed(x) (((x) & 1 ? -(x)-1 : (x))/2) #define div_ceil(x, y) (((x) - 1) / (y) + 1) @@ -724,16 +726,17 @@ static int read_vlc_prefix(GetBitContext *gb, JXLEntropyDecoder *dec, JXLSymbolD if (ret < 0) goto end; - buf = av_calloc(1, 262148); // 32768 * 8 + 4 + buf = av_mallocz(dist->alphabet_size * (2 * sizeof(int8_t) + sizeof(int16_t) + sizeof(uint32_t)) + + sizeof(uint32_t)); if (!buf) { ret = AVERROR(ENOMEM); goto end; } level2_lens = (int8_t *)buf; - level2_lens_s = (int8_t *)(buf + 32768); - level2_syms = (int16_t *)(buf + 65536); - level2_codecounts = (uint32_t *)(buf + 131072); + level2_lens_s = (int8_t *)(buf + dist->alphabet_size * sizeof(int8_t)); + level2_syms = (int16_t *)(buf + dist->alphabet_size * (2 * sizeof(int8_t))); + level2_codecounts = (uint32_t *)(buf + dist->alphabet_size * (2 * sizeof(int8_t) + sizeof(int16_t))); total_code = 0; for (int i = 0; i < dist->alphabet_size; i++) { @@ -742,6 +745,10 @@ static int read_vlc_prefix(GetBitContext *gb, JXLEntropyDecoder *dec, JXLSymbolD int extra = 3 + get_bits(gb, 2); if (repeat_count_prev) extra = 4 * (repeat_count_prev - 2) - repeat_count_prev + extra; + if (i + extra > dist->alphabet_size) { + ret = AVERROR_INVALIDDATA; + goto end; + } for (int j = 0; j < extra; j++) level2_lens[i + j] = prev; total_code += (32768 >> prev) * extra; @@ -772,8 +779,10 @@ static int read_vlc_prefix(GetBitContext *gb, JXLEntropyDecoder *dec, JXLSymbolD } } - if (total_code != 32768 && level2_codecounts[0] < dist->alphabet_size - 1) - return AVERROR_INVALIDDATA; + if (total_code != 32768 && level2_codecounts[0] < dist->alphabet_size - 1) { + ret = AVERROR_INVALIDDATA; + goto end; + } for (int i = 1; i < dist->alphabet_size + 1; i++) level2_codecounts[i] += level2_codecounts[i - 1]; @@ -848,6 +857,8 @@ static int read_distribution_bundle(GetBitContext *gb, JXLEntropyDecoder *dec, if (get_bits1(gb)) { int n = get_bits(gb, 4); dist->alphabet_size = 1 + (1 << n) + get_bitsz(gb, n); + if (dist->alphabet_size > MAX_PREFIX_ALPHABET_SIZE) + return AVERROR_INVALIDDATA; } else { dist->alphabet_size = 1; } -- 2.42.0 _______________________________________________ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".