From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org [79.124.17.100]) by master.gitmailbox.com (Postfix) with ESMTP id A46ED46DAA for ; Thu, 10 Aug 2023 23:49:15 +0000 (UTC) Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 44EF768C823; Fri, 11 Aug 2023 02:49:12 +0300 (EEST) Received: from mail-pj1-f53.google.com (mail-pj1-f53.google.com [209.85.216.53]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 7A49668C789 for ; Fri, 11 Aug 2023 02:49:05 +0300 (EEST) Received: by mail-pj1-f53.google.com with SMTP id 98e67ed59e1d1-2682e33509bso1001921a91.1 for ; Thu, 10 Aug 2023 16:49:05 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1691711343; x=1692316143; h=content-transfer-encoding:mime-version:message-id:date:subject:cc :to:from:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=FGAgROnEX913BGP+cFkbGKCNJq4SCDfwWwmWPkbK6X8=; b=M1TReZwGtHnXtWDd6zsXZNtludit+a73RIOKGpf8oyEOOdpZvRbm/l/P+O6Y7Qz3JL PC/6SeE2G3qDewT+KFO5+XpCl+6akc0IqxdXcKkZa3lOTg0d0G8OvlbGZv1QhCMvIN3J ei3xgyn29aSGrzC8e33IzEQ74z3xZSGqVwLZgmZxMLmsp/QTRJB4rr2aRAhecTb6HjVl xRwtMuu8NcafcGlF2brBR9nVgYLgNhQx813JlpdnExz2CEdU2ZHa9w2kXOKYatR2cHSz mS0B3YT3iT7jGq/OzdwVPvX4v/uOb/QAmQQ6V1auvz29dR4sEDvtSDMIc1mURllPDLEk 1XKA== X-Gm-Message-State: AOJu0YxA1GC+Wucsqd4VKRQU5ooZbtB6pzWlqeCgS+hYIIzeRGXPUn7C 0vPbxc+5lLNnXxNETVzL1FVQaYbxCv4= X-Google-Smtp-Source: AGHT+IF3prqH3nC/2eD9DY82XeKNLvI2IgG9xIbf3NQRuwioM3nuSw9irtalKP7wcdVlOZHq7BHvuA== X-Received: by 2002:a17:90b:1884:b0:263:311f:9bcc with SMTP id mn4-20020a17090b188400b00263311f9bccmr93614pjb.35.1691711342544; Thu, 10 Aug 2023 16:49:02 -0700 (PDT) Received: from localhost (76-14-89-2.sf-cable.astound.net. [76.14.89.2]) by smtp.gmail.com with ESMTPSA id p14-20020a17090adf8e00b0026971450601sm2041721pjv.7.2023.08.10.16.49.00 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 10 Aug 2023 16:49:01 -0700 (PDT) Received: by localhost (sSMTP sendmail emulation); Thu, 10 Aug 2023 16:48:58 -0700 From: pal@sandflow.com To: ffmpeg-devel@ffmpeg.org Date: Thu, 10 Aug 2023 16:48:56 -0700 Message-Id: <20230810234856.2636-1-pal@sandflow.com> X-Mailer: git-send-email 2.25.1 MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH v1] avcodec/jpeg2000htdec: check if block decoding will exceed internal precision X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Pierre-Anthony Lemieux Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" Archived-At: List-Archive: List-Post: From: Pierre-Anthony Lemieux Intended to replace https://patchwork.ffmpeg.org/project/ffmpeg/patch/20230802000135.26482-3-michael@niedermayer.cc/ with a more accurate block decoding magnitude bound. --- libavcodec/jpeg2000htdec.c | 32 +++++++++++++++++++++++++++++++- 1 file changed, 31 insertions(+), 1 deletion(-) diff --git a/libavcodec/jpeg2000htdec.c b/libavcodec/jpeg2000htdec.c index 2c4cea5dd9..713dba17e3 100644 --- a/libavcodec/jpeg2000htdec.c +++ b/libavcodec/jpeg2000htdec.c @@ -572,6 +572,14 @@ static int jpeg2000_decode_ht_cleanup_segment(const Jpeg2000DecoderContext *s, const uint8_t *vlc_buf = Dcup + Pcup; + /* + * Bound on the recision needed to process the codeblock. The number of + * decoded bit planes is equal to at most cblk->zbp + 2 since S_blk = P if + * there are no placeholder passes or HT Sets and P = cblk->zbp. See Rec. + * ITU-T T.814, 7.6. + */ + int maxbp = cblk->zbp + 2; + /* convert to raster-scan */ const uint16_t is_border_x = width % 2; const uint16_t is_border_y = height % 2; @@ -590,6 +598,10 @@ static int jpeg2000_decode_ht_cleanup_segment(const Jpeg2000DecoderContext *s, goto free; } + /* do we have enough precision, assuming a 32-bit decoding path */ + if (maxbp >= 32) + return AVERROR_INVALIDDATA; + sigma = sigma_n; mu = mu_n; @@ -676,6 +688,10 @@ static int jpeg2000_decode_ht_cleanup_segment(const Jpeg2000DecoderContext *s, } U[J2K_Q1] = kappa[J2K_Q1] + u[J2K_Q1]; U[J2K_Q2] = kappa[J2K_Q2] + u[J2K_Q2]; + if (U[J2K_Q1] > maxbp || U[J2K_Q2] > maxbp) { + ret = AVERROR_INVALIDDATA; + goto free; + } for (int i = 0; i < 4; i++) { m[J2K_Q1][i] = sigma_n[4 * q1 + i] * U[J2K_Q1] - ((emb_pat_k[J2K_Q1] >> i) & 1); @@ -713,6 +729,10 @@ static int jpeg2000_decode_ht_cleanup_segment(const Jpeg2000DecoderContext *s, } U[J2K_Q1] = kappa[J2K_Q1] + u[J2K_Q1]; + if (U[J2K_Q1] > maxbp) { + ret = AVERROR_INVALIDDATA; + goto free; + } for (int i = 0; i < 4; i++) m[J2K_Q1][i] = sigma_n[4 * q1 + i] * U[J2K_Q1] - ((emb_pat_k[J2K_Q1] >> i) & 1); @@ -842,6 +862,10 @@ static int jpeg2000_decode_ht_cleanup_segment(const Jpeg2000DecoderContext *s, U[J2K_Q1] = kappa[J2K_Q1] + u[J2K_Q1]; U[J2K_Q2] = kappa[J2K_Q2] + u[J2K_Q2]; + if (U[J2K_Q1] > maxbp || U[J2K_Q2] > maxbp) { + ret = AVERROR_INVALIDDATA; + goto free; + } for (int i = 0; i < 4; i++) { m[J2K_Q1][i] = sigma_n[4 * q1 + i] * U[J2K_Q1] - ((emb_pat_k[J2K_Q1] >> i) & 1); @@ -910,6 +934,10 @@ static int jpeg2000_decode_ht_cleanup_segment(const Jpeg2000DecoderContext *s, kappa[J2K_Q1] = FFMAX(1, gamma[J2K_Q1] * (max_e[J2K_Q1] - 1)); U[J2K_Q1] = kappa[J2K_Q1] + u[J2K_Q1]; + if (U[J2K_Q1] > maxbp) { + ret = AVERROR_INVALIDDATA; + goto free; + } for (int i = 0; i < 4; i++) m[J2K_Q1][i] = sigma_n[4 * q1 + i] * U[J2K_Q1] - ((emb_pat_k[J2K_Q1] >> i) & 1); @@ -1238,8 +1266,10 @@ ff_jpeg2000_decode_htj2k(const Jpeg2000DecoderContext *s, Jpeg2000CodingStyle *c } if ((ret = jpeg2000_decode_ht_cleanup_segment(s, cblk, t1, &mel_state, &mel, &vlc, &mag_sgn, Dcup, Lcup, Pcup, pLSB, width, - height, sample_buf, block_states)) < 0) + height, sample_buf, block_states)) < 0) { + av_log(s->avctx, AV_LOG_ERROR, "Bad HT cleanup segment\n"); goto free; + } if (cblk->npasses > 1) jpeg2000_decode_sigprop_segment(cblk, width, height, Dref, Lref, -- 2.25.1 _______________________________________________ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".