From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org [79.124.17.100]) by master.gitmailbox.com (Postfix) with ESMTP id 6BC3040A2E for ; Sat, 5 Mar 2022 10:14:13 +0000 (UTC) Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id D17EB68B0DB; Sat, 5 Mar 2022 12:14:10 +0200 (EET) Received: from mail-ej1-f54.google.com (mail-ej1-f54.google.com [209.85.218.54]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 169D668B0B9 for ; Sat, 5 Mar 2022 12:14:04 +0200 (EET) Received: by mail-ej1-f54.google.com with SMTP id r13so22309845ejd.5 for ; Sat, 05 Mar 2022 02:14:04 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:subject:date:message-id:mime-version :content-transfer-encoding; bh=JKNd4Z6O9lRCTSLvCEwQfyIYW7QYMyJS39c2lKOs9aE=; b=htMMbEETZFCmU+TMlQTp2FgQOMVAF9iCSi+3dL/e+fyz9Y+oEPKDBbTw1an+2yMlq3 tOy+Wv2kk6cjlIlRlLiLCDyrNgznVCzzyaYf9hIQ+Gl7ySpkfnpckEpK6UWCtqzdSAEf IWOPVTpWEte/hrgxHvb7Tyxwlkj0skoiFBkp517iRlPzgM14yCSUETGK/AxTjN4XM3Gr nnWVi2tT63lcI8e3vhUXIsc+BrRYkeUFOPQuU6v7po4L4g5yR8pHf5s2mJd13bK1YjI6 Dv3ZtGyMNlbj1+6qGlWHd9i9i8SMjy6wBYeY+V8a0NL+eh4g2qavw4jy9b3FmJW7BaPy wLRQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:subject:date:message-id:mime-version :content-transfer-encoding; bh=JKNd4Z6O9lRCTSLvCEwQfyIYW7QYMyJS39c2lKOs9aE=; b=0orhzQUxhOz2GAHRdSd+UxJ6i0VpjPsEXDfTth0XNEjG0F7AWwAQ2NSW4/BMztC++Y Kh5VT9XX23hr8ED0qgr2a2hSNQ+2IVYncjRiGIc84h2H38iPG/gKeVXrDfq97VPiCw4A /BYy9OzoJU2RXjIN53iLx6lAh5Mq9qkJA9d5QHk2A1pKptSR+eUEk4sfQcqH1wndxLpC Tl9K64HM5G5ND8p8j+inQQ8/ERYwzTjx7+ZdiMaRddaMtiwGTxfXrxKQNIkWoXmjnhFm GlMU7L0P3Q3sW/QbxDzynGc4i8dUkF1UlwvzrBu0fk93JnjMBbb4SGPw/OhU+/E+8KeT HxMA== X-Gm-Message-State: AOAM531kviK5ynD61xt+bW2rB3p4wn7Nq2ZsogV+iIvoCyA6hXXPp2+g SLiLhUWww9857v+cDvY1EeHT2waC5lSYFw== X-Google-Smtp-Source: ABdhPJxzpLWhm2epEViI6Kz6fT6zvmBzLlIqz64+i+qi98J1zyalNicgSmI+AkYkNq4CEl0qyL79dA== X-Received: by 2002:a17:906:4ad9:b0:6cf:93f:f77e with SMTP id u25-20020a1709064ad900b006cf093ff77emr2280216ejt.558.1646475243517; Sat, 05 Mar 2022 02:14:03 -0800 (PST) Received: from localhost.localdomain ([95.168.118.13]) by smtp.gmail.com with ESMTPSA id d4-20020a056402000400b00412d60fee38sm3294853edu.11.2022.03.05.02.14.02 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sat, 05 Mar 2022 02:14:03 -0800 (PST) From: Paul B Mahol To: ffmpeg-devel@ffmpeg.org Date: Sat, 5 Mar 2022 11:15:44 +0100 Message-Id: <20220305101544.507071-1-onemda@gmail.com> X-Mailer: git-send-email 2.33.0 MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH] avcodec/dnxhdenc: retry increasing qscale to not overflow max_bits X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" Archived-At: List-Archive: List-Post: Increase mb_bits type from uint16_t to uint32_t to fix possible oveflows in bit size calculations. Update fate test that needs change. Signed-off-by: Paul B Mahol --- libavcodec/dnxhdenc.c | 8 +++++--- libavcodec/dnxhdenc.h | 2 +- tests/ref/lavf/mxf_opatom | 4 ++-- 3 files changed, 8 insertions(+), 6 deletions(-) diff --git a/libavcodec/dnxhdenc.c b/libavcodec/dnxhdenc.c index 374fec499b..9bdecec284 100644 --- a/libavcodec/dnxhdenc.c +++ b/libavcodec/dnxhdenc.c @@ -924,7 +924,7 @@ static void dnxhd_setup_threads_slices(DNXHDEncContext *ctx) unsigned mb = mb_y * ctx->m.mb_width + mb_x; ctx->slice_size[mb_y] += ctx->mb_bits[mb]; } - ctx->slice_size[mb_y] = (ctx->slice_size[mb_y] + 31) & ~31; + ctx->slice_size[mb_y] = (ctx->slice_size[mb_y] + 31U) & ~31U; ctx->slice_size[mb_y] >>= 3; thread_size = ctx->slice_size[mb_y]; offset += thread_size; @@ -1220,17 +1220,19 @@ static int dnxhd_encode_fast(AVCodecContext *avctx, DNXHDEncContext *ctx) avctx->execute2(avctx, dnxhd_mb_var_thread, NULL, NULL, ctx->m.mb_height); radix_sort(ctx->mb_cmp, ctx->mb_cmp_tmp, ctx->m.mb_num); +retry: for (x = 0; x < ctx->m.mb_num && max_bits > ctx->frame_bits; x++) { int mb = ctx->mb_cmp[x].mb; int rc = (ctx->qscale * ctx->m.mb_num ) + mb; max_bits -= ctx->mb_rc[rc].bits - ctx->mb_rc[rc + ctx->m.mb_num].bits; - ctx->mb_qscale[mb] = ctx->qscale + 1; + if (ctx->mb_qscale[mb] < 255) + ctx->mb_qscale[mb]++; ctx->mb_bits[mb] = ctx->mb_rc[rc + ctx->m.mb_num].bits; } if (max_bits > ctx->frame_bits) - return AVERROR(EINVAL); + goto retry; } return 0; } diff --git a/libavcodec/dnxhdenc.h b/libavcodec/dnxhdenc.h index 30ae8c15e3..7726a3915f 100644 --- a/libavcodec/dnxhdenc.h +++ b/libavcodec/dnxhdenc.h @@ -100,7 +100,7 @@ typedef struct DNXHDEncContext { unsigned qscale; unsigned lambda; - uint16_t *mb_bits; + uint32_t *mb_bits; uint8_t *mb_qscale; RCCMPEntry *mb_cmp; diff --git a/tests/ref/lavf/mxf_opatom b/tests/ref/lavf/mxf_opatom index e34cf2559e..359422ce9a 100644 --- a/tests/ref/lavf/mxf_opatom +++ b/tests/ref/lavf/mxf_opatom @@ -1,3 +1,3 @@ -aab6397829bd90f0c77a3f9fde53bb9c *tests/data/lavf/lavf.mxf_opatom +b1e32792b835ac51b0b1d5dcb9497e4e *tests/data/lavf/lavf.mxf_opatom 4717625 tests/data/lavf/lavf.mxf_opatom -tests/data/lavf/lavf.mxf_opatom CRC=0xf55aa22a +tests/data/lavf/lavf.mxf_opatom CRC=0xb13ba2f8 -- 2.33.0 _______________________________________________ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".