From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org [79.124.17.100]) by master.gitmailbox.com (Postfix) with ESMTP id 1006449B68 for ; Tue, 2 Apr 2024 21:49:03 +0000 (UTC) Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 4218F68CFD1; Wed, 3 Apr 2024 00:49:01 +0300 (EEST) Received: from wrqvqshf.outbound-mail.sendgrid.net (wrqvqshf.outbound-mail.sendgrid.net [149.72.70.15]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 75AD368CD83 for ; Wed, 3 Apr 2024 00:48:54 +0300 (EEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=frankplowman.com; h=from:subject:mime-version:to:cc:content-transfer-encoding: content-type:cc:content-type:from:subject:to; s=s1; bh=L0MQ2dPlV34/EvOvr29Z5eXHdEp8niQd/yDY09ravjE=; b=s7q+VWw/WSo9tqmbBdiirMi+s6JjUemi7OJL3mXFztA7iIGk7wozcUNfNxLoUzy8hN++ V45RdQ4pfbP0CRlBIjJ0n6apuOfhe+IbMv/S3WZD85176Xkh8Lj4usOp8wp2IRCmdGg7KT cPkEHijgGOEueo4aZDSxULejB1m2dV/yanxymO3RfqYqiVBxToIWhrQ6jT6QKH9agFvmWV Qr8O2btKIdEyZP53nw/fb6Y1kFW/p0naoXtUPxux1rg2dvi0GgIgElIcaQA7ihi5tpt3Ht /S5Ue6OFQr5yJ6vSpsyWWjGGMXQPn76/EnZE8v3PG61Y7aiXdSmsQiErmv5hqODA== Received: by filterdrecv-86675bc4b9-2clcd with SMTP id filterdrecv-86675bc4b9-2clcd-1-660C7D44-9 2024-04-02 21:48:52.592506206 +0000 UTC m=+1300989.153488583 Received: from localhost.localdomain (unknown) by geopod-ismtpd-3 (SG) with ESMTP id c8Mm-crfSdGId839eDTPaA Tue, 02 Apr 2024 21:48:52.280 +0000 (UTC) From: Frank Plowman Date: Tue, 02 Apr 2024 21:48:52 +0000 (UTC) Message-ID: <20240402214843.474910-1-post@frankplowman.com> X-Mailer: git-send-email 2.44.0 MIME-Version: 1.0 X-SG-EID: =?us-ascii?Q?u001=2EZ0KJCHpts8tvDq7PHgz5cpqJ+vJcSmdTtST=2Fg91WT3qAbNxUpEMGIDQq9?= =?us-ascii?Q?1mXy3Yjvepp8fYsFUJzGsyhT1EwBe2kwsjy2Y9D?= =?us-ascii?Q?ttckmWRt2u002kw9YwUj6KEKw7Wplkh4Xyln4Mi?= =?us-ascii?Q?MLKg6u5RfgFogzekaWjgBICiC2TwkGawmn+Y2rm?= =?us-ascii?Q?RqYjafDe1I7vDMMKPo1IdLApCh=2FYB1XzcfD9lK=2F?= =?us-ascii?Q?j0ErnXQNHyZoH9au+KTg8pEMMWZFDAZAPznePi8?= =?us-ascii?Q?8KEl?= To: ffmpeg-devel@ffmpeg.org X-Entity-ID: u001.qzljkbu34TNIX4NwfTiKWA== Subject: [FFmpeg-devel] [PATCH] lavc/vvc: Only read split_cu_flag if a split is allowed X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Frank Plowman Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" Archived-At: List-Archive: List-Post: Add a check to ensure some split is possible before reading the split_cu_flag. This is present in the spec, in VVCv3 section 7.3.11.4. Its omission could lead to infinite loops and ultimately crashing due to stack overflow. --- libavcodec/vvc/vvc_ctu.c | 7 ++++++- 1 file changed, 6 insertions(+), 1 deletion(-) diff --git a/libavcodec/vvc/vvc_ctu.c b/libavcodec/vvc/vvc_ctu.c index 8ba12c8d9f..32d8bc8f5c 100644 --- a/libavcodec/vvc/vvc_ctu.c +++ b/libavcodec/vvc/vvc_ctu.c @@ -2095,6 +2095,7 @@ static int hls_coding_tree(VVCLocalContext *lc, const int ch_type = tree_type_curr == DUAL_TREE_CHROMA; int ret; VVCAllowedSplit allowed; + int split_cu_flag; if (pps->r->pps_cu_qp_delta_enabled_flag && qg_on_y && cb_sub_div <= sh->cu_qp_delta_subdiv) { lc->parse.is_cu_qp_delta_coded = 0; @@ -2109,7 +2110,11 @@ static int hls_coding_tree(VVCLocalContext *lc, can_split(lc, x0, y0, cb_width, cb_height, mtt_depth, depth_offset, part_idx, last_split_mode, tree_type_curr, mode_type_curr, &allowed); - if (ff_vvc_split_cu_flag(lc, x0, y0, cb_width, cb_height, ch_type, &allowed)) { + if (allowed.btv || allowed.bth || allowed.ttv || allowed.tth || allowed.qt) + split_cu_flag = ff_vvc_split_cu_flag(lc, x0, y0, cb_width, cb_height, ch_type, &allowed); + else + split_cu_flag = 0; + if (split_cu_flag) { VVCSplitMode split = ff_vvc_split_mode(lc, x0, y0, cb_width, cb_height, cqt_depth, mtt_depth, ch_type, &allowed); VVCModeType mode_type = mode_type_decode(lc, x0, y0, cb_width, cb_height, split, ch_type, mode_type_curr); -- 2.44.0 _______________________________________________ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".