From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org [79.124.17.100]) by master.gitmailbox.com (Postfix) with ESMTP id E4BAC4873D for ; Wed, 17 Jan 2024 02:16:56 +0000 (UTC) Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 3C8FA68D089; Wed, 17 Jan 2024 04:16:50 +0200 (EET) Received: from APC01-SG2-obe.outbound.protection.outlook.com (mail-sgaapc01olkn2100.outbound.protection.outlook.com [40.92.53.100]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 923D868D081 for ; Wed, 17 Jan 2024 04:16:43 +0200 (EET) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=XoYhTjqS0GFV+c1knHtrIGvN3zrWomrd0fRIKwD9J1RRhngn6GgwznbwnR2KdV7akFnBxQfJZFeqmubnvLyQiywqodlegkwQafBs4+nZgAlwW0VqP9RAdXetgEpS8TGeIOCn3S/1iZeL3hZl8Bf3T1phyXSI0y9ye9F7xlakDExwoHYpPYT4NFWA/PZQwglrlkeyJRP9TkaTS33S0qTfAeznZIL42mSPSh6oISVJhSWrP1gq3Gq2vZd1nJEcN2gauIfVBg1Y81OULQfmNJyLWsPUbB+UzyM8kcfmTtZECTa9jWQdzaQdWxdclJw4Ix0bIrRxd9ElOoz5RHxB4zeZxg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=NbVdQ7Mhx5buZ9IsnGN4CCXkXoBUQk+2XUsakLWspaE=; b=IKKkLzouHzOHz/GYz+66oQJtClBFBpI4i60lDdb6yS1HINq6U4wtLVHDPTeF448TIgYKCj0OEJyyAK6/3kMYWXsSbCfZu82/uNnpssQk/j1cFnhM3WCzNAaptJt3L+mbg8Kb7R27haweG3soH4c51Nbc5hFfl0nAgWuy5KOG6SaRBppwBGZbcbRmKo4PwQbSDHWpNTTkEzunA9VYfUyA6oZ3Zfc+RCGn6WdtuHKjggkpRgX03Mia4LbdSQAmULjdqt8taR4yrweSvDg7103U4n9F3/Scu7iTMjRXRoUQtCrhlqnPImSMf8a3QbTbG6R+Fz868t/YVuj85o7EnTGUFQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=none; dmarc=none; dkim=none; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=outlook.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=NbVdQ7Mhx5buZ9IsnGN4CCXkXoBUQk+2XUsakLWspaE=; b=mdYznF5aBIozGGd+yEI0Q3+ip++mcHKjs0c1wEJdUC9U7rP8Ueoot42lElv1tDbgcLVv2YMEOzD7oPi5Y5+7Pq10OpRy7hHKIofepA0a03EfA6SFl1u4VvHR3AQ6N1PKU0IFfsOjiLfP8N9vlnYgxokR7PiqcwfzDckgY7CSy+E7yhKY8ILE5yzFPH2egkg+ZOssbIxddZHUmbn1xdGiO608drBgepaXx7hg9CgTWhg5Us6P526hWkKb8AfQo0Cs6bsByZiLAYtx09w2TooV2YUtLByzvq416D1eJPv5qMD7sjrYpZiBzOPgSsJTKbSDSoUa2v0e/4+PFd+2z8zqYg== Received: from TYSPR06MB6433.apcprd06.prod.outlook.com (2603:1096:400:47a::6) by SEYPR06MB5255.apcprd06.prod.outlook.com (2603:1096:101:84::10) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7181.26; Wed, 17 Jan 2024 02:16:22 +0000 Received: from TYSPR06MB6433.apcprd06.prod.outlook.com ([fe80::e139:fd7a:1b66:a3f7]) by TYSPR06MB6433.apcprd06.prod.outlook.com ([fe80::e139:fd7a:1b66:a3f7%7]) with mapi id 15.20.7202.020; Wed, 17 Jan 2024 02:16:22 +0000 From: Nuo Mi To: ffmpeg-devel@ffmpeg.org Date: Wed, 17 Jan 2024 10:15:32 +0800 Message-ID: X-Mailer: git-send-email 2.25.1 In-Reply-To: <20240117021533.2965-1-nuomi2021@gmail.com> References: <20240117021533.2965-1-nuomi2021@gmail.com> X-TMN: [CqEzawnRzUi/i8+XvMJG+uibgZex708b] X-ClientProxiedBy: TYCP286CA0261.JPNP286.PROD.OUTLOOK.COM (2603:1096:400:455::12) To TYSPR06MB6433.apcprd06.prod.outlook.com (2603:1096:400:47a::6) X-Microsoft-Original-Message-ID: <20240117021533.2965-2-nuomi2021@gmail.com> MIME-Version: 1.0 X-MS-Exchange-MessageSentRepresentingType: 2 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: TYSPR06MB6433:EE_|SEYPR06MB5255:EE_ X-MS-Office365-Filtering-Correlation-Id: 56556de0-682e-4deb-89b8-08dc17024c9f X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: lhAb6f3kJa/RpF1Y9y90M+SVwKdSnYdCdhGUdv4Yj9/n0CRm2Y/vWAjkm0nxEJcDDI4mGGMBxg7PEvXBnWDXnPW9jsagr8RM0hh+C48cL6GLVtFT4BsE86n+nHALT4RGhY5Du4esmhRaQwPlhXprOJoryTFj+P+/7fyPoyiSP1S0Td3YNax2sLY+MFLp7AeVL1ttqTmVBjuNHLTM9oPne0zrRntxkLruKKh0mjBQsa0dwi5OsTjaZcpJyvxTlETQkZJ06K7/pnVQohTbrA8hoXb4px9NW0ANdFJYI+ITwVkAkfrlqAfP5jiaDRgOLSmpq5rcXaavOgGUeCgwJica5iSx4K/Qmg9KUjdV4Gv+fily59d5BHQRjvR7YJy1Z8nSX+VCHZPTLX8WY2ElLTbMMKOWXzrI/WBnH9W4RoR8otAe4kXQx72ZCgynlvgsLfJ+lEtw+47jhX4dNWYMQMEF/GLBsBDn4RQDIPrdAAF6IHIsXR+yxSk4QkI4qEV28f6v2niYhxeysskunV2P6suaS0sbbAVyxZcJM+9NEVfrg2FkyrcsdLliybkB+EVCXzW/Mej1c0useGS1ScJeEe+Ge66j41OwdgYZDzJGkpHafXqbMKNU/XAh2WtiyUP3Ehf6 X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?us-ascii?Q?JZ4d5zT4PJjvv+kwXDl3Fl++Xp0ETHhXl9KFxTbkPrd+fQO3GZAGfG3aHGEH?= =?us-ascii?Q?6LyMKJRUicaVSfyU0RNWkhZD5OS20SnwhihL8qkC+hvDVtLjGmb4gbRnoIiJ?= =?us-ascii?Q?LDo27oIhvbMnVNSDrGBnt0okWantgtuF2XrKkhliKLSQnHQAqwAfFqHucYNI?= =?us-ascii?Q?vPUUNrSIV9qtGBJe+wuo/3iLt6v5LhM86sj23tPbPIiSsnb6vujwAQtBVRAz?= =?us-ascii?Q?fy8ax2yd8Ar29otbyGv6ymaxRcKuFxxgjRDYv26kB6FGlUdMPRwePFpM2Qge?= =?us-ascii?Q?nwm/TjIqwnHcDmOPhPtfVBSmPjO0ELcOAqCRPP01NUAWGo0Smbfqdxm1vit9?= =?us-ascii?Q?oEs1zQBs2OEW9eMEsKFwZXeXMUWBCGdyQIxQu2lvzf/256ZYJUnhJz9FX145?= =?us-ascii?Q?RZKIyn7V/xIs4d/mhcxHNWVWjE9dqGPUc2ToNXl73gibFdNCFKNfn1JDP4o/?= =?us-ascii?Q?mIYFQc/JlFI/7J5lT3rgsRplr2ks2t07T2GmIar2qzdt1Kbjq8AqNS4/yWUd?= =?us-ascii?Q?E3ok6pzM6syAKhsx/o+VLjnCZgv9VOCgw/zztyVG4XwF6oiSi3Z25ualw9hy?= =?us-ascii?Q?CgeD7g9EfzsJ5knIYnqOhcJ/Ts4d88fw/kucVowtWE1R6PD1oZnEnIyFnCCg?= =?us-ascii?Q?sS0Em1sKH3WcKAnGZLOKsBS715tfkpBzswc9nUvR63JfAo84PC8GXfBKIL7V?= =?us-ascii?Q?Al2EUym7vujcrXVGMCX31IIiM/6DDvNigzLXs/pafXiQzDUgFy9pPjnTa4rQ?= =?us-ascii?Q?oKF5SOeLVRYoj4x+k+DrnXwwX39uQ/4eSu6YRzaGdbbwhrjGSjAmHT9o7+5r?= =?us-ascii?Q?hUmqKjrdToszRUd8qwWKHkG7axNvPBygZyynwiWxJhRhJnpSCZPy6WyT2Thd?= =?us-ascii?Q?IW47lk4uEufuMv8QdeS7zX8eRSHq3iVBjHiSpoVMeshtkXCUp8O3/IDNGL8j?= =?us-ascii?Q?5MCQNwF+zMjk+Em+l3Ew3hzKAW9M5xHqUKj3Vz8CHOivhNPcpSMELeRaT+vf?= =?us-ascii?Q?8YmfOgUKgO1VrUufFQuY0IDBxkrL+dm7R3Ue2cC04CcA2H38r6CMYyA/+w86?= =?us-ascii?Q?QnOWrVI3jVB6ubirmDjkDSBSU/7su5r5fJefxVI7NVzhoTGmXnk/RgbY+OYW?= =?us-ascii?Q?xRkHeBXa31OWY1Gqi/XTZmZlcsxj80n7ObbAhu6/m3JFqd2LIzYlNEBfepwH?= =?us-ascii?Q?9TAOnS2a6Ar6noobjFE7mseWLIkJ4LFXXxYjYYrwbqQXjZVKKr+RV/Rga150?= =?us-ascii?Q?fDLVH10sHITcr5jeVe++?= X-OriginatorOrg: outlook.com X-MS-Exchange-CrossTenant-Network-Message-Id: 56556de0-682e-4deb-89b8-08dc17024c9f X-MS-Exchange-CrossTenant-AuthSource: TYSPR06MB6433.apcprd06.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 17 Jan 2024 02:16:22.6320 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 84df9e7f-e9f6-40af-b435-aaaaaaaaaaaa X-MS-Exchange-CrossTenant-RMS-PersistedConsumerOrg: 00000000-0000-0000-0000-000000000000 X-MS-Exchange-Transport-CrossTenantHeadersStamped: SEYPR06MB5255 Subject: [FFmpeg-devel] [PATCH v2 1/2] avcodec/vvcdec: deblock, fix uninitialized values X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Nuo Mi , James Almer Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" Archived-At: List-Archive: List-Post: see https://fate.ffmpeg.org/report.cgi?slot=x86_64-archlinux-gcc-valgrind&time=20240105201935 If tc is zero, the max_len_q, max_len_p are uninitialized. Reported-by: James Almer --- libavcodec/vvc/vvc_filter_template.c | 236 ++++++++++++++------------- 1 file changed, 119 insertions(+), 117 deletions(-) diff --git a/libavcodec/vvc/vvc_filter_template.c b/libavcodec/vvc/vvc_filter_template.c index 671ed7de4e..845c8a4181 100644 --- a/libavcodec/vvc/vvc_filter_template.c +++ b/libavcodec/vvc/vvc_filter_template.c @@ -551,81 +551,80 @@ static void FUNC(vvc_loop_filter_luma)(uint8_t* _pix, ptrdiff_t _xstride, ptrdif const ptrdiff_t ystride = _ystride / sizeof(pixel); for (int i = 0; i < 2; i++) { - pixel* pix = (pixel*)_pix + i * 4 * ystride; - const int dp0 = abs(P2 - 2 * P1 + P0); - const int dq0 = abs(Q2 - 2 * Q1 + Q0); - const int dp3 = abs(TP2 - 2 * TP1 + TP0); - const int dq3 = abs(TQ2 - 2 * TQ1 + TQ0); - const int d0 = dp0 + dq0; - const int d3 = dp3 + dq3; #if BIT_DEPTH < 10 const int tc = (_tc[i] + (1 << (9 - BIT_DEPTH))) >> (10 - BIT_DEPTH); #else const int tc = _tc[i] << (BIT_DEPTH - 10); #endif - const int tc25 = ((tc * 5 + 1) >> 1); - - const int no_p = _no_p[i]; - const int no_q = _no_q[i]; - - int max_len_p = _max_len_p[i]; - int max_len_q = _max_len_q[i]; - - const int large_p = (max_len_p > 3 && !hor_ctu_edge); - const int large_q = max_len_q > 3; - const int beta = _beta[i] << BIT_DEPTH - 8; - - const int beta_3 = beta >> 3; - const int beta_2 = beta >> 2; - - if (!tc) - continue; - - if (large_p || large_q) { - const int dp0l = large_p ? ((dp0 + abs(P5 - 2 * P4 + P3) + 1) >> 1) : dp0; - const int dq0l = large_q ? ((dq0 + abs(Q5 - 2 * Q4 + Q3) + 1) >> 1) : dq0; - const int dp3l = large_p ? ((dp3 + abs(TP5 - 2 * TP4 + TP3) + 1) >> 1) : dp3; - const int dq3l = large_q ? ((dq3 + abs(TQ5 - 2 * TQ4 + TQ3) + 1) >> 1) : dq3; - const int d0l = dp0l + dq0l; - const int d3l = dp3l + dq3l; - const int beta53 = beta * 3 >> 5; - const int beta_4 = beta >> 4; - max_len_p = large_p ? max_len_p : 3; - max_len_q = large_q ? max_len_q : 3; - - if (d0l + d3l < beta) { - const int sp0l = abs(P3 - P0) + (max_len_p == 7 ? abs(P7 - P6 - P5 + P4) : 0); - const int sq0l = abs(Q0 - Q3) + (max_len_q == 7 ? abs(Q4 - Q5 - Q6 + Q7) : 0); - const int sp3l = abs(TP3 - TP0) + (max_len_p == 7 ? abs(TP7 - TP6 - TP5 + TP4) : 0); - const int sq3l = abs(TQ0 - TQ3) + (max_len_q == 7 ? abs(TQ4 - TQ5 - TQ6 + TQ7) : 0); - const int sp0 = large_p ? ((sp0l + abs(P3 - P(max_len_p)) + 1) >> 1) : sp0l; - const int sp3 = large_p ? ((sp3l + abs(TP3 - TP(max_len_p)) + 1) >> 1) : sp3l; - const int sq0 = large_q ? ((sq0l + abs(Q3 - Q(max_len_q)) + 1) >> 1) : sq0l; - const int sq3 = large_q ? ((sq3l + abs(TQ3 - TQ(max_len_q)) + 1) >> 1) : sq3l; - if (sp0 + sq0 < beta53 && abs(P0 - Q0) < tc25 && - sp3 + sq3 < beta53 && abs(TP0 - TQ0) < tc25 && - (d0l << 1) < beta_4 && (d3l << 1) < beta_4) { - FUNC(loop_filter_luma_large)(pix, xstride, ystride, tc, no_p, no_q, max_len_p, max_len_q); - continue; + if (tc) { + pixel* pix = (pixel*)_pix + i * 4 * ystride; + const int dp0 = abs(P2 - 2 * P1 + P0); + const int dq0 = abs(Q2 - 2 * Q1 + Q0); + const int dp3 = abs(TP2 - 2 * TP1 + TP0); + const int dq3 = abs(TQ2 - 2 * TQ1 + TQ0); + const int d0 = dp0 + dq0; + const int d3 = dp3 + dq3; + const int tc25 = ((tc * 5 + 1) >> 1); + + const int no_p = _no_p[i]; + const int no_q = _no_q[i]; + + int max_len_p = _max_len_p[i]; + int max_len_q = _max_len_q[i]; + + const int large_p = (max_len_p > 3 && !hor_ctu_edge); + const int large_q = max_len_q > 3; + + const int beta = _beta[i] << BIT_DEPTH - 8; + const int beta_3 = beta >> 3; + const int beta_2 = beta >> 2; + + if (large_p || large_q) { + const int dp0l = large_p ? ((dp0 + abs(P5 - 2 * P4 + P3) + 1) >> 1) : dp0; + const int dq0l = large_q ? ((dq0 + abs(Q5 - 2 * Q4 + Q3) + 1) >> 1) : dq0; + const int dp3l = large_p ? ((dp3 + abs(TP5 - 2 * TP4 + TP3) + 1) >> 1) : dp3; + const int dq3l = large_q ? ((dq3 + abs(TQ5 - 2 * TQ4 + TQ3) + 1) >> 1) : dq3; + const int d0l = dp0l + dq0l; + const int d3l = dp3l + dq3l; + const int beta53 = beta * 3 >> 5; + const int beta_4 = beta >> 4; + max_len_p = large_p ? max_len_p : 3; + max_len_q = large_q ? max_len_q : 3; + + if (d0l + d3l < beta) { + const int sp0l = abs(P3 - P0) + (max_len_p == 7 ? abs(P7 - P6 - P5 + P4) : 0); + const int sq0l = abs(Q0 - Q3) + (max_len_q == 7 ? abs(Q4 - Q5 - Q6 + Q7) : 0); + const int sp3l = abs(TP3 - TP0) + (max_len_p == 7 ? abs(TP7 - TP6 - TP5 + TP4) : 0); + const int sq3l = abs(TQ0 - TQ3) + (max_len_q == 7 ? abs(TQ4 - TQ5 - TQ6 + TQ7) : 0); + const int sp0 = large_p ? ((sp0l + abs(P3 - P(max_len_p)) + 1) >> 1) : sp0l; + const int sp3 = large_p ? ((sp3l + abs(TP3 - TP(max_len_p)) + 1) >> 1) : sp3l; + const int sq0 = large_q ? ((sq0l + abs(Q3 - Q(max_len_q)) + 1) >> 1) : sq0l; + const int sq3 = large_q ? ((sq3l + abs(TQ3 - TQ(max_len_q)) + 1) >> 1) : sq3l; + if (sp0 + sq0 < beta53 && abs(P0 - Q0) < tc25 && + sp3 + sq3 < beta53 && abs(TP0 - TQ0) < tc25 && + (d0l << 1) < beta_4 && (d3l << 1) < beta_4) { + FUNC(loop_filter_luma_large)(pix, xstride, ystride, tc, no_p, no_q, max_len_p, max_len_q); + continue; + } } } - } - if (d0 + d3 < beta) { - if (max_len_p > 2 && max_len_q > 2 && - abs(P3 - P0) + abs(Q3 - Q0) < beta_3 && abs(P0 - Q0) < tc25 && - abs(TP3 - TP0) + abs(TQ3 - TQ0) < beta_3 && abs(TP0 - TQ0) < tc25 && - (d0 << 1) < beta_2 && (d3 << 1) < beta_2) { - FUNC(loop_filter_luma_strong)(pix, xstride, ystride, tc, tc << 1, tc * 3, no_p, no_q); - } else { // weak filtering - int nd_p = 1; - int nd_q = 1; - if (max_len_p > 1 && max_len_q > 1) { - if (dp0 + dp3 < ((beta + (beta >> 1)) >> 3)) - nd_p = 2; - if (dq0 + dq3 < ((beta + (beta >> 1)) >> 3)) - nd_q = 2; + if (d0 + d3 < beta) { + if (max_len_p > 2 && max_len_q > 2 && + abs(P3 - P0) + abs(Q3 - Q0) < beta_3 && abs(P0 - Q0) < tc25 && + abs(TP3 - TP0) + abs(TQ3 - TQ0) < beta_3 && abs(TP0 - TQ0) < tc25 && + (d0 << 1) < beta_2 && (d3 << 1) < beta_2) { + FUNC(loop_filter_luma_strong)(pix, xstride, ystride, tc, tc << 1, tc * 3, no_p, no_q); + } else { + int nd_p = 1; + int nd_q = 1; + if (max_len_p > 1 && max_len_q > 1) { + if (dp0 + dp3 < ((beta + (beta >> 1)) >> 3)) + nd_p = 2; + if (dq0 + dq3 < ((beta + (beta >> 1)) >> 3)) + nd_q = 2; + } + FUNC(loop_filter_luma_weak)(pix, xstride, ystride, tc, beta, no_p, no_q, nd_p, nd_q); } - FUNC(loop_filter_luma_weak)(pix, xstride, ystride, tc, beta, no_p, no_q, nd_p, nd_q); } } } @@ -689,65 +688,68 @@ static void FUNC(vvc_loop_filter_chroma)(uint8_t *_pix, const ptrdiff_t _xstrid const int end = 8 / size; // 8 samples a loop for (int i = 0; i < end; i++) { - pixel *pix = (pixel *)_pix + i * size * ystride; - const uint8_t no_p = _no_p[i]; - const uint8_t no_q = _no_q[i]; - const int beta = _beta[i] << (BIT_DEPTH - 8); - const int beta_3 = beta >> 3; - const int beta_2 = beta >> 2; - #if BIT_DEPTH < 10 const int tc = (_tc[i] + (1 << (9 - BIT_DEPTH))) >> (10 - BIT_DEPTH); #else const int tc = _tc[i] << (BIT_DEPTH - 10); #endif - const int tc25 = ((tc * 5 + 1) >> 1); - - uint8_t max_len_p = _max_len_p[i]; - uint8_t max_len_q = _max_len_q[i]; - - if (!max_len_p || !max_len_q || !tc) - continue; - - if (max_len_q == 3){ - const int p1n = shift ? FP1 : TP1; - const int p2n = max_len_p == 1 ? p1n : (shift ? FP2 : TP2); - const int p0n = shift ? FP0 : TP0; - const int q0n = shift ? FQ0 : TQ0; - const int q1n = shift ? FQ1 : TQ1; - const int q2n = shift ? FQ2 : TQ2; - const int p3 = max_len_p == 1 ? P1 : P3; - const int p2 = max_len_p == 1 ? P1 : P2; - const int p1 = P1; - const int p0 = P0; - const int dp0 = abs(p2 - 2 * p1 + p0); - const int dq0 = abs(Q2 - 2 * Q1 + Q0); + if (tc) { + pixel *pix = (pixel *)_pix + i * size * ystride; + const uint8_t no_p = _no_p[i]; + const uint8_t no_q = _no_q[i]; + + const int beta = _beta[i] << (BIT_DEPTH - 8); + const int beta_3 = beta >> 3; + const int beta_2 = beta >> 2; + + const int tc25 = ((tc * 5 + 1) >> 1); + + uint8_t max_len_p = _max_len_p[i]; + uint8_t max_len_q = _max_len_q[i]; - const int dp1 = abs(p2n - 2 * p1n + p0n); - const int dq1 = abs(q2n - 2 * q1n + q0n); - const int d0 = dp0 + dq0; - const int d1 = dp1 + dq1; - - if (d0 + d1 < beta) { - const int p3n = max_len_p == 1 ? p1n : (shift ? FP3 : TP3); - const int q3n = shift ? FQ3 : TQ3; - const int dsam0 = (d0 << 1) < beta_2 && (abs(p3 - p0) + abs(Q0 - Q3) < beta_3) && - abs(p0 - Q0) < tc25; - const int dsam1 = (d1 << 1) < beta_2 && (abs(p3n - p0n) + abs(q0n - q3n) < beta_3) && - abs(p0n - q0n) < tc25; - if (!dsam0 || !dsam1) + if (!max_len_p || !max_len_q) + continue; + + if (max_len_q == 3){ + const int p1n = shift ? FP1 : TP1; + const int p2n = max_len_p == 1 ? p1n : (shift ? FP2 : TP2); + const int p0n = shift ? FP0 : TP0; + const int q0n = shift ? FQ0 : TQ0; + const int q1n = shift ? FQ1 : TQ1; + const int q2n = shift ? FQ2 : TQ2; + const int p3 = max_len_p == 1 ? P1 : P3; + const int p2 = max_len_p == 1 ? P1 : P2; + const int p1 = P1; + const int p0 = P0; + const int dp0 = abs(p2 - 2 * p1 + p0); + const int dq0 = abs(Q2 - 2 * Q1 + Q0); + + const int dp1 = abs(p2n - 2 * p1n + p0n); + const int dq1 = abs(q2n - 2 * q1n + q0n); + const int d0 = dp0 + dq0; + const int d1 = dp1 + dq1; + + if (d0 + d1 < beta) { + const int p3n = max_len_p == 1 ? p1n : (shift ? FP3 : TP3); + const int q3n = shift ? FQ3 : TQ3; + const int dsam0 = (d0 << 1) < beta_2 && (abs(p3 - p0) + abs(Q0 - Q3) < beta_3) && + abs(p0 - Q0) < tc25; + const int dsam1 = (d1 << 1) < beta_2 && (abs(p3n - p0n) + abs(q0n - q3n) < beta_3) && + abs(p0n - q0n) < tc25; + if (!dsam0 || !dsam1) + max_len_p = max_len_q = 1; + } else { max_len_p = max_len_q = 1; - } else { - max_len_p = max_len_q = 1; + } } - } - if (max_len_p == 3 && max_len_q == 3) - FUNC(loop_filter_chroma_strong)(pix, xstride, ystride, size, tc, no_p, no_q); - else if (max_len_q == 3) - FUNC(loop_filter_chroma_strong_one_side)(pix, xstride, ystride, size, tc, no_p, no_q); - else - FUNC(loop_filter_chroma_weak)(pix, xstride, ystride, size, tc, no_p, no_q); + if (max_len_p == 3 && max_len_q == 3) + FUNC(loop_filter_chroma_strong)(pix, xstride, ystride, size, tc, no_p, no_q); + else if (max_len_q == 3) + FUNC(loop_filter_chroma_strong_one_side)(pix, xstride, ystride, size, tc, no_p, no_q); + else + FUNC(loop_filter_chroma_weak)(pix, xstride, ystride, size, tc, no_p, no_q); + } } } -- 2.25.1 _______________________________________________ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".