From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org [79.124.17.100]) by master.gitmailbox.com (Postfix) with ESMTPS id 597DF4CAFE for ; Tue, 11 Feb 2025 12:12:15 +0000 (UTC) Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id DF11468BD13; Tue, 11 Feb 2025 14:12:10 +0200 (EET) Received: from EUR05-DB8-obe.outbound.protection.outlook.com (mail-db8eur05olkn2012.outbound.protection.outlook.com [40.92.89.12]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id EA13E68B3FA for ; Tue, 11 Feb 2025 14:12:04 +0200 (EET) ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=yxHP0MkDYdtyzzvuxABpwwiBEAOctduF4//BtBXysXPrI6Ro+VzioGq1DHOwcmyKIQTjNBqtyvX5v7qlI8HBC6j6Dqd5gAlMaAb+BjDKvpoJgP0CF/ifU/AM6b/krGZ0El6ezkXif2nJaiJ24XbwG3rAj6bx0pKGPaPD1oTKWMhjHt9iRzQ9PWPeuUjO5U7lMLjixh6x/NDQ1vl4wnPBaifl104iVuWDDfnJyHcM002bpuEQ8PccZJoci6G//nc5YNdes/76ymoBb9Di2Hhi4iqDFVeAcd1UGQezF+y4Evjnp8uZEPjTLUnSI1rFWEjlW/adZLuoND38P9/T8UvmAA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=/ZFZ1dy7MuUYpOxAcHhDhjzvnA01ALFdWBFa9omADDI=; b=Bnav74+78Ab+b0EurgEUUiEmClvAEczV6dOiqDe1RYdYVAFBLNhR+ASirDzqeALZVu7Ux2MqOJ075fXmz9bGqQdyiNGssfcZcTlHAFJyL2NzuO3nkBoq/Rd9283rr3d2/VSpG490Kcr//HAPA1izdBQ7byHqL9EvCrpSzCVQ+vEqIV+KNJGymtcw8hgB1ZsvSwt0d+YQv2bksscNYz5g+NtOTEA84iDqAsqhPpJrpy7OxuoHKxPoUr6PEPSdZa92eCI79WMK+uT7vu816+qSijJP5Cswsl09QJI/td0sAOF/ywfoS7wo11Zh+42d3As+XIkfWptPVKmstOHaBI4FvQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=none; dmarc=none; dkim=none; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=outlook.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=/ZFZ1dy7MuUYpOxAcHhDhjzvnA01ALFdWBFa9omADDI=; b=o1yt4DuPI+Fd3+P+esU2rIee5kt2utfyaDD5K3LbEAfD5QKIhgu2dML7imEwSGA8fdhPoYhbLyUwPTeT3bY1fcSZ3umcCjPPPIf1U8uYggNqE2cctkdtsRhvAR/TfXVUs4ge5DL1BHdVtv1FxCVhLkTl6ODXwRY870/JHm9ZePNpOykjZMXVSYe76suyJ9knPGORWhUWr87Hx7pJbgzDjl4CDE+fafC11oln2APldrDfszMcfUawU2EPRy7bdmtkj3YrIElVBZ7f42GUi4no0I1vf8BPCEk1nuq6mZ79QgpLmEUbInCSJoin9y/UlN6MmTU4B/DoJ4v6/43/efauAQ== Received: from AS8P250MB0744.EURP250.PROD.OUTLOOK.COM (2603:10a6:20b:541::14) by AS1P250MB0629.EURP250.PROD.OUTLOOK.COM (2603:10a6:20b:4af::8) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.8422.19; Tue, 11 Feb 2025 12:12:03 +0000 Received: from AS8P250MB0744.EURP250.PROD.OUTLOOK.COM ([fe80::384d:40d4:ecb7:1c9]) by AS8P250MB0744.EURP250.PROD.OUTLOOK.COM ([fe80::384d:40d4:ecb7:1c9%4]) with mapi id 15.20.8422.015; Tue, 11 Feb 2025 12:12:03 +0000 Message-ID: Date: Tue, 11 Feb 2025 13:12:02 +0100 User-Agent: Mozilla Thunderbird To: ffmpeg-devel@ffmpeg.org References: Content-Language: en-US From: Andreas Rheinhardt In-Reply-To: X-ClientProxiedBy: FR0P281CA0257.DEUP281.PROD.OUTLOOK.COM (2603:10a6:d10:b5::20) To AS8P250MB0744.EURP250.PROD.OUTLOOK.COM (2603:10a6:20b:541::14) X-Microsoft-Original-Message-ID: <541a1aec-9fd2-4aad-a5a2-3b63e587ff61@outlook.com> MIME-Version: 1.0 X-MS-Exchange-MessageSentRepresentingType: 1 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: AS8P250MB0744:EE_|AS1P250MB0629:EE_ X-MS-Office365-Filtering-Correlation-Id: 59585d21-980d-48ad-e145-08dd4a954b58 X-Microsoft-Antispam: BCL:0; ARA:14566002|5072599009|8060799006|19110799003|6090799003|7092599003|461199028|15080799006|440099028|10035399004|3412199025|41001999003|56899033; X-Microsoft-Antispam-Message-Info: =?utf-8?B?SmFocDRvK2U3eXR1OWg5eC9RaWVPcFRnUDdWdTREYkR5UzVXU090ZDU1LzlB?= =?utf-8?B?WTBMTTdQYXVuSDU4OHhoREFGT1hjSmlYZXA1SlNjUmNpaXhBQjFUVkhQcjI5?= =?utf-8?B?TGovYWVmbmNFWTBVQUFwNUhoT3NJa2c0QnhrTjVzVEFzckRsc3VPY0JrNDRo?= =?utf-8?B?WDRaQ05BRUNjRkVENlE3dnhNcGIxSUNtR0RjeldFdGJ4eHEyZTYvbTBTc1RP?= =?utf-8?B?VUp5M0I1ZnluZCt2TnhsNE1PUE1VUndiSU1YdU9nUm04Zm02THpzYWNHaTZV?= =?utf-8?B?YmVRcWljTUh2ZlAvVk12ZEt0U1BjaUY1THV0b1ZqOVBPaTkrUk9XMEwrMlRW?= =?utf-8?B?eFpaa2lYcm1MckVPazdsamwwc01zNXNSYWE4WkVHaHVpclNNVmozOWhpSENZ?= =?utf-8?B?TWk3Kys2TFZReklnRCtoUWtiRXcxWnEvVzVWSEdWT29QOGVFVExzVXFGYUUy?= =?utf-8?B?WnF1MnZpNVpiNm1JWThPUGEvR0FHakdnVnBwbUJxUllDRVJzbDRTN2ZZVGFI?= =?utf-8?B?U0dDajJOZGtyV0dKNkwrbThCMGlpNy9zdWtVZkZhMlo3RHhnSndFWVdJd3hl?= =?utf-8?B?angyRnFQek9sSEJuNS94ZE1QTllrRHJ6R1lzYzBqdDJneFNBRHNtZ05zTTVm?= =?utf-8?B?N2w2OTVPWUlRLzA2NkUvVE1GdVhVSGdzUDBZdXpqdk8wTVJDZkZCanR2THdh?= =?utf-8?B?dm0xQVY1WWpxTEZhUW5Uai94Mnh4Q0hUN0ZVSmorSG4vM2c4NkRobmp0ODdL?= =?utf-8?B?a3h6VlpYV1RyN2dLQU0relZQMVg1YnFyZTh3eXg3TDJkV3YvVlVrVld6QU5i?= =?utf-8?B?QkZzT2tGSGdUOG1ETXlnOU8xRk9JMUtnK1A3VzZrZnZCMlhFbmxUckxBelZm?= =?utf-8?B?cExJbytkM2RLaytMWW1SUzdjVFQ4blhsTEY2UnFMdnA4QWs5eFRqK05uZndR?= =?utf-8?B?RGpXTTlFL0ZaL3g2eHdWam80ZnQ5Vys0cDNhcXdqTms5WDRDTEhlTXVPc29x?= =?utf-8?B?c1MraVdvb0VPQWJqSlNTN2czSW82SG83OEFDSXIzSC84S3JoNCtRQ2VyZ1l5?= =?utf-8?B?UU9YVkZTYnk1dHdMSjRtWXdnQ3ViajdxK1l6bFlpYlQ3RWxiclhIRzJvWmox?= =?utf-8?B?K3Fta254RjFQbS9jV3N4cE40Z2ZWZ29iYUNObGZsb0VwTHc3RFpNMVluUk1T?= =?utf-8?B?dmlKV1p2ZW9Hc3QwTStvZmt0NUhoUmc3V0ppdGdIaWJ6RENSV2doOFN5RDBD?= =?utf-8?B?M2krZkkzOW05OTJhTDJiaFlhaHc3T1Rqc1VSMGJiNUtCT3l1OVFacVdHbDRl?= =?utf-8?B?eG5wM1VTU0NUUnNRWkYzOXArWHdITjQwWWpiRXlOOTJicG1aQStHaXRkdHN6?= =?utf-8?B?NDlQWkZvZWlYY0ROdkd0NW9zekFRL0ZKN1ZieU1vdnM5Y3FjbHA2NFlIUUZq?= =?utf-8?B?RmdLU3lhVTlaRUJVZENLQnJreUxoaUNqVzcwVmc2c1Z5ZHM4OW9ab2lyZDk3?= =?utf-8?B?c2pnUVNFSzFiY3dhZ0pPY21HendZMGlkU3RlRjNhanZUYUF6Wm5mVm9oYkxI?= =?utf-8?Q?/8/hm1Q196mwf3wTAEu2TODL5EnFUhCoVxLGkd0sKmmFWF?= X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?B?ZEVJNFBIOFp6a3JrRk0yVS9xUVBaQ0VWdE5xNTk2OWpJRm1vSElMNndGVk9S?= =?utf-8?B?WlFLV01ESU5QZkxMVGV6YXV3R1JhV2diZks3NWpDSndVeHM5cVRGM1NUZDRQ?= =?utf-8?B?RllsYzVyOUZFTENHTUpOYzJjSDcxRS9XWk1ObkdUODNaQWtzRDQ5LzhyMWx3?= =?utf-8?B?cnBkdFFJTURoelVYZjRWUit4a3V2RFhQczZPV0ZRZlpZdUpzS1FxUnhZSE9X?= =?utf-8?B?dE5TWml2UVEvSkZCRk4yb2doQXRLOTAxUnNYd3RxRU9iaGo5UitKV0cybzVP?= =?utf-8?B?eSsyY3l1QzVIUlZkRGZqeFp6czVUTlFObzk2ZjFUaWxUUHJiQSt0OXloQVdm?= =?utf-8?B?QkdNNElYNy80Y1BLZmlhelhNMmhkZGNqNkhrRkltYld6Z1FmbGFJdFZTdURr?= =?utf-8?B?cm9PUG9ibTVrakJrU2c1d2o0MHprcHdaYVFNZFA2OStOYnR5Y1FINVZRQ2da?= =?utf-8?B?VE9TQlBiY0FqdHRYOEpWa3daZlFaYnVSbFl5b0VnclZvZE9GQTFvTXdnMEtm?= =?utf-8?B?eU42eUxBeWxaQjk4RUU2VXdreStxWjU4bEpEcUh0QmNiRFVvTGh2bkFQT3Ro?= =?utf-8?B?eDFkOFgxWEtWakdBZFlDcHQ4QjhTeTFSMEhNYXY2NG5tNlp3a0VmVTdYMWJi?= =?utf-8?B?QStlU3M0V1BBdzI5dzNMckZQemt3dmhlQ3NjRE9MbDZ5V3pjRWUvWk9hOUVV?= =?utf-8?B?cC9IOUdwb0FCWktUc21LcnFuUTFDQzNsUFQxV29OWC8xWktTWkdZb3ZxZ1pl?= =?utf-8?B?NzQrWmRWdG1zWjliUmdJOWViT0YxYjF4MGQ5eFZyZmo1bGw5eXNWNU1hNXN0?= =?utf-8?B?ZHUvL2VLckw0ekYwbVNSR1h1L2FLeVFVaUMvdUlPOTJXTnV4d0NGTWYrVCtF?= =?utf-8?B?aDY4UXZGMStPd29zUjEvemg1LzliU0dvc0hUSVdKMCtIN2JUUE1xWHlOS2sw?= =?utf-8?B?ZStUc2gyNS9QSlZWZWdVeEU3L1ZlV0dyblZWQ0FBSUVqblBRSGpxa1hhQW9F?= =?utf-8?B?b1l3aTJNa2JSaURMM1F2aUZROGJtNmQrUk9QTjZoWjczWnNHZ0FSWEx6WXlt?= =?utf-8?B?cHBqaHJhNmVHbk85NzljbzVaY2ZkWjlkSHpwckNZTCtGdEhxdHBOelpEdEFF?= =?utf-8?B?VUJqb1p3U1oxb20wVm0zODdUeGxjekZzVFB2SVJxRG1QdjNyejhjN2tiTGZT?= =?utf-8?B?bmJ1bnp0WU9kbENWWTJpU29TM2sybnRxdE9zK2ZSQkJOZ0N4YkZUQkx5dDBS?= =?utf-8?B?QnZSWlNVTUw0TlQrcVFIWUJ2VEhQaDhEd21odW16UHlndFlacU1NTFdaazlv?= =?utf-8?B?V2pIeFBmRnBpSlhpWHBzcE90WWtJb1NQQXVsSGNURkcyd1VKNWlCUk9tWTBX?= =?utf-8?B?RUNFd3EyWDQvWkZ2TDZJelE5SEwvSGd5dGZIc29Sd0lvdXhSWUZkQkFaVmlM?= =?utf-8?B?V1lqY2VyTnNNU003RmQrYjAyZWE0b2J5dEpOZGgxWXZUblZDWWNBeTk0Z1lY?= =?utf-8?B?N0N3bWhHeXhTUGtnYlZHNWRJa1RjTVZVUS9OaWpqMVJxakMzR3AyV09say9i?= =?utf-8?B?OHJ2ZkZBbmFCSU1OajhaTDhvNm5tYzkxbkI2WDZwbDl2Q1d3OHhkcng0dkxn?= =?utf-8?B?YzcrVG5CV24xbllmWmRSZkhqMUM3Sjc3NXRldnVqTzk5Rkw2UlpYYzQ4RWNy?= =?utf-8?B?UUtiT1hmeDIyYlRELzZVQW8vY0VHQmR2SU5DV3JYdnExTTBkMWlyY3hRPT0=?= X-OriginatorOrg: outlook.com X-MS-Exchange-CrossTenant-Network-Message-Id: 59585d21-980d-48ad-e145-08dd4a954b58 X-MS-Exchange-CrossTenant-AuthSource: AS8P250MB0744.EURP250.PROD.OUTLOOK.COM X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 11 Feb 2025 12:12:03.4528 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 84df9e7f-e9f6-40af-b435-aaaaaaaaaaaa X-MS-Exchange-CrossTenant-RMS-PersistedConsumerOrg: 00000000-0000-0000-0000-000000000000 X-MS-Exchange-Transport-CrossTenantHeadersStamped: AS1P250MB0629 Subject: Re: [FFmpeg-devel] [PATCH v6 3/3] avcodec/hevc: Add alpha layer support X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" Archived-At: List-Archive: List-Post: Zhao Zhili: > > >> On Feb 11, 2025, at 19:23, Andreas Rheinhardt wrote: >> >> Zhao Zhili: >>> From: Zhao Zhili >>> >>> Signed-off-by: Zhao Zhili >>> --- >>> libavcodec/hevc/hevcdec.c | 73 ++++++++++++++++++++++++++++++++++++++- >>> libavcodec/hevc/hevcdec.h | 2 ++ >>> libavcodec/hevc/refs.c | 34 +++++++++++++++++- >>> 3 files changed, 107 insertions(+), 2 deletions(-) >>> >>> diff --git a/libavcodec/hevc/hevcdec.c b/libavcodec/hevc/hevcdec.c >>> index e9c045f7a1..f71edf213b 100644 >>> --- a/libavcodec/hevc/hevcdec.c >>> +++ b/libavcodec/hevc/hevcdec.c >>> @@ -466,6 +466,25 @@ static int export_multilayer(HEVCContext *s, const HEVCVPS *vps) >>> return 0; >>> } >>> >>> +int ff_hevc_is_alpha_video(const HEVCContext *s) >>> +{ >>> + const HEVCVPS *vps = s->vps; >>> + int ret = 0; >>> + >>> + if (vps->nb_layers != 2 || !vps->layer_id_in_nuh[1]) >>> + return 0; >>> + >>> + /* decode_vps_ext() guarantees that SCALABILITY_AUXILIARY with AuxId other >>> + * than alpha cannot reach here. >>> + */ >>> + ret = (s->vps->scalability_mask_flag & HEVC_SCALABILITY_AUXILIARY); >>> + >>> + av_log(s->avctx, AV_LOG_DEBUG, "Multi layer video, %s alpha video\n", >>> + ret ? "is" : "not"); >>> + >>> + return ret; >>> +} >>> + >>> static int setup_multilayer(HEVCContext *s, const HEVCVPS *vps) >>> { >>> unsigned layers_active_output = 0, highest_layer; >>> @@ -473,6 +492,18 @@ static int setup_multilayer(HEVCContext *s, const HEVCVPS *vps) >>> s->layers_active_output = 1; >>> s->layers_active_decode = 1; >>> >>> + if (ff_hevc_is_alpha_video(s)) { >>> + const AVPixFmtDescriptor *desc = av_pix_fmt_desc_get(s->avctx->pix_fmt); >>> + >>> + if (!(desc->flags & AV_PIX_FMT_FLAG_ALPHA)) >>> + return 0; >>> + >>> + s->layers_active_decode = (1 << vps->nb_layers) - 1; >>> + s->layers_active_output = 1; >>> + >>> + return 0; >>> + } >>> + >>> // nothing requested - decode base layer only >>> if (!s->nb_view_ids) >>> return 0; >>> @@ -530,6 +561,34 @@ static int setup_multilayer(HEVCContext *s, const HEVCVPS *vps) >>> return 0; >>> } >>> >>> +static enum AVPixelFormat map_to_alpha_format(HEVCContext *s, >>> + enum AVPixelFormat pix_fmt) >>> +{ >>> + switch (pix_fmt) { >>> + case AV_PIX_FMT_YUV420P: >>> + case AV_PIX_FMT_YUVJ420P: >>> + return AV_PIX_FMT_YUVA420P; >>> + case AV_PIX_FMT_YUV420P10: >>> + return AV_PIX_FMT_YUVA420P10; >>> + case AV_PIX_FMT_YUV444P: >>> + return AV_PIX_FMT_YUVA444P; >>> + case AV_PIX_FMT_YUV422P: >>> + return AV_PIX_FMT_YUVA422P; >>> + case AV_PIX_FMT_YUV422P10LE: >>> + return AV_PIX_FMT_YUVA422P10LE; >>> + case AV_PIX_FMT_YUV444P10: >>> + return AV_PIX_FMT_YUVA444P10; >>> + case AV_PIX_FMT_YUV444P12: >>> + return AV_PIX_FMT_YUVA444P12; >>> + case AV_PIX_FMT_YUV422P12: >>> + return AV_PIX_FMT_YUVA422P12; >>> + default: >>> + av_log(s->avctx, AV_LOG_WARNING, "No alpha pixel format map for %s\n", >>> + av_get_pix_fmt_name(pix_fmt)); >>> + return AV_PIX_FMT_NONE; >>> + } >>> +} >>> + >>> static enum AVPixelFormat get_format(HEVCContext *s, const HEVCSPS *sps) >>> { >>> #define HWACCEL_MAX (CONFIG_HEVC_DXVA2_HWACCEL + \ >>> @@ -540,9 +599,13 @@ static enum AVPixelFormat get_format(HEVCContext *s, const HEVCSPS *sps) >>> CONFIG_HEVC_VIDEOTOOLBOX_HWACCEL + \ >>> CONFIG_HEVC_VDPAU_HWACCEL + \ >>> CONFIG_HEVC_VULKAN_HWACCEL) >>> - enum AVPixelFormat pix_fmts[HWACCEL_MAX + 2], *fmt = pix_fmts; >>> + enum AVPixelFormat pix_fmts[HWACCEL_MAX + 3], *fmt = pix_fmts; >>> + enum AVPixelFormat alpha_fmt = AV_PIX_FMT_NONE; >>> int ret; >>> >>> + if (ff_hevc_is_alpha_video(s)) >>> + alpha_fmt = map_to_alpha_format(s, sps->pix_fmt); >>> + >>> switch (sps->pix_fmt) { >>> case AV_PIX_FMT_YUV420P: >>> case AV_PIX_FMT_YUVJ420P: >>> @@ -664,6 +727,8 @@ static enum AVPixelFormat get_format(HEVCContext *s, const HEVCSPS *sps) >>> break; >>> } >>> >>> + if (alpha_fmt != AV_PIX_FMT_NONE) >>> + *fmt++ = alpha_fmt; >>> *fmt++ = sps->pix_fmt; >>> *fmt = AV_PIX_FMT_NONE; >>> >>> @@ -3194,6 +3259,12 @@ static int hevc_frame_start(HEVCContext *s, HEVCLayerContext *l, >>> !sps->vui.common.video_signal_type_present_flag) >>> pix_fmt = sps_base->pix_fmt; >>> >>> + // Ignore range mismatch between base layer and alpha layer >>> + if (ff_hevc_is_alpha_video(s) && >>> + sps_base->pix_fmt == AV_PIX_FMT_YUV420P && >>> + pix_fmt == AV_PIX_FMT_YUVJ420P) >>> + pix_fmt = sps_base->pix_fmt; >>> + >>> if (pix_fmt != sps_base->pix_fmt || >>> sps->width != sps_base->width || >>> sps->height != sps_base->height) { >>> diff --git a/libavcodec/hevc/hevcdec.h b/libavcodec/hevc/hevcdec.h >>> index 4e95035688..b2b725b5cd 100644 >>> --- a/libavcodec/hevc/hevcdec.h >>> +++ b/libavcodec/hevc/hevcdec.h >>> @@ -714,6 +714,8 @@ void ff_hevc_hls_residual_coding(HEVCLocalContext *lc, const HEVCPPS *pps, >>> >>> void ff_hevc_hls_mvd_coding(HEVCLocalContext *lc, int x0, int y0, int log2_cb_size); >>> >>> +int ff_hevc_is_alpha_video(const HEVCContext *s); >>> + >>> extern const uint8_t ff_hevc_qpel_extra_before[4]; >>> extern const uint8_t ff_hevc_qpel_extra_after[4]; >>> extern const uint8_t ff_hevc_qpel_extra[4]; >>> diff --git a/libavcodec/hevc/refs.c b/libavcodec/hevc/refs.c >>> index dd7f7f95a8..da0bac8d9c 100644 >>> --- a/libavcodec/hevc/refs.c >>> +++ b/libavcodec/hevc/refs.c >>> @@ -79,6 +79,30 @@ void ff_hevc_flush_dpb(HEVCContext *s) >>> } >>> } >>> >>> +static int replace_alpha_plane(AVFrame *alpha, AVFrame *base) >>> +{ >>> + AVBufferRef *base_a = av_frame_get_plane_buffer(base, 3); >>> + uint8_t *data = alpha->data[0]; >>> + int ret; >>> + >>> + for (int i = 0; i < FF_ARRAY_ELEMS(alpha->buf) && alpha->buf[i]; i++) { >>> + AVBufferRef *buf = alpha->buf[i]; >>> + >>> + if (data >= buf->data && data < buf->data + buf->size) { >> >> Pointer comparisons (other than equality/inquality comparisons) are >> undefined behavior unless the two pointers point into the same array (or >> one past the last element of an array). > > av_frame_get_plane_buffer does the same compare. Any idea to fix this UB? ISO C does not have a portable way to achieve this (except iterating over buf->data[0..buf->size] and checking via ==). In practice (in a flat memory space) casting to uintptr_t should work. > >> >>> + ret = av_buffer_replace(&alpha->buf[i], base_a); >>> + if (ret < 0) >>> + return ret; >>> + >>> + alpha->linesize[0] = base->linesize[3]; >>> + alpha->data[0] = base->data[3]; >>> + >>> + return 0; >>> + } >>> + } >>> + >>> + return AVERROR_BUG; >>> +} >>> + >>> static HEVCFrame *alloc_frame(HEVCContext *s, HEVCLayerContext *l) >>> { >>> const HEVCVPS *vps = l->sps->vps; >>> @@ -103,7 +127,7 @@ static HEVCFrame *alloc_frame(HEVCContext *s, HEVCLayerContext *l) >>> } >>> >>> // add view ID side data if it's nontrivial >>> - if (vps->nb_layers > 1 || view_id) { >>> + if (!ff_hevc_is_alpha_video(s) && (vps->nb_layers > 1 || view_id)) { >>> HEVCSEITDRDI *tdrdi = &s->sei.tdrdi; >>> AVFrameSideData *sd = av_frame_side_data_new(&frame->f->side_data, >>> &frame->f->nb_side_data, >>> @@ -161,7 +185,15 @@ static HEVCFrame *alloc_frame(HEVCContext *s, HEVCLayerContext *l) >>> if (ret < 0) >>> goto fail; >>> >>> + >>> frame->pps = av_refstruct_ref_c(s->pps); >>> + if (l != &s->layers[0] && ff_hevc_is_alpha_video(s)) { >>> + AVFrame *alpha = frame->f; >>> + AVFrame *base = s->layers[0].cur_frame->f; >>> + ret = replace_alpha_plane(alpha, base); >>> + if (ret < 0) >>> + goto fail; >>> + } >>> >>> return frame; >>> fail: >> >> _______________________________________________ >> ffmpeg-devel mailing list >> ffmpeg-devel@ffmpeg.org >> https://ffmpeg.org/mailman/listinfo/ffmpeg-devel >> >> To unsubscribe, visit link above, or email >> ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe". > > _______________________________________________ > ffmpeg-devel mailing list > ffmpeg-devel@ffmpeg.org > https://ffmpeg.org/mailman/listinfo/ffmpeg-devel > > To unsubscribe, visit link above, or email > ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe". _______________________________________________ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".