From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org [79.124.17.100]) by master.gitmailbox.com (Postfix) with ESMTP id 16AB0436BC for ; Thu, 21 Jul 2022 21:56:36 +0000 (UTC) Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 2D5F068B68F; Fri, 22 Jul 2022 00:56:34 +0300 (EEST) Received: from EUR04-DB3-obe.outbound.protection.outlook.com (mail-oln040092074011.outbound.protection.outlook.com [40.92.74.11]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id A924568A5E1 for ; Fri, 22 Jul 2022 00:56:27 +0300 (EEST) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=cgI6NyhlAN0noxByjvLralHRp+FjobxMrGpvYh7ntALjHgp98i4h3LPmjtGjwWU3QOP+iu1XkwZ3p+FPClxFcyBmm80wiGTBnX1MYH8tHiZbu/dfYy+9Pcjm7309SRrI3xxUOXhdPrl2qbIuipWDPg3EgGZrhXM+QqA5ANreBL6nh6sopOdMHr4ZSn3OjGXP2mcp3wusMPICrFum1rpvD48kGjMkbliCrejSzjDVDsm1n94ZePlkmWJFoJkby8xnzX7IphOBuh34nGjSx5Z3H21mmuQdAxuNyi93g7wRvUD0emOv0brhGkzz4eMg3NJx9vxTYhXFtvp6tC+QjreWgg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=is0OVVKaxwX1i1VJ59S5sUFiVRih2GaA4NBHYcted0M=; b=U5V8htPOq4yLImq0h3we4DYGBeHvtdiWY+nPv7Ea4Ia9v5s3lDMalnsKD1EtHrc0cvS0dQ2Vdu8S/v4kQr1a4VMH2ALzpOW02+yUdO2XvQbO48W1cp5QogiN/0HF4Qbk3OFa0DtLmSerEaB1j95X82Op4zmy2JrKGp3dojJzLbqDGj2fOpvfgrHrTf1c/55zQherq3ImWxbolgS47fs1cUY83mDWpki3nyfdZZknB3mXgU63vCxIvJ2R5kG/w7k5z5C4oD4VYtyGeDCW2MV44G9M4atkLmtlxNcSK+l//U36yZqGzaOL6hwiRWQoC15IBxPc0dlQ5/qtTjTvQF+Wew== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=none; dmarc=none; dkim=none; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=outlook.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=is0OVVKaxwX1i1VJ59S5sUFiVRih2GaA4NBHYcted0M=; b=S3VyRrkrbT0tN5VgUfH1YbL8ZpR7DvDIHnpNcNBm38vd0zCjEf3qSeJ6fKlk3nliuw7e3Hndf8QnpHjMpHwjvYR7O/g8oDEgQGAPEa5+JKxrKTYikQKOSk3UqqwUbgkhfTQNH09ctK4NpK5T9iPFxOhBDSBEIQjbvoYHQfGY8p6fpy+1lYQhiuIa+vSHiHo6bJkz7lXOPuB9dzsgIuXsMK7ax41WM+OTTrmswDXrlWC2br8qttRGKba9WOzBf/Ta1w5D3UxDxws2R4uxk+8REHhC3hj80AEXVQBjkJQnaXI8Fo/JFLewNkJJxCquDr7aMn/hkDbcMb4Zl0j9nsL2+A== Received: from DB6PR0101MB2214.eurprd01.prod.exchangelabs.com (2603:10a6:4:42::27) by AM0PR0102MB3393.eurprd01.prod.exchangelabs.com (2603:10a6:208:9::12) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5458.18; Thu, 21 Jul 2022 21:56:24 +0000 Received: from DB6PR0101MB2214.eurprd01.prod.exchangelabs.com ([fe80::210e:b627:bcc9:8c46]) by DB6PR0101MB2214.eurprd01.prod.exchangelabs.com ([fe80::210e:b627:bcc9:8c46%11]) with mapi id 15.20.5438.023; Thu, 21 Jul 2022 21:56:24 +0000 Message-ID: Date: Thu, 21 Jul 2022 23:56:22 +0200 Content-Language: en-US To: "Xiang, Haihao" , "ffmpeg-devel@ffmpeg.org" References: <57cda049766225c291fae75c31d651fdff57f7a2.camel@intel.com> From: Andreas Rheinhardt In-Reply-To: <57cda049766225c291fae75c31d651fdff57f7a2.camel@intel.com> X-TMN: [5X+FU1Yn6Qs8WyAOt6pCraoSFWaj1Xd3] X-ClientProxiedBy: ZR0P278CA0065.CHEP278.PROD.OUTLOOK.COM (2603:10a6:910:21::16) To DB6PR0101MB2214.eurprd01.prod.exchangelabs.com (2603:10a6:4:42::27) X-Microsoft-Original-Message-ID: MIME-Version: 1.0 X-MS-Exchange-MessageSentRepresentingType: 1 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: a5a2f70b-9e06-431e-aa6f-08da6b63dad3 X-MS-TrafficTypeDiagnostic: AM0PR0102MB3393:EE_ X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: C4R38bIM3fa8xa3kfBi5b+EJ+uApGQpwWyPfxKy4E1rq5FJhV2AKBIsEcZTC9YhYJOjXmXnXcdqbbQRADGCJ+nRVeUTpKwI0ebbFknSx9ybJ1d7/nO4Lhu3CHWvXBZvW4P9GLhGIdd39dyv2R1uswlcZJ9Rcd3j6zOn5XzsqPNbe9snDJBFucMAN+MCeIZ+pyrPMTsup/1GSmsc7I1W0jp+sGlJ7zKYd106pS04WKauDZZSZq326ov/GdLyJleImHtK7RYCAwvwoAfxGT+RflnS/8djgX/3VDiu4qLTK0eZ/BmkRG+mwK8f9LwpWQ2oim7UQKnx+UxduxZ/Jl1XwSlPsV9J6T1ulWZt+C5hJm9L2jqKIH0r17wXGYv9Qk99YddbzRHl7TrHdAk4QPaq5JRPwLxiY7IzOz9pL2/z63EjEkFqWgqI03nihRzzyDsvsVXAVcC+rnCdUCnW9vvD1f714TIg89Y+suGqDh4IDTFnhSIPLmbKBdFe/nG2zAY208+nNBh1g1miTDIqPhOmMhsX1abtzLWXEUVwzNBKeKS8dIGEWK8tpsbLjZg1lbJX2Ojz1lMFU6AUConWh6rz1rGAjJnvdx56aE/+5Bw6Xj3kqie3eLxXVI2WpinMtfw66r/3sRSWFtOuCNvyZZh+UqQyrIDiwfFl9xrkDmub5Nrffd+ZcwFzmbKnGro/mkb1Zt4YL+ggIIIPUWCdFd+KCskhWxkZ75qucUgPg5nKLGOg= X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?B?amkyZTRVdHhWYkZOYTdvbnhaSm9hZGdVSDN3c2xUUlh0c3F2QUUraDlnVkNa?= =?utf-8?B?SmQ5MzZQTjBXQ1BHMzdSYi83RE9kTXpPckRtUm5pRGpEVGx5dlYvUVVBVTl4?= =?utf-8?B?Ymp3c0RpSXBiejJLaDFOV2c1eXB6aUcrcTBhYlZJamdQNnlSRzB2aWpRZHNq?= =?utf-8?B?MlJxOW1rcFBsbmFqYXVUUGZPNEtWYVVNa1c5ZzE5bFRWNmh5ZVc1cFIxQTVD?= =?utf-8?B?MjFCMEtFc2ZoclM1ZG5pR0k5NVBEc2lTY3VXMDY2Ym96ckVoS1J6S3BKYnJh?= =?utf-8?B?ZEc1anpFRTl2bmV5SUNkaUZibFB6QzZnMVlxWE13ZzJxU2lSUWtTUUxmWXpY?= =?utf-8?B?VDljQTEwNmZPWTR3cFB2NW1INWd4SXFIQzBCMG12cmZobUxzbi83d0VhNG9U?= =?utf-8?B?UE52WWdiNldTRDlxRTBIbDRZMmtzRlppb1dENXduN3NDbHZXdGJSWm5mNjlH?= =?utf-8?B?SEJlcFFnNWQ0cG1COXgyYkxNaHlOdGFCWjhFVHFEaDhwRGZUaGtMR0VWN0RJ?= =?utf-8?B?aC9EMjJoMDlUR0VzMTBmZ202WGd5ZlpTNTZqanVYcGxST1VBaUFuQ0E1eUFo?= =?utf-8?B?QkF1N1N1RWh6aEZaR2pYY2tZa3AvY2xUUk5ZVzYwaitBalRRM1cxcmcvWUJK?= =?utf-8?B?Q3BRTStacGNkN0hmTC9kM1NXaWF5RXIyQW9TejFnTmQydW5tRWV6YnVIZ0Fw?= =?utf-8?B?UTlObHoxalJmbktEWW81aXA5MDBZUGtIek1zZEdMbGduN2NZNDNuZStsTnl5?= =?utf-8?B?TnprcGhWVm5SVm1WWmhEQnFyaVlMT0Y5dDRZRXJXOXdKUlRPNUVLNllnQklX?= =?utf-8?B?TWxFWTZXM1kwT3R1V2U2ODFSYVZKM0FSY3EzUGk2NVhlZnBjZTdoVUxEUGtK?= =?utf-8?B?YVR6aHgwRmZSaXM1cGRqRldwZEk5S0MrVVJybmRXb2JPQk50KzdGamJ3K2Uy?= =?utf-8?B?a3hDNVB5anBaZWVjQ0RYVTJjRzBFWituWW8xUTFoVVh4YTlsRzhmUzZhbGNm?= =?utf-8?B?UmZJN0ZzQUh1T25LSllaRFF0cVV5SDkwalJ5T1FNUXFmZldBeEpmMUdtdXdx?= =?utf-8?B?ZTV0dE1VTzhXbmVyckd6ZkZ1Yi9yZHl3U0dFSmhvVm1QcjEveTM4dzhMZEE3?= =?utf-8?B?MG9RVy9GcERLN0J5Y1VVQ29LN3V5cUJycG8rUDJVUUJ1K3VyZ1VsSnN6Y1Rl?= =?utf-8?B?UkpHeXFZR2hMOXF1TVNiYStMWDVhck9kZVo3Qzg4bVNrSmVJalVVeVBscmFw?= =?utf-8?B?aEIzN1VoRFRUdmNrelpXYmorWWdWRG9uR1Fmc2c0aGFlNTVzcWViT1hDdnQ3?= =?utf-8?B?YStWanBRTzNUMXVuN1RibTJaaXhJc3I0Nm5Zbzg5enhtQ2pJa3dsa1hqR2k2?= =?utf-8?B?QWMvanBqeWh5TUhVZ3E2L0diS2ZWeTlLckd3WmVPMkFCUUNIYkppdjJKdkVJ?= =?utf-8?B?dVBJeHVhV1EvM01vVHc5bTBEQlBpUVp5aFdycTFTTWVreXBDbmZWaGRkN1Zv?= =?utf-8?B?SWExMElFZWQzSndLcWdsbEFGZXE2ZUZaNWlZT1pXQzhqRHIvSzNGelV0UHlt?= =?utf-8?B?OHhKV2RjMGVtUHpkZWhpb293aUgrczR5cmVybWRKbWphczVWUllBb2s4clQ3?= =?utf-8?B?TTkrSWExYnZvVEhacnd2c21HK08zbzI2RTZvbFV1enlDS3ZhcExFWi9xTzlo?= =?utf-8?B?TlZySXRxaGR2cklpUTVtUjg0RktYQ0ZMRUtEMGpVVi9BdkhsZUpKVzlpYTI4?= =?utf-8?Q?vZV3ftgfqd9/kdQYRg=3D?= X-OriginatorOrg: outlook.com X-MS-Exchange-CrossTenant-Network-Message-Id: a5a2f70b-9e06-431e-aa6f-08da6b63dad3 X-MS-Exchange-CrossTenant-AuthSource: DB6PR0101MB2214.eurprd01.prod.exchangelabs.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 21 Jul 2022 21:56:24.7335 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 84df9e7f-e9f6-40af-b435-aaaaaaaaaaaa X-MS-Exchange-CrossTenant-RMS-PersistedConsumerOrg: 00000000-0000-0000-0000-000000000000 X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM0PR0102MB3393 Subject: Re: [FFmpeg-devel] [PATCH v5 0/6] Implement SEI parsing for QSV decoders X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" Archived-At: List-Archive: List-Post: Xiang, Haihao: > On Fri, 2022-07-01 at 20:48 +0000, ffmpegagent wrote: >> Missing SEI information has always been a major drawback when using the QSV >> decoders. I used to think that there's no chance to get at the data without >> explicit implementation from the MSDK side (or doing something weird like >> parsing in parallel). It turned out that there's a hardly known api method >> that provides access to all SEI (h264/hevc) or user data (mpeg2video). >> >> This allows to get things like closed captions, frame packing, display >> orientation, HDR data (mastering display, content light level, etc.) without >> having to rely on those data being provided by the MSDK as extended buffers. >> >> The commit "Implement SEI parsing for QSV decoders" includes some hard-coded >> workarounds for MSDK bugs which I reported: >> > https://github.com/Intel-Media-SDK/MediaSDK/issues/2597#issuecomment-1072795311 >> >> But that doesn't help. Those bugs exist and I'm sharing my workarounds, >> which are empirically determined by testing a range of files. If someone is >> interested, I can provide private access to a repository where we have been >> testing this. Alternatively, I could also leave those workarounds out, and >> just skip those SEI types. >> >> In a previous version of this patchset, there was a concern that payload >> data might need to be re-ordered. Meanwhile I have researched this carefully >> and the conclusion is that this is not required. >> >> My detailed analysis can be found here: >> https://gist.github.com/softworkz/36c49586a8610813a32270ee3947a932 >> >> v4 >> >> * add new dependencies in makefile Now, build still works when someone uses >> configure --disable-decoder=h264 --disable-decoder=hevc >> --disable-decoder=mpegvideo --disable-decoder=mpeg1video >> --disable-decoder=mpeg2video --enable-libmfx >> >> v3 >> >> * frame.h: clarify doc text for av_frame_copy_side_data() >> >> v2 >> >> * qsvdec: make error handling consistent and clear >> * qsvdec: remove AV_CODEC_ID_MPEG1VIDEO constants >> * hevcdec: rename function to ff_hevc_set_side_data(), add doc text >> >> v3 >> >> * qsvdec: fix c/p error >> >> softworkz (6): >> avutil/frame: Add av_frame_copy_side_data() and >> av_frame_remove_all_side_data() >> avcodec/vpp_qsv: Copy side data from input to output frame >> avcodec/mpeg12dec: make mpeg_decode_user_data() accessible >> avcodec/hevcdec: make set_side_data() accessible >> avcodec/h264dec: make h264_export_frame_props() accessible >> avcodec/qsvdec: Implement SEI parsing for QSV decoders >> >> doc/APIchanges | 4 + >> libavcodec/Makefile | 6 +- >> libavcodec/h264_slice.c | 98 ++++++++------- >> libavcodec/h264dec.h | 2 + >> libavcodec/hevcdec.c | 117 +++++++++--------- >> libavcodec/hevcdec.h | 9 ++ >> libavcodec/hevcdsp.c | 4 + >> libavcodec/mpeg12.h | 28 +++++ >> libavcodec/mpeg12dec.c | 40 +----- >> libavcodec/qsvdec.c | 234 +++++++++++++++++++++++++++++++++++ >> libavfilter/qsvvpp.c | 6 + >> libavfilter/vf_overlay_qsv.c | 19 ++- >> libavutil/frame.c | 67 ++++++---- >> libavutil/frame.h | 32 +++++ >> libavutil/version.h | 2 +- >> 15 files changed, 494 insertions(+), 174 deletions(-) >> >> >> base-commit: 6a82412bf33108111eb3f63076fd5a51349ae114 >> Published-As: >> https://github.com/ffstaging/FFmpeg/releases/tag/pr-ffstaging-31%2Fsoftworkz%2Fsubmit_qsv_sei-v5 >> Fetch-It-Via: git fetch https://github.com/ffstaging/FFmpeg pr-ffstaging- >> 31/softworkz/submit_qsv_sei-v5 >> Pull-Request: https://github.com/ffstaging/FFmpeg/pull/31 >> >> Range-diff vs v4: >> >> 1: 7656477360 = 1: 7656477360 avutil/frame: Add av_frame_copy_side_data() >> and av_frame_remove_all_side_data() >> 2: 06976606c5 = 2: 06976606c5 avcodec/vpp_qsv: Copy side data from input to >> output frame >> 3: 320a8a535c = 3: 320a8a535c avcodec/mpeg12dec: make >> mpeg_decode_user_data() accessible >> 4: e58ad6564f = 4: e58ad6564f avcodec/hevcdec: make set_side_data() >> accessible >> 5: a57bfaebb9 = 5: 4c0b6eb4cb avcodec/h264dec: make >> h264_export_frame_props() accessible >> 6: 3f2588563e ! 6: 19bc00be4d avcodec/qsvdec: Implement SEI parsing for QSV >> decoders >> @@ Commit message >> >> Signed-off-by: softworkz >> >> + ## libavcodec/Makefile ## >> +@@ libavcodec/Makefile: OBJS-$(CONFIG_MSS34DSP) += >> mss34dsp.o >> + OBJS-$(CONFIG_PIXBLOCKDSP) += pixblockdsp.o >> + OBJS-$(CONFIG_QPELDSP) += qpeldsp.o >> + OBJS-$(CONFIG_QSV) += qsv.o >> +-OBJS-$(CONFIG_QSVDEC) += qsvdec.o >> ++OBJS-$(CONFIG_QSVDEC) += qsvdec.o h264_slice.o >> h264_cabac.o h264_cavlc.o \ >> ++ h264_direct.o h264_mb.o >> h264_picture.o h264_loopfilter.o \ >> ++ h264dec.o h264_refs.o cabac.o >> hevcdec.o hevc_refs.o \ >> ++ >> hevc_filter.o hevc_cabac.o hevc_mvs.o hevcpred.o hevcdsp.o \ >> ++ >> h274.o dovi_rpu.o mpeg12dec.o >> + OBJS-$(CONFIG_QSVENC) += qsvenc.o >> + OBJS-$(CONFIG_RANGECODER) += rangecoder.o >> + OBJS-$(CONFIG_RDFT) += rdft.o >> + >> + ## libavcodec/hevcdsp.c ## >> +@@ >> + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110- >> 1301 USA >> + */ >> + >> ++#include "config_components.h" >> ++ >> + #include "hevcdsp.h" >> + >> + static const int8_t transform[32][32] = { >> +@@ libavcodec/hevcdsp.c: int i = 0; >> + break; >> + } >> + >> ++#if CONFIG_HEVC_DECODER >> + #if ARCH_AARCH64 >> + ff_hevc_dsp_init_aarch64(hevcdsp, bit_depth); >> + #elif ARCH_ARM >> +@@ libavcodec/hevcdsp.c: int i = 0; >> + #elif ARCH_LOONGARCH >> + ff_hevc_dsp_init_loongarch(hevcdsp, bit_depth); >> + #endif >> ++#endif >> + } >> + >> ## libavcodec/qsvdec.c ## >> @@ >> #include "hwconfig.h" > > > Is there any comment on this patchset ? If not, I'd like to merge it to make QSV > decoders works with SEI info. > > Thanks > Haihao > This patchset has several issues, namely: 1. It tries to share the functions that are used for processing user/SEI data as they are, even the parts that are not intended to be used by QSV (like the picture structure stuff for H.264 or tmpgexs in case of MPEG-1/2). 2. It tries to keep the functions where they are, leading to the insanely long Makefile line in patch 6/6 (which I believe to be still incomplete: mpeg12dec.o pulls in mpegvideo.o mpegvideo_dec.o (which in turn pull in lots of dsp stuff) and where is h264dsp.o? (it seems like there is a reliance on the H.264 parser for this)). This is the opposite of modularity. 3. It just puts a huge Mpeg1Context in the QSVContext, although only a miniscule part of it is actually used. One should use a small context of its own instead. 4. It does not take into account that buffers need to be padded to be usable by the GetBit-API. (I have made an attempt to factor out the common parts of H.264 and H.265 SEI handling, which should make this here much easier.) - Andreas _______________________________________________ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".