From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org [79.124.17.100]) by master.gitmailbox.com (Postfix) with ESMTP id AB8FA4459D for ; Fri, 21 Oct 2022 07:43:02 +0000 (UTC) Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id C83B768BE4F; Fri, 21 Oct 2022 10:42:59 +0300 (EEST) Received: from NAM12-BN8-obe.outbound.protection.outlook.com (mail-bn8nam12olkn2020.outbound.protection.outlook.com [40.92.21.20]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 20CC768BA40 for ; Fri, 21 Oct 2022 10:42:53 +0300 (EEST) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=OYCqcFhUxwPQ1dqxD7iQ3lIwaRQ1doNKC7RH+Ud+7xieRrUKWQ06mBB1OtXoeCmQDBShLlnFqkblCcmPrtQTAF/vAcH4PsQb9q5eViCHa01zSJsDJZJD4737R1O7XXcn6/tH2D7roQdi8Btme00uWY5jChXsx8KXQNwqyINhLC7IGcVLJ9oGqY7YW8H6JWFuqZ5RqskLYxbi4cQs2OMv+SMeJtADyVJLg5i4xCbBd0ZidqsjUF/cVhhIvjnv/gIxErv2uMB9HU8PAYwPdjIfuT1qhfJCKOq4BKeyqsNYT4JCRf8OAEu0puAc5SO0D5oXnaqlndanDDYLOvL2gvQFFA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=ViZAgWh/NrY6xUb0jFuy0qGkPP7EOpzksf1e8Svunj0=; b=GokabTQDK6xbjKoXolFo1kleIeNlpYZAH8yhy7saMbN0WgzTViRcg05VsTx04puNUwC0EYByjga737+k3ioH9DSqoY2xAsIopaCp/lFxiCKfYWhE7c0bO0QmW9JECmGtc7WrNbV55Nmdgmn79mK2K2ZKf1h8z8CiqCVPmonn8ZjMliSJwadNOQt4N+PbjkMqRINfkvUc8keKQgUNjYFVysLvpRagUwrYNLol2C6gfmQkjG6iMSa1zokY+AIWvOYb2ZsNJIP22ZUU+KaATJjty5FeJYVYhg0+U0JS2qQEA3bEzJJ8dVjBQ2NzovSoPs7UdRB/gnxomVjXoeXhwqGadw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=none; dmarc=none; dkim=none; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=hotmail.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=ViZAgWh/NrY6xUb0jFuy0qGkPP7EOpzksf1e8Svunj0=; b=cfBQFxvcxpBVbbdQYyiPrEGPjkzDsISDK0egv86ZN6HDVgoFqmZxDpsVpa64WXJN6mBT6hOXheBszDQ8SscZrrD+1uzHW0/p05XTNtteZLGgYSyGEu17DZAPyp6/vnywpyLV5hC1WyHfaWcqV5qPL05FlV7dBSZTZjwvceWYjnYKAbDpqzHl0NesdLs6BQzQULf/xcgcyFNppuKvjBaEvybBrPa21FXBoAGuZrkFWTzwcv2t9QlL7zJJ1FdVNd7TS2RYfMqVe3wrbRg0N/SACwQOg4DFCWhx6Zdr37ISWvwYuDDQ+gT9sUIMaxRZ3kxkI5/aJ3CH6ylS6sf3muXU2w== Received: from BN0P223MB0358.NAMP223.PROD.OUTLOOK.COM (2603:10b6:408:145::16) by MW4P223MB0612.NAMP223.PROD.OUTLOOK.COM (2603:10b6:303:20e::9) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5723.34; Fri, 21 Oct 2022 07:42:49 +0000 Received: from BN0P223MB0358.NAMP223.PROD.OUTLOOK.COM ([fe80::a0b4:5dce:92a2:8c14]) by BN0P223MB0358.NAMP223.PROD.OUTLOOK.COM ([fe80::a0b4:5dce:92a2:8c14%7]) with mapi id 15.20.5723.034; Fri, 21 Oct 2022 07:42:48 +0000 From: Soft Works To: FFmpeg development discussions and patches , "Xiang, Haihao" Thread-Topic: [FFmpeg-devel] [PATCH v5 0/6] Implement SEI parsing for QSV decoders Thread-Index: AQHYjYv91+b3D1Tr2UWOflh6ddwYuq2FXfIAgAQgYwCAj6NIoA== Date: Fri, 21 Oct 2022 07:42:48 +0000 Message-ID: References: <57cda049766225c291fae75c31d651fdff57f7a2.camel@intel.com> In-Reply-To: Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-tmn: [K6SSsnj0c28FL6Ku95oLdHiz0ZcYgFpS] x-ms-publictraffictype: Email x-ms-traffictypediagnostic: BN0P223MB0358:EE_|MW4P223MB0612:EE_ x-ms-office365-filtering-correlation-id: 902b035e-507d-44c3-fed7-08dab337d9e8 x-microsoft-antispam: BCL:0; x-microsoft-antispam-message-info: 9s7wNwnYLElrtLF75CYweI9Zsab9sF8GkxMiMqWQ5ioFm1xiw54pH8btNJjln62HBp9Zl1SjwM7pWyuxm2qzmCKz68ECQuYxiX+cNvTM9bia7+u8khBzf12s79uJifdi1KDml1cuMekEoKdTVmNJMV+9UqT7UvVdBZ2OfHcQnQwvFDbAY+xn5oLAgsLzG6PGzarFkvlFvTJRCU6OK3Ly+Y6ecu1YGh3rJ35DNP5pcVibQ1t0r35GJ9Zkv7spBf/dfAw2fH8OCDPugjwNtvZanrPC/dUVTAvvfty4ORdzrdma9bGcN/xpcoEW0K/wdRmEuKsMYCzofkRVVDb1dqvlyDI+Ex7eZgW7uIQmOs4ZhlFvsWbS3283vh2+mmHEgBKCnwQBXc+u764ouoyAMnW2dYS6VPOBciU3XFyxGQqdQfAg9C3kA/Gbq4CcuZ6dXyTMxb0KSMboDFzJysSpYFasrpaQu0vjIY670jcweQuSXcg1cHY8XKZokDFf9HN4YXN6H4XAx5PoDMOWgodkRn2ief4X3NNY4yZ+geMp75G03cUR3DE/Vo5Fx8VrpDvBpxWd4R0WOyaYU46YzSWYpcYcuXUJBH0yaIFRgtuUdiCrIT24MBvCQi+TBvVQ0Cv7XSVyINbgOWYhtHlTlxsQw4fBfDfwlYy2eNQc0BYqxEM/GQw+3e/ZXwOK8SNmbvTebnV//x+rm0zvv35rpQTmfDopVQ== x-ms-exchange-antispam-messagedata-chunkcount: 1 x-ms-exchange-antispam-messagedata-0: =?utf-8?B?dXI4VUlnajBSRWxKalM5Wjc3U0Vva2Y0OXFMeUNuVDdlTCtRbldVcTFIR1c3?= =?utf-8?B?UUM2UTVza2QzYTMwR2pqdUVLMVdJa0VwQTZLZjlGUzc2WlFOU0tuZ1lDQWpE?= =?utf-8?B?MG8yaXV4Q2NZNnc1UGlDQll5MlBrZmZ1QXJQclI0M0dSU0F3YWdxK3o5Z09a?= =?utf-8?B?RWQyT2NoT3RJUEtDa09wSkdHbnZnMkprQlpPUlBSeUNNdG10VEticGFnWWhM?= =?utf-8?B?SG84ekxjenkwMWpqVXkvSFJPeVRHU1VoZTNXUGdTOWVGdW1NeXJXQzh5amlp?= =?utf-8?B?TjNNRVp5SDkvcU1YamhGdEYreDhPMFBteS84MU1mRVBBc1FYQjBPSm5uVGxx?= =?utf-8?B?bzJETm9tUXpFWmNqN3duNEpwUnk2OU5TRHN2YkRGcnVtbU43cE9MZlpxeXlU?= =?utf-8?B?MGI4RXBZVFhLa3RGV3haSjJFdTRzc0NCYUR1WlZrN2pJZml3eW9hci9obFVx?= =?utf-8?B?Z0VpSTNhWW5hU0tFUjg3KzVxK2pLUkE3L0IwTCtRZWtFTHF5ZVRwZmZLZjhJ?= =?utf-8?B?TmVDcDdBeXI1WDF1RG1QWmp6KzUzUWdOZlNOWkpyVWQzNEMram9iVWxITGxv?= =?utf-8?B?b29GV1huUDl5SGVuN1ZjbHpNUjQrUjliRU5ieFB0UHdTQm43Q05tUEZEK25j?= =?utf-8?B?a0FlWm81Y0VCaXdlR0ZVRVJrNHZmeFIvNnFZYnJkck1OYXR3dG1jRHpVR0dt?= =?utf-8?B?VWRZNVRmVGxETEZtL3FTWUJNdy9FWVlFYnNYbzkzMzgvOXQxZDVCaE1Kd1l0?= =?utf-8?B?RDYreGhYcjZUaFlzNk9aS3p3T1ZEZWhPUHJ1OTFFTS9CMXZWQ2VaVUJHZE4w?= =?utf-8?B?NDBWQ3FLNnNmWkR2THgxUGtnM0tPM1ovOFhObm5ZTktHeFFvZUtFVkNSaGtL?= =?utf-8?B?ZzNJVS9SNnZ5ZCt4WUNpb2pSVUk1MFRKcUF6b0hmQnFXSUViY1NUTGgxcW5p?= =?utf-8?B?VHpabUp6QjFnU2t3NEVQakNCV3Jwd1RIYUZNN2IrL2Yzb1NLajJ6RXhIT25P?= =?utf-8?B?elhLU2RHcEFLWnFVV2NlQ0liQmJDMWNEY2tSdHdTeXJRNkdnQm1kQWp3VGJv?= =?utf-8?B?V1A1MTZyVmtxUTRYOGg5c21Tc1dMcEpYZlczOVQvYk04YmZ5cEJWYVFzMzdm?= =?utf-8?B?aXBOQWtGSFN2UEJVYVFpRklrMll6Z0tIT2xJUW1EUnczTkhNQTAxTDcyV0Ro?= =?utf-8?B?UmYxRHJ1czhObVhpL2hmblRRdFJRN2tYWU9KNjRvV3RvQTllQnIxVDRTNHZF?= =?utf-8?B?NWcrQWJwWnVXZm1iUisrQnVZWUxWYm54TzUwdEVabnR4cFA2SFA3bG9FTUc5?= =?utf-8?B?Z3lkbjE2U1lvTTdaVnhHQTFYckhnTkRpaG1McDRNNjV4UWZkRDl6RDkvblZY?= =?utf-8?B?V1E5bys0OE1MT29rSHFqZkYxRUVpUTlaTlV1ekl2Q1VOQnJQY09LdXF5czNM?= =?utf-8?B?dTJXMm5xU0tEUmdxU2dySWE5VlJ0d2NqSjkrSTFyRjBPbUswNGh6SXZ0Vkdl?= =?utf-8?B?aXBlYXc2SmhOSWxYRzJMNGRFN3l2V2xtNkttYkx3djRBN3B5YWx1ZHBKc0xV?= =?utf-8?B?Y3lIcjlzNDZBTWVkZzFHNzZWTFdBWjJlcVQ1OXVDZURzaWh3c3U1a1I4MGxu?= =?utf-8?B?amV3Vnp6VEhySDRZTWg1SGkwMTB1SXBFZ3VZUkZSOGs5RTZOSTRBQjF2SjV0?= =?utf-8?Q?YGK3K20IaPWQNOvd/E9d?= MIME-Version: 1.0 X-OriginatorOrg: sct-15-20-4755-11-msonline-outlook-1ff67.templateTenant X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-AuthSource: BN0P223MB0358.NAMP223.PROD.OUTLOOK.COM X-MS-Exchange-CrossTenant-RMS-PersistedConsumerOrg: 00000000-0000-0000-0000-000000000000 X-MS-Exchange-CrossTenant-Network-Message-Id: 902b035e-507d-44c3-fed7-08dab337d9e8 X-MS-Exchange-CrossTenant-originalarrivaltime: 21 Oct 2022 07:42:48.6220 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 84df9e7f-e9f6-40af-b435-aaaaaaaaaaaa X-MS-Exchange-CrossTenant-rms-persistedconsumerorg: 00000000-0000-0000-0000-000000000000 X-MS-Exchange-Transport-CrossTenantHeadersStamped: MW4P223MB0612 Subject: Re: [FFmpeg-devel] [PATCH v5 0/6] Implement SEI parsing for QSV decoders X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" Archived-At: List-Archive: List-Post: > -----Original Message----- > From: ffmpeg-devel On Behalf Of > Andreas Rheinhardt > Sent: Thursday, July 21, 2022 11:56 PM > To: Xiang, Haihao ; ffmpeg-devel@ffmpeg.org > Subject: Re: [FFmpeg-devel] [PATCH v5 0/6] Implement SEI parsing for > QSV decoders > > Xiang, Haihao: > > On Fri, 2022-07-01 at 20:48 +0000, ffmpegagent wrote: > >> Missing SEI information has always been a major drawback when > using the QSV > >> decoders. I used to think that there's no chance to get at the > data without > >> explicit implementation from the MSDK side (or doing something > weird like > >> parsing in parallel). It turned out that there's a hardly known > api method > >> that provides access to all SEI (h264/hevc) or user data > (mpeg2video). > >> > >> This allows to get things like closed captions, frame packing, > display > >> orientation, HDR data (mastering display, content light level, > etc.) without > >> having to rely on those data being provided by the MSDK as > extended buffers. > >> > >> The commit "Implement SEI parsing for QSV decoders" includes some > hard-coded > >> workarounds for MSDK bugs which I reported: > >> > > https://github.com/Intel-Media- > SDK/MediaSDK/issues/2597#issuecomment-1072795311 > >> > >> But that doesn't help. Those bugs exist and I'm sharing my > workarounds, > >> which are empirically determined by testing a range of files. If > someone is > >> interested, I can provide private access to a repository where we > have been > >> testing this. Alternatively, I could also leave those workarounds > out, and > >> just skip those SEI types. > >> > >> In a previous version of this patchset, there was a concern that > payload > >> data might need to be re-ordered. Meanwhile I have researched this > carefully > >> and the conclusion is that this is not required. > >> > >> My detailed analysis can be found here: > >> https://gist.github.com/softworkz/36c49586a8610813a32270ee3947a932 > >> > >> v4 > >> > >> * add new dependencies in makefile Now, build still works when > someone uses > >> configure --disable-decoder=h264 --disable-decoder=hevc > >> --disable-decoder=mpegvideo --disable-decoder=mpeg1video > >> --disable-decoder=mpeg2video --enable-libmfx > >> > >> v3 > >> > >> * frame.h: clarify doc text for av_frame_copy_side_data() > >> > >> v2 > >> > >> * qsvdec: make error handling consistent and clear > >> * qsvdec: remove AV_CODEC_ID_MPEG1VIDEO constants > >> * hevcdec: rename function to ff_hevc_set_side_data(), add doc > text > >> > >> v3 > >> > >> * qsvdec: fix c/p error > >> > >> softworkz (6): > >> avutil/frame: Add av_frame_copy_side_data() and > >> av_frame_remove_all_side_data() > >> avcodec/vpp_qsv: Copy side data from input to output frame > >> avcodec/mpeg12dec: make mpeg_decode_user_data() accessible > >> avcodec/hevcdec: make set_side_data() accessible > >> avcodec/h264dec: make h264_export_frame_props() accessible > >> avcodec/qsvdec: Implement SEI parsing for QSV decoders > >> > >> doc/APIchanges | 4 + > >> libavcodec/Makefile | 6 +- > >> libavcodec/h264_slice.c | 98 ++++++++------- > >> libavcodec/h264dec.h | 2 + > >> libavcodec/hevcdec.c | 117 +++++++++--------- > >> libavcodec/hevcdec.h | 9 ++ > >> libavcodec/hevcdsp.c | 4 + > >> libavcodec/mpeg12.h | 28 +++++ > >> libavcodec/mpeg12dec.c | 40 +----- > >> libavcodec/qsvdec.c | 234 > +++++++++++++++++++++++++++++++++++ > >> libavfilter/qsvvpp.c | 6 + > >> libavfilter/vf_overlay_qsv.c | 19 ++- > >> libavutil/frame.c | 67 ++++++---- > >> libavutil/frame.h | 32 +++++ > >> libavutil/version.h | 2 +- > >> 15 files changed, 494 insertions(+), 174 deletions(-) > >> > >> > >> base-commit: 6a82412bf33108111eb3f63076fd5a51349ae114 > >> Published-As: > >> https://github.com/ffstaging/FFmpeg/releases/tag/pr-ffstaging- > 31%2Fsoftworkz%2Fsubmit_qsv_sei-v5 > >> Fetch-It-Via: git fetch https://github.com/ffstaging/FFmpeg pr- > ffstaging- > >> 31/softworkz/submit_qsv_sei-v5 > >> Pull-Request: https://github.com/ffstaging/FFmpeg/pull/31 > >> > >> Range-diff vs v4: > >> > >> 1: 7656477360 = 1: 7656477360 avutil/frame: Add > av_frame_copy_side_data() > >> and av_frame_remove_all_side_data() > >> 2: 06976606c5 = 2: 06976606c5 avcodec/vpp_qsv: Copy side data > from input to > >> output frame > >> 3: 320a8a535c = 3: 320a8a535c avcodec/mpeg12dec: make > >> mpeg_decode_user_data() accessible > >> 4: e58ad6564f = 4: e58ad6564f avcodec/hevcdec: make > set_side_data() > >> accessible > >> 5: a57bfaebb9 = 5: 4c0b6eb4cb avcodec/h264dec: make > >> h264_export_frame_props() accessible > >> 6: 3f2588563e ! 6: 19bc00be4d avcodec/qsvdec: Implement SEI > parsing for QSV > >> decoders > >> @@ Commit message > >> > >> Signed-off-by: softworkz > >> > >> + ## libavcodec/Makefile ## > >> +@@ libavcodec/Makefile: OBJS-$(CONFIG_MSS34DSP) > += > >> mss34dsp.o > >> + OBJS-$(CONFIG_PIXBLOCKDSP) += pixblockdsp.o > >> + OBJS-$(CONFIG_QPELDSP) += qpeldsp.o > >> + OBJS-$(CONFIG_QSV) += qsv.o > >> +-OBJS-$(CONFIG_QSVDEC) += qsvdec.o > >> ++OBJS-$(CONFIG_QSVDEC) += qsvdec.o > h264_slice.o > >> h264_cabac.o h264_cavlc.o \ > >> ++ h264_direct.o > h264_mb.o > >> h264_picture.o h264_loopfilter.o \ > >> ++ h264dec.o > h264_refs.o cabac.o > >> hevcdec.o hevc_refs.o \ > >> ++ > > >> hevc_filter.o hevc_cabac.o hevc_mvs.o hevcpred.o hevcdsp.o \ > >> ++ > > >> h274.o dovi_rpu.o mpeg12dec.o > >> + OBJS-$(CONFIG_QSVENC) += qsvenc.o > >> + OBJS-$(CONFIG_RANGECODER) += rangecoder.o > >> + OBJS-$(CONFIG_RDFT) += rdft.o > >> + > >> + ## libavcodec/hevcdsp.c ## > >> +@@ > >> + * Foundation, Inc., 51 Franklin Street, Fifth Floor, > Boston, MA 02110- > >> 1301 USA > >> + */ > >> + > >> ++#include "config_components.h" > >> ++ > >> + #include "hevcdsp.h" > >> + > >> + static const int8_t transform[32][32] = { > >> +@@ libavcodec/hevcdsp.c: int i = 0; > >> + break; > >> + } > >> + > >> ++#if CONFIG_HEVC_DECODER > >> + #if ARCH_AARCH64 > >> + ff_hevc_dsp_init_aarch64(hevcdsp, bit_depth); > >> + #elif ARCH_ARM > >> +@@ libavcodec/hevcdsp.c: int i = 0; > >> + #elif ARCH_LOONGARCH > >> + ff_hevc_dsp_init_loongarch(hevcdsp, bit_depth); > >> + #endif > >> ++#endif > >> + } > >> + > >> ## libavcodec/qsvdec.c ## > >> @@ > >> #include "hwconfig.h" > > > > > > Is there any comment on this patchset ? If not, I'd like to merge > it to make QSV > > decoders works with SEI info. > > > > Thanks > > Haihao > > > > This patchset has several issues, namely: > 1. It tries to share the functions that are used for processing > user/SEI > data as they are, even the parts that are not intended to be used by > QSV > (like the picture structure stuff for H.264 or tmpgexs in case of > MPEG-1/2). > 2. It tries to keep the functions where they are, leading to the > insanely long Makefile line in patch 6/6 (which I believe to be still > incomplete: mpeg12dec.o pulls in mpegvideo.o mpegvideo_dec.o (which > in > turn pull in lots of dsp stuff) and where is h264dsp.o? (it seems > like > there is a reliance on the H.264 parser for this)). This is the > opposite > of modularity. > 3. It just puts a huge Mpeg1Context in the QSVContext, although only > a > miniscule part of it is actually used. One should use a small context > of > its own instead. > 4. It does not take into account that buffers need to be padded to be > usable by the GetBit-API. Hi Andreas, thanks for pointing out (4), in fact I wasn't aware of this. I agree to your other points (1-3). Not that I wouldn't have been aware of those implications, I've just been afraid that larger refactorings could have minimized acceptance. > (I have made an attempt to factor out the common parts of H.264 and > H.265 SEI handling, which should make this here much easier.) Your patchset would in fact be very helpful and allow me to provide a much better and focused revision. Though, it is still pending at this time - are you planning to push it? Thanks, softworkz _______________________________________________ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".