From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org [79.124.17.100]) by master.gitmailbox.com (Postfix) with ESMTP id 903FE4375F for ; Mon, 27 Jun 2022 21:09:53 +0000 (UTC) Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 4BDDD68B8EB; Tue, 28 Jun 2022 00:09:51 +0300 (EEST) Received: from NAM11-CO1-obe.outbound.protection.outlook.com (mail-co1nam11olkn2024.outbound.protection.outlook.com [40.92.18.24]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 33D8168B755 for ; Tue, 28 Jun 2022 00:09:45 +0300 (EEST) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=U1/buRbF6CLLWVtPjvdGUTyOkwtP0qG5qv4VGJThKWg+JtsrcW6LHhGVPF2rkfiIL1ZQPfnDVN1TuJVWIs1vNXRnI2sUWQid/i+Mq4hVu9rYbGu3+1bosHgNtAEIecRoq08zUfhUp8C0CjZOcJsafz8B9R9kmYr6N7HLE97zaKa93RH+lGUR1Yx6lPQBIRzei5HaAF3eIpa8j6PzQT4Iyc1qMGRQf9buFktIb7mPwM+qeoQ9tIHcHgyfghoFAXHVxTvL01Wt1gYfQq9frbZxoge9apD23eV0MG1PZA4waqt7ZyZaJxsjO8PzfDwGg+Xb1I/GN3YXbSxJXdD00C+LDw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=ChTcDDX+O8XVFE008pVU/TIgBlzkmEWM7ft4XJ8SmB8=; b=llWzvQXk181xEAur1K872NnHH1goPBewdqS5Iw9hkRHrrzbY+2keRznxyk6p9f69YlAXgYotn9MqXiWhAtc8GfS/SGBd+f/6EVa/T7xWh++Dptet9tNe7pQPIGKNgoKOxgK846xp38P50E9oJrLKLBnSvfKAN7z3yILNbDYejHz86nfGjie3lggDalyGiz+SfD5hmVgg9z0tgQXZppqElI/UNlz2lzro+H5Ks34FvdxQ1wiNEQKSSmhy0yyjY9h3sl8nUFORCBWRPt6rTlAvRLFNyh6bFNVoY82TqnO02gFo6tFshqRm90bnTYhGIPAuC/0bs05rMlvm7nPXFRpXCg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=none; dmarc=none; dkim=none; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=hotmail.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=ChTcDDX+O8XVFE008pVU/TIgBlzkmEWM7ft4XJ8SmB8=; b=ASPhQwvfPY+2c3NsY6x0Rizd3mvGnz5b6KlCW3VZBeKlK/O6NPgAJoBmxH9fFYok7170VfK7ACrziC/T8X4iUHQRLvsYVzYQZzkhc8sjbmmvU3PVOToVrz7XsBx3kVKSVNWn7XjCe41EcMAaM6BYiXHYTi6yd5IPD+BMHv8NRblGWhb8jYHoCzYwA/c/CBeHY6m96pTKxTb0kY6l7bckeFdaUbtbUDe2DNMDhibN1sls7i0ZboNl2PWxmQUFjqQn+dlB6lafWV1V+gw14ZUTZMj4Xyv1Mld2Yrlqyw/6plPQKmKarkgvyfm5DpY/J/BzD5eBjMEsyiXwysnKLiexDQ== Received: from DM8P223MB0365.NAMP223.PROD.OUTLOOK.COM (2603:10b6:8:b::20) by BN0P223MB0231.NAMP223.PROD.OUTLOOK.COM (2603:10b6:408:15a::14) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5373.17; Mon, 27 Jun 2022 21:09:41 +0000 Received: from DM8P223MB0365.NAMP223.PROD.OUTLOOK.COM ([fe80::d9e4:ced6:ab31:c231]) by DM8P223MB0365.NAMP223.PROD.OUTLOOK.COM ([fe80::d9e4:ced6:ab31:c231%2]) with mapi id 15.20.5373.018; Mon, 27 Jun 2022 21:09:41 +0000 From: Soft Works To: FFmpeg development discussions and patches Thread-Topic: [FFmpeg-devel] [PATCH] libavcodec/hevc_mp4toannexb_bsf: insert extradata before non-AUD unit Thread-Index: AQHYOclKR/9cbLsBwUOHkCwe1WLL56zLKx+AgJkw+5A= Date: Mon, 27 Jun 2022 21:09:41 +0000 Message-ID: References: <20220317063546.7429-1-haihao.xiang@intel.com> In-Reply-To: Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-tmn: [diHeLYsdfXqyc4y8PE8uxmoj5K1T6WKQ] x-ms-publictraffictype: Email x-ms-office365-filtering-correlation-id: cd6973a0-9ee6-46f2-0651-08da58815a2b x-ms-traffictypediagnostic: BN0P223MB0231:EE_ x-microsoft-antispam: BCL:0; x-microsoft-antispam-message-info: ha3+L5e7pXimsyB8LMYWxkb/BqID9c2oQTOLQGHpD1zjSHrOrG3y2I2qy+Fkmy0+mSAH1SCtAdUngXWYaIA7lh/+DH1nEeyEcuDnUPhBiqcf7PxelyoQpmPC/g8i3qJdEWncCUrb0niKoE/fy1Xu1NhPpXZy6enCpVUFNxMHJa/Ua1v1wxdy03GI7XKAyZGQo8khcbvqINh/hej5pFs+cXxn7Jp/u9T2oUHJbHOafqC3tMJ/Xuda/g/8OWPqoTIblIpf3VGb9m8G3K/BMLRn4TLOw2mi/kQgxq8jwTD3+viDAmuRQ6+v/rfnGB87UvCjveiS0OaVRfQ7Pf9HmA+cH6R0T2de86pskEZ1xwFezgkUYISMR/NuZBZOzhGpCJcXqA7TLlQA3ns7y28VmFbAbzBoF2F2WXOoBBArkCM75N/JlFRv7vPqVLnPhtPOT41elxYwUbgCKlIbaYSIne9q5KwlMYP2ORd6JuxPkzYkwRj/1r50Cvj3YSCRQHL2rLO/akFlRZzmyDU3Ja+EUl5jR4sUo5PxoXtyBvER+ADdtLeUu8vW7cXaleMrdNgdzquYxf0No532z+7QHa19CfSLDFrxq+xDCaB+MTJsbhJEoyylw05K9xf/17NrhIzDPVaf x-ms-exchange-antispam-messagedata-chunkcount: 1 x-ms-exchange-antispam-messagedata-0: =?utf-8?B?Q29BTW9LNVN2MENhQXNqM1R0K3c0NU0wbzg3czlzbk9hUURQYWYwcnhyWGdM?= =?utf-8?B?a04vV0JtZ0JJUy9laDMrK2FBZlBUcjRpOEwvL1FTeGNnQXU4UVhtTzBERStT?= =?utf-8?B?bTFBN2FyZW9RUnZ2aHBhc0lPc0ZJdmRIUlFaeERkRnN6TkpvUVRzc2Y4RTdX?= =?utf-8?B?WFlWdkc5N2hFd2pyaGdiNUNuZG1MUkVoc29wMjNSK3NHNXZ3b2dlQ0tncHE2?= =?utf-8?B?TStwTEdnOWZSNGJwa1ZxVTlUblpRd0hqT28rRnVnRjFTekpmcU5XU1huVExH?= =?utf-8?B?aXpNUXdOYVc5Titzd0dFcHptNnkrNGE1M3BLSmt4NFNJVGNDUWRlbm4wUWZj?= =?utf-8?B?ektNSkJFL2JnNVBYTWpDTkp4bjF4QStHUU90VU05b1owOTJ5VDNTTzY1MVYz?= =?utf-8?B?dUM0TUhqVGllR0c5cnNHNmMxRzZ3SkduYmtlSUJzRUxwS0ZUMEd0SkR6T0s1?= =?utf-8?B?WDdENC9KM1lLZklFQ2JnMW10RE9vQkR6RW04ai9ja3NuTE9MZDdrZXVJNVhp?= =?utf-8?B?cXVMeWJOTkF0MXV2bzZqOURBR0x1K0lVaXhHVXVOK04vMm95czNhZ2pTMERZ?= =?utf-8?B?ZzFadEhPcTJTUU9lNld5SWRPYnFweDdTQURMaGVQcVVCeEJYdTE5aU40bEdV?= =?utf-8?B?MGVMZStDbmlXd0hTM2NqNGFlSkE1VTBTa0h6L2x2S2tOakNwd3NFQlhZQjhC?= =?utf-8?B?M085YmJ3M1VRaWpSYTRoekJqWWloK3Y1OGpQWU9lUXYrUHpreHN1cWttUkEw?= =?utf-8?B?cXJJckVia0FaL1lNVVErRHNVUDhHcXhHc3RDN0YvLzBYQ0tKTittT0YvWWtG?= =?utf-8?B?QzFDaHVvaVVvVEZONXVTRkI4VGNkbFd1aEp4TTdjcjhHSGh2cUJKbXdGWmpO?= =?utf-8?B?MVZBZ01kZ2VmVHhkNE1tcFdsQlZUVGhNSWtJTEVsb3dlQWp1RFJxMmE3dVFN?= =?utf-8?B?bVNsTE5vSExqaFZGRXAvZUFXYnJpVUFjWC9xd29wY3VpdkQ2UVNwMFk5Yk9v?= =?utf-8?B?MXlLVGpQK2JTcXlwNzhnSkhjckEzSWV1UzhhWkVIaWxBZHo0clRoa3JNTnFa?= =?utf-8?B?SVdpQjNYVEpGcFhaSjN2RG5QZUhBeDZCQ3hsTlFldkp3eDJWMCtqckZMbGd0?= =?utf-8?B?OWNiQ3Y2amRYbXFnQWE0S0x1U2ZJLzJhVkc3dXJjdWJrWXc0QUZiVjQzSzdF?= =?utf-8?B?YU05Rm5WSzBqWW5tdGR6L1hkVWFnZXZ3TVFmMUVqWGFQR1RQNnpzQ04ycG1w?= =?utf-8?B?cjlRTE5ENnp1V201SXgwT1JWY3FNMzVjdkRxbUtHMFZmMm5ySzl3NXRzaUIw?= =?utf-8?B?SlpuZnBJcDNlOWN6S21ramd3RTBNdmFJM2phNzdWMkN4RjFJdkdCV0F5SUFi?= =?utf-8?B?Y0JFbmUwQ1lsUWl5cTM4Z2FjV01USGtWRDhJdjErL1l3Tm1SRmt3OXFIVnl6?= =?utf-8?B?bWkyTzRMQnI2L2RMN1ZiL3E3aE1lZFlCOHkzeFRtaDlyOFVwOVFuUWFFYXpt?= =?utf-8?B?eTlCRWUvTkNpSjZaZmNRTGZKdFdmRVpZbkwzcjBqSzdrQUxxb2dJS2VZaGtG?= =?utf-8?B?OTJ6MFh6eTRsSDk5VjhLaHQ2MEJoakVyREt2NVdDLzdaYjRxRTROUzU3MzVz?= =?utf-8?B?R2VSS3RWZVA1di9DRzV2aDZVWFFZejVwY0FMVCtaSVQxYUJXSWMxanp2cVZ2?= =?utf-8?B?bUxkTXNPRU1Qam45S2dKUkRibTc3eTZ4RFBRVVJDS1BESXFWcHl1c3RBPT0=?= MIME-Version: 1.0 X-OriginatorOrg: sct-15-20-4755-11-msonline-outlook-1ff67.templateTenant X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-AuthSource: DM8P223MB0365.NAMP223.PROD.OUTLOOK.COM X-MS-Exchange-CrossTenant-RMS-PersistedConsumerOrg: 00000000-0000-0000-0000-000000000000 X-MS-Exchange-CrossTenant-Network-Message-Id: cd6973a0-9ee6-46f2-0651-08da58815a2b X-MS-Exchange-CrossTenant-originalarrivaltime: 27 Jun 2022 21:09:41.2515 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 84df9e7f-e9f6-40af-b435-aaaaaaaaaaaa X-MS-Exchange-CrossTenant-rms-persistedconsumerorg: 00000000-0000-0000-0000-000000000000 X-MS-Exchange-Transport-CrossTenantHeadersStamped: BN0P223MB0231 Subject: Re: [FFmpeg-devel] [PATCH] libavcodec/hevc_mp4toannexb_bsf: insert extradata before non-AUD unit X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" Archived-At: List-Archive: List-Post: > -----Original Message----- > From: ffmpeg-devel On Behalf Of > Andreas Rheinhardt > Sent: Tuesday, March 22, 2022 10:30 AM > To: ffmpeg-devel@ffmpeg.org > Subject: Re: [FFmpeg-devel] [PATCH] libavcodec/hevc_mp4toannexb_bsf: > insert extradata before non-AUD unit > > Xiang, Haihao: > > From: Haihao Xiang > > > > It is possible that an IRAP frame in input AVPacket contains VPS, > SPS > > and PPS, and these headers should take effect. However the > prepended > > extradata might override these headers. This patch inserts > extradata > > before non-AUD unit, hence VPS, SPS and PPS from the input AVPacket > will > > take effect if they are present. > > > > This should fix #7799 > > > > Signed-off-by: Haihao Xiang > > --- > > libavcodec/hevc_mp4toannexb_bsf.c | 12 ++++++++++-- > > 1 file changed, 10 insertions(+), 2 deletions(-) > > > > diff --git a/libavcodec/hevc_mp4toannexb_bsf.c > b/libavcodec/hevc_mp4toannexb_bsf.c > > index 790dfb0394..77551ba221 100644 > > --- a/libavcodec/hevc_mp4toannexb_bsf.c > > +++ b/libavcodec/hevc_mp4toannexb_bsf.c > > @@ -124,6 +124,7 @@ static int hevc_mp4toannexb_filter(AVBSFContext > *ctx, AVPacket *out) > > > > int got_irap = 0; > > int i, ret = 0; > > + int prev_nalu_is_aud = 0, extradata_offset = 0; > > > > ret = ff_bsf_get_packet(ctx, &in); > > if (ret < 0) > > @@ -169,14 +170,21 @@ static int > hevc_mp4toannexb_filter(AVBSFContext *ctx, AVPacket *out) > > > > prev_size = out->size; > > > > + if (prev_nalu_is_aud) > > + extradata_offset = prev_size; > > + > > ret = av_grow_packet(out, 4 + nalu_size + extra_size); > > if (ret < 0) > > goto fail; > > > > - if (extra_size) > > - memcpy(out->data + prev_size, ctx->par_out->extradata, > extra_size); > > + if (extra_size) { > > + memmove(out->data + extradata_offset + extra_size, > out->data + extradata_offset, prev_size - extradata_offset); > > + memcpy(out->data + extradata_offset, ctx->par_out- > >extradata, extra_size); > > + } > > + > > AV_WB32(out->data + prev_size + extra_size, 1); > > bytestream2_get_buffer(&gb, out->data + prev_size + 4 + > extra_size, nalu_size); > > + prev_nalu_is_aud = nalu_type == HEVC_NAL_AUD; > > } > > > > ret = av_packet_copy_props(out, in); > > 1. prev_nalu_is_aud is unnecessary: You can just use "if (nalu_type > == > HEVC_NAL_AUD) extradata_offset = out->size;" at the end of the loop. > 2. This only mitigates a certain case of wrongly inserted extradata; > it > does not fix the underlying issue which is that this BSF does not > track > the current state of extradata and therefore inserts parameter sets > that > have already been superseded by new in-band extradata. Your patch > ensures that the extradata parameter sets will be prepended to the > in-band extradata. Yet the already deactivated parameter sets will > still > be inserted. The output can still be invalid, because 7.4.2.4.2 of > the > HEVC spec requires the following: "Any SPS NAL unit with nuh_layer_id > equal to 0 containing the value of sps_seq_parameter_set_id for the > active SPS > RBSP for the base layer for a CVS shall have the same content as that > of > the active SPS RBSP for the base layer for the CVS, unless it follows > the last access unit of the CVS and precedes the first VCL NAL unit > and > the first SEI NAL unit containing an active parameter sets SEI > message > (when present) of another CVS." Furthermore in case a preceding > packet > contained updated parameter sets that (perhaps partially) overwrite > parameter sets from extradata and the current packet does not > contain/repeat these parameter sets, then the above code will still > insert the outdated and incorrect parameter set and these parameter > sets > will not be overwritten before being used. > 3. Andriy Gelman once proposed a patchset that tracked the parameter > sets and inserted only the needed ones. See > https://patchwork.ffmpeg.org/project/ffmpeg/patch/20191016025040.3127 > 3-2-andriy.gelman@gmail.com/ > The problem with this patchset was the complexity emanating from > HEVC's > layers. > 4. Lacking proper tracking of parameter sets we should probably err > on > the side of caution and stop inserting parameter sets if the input > contained in-band parameter sets (similar to h264_mp4toannexb). I can > write a patch for this. Hi Andreas, I had missed this conversation and just recently noticed that Haihao's Patch fixes HEVC decoding issues with certain files that I'm seeing for a long time already. The cases I know are with the hevc_qsv, hevc_cuvid and mediacodec hevc decoders, but obviously it's about all usages of the bitstream filter. You mentioned you might want to write a patch. Would you still be able to do so? Would this be a complete fix for the problem or just about a part of it? Or would this patch be acceptable as a temporary workaround? Thanks, softworkz _______________________________________________ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".