From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org [79.124.17.100]) by master.gitmailbox.com (Postfix) with ESMTP id BD71244D5E for ; Mon, 21 Nov 2022 02:44:57 +0000 (UTC) Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 6740968BCFC; Mon, 21 Nov 2022 04:44:54 +0200 (EET) Received: from mga14.intel.com (mga14.intel.com [192.55.52.115]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id F2BD368BCAC for ; Mon, 21 Nov 2022 04:44:41 +0200 (EET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1668998692; x=1700534692; h=from:to:cc:subject:date:message-id:references: in-reply-to:content-id:content-transfer-encoding: mime-version; bh=Bpr7FKLx4vQ/4Y4/8H4TmAKyi5WkC+5njP6afyxTr9o=; b=eLdBfEdPLfJ3uufSOCPt7CI4GAMybIcV1M13DQ2VsxmxRquldKKmxatc Iy2vrHf3kzL1D1RvZqZn57gFxRpjr8tixcHicMGy7UrTpqOsD5CMQSir7 P4yBOMTCAHY8Syi+7fsEM3Ozz3HYPDLe6/x0L4UQx4CGGZp8pQ1Gagwvn LKdZA9DBl7uoJShWnmuTZYhLF8Ot0DMLnwo/hEKcOFPtcJuaboixqavep xqFi57D+zRnJX59dcBxpsFHPweoczKIKvrYXiHykVQEnhQkA88XQTlSLd NqhCQpytyOemxfaovyT9cLrM2hMVUr7OH1RrGGyd6l1QuvkSciVWVjOoh w==; X-IronPort-AV: E=McAfee;i="6500,9779,10537"; a="313488313" X-IronPort-AV: E=Sophos;i="5.96,180,1665471600"; d="scan'208";a="313488313" Received: from fmsmga003.fm.intel.com ([10.253.24.29]) by fmsmga103.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 20 Nov 2022 18:44:40 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6500,9779,10537"; a="729849993" X-IronPort-AV: E=Sophos;i="5.96,180,1665471600"; d="scan'208";a="729849993" Received: from fmsmsx602.amr.corp.intel.com ([10.18.126.82]) by FMSMGA003.fm.intel.com with ESMTP; 20 Nov 2022 18:44:39 -0800 Received: from fmsmsx610.amr.corp.intel.com (10.18.126.90) by fmsmsx602.amr.corp.intel.com (10.18.126.82) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2375.31; Sun, 20 Nov 2022 18:44:39 -0800 Received: from fmsmsx610.amr.corp.intel.com (10.18.126.90) by fmsmsx610.amr.corp.intel.com (10.18.126.90) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2375.31; Sun, 20 Nov 2022 18:44:39 -0800 Received: from FMSEDG603.ED.cps.intel.com (10.1.192.133) by fmsmsx610.amr.corp.intel.com (10.18.126.90) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2375.31 via Frontend Transport; Sun, 20 Nov 2022 18:44:39 -0800 Received: from NAM11-CO1-obe.outbound.protection.outlook.com (104.47.56.175) by edgegateway.intel.com (192.55.55.68) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.1.2375.31; Sun, 20 Nov 2022 18:44:38 -0800 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=e3TwphbsiIciGQxG2lPB7vS1EcSBJVGxB7mRND9JDVBOGU7pb64nB2MxgdyqBY42/xv8tZmAWwx3c1zsDXivVWBpkJ9awcwJIWKPJL6o9FYpd09ZRb9o46BdMS7NR31X2a1ns0j7Ab+tCZZVg0N8aRO9vEXjc79hPubZbUCbg6jDhqq4RN3IYlXzn+9T/yCgXscngOoktQsZ7/iCZaGJvgUZ3YV8sAlPx+ksh34LN2SSBdn+Foyt5BQqGN5gFAb9s5SMuSVnliFvm5M4Rd1cqIyWYzvyulIQbyw/oU/3mEttbGTJy0FHpi4KxwZU48vQx6CWgciABQvqyMT5bKPcIg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=Bpr7FKLx4vQ/4Y4/8H4TmAKyi5WkC+5njP6afyxTr9o=; b=ZTX+egsUfkG6kj4FOt5GKwT2y0QWBa+Eojz9wkVbE67EzF1Z1Zu5sntJxTX8hqelNcuxNUYxMO3kaZUTjJIDqKEPpbXDa4Sg8PLff0adGKNmGMgj4G9yWzr/qaoaax5SlNkpqoC8xu0lpffSj6/AgBwegEOsoQJhcmFWpX6rCKNpwp4e/i65cNIMmrllQPozl+/m3dSEkl/dre0UeOGnowEHWSWGHMZLg/wKhVSaEHHGRrS+62b5jJU4oZEIdm6z3ifLAaneIS/Acs6eo3+8s+LiN60EljAYUjeHxuhm06ZpIKTzTCLdNVYKSJhKJEbPpdk8+clpRbMQci+Uo852Hg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=intel.com; dmarc=pass action=none header.from=intel.com; dkim=pass header.d=intel.com; arc=none Received: from BN9PR11MB5515.namprd11.prod.outlook.com (2603:10b6:408:104::8) by CH0PR11MB5252.namprd11.prod.outlook.com (2603:10b6:610:e3::15) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5834.15; Mon, 21 Nov 2022 02:44:37 +0000 Received: from BN9PR11MB5515.namprd11.prod.outlook.com ([fe80::ca2a:c3a1:b0cb:c003]) by BN9PR11MB5515.namprd11.prod.outlook.com ([fe80::ca2a:c3a1:b0cb:c003%9]) with mapi id 15.20.5834.015; Mon, 21 Nov 2022 02:44:37 +0000 From: "Xiang, Haihao" To: "ffmpeg-devel@ffmpeg.org" Thread-Topic: [FFmpeg-devel] [PATCH v6 3/3] avcodec/qsvdec: Implement SEI parsing for QSV decoders Thread-Index: AQHY6Cbw3gWDNljL2Uyikyl9I8ywha5I1hQA Date: Mon, 21 Nov 2022 02:44:36 +0000 Message-ID: <86cdc94085af9e858fb91de81d223efb175d7620.camel@intel.com> References: <61626ebb78fcf603f58c31983fa5916c6ebeaafe.1666670616.git.ffmpegagent@gmail.com> In-Reply-To: <61626ebb78fcf603f58c31983fa5916c6ebeaafe.1666670616.git.ffmpegagent@gmail.com> Accept-Language: en-AS, zh-CN, en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: user-agent: Evolution 3.36.5-0ubuntu1 authentication-results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=intel.com; x-ms-publictraffictype: Email x-ms-traffictypediagnostic: BN9PR11MB5515:EE_|CH0PR11MB5252:EE_ x-ms-office365-filtering-correlation-id: 9ad83592-b9fe-4a7f-8a8a-08dacb6a547b x-ms-exchange-senderadcheck: 1 x-ms-exchange-antispam-relay: 0 x-microsoft-antispam: BCL:0; x-microsoft-antispam-message-info: ZpTmzIjhYqnKSN/HqCL6xa3qPmLhZT/so1aCkyiEtA3rXJ6edotTOyY3nc257G3z9zYmgMeMdlg4H4pDVjNVVWWMc75USQevrKvlI7VHFDqIFKEkvn2Xez1uIBuRu4ZWVklB1yHWJuZDrtIVLT26Alvuhtfmdh5bD78zTlqX4SsL/KyxrZ95dMk08BfZ77oIhI1OQGmnv6IwKMbWp8/WLq9qlt3O3LPbrIGB+BPiJcgJoQWiUG3Edc+v7ORYS8+0Jv0TSoYrRsN9CGVjBm2XpymjmyYIjvKg30lOs7UhkoOZvdqnLgPsYbmDf+3FRZ+2mcuAKMCm3p+N1m8LcCPG4/Y/FgY+LdlgowEl/3a7VSy/3KgxIWqngSMv9ZxfsZJiN1pBuJtqS17yfZHmQ6ztcGyqSEnuWgkG+lJpa1FYRFlzgHsPwG+nJJyD/sjv+6ylL5Tt3DlFbVJVQjZKpitsz192VB4pgCHYnmJTzf+nxexOM1/UStG3hY6bydHFE+gMM52rdknILSSXhLG02ORIRseSOsgqLla7NM8qzsqYXO985n3gdYUK5FMfy27iS2vnRWVBBddoNqWKqHTN3ggralzoeDfoROuiaOWYmqIXH9m783x4+AViD4T1SKjkBCGbHMFdjRAmwWwZTgbuIDLEu3UFnOFNt16JM3ysPMQFPTYkCrHpHTd4Whp3LFeD1R172yvo8NwKPhZzfHBKG5o7e+rv45gvSquxNGhPAeAs1pc= x-forefront-antispam-report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:BN9PR11MB5515.namprd11.prod.outlook.com; PTR:; CAT:NONE; SFS:(13230022)(376002)(136003)(366004)(39860400002)(396003)(346002)(451199015)(36756003)(38070700005)(86362001)(30864003)(64756008)(76116006)(5660300002)(2906002)(26005)(4001150100001)(2616005)(186003)(6512007)(82960400001)(38100700002)(122000001)(83380400001)(91956017)(54906003)(966005)(6486002)(71200400001)(8936002)(41300700001)(66476007)(66556008)(66946007)(66446008)(19627235002)(316002)(6916009)(478600001)(8676002)(4326008)(6506007); DIR:OUT; SFP:1102; x-ms-exchange-antispam-messagedata-chunkcount: 1 x-ms-exchange-antispam-messagedata-0: =?utf-8?B?aDRUak1ZUzRlS2R1WFJjTTJBMWxDZW5ZUkZid0NwczJrZ1poazRPaGwxaTB4?= =?utf-8?B?SVBMUmEwUXJMTXhSRWZnTlpsOVd3NXlGZXVTN1dnLzVpUkkvTVk2L3R1aExK?= =?utf-8?B?Sm9HZmlWb3JxZ29Tb3ZybXZuVG9yOXRENVhtMnpRd3R4ekVucVVLWkFKaml2?= =?utf-8?B?QlBTc1VHOFIwVlpvWkErMW01bnNzR0dyNmNYa0JlTG55Ry9SMXlNRnBxQzAy?= =?utf-8?B?Y1F0ZkpEV1ZJQ0tyMkd5WFBqVzB6L3RwU0dPcnE1TTYycUdicmcrckpVWlh6?= =?utf-8?B?VGtOY3pEQldIMThnRHVkNlBBWGZpcFpFWDMrTys4Q2g0TFJ2ZmthdGU1MUdK?= =?utf-8?B?bHc3eGdYSWdiblVrVGRRMGJJaWtvL1dLcmpPdWZ3UHUrbUplSUJ1VVdoRElE?= =?utf-8?B?WnVkS0RYblZLcklCdHowRzdaWVRZeHZhUHRaTWdhR0lXTzNXbTFTbElLMEIr?= =?utf-8?B?a3BWL1N0S0ViczdHc2pCK0ZYSlp3TFgweW5tT2ROVUt0N1RhdStxbGhIeWh3?= =?utf-8?B?VFdwaTVpSVFkaDhXeU5vb0hmcnBkS1ZDTTVML1dYb2JvZHdGQmpNazFyU1Ay?= =?utf-8?B?akRPbE5XUVBLYktZaFNMbk5lZ0p3NEFsYlR6d0ZudW5YQXlvbkJnNXVkcEh2?= =?utf-8?B?TGVvZHJoamo4MjlNcGlJUTk4OHZwRXBmcXJsM2dsNjUvMFM5UVlXRkZvS2pR?= =?utf-8?B?dERXRXk4RnZLOFZTY25NMTRqS2o5eHBpN2FMSks1NWw0QUtUVHRjMDR6c2Fk?= =?utf-8?B?VlpGUnluMFI1SVhHekFteUZsdWQvTHZoTklxVDA2LzkxaVJvbHFBYUNoVEw4?= =?utf-8?B?UUg5TG9JbFNub09tZGZ6SVpqZDJ1OW93R2pHNDllOTRZZ2NvZ2V2T3BzNDdV?= =?utf-8?B?L2E1YzdoL2R0cDdJNGRCbVhhd1pkb1pNNkF0K2l6U1IxWkM3c1N6WHl5MnVS?= =?utf-8?B?eWJGQnJiMVBHSTRqYVk4TUNIUjlNMnQ3QUFFdFIyb2VwVlBDcHd1Z2g0M2F6?= =?utf-8?B?ZVFiUllsZTZnbHNDMWNSWjdxWUJrcEw5REtlU2pDbThBcWNEMXk4OG5YTE5J?= =?utf-8?B?UmR0akxqL3d4bFR6enZ6b25pNGtrNFoyanhaaG1jR2xpdnlRZlJJQXZkbzVG?= =?utf-8?B?TGJhcTlWU2hhRWRXQisyMWxOaUp3MERmbEtXc20zMVN0aHlBUmtOeXZlRU54?= =?utf-8?B?aXQ3T2FwdlRyaStzZHk2ejNndTMwbjZVQ0wrZmNLQXdpaDJ0cXBLQ2NNRllS?= =?utf-8?B?dlVHVGxnSE5QOEVTd21STXY1TWRhdkM5Q3MyRGIwKysrVWJ4ZG5GcnNjbU5O?= =?utf-8?B?dnNIQjFoT0x1RUhqeGU1ZXp0REJ3Qm9vRUxNSmdGbUk3c0t4NnIvWTFuekxa?= =?utf-8?B?OW94SGdtTUNBMHc5V0lCT0IvUitsM0w2VitENGg1WDUxQTlXOWMwYWJmQUlW?= =?utf-8?B?anUvRkRlZUQyUzk0UTVFM2NoZGJBQ09iM0JIZzBSd3BFeGVPa0NkdjY3ZlBp?= =?utf-8?B?RE9Ha2g3bkY3N3V6NnpwQU9NZUY5N2RNMFhYVStSTWswNzZJNHBrdS9LWGdo?= =?utf-8?B?YmdMbWE5MkkzL0RQb0JtRWtXRDVDMk9TSG45eHNmQWVkMFZsVlJPeEZLOW96?= =?utf-8?B?bjZXZ3JpRjZFTENlNHhXZ3FsTjBDaTNHalVHSjNZek9kcFFMUjVXTkNWU1Bj?= =?utf-8?B?bWhQNFJyazdZTTc4eGp0QkwySldXVHZ0SXRiYWdXamg0UzRuWjREbThoT3Mv?= =?utf-8?B?aCtqQStwNDF1d010b25qV2JwKzZuVWtNSVV2MDhmNFBleGxoWEZ6ODQ1TTVV?= =?utf-8?B?c2hhbHpzMW81QXpJSkcvejYvNTIvcENRTWQxdDc3TGErVFN1OWszS1RlTlE0?= =?utf-8?B?NWErRjk2ZDdRSDN1SjRpdnQ1eW5sdGVEcXowdDlqZzh2Q3VFUmZGQWt4eE1l?= =?utf-8?B?STh3a2lEbWtud3JCSHZRYmpSOVIwckdoOEp1WVJ5S2dFUnEzMGtBM1RNRC9m?= =?utf-8?B?aVhOVlNxYk1nbWpXMVN5Yk1ZS2FIeEpwL1hhV1lPNERvVnlkY1hKZVF5TW1R?= =?utf-8?B?c3RkT2IwaksveGtDS1M5V3ppR0plS2JKbUZ4WG9ybHFRd2JsZUl2MG9oQUhW?= =?utf-8?B?QWNoWFViWFV3SWlOZGVxVFlZYVB3dzNaY04wRlVJK1JFdS9aSXd6c1hDbWhv?= =?utf-8?B?U2c9PQ==?= Content-ID: MIME-Version: 1.0 X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-AuthSource: BN9PR11MB5515.namprd11.prod.outlook.com X-MS-Exchange-CrossTenant-Network-Message-Id: 9ad83592-b9fe-4a7f-8a8a-08dacb6a547b X-MS-Exchange-CrossTenant-originalarrivaltime: 21 Nov 2022 02:44:37.0076 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 46c98d88-e344-4ed4-8496-4ed7712e255d X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-CrossTenant-userprincipalname: 6GlcwsOFmbNZSO/FaGNSayDoOOsmaPeZrbwViggHXSk4sJtHBCX6lfFoevHuSNj/E1cPIDtGKECnNsYJnrhTCQ== X-MS-Exchange-Transport-CrossTenantHeadersStamped: CH0PR11MB5252 X-OriginatorOrg: intel.com Subject: Re: [FFmpeg-devel] [PATCH v6 3/3] avcodec/qsvdec: Implement SEI parsing for QSV decoders X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: "kierank@obe.tv" , "softworkz@hotmail.com" , "haihao.xiang-at-intel.com@ffmpeg.org" , "andreas.rheinhardt@outlook.com" Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" Archived-At: List-Archive: List-Post: On Tue, 2022-10-25 at 04:03 +0000, softworkz wrote: > From: softworkz > > Signed-off-by: softworkz > --- > libavcodec/Makefile | 2 +- > libavcodec/qsvdec.c | 321 ++++++++++++++++++++++++++++++++++++++++++++ > 2 files changed, 322 insertions(+), 1 deletion(-) > > diff --git a/libavcodec/Makefile b/libavcodec/Makefile > index 90c7f113a3..cbddbb0ace 100644 > --- a/libavcodec/Makefile > +++ b/libavcodec/Makefile > @@ -146,7 +146,7 @@ OBJS-$(CONFIG_MSS34DSP) += mss34dsp.o > OBJS-$(CONFIG_PIXBLOCKDSP) += pixblockdsp.o > OBJS-$(CONFIG_QPELDSP) += qpeldsp.o > OBJS-$(CONFIG_QSV) += qsv.o > -OBJS-$(CONFIG_QSVDEC) += qsvdec.o > +OBJS-$(CONFIG_QSVDEC) += qsvdec.o h264_sei.o hevc_sei.o > OBJS-$(CONFIG_QSVENC) += qsvenc.o > OBJS-$(CONFIG_RANGECODER) += rangecoder.o > OBJS-$(CONFIG_RDFT) += rdft.o > diff --git a/libavcodec/qsvdec.c b/libavcodec/qsvdec.c > index 73405b5747..467a248224 100644 > --- a/libavcodec/qsvdec.c > +++ b/libavcodec/qsvdec.c > @@ -41,6 +41,7 @@ > #include "libavutil/time.h" > #include "libavutil/imgutils.h" > #include "libavutil/film_grain_params.h" > +#include > > #include "avcodec.h" > #include "codec_internal.h" > @@ -49,6 +50,9 @@ > #include "hwconfig.h" > #include "qsv.h" > #include "qsv_internal.h" > +#include "h264_sei.h" > +#include "hevc_ps.h" > +#include "hevc_sei.h" > > #if QSV_ONEVPL > #include > @@ -66,6 +70,8 @@ static const AVRational mfx_tb = { 1, 90000 }; > AV_NOPTS_VALUE : pts_tb.num ? \ > av_rescale_q(mfx_pts, mfx_tb, pts_tb) : mfx_pts) > > +#define PAYLOAD_BUFFER_SIZE 65535 > + > typedef struct QSVAsyncFrame { > mfxSyncPoint *sync; > QSVFrame *frame; > @@ -107,6 +113,9 @@ typedef struct QSVContext { > > mfxExtBuffer **ext_buffers; > int nb_ext_buffers; > + > + mfxU8 payload_buffer[PAYLOAD_BUFFER_SIZE]; > + AVBufferRef *a53_buf_ref; > } QSVContext; > > static const AVCodecHWConfigInternal *const qsv_hw_configs[] = { > @@ -628,6 +637,299 @@ static int qsv_export_film_grain(AVCodecContext *avctx, > mfxExtAV1FilmGrainParam > } > #endif > > +static int find_start_offset(mfxU8 data[4]) > +{ > + if (data[0] == 0 && data[1] == 0 && data[2] == 1) > + return 3; > + > + if (data[0] == 0 && data[1] == 0 && data[2] == 0 && data[3] == 1) > + return 4; > + > + return 0; > +} > + > +static int parse_sei_h264(AVCodecContext* avctx, QSVContext* q, AVFrame* out) > +{ > + H264SEIContext sei = { 0 }; > + GetBitContext gb = { 0 }; > + mfxPayload payload = { 0, .Data = &q->payload_buffer[0], .BufSize = > sizeof(q->payload_buffer) - AV_INPUT_BUFFER_PADDING_SIZE }; > + mfxU64 ts; > + int ret; > + > + while (1) { > + int start; > + memset(payload.Data, 0, payload.BufSize); > + > + ret = MFXVideoDECODE_GetPayload(q->session, &ts, &payload); > + if (ret == MFX_ERR_NOT_ENOUGH_BUFFER) { > + av_log(avctx, AV_LOG_WARNING, "Warning: Insufficient buffer on > GetPayload(). Size: %"PRIu64" Needed: %d\n", sizeof(q->payload_buffer), > payload.BufSize); > + return 0; > + } > + if (ret != MFX_ERR_NONE) > + return ret; > + > + if (payload.NumBit == 0 || payload.NumBit >= payload.BufSize * 8) > + break; > + > + start = find_start_offset(payload.Data); > + > + switch (payload.Type) { > + case SEI_TYPE_BUFFERING_PERIOD: > + case SEI_TYPE_PIC_TIMING: > + continue; > + } > + > + if (init_get_bits(&gb, &payload.Data[start], payload.NumBit - start * > 8) < 0) > + av_log(avctx, AV_LOG_ERROR, "Error initializing bitstream reader > SEI type: %d Numbits %d error: %d\n", payload.Type, payload.NumBit, ret); > + else { > + ret = ff_h264_sei_decode(&sei, &gb, NULL, avctx); > + > + if (ret < 0) > + av_log(avctx, AV_LOG_WARNING, "Failed to parse SEI type: > %d Numbits %d error: %d\n", payload.Type, payload.NumBit, ret); > + else > + av_log(avctx, AV_LOG_DEBUG, "mfxPayload Type: %d Numbits > %d\n", payload.Type, payload.NumBit); > + } > + } > + > + if (out) > + return ff_h264_set_sei_to_frame(avctx, &sei, out, NULL, 0); > + > + return 0; > +} > + > +static int parse_sei_hevc(AVCodecContext* avctx, QSVContext* q, QSVFrame* > out) > +{ > + HEVCSEI sei = { 0 }; > + HEVCParamSets ps = { 0 }; > + GetBitContext gb = { 0 }; > + mfxPayload payload = { 0, .Data = &q->payload_buffer[0], .BufSize = > sizeof(q->payload_buffer) - AV_INPUT_BUFFER_PADDING_SIZE }; > + mfxFrameSurface1 *surface = &out->surface; > + mfxU64 ts; > + int ret, has_logged = 0; > + > + while (1) { > + int start; > + memset(payload.Data, 0, payload.BufSize); > + > + ret = MFXVideoDECODE_GetPayload(q->session, &ts, &payload); > + if (ret == MFX_ERR_NOT_ENOUGH_BUFFER) { > + av_log(avctx, AV_LOG_WARNING, "Warning: Insufficient buffer on > GetPayload(). Size: %"PRIu64" Needed: %d\n", sizeof(q->payload_buffer), > payload.BufSize); > + return 0; > + } > + if (ret != MFX_ERR_NONE) > + return ret; > + > + if (payload.NumBit == 0 || payload.NumBit >= payload.BufSize * 8) > + break; > + > + if (!has_logged) { > + has_logged = 1; > + av_log(avctx, AV_LOG_VERBOSE, "-------------------------------- > ---------\n"); > + av_log(avctx, AV_LOG_VERBOSE, "Start reading SEI - payload > timestamp: %llu - surface timestamp: %llu\n", ts, surface->Data.TimeStamp); > + } > + > + if (ts != surface->Data.TimeStamp) { > + av_log(avctx, AV_LOG_WARNING, "GetPayload timestamp (%llu) does > not match surface timestamp: (%llu)\n", ts, surface->Data.TimeStamp); > + } > + > + start = find_start_offset(payload.Data); > + > + av_log(avctx, AV_LOG_VERBOSE, "parsing SEI type: %3d Numbits > %3d Start: %d\n", payload.Type, payload.NumBit, start); > + > + switch (payload.Type) { > + case SEI_TYPE_BUFFERING_PERIOD: > + case SEI_TYPE_PIC_TIMING: > + continue; > + case SEI_TYPE_MASTERING_DISPLAY_COLOUR_VOLUME: > + // There seems to be a bug in MSDK > + payload.NumBit -= 8; > + > + break; > + case SEI_TYPE_CONTENT_LIGHT_LEVEL_INFO: > + // There seems to be a bug in MSDK > + payload.NumBit = 48; > + > + break; > + case SEI_TYPE_USER_DATA_REGISTERED_ITU_T_T35: > + // There seems to be a bug in MSDK > + if (payload.NumBit == 552) > + payload.NumBit = 528; > + break; > + } > + > + if (init_get_bits(&gb, &payload.Data[start], payload.NumBit - start * > 8) < 0) > + av_log(avctx, AV_LOG_ERROR, "Error initializing bitstream reader > SEI type: %d Numbits %d error: %d\n", payload.Type, payload.NumBit, ret); > + else { > + ret = ff_hevc_decode_nal_sei(&gb, avctx, &sei, &ps, > HEVC_NAL_SEI_PREFIX); > + > + if (ret < 0) > + av_log(avctx, AV_LOG_WARNING, "Failed to parse SEI type: > %d Numbits %d error: %d\n", payload.Type, payload.NumBit, ret); > + else > + av_log(avctx, AV_LOG_DEBUG, "mfxPayload Type: %d Numbits > %d\n", payload.Type, payload.NumBit); > + } > + } > + > + if (has_logged) { > + av_log(avctx, AV_LOG_VERBOSE, "End reading SEI\n"); > + } > + > + if (out && out->frame) > + return ff_hevc_set_sei_to_frame(avctx, &sei, out->frame, avctx- > >framerate, 0, &ps.sps->vui, ps.sps->bit_depth, ps.sps->bit_depth_chroma); I got segfault when trying your patchset, Thread 1 "ffmpeg" received signal SIGSEGV, Segmentation fault. 0x00007ffff67c0497 in parse_sei_hevc (avctx=avctx@entry=0x5555555e4280, q=q@entry=0x555555625288, out=out@entry=0x5555559b6f80) at libavcodec/qsvdec.c:777 777 return ff_hevc_set_sei_to_frame(avctx, &sei, out->frame, avctx->framerate, 0, &ps.sps->vui, ps.sps->bit_depth, ps.sps->bit_depth_chroma); (gdb) bt #0 0x00007ffff67c0497 in parse_sei_hevc (avctx=avctx@entry=0x5555555e4280, q=q@entry=0x555555625288, out=out@entry=0x5555559b6f80) at libavcodec/qsvdec.c:777 #1 0x00007ffff67c1afe in qsv_decode (avctx=avctx@entry=0x5555555e4280, q=q@entry=0x555555625288, frame=frame@entry=0x5555556df740, got_frame=got_frame@entry=0x7fffffffd6bc, avpkt=avpkt@entry=0x555555635398) at libavcodec/qsvdec.c:1020 BTW the SDK provides support for hevc HDR metadata, we needn't parse SEI payload in qsvdec and may get the corresponding info from the SDK, see https://ffmpeg.org/pipermail/ffmpeg-devel/2022-November/304142.html Thanks Haihao > + > + return 0; > +} > + > +#define A53_MAX_CC_COUNT 2000 > + > +static int mpeg_decode_a53_cc(AVCodecContext *avctx, QSVContext *s, > + const uint8_t *p, int buf_size) > +{ > + if (buf_size >= 6 && > + p[0] == 'G' && p[1] == 'A' && p[2] == '9' && p[3] == '4' && > + p[4] == 3 && (p[5] & 0x40)) { > + /* extract A53 Part 4 CC data */ > + unsigned cc_count = p[5] & 0x1f; > + if (cc_count > 0 && buf_size >= 7 + cc_count * 3) { > + const uint64_t old_size = s->a53_buf_ref ? s->a53_buf_ref->size : > 0; > + const uint64_t new_size = (old_size + cc_count > + * UINT64_C(3)); > + int ret; > + > + if (new_size > 3*A53_MAX_CC_COUNT) > + return AVERROR(EINVAL); > + > + ret = av_buffer_realloc(&s->a53_buf_ref, new_size); > + if (ret >= 0) > + memcpy(s->a53_buf_ref->data + old_size, p + 7, cc_count * > UINT64_C(3)); > + > + avctx->properties |= FF_CODEC_PROPERTY_CLOSED_CAPTIONS; > + } > + return 1; > + } else if (buf_size >= 2 && p[0] == 0x03 && (p[1]&0x7f) == 0x01) { > + /* extract SCTE-20 CC data */ > + GetBitContext gb; > + unsigned cc_count = 0; > + int ret; > + > + init_get_bits8(&gb, p + 2, buf_size - 2); > + cc_count = get_bits(&gb, 5); > + if (cc_count > 0) { > + uint64_t old_size = s->a53_buf_ref ? s->a53_buf_ref->size : 0; > + uint64_t new_size = (old_size + cc_count * UINT64_C(3)); > + if (new_size > 3 * A53_MAX_CC_COUNT) > + return AVERROR(EINVAL); > + > + ret = av_buffer_realloc(&s->a53_buf_ref, new_size); > + if (ret >= 0) { > + uint8_t field, cc1, cc2; > + uint8_t *cap = s->a53_buf_ref->data; > + > + memset(s->a53_buf_ref->data + old_size, 0, cc_count * 3); > + for (unsigned i = 0; i < cc_count && get_bits_left(&gb) >= > 26; i++) { > + skip_bits(&gb, 2); // priority > + field = get_bits(&gb, 2); > + skip_bits(&gb, 5); // line_offset > + cc1 = get_bits(&gb, 8); > + cc2 = get_bits(&gb, 8); > + skip_bits(&gb, 1); // marker > + > + if (!field) { // forbidden > + cap[0] = cap[1] = cap[2] = 0x00; > + } else { > + field = (field == 2 ? 1 : 0); > + ////if (!s1->mpeg_enc_ctx.top_field_first) field = > !field; > + cap[0] = 0x04 | field; > + cap[1] = ff_reverse[cc1]; > + cap[2] = ff_reverse[cc2]; > + } > + cap += 3; > + } > + } > + avctx->properties |= FF_CODEC_PROPERTY_CLOSED_CAPTIONS; > + } > + return 1; > + } else if (buf_size >= 11 && p[0] == 'C' && p[1] == 'C' && p[2] == 0x01 > && p[3] == 0xf8) { > + int cc_count = 0; > + int i, ret; > + // There is a caption count field in the data, but it is often > + // incorrect. So count the number of captions present. > + for (i = 5; i + 6 <= buf_size && ((p[i] & 0xfe) == 0xfe); i += 6) > + cc_count++; > + // Transform the DVD format into A53 Part 4 format > + if (cc_count > 0) { > + int old_size = s->a53_buf_ref ? s->a53_buf_ref->size : 0; > + uint64_t new_size = (old_size + cc_count > + * UINT64_C(6)); > + if (new_size > 3*A53_MAX_CC_COUNT) > + return AVERROR(EINVAL); > + > + ret = av_buffer_realloc(&s->a53_buf_ref, new_size); > + if (ret >= 0) { > + uint8_t field1 = !!(p[4] & 0x80); > + uint8_t *cap = s->a53_buf_ref->data; > + p += 5; > + for (i = 0; i < cc_count; i++) { > + cap[0] = (p[0] == 0xff && field1) ? 0xfc : 0xfd; > + cap[1] = p[1]; > + cap[2] = p[2]; > + cap[3] = (p[3] == 0xff && !field1) ? 0xfc : 0xfd; > + cap[4] = p[4]; > + cap[5] = p[5]; > + cap += 6; > + p += 6; > + } > + } > + avctx->properties |= FF_CODEC_PROPERTY_CLOSED_CAPTIONS; > + } > + return 1; > + } > + return 0; > +} > + > +static int parse_sei_mpeg12(AVCodecContext* avctx, QSVContext* q, AVFrame* > out) > +{ > + mfxPayload payload = { 0, .Data = &q->payload_buffer[0], .BufSize = > sizeof(q->payload_buffer) - AV_INPUT_BUFFER_PADDING_SIZE }; > + mfxU64 ts; > + int ret; > + > + while (1) { > + int start; > + > + memset(payload.Data, 0, payload.BufSize); > + ret = MFXVideoDECODE_GetPayload(q->session, &ts, &payload); > + if (ret == MFX_ERR_NOT_ENOUGH_BUFFER) { > + av_log(avctx, AV_LOG_WARNING, "Warning: Insufficient buffer on > GetPayload(). Size: %"PRIu64" Needed: %d\n", sizeof(q->payload_buffer), > payload.BufSize); > + return 0; > + } > + if (ret != MFX_ERR_NONE) > + return ret; > + > + if (payload.NumBit == 0 || payload.NumBit >= payload.BufSize * 8) > + break; > + > + start = find_start_offset(payload.Data); > + > + start++; > + > + mpeg_decode_a53_cc(avctx, q, &payload.Data[start], > (int)((payload.NumBit + 7) / 8) - start); > + > + av_log(avctx, AV_LOG_DEBUG, "mfxPayload Type: %d Numbits %d start %d > -> %.s\n", payload.Type, payload.NumBit, start, (char > *)(&payload.Data[start])); > + } > + > + if (!out) > + return 0; > + > + if (q->a53_buf_ref) { > + > + AVFrameSideData *sd = av_frame_new_side_data_from_buf(out, > AV_FRAME_DATA_A53_CC, q->a53_buf_ref); > + if (!sd) > + av_buffer_unref(&q->a53_buf_ref); > + q->a53_buf_ref = NULL; > + } > + > + return 0; > +} > + > static int qsv_decode(AVCodecContext *avctx, QSVContext *q, > AVFrame *frame, int *got_frame, > const AVPacket *avpkt) > @@ -664,6 +966,8 @@ static int qsv_decode(AVCodecContext *avctx, QSVContext > *q, > insurf, &outsurf, sync); > if (ret == MFX_WRN_DEVICE_BUSY) > av_usleep(500); > + else if (avctx->codec_id == AV_CODEC_ID_MPEG2VIDEO) > + parse_sei_mpeg12(avctx, q, NULL); > > } while (ret == MFX_WRN_DEVICE_BUSY || ret == MFX_ERR_MORE_SURFACE); > > @@ -705,6 +1009,23 @@ static int qsv_decode(AVCodecContext *avctx, QSVContext > *q, > return AVERROR_BUG; > } > > + switch (avctx->codec_id) { > + case AV_CODEC_ID_MPEG2VIDEO: > + ret = parse_sei_mpeg12(avctx, q, out_frame->frame); > + break; > + case AV_CODEC_ID_H264: > + ret = parse_sei_h264(avctx, q, out_frame->frame); > + break; > + case AV_CODEC_ID_HEVC: > + ret = parse_sei_hevc(avctx, q, out_frame); > + break; > + default: > + ret = 0; > + } > + > + if (ret < 0) > + av_log(avctx, AV_LOG_ERROR, "Error parsing SEI data: %d\n", ret); > + > out_frame->queued += 1; > > aframe = (QSVAsyncFrame){ sync, out_frame }; _______________________________________________ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".