From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org [79.124.17.100]) by master.gitmailbox.com (Postfix) with ESMTP id 7ABB841332 for ; Thu, 28 Apr 2022 01:14:58 +0000 (UTC) Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id BC9E368B45B; Thu, 28 Apr 2022 04:14:55 +0300 (EEST) Received: from EUR01-HE1-obe.outbound.protection.outlook.com (mail-oln040092065086.outbound.protection.outlook.com [40.92.65.86]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 41D0268B3C7 for ; Thu, 28 Apr 2022 04:14:49 +0300 (EEST) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=KB+g6uYb9BNLtwD47O/UkOjJO/4vEkl6qzKdIXb2m5Rq9GPj98eaGTR/ZIdsa8SiJKhXkPFpWC1Whn8CG6iUhvXhAhaIref7F8+YihMx7tlobcrrvJs4iF0LmTECYYgS9WoI7whjHnOzNEeUyNu95lYMp3B/7t4qTdTKvTY3eW7LfOHHYEUSMhfyYqosuJJZSH8HgfDNYF47tGYP9sv32iuSUDSUJYISl0wkHMlOgnGdCHZoTJPQzDo3DmkERIRHKIamjVekYl7mR7UaZ0k6BnpsJUHFMZUttPZiRWMKNf/YseoKx5bTAYQIO2zV4rvthi9h/EP+TbZIS1ByUkeVSA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=zC+bME9XWD8EIjlJiPWuTSoe72c44iURCjDiBNhEFYk=; b=Ffq/W4Kjfw8ueQX9iYaDayGFZiajQ1zGhjT3HPT0HulCcB7aVAZV2f7ircwGY56hVH6Ij2aMZNmW/BxDRTGLLgFcMCD+QNrDGryLC8Bw4KGnIqUzVlxRSEpmdfVQD6sR99+eAlnn4ZmwWU8Pfi+LxgJENBbBbPm4AvEI8QSloqmllMN0SjBS0SctfuNvddF4Z/FXIfriwhUxWC3q+wd6ICVnmbJbjOJJqqQkrzfWSxEFjs454EGe235QOLvbmNyphDS6gEYygjl2iD70BFYpPkVOFiHP25El1OGQ3rWcSwGhpxC0VSgzlEnmJ/msbOykgHsZP1gxkJv8L+vpPcbVTA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=none; dmarc=none; dkim=none; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=outlook.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=zC+bME9XWD8EIjlJiPWuTSoe72c44iURCjDiBNhEFYk=; b=WgdkQYmoRdkr6nTl6DUJmwlsjXSLT/YkQOU623vxTI2RrgMEgVVAKyBqDkHJL20FtSFVr2vvit3DZAa6ccENwWuwpm3eqU/cd7D13uPQG9fhTMbTiWRyza4fue5B3CDUF1RBHenE0zvbIEHO1d7EaTSIpc6DKm3TXHXgrhj9a/76gOr1Fw+Ly4unIBrrkr3jNCMtiY+vZAG6DCI0A8a7bRG9SZGgQ+T36EArV+b7+NzCMA6A20iQSloGLJdtDXWj8nMgD7vx684dri9xJI4ocEasOuoFRqFpG7bit3Ds8fAqkZ/pzphsE9wlKX7o63X6m9OQMkUOexDQLyA2J9iU5Q== Received: from AS8PR01MB7944.eurprd01.prod.exchangelabs.com (2603:10a6:20b:373::5) by PAXPR01MB9121.eurprd01.prod.exchangelabs.com (2603:10a6:102:2b4::9) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5186.18; Thu, 28 Apr 2022 01:14:46 +0000 Received: from AS8PR01MB7944.eurprd01.prod.exchangelabs.com ([fe80::1854:2c30:7ba1:c431]) by AS8PR01MB7944.eurprd01.prod.exchangelabs.com ([fe80::1854:2c30:7ba1:c431%6]) with mapi id 15.20.5186.021; Thu, 28 Apr 2022 01:14:46 +0000 Message-ID: Date: Thu, 28 Apr 2022 03:14:44 +0200 Content-Language: en-US To: ffmpeg-devel@ffmpeg.org References: <20220426100035.2026406-1-wenbin.chen@intel.com> From: Andreas Rheinhardt In-Reply-To: <20220426100035.2026406-1-wenbin.chen@intel.com> X-TMN: [cAgSczbi246wh+2VtdfuwcylgVMuyI5m] X-ClientProxiedBy: ZR0P278CA0054.CHEP278.PROD.OUTLOOK.COM (2603:10a6:910:1d::23) To AS8PR01MB7944.eurprd01.prod.exchangelabs.com (2603:10a6:20b:373::5) X-Microsoft-Original-Message-ID: <408e0218-9582-7d28-52d0-0b7c88022994@outlook.com> MIME-Version: 1.0 X-MS-Exchange-MessageSentRepresentingType: 1 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 179cc016-30ab-4d1c-4e64-08da28b47b72 X-MS-TrafficTypeDiagnostic: PAXPR01MB9121:EE_ X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: R61RylE2eFa6mQVP1vCSS2mQs2g/cjHZS9+cy9C/JyWWuZk3H5bw2P4plU/CIS2b1IaA9kbweiNnrpfQMMpQbZ5kB7Bwah1M20DG5kCW5eR7xIIvkMsJIaYE578zGTQEPyWtWok2uIBFwJNbfdKJNtQXgdtAzrFdqemfaKgiOgHKTLVyHapGEoWq/tWMMco5QcRMHUKwfW1+v/31PdcVD0N35xmKI4VCuIF2tJaED+DpVHnC0q68xbCziyIIEXP2YOmIGFv/aVckETMKOinMbRVI+8b/7tWB3pSWqAV6WYVkzee/clkbCixkx9sgFTYdOm2DTlJ13GhQZPIISXa/k0XJisOReZOf3/ZmhPCnU/o4lQFjcyHRGli+IVv7/7oCxxAO27zwZzqti9xpBbkob8S5Ub39Zlg0wyl2JNt7Ay1e1M3y9rMU8BmZHQXTysUKJ3FJmUcIoteZUrJELCDVg98BB3+i2ZTX5O/IMXaLxQSbkMylpdFB0X6o27yfi0yChtuG4RMATVNo5w+jvDwjGOEGgrQMPvA7lfNYqSKIp53rbVaukGpTKz4YNAgDh735zSvqEu0ylpIm6T9V9scyMA== X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?B?TXdoNERLcGxJWi9XWExnOWZKYWpRV21GaEpLQ2VJKzYzZVI3TmVyMUttZmlh?= =?utf-8?B?MDVSc3VkUlF5aXZxVyt2V3B6RHZOOEk5dVc5ajZlN2ZNUGtIWVNyR29HbEJv?= =?utf-8?B?a05pQUladENQNnNOWCtjRWpYVUlXTDA5SjhmWDJXNG5BUEVSR2xGejlpMzVq?= =?utf-8?B?SERLOVBEbGlybGNCK1Q1RTh2VldiUi96SEtWZjlKcWo4cDVMVitxUi8rWUJ2?= =?utf-8?B?eFludU5ldG4rSitCZFZDVTF1czlOOW9lNzJxbjN2TWtvWjZmaXpxVTd2QVdB?= =?utf-8?B?UVY0VTFjZmNXbFczeTJyMW1PeWJrRHNWbTVucnh1bVVtMGRZNnJXYmQyVjkx?= =?utf-8?B?OUN6WW9MaU9yZFVaOGpXOENZTjFEYWlValpSRFNobzVXMkl2akVGaUZ2ZjYv?= =?utf-8?B?dm54R1FGcW11UE4vWFRoeWxwQitrR2NYMVlsZlRGR1dyVWFuZlV1NE9sb1N6?= =?utf-8?B?OFFvYUpKeWNBbkhmM0xBamhldDU0bnYzdHRtM1kxNDUyVWJrK0ZZM0VlZS9V?= =?utf-8?B?cm5QMExvKzdNSVNSS29JQlZ5blI4cXNMSDMvQytxV0ttNUhiY2hTV0VhMDdZ?= =?utf-8?B?bGgyMk5HTVVYRGQ1cEFTQWVYV3NCOHFZWGxHSkF4bklCTzUycnRCeENZdndw?= =?utf-8?B?Ym9HK2hUT256WkJWbWxhNWpKeUU2eS9SZkpxREZSWXNtUldka1U4SngrM2Fy?= =?utf-8?B?VlB1NHUrZTNYdkVaL0FRZXVWUUFkM1JLQzdQcWZ0RGVVUDI5ODcrS1h1bXdx?= =?utf-8?B?Z3ROdlNNWDN3OFpPSDZIUWx1RVU2bm42VWhZdjlvbVJPUzY2TkFrS0VGMnhi?= =?utf-8?B?Y3NYVm5QN3AzOUNQZEk2elFtMG5ocGc0bCtmZUNqREVDaWRuVG1WbE5odG43?= =?utf-8?B?ZXdrSFBSYXJ3ck42N3hQdU1BQmkybGpYK0NzTlVzUUVTQzFJOTBCcjM5ZUlU?= =?utf-8?B?Qnc3aFBubXprRkd3bURPZ1grM25IMnZTeWJ4Qk5lblVkZWR6VzhlbG5HdFcw?= =?utf-8?B?Z2c0K2J1cGNyMjV4Q3ZnOGxGYjRmNnp5MUtmTEFrUmJSSkRMZHFKTHN3NjJG?= =?utf-8?B?MzM2Y1hseXluVEFzdzJjaXJkdmpQdkVRZTFSTmptLzdpMDF2OEozR3F0cjdH?= =?utf-8?B?cFhZRnhydk9lYko5RVMyN2lZeVdLV3R5WWs4QXBjWHpiSFRYNEpSbjVWK3BK?= =?utf-8?B?VUVrWTAxM28vZCt6ZG9sbjNKV1AxUUUrdWQvcW55NS80NWoxOFN2NnRyL0d0?= =?utf-8?B?NnM4c2x4VnVaQ0s0alZGQWw4YlRyQlpMVjhLUHpiQjRXcXhWZ1NiMjJVc1dn?= =?utf-8?B?aGtMa0FvUDFieldYVEhBUmd4RXZReUZRZ05TeDJBVGNXeXVhQW5wMU1OTk9G?= =?utf-8?B?cUVsZFFkUC9TbUlpdXBsV2wwdDhwRFNrNUp1cEdkMFZzdzdoWTRkMUZxQTF1?= =?utf-8?B?akFtRFlDcGtFbHVIb0g4M1o5WDJIMTcxM2JVUFphb1haenllaDAyeklqLzNW?= =?utf-8?B?YUFNTmMyUlI0UGFNbnJnWldscHBVbHp3Z2NHYlBlaVVOZmZMbzY5NnNKdW9D?= =?utf-8?B?bWRmMEk4N0poMzFreEdzK1ZLbG5OeFhPZUdtNDdENWpWSy9UZDBJVVhLajRZ?= =?utf-8?B?NHZkd2RDNlEwd2cya2dpaVJ0cmNGYThlMTkzemhPSVpRN1RjRTFlVk5KVUg2?= =?utf-8?B?Y2tPc1dFVHp2SnJzajRXL2xDMDZueHZXUzRnMEkvMWNXUHdQaWZHVnltNVN3?= =?utf-8?B?OTZXRTduenZ5TmJEZTFOeEVYTTVvemx6K1JIUElpYkhFZ3p3MFVwdkUxNi85?= =?utf-8?B?SUY4dnQvem9OU0JXSm9rSHFpUlhuYUZZazZpMDFqSzNrNGw3YThIckRXeFdl?= =?utf-8?B?NTltZjRZVTI1YktELzhKUFVPQ1NySlArVVgyMnQ4ODg2WmdXOEJDakE2UHBx?= =?utf-8?B?TVVkZnYzVk9Gc2d2WGVaUll4S3Vzd0VWSGpBczdkMDBZSmNiWDc0eTFGWWM1?= =?utf-8?B?K0dOTnFRUFdnPT0=?= X-OriginatorOrg: outlook.com X-MS-Exchange-CrossTenant-Network-Message-Id: 179cc016-30ab-4d1c-4e64-08da28b47b72 X-MS-Exchange-CrossTenant-AuthSource: AS8PR01MB7944.eurprd01.prod.exchangelabs.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 28 Apr 2022 01:14:46.4169 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 84df9e7f-e9f6-40af-b435-aaaaaaaaaaaa X-MS-Exchange-CrossTenant-RMS-PersistedConsumerOrg: 00000000-0000-0000-0000-000000000000 X-MS-Exchange-Transport-CrossTenantHeadersStamped: PAXPR01MB9121 Subject: Re: [FFmpeg-devel] [PATCH] libavcodec/qsvdec.c: extract frame packing arrangement data X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" Archived-At: List-Archive: List-Post: Wenbin Chen: > Use h264_sei to parse SEI data got from MediaSDK. Extract frame > packing arrangement information from SEI data and config AVStereo3D > side data for decoded frame. > > Signed-off-by: Wenbin Chen > Signed-off-by: Tong Wu > --- > libavcodec/qsv_internal.h | 2 + > libavcodec/qsvdec.c | 160 ++++++++++++++++++++++++++++++++++++++ > 2 files changed, 162 insertions(+) > > diff --git a/libavcodec/qsv_internal.h b/libavcodec/qsv_internal.h > index e2aecdcbd6..a804c392c1 100644 > --- a/libavcodec/qsv_internal.h > +++ b/libavcodec/qsv_internal.h > @@ -54,6 +54,8 @@ > > #define QSV_MAX_FRAME_EXT_PARAMS 4 > > +#define QSV_PAYLOAD_SIZE 1024 > + > #define QSV_VERSION_ATLEAST(MAJOR, MINOR) \ > (MFX_VERSION_MAJOR > (MAJOR) || \ > MFX_VERSION_MAJOR == (MAJOR) && MFX_VERSION_MINOR >= (MINOR)) > diff --git a/libavcodec/qsvdec.c b/libavcodec/qsvdec.c > index 5fc5bed4c8..26fa178b4d 100644 > --- a/libavcodec/qsvdec.c > +++ b/libavcodec/qsvdec.c > @@ -41,13 +41,16 @@ > #include "libavutil/time.h" > #include "libavutil/imgutils.h" > #include "libavutil/film_grain_params.h" > +#include "libavutil/stereo3d.h" > > #include "avcodec.h" > #include "codec_internal.h" > #include "internal.h" > #include "decode.h" > #include "hwconfig.h" > +#include "get_bits.h" > #include "qsv.h" > +#include "h264_sei.h" > #include "qsv_internal.h" > > static const AVRational mfx_tb = { 1, 90000 }; > @@ -101,6 +104,10 @@ typedef struct QSVContext { > > mfxExtBuffer **ext_buffers; > int nb_ext_buffers; > + > + mfxPayload payload; > + H264SEIContext sei; > + H264ParamSets ps; > } QSVContext; > > static const AVCodecHWConfigInternal *const qsv_hw_configs[] = { > @@ -600,6 +607,150 @@ static int qsv_export_film_grain(AVCodecContext *avctx, mfxExtAV1FilmGrainParam > } > #endif > > +static int h264_decode_fpa(H264SEIFramePacking *fpa, AVFrame *frame) > +{ > + if (!fpa || !frame) > + return AVERROR(EINVAL); > + > + if (!fpa->arrangement_cancel_flag && > + fpa->arrangement_type <= 6 && > + fpa->content_interpretation_type > 0 && > + fpa->content_interpretation_type < 3) { > + AVStereo3D *stereo = av_stereo3d_create_side_data(frame); > + if (stereo) { > + switch (fpa->arrangement_type) { > + case 0: > + stereo->type = AV_STEREO3D_CHECKERBOARD; > + break; > + case 1: > + stereo->type = AV_STEREO3D_COLUMNS; > + break; > + case 2: > + stereo->type = AV_STEREO3D_LINES; > + break; > + case 3: > + if (fpa->quincunx_sampling_flag) > + stereo->type = AV_STEREO3D_SIDEBYSIDE_QUINCUNX; > + else > + stereo->type = AV_STEREO3D_SIDEBYSIDE; > + break; > + case 4: > + stereo->type = AV_STEREO3D_TOPBOTTOM; > + break; > + case 5: > + stereo->type = AV_STEREO3D_FRAMESEQUENCE; > + if (fpa->current_frame_is_frame0_flag) > + stereo->view = AV_STEREO3D_VIEW_LEFT; > + else > + stereo->view = AV_STEREO3D_VIEW_RIGHT; > + break; > + case 6: > + stereo->type = AV_STEREO3D_2D; > + break; > + } > + > + if (fpa->content_interpretation_type == 2) > + stereo->flags = AV_STEREO3D_FLAG_INVERT; > + } > + } > + return 0; > +} > + > +static int h264_parse_side_data(AVCodecContext *avctx, QSVContext *q, AVFrame *frame) > +{ > + GetBitContext gb_payload; > + uint8_t *sei_buffer; > + int sei_buffer_index; > + int ret; > + > + if (q->payload.Type != SEI_TYPE_FRAME_PACKING_ARRANGEMENT) > + return 0; > + > + sei_buffer = (uint8_t *)av_mallocz(q->payload.NumBit / 8); > + if (!sei_buffer) { > + av_freep(&sei_buffer); > + return AVERROR(ENOMEM); > + } > + /* remove emulation prevention bytes */ > + sei_buffer_index = 0; > + for (int i = 0; i < q->payload.NumBit / 8; i++) { > + if (q->payload.Data[i] == 3) > + i++; > + sei_buffer[sei_buffer_index] = q->payload.Data[i]; > + sei_buffer_index += 1; Not every 0x03 is an emulation prevention byte. > + } > + > + ret = init_get_bits8(&gb_payload, sei_buffer, sei_buffer_index+1); Buffers used with the get-bits API need to be padded. Best to use av_fast_padded_malloc and ff_h2645_extract_rbsp. > + if (ret < 0) { > + av_freep(&sei_buffer); > + return ret; > + } > + > + ret = ff_h264_sei_decode(&q->sei, &gb_payload, &q->ps, avctx); > + if (ret < 0) { > + av_freep(&sei_buffer); > + return ret; > + } > + > + switch (q->payload.Type) { > + case SEI_TYPE_FRAME_PACKING_ARRANGEMENT: > + ret = h264_decode_fpa(&q->sei.frame_packing, frame); > + break; > + default: > + break; > + } > + > + av_freep(&sei_buffer); > + return ret; > +} > + > +static int extract_frame_side_data(AVCodecContext *avctx, QSVContext *q, AVFrame *frame) > +{ > + mfxU64 ts; > + mfxStatus sts; > + int ret = 0; > + > + if (q->payload.BufSize == 0) { > + q->payload.Data = av_mallocz(QSV_PAYLOAD_SIZE); > + if (!q->payload.Data) > + return AVERROR(ENOMEM); > + q->payload.BufSize = QSV_PAYLOAD_SIZE; > + } > + > + sts = MFX_ERR_NONE; > + while (sts == MFX_ERR_NONE) { > + > + sts = MFXVideoDECODE_GetPayload(q->session, &ts, &q->payload); > + if (sts == MFX_ERR_NOT_ENOUGH_BUFFER) { > + av_log(avctx, AV_LOG_WARNING, "Space for SEI is not enough." > + "Realloc buffer\n"); Why this warning? Why a log-message at all? > + if (q->payload.BufSize >= INT16_MAX / 2) Where does this limit come from? > + return AVERROR(ENOMEM); > + q->payload.BufSize = q->payload.BufSize * 2; > + av_freep(&q->payload.Data); > + q->payload.Data = av_mallocz(q->payload.BufSize); > + if (!q->payload.Data) > + return AVERROR(ENOMEM); q->payload.Data is NULL if this happens, but q->payload.BufSize will be > 0, so that q->payload used in the next MFXVideoDECODE_GetPayload call will be inconsistent. > + continue; > + } else if (sts != MFX_ERR_NONE || q->payload.NumBit == 0) { > + break; > + } > + > + switch (avctx->codec_id) { > + case AV_CODEC_ID_H264: > + ret = h264_parse_side_data(avctx, q, frame); > + break; > + default: > + break; > + } > + if (ret < 0) { > + av_log(avctx, AV_LOG_WARNING, "parse side data failed\n"); > + break; > + } > + } > + return ret; > +} > + > static int qsv_decode(AVCodecContext *avctx, QSVContext *q, > AVFrame *frame, int *got_frame, > const AVPacket *avpkt) > @@ -709,6 +860,14 @@ static int qsv_decode(AVCodecContext *avctx, QSVContext *q, > > outsurf = &aframe.frame->surface; > > + ret = extract_frame_side_data(avctx, q, frame); > + if (ret == AVERROR_INVALIDDATA) > + av_log(avctx, AV_LOG_WARNING, "Side data is invalid\n"); > + else if (ret < 0) { > + av_log(avctx, AV_LOG_ERROR, "Error extracting side data from packet\n"); > + return ret; > + } > + > frame->pts = MFX_PTS_TO_PTS(outsurf->Data.TimeStamp, avctx->pkt_timebase); > #if QSV_VERSION_ATLEAST(1, 34) > if ((avctx->export_side_data & AV_CODEC_EXPORT_DATA_FILM_GRAIN) && > @@ -770,6 +929,7 @@ static void qsv_decode_close_qsvcontext(QSVContext *q) > av_buffer_unref(&q->frames_ctx.hw_frames_ctx); > av_buffer_unref(&q->frames_ctx.mids_buf); > av_buffer_pool_uninit(&q->pool); > + av_freep(&q->payload.Data); > } > > static int qsv_process_data(AVCodecContext *avctx, QSVContext *q, _______________________________________________ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".