From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org [79.124.17.100]) by master.gitmailbox.com (Postfix) with ESMTP id A6F95406BF for ; Wed, 22 Dec 2021 14:29:43 +0000 (UTC) Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 71E4868B008; Wed, 22 Dec 2021 16:29:40 +0200 (EET) Received: from EUR05-AM6-obe.outbound.protection.outlook.com (mail-am6eur05olkn2043.outbound.protection.outlook.com [40.92.91.43]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 48DE668AACC for ; Wed, 22 Dec 2021 16:29:38 +0200 (EET) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=JZ2lB1TfpPJv2XWyAgQurTmKtxn7O5bn+QnTMSLqflJxg1NenDuoJSF1ntQTvVArrCxCjjYgwYL6GrXxDYSqZ5I7MJ+nsYcXFc/q9nxC/OQ3OeoyEWNZTV6GvkglPOLYlHrd6rhKcNEgeO1PX90zel6VUwK48pdY+eDMRb77PRT4ZaEU+Py0wwW0fXbnZCPPS+We5ZPzAfArkiChVXsZu+/zUy3RAew8LhtLODFbE4tAEatQjnuMHPAHVEebCnGQJmB80WeXxIztKWWFocTLC7e08lG/XrUR6IoPb01CusFHoBFkD/kIDpjFBNYcvgjTFQCg0Zu0FOTDypbagmQ3oQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=MrVuAgA/HfYmEJ4suyOlz3fThZ5ocEDQio6ZZae3k78=; b=PqMbdbbvNjLgOLZv3jzhBOZnUnQnu0TE6j0hhfjf4o2zBafRmqHxCmM6ZiHF/DN56magvPesZpgigJUPMYHITDnm5YJDNhi5Bzn/fwql5QwYN5PfGNauyzuqi1DXTgee8x6PEZf7LsThQxSx1Lv9SW74qDiXLzqHI/l+pQNLSjQMXUsw0DvDm3JC5/IPslbd/qKh3HSWoPZ289V8GWYeeZ+mU0JlJb0ocYA+6lOsT3OKPi89x9uc1Tj4H9FYYDU1YbxScpUA70H6y4RzSQZPooCe9j/fK7KWV+7lUVp3okocNhGO0/pnth6T3Ck9AIrQ/DXAsWhQVcqRFkGT47vYCw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=none; dmarc=none; dkim=none; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=outlook.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=MrVuAgA/HfYmEJ4suyOlz3fThZ5ocEDQio6ZZae3k78=; b=AM2rnfs9JTSjpuxe5/5cjWAiBQUnh3nfdYY0PcKPpSdnFRDLfjs6OzMfXocIXAKs1Ie9YXnqDITFDZnHOnTxQBV1qG6Cywx0oEMS1O+Izxqc+DgaaoLFPapHsXMBO4D0vn2q0Xb5EuVxF0qj0RDZbe+cvujeS/sNHxw2oyvjZZ+3njOb64t4rlBW2B1kwTD4t68rPkyJfzk8Gr+Wjanbw5OPy2dk3+ReBs0fBNSeACsKWJ+LCpR/lhyeU2OTwl66gmW7p8Ofg8f/SUgh3Ky7B/GhxVou2570svxjorR7yVu1vtkxu1JwMTdWQRIsbq50BQlDHSrZLSsl8f0rbKn1LQ== Received: from AM7PR03MB6660.eurprd03.prod.outlook.com (2603:10a6:20b:1c1::22) by AM6PR03MB3525.eurprd03.prod.outlook.com (2603:10a6:209:32::16) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4801.17; Wed, 22 Dec 2021 14:29:37 +0000 Received: from AM7PR03MB6660.eurprd03.prod.outlook.com ([fe80::f0dc:92f5:6bc2:45ca]) by AM7PR03MB6660.eurprd03.prod.outlook.com ([fe80::f0dc:92f5:6bc2:45ca%6]) with mapi id 15.20.4801.022; Wed, 22 Dec 2021 14:29:37 +0000 Message-ID: Date: Wed, 22 Dec 2021 15:29:35 +0100 Content-Language: en-US To: ffmpeg-devel@ffmpeg.org References: <20211220134420.69078-1-ffmpeg@haasn.xyz> <20211220153122.54839-1-ffmpeg@haasn.xyz> From: Andreas Rheinhardt In-Reply-To: X-TMN: [Y+GHhJlXlLjF17KfzvNCOWs3fwtfMjGp] X-ClientProxiedBy: AM0PR02CA0027.eurprd02.prod.outlook.com (2603:10a6:208:3e::40) To AM7PR03MB6660.eurprd03.prod.outlook.com (2603:10a6:20b:1c1::22) X-Microsoft-Original-Message-ID: <2dc32117-3535-2937-0c79-dd8412e2f527@outlook.com> MIME-Version: 1.0 X-MS-Exchange-MessageSentRepresentingType: 1 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 0e9b631a-2d83-4ebd-862a-08d9c5577b19 X-MS-TrafficTypeDiagnostic: AM6PR03MB3525:EE_ X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: bU4VbiT447Fsz+AQIPbVGESEVah3laJLWXnatYhjucN/umv1kXazGDCAQU06dgFXOYtWpVaMNKssVNmFoVYwGciZKrkzCp/W9tyudNU0QU/Y1VW7IdNHPkQ960ghsVaBLiZowCdZoSzDIQ7RkeLwEgOz67y7RqqqeRE8E9OUfhun87IZiuiWgJv0AcRKq7HZ7iiMyQggF53HvaTvq+GZS1rqMagxqVt9jGCq/MC5Mi8CHqWfkjn7loahB8Az1dQP9+feKp6Qxw76MmvVLyxr4q+E2n8qFtQGmfyyQuS2BSycvb2WUp9C5Hb5J9ILVKXdoUycWohLd+UumANXLNi22a0gDadQm6IHF/5cAvr/JtBg5RN6y8XfvgaGT8OH5c9e8/Q3BJ/q7op0m9uqcQOy2Aoy5AgSKbJR3C8UWcM5AHzzulTBtssoWvBlWPVZXHqmVNeA3jL55InsS5OfgqewpDXNlm9Bl+BfqBJjicqBhvcyj9VpjafHU7+CwVY1oD0VZIJmpGnlP6azpAFNqjrUKwZp4ydxR/b1bcBL5yoOLVt738bbPJk6vhNgJGS5c/hbPn1NcPEbV7Gii3M7854LAQ== X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?B?b284YTU4Z3FRU1QyZWZ4R0RyRFRST1BiWERFcXduWXZubE5HM0p3OUZLVGIz?= =?utf-8?B?SUNvQlFjdVROZzRLQWNBQUk2UUw5cjg0UlNCZXIyMTFsUGNUN2U1SFF5Vjlw?= =?utf-8?B?SkJ0T2dRd3VxRm5iMFdVaGE5VDkwT3dCcGV6NmRNdDFTSlpYK0hMK3dqMjg2?= =?utf-8?B?RENXdEVCblJXRGVIU28wTlBJNDFldFJvTkJoUmpZSXBKcGNaZE10bThYQXFj?= =?utf-8?B?cjRyZFp0U1dDSUJNbGNTMXZaL2VKVFlIMEIzcUo0VVZsVE5pdkl3OXI3azk4?= =?utf-8?B?VGxtY2ZIeTZKenIyMGJBWW1XN0p0NEs2WFREbityRHIyMHZMRVVMekVlc0RS?= =?utf-8?B?RjFFNEgvVHpNd3ozWGlCR1IxN2swbm4weVZEdTk5VjMycGdZeFBaKzhXK2Iz?= =?utf-8?B?Mkkwc0tLMDRabnltSHZocjN6ckZrTlB4WGhIR0NUUjFBVFgyNURLR0kvS3Fz?= =?utf-8?B?U0tVajlhUTE4NFpHMEN5SFl5OGR4VzhMcGtFN1pzTlhxV3I1RWtXbTRIRDFj?= =?utf-8?B?NlBlWUtjbzJmRkZmT2Q1UTNsMEVxQlpqc0VnOHNYTUZJbFpDTEF2RWI2MnJP?= =?utf-8?B?d0pmZFVQZHNndFFrK0E5b3cvSnI0M1ZzQ3IwcmwwVUg2RGlqRXVsb09YaHVU?= =?utf-8?B?ZSs2STVhOG0ybXNSZm85OVppTVF2RVVVNlZZVmUrUVFtKzg1dXN4RTFOTXNp?= =?utf-8?B?Ui9wOFh0WlByOGtnSlFVaFFpM1c0Y05UVEZtYmtMVnlwNzVwTVdwbSt1d1Zz?= =?utf-8?B?UmFMWjdPaktBK2lSNXBCRXFVVitDRjkyOHdKODIrMWZFSHB4T0JONHlXSzJR?= =?utf-8?B?U0xCNEdIM2tpbzFoS2h1QnVGTlpkV25BUzV5eGZ4VTB4Z0JoakVtTTUyTVhF?= =?utf-8?B?bjg1VFJ2VkFhcVpZQ1IvN0VYWmdBUEluTEFtNG02UDA5RHJhczlGZUxnUE1I?= =?utf-8?B?Y3NNMjdaOEFmNzFpcllqUFJYdm9xZ3JOQmdMYzZycU1UZ3BwRUppdUpUcTJL?= =?utf-8?B?a0dQWGNvN0QxOU5DN2paM09JV3c4eGJJQktva3h3eVNSbzBFakFzNDZReFRS?= =?utf-8?B?L0x6YVVodkV2UEQ3ZWswTEw1bEtYUWc4RVRZYzZOSHV4SEU3MzRnRkxmRUJv?= =?utf-8?B?RGI3cGVLc3YrWUFrUm4yMFIvQWFhNktRZE53ZmJHWUZLNlNBMnhQYWd6enBQ?= =?utf-8?B?MmpnV2ZpVVA5TXdCVVVWTEo3WHpraW01YU1Fb0o3YUFDVlNRb1pyMk1keXZY?= =?utf-8?B?aVZRK3NhRjBIZWZVU25aSkNwZlB6SVdNQWFQSmJEWDIyc0JBQT09?= X-OriginatorOrg: outlook.com X-MS-Exchange-CrossTenant-Network-Message-Id: 0e9b631a-2d83-4ebd-862a-08d9c5577b19 X-MS-Exchange-CrossTenant-AuthSource: AM7PR03MB6660.eurprd03.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 22 Dec 2021 14:29:36.9933 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 84df9e7f-e9f6-40af-b435-aaaaaaaaaaaa X-MS-Exchange-CrossTenant-RMS-PersistedConsumerOrg: 00000000-0000-0000-0000-000000000000 X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM6PR03MB3525 Subject: Re: [FFmpeg-devel] [PATCH v8 1/6] lavu/frame: Add Dolby Vision metadata side data type X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" Archived-At: List-Archive: List-Post: Hendrik Leppkes: > On Wed, Dec 22, 2021 at 2:33 PM Andreas Rheinhardt > wrote: >> >> Hendrik Leppkes: >>> On Wed, Dec 22, 2021 at 2:00 PM Lynne wrote: >>>> >>>> 20 Dec 2021, 16:31 by ffmpeg@haasn.xyz: >>>> >>>>> From: Niklas Haas >>>>> >>>>> Signed-off-by: Niklas Haas >>>>> --- >>>>> doc/APIchanges | 3 + >>>>> libavutil/dovi_meta.c | 12 ++++ >>>>> libavutil/dovi_meta.h | 143 ++++++++++++++++++++++++++++++++++++++++++ >>>>> libavutil/frame.c | 1 + >>>>> libavutil/frame.h | 9 ++- >>>>> libavutil/version.h | 2 +- >>>>> 6 files changed, 168 insertions(+), 2 deletions(-) >>>>> >>>>> diff --git a/doc/APIchanges b/doc/APIchanges >>>>> index 17aa664ca3..ff78edec88 100644 >>>>> --- a/doc/APIchanges >>>>> +++ b/doc/APIchanges >>>>> @@ -14,6 +14,9 @@ libavutil: 2021-04-27 >>>>> >>>>> API changes, most recent first: >>>>> >>>>> +2021-12-xx - xxxxxxxxxx - lavu 57.12.100 - frame.h >>>>> + Add AV_FRAME_DATA_DOVI_METADATA. >>>>> + >>>>> 2021-12-xx - xxxxxxxxxx - lavf 59.10.100 - avformat.h >>>>> Add AVFormatContext io_close2 which returns an int >>>>> >>>>> diff --git a/libavutil/dovi_meta.c b/libavutil/dovi_meta.c >>>>> index 7bd08f6c54..60b4cb2376 100644 >>>>> --- a/libavutil/dovi_meta.c >>>>> +++ b/libavutil/dovi_meta.c >>>>> @@ -33,3 +33,15 @@ AVDOVIDecoderConfigurationRecord *av_dovi_alloc(size_t *size) >>>>> >>>>> return dovi; >>>>> } >>>>> + >>>>> +AVDOVIMetadata *av_dovi_metadata_alloc(size_t *size) >>>>> +{ >>>>> + AVDOVIMetadata *dovi = av_mallocz(sizeof(AVDOVIMetadata)); >>>>> + if (!dovi) >>>>> + return NULL; >>>>> + >>>>> + if (size) >>>>> + *size = sizeof(*dovi); >>>>> + >>>>> + return dovi; >>>>> +} >>>>> diff --git a/libavutil/dovi_meta.h b/libavutil/dovi_meta.h >>>>> index 299911d434..25e6d7b42f 100644 >>>>> --- a/libavutil/dovi_meta.h >>>>> +++ b/libavutil/dovi_meta.h >>>>> @@ -29,6 +29,7 @@ >>>>> >>>>> #include >>>>> #include >>>>> +#include "rational.h" >>>>> >>>>> /* >>>>> * DOVI configuration >>>>> @@ -67,4 +68,146 @@ typedef struct AVDOVIDecoderConfigurationRecord { >>>>> */ >>>>> AVDOVIDecoderConfigurationRecord *av_dovi_alloc(size_t *size); >>>>> >>>>> +/** >>>>> + * Dolby Vision RPU data header. >>>>> + * >>>>> + * @note Cannot be extended without an ABI bump. >>>>> + */ >>>>> +typedef struct AVDOVIRpuDataHeader { >>>>> + uint8_t rpu_type; >>>>> + uint16_t rpu_format; >>>>> + uint8_t vdr_rpu_profile; >>>>> + uint8_t vdr_rpu_level; >>>>> + uint8_t chroma_resampling_explicit_filter_flag; >>>>> + uint8_t coef_data_type; /* informative, lavc always converts to fixed */ >>>>> + uint8_t coef_log2_denom; >>>>> + uint8_t vdr_rpu_normalized_idc; >>>>> + uint8_t bl_video_full_range_flag; >>>>> + uint8_t bl_bit_depth; /* [8, 16] */ >>>>> + uint8_t el_bit_depth; /* [8, 16] */ >>>>> + uint8_t vdr_bit_depth; /* [8, 16] */ >>>>> + uint8_t spatial_resampling_filter_flag; >>>>> + uint8_t el_spatial_resampling_filter_flag; >>>>> + uint8_t disable_residual_flag; >>>>> +} AVDOVIRpuDataHeader; >>>>> + >>>>> +enum AVDOVIMappingMethod { >>>>> + AV_DOVI_MAPPING_POLYNOMIAL = 0, >>>>> + AV_DOVI_MAPPING_MMR = 1, >>>>> +}; >>>>> + >>>>> +/** >>>>> + * Coefficients of a piece-wise function. The pieces of the function span the >>>>> + * value ranges between two adjacent pivot values. >>>>> + * >>>>> + * @note Cannot be extended without an ABI bump. >>>>> + */ >>>>> +#define AV_DOVI_MAX_PIECES 8 >>>>> +typedef struct AVDOVIReshapingCurve { >>>>> + uint8_t num_pivots; /* [2, 9] */ >>>>> + uint16_t pivots[AV_DOVI_MAX_PIECES + 1]; /* sorted ascending */ >>>>> + enum AVDOVIMappingMethod mapping_idc[AV_DOVI_MAX_PIECES]; >>>>> + /* AV_DOVI_MAPPING_POLYNOMIAL */ >>>>> + uint8_t poly_order[AV_DOVI_MAX_PIECES]; /* [1, 2] */ >>>>> + int64_t poly_coef[AV_DOVI_MAX_PIECES][3]; /* x^0, x^1, x^2 */ >>>>> + /* AV_DOVI_MAPPING_MMR */ >>>>> + uint8_t mmr_order[AV_DOVI_MAX_PIECES]; /* [1, 3] */ >>>>> + int64_t mmr_constant[AV_DOVI_MAX_PIECES]; >>>>> + int64_t mmr_coef[AV_DOVI_MAX_PIECES][3/* order - 1 */][7]; >>>>> +} AVDOVIReshapingCurve; >>>>> + >>>>> +enum AVDOVINLQMethod { >>>>> + AV_DOVI_NLQ_NONE = -1, >>>>> + AV_DOVI_NLQ_LINEAR_DZ = 0, >>>>> +}; >>>>> + >>>>> +/** >>>>> + * Coefficients of the non-linear inverse quantization. For the interpretation >>>>> + * of these, see ETSI GS CCM 001. >>>>> + * >>>>> + * @note Cannot be extended without an ABI bump. >>>>> + */ >>>>> +typedef struct AVDOVINLQParams { >>>>> + uint64_t nlq_offset; >>>>> + uint64_t vdr_in_max; >>>>> + /* AV_DOVI_NLQ_LINEAR_DZ */ >>>>> + uint64_t linear_deadzone_slope; >>>>> + uint64_t linear_deadzone_threshold; >>>>> +} AVDOVINLQParams; >>>>> + >>>>> +/** >>>>> + * Dolby Vision RPU data mapping parameters. >>>>> + * >>>>> + * @note Cannot be extended without an ABI bump. >>>>> + */ >>>>> +typedef struct AVDOVIDataMapping { >>>>> + uint8_t vdr_rpu_id; >>>>> + uint8_t mapping_color_space; >>>>> + uint8_t mapping_chroma_format_idc; >>>>> + AVDOVIReshapingCurve curves[3]; /* per component */ >>>>> + >>>>> + /* Non-linear inverse quantization */ >>>>> + enum AVDOVINLQMethod nlq_method_idc; >>>>> + uint32_t num_x_partitions; >>>>> + uint32_t num_y_partitions; >>>>> + AVDOVINLQParams nlq[3]; /* per component */ >>>>> +} AVDOVIDataMapping; >>>>> + >>>>> +typedef struct AVDOVIColorMetadata { >>>>> + uint8_t dm_metadata_id; >>>>> + uint8_t scene_refresh_flag; >>>>> + >>>>> + /** >>>>> + * Coefficients of the custom Dolby Vision IPT-PQ matrices. These are to be >>>>> + * used instead of the matrices indicated by the frame's colorspace tags. >>>>> + * The output of rgb_to_lms_matrix is to be fed into a BT.2020 LMS->RGB >>>>> + * matrix based on a Hunt-Pointer-Estevez transform, but without any >>>>> + * crosstalk. (See the definition of the ICtCp colorspace for more >>>>> + * information.) >>>>> + */ >>>>> + AVRational ycc_to_rgb_matrix[9]; /* before PQ linearization */ >>>>> + AVRational ycc_to_rgb_offset[3]; /* input offset of neutral value */ >>>>> + AVRational rgb_to_lms_matrix[9]; /* after PQ linearization */ >>>>> + >>>>> + /** >>>>> + * Extra signal metadata (see Dolby patents for more info). >>>>> + */ >>>>> + uint16_t signal_eotf; >>>>> + uint16_t signal_eotf_param0; >>>>> + uint16_t signal_eotf_param1; >>>>> + uint32_t signal_eotf_param2; >>>>> + uint8_t signal_bit_depth; >>>>> + uint8_t signal_color_space; >>>>> + uint8_t signal_chroma_format; >>>>> + uint8_t signal_full_range_flag; /* [0, 3] */ >>>>> + uint16_t source_min_pq; >>>>> + uint16_t source_max_pq; >>>>> + uint16_t source_diagonal; >>>>> +} AVDOVIColorMetadata; >>>>> + >>>>> +/** >>>>> + * Combined struct representing a combination of header, mapping and color >>>>> + * metadata, for attaching to frames as side data. >>>>> + * >>>>> + * @note The struct must be allocated with av_dovi_metadata_alloc() and >>>>> + * its size is not a part of the public ABI. >>>>> + */ >>>>> + >>>>> +typedef struct AVDOVIMetadata { >>>>> + AVDOVIRpuDataHeader header; >>>>> + AVDOVIDataMapping mapping; >>>>> + AVDOVIColorMetadata color; >>>>> >>>> >>>> I think we should version this by adding an integer version >>>> field, and a compile-time header #define. >>>> just in case more extra signalling is added. Then >>>> we could support it in a sort of backwards-compatible way >>>> by just documenting what has changed, and users could >>>> make their own decisions about whether to present it >>>> as-is or error out. >>>> >>> >>> That seems rather pointless to me when API users can just tie that to >>> avutil or avcodec version. >>> Additionally, how would I ever make use of that? We never support >>> running binaries older then the headers you have, and newer data can't >>> be used because you don't have the struct definition that corresponds >>> with it. >>> >> >> One way to make these structs extensible while keeping this side data a >> single buffer is by adding the offsets of header, mapping and color to >> the start of AVDOVIMetadata at the start of AVDOVIMetadata. That way >> users could use the parts they know and ignore the rest. It would even >> allow for adding further structs to AVDOVIMetadata. >> > > How would you define the struct then? If we define it as a normal > struct and people naively access stuff, it might suddenly point to > something else - so if we allow us to change it, then we probably > should define it in a way that doesn't even allow the naive accidental > mis-reading. > Keeping that in mind then, it sounds like it would get rather > complicated to use. > Similar to how it is done in detection_bbox.h: Add some inline accessor functions. We could even remove the substructs from the publically visibly part of the struct altogether and only allow access through these accessors. - Andreas _______________________________________________ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".