From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org [79.124.17.100]) by master.gitmailbox.com (Postfix) with ESMTP id CE50846976 for ; Thu, 29 Jun 2023 20:34:50 +0000 (UTC) Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 79E5668C29B; Thu, 29 Jun 2023 23:34:47 +0300 (EEST) Received: from EUR05-AM6-obe.outbound.protection.outlook.com (mail-am6eur05olkn2029.outbound.protection.outlook.com [40.92.91.29]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 71CD268BEFF for ; Thu, 29 Jun 2023 23:34:41 +0300 (EEST) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=SXM4QgrwxfQxh9rsSBDDPwNHXCdjcDRz234EoSQb4WFiUnE/LccJCEezBFIcDuAfJ05jTWhC+y1hlsS+CWsl5v9uRQygjecJvgh5rk1PrPATebDw+/TfhV9U/4h3rGM0d/yaq/06l/WdFuzcMC+Hv6+f7adB4T2KrMLBgHijLKYreypF37deqbZq+MRmWyTdlCTgnkzJquNXConjT5BXdppw3eFL5A89GCNxy8YaqAztIGOMVEPiKtMDSHUo2Hf+GEFgE0Xk9NBTPQGzXpgBCmfrKMumSfdVI8ewm9P7up7JsJ6bugMiFafNgcL4FEbELz650No3ExelLskPe0zPfQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=dNXYyaWw7Gj9XfS3ZcZyOM2IjO11bMXaesj7n/kCVbw=; b=Mm+/l4vm64JVd8WkTrxEkPoluoolbmO/z9LlZpceDMS3hctTxOW6Bs0d4ECqErZhL0EvmMFg7Ouq1/yIsx/yQadNBWgbhpcRwau8XeJMxv0eESoA+3uL8+KMdkFvCPF1YqOH9M3+jy7wGq5CwSyFvZIrQo4DG3whnMr4Vxi7+XfQCzTJdzmYMRU6EqPi01wgDUmJ7R7ivXOaCqbIUP3OVYYYxWqmzkx664a81Uws/Pw5I+Z/gvw233QA02X6cfEwyR2eDBECCADtSrdYABWweqEtum9i+MYVgzzbEYSx6GfAcxgRDca7ak5dCnV8DPvtdveSYrR8PZUrch0VMU77eA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=none; dmarc=none; dkim=none; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=outlook.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=dNXYyaWw7Gj9XfS3ZcZyOM2IjO11bMXaesj7n/kCVbw=; b=Mx5zniszeJ+j5pvT+zJ6D2vRV3ytBuRNloTc49dbvHUOaHAR8xs+/lTFShYsR9CXECGcdwh6Ed8R5KiJuxOIXnVTar3rfmOQ+ppnQGhP+y02HMK5Pwe6O6IMbEN+nguadnn7DqUErdQhEfy7cmOsoO0lvKbgRfqdPtGIZTccFgtarua3slK7LqxGVFZAo+KMLOQs7fswzUYEEdW3TxEHP8cdkQZ9SSId3FqhEKPTH9y4NQpgj22IY6hKfL2Jg4AjvlmLvkHgvNQgj5Vlg7Xp8mjwLpGgYectU5HvhmVX4SgeX0KHLYL6wyMgkwgy5u7eZjMHkI1BPWlol8KaIcC7Kw== Received: from AS8P250MB0744.EURP250.PROD.OUTLOOK.COM (2603:10a6:20b:541::14) by AS8P250MB0330.EURP250.PROD.OUTLOOK.COM (2603:10a6:20b:37d::14) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6544.19; Thu, 29 Jun 2023 20:34:38 +0000 Received: from AS8P250MB0744.EURP250.PROD.OUTLOOK.COM ([fe80::a38:e93f:c7e0:f473]) by AS8P250MB0744.EURP250.PROD.OUTLOOK.COM ([fe80::a38:e93f:c7e0:f473%7]) with mapi id 15.20.6521.026; Thu, 29 Jun 2023 20:34:38 +0000 Message-ID: Date: Thu, 29 Jun 2023 22:35:47 +0200 Content-Language: en-US To: ffmpeg-devel@ffmpeg.org References: <20230321150124.21999-1-thomas.ff@spin-digital.com> <20230321150124.21999-4-thomas.ff@spin-digital.com> From: Andreas Rheinhardt In-Reply-To: <20230321150124.21999-4-thomas.ff@spin-digital.com> X-TMN: [0aSSWwb99CBDZeH/Yc94PetdTQyhZydV] X-ClientProxiedBy: FR0P281CA0166.DEUP281.PROD.OUTLOOK.COM (2603:10a6:d10:b4::18) To AS8P250MB0744.EURP250.PROD.OUTLOOK.COM (2603:10a6:20b:541::14) X-Microsoft-Original-Message-ID: <391a85b6-eb8f-117b-ff07-d65b0962373c@outlook.com> MIME-Version: 1.0 X-MS-Exchange-MessageSentRepresentingType: 1 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: AS8P250MB0744:EE_|AS8P250MB0330:EE_ X-MS-Office365-Filtering-Correlation-Id: 167fe591-10de-4887-af5f-08db78e0427a X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: VpQDLz6wZueuLtB7tWtmXLLWQJwgzC4G/AeCBDjyYxz8mXWUGlxVDv3ycGsW6LvtsgGflhOGAIZlswdSGdGnAUgOJ8fzOHkjFPKz22BMaIausf/pkTPp8+sfE1BKt+J83gQQ2dS74UTkKwRmWrhA0d9r+5dKejb7+3mwwt3jrokcaPhufbBlpNxVIE9uf1qcIHCfHenYshYPb8u5awxxvWJo6hJWspkpab2O6x7HJwM4IjZ9X+4UYSU/iEYTTbOJyUL6agLh1LHP//zd8Qg3uv8ny8mzm/8GIBEhqjDt+WljH9AGRCsV4HBXRJCIvpHjglhxxmxTrU9UU8bVYeia35bTNiLqs9bJhprJN7ZI/Vx6N3EQPEZ9X8KgC2+p74XciepjTH/A+RQIt361VkuPK4klK26Ns6W/oJoTWv+RPR8bz0lwP2ILmiclZ93UN+BxFoFTLtfUFO3U8ofWb+uuDwSi96YO/kjuXZgv0MSmtRz8r6QiQzYOd3Kz72sFDRP/bu6LshXcYql74yYNO2PGDzVVvbucEfCRSIj6n9Uw7LOJecYSp3cm40HP7mw+dbkx X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?B?ZGJIOFVCamkxMnV1ZUFKdjNOZWUvbGJiVWMyRkdUN084cjd0Z1pvRG9zaWJB?= =?utf-8?B?alh0S1hWbGFidFRDTm9NdmFVWG1kUHozRHUvVjZSSzFTamZNbEsvQ3NtTEIw?= =?utf-8?B?Z0RXTjVoemc4SkQyMFlwVHIzL2R6Z1pTV0phYnlGaE5VZldWaTZUTVpwZDFs?= =?utf-8?B?T2FTZmdHT3lrUSsrdnA3SWxzUVpOS3RHUFRNd0VqaWhhSzBkd0ZwZ1FMSWJV?= =?utf-8?B?Z0E2SVdGNDFQcTludkFyWGpYTnNrZ3k2R21CRUYveWcwSjlkYXN3NG1WMVYw?= =?utf-8?B?a3ZMUnR6dWMzckdCUmtZSEJpb0NlS0RZUHk0OWFnZ3B0OFd4UzZETFhZeE1M?= =?utf-8?B?aW9PTDNIRGsyWGVWYVZHRUZuTTAzSnZXWHVkRGVYVEk5TVhvT0hpK1JtMkp6?= =?utf-8?B?MlBXWjl3UTh0dWVLVUQ5U1ozQmZIako5QXM4VG1ZNnFiRjNUV1N3SmkvaUVI?= =?utf-8?B?RlNVTmJMNUxTbmRSWHpEOHczU2hhVFRnM0twdk9WeUc1OGFJblJaK0RENi9j?= =?utf-8?B?TjZyNWtPQ1NMbTd5UXg1OFpjNDJ1U1A2V1VKaElMbDVwdGUrcFkzcjhjNmRT?= =?utf-8?B?c0RTUm9HeFpEaXFLRzkyVzlCTytJYVNFdjNtM29xN1RJSWdyanh6S1VYUFJz?= =?utf-8?B?REJMWGJBck1Wamc1UTJJQ25QeDdOWFB4cC9kZmNycXoyTktXNGFYdzlXc1hT?= =?utf-8?B?Mkw3ZU02MUZwS0g1cWlVQi96ZmZIREk5RFNkUlZBWk01SlVlNVp2aC83cFZv?= =?utf-8?B?LzltaGRQVSs1VVNPdHJ3VWNTNWVkY2p3cHpoTUlMRStOWjVRNHFuL0sxZG5N?= =?utf-8?B?ckNCTURqOWswbk83VElKMTR2eFlqcjV1ME9YODE2ZE9kVWRlckJHT1VoZWs0?= =?utf-8?B?UktSRW9GWkg3dzlHSlA1ZURWWklzSzFZQUorSXBSTmFUWlZoVTM4YlNUemQv?= =?utf-8?B?T2tsSnM2STY1REZxZ1EzNnE4NlR0QUxjMjhJTGxCSEtPWDRtLzJtRjlTTXhV?= =?utf-8?B?OE16OG9IeGsyYW1paDgvREw5Mjl1LzlQeWNyb1V2VHN5T1JuMHFrc0ZiaHoy?= =?utf-8?B?bVM0eHlyUUJST3JONHAwclYweHhWY1Yzb3JEVUdnelIyM1ZrVVdJOVVKaDkv?= =?utf-8?B?KzdaaUFzdVorejBSSFFEbFR6Y1BpKzBLNUpIOTZiWGNHM2JZNFFkdFpnWG1D?= =?utf-8?B?ZGdhdFZrQTNSVXpUa1ZtVVFNbEd5MmVlTzQwNFpBakFWVm05KzZkTXB0LzNB?= =?utf-8?B?N2h2dHZXZTlDOXBuVW5rbDNQRmxOaVFUZHFyR0FDa00wb09QeVZpdzhEQWFR?= =?utf-8?B?cWlleFFkWFovNmtqWGt4aG9rTDFmSUVNT3hSL2xLUEFtc0dTLzdOYXlXK0I2?= =?utf-8?B?bWdDL1ljdXo1c2Zvd0Nqa21yM1htSm83dmU2TUlPMHR6QTFyRzU0RVJHeFI5?= =?utf-8?B?N2JEWDlGNS9MWUxpZS9Vc09pK0tWMmZBd0ZXeUlaSzJvbHo4elN5UEtPbUVl?= =?utf-8?B?MG1GNzRUTk1LNVpVTnNqWXNkL21tbVJCdXJlbDlQNEdpT3lnODVEQlRTVWNU?= =?utf-8?B?Q1ZmWFAyZVdXODFVNGppaXFPbGZEN0hHaHhNNk5QSmZHaXRTcSt4NUhHNXhR?= =?utf-8?B?V0I2VzFLa2RHM2lDUnZVSDJoRmY1V09laXhST1FEakp5RHZYMHUvbzQ1K3RG?= =?utf-8?Q?zkpsyb/Oxl0pRJvs2OZo?= X-OriginatorOrg: outlook.com X-MS-Exchange-CrossTenant-Network-Message-Id: 167fe591-10de-4887-af5f-08db78e0427a X-MS-Exchange-CrossTenant-AuthSource: AS8P250MB0744.EURP250.PROD.OUTLOOK.COM X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 29 Jun 2023 20:34:38.8695 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 84df9e7f-e9f6-40af-b435-aaaaaaaaaaaa X-MS-Exchange-CrossTenant-RMS-PersistedConsumerOrg: 00000000-0000-0000-0000-000000000000 X-MS-Exchange-Transport-CrossTenantHeadersStamped: AS8P250MB0330 Subject: Re: [FFmpeg-devel] [PATCH v7 03/11] avcodec: add bitstream parser for H266/VVC X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" Archived-At: List-Archive: List-Post: Thomas Siedel: > From: Nuo Mi > > Add nal parser ff_vvc_parser to parse vvc elementary bitstreams. > > Co-authored-by: Thomas Siedel > --- > configure | 1 + > libavcodec/Makefile | 2 + > libavcodec/h2645_parse.c | 71 ++++- > libavcodec/h266_parser.c | 601 +++++++++++++++++++++++++++++++++++++++ > libavcodec/parsers.c | 1 + > 5 files changed, 675 insertions(+), 1 deletion(-) > create mode 100644 libavcodec/h266_parser.c > > diff --git a/configure b/configure > index 91964edc8f..2fb167c7c3 100755 > --- a/configure > +++ b/configure > @@ -3266,6 +3266,7 @@ av1_amf_encoder_deps="amf" > aac_parser_select="adts_header mpeg4audio" > av1_parser_select="cbs_av1" > h264_parser_select="golomb h264dsp h264parse h264_sei" > +h266_parser_select="cbs_h266" > hevc_parser_select="hevcparse hevc_sei" > mpegaudio_parser_select="mpegaudioheader" > mpeg4video_parser_select="h263dsp mpegvideodec qpeldsp" > diff --git a/libavcodec/Makefile b/libavcodec/Makefile > index abae4909d2..79c4363f3d 100644 > --- a/libavcodec/Makefile > +++ b/libavcodec/Makefile > @@ -76,6 +76,7 @@ OBJS-$(CONFIG_CBS) += cbs.o cbs_bsf.o > OBJS-$(CONFIG_CBS_AV1) += cbs_av1.o > OBJS-$(CONFIG_CBS_H264) += cbs_h2645.o cbs_sei.o h2645_parse.o > OBJS-$(CONFIG_CBS_H265) += cbs_h2645.o cbs_sei.o h2645_parse.o > +OBJS-$(CONFIG_CBS_H266) += cbs_h2645.o cbs_sei.o h2645_parse.o > OBJS-$(CONFIG_CBS_JPEG) += cbs_jpeg.o > OBJS-$(CONFIG_CBS_MPEG2) += cbs_mpeg2.o > OBJS-$(CONFIG_CBS_VP9) += cbs_vp9.o > @@ -1166,6 +1167,7 @@ OBJS-$(CONFIG_GSM_PARSER) += gsm_parser.o > OBJS-$(CONFIG_H261_PARSER) += h261_parser.o > OBJS-$(CONFIG_H263_PARSER) += h263_parser.o > OBJS-$(CONFIG_H264_PARSER) += h264_parser.o h264data.o > +OBJS-$(CONFIG_H266_PARSER) += h266_parser.o > OBJS-$(CONFIG_HEVC_PARSER) += hevc_parser.o hevc_data.o > OBJS-$(CONFIG_HDR_PARSER) += hdr_parser.o > OBJS-$(CONFIG_IPU_PARSER) += ipu_parser.o > diff --git a/libavcodec/h2645_parse.c b/libavcodec/h2645_parse.c > index 90944177c7..5261c3e568 100644 > --- a/libavcodec/h2645_parse.c > +++ b/libavcodec/h2645_parse.c > @@ -30,6 +30,7 @@ > #include "hevc.h" > #include "h264.h" > #include "h2645_parse.h" > +#include "h266.h" > > int ff_h2645_extract_rbsp(const uint8_t *src, int length, > H2645RBSP *rbsp, H2645NAL *nal, int small_padding) > @@ -145,6 +146,47 @@ nsc: > return si; > } > > +static const char *const h266_nal_type_name[32] = { > + "TRAIL_NUT", // VVC_TRAIL_NUT > + "STSA_NUT", // VVC_STSA_NUT > + "RADL_NUT", // VVC_RADL_NUT > + "RASL_NUT", // VVC_RASL_NUT > + "RSV_VCL_4", // VVC_RSV_VCL_4 > + "RSV_VCL_5", // VVC_RSV_VCL_5 > + "RSV_VCL_6", // VVC_RSV_VCL_6 > + "IDR_W_RADL", // VVC_IDR_W_RADL > + "IDR_N_LP", // VVC_IDR_N_LP > + "CRA_NUT", // VVC_CRA_NUT > + "GDR_NUT", // VVC_GDR_NUT > + "RSV_IRAP_11", // VVC_RSV_IRAP_11 > + "OPI_NUT", // VVC_OPI_NUT > + "DCI_NUT", // VVC_DCI_NUT > + "VPS_NUT", // VVC_VPS_NUT > + "SPS_NUT", // VVC_SPS_NUT > + "PPS_NUT", // VVC_PPS_NUT > + "PREFIX_APS_NUT",// VVC_PREFIX_APS_NUT > + "SUFFIX_APS_NUT",// VVC_SUFFIX_APS_NUT > + "PH_NUT", // VVC_PH_NUT > + "AUD_NUT", // VVC_AUD_NUT > + "EOS_NUT", // VVC_EOS_NUT > + "EOB_NUT", // VVC_EOB_NUT > + "PREFIX_SEI_NUT",// VVC_PREFIX_SEI_NUT > + "SUFFIX_SEI_NUT",// VVC_SUFFIX_SEI_NUT > + "FD_NUT", // VVC_FD_NUT > + "RSV_NVCL_26", // VVC_RSV_NVCL_26 > + "RSV_NVCL_27", // VVC_RSV_NVCL_27 > + "UNSPEC_28", // VVC_UNSPEC_28 > + "UNSPEC_29", // VVC_UNSPEC_29 > + "UNSPEC_30", // VVC_UNSPEC_30 > + "UNSPEC_31", // VVC_UNSPEC_31 > +}; > + > +static const char *h266_nal_unit_name(int nal_type) > +{ > + av_assert0(nal_type >= 0 && nal_type < 32); > + return h266_nal_type_name[nal_type]; > +} > + > static const char *const hevc_nal_type_name[64] = { > "TRAIL_N", // HEVC_NAL_TRAIL_N > "TRAIL_R", // HEVC_NAL_TRAIL_R > @@ -293,6 +335,31 @@ static int get_bit_length(H2645NAL *nal, int min_size, int skip_trailing_zeros) > * @return AVERROR_INVALIDDATA if the packet is not a valid NAL unit, > * 0 otherwise > */ > +static int h266_parse_nal_header(H2645NAL *nal, void *logctx) > +{ > + GetBitContext *gb = &nal->gb; > + > + if (get_bits1(gb) != 0) //forbidden_zero_bit > + return AVERROR_INVALIDDATA; > + > + skip_bits1(gb); //nuh_reserved_zero_bit > + > + nal->nuh_layer_id = get_bits(gb, 6); > + nal->type = get_bits(gb, 5); > + nal->temporal_id = get_bits(gb, 3) - 1; > + if (nal->temporal_id < 0) > + return AVERROR_INVALIDDATA; > + > + if ((nal->type >= VVC_IDR_W_RADL && nal->type <= VVC_RSV_IRAP_11) && nal->temporal_id) > + return AVERROR_INVALIDDATA; > + > + av_log(logctx, AV_LOG_DEBUG, > + "nal_unit_type: %d(%s), nuh_layer_id: %d, temporal_id: %d\n", > + nal->type, h266_nal_unit_name(nal->type), nal->nuh_layer_id, nal->temporal_id); > + > + return 0; > +} > + > static int hevc_parse_nal_header(H2645NAL *nal, void *logctx) > { > GetBitContext *gb = &nal->gb; > @@ -509,7 +576,9 @@ int ff_h2645_packet_split(H2645Packet *pkt, const uint8_t *buf, int length, > /* Reset type in case it contains a stale value from a previously parsed NAL */ > nal->type = 0; > > - if (codec_id == AV_CODEC_ID_HEVC) > + if (codec_id == AV_CODEC_ID_VVC) > + ret = h266_parse_nal_header(nal, logctx); > + else if (codec_id == AV_CODEC_ID_HEVC) > ret = hevc_parse_nal_header(nal, logctx); > else > ret = h264_parse_nal_header(nal, logctx); > diff --git a/libavcodec/h266_parser.c b/libavcodec/h266_parser.c > new file mode 100644 > index 0000000000..fbf8177261 > --- /dev/null > +++ b/libavcodec/h266_parser.c > @@ -0,0 +1,601 @@ > +/* > + * H.266 / VVC parser > + * > + * Copyright (C) 2021 Nuo Mi > + * > + * This file is part of FFmpeg. > + * > + * FFmpeg is free software; you can redistribute it and/or > + * modify it under the terms of the GNU Lesser General Public > + * License as published by the Free Software Foundation; either > + * version 2.1 of the License, or (at your option) any later version. > + * > + * FFmpeg is distributed in the hope that it will be useful, > + * but WITHOUT ANY WARRANTY; without even the implied warranty of > + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU > + * Lesser General Public License for more details. > + * > + * You should have received a copy of the GNU Lesser General Public > + * License along with FFmpeg; if not, write to the Free Software > + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA > + */ > + > +#include "cbs.h" > +#include "cbs_h266.h" > +#include "internal.h" > +#include "parser.h" > +#include "decode.h" > + > +#define START_CODE 0x000001 ///< start_code_prefix_one_3bytes > +#define IS_IDR(nut) (nut == VVC_IDR_W_RADL || nut == VVC_IDR_N_LP) > + > +typedef struct PuInfo { > + AVBufferRef *sps_ref; > + AVBufferRef *pps_ref; > + AVBufferRef *slice_ref; > + AVBufferRef *ph_ref; > + > + const H266RawPPS *pps; > + const H266RawSPS *sps; > + const H266RawPH *ph; > + const H266RawSlice *slice; > + int pic_type; > +} PuInfo; > + > +typedef struct AuDetector { > + uint8_t prev_layer_id; > + int prev_tid0_poc; > + int prev_poc; > +} AuDetector; > + > +typedef struct H266ParserContext { > + ParseContext pc; > + CodedBitstreamContext *cbc; > + > + CodedBitstreamFragment picture_unit; > + > + PuInfo au_info; > + AVPacket au; > + AVPacket last_au; > + > + AuDetector au_detector; > + > + int parsed_extradata; > +} H266ParserContext; > + > +static const enum AVPixelFormat pix_fmts_8bit[] = { > + AV_PIX_FMT_GRAY8, AV_PIX_FMT_YUV420P, > + AV_PIX_FMT_YUV422P, AV_PIX_FMT_YUV444P > +}; > + > +static const enum AVPixelFormat pix_fmts_10bit[] = { > + AV_PIX_FMT_GRAY10, AV_PIX_FMT_YUV420P10, > + AV_PIX_FMT_YUV422P10, AV_PIX_FMT_YUV444P10 > +}; > + > +static int get_format(const H266RawSPS *sps) > +{ > + switch (sps->sps_bitdepth_minus8) { > + case 0: > + return pix_fmts_8bit[sps->sps_chroma_format_idc]; > + case 2: > + return pix_fmts_10bit[sps->sps_chroma_format_idc]; > + } > + return AV_PIX_FMT_NONE; > +} > + > +/** > + * Find the end of the current frame in the bitstream. > + * @return the position of the first byte of the next frame, or END_NOT_FOUND > + */ > +static int find_frame_end(AVCodecParserContext *s, const uint8_t *buf, > + int buf_size) > +{ > + H266ParserContext *ctx = s->priv_data; > + ParseContext *pc = &ctx->pc; > + int i; > + > + for (i = 0; i < buf_size; i++) { > + int nut, code_len; > + > + pc->state64 = (pc->state64 << 8) | buf[i]; > + > + if (((pc->state64 >> 3 * 8) & 0xFFFFFF) != START_CODE) > + continue; Reading one byte at a time is the slowest way to search for a startcode. We have special functions for that. > + > + code_len = ((pc->state64 >> 3 * 8) & 0xFFFFFFFF) == 0x01 ? 4 : 3; > + > + nut = (pc->state64 >> (8 + 3)) & 0x1F; > + // 7.4.2.4.3 and 7.4.2.4.4 > + if ((nut >= VVC_OPI_NUT && nut <= VVC_PREFIX_APS_NUT && > + nut != VVC_PH_NUT) || nut == VVC_AUD_NUT > + || (nut == VVC_PREFIX_SEI_NUT && !pc->frame_start_found) > + || nut == VVC_RSV_NVCL_26 || nut == VVC_UNSPEC_28 > + || nut == VVC_UNSPEC_29) { > + if (pc->frame_start_found) { > + pc->frame_start_found = 0; > + return i - (code_len + 2); > + } > + } else if (nut == VVC_PH_NUT || IS_H266_SLICE(nut)) { > + int sh_picture_header_in_slice_header_flag = buf[i] >> 7; > + > + if (nut == VVC_PH_NUT || sh_picture_header_in_slice_header_flag) { > + if (!pc->frame_start_found) { > + pc->frame_start_found = 1; > + } else { // First slice of next frame found > + pc->frame_start_found = 0; > + return i - (code_len + 2); > + } > + } > + } > + } > + return END_NOT_FOUND; > +} > + > +static int get_pict_type(const CodedBitstreamFragment *pu) > +{ > + int has_p = 0; > + for (int i = 0; i < pu->nb_units; i++) { > + CodedBitstreamUnit *unit = &pu->units[i]; > + if (IS_H266_SLICE(unit->type)) { > + const H266RawSlice *slice = unit->content; > + uint8_t type = slice->header.sh_slice_type; > + if (type == VVC_SLICE_TYPE_B) { > + return AV_PICTURE_TYPE_B; > + } > + if (type == VVC_SLICE_TYPE_P) { > + has_p = 1; > + } > + } > + } > + return has_p ? AV_PICTURE_TYPE_P : AV_PICTURE_TYPE_I; > +} > + > +static void pu_info_unref(PuInfo *info) > +{ > + av_buffer_unref(&info->slice_ref); > + av_buffer_unref(&info->ph_ref); > + av_buffer_unref(&info->pps_ref); > + av_buffer_unref(&info->sps_ref); > + info->slice = NULL; > + info->ph = NULL; > + info->pps = NULL; > + info->sps = NULL; > + info->pic_type = AV_PICTURE_TYPE_NONE; > +} > + > +static int pu_info_ref(PuInfo *dest, const PuInfo *src) > +{ > + pu_info_unref(dest); > + dest->sps_ref = av_buffer_ref(src->sps_ref); > + dest->pps_ref = av_buffer_ref(src->pps_ref); > + if (src->ph_ref) > + dest->ph_ref = av_buffer_ref(src->ph_ref); > + dest->slice_ref = av_buffer_ref(src->slice_ref); > + if (!dest->sps_ref || !dest->pps_ref || (src->ph_ref && !dest->ph_ref) > + || !dest->slice_ref) { > + pu_info_unref(dest); > + return AVERROR(ENOMEM); > + } > + > + dest->sps = src->sps; > + dest->pps = src->pps; > + dest->ph = src->ph; > + dest->slice = src->slice; > + dest->pic_type = src->pic_type; > + return 0; > +} > + > +static int set_parser_ctx(AVCodecParserContext *s, AVCodecContext *avctx, > + const PuInfo *pu) > +{ > + int ret, num = 0, den = 0; > + static const uint8_t h266_sub_width_c[] = { > + 1, 2, 2, 1 > + }; > + static const uint8_t h266_sub_height_c[] = { > + 1, 2, 1, 1 > + }; > + const H266RawSPS *sps = pu->sps; > + const H266RawPPS *pps = pu->pps; > + //const H266RawPH *ph = pu->ph; > + const H266RawNALUnitHeader *nal = &pu->slice->header.nal_unit_header; > + > + /* set some sane default values */ > + s->pict_type = AV_PICTURE_TYPE_I; > + s->key_frame = 0; > + s->picture_structure = AV_PICTURE_STRUCTURE_FRAME; > + > + s->key_frame = nal->nal_unit_type == VVC_IDR_W_RADL || > + nal->nal_unit_type == VVC_IDR_N_LP || > + nal->nal_unit_type == VVC_CRA_NUT || > + nal->nal_unit_type == VVC_GDR_NUT; > + > + s->coded_width = pps->pps_pic_width_in_luma_samples; > + s->coded_height = pps->pps_pic_height_in_luma_samples; > + s->width = pps->pps_pic_width_in_luma_samples - > + (pps->pps_conf_win_left_offset + pps->pps_conf_win_right_offset) * > + h266_sub_width_c[sps->sps_chroma_format_idc]; > + s->height = pps->pps_pic_height_in_luma_samples - > + (pps->pps_conf_win_top_offset + pps->pps_conf_win_bottom_offset) * > + h266_sub_height_c[sps->sps_chroma_format_idc];; > + s->pict_type = pu->pic_type; > + s->format = get_format(sps); > + > + avctx->profile = sps->profile_tier_level.general_profile_idc; > + avctx->level = sps->profile_tier_level.general_level_idc; > + > + avctx->colorspace = (enum AVColorSpace) sps->vui.vui_matrix_coeffs; > + avctx->color_primaries = (enum AVColorPrimaries) sps->vui.vui_colour_primaries; > + avctx->color_trc = (enum AVColorTransferCharacteristic) sps->vui.vui_transfer_characteristics; > + avctx->color_range = > + sps->vui.vui_full_range_flag ? AVCOL_RANGE_JPEG : AVCOL_RANGE_MPEG; > + > + if (s->width != avctx->width || s->height != avctx->height) { > + ret = ff_set_dimensions(avctx, s->width, s->height); > + if (ret < 0) > + return ret; > + } > + avctx->pix_fmt = s->format; > + avctx->has_b_frames = (sps->sps_max_sublayers_minus1 + 1) > 2 ? 2 : > + sps->sps_max_sublayers_minus1; > + avctx->max_b_frames = sps->sps_max_sublayers_minus1; > + > + if (sps->sps_ptl_dpb_hrd_params_present_flag && > + sps->sps_timing_hrd_params_present_flag) { > + num = sps->sps_general_timing_hrd_parameters.num_units_in_tick; > + den = sps->sps_general_timing_hrd_parameters.time_scale; > + } else { > + return 1; > + } > + if (num != 0 && den != 0) > + av_reduce(&avctx->framerate.den, &avctx->framerate.num, > + num, den, 1 << 30); > + > + if (avctx->framerate.num) > + avctx->time_base = av_inv_q(av_mul_q(avctx->framerate, (AVRational) { > + avctx->ticks_per_frame, 1} )); > + > + return 1; > +} > + > +static int set_ctx(AVCodecParserContext *s, AVCodecContext *avctx, > + const PuInfo *next_pu) > +{ > + H266ParserContext *ctx = s->priv_data; > + int ret; > + if (ctx->au_info.slice) { > + if ((ret = set_parser_ctx(s, avctx, &ctx->au_info)) < 0) > + return ret; > + } > + ret = pu_info_ref(&ctx->au_info, next_pu); > + return ret; > +} > + > +//8.3.1 Decoding process for picture order count. > +//VTM did not follow the spec, and it's much simpler than spec. > +//We follow the VTM. > +static void get_slice_poc(H266ParserContext *s, int *poc, > + const H266RawSPS *sps, > + const H266RawPH *ph, > + const H266RawSliceHeader *slice, void *log_ctx) > +{ > + int poc_msb, max_poc_lsb, poc_lsb; > + AuDetector *d = &s->au_detector; > + max_poc_lsb = 1 << (sps->sps_log2_max_pic_order_cnt_lsb_minus4 + 4); > + poc_lsb = ph->ph_pic_order_cnt_lsb; > + if (IS_IDR(slice->nal_unit_header.nal_unit_type)) { > + if (ph->ph_poc_msb_cycle_present_flag) > + poc_msb = ph->ph_poc_msb_cycle_val * max_poc_lsb; > + else > + poc_msb = 0; > + } else { > + int prev_poc = d->prev_tid0_poc; > + int prev_poc_lsb = prev_poc & (max_poc_lsb - 1); > + int prev_poc_msb = prev_poc - prev_poc_lsb; > + if (ph->ph_poc_msb_cycle_present_flag) { > + poc_msb = ph->ph_poc_msb_cycle_val * max_poc_lsb; > + } else { > + if ((poc_lsb < prev_poc_lsb) && ((prev_poc_lsb - poc_lsb) >= > + (max_poc_lsb / 2))) > + poc_msb = prev_poc_msb + max_poc_lsb; > + else if ((poc_lsb > prev_poc_lsb) && ((poc_lsb - prev_poc_lsb) > > + (max_poc_lsb / 2))) > + poc_msb = prev_poc_msb - max_poc_lsb; > + else > + poc_msb = prev_poc_msb; > + } > + } > + > + *poc = poc_msb + poc_lsb; > +} > + > +static void au_detector_init(AuDetector *d) > +{ > + d->prev_layer_id = UINT8_MAX; > + d->prev_poc = INT_MAX; > + d->prev_tid0_poc = INT_MAX; > +} > + > +static int is_au_start(H266ParserContext *s, const PuInfo *pu, void *log_ctx) > +{ > + //7.4.2.4.3 > + AuDetector *d = &s->au_detector; > + const H266RawSPS *sps = pu->sps; > + const H266RawNALUnitHeader *nal = &pu->slice->header.nal_unit_header; > + const H266RawPH *ph = pu->ph; > + const H266RawSlice *slice = pu->slice; > + int ret, poc, nut; > + > + get_slice_poc(s, &poc, sps, ph, &slice->header, log_ctx); > + > + ret = (nal->nuh_layer_id <= d->prev_layer_id) || (poc != d->prev_poc); > + > + nut = nal->nal_unit_type; > + d->prev_layer_id = nal->nuh_layer_id; > + d->prev_poc = poc; > + if (nal->nuh_temporal_id_plus1 == 1 && > + !ph->ph_non_ref_pic_flag && nut != VVC_RADL_NUT > + && nut != VVC_RASL_NUT) { > + d->prev_tid0_poc = poc; > + } > + return ret; > +} > + > +static int get_pu_info(PuInfo *info, const CodedBitstreamH266Context *h266, > + const CodedBitstreamFragment *pu, void *logctx) > +{ > + const H266RawNALUnitHeader *nal; > + int ret; > + > + memset(info, 0, sizeof(*info)); > + for (int i = 0; i < pu->nb_units; i++) { > + nal = pu->units[i].content; > + if (!nal) > + continue; > + if ( nal->nal_unit_type == VVC_PH_NUT ) { > + info->ph = pu->units[i].content; > + info->ph_ref = pu->units[i].content_ref; > + } else if (IS_H266_SLICE(nal->nal_unit_type)) { > + info->slice = pu->units[i].content; > + info->slice_ref = pu->units[i].content_ref; > + if (info->slice->header.sh_picture_header_in_slice_header_flag) > + info->ph = &info->slice->header.sh_picture_header; > + if (!info->ph) { > + av_log(logctx, AV_LOG_ERROR, > + "can't find picture header in picture unit.\n"); > + ret = AVERROR_INVALIDDATA; > + goto error; > + } > + break; > + } > + } > + if (!info->slice) { > + av_log(logctx, AV_LOG_ERROR, "can't find slice in picture unit.\n"); > + ret = AVERROR_INVALIDDATA; > + goto error; > + } > + info->pps = h266->pps[info->ph->ph_pic_parameter_set_id]; > + if (!info->pps) { > + av_log(logctx, AV_LOG_ERROR, "PPS id %d is not avaliable.\n", > + info->ph->ph_pic_parameter_set_id); > + ret = AVERROR_INVALIDDATA; > + goto error; > + } > + info->pps_ref = h266->pps_ref[info->ph->ph_pic_parameter_set_id]; Copying AVBufferRef* is not nice, although I see that it is correct here. > + info->sps = h266->sps[info->pps->pps_seq_parameter_set_id]; > + if (!info->sps) { > + av_log(logctx, AV_LOG_ERROR, "SPS id %d is not avaliable.\n", > + info->pps->pps_seq_parameter_set_id); > + ret = AVERROR_INVALIDDATA; > + goto error; > + } > + info->sps_ref = h266->sps_ref[info->pps->pps_seq_parameter_set_id]; > + info->pic_type = get_pict_type(pu); > + return 0; > + error: > + memset(info, 0, sizeof(*info)); > + return ret; > +} > + > +static int append_au(AVPacket *pkt, const uint8_t *buf, int buf_size) > +{ > + int offset = pkt->size; > + int ret; > + if ((ret = av_grow_packet(pkt, buf_size)) < 0) > + goto end; > + memcpy(pkt->data + offset, buf, buf_size); > + end: > + return ret; > +} > + > +/** > + * Parse NAL units of found picture and decode some basic information. > + * > + * @param s parser context. > + * @param avctx codec context. > + * @param buf buffer with field/frame data. > + * @param buf_size size of the buffer. > + * @return < 0 for error, == 0 for a complete au, > 0 is not a completed au. > + */ > +static int parse_nal_units(AVCodecParserContext *s, const uint8_t *buf, > + int buf_size, AVCodecContext *avctx) > +{ > + H266ParserContext *ctx = s->priv_data; > + const CodedBitstreamH266Context *h266 = ctx->cbc->priv_data; > + > + CodedBitstreamFragment *pu = &ctx->picture_unit; > + int ret; > + PuInfo info; > + > + if (!buf_size) { > + if (ctx->au.size) { > + if ((ret = av_packet_ref(&ctx->last_au, &ctx->au)) < 0) > + goto end; > + av_packet_unref(&ctx->au); > + return 0; > + } > + return 1; > + } > + > + if ((ret = ff_cbs_read(ctx->cbc, pu, buf, buf_size)) < 0) { > + av_log(avctx, AV_LOG_ERROR, "Failed to parse picture unit.\n"); > + goto end; > + } > + if ((ret = get_pu_info(&info, h266, pu, avctx)) < 0) > + goto end; > + if (append_au(&ctx->au, buf, buf_size) < 0){ Why is this done unconditionally, even with PARSER_FLAG_COMPLETE_FRAMES? > + ret = AVERROR(ENOMEM); > + goto end; > + } > + if (is_au_start(ctx, &info, avctx)) { Why are you checking again? Does this mean that find_frame_end() is not correct? > + if ((ret = set_ctx(s, avctx, &info)) < 0) > + goto end; > + if ((ret = av_packet_ref(&ctx->last_au, &ctx->au)) < 0) > + goto end; > + av_packet_unref(&ctx->au); > + } else { > + ret = 1; //not a completed au > + } > + end: > + ff_cbs_fragment_reset(pu); > + return ret; > +} > + > +/** > + * Combine PU to AU > + * > + * @param s parser context. > + * @param avctx codec context. > + * @param buf buffer to a PU. > + * @param buf_size size of the buffer. > + * @return < 0 for error, == 0 a complete au, > 0 not a completed au. > + */ > +static int combine_au(AVCodecParserContext *s, AVCodecContext *avctx, > + const uint8_t **buf, int *buf_size) > +{ > + H266ParserContext *ctx = s->priv_data; > + int ret; > + > + ctx->cbc->log_ctx = avctx; > + > + av_packet_unref(&ctx->last_au); > + ret = parse_nal_units(s, *buf, *buf_size, avctx); > + if (ret == 0) { > + if (ctx->last_au.size) { > + *buf = ctx->last_au.data; > + *buf_size = ctx->last_au.size; > + } else { > + ret = 1; //no output > + } > + } > + ctx->cbc->log_ctx = NULL; > + return ret; > +} > + > +static int h266_parser_parse(AVCodecParserContext *s, AVCodecContext *avctx, > + const uint8_t **poutbuf, int *poutbuf_size, > + const uint8_t *buf, int buf_size) > +{ > + int next, ret; > + H266ParserContext *ctx = s->priv_data; > + ParseContext *pc = &ctx->pc; > + CodedBitstreamFragment *pu = &ctx->picture_unit; > + > + int is_dummy_buf = !buf_size; > + int flush = !buf_size; > + const uint8_t *dummy_buf = buf; Your dummy_buf might not be a dummy_buf at all; better call it "orig_buf" or so. > + > + if (avctx->extradata_size && !ctx->parsed_extradata) { > + ctx->parsed_extradata = 1; > + > + ret = ff_cbs_read_extradata_from_codec(ctx->cbc, pu, avctx); > + if (ret < 0) > + av_log(avctx, AV_LOG_WARNING, "Failed to parse extradata.\n"); > + > + ff_cbs_fragment_reset(pu); > + } > + > + if (s->flags & PARSER_FLAG_COMPLETE_FRAMES) { > + next = buf_size; > + } else { > + next = find_frame_end(s, buf, buf_size); > + if (ff_combine_frame(pc, next, &buf, &buf_size) < 0) > + goto no_out; > + } > + > + is_dummy_buf &= (dummy_buf == buf); > + > + if (!is_dummy_buf) { I don't really understand this. The above check is only relevant if we are flushing; but if we are flushing, then we should always analyze everything we have. It seems to me that the above check is actually meant to be equivalent to "if (buf_size)". (If PARSER_FLAG_COMPLETE_FRAMES is set, then we have no leftover stuff from last time and is_dummy_buf is 1 for flushing; if PARSER_FLAG_COMPLETE_FRAMES is not true, then find_frame_end() won't find an end in our 0-sized buffer, but ff_combine_frame() will (as a special case when parsing) return a value >= 0 and the line above will make is_dummy_buf 0, so we analyze the leftover.) > + ret = combine_au(s, avctx, &buf, &buf_size); > + if (ret > 0 && flush) { > + buf_size = 0; > + ret = combine_au(s, avctx, &buf, &buf_size); > + } > + if (ret != 0) { > + buf_size = next; > + goto no_out; > + } > + } > + > + *poutbuf = buf; > + *poutbuf_size = buf_size; > + return next; > + no_out: > + *poutbuf = NULL; > + *poutbuf_size = 0; > + return buf_size; > +} > + > +static const CodedBitstreamUnitType decompose_unit_types[] = { > + VVC_TRAIL_NUT, > + VVC_STSA_NUT, > + VVC_RADL_NUT, > + VVC_RASL_NUT, > + VVC_IDR_W_RADL, > + VVC_IDR_N_LP, > + VVC_CRA_NUT, > + VVC_GDR_NUT, > + VVC_VPS_NUT, > + VVC_SPS_NUT, > + VVC_PPS_NUT, > + VVC_PH_NUT, > + VVC_AUD_NUT, > +}; > + > +static av_cold int h266_parser_init(AVCodecParserContext *s) > +{ > + H266ParserContext *ctx = s->priv_data; > + int ret; > + > + ret = ff_cbs_init(&ctx->cbc, AV_CODEC_ID_VVC, NULL); > + if (ret < 0) > + return ret; > + au_detector_init(&ctx->au_detector); > + > + ctx->cbc->decompose_unit_types = decompose_unit_types; > + ctx->cbc->nb_decompose_unit_types = FF_ARRAY_ELEMS(decompose_unit_types); > + > + return ret; > +} > + > +static void h266_parser_close(AVCodecParserContext *s) > +{ > + H266ParserContext *ctx = s->priv_data; > + > + pu_info_unref(&ctx->au_info); > + av_packet_unref(&ctx->au); > + av_packet_unref(&ctx->last_au); > + ff_cbs_fragment_free(&ctx->picture_unit); > + > + ff_cbs_close(&ctx->cbc); > + av_freep(&ctx->pc.buffer); > +} > + > +AVCodecParser ff_h266_parser = { > + .codec_ids = { AV_CODEC_ID_VVC }, > + .priv_data_size = sizeof(H266ParserContext), > + .parser_init = h266_parser_init, > + .parser_close = h266_parser_close, > + .parser_parse = h266_parser_parse, > +}; > diff --git a/libavcodec/parsers.c b/libavcodec/parsers.c > index d355808018..aa2043ee1b 100644 > --- a/libavcodec/parsers.c > +++ b/libavcodec/parsers.c > @@ -50,6 +50,7 @@ extern const AVCodecParser ff_gsm_parser; > extern const AVCodecParser ff_h261_parser; > extern const AVCodecParser ff_h263_parser; > extern const AVCodecParser ff_h264_parser; > +extern const AVCodecParser ff_h266_parser; > extern const AVCodecParser ff_hevc_parser; > extern const AVCodecParser ff_hdr_parser; > extern const AVCodecParser ff_ipu_parser; _______________________________________________ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".