From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org [79.124.17.100]) by master.gitmailbox.com (Postfix) with ESMTP id 0E84144EF1 for ; Wed, 29 Mar 2023 14:57:31 +0000 (UTC) Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 283E568C1A9; Wed, 29 Mar 2023 17:57:29 +0300 (EEST) Received: from mail-oi1-f174.google.com (mail-oi1-f174.google.com [209.85.167.174]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id B21F568C139 for ; Wed, 29 Mar 2023 17:57:22 +0300 (EEST) Received: by mail-oi1-f174.google.com with SMTP id bj20so11777178oib.3 for ; Wed, 29 Mar 2023 07:57:22 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; t=1680101840; h=content-transfer-encoding:in-reply-to:from:references:to :content-language:subject:user-agent:mime-version:date:message-id :from:to:cc:subject:date:message-id:reply-to; bh=FY0eJ1BFWfn5dWzFV8uIgviLmGwe9FxTWR4AEboDXak=; b=hbAM5POuLsR94spCexwALaYZdNW5CYVq8aO5PYofJKcWGCKDlT8scEHt6gdMMldWD1 FyLjfuRXZzB7uwOLvPaGE9ZyexR7Jum0e0EGUNFf3iGYiHTLYHfVRmZZgVe7y9N2CjuF SIN8CEeLRiFQXPSiCqFLK2MeahU8UrAh7QeXsAPWaHoo5ulz5utt2zDu2F+iYGZnfd2w mjhwGobXjvhR1xyZtVTKJcd8owaci8Lq5c0z/iCZuWU+f7fpZ8GxibX/HNsRu3BA5uFz HArSI833f2X+PGzAQh7kS7883c9iJMoei4NY10W/3pM0uDfSWm4rP6jNlXp/6qRxb7Uo odTA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; t=1680101840; h=content-transfer-encoding:in-reply-to:from:references:to :content-language:subject:user-agent:mime-version:date:message-id :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=FY0eJ1BFWfn5dWzFV8uIgviLmGwe9FxTWR4AEboDXak=; b=sAYSn3H2Fwng9wHYbbkZRtVwWGcrx5B1Bzs1uT7aIAJ48UswpJaK9QMITv5weiLBQw ATMPzkz2B8JMxdxu87y5GWmvXcgIvzfyaQy2534Aiy+Zdap9dwuv7g1DigfsCYS1tI3m MxyxFRXnBAN4SvTBsl0omRqN1++6R62MlEOYNslgm/uMwdIgdMsovzuwQ+sd/HfxaxbA +2MyNMOAcmwMd95hoFifhfxoX0KO8cN22W30cefaVAjMg6mh2D5mqhTLXGs9j99mjfke qD3G+OUHMnpl7289j5/2CUnuuX9notcvkEhkeKaat727CAAyqjpm3S35KnrVB/Mzlmde tBgg== X-Gm-Message-State: AAQBX9cxn6N7HwB3FnV0YNR0v51A63qwxYsqf6WFL289EVXqjP8i2QHi n25TPW0HXuUa+nE8PmQF0iavOVzv4OA= X-Google-Smtp-Source: AKy350b8FYZLA+hGL+iONbVq2mair/HOEI3w0OvRUihwlqkoWtsurRsh9+9s1lDu5vkO2COgDO62tQ== X-Received: by 2002:a54:4898:0:b0:389:4cf7:47ee with SMTP id r24-20020a544898000000b003894cf747eemr3626475oic.47.1680101840298; Wed, 29 Mar 2023 07:57:20 -0700 (PDT) Received: from [192.168.0.14] (host197.190-225-105.telecom.net.ar. [190.225.105.197]) by smtp.gmail.com with ESMTPSA id c192-20020a4a4fc9000000b0053b88b03e24sm8036020oob.18.2023.03.29.07.57.19 for (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Wed, 29 Mar 2023 07:57:19 -0700 (PDT) Message-ID: Date: Wed, 29 Mar 2023 11:57:18 -0300 MIME-Version: 1.0 User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:102.0) Gecko/20100101 Thunderbird/102.9.0 Content-Language: en-US To: ffmpeg-devel@ffmpeg.org References: <20230328134636.234-1-d.kozinski@samsung.com> From: James Almer In-Reply-To: <20230328134636.234-1-d.kozinski@samsung.com> Subject: Re: [FFmpeg-devel] [PATCH v18 02/10] avcodec/evc_parser: Added parser implementation for EVC format X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Content-Transfer-Encoding: 7bit Content-Type: text/plain; charset="us-ascii"; Format="flowed" Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" Archived-At: List-Archive: List-Post: On 3/28/2023 10:46 AM, Dawid Kozinski wrote: > - Added constants definitions for EVC parser > - Provided NAL units parsing following ISO_IEC_23094-1 > - EVC parser registration > > Signed-off-by: Dawid Kozinski > --- > libavcodec/Makefile | 1 + > libavcodec/evc.h | 155 +++++ > libavcodec/evc_parser.c | 1270 +++++++++++++++++++++++++++++++++++++++ > libavcodec/parsers.c | 1 + > 4 files changed, 1427 insertions(+) > create mode 100644 libavcodec/evc.h > create mode 100644 libavcodec/evc_parser.c > [...] > +static int parse_nal_unit(AVCodecParserContext *s, const uint8_t *buf, > + int buf_size, AVCodecContext *avctx) > +{ > + EVCParserContext *ev = s->priv_data; > + int nalu_type, nalu_size; > + int tid; > + const uint8_t *data = buf; > + int data_size = buf_size; > + > + s->picture_structure = AV_PICTURE_STRUCTURE_FRAME; > + s->key_frame = -1; > + > + > + nalu_size = buf_size; > + if (nalu_size <= 0) { > + av_log(avctx, AV_LOG_ERROR, "Invalid NAL unit size: (%d)\n", nalu_size); > + return AVERROR_INVALIDDATA; > + } > + > + // @see ISO_IEC_23094-1_2020, 7.4.2.2 NAL unit header semantic (Table 4 - NAL unit type codes and NAL unit type classes) > + // @see enum EVCNALUnitType in evc.h > + nalu_type = get_nalu_type(data, data_size, avctx); > + if (nalu_type < EVC_NOIDR_NUT || nalu_type > EVC_UNSPEC_NUT62) { > + av_log(avctx, AV_LOG_ERROR, "Invalid NAL unit type: (%d)\n", nalu_type); > + return AVERROR_INVALIDDATA; > + } > + ev->nalu_type = nalu_type; > + > + tid = get_temporal_id(data, data_size, avctx); > + if (tid < 0) { > + av_log(avctx, AV_LOG_ERROR, "Invalid temporial id: (%d)\n", tid); > + return AVERROR_INVALIDDATA; > + } > + ev->nuh_temporal_id = tid; > + > + if (data_size < nalu_size) { > + av_log(avctx, AV_LOG_ERROR, "NAL unit does not fit in the data buffer\n"); > + return AVERROR_INVALIDDATA; > + } > + > + data += EVC_NALU_HEADER_SIZE; > + data_size -= EVC_NALU_HEADER_SIZE; > + > + if (nalu_type == EVC_SPS_NUT) { > + EVCParserSPS *sps; > + int SubGopLength; > + > + sps = parse_sps(data, nalu_size, ev); > + if (!sps) { > + av_log(avctx, AV_LOG_ERROR, "SPS parsing error\n"); > + return AVERROR_INVALIDDATA; > + } > + > + s->coded_width = sps->pic_width_in_luma_samples; > + s->coded_height = sps->pic_height_in_luma_samples; > + s->width = sps->pic_width_in_luma_samples - sps->picture_crop_left_offset - sps->picture_crop_right_offset; > + s->height = sps->pic_height_in_luma_samples - sps->picture_crop_top_offset - sps->picture_crop_bottom_offset; > + > + SubGopLength = (int)pow(2.0, sps->log2_sub_gop_length); > + avctx->gop_size = SubGopLength; > + > + avctx->delay = (sps->sps_max_dec_pic_buffering_minus1) ? sps->sps_max_dec_pic_buffering_minus1 - 1 : SubGopLength + sps->max_num_tid0_ref_pics - 1; > + > + if (sps->profile_idc == 1) avctx->profile = FF_PROFILE_EVC_MAIN; > + else avctx->profile = FF_PROFILE_EVC_BASELINE; > + > + ev->time_base = avctx->time_base.den; This looks like a write only field. Also, avctx->time_base is no longer used for decoding. What you can do is setting avctx->frame_rate using vui->num_units_in_tick and vui->time_scale if present. See h264_parser.c > + > + switch (sps->chroma_format_idc) { > + case 0: /* YCBCR400_10LE */ > + av_log(avctx, AV_LOG_ERROR, "YCBCR400_10LE: Not supported chroma format\n"); > + s->format = AV_PIX_FMT_GRAY10LE; > + return -1; > + case 1: /* YCBCR420_10LE */ > + s->format = AV_PIX_FMT_YUV420P10LE; > + break; > + case 2: /* YCBCR422_10LE */ > + av_log(avctx, AV_LOG_ERROR, "YCBCR422_10LE: Not supported chroma format\n"); > + s->format = AV_PIX_FMT_YUV422P10LE; > + return -1; > + case 3: /* YCBCR444_10LE */ > + av_log(avctx, AV_LOG_ERROR, "YCBCR444_10LE: Not supported chroma format\n"); > + s->format = AV_PIX_FMT_YUV444P10LE; > + return -1; > + default: > + s->format = AV_PIX_FMT_NONE; > + av_log(avctx, AV_LOG_ERROR, "Unknown supported chroma format\n"); > + return -1; > + } > + } else if (nalu_type == EVC_PPS_NUT) { > + EVCParserPPS *pps; > + > + pps = parse_pps(data, nalu_size, ev); > + if (!pps) { > + av_log(avctx, AV_LOG_ERROR, "PPS parsing error\n"); > + return AVERROR_INVALIDDATA; > + } > + } else if (nalu_type == EVC_SEI_NUT) // Supplemental Enhancement Information > + return 0; > + else if (nalu_type == EVC_APS_NUT) // Adaptation parameter set > + return 0; > + else if (nalu_type == EVC_FD_NUT) /* Filler data */ > + return 0; Use a switch() statement for this instead. > + else if (nalu_type == EVC_IDR_NUT || nalu_type == EVC_NOIDR_NUT) { // Coded slice of a IDR or non-IDR picture > + EVCParserSliceHeader *sh; > + EVCParserSPS *sps; > + int slice_pic_parameter_set_id; > + > + sh = parse_slice_header(data, nalu_size, ev); > + if (!sh) { > + av_log(avctx, AV_LOG_ERROR, "Slice header parsing error\n"); > + return AVERROR_INVALIDDATA; > + } > + > + switch (sh->slice_type) { > + case EVC_SLICE_TYPE_B: { > + s->pict_type = AV_PICTURE_TYPE_B; > + break; > + } > + case EVC_SLICE_TYPE_P: { > + s->pict_type = AV_PICTURE_TYPE_P; > + break; > + } > + case EVC_SLICE_TYPE_I: { > + s->pict_type = AV_PICTURE_TYPE_I; > + break; > + } > + default: { > + s->pict_type = AV_PICTURE_TYPE_NONE; > + } > + } > + > + s->key_frame = (nalu_type == EVC_IDR_NUT) ? 1 : 0; > + > + // POC (picture order count of the current picture) derivation > + // @see ISO/IEC 23094-1:2020(E) 8.3.1 Decoding process for picture order count > + slice_pic_parameter_set_id = sh->slice_pic_parameter_set_id; > + sps = &ev->sps[slice_pic_parameter_set_id]; > + > + if (sps->sps_pocs_flag) { > + > + int PicOrderCntMsb = 0; > + ev->poc.prevPicOrderCntVal = ev->poc.PicOrderCntVal; > + > + if (nalu_type == EVC_IDR_NUT) > + PicOrderCntMsb = 0; > + else { > + int MaxPicOrderCntLsb = 1 << (sps->log2_max_pic_order_cnt_lsb_minus4 + 4); > + > + int prevPicOrderCntLsb = ev->poc.PicOrderCntVal & (MaxPicOrderCntLsb - 1); > + int prevPicOrderCntMsb = ev->poc.PicOrderCntVal - prevPicOrderCntLsb; > + > + > + if ((sh->slice_pic_order_cnt_lsb < prevPicOrderCntLsb) && > + ((prevPicOrderCntLsb - sh->slice_pic_order_cnt_lsb) >= (MaxPicOrderCntLsb / 2))) > + > + PicOrderCntMsb = prevPicOrderCntMsb + MaxPicOrderCntLsb; > + > + else if ((sh->slice_pic_order_cnt_lsb > prevPicOrderCntLsb) && > + ((sh->slice_pic_order_cnt_lsb - prevPicOrderCntLsb) > (MaxPicOrderCntLsb / 2))) > + > + PicOrderCntMsb = prevPicOrderCntMsb - MaxPicOrderCntLsb; > + > + else > + PicOrderCntMsb = prevPicOrderCntMsb; > + } > + ev->poc.PicOrderCntVal = PicOrderCntMsb + sh->slice_pic_order_cnt_lsb; > + > + } else { > + if (nalu_type == EVC_IDR_NUT) { > + ev->poc.PicOrderCntVal = 0; > + ev->poc.DocOffset = -1; > + } else { > + int SubGopLength = (int)pow(2.0, sps->log2_sub_gop_length); > + if (tid == 0) { > + ev->poc.PicOrderCntVal = ev->poc.prevPicOrderCntVal + SubGopLength; > + ev->poc.DocOffset = 0; > + ev->poc.prevPicOrderCntVal = ev->poc.PicOrderCntVal; > + } else { > + int ExpectedTemporalId; > + int PocOffset; > + int prevDocOffset = ev->poc.DocOffset; > + > + ev->poc.DocOffset = (prevDocOffset + 1) % SubGopLength; > + if (ev->poc.DocOffset == 0) { > + ev->poc.prevPicOrderCntVal += SubGopLength; > + ExpectedTemporalId = 0; > + } else > + ExpectedTemporalId = 1 + (int)log2(ev->poc.DocOffset); > + while (tid != ExpectedTemporalId) { > + ev->poc.DocOffset = (ev->poc.DocOffset + 1) % SubGopLength; > + if (ev->poc.DocOffset == 0) > + ExpectedTemporalId = 0; > + else > + ExpectedTemporalId = 1 + (int)log2(ev->poc.DocOffset); > + } > + PocOffset = (int)(SubGopLength * ((2.0 * ev->poc.DocOffset + 1) / (int)pow(2.0, tid) - 2)); > + ev->poc.PicOrderCntVal = ev->poc.prevPicOrderCntVal + PocOffset; > + } > + } > + } > + > + s->output_picture_number = ev->poc.PicOrderCntVal; > + s->key_frame = (nalu_type == EVC_IDR_NUT) ? 1 : 0; > + > + return 0; > + } > + data += (nalu_size - EVC_NALU_HEADER_SIZE); > + data_size -= (nalu_size - EVC_NALU_HEADER_SIZE); What does this even do? They are local variables, and this is not in a loop. > + > + return 0; > +} [...] > +// Find the end of the current frame in the bitstream. > +// The end of frame is the end of Access Unit. > +// Function returns the position of the first byte of the next frame, or END_NOT_FOUND > +static int evc_find_frame_end(AVCodecParserContext *s, const uint8_t *buf, > + int buf_size, AVCodecContext *avctx) > +{ > + EVCParserContext *ctx = s->priv_data; > + > + const uint8_t *data = buf; > + int data_size = buf_size; > + > + while (data_size > 0) { > + > + if (ctx->to_read == 0) { > + // Nothing must be read and appended to the data from previous chunks. > + // The previous chunk of data provided the complete NALU prefix or provided the complete NALU. > + > + if (ctx->nalu_prefix_assembled) // NALU prefix has been assembled from previous and current chunks of incoming data > + ctx->nalu_prefix_assembled = 0; > + else { // Buffer size is not enough for buffer to store NAL unit 4-bytes prefix (length) > + if (data_size < EVC_NALU_LENGTH_PREFIX_SIZE) { > + ctx->to_read = EVC_NALU_LENGTH_PREFIX_SIZE - data_size; > + ctx->incomplete_nalu_prefix_read = 1; > + return END_NOT_FOUND; > + } > + > + ctx->nalu_size = read_nal_unit_length(data, data_size, avctx); > + ctx->bytes_read += EVC_NALU_LENGTH_PREFIX_SIZE; > + > + data += EVC_NALU_LENGTH_PREFIX_SIZE; > + data_size -= EVC_NALU_LENGTH_PREFIX_SIZE; > + } > + > + if (data_size < ctx->nalu_size) { > + > + ctx->to_read = ctx->nalu_size - data_size; > + ctx->incomplete_nalu_read = 1; > + return END_NOT_FOUND; > + } > + > + // the entire NALU can be read > + if (parse_nal_unit(s, data, ctx->nalu_size, avctx) != 0) { NALU parsing should happen always, not only when the parser is required to assemble a full access unit. > + av_log(avctx, AV_LOG_ERROR, "Parsing of NAL unit failed\n"); > + return AVERROR_INVALIDDATA; > + } > + > + data += ctx->nalu_size; > + data_size -= ctx->nalu_size; > + > + ctx->bytes_read += ctx->nalu_size; > + > + if (end_of_access_unit_found(s, avctx)) { > + > + // parser should return buffer that contains complete AU > + int read_bytes = ctx->bytes_read; > + ctx->bytes_read = 0; > + return read_bytes; > + } > + > + // go to the next iteration > + continue; > + > + } else { > + // The previous chunk of input data did not contain the complete valid NALU prefix or did not contain the complete NALU. > + // > + // Missing data must be read from the current data chunk and merged with the data from the previous data chunk > + // to assemble a complete NALU or complete NALU prefix. > + // > + // The data from the previous data chunk are stored in pc->buf > + > + if (ctx->to_read < data_size) { > + > + if (ctx->incomplete_nalu_prefix_read == 1) { > + > + uint8_t nalu_prefix[EVC_NALU_LENGTH_PREFIX_SIZE]; > + evc_assemble_nalu_prefix(s, data, data_size, nalu_prefix, avctx); > + > + ctx->nalu_size = read_nal_unit_length(nalu_prefix, EVC_NALU_LENGTH_PREFIX_SIZE, avctx); > + > + // update variable storing amout of read bytes for teh current AU > + ctx->bytes_read += ctx->to_read; > + > + // update data pointer and data size > + data += ctx->to_read; > + data_size -= ctx->to_read; > + > + // reset variable storing amount of bytes to read from the new data chunk > + ctx->to_read = 0; > + > + ctx->incomplete_nalu_prefix_read = 0; > + ctx->nalu_prefix_assembled = 1; > + > + continue; > + } > + if (ctx->incomplete_nalu_read == 1) { > + > + uint8_t *nalu = (uint8_t *)av_malloc(ctx->nalu_size); > + > + // assemble NAL unit using data from previous data chunks (pc->buffer) and the current one (data) > + evc_assemble_nalu(s, data, ctx->to_read, nalu, ctx->nalu_size, avctx); > + > + if (parse_nal_unit(s, nalu, ctx->nalu_size, avctx) != 0) { > + av_log(avctx, AV_LOG_ERROR, "Parsing of NAL unit failed\n"); > + return AVERROR_INVALIDDATA; > + } > + av_free(nalu); > + > + // update variable storing amout of read bytes for teh current AU > + ctx->bytes_read += ctx->nalu_size; > + > + // update data pointer and data size > + data += ctx->to_read; > + data_size -= ctx->to_read; > + > + ctx->incomplete_nalu_read = 0; > + > + if (end_of_access_unit_found(s, avctx)) { > + > + // parser should return buffer that contains complete AU > + int read_bytes = ctx->to_read; > + > + ctx->to_read = 0; > + ctx->bytes_read = 0; > + > + return read_bytes; > + } > + > + // reset variable storing amount of bytes to read from the new data chunk > + ctx->to_read = 0; > + > + continue; > + } > + } else { > + // needed more input data to assemble complete valid NAL Unit > + ctx->to_read = ctx->to_read - data_size; > + return END_NOT_FOUND; > + } > + } > + } > + > + return END_NOT_FOUND; > +} > + > +static int evc_parse(AVCodecParserContext *s, AVCodecContext *avctx, > + const uint8_t **poutbuf, int *poutbuf_size, > + const uint8_t *buf, int buf_size) > +{ > + int next; > + EVCParserContext *ev = s->priv_data; > + ParseContext *pc = &ev->pc; > + > + if (s->flags & PARSER_FLAG_COMPLETE_FRAMES) > + next = buf_size; > + else { > + next = evc_find_frame_end(s, buf, buf_size, avctx); > + if (ff_combine_frame(pc, next, &buf, &buf_size) < 0) { > + *poutbuf = NULL; > + *poutbuf_size = 0; > + return buf_size; > + } > + } Case in point, you should call the function to parse the packet's NALUs here, so it's done also when PARSER_FLAG_COMPLETE_FRAMES is signaled. > + > + // poutbuf contains just one Access Unit > + *poutbuf = buf; > + *poutbuf_size = buf_size; > + > + return next; > +} > + > +static int evc_parser_init(AVCodecParserContext *s) > +{ > + EVCParserContext *ev = s->priv_data; > + ev->incomplete_nalu_prefix_read = 0; > + > + return 0; > +} > + > +const AVCodecParser ff_evc_parser = { > + .codec_ids = { AV_CODEC_ID_EVC }, > + .priv_data_size = sizeof(EVCParserContext), > + .parser_init = evc_parser_init, > + .parser_parse = evc_parse, > + .parser_close = ff_parse_close, > +}; > diff --git a/libavcodec/parsers.c b/libavcodec/parsers.c > index d355808018..2c077ec3ae 100644 > --- a/libavcodec/parsers.c > +++ b/libavcodec/parsers.c > @@ -41,6 +41,7 @@ extern const AVCodecParser ff_dvaudio_parser; > extern const AVCodecParser ff_dvbsub_parser; > extern const AVCodecParser ff_dvdsub_parser; > extern const AVCodecParser ff_dvd_nav_parser; > +extern const AVCodecParser ff_evc_parser; > extern const AVCodecParser ff_flac_parser; > extern const AVCodecParser ff_ftr_parser; > extern const AVCodecParser ff_g723_1_parser; _______________________________________________ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".