From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org [79.124.17.100]) by master.gitmailbox.com (Postfix) with ESMTP id F284A4B84C for ; Wed, 17 Jul 2024 10:02:20 +0000 (UTC) Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 998EA68D9F0; Wed, 17 Jul 2024 13:02:17 +0300 (EEST) Received: from btbn.de (btbn.de [144.76.60.213]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id DAC7768D98A for ; Wed, 17 Jul 2024 13:02:10 +0300 (EEST) Received: from [authenticated] by btbn.de (Postfix) with ESMTPSA id 5A3E328A8E302 for ; Wed, 17 Jul 2024 12:02:07 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=rothenpieler.org; s=mail; t=1721210527; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=zWzcd2jULszUwoGBwtlt+Tma5UoR+OVC9Ff3qURu6tU=; b=hWmoRxmjRfEMpb1Gf4Iz6cSjgAKHB+N6xEKhsiC1gBks+HhftLaIvWTqyt1FIe8f4hl+Mi 7g4Yltr4FZRdt71kWDvHCk0A/j1Yc1osW4Q2M9W5B98hCO8KWt6nvKQ0RVTAsbDZXwTLdu MlppvY6wYsFGiRQ1soTpvTuD+fQrHTyUTPAZMYdUbnD8bT5I3XW0ciP+4vW7rpfbCtW6SW +OMZuyCtGPKCLcPzedmFb0n06TtOrEP/o4UwvgvduEQRVFtDGmYUTbl3nDaHhtbuQia6+4 UucW77qUwSfr0PKe3Iy0yfx8qrwOkbWSwOPI+/hkJPlePfc6Yrejah+CKNJw9w== Message-ID: <8a1f7d6b-b5ad-47aa-bcbd-ce067407036c@rothenpieler.org> Date: Wed, 17 Jul 2024 12:03:03 +0200 MIME-Version: 1.0 User-Agent: Mozilla Thunderbird To: ffmpeg-devel@ffmpeg.org References: Content-Language: en-US, de-DE From: Timo Rothenpieler In-Reply-To: Subject: Re: [FFmpeg-devel] [PATCH] avcodec/nvenc: Rework on DTS generation X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Content-Transfer-Encoding: 7bit Content-Type: text/plain; charset="us-ascii"; Format="flowed" Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" Archived-At: List-Archive: List-Post: On 17/07/2024 11:47, Zhao Zhili wrote: > From: Zhao Zhili > > Before the patch, the method to generate DTS only works with > timebase equal to 1/fps. With timebase like 1/1000 > > ./ffmpeg -i foo.mp4 -an -c:v h264_nvenc -enc_time_base 1/1000 bar.mp4 > > pts 0 dts -3 > pts 160 dts 37 > pts 80 dts 77 > pts 40 dts 117 <-- invalid > pts 120 dts 157 > pts 320 dts 197 > pts 240 dts 237 > pts 200 dts 277 <-- invalid > pts 280 dts 317 <-- invalid > > The generated DTS can be larger than PTS, since it only reorder the > input PTS and minus the number of frame delay, which doesn't take > timebase into account. It should minus the "time" of frame delay. > > 9a245bd trying to fix the issue, but the implementation is incomplete, > which only use time_base.num. Then it got reverted by ac7c265b33b. > > After this patch: > > pts 0 dts -120 > pts 160 dts -80 > pts 80 dts -40 > pts 40 dts 0 > pts 120 dts 40 > pts 320 dts 80 > pts 240 dts 120 > pts 200 dts 160 > pts 280 dts 200 > --- > libavcodec/nvenc.c | 62 ++++++++++++++++++++++++++++++++++++++-------- > libavcodec/nvenc.h | 3 +++ > 2 files changed, 55 insertions(+), 10 deletions(-) > > diff --git a/libavcodec/nvenc.c b/libavcodec/nvenc.c > index ab92395ed6..34448462f0 100644 > --- a/libavcodec/nvenc.c > +++ b/libavcodec/nvenc.c > @@ -1893,7 +1893,8 @@ static av_cold int nvenc_setup_surfaces(AVCodecContext *avctx) > if (!ctx->frame_data_array) > return AVERROR(ENOMEM); > > - ctx->timestamp_list = av_fifo_alloc2(ctx->nb_surfaces, sizeof(int64_t), 0); > + ctx->timestamp_list = av_fifo_alloc2(ctx->nb_surfaces + ctx->encode_config.frameIntervalP, > + sizeof(int64_t), 0); > if (!ctx->timestamp_list) > return AVERROR(ENOMEM); > > @@ -2347,26 +2348,65 @@ static inline int64_t timestamp_queue_dequeue(AVFifo *queue) > return timestamp; > } > > +static inline int64_t timestamp_queue_peek(AVFifo *queue, size_t index) > +{ > + int64_t timestamp = AV_NOPTS_VALUE; > + av_fifo_peek(queue, ×tamp, 1, index); > + > + return timestamp; > +} > + > static int nvenc_set_timestamp(AVCodecContext *avctx, > NV_ENC_LOCK_BITSTREAM *params, > AVPacket *pkt) > { > NvencContext *ctx = avctx->priv_data; > + int delay; > + int64_t delay_time; > > pkt->pts = params->outputTimeStamp; > > - if (avctx->codec_descriptor->props & AV_CODEC_PROP_REORDER) { > -FF_DISABLE_DEPRECATION_WARNINGS > - pkt->dts = timestamp_queue_dequeue(ctx->timestamp_list) - > -#if FF_API_TICKS_PER_FRAME > - FFMAX(avctx->ticks_per_frame, 1) * > -#endif > - FFMAX(ctx->encode_config.frameIntervalP - 1, 0); > -FF_ENABLE_DEPRECATION_WARNINGS > - } else { > + if (!(avctx->codec_descriptor->props & AV_CODEC_PROP_REORDER)) { > pkt->dts = pkt->pts; > + return 0; > + } > + > + // This can be more than necessary, but we don't know the real reorder delay. > + delay = FFMAX(ctx->encode_config.frameIntervalP - 1, 0); > + if (ctx->output_frame_num >= delay) { > + pkt->dts = timestamp_queue_dequeue(ctx->timestamp_list); > + ctx->output_frame_num++; > + return 0; > } > > + delay_time = ctx->initial_delay_time; > + if (!delay_time) { > + int64_t t1, t2, t3; > + t1 = timestamp_queue_peek(ctx->timestamp_list, delay); > + t2 = timestamp_queue_peek(ctx->timestamp_list, 0); > + t3 = (delay > 1) ? timestamp_queue_peek(ctx->timestamp_list, 1) : t1; > + > + if (t1 != AV_NOPTS_VALUE) { > + delay_time = t1 - t2; > + } else if (avctx->framerate.num > 0 && avctx->framerate.den > 0) { > + delay_time = av_rescale_q(delay, (AVRational) {avctx->framerate.den, avctx->framerate.num}, > + avctx->time_base); > + } else if (t3 != AV_NOPTS_VALUE) { > + delay_time = delay * (t3 - t2); > + } else { > + delay_time = delay; > + } > + ctx->initial_delay_time = delay_time; > + } > + > + /* The following method is simple, but doesn't guarantee monotonic with VFR > + * when delay_time isn't accurate (that is, t1 == AV_NOPTS_VALUE) > + * > + * dts = timestamp_queue_peek(ctx->timestamp_list, ctx->output_frame_num) - delay_time > + */ > + pkt->dts = timestamp_queue_peek(ctx->timestamp_list, 0) - delay_time * (delay - ctx->output_frame_num) / delay; > + ctx->output_frame_num++; > + > return 0; > } > > @@ -2902,4 +2942,6 @@ av_cold void ff_nvenc_encode_flush(AVCodecContext *avctx) > > nvenc_send_frame(avctx, NULL); > av_fifo_reset2(ctx->timestamp_list); > + ctx->output_frame_num = 0; > + ctx->initial_delay_time = 0; > } > diff --git a/libavcodec/nvenc.h b/libavcodec/nvenc.h > index 09de00badc..dc7fe41951 100644 > --- a/libavcodec/nvenc.h > +++ b/libavcodec/nvenc.h > @@ -206,6 +206,9 @@ typedef struct NvencContext > AVFifo *output_surface_queue; > AVFifo *output_surface_ready_queue; > AVFifo *timestamp_list; > + // This is for DTS calculating, reset after flush > + int64_t output_frame_num; > + int64_t initial_delay_time; > > NV_ENC_SEI_PAYLOAD *sei_data; > int sei_data_size; This looks good on first glance, but I'm away from any nvenc capable PC for the next two weeks, so it'll unfortunately be a while until I can properly test and apply. Timo _______________________________________________ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".