From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org [79.124.17.100]) by master.gitmailbox.com (Postfix) with ESMTP id 714FA4AA2D for ; Wed, 7 Aug 2024 10:18:35 +0000 (UTC) Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 3D2F668DB12; Wed, 7 Aug 2024 13:18:33 +0300 (EEST) Received: from EUR05-AM6-obe.outbound.protection.outlook.com (mail-am6eur05olkn2041.outbound.protection.outlook.com [40.92.91.41]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id B36B168DA1F for ; Wed, 7 Aug 2024 13:18:26 +0300 (EEST) ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=o0ND5UEzdfwd5wsVry5QSEGkDP5TNbgq+nEXGYCTg2A5JVy3VD+UGIiLpxsOWKEq+98gJhtZ9VNOnrQXs3aHKQeasxMduHN9V+NjGjCgprohs9oLGmsc9cP5+kaYwNFqdTDtl5/tWSRlQ90zPeoZ4jgQoDn556F7pzZF4oEIayPj1KVuDnu4p1QRc10+xvxMrQXHgXvonPjgJl92cO/WvD7LmF6fcwhcE6JHKTNxYyMDT1MaTtjRlrInyS1jybdqMvwBtDxc4ITqkP6LgBAp3gcVoFe7PehEswZqqbwpxu0Mfd9k2l9nr62ufcYD44MmqG0+nmzRB0d0JPhx/13A1w== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=omyzuuUHj/tw9lmRsNAXc5LRPtpZloc8zMSSw3jRwjM=; b=Zo1AQ2BLsKDWq/EQYb71YvCmjc9TSEQW64tO4ZMcoGGsRnVTJiHZLROkrOgUf1IHTPUhHjcwKabDZqTlj/+DHAN67tf7lrmgM8vGfMjxYzvLdfMtFa0JIl7aU3362qdSHVvdCSSqdJ4ibb5DQznf6ZsSywMt+CDP//iPIsBz1c70t//4F9TvAh43fpWz3L7WaF1g1UgQed1wg+ttp4yhCxQnDMvS0wWEoSjNgL3uRHxDFHqDFo8J5FKOyi2QELDk4+efTAAIMJ1ddhOJZYFWSOwKNqC17yYx/w/GDVlvWnj+zvPLE2xt8tUlxZ9DCAX8eHXh9gSx5qPmMe3AWGozzQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=none; dmarc=none; dkim=none; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=outlook.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=omyzuuUHj/tw9lmRsNAXc5LRPtpZloc8zMSSw3jRwjM=; b=VgKXWdyOb3BfSDl80sEYT7cpsxSofNPQZqDQTOlMe2d6/UydkI4I53+Jz++vMzOt0qrMhBdiBarJSPgIIdvSTx6uFnb96BsEm+7Ju0GMVusI3+S2iCqe5srL/XdXBDBGAp6NZn5SUweUsMHzBrIkWn8+jtjSF3EHk5XoMqO1oUHoRYH+jXGaZjIzwSdeV8aeA0GxRgeqUFzq2U2zu1shD0ud5xceXudtPwhgqN42fMyRKa9oT1JSbqvp7u26f/LZGW4m9qgM0QU7Jz4hUJZ+jTwQC2b4o3xxD4xs1OiEP+4v3KgOVtCXsRWXKm4GmYeRInpFLPE3LvNe2PLwMwSTHw== Received: from AS8P250MB0744.EURP250.PROD.OUTLOOK.COM (2603:10a6:20b:541::14) by PR3P250MB0306.EURP250.PROD.OUTLOOK.COM (2603:10a6:102:17d::20) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7849.13; Wed, 7 Aug 2024 10:18:24 +0000 Received: from AS8P250MB0744.EURP250.PROD.OUTLOOK.COM ([fe80::384d:40d4:ecb7:1c9]) by AS8P250MB0744.EURP250.PROD.OUTLOOK.COM ([fe80::384d:40d4:ecb7:1c9%3]) with mapi id 15.20.7828.023; Wed, 7 Aug 2024 10:18:24 +0000 Message-ID: Date: Wed, 7 Aug 2024 12:18:22 +0200 User-Agent: Mozilla Thunderbird To: ffmpeg-devel@ffmpeg.org References: Content-Language: en-US From: Andreas Rheinhardt In-Reply-To: X-TMN: [GIaiW7rOb4kovs3G8OBmcfM3CMC2loikBeQNFhzKlmU=] X-ClientProxiedBy: ZR2P278CA0006.CHEP278.PROD.OUTLOOK.COM (2603:10a6:910:50::12) To AS8P250MB0744.EURP250.PROD.OUTLOOK.COM (2603:10a6:20b:541::14) X-Microsoft-Original-Message-ID: MIME-Version: 1.0 X-MS-Exchange-MessageSentRepresentingType: 1 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: AS8P250MB0744:EE_|PR3P250MB0306:EE_ X-MS-Office365-Filtering-Correlation-Id: 7d25905f-59f8-44a1-5c68-08dcb6ca4549 X-Microsoft-Antispam: BCL:0; ARA:14566002|461199028|19110799003|5072599009|8060799006|440099028|3412199025; X-Microsoft-Antispam-Message-Info: tRdLxvobpsVsqSRPyyTFrkYTp6+o4CwOIVXKFp8d2/4J4muujf/50FDicbZJT0s1vOnoJMJRXZzDLtYXkxGR/0fb57dm+AImRdRkiM7exmWVnBD0cRX6emnxT7sU9upzMXVY4Wec1JP8s5R3BIO7rcqrnDkxVNMG6eboNDxH9L9VL0EEamTPu/Pc8TbMi8ILs0Cv4mhyiUK44nb5lmniqRLWRW+BYHansYaIDTu0J8vY5TLYl/ZqpPnMW8aKIFsnE5U/jFRjgsuh3EW9B6HktObKw8Z5V7kyz8TlIjEAE7/wxwFnZseBlEKOGH900VNszYn/9wpjHnRuUKKYxYN5//0f1qVz1tyLBCaKC8kbIrihyqLL6XrAkX1Zm3XYL0F/LuUeib8hd7Pe36pvpyikH8cZcOkOpQ6OwbZ7hCQQElmqzTMKA37ON0tHFGQaPTck9cXn7tezNuUxO50QteI7l10wQTpV+0B+epzNb3VkrZRTYnIsFP49D/2mu7m4np97N2fG+aaHknvCWWkmNe+snTYjjadGzICIEk3VV5oejqfil3wQVSQJqdUdFH9VXTThP7NALgvHX7BfzI8rU0uSuec+ZfH/H3AyCy0B6rQLfuHjv+Ws3b4QscGq86TG9HjV0YoTUu8Vo9Hsvz+5cocBCIGu5kQT9YsTXnurF5mt5syaJ+8rCOTjkmHSF4eZom5y X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?B?dGVrSWxPTDF2cHF6OHZvUkRnMmJ1cGc4MzFIWDdrdm9pRHBBZUIrdXh2dVFD?= =?utf-8?B?T3ltSEQ1bzZhSlF0ZFM1VG1JZ0xncEVxYi9mM0RCRy9NVUpQZXFuTEx6MVN5?= =?utf-8?B?WmVyTnAyRTZScnZXVE4xZ2E3RFc2WVhVVk5HYi8xSkRTMUNRdis2ZWlXa0dT?= =?utf-8?B?S2Q0L3V4V0pVcjI4QVNNQ1FERmE4aGZidVduZ0d5Y1RQQnRJYVllUUcxbFRS?= =?utf-8?B?c0IzUWR0QnIyQy9Qd2pXY0NiOFlMcStuUG8zMWVnQlFGUWNFQWJTUDN2MFhX?= =?utf-8?B?cms1MURWMy9mQURFMEJ5VlpyaW5aTzNiZXZjVWFqOCtrT0tRNmNodUQ5c0VF?= =?utf-8?B?eFgxNXBhRzdtMXN2TURkNTlKVHZBME1aSHBiUXZrUis5RWhtUDBZVlA1dGtI?= =?utf-8?B?UjhFTWFrbVBSeWdWa3lEOGd4aXpzNDN1Uk1zQmdIb2g0bWllVjNJRmVHSEhT?= =?utf-8?B?RG5HNGNvakdtVXVqTmRwWFgydWhxU0xKNml6Q0JuK1hobjRkU1BaZ0IxSHdJ?= =?utf-8?B?SXpBcDJTcGFwNUZkTzA2SnZMSmxOQXdqQ1pHaHlNQ1B4NllVK2s4N1p0UENX?= =?utf-8?B?NzBZalV6SjVhM3lTOU1OZm9BU0tiWGFGTjBiWi8yR1dvZGQ1SmpLWFpYTGo1?= =?utf-8?B?TWNxTVRJbUJCd0dhZ3lldUZCeTBvZUZSbDVYbTZ3cXo3YTRpT3VEalVreUtM?= =?utf-8?B?Mm15dzBEKzJuWG14VjRtQ2dRU0hHenIxcjFXbkh1N0c4SVBxYXA0bExLaGoy?= =?utf-8?B?dGJHS3B3ZnJFR0FwbWFTQXc1czkySkM0MkRRelo0cEFsU1o4QlU4S3dRM3FU?= =?utf-8?B?d2EreVNPdzl5YllZVmhyeXpqRnU5UlorVHppSnVHb25weFFxWEpwUFlpdDcr?= =?utf-8?B?YTBLNElLbFN2YVdTUkMxNGRwdnNBUUNvV0luZUVuazNPVTFLbkR1MlVQc0Jp?= =?utf-8?B?N0JLTHFBbTBJcndwbW42aE9peUhlUVRTZjVJbHVMaG43YmJ3QXFXcURZTVpB?= =?utf-8?B?ZCtFcU40MFpMRXhuTEpidmtEa3hNZUZmaTVQTEZ1N0Y4ZHNaMkRyUDJFd0dh?= =?utf-8?B?cWV5SEV4NmRVVnVkQVFlTHlZZm52Z3VzZjM4NW14azNFZDRKYU9iR2V3OThK?= =?utf-8?B?aFVXanBkK2FjUEE2M1R1TWY2ZENPbDh2aUtvYjBDU3NXT05QZXZLMmxrNmFM?= =?utf-8?B?RDA4TzBLaUlwczBQaStMUUpGdEhLblBoRnhXdGg0MVIwenVyTUdyczY0U2hK?= =?utf-8?B?Z29YWllES3NTNWs2emNrMUdjUmJaZTlRaGM5TklRUjdkenhxMDBhcXJJYVk1?= =?utf-8?B?SFpyOWdvYlJobVNXRHdSbmtPYzNBWmo1UXVRRFcxVVBkTGMwU2lNVDhMN3Vn?= =?utf-8?B?a3h1LzhHUFM2TmRtNFNWMjBrYUtnZmY4aXBTTnlKc0ZabmsyYmE4M3loNDgv?= =?utf-8?B?NERCSFJha2krTGhkWmFLZlN4Q3NPU1BjM21lOG90cElUZ3ZCWFA0RVFab3Fj?= =?utf-8?B?cUdxK3pFTzNMSmxiWE13Vm5RU2h5KzdRMk5IVjFyMmQyQWVTUGlNN1ZwWm9o?= =?utf-8?B?WWJxckcvc1ZWR242RXBRekZNNWdiNmJkaHdWcUxWT1lMYmpJbU80bjlsR0JH?= =?utf-8?B?SEtSc1dtajVkTnRSekR0eE5JZmoxdlR1MEt1dE1icU9LWi9oR0JkWVloNk42?= =?utf-8?B?Z0NUMnFIVjl3bmRGMnFqMXp0T1pqZWhDeFhxeVk4MDFJQzlnRVVxVUxRPT0=?= X-OriginatorOrg: outlook.com X-MS-Exchange-CrossTenant-Network-Message-Id: 7d25905f-59f8-44a1-5c68-08dcb6ca4549 X-MS-Exchange-CrossTenant-AuthSource: AS8P250MB0744.EURP250.PROD.OUTLOOK.COM X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 07 Aug 2024 10:18:24.5898 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 84df9e7f-e9f6-40af-b435-aaaaaaaaaaaa X-MS-Exchange-CrossTenant-RMS-PersistedConsumerOrg: 00000000-0000-0000-0000-000000000000 X-MS-Exchange-Transport-CrossTenantHeadersStamped: PR3P250MB0306 Subject: Re: [FFmpeg-devel] [PATCH] avcodec/nvenc: Rework on DTS generation X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" Archived-At: List-Archive: List-Post: Zhao Zhili: > From: Zhao Zhili > > Before the patch, the method to generate DTS only works with > timebase equal to 1/fps. With timebase like 1/1000 > > ./ffmpeg -i foo.mp4 -an -c:v h264_nvenc -enc_time_base 1/1000 bar.mp4 > > pts 0 dts -3 > pts 160 dts 37 > pts 80 dts 77 > pts 40 dts 117 <-- invalid > pts 120 dts 157 > pts 320 dts 197 > pts 240 dts 237 > pts 200 dts 277 <-- invalid > pts 280 dts 317 <-- invalid > > The generated DTS can be larger than PTS, since it only reorder the > input PTS and minus the number of frame delay, which doesn't take > timebase into account. It should minus the "time" of frame delay. > > 9a245bd trying to fix the issue, but the implementation is incomplete, > which only use time_base.num. Then it got reverted by ac7c265b33b. > > After this patch: > > pts 0 dts -120 > pts 160 dts -80 > pts 80 dts -40 > pts 40 dts 0 > pts 120 dts 40 > pts 320 dts 80 > pts 240 dts 120 > pts 200 dts 160 > pts 280 dts 200 It looks like you only have two reorder frames here, so shouldn't the first dts be -80? > --- > libavcodec/nvenc.c | 62 ++++++++++++++++++++++++++++++++++++++-------- > libavcodec/nvenc.h | 3 +++ > 2 files changed, 55 insertions(+), 10 deletions(-) > > diff --git a/libavcodec/nvenc.c b/libavcodec/nvenc.c > index ab92395ed6..34448462f0 100644 > --- a/libavcodec/nvenc.c > +++ b/libavcodec/nvenc.c > @@ -1893,7 +1893,8 @@ static av_cold int nvenc_setup_surfaces(AVCodecContext *avctx) > if (!ctx->frame_data_array) > return AVERROR(ENOMEM); > > - ctx->timestamp_list = av_fifo_alloc2(ctx->nb_surfaces, sizeof(int64_t), 0); > + ctx->timestamp_list = av_fifo_alloc2(ctx->nb_surfaces + ctx->encode_config.frameIntervalP, > + sizeof(int64_t), 0); > if (!ctx->timestamp_list) > return AVERROR(ENOMEM); > > @@ -2347,26 +2348,65 @@ static inline int64_t timestamp_queue_dequeue(AVFifo *queue) > return timestamp; > } > > +static inline int64_t timestamp_queue_peek(AVFifo *queue, size_t index) > +{ > + int64_t timestamp = AV_NOPTS_VALUE; > + av_fifo_peek(queue, ×tamp, 1, index); > + > + return timestamp; > +} > + > static int nvenc_set_timestamp(AVCodecContext *avctx, > NV_ENC_LOCK_BITSTREAM *params, > AVPacket *pkt) > { > NvencContext *ctx = avctx->priv_data; > + int delay; > + int64_t delay_time; > > pkt->pts = params->outputTimeStamp; > > - if (avctx->codec_descriptor->props & AV_CODEC_PROP_REORDER) { > -FF_DISABLE_DEPRECATION_WARNINGS > - pkt->dts = timestamp_queue_dequeue(ctx->timestamp_list) - > -#if FF_API_TICKS_PER_FRAME > - FFMAX(avctx->ticks_per_frame, 1) * > -#endif > - FFMAX(ctx->encode_config.frameIntervalP - 1, 0); > -FF_ENABLE_DEPRECATION_WARNINGS > - } else { > + if (!(avctx->codec_descriptor->props & AV_CODEC_PROP_REORDER)) { > pkt->dts = pkt->pts; > + return 0; > + } > + > + // This can be more than necessary, but we don't know the real reorder delay. > + delay = FFMAX(ctx->encode_config.frameIntervalP - 1, 0); > + if (ctx->output_frame_num >= delay) { > + pkt->dts = timestamp_queue_dequeue(ctx->timestamp_list); > + ctx->output_frame_num++; > + return 0; > } > > + delay_time = ctx->initial_delay_time; > + if (!delay_time) { > + int64_t t1, t2, t3; > + t1 = timestamp_queue_peek(ctx->timestamp_list, delay); > + t2 = timestamp_queue_peek(ctx->timestamp_list, 0); > + t3 = (delay > 1) ? timestamp_queue_peek(ctx->timestamp_list, 1) : t1; > + > + if (t1 != AV_NOPTS_VALUE) { > + delay_time = t1 - t2; > + } else if (avctx->framerate.num > 0 && avctx->framerate.den > 0) { > + delay_time = av_rescale_q(delay, (AVRational) {avctx->framerate.den, avctx->framerate.num}, > + avctx->time_base); > + } else if (t3 != AV_NOPTS_VALUE) { > + delay_time = delay * (t3 - t2); > + } else { > + delay_time = delay; > + } > + ctx->initial_delay_time = delay_time; > + } > + > + /* The following method is simple, but doesn't guarantee monotonic with VFR > + * when delay_time isn't accurate (that is, t1 == AV_NOPTS_VALUE) > + * > + * dts = timestamp_queue_peek(ctx->timestamp_list, ctx->output_frame_num) - delay_time > + */ > + pkt->dts = timestamp_queue_peek(ctx->timestamp_list, 0) - delay_time * (delay - ctx->output_frame_num) / delay; > + ctx->output_frame_num++; > + > return 0; > } > > @@ -2902,4 +2942,6 @@ av_cold void ff_nvenc_encode_flush(AVCodecContext *avctx) > > nvenc_send_frame(avctx, NULL); > av_fifo_reset2(ctx->timestamp_list); > + ctx->output_frame_num = 0; > + ctx->initial_delay_time = 0; > } > diff --git a/libavcodec/nvenc.h b/libavcodec/nvenc.h > index 09de00badc..dc7fe41951 100644 > --- a/libavcodec/nvenc.h > +++ b/libavcodec/nvenc.h > @@ -206,6 +206,9 @@ typedef struct NvencContext > AVFifo *output_surface_queue; > AVFifo *output_surface_ready_queue; > AVFifo *timestamp_list; > + // This is for DTS calculating, reset after flush > + int64_t output_frame_num; > + int64_t initial_delay_time; > > NV_ENC_SEI_PAYLOAD *sei_data; > int sei_data_size; _______________________________________________ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".