From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org [79.124.17.100]) by master.gitmailbox.com (Postfix) with ESMTP id D995E43AA1 for ; Mon, 8 Aug 2022 00:01:52 +0000 (UTC) Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id C23E368B701; Mon, 8 Aug 2022 03:01:48 +0300 (EEST) Received: from mail-vk1-f177.google.com (mail-vk1-f177.google.com [209.85.221.177]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 44E0568B6B0 for ; Mon, 8 Aug 2022 03:01:43 +0300 (EEST) Received: by mail-vk1-f177.google.com with SMTP id w129so3673041vkg.10 for ; Sun, 07 Aug 2022 17:01:43 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=content-transfer-encoding:mime-version:message-id:date:subject:to :from:from:to:cc; bh=bU3DO25wcDB9RsjmBxU1dAHqiEQt/I3Kw38srP6/kmo=; b=C75kXaLvSlcepbcVzxN0J6VqcKmfNnCTj+nEBbp0pJrfArcZ0nnsjMG1Km+i/JnEKk rotB/Da4wwk23crPrVmDNpLVksHjS7rGx4dgjINA9GCR88/LEfRS7v0Oja+2pLbBfJbr H8dcQiLnBCFgBsjaeCo5gzEx7YmwBBBHYfns44OJoLc+UKTlyqZyO7be1vX40fg2FGPk gGTb1D4WDWlcU+Nmn5iJ2lUI9ahZfuLA1Z4PhlLLprBIQnNv6zBAzbzYpvOnJpzU9KY8 iEUchHlbbI83iFD05ZxF5Y6kDse8YQvg7A0TC83PNV0wZlUn5cqNPwSj1ytZdii0i1pD VEjg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:message-id:date:subject:to :from:x-gm-message-state:from:to:cc; bh=bU3DO25wcDB9RsjmBxU1dAHqiEQt/I3Kw38srP6/kmo=; b=uCze1+AHjfMEAJbCsWhpvn75DV4m9cM6LZ5GGzYIpZ0GMQwqxx7fUBbaRTgrnUtTEB 1O19hvcguFLEVBUvfYD+mvCU4KTEl7Or63VaLJ4TdAuLj+Bgk2HveEk9QI9kSubfa8F0 CBE9IV19IC6tDZaAzEdSS2Sf3dWmLGXXsMDPKxZrpVx2vxXMadwKa5OdEoKtoLVVGv17 pJ1yBHISHxf9+8YqmKV26nc/aKt7EbCjwy1Ogs71Q+PALVbO76CZnVooN1RZcHyH1DNk xmbozdN0SewphE4CRSa/KHKUvyi1elgyzJoOaTPv3qOLbkDNUW9SOKb2g1vjReKVOYg7 gpbg== X-Gm-Message-State: ACgBeo1xB/OsIWAM/u/Bcq0bUUosEEjgbGZjwKk49yxExZqN0ccNLJRo 1C1Pn14pPFt5ApNnCpOO4LPgGTlTpaA= X-Google-Smtp-Source: AA6agR7psu9XDS8KrHLv7ix4u9i1jnOSpPqYuDGk1RqUMtSsq7JuRk4AkBEiUgY75MipPj3/npcsRA== X-Received: by 2002:a05:6122:643:b0:378:d1a9:b768 with SMTP id h3-20020a056122064300b00378d1a9b768mr2250634vkp.41.1659916901051; Sun, 07 Aug 2022 17:01:41 -0700 (PDT) Received: from localhost.localdomain ([186.136.131.204]) by smtp.gmail.com with ESMTPSA id t2-20020a0561020a0200b00385d88375f4sm6937561vsa.32.2022.08.07.17.01.39 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 07 Aug 2022 17:01:40 -0700 (PDT) From: James Almer To: ffmpeg-devel@ffmpeg.org Date: Sun, 7 Aug 2022 21:01:24 -0300 Message-Id: <20220808000124.12794-1-jamrial@gmail.com> X-Mailer: git-send-email 2.37.1 MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH] avfilter/framesync: add a new option to set how to sync streams based on secondary input timestamps X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" Archived-At: List-Archive: List-Post: Include two values for it, a default one that sets/keeps the current behavior, where the frame event generated by the primary input will have a timestamp equal or higher than frames in secondary input, plus a new one where the secondary input frame will be that with the absolute closest timestamp to that of the frame event one. Addresses ticket #9689, where the new optional behavior produces better frame syncronization. Signed-off-by: James Almer --- libavfilter/framesync.c | 19 +++++++++++++++++++ libavfilter/framesync.h | 23 +++++++++++++++++++++++ 2 files changed, 42 insertions(+) diff --git a/libavfilter/framesync.c b/libavfilter/framesync.c index 7510550d8e..bd784f6e0b 100644 --- a/libavfilter/framesync.c +++ b/libavfilter/framesync.c @@ -42,6 +42,13 @@ static const AVOption framesync_options[] = { { "pass", "Pass through the main input.", 0, AV_OPT_TYPE_CONST, { .i64 = EOF_ACTION_PASS }, .flags = FLAGS, "eof_action" }, { "shortest", "force termination when the shortest input terminates", OFFSET(opt_shortest), AV_OPT_TYPE_BOOL, { .i64 = 0 }, 0, 1, FLAGS }, { "repeatlast", "extend last frame of secondary streams beyond EOF", OFFSET(opt_repeatlast), AV_OPT_TYPE_BOOL, { .i64 = 1 }, 0, 1, FLAGS }, + { "ts_sync_mode", "How strictly to sync streams based on secondary input timestamps", + OFFSET(opt_ts_sync_mode), AV_OPT_TYPE_INT, { .i64 = TS_DEFAULT }, + TS_DEFAULT, TS_NEAREST, .flags = FLAGS, "ts_sync_mode" }, + { "default", "Frame from secondary input with the nearest lower or equal timestamp to the primary input frame", + 0, AV_OPT_TYPE_CONST, { .i64 = TS_DEFAULT }, .flags = FLAGS, "ts_sync_mode" }, + { "nearest", "Frame from secondary input with the absolute nearest timestamp to the primary input frame", + 0, AV_OPT_TYPE_CONST, { .i64 = TS_NEAREST }, .flags = FLAGS, "ts_sync_mode" }, { NULL } }; static const AVClass framesync_class = { @@ -110,6 +117,14 @@ static void framesync_sync_level_update(FFFrameSync *fs) av_assert0(level <= fs->sync_level); if (level < fs->sync_level) av_log(fs, AV_LOG_VERBOSE, "Sync level %u\n", level); + if (fs->opt_ts_sync_mode > TS_DEFAULT) { + for (i = 0; i < fs->nb_in; i++) { + if (fs->in[i].sync < level) + fs->in[i].ts_mode = fs->opt_ts_sync_mode; + else + fs->in[i].ts_mode = TS_DEFAULT; + } + } if (level) fs->sync_level = level; else @@ -187,6 +202,10 @@ static int framesync_advance(FFFrameSync *fs) } for (i = 0; i < fs->nb_in; i++) { if (fs->in[i].pts_next == pts || + (fs->in[i].ts_mode == TS_NEAREST && + fs->in[i].have_next && + fs->in[i].pts_next != INT64_MAX && fs->in[i].pts != AV_NOPTS_VALUE && + FFABS(pts - fs->in[i].pts_next) < FFABS(pts - fs->in[i].pts)) || (fs->in[i].before == EXT_INFINITY && fs->in[i].state == STATE_BOF)) { av_frame_free(&fs->in[i].frame); diff --git a/libavfilter/framesync.h b/libavfilter/framesync.h index a246d2d1e5..233f50a0eb 100644 --- a/libavfilter/framesync.h +++ b/libavfilter/framesync.h @@ -75,6 +75,27 @@ enum FFFrameSyncExtMode { EXT_INFINITY, }; +/** + * Timestamp syncronization mode + * + * Describe how the frames of a stream are syncronized based on timestamp + * distance. + */ +enum FFFrameTSSyncMode { + + /** + * Sync to frames from secondary input with the nearest, lower or equal + * timestamp to the frame event one. + */ + TS_DEFAULT, + + /** + * Sync to frames from secondary input with the absolute nearest timestamp + * to the frame event one. + */ + TS_NEAREST, +}; + /** * Input stream structure */ @@ -138,6 +159,7 @@ typedef struct FFFrameSyncIn { */ unsigned sync; + enum FFFrameTSSyncMode ts_mode; } FFFrameSyncIn; /** @@ -205,6 +227,7 @@ typedef struct FFFrameSync { int opt_repeatlast; int opt_shortest; int opt_eof_action; + int opt_ts_sync_mode; } FFFrameSync; -- 2.37.1 _______________________________________________ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".