From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from ffbox0-bg.ffmpeg.org (ffbox0-bg.ffmpeg.org [79.124.17.100]) by master.gitmailbox.com (Postfix) with ESMTPS id 460674D294 for ; Sun, 16 Nov 2025 06:30:24 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ffmpeg.org; i=@ffmpeg.org; q=dns/txt; s=mail; t=1763274587; h=to : date : message-id : mime-version : reply-to : subject : list-id : list-archive : list-archive : list-help : list-owner : list-post : list-subscribe : list-unsubscribe : from : cc : content-type : content-transfer-encoding : from; bh=zDJf6vO6oAnmQHj9xcA/4fn2gbyTPj6EV1YKpj5Bb6g=; b=m/CIQunNNrSEpGwQDW7Fzey52a/E7NpP7gaIOBWnsY29UXZyVKc1Oi9N1OFbP/NXmIDps +FagD/IVTMsONb6M8EL7bSqMgWoSfWf+KUbWfTsekzgSSjp59jW7PStpdFk2p+bPm+CcLj3 HSj8EiQQGD0MWfNb3oXaW+LKHNqkbYMSi0vp9fD8FTd5DGEkIXrJ0e1MFVDC4tWYOF4htlh ebtnKCJbVhVo8bZAP95mS8Z/++7rmAluPtKdCAn+8lw6O5J+sLuZ4MCET853q7ZhG5Z0RuF mloMzlvZnQ8mrtWkvZYOgF732W3TfR5rUn11t/t2+JiKG/oW3RfBWTlDpvvQ== Received: from [172.19.0.2] (unknown [172.19.0.2]) by ffbox0-bg.ffmpeg.org (Postfix) with ESMTP id 6AB9A68FE66; Sun, 16 Nov 2025 08:29:47 +0200 (EET) ARC-Seal: i=1; cv=none; a=rsa-sha256; d=ffmpeg.org; s=arc; t=1763274552; b=m7rfh3Yfs1YftXKuNqILL7XIW6cmByF4HlyRYyF8AmmTYksjaxJgVUH+gpktClzge/TOH GQVfk0JGqtJzWX2a+htSzvZNobVsZzwEdJcxzijLSlY6l+NJ+IHKLYPzvnS3QJz4AIDIIra Ypfz8UWTupUN2N2WtgZtxWIwIj6dX2GZxeaAgmqTlmltv+xJuw/sQ7lsyj5EM+3QuapkcjB 4Fu9IMzFzz/cCIVf69g8bzfugShbFsjA/Kw3j0EhdCReqhTpo//G4I5Dp5mDBRh9apm+W7r anVayVbjwLXKu7ciFWxMPqhu+gpsLuR8m1kfR3xvPSvKmFE+mIFlu9Pdrg7Q== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=ffmpeg.org; s=arc; t=1763274552; h=from : sender : reply-to : subject : date : message-id : to : cc : mime-version : content-type : content-transfer-encoding : content-id : content-description : resent-date : resent-from : resent-sender : resent-to : resent-cc : resent-message-id : in-reply-to : references : list-id : list-help : list-unsubscribe : list-subscribe : list-post : list-owner : list-archive; bh=0F9ZvJwQkQLzMblyjPz/8NBlNzpXFVDyflIcSTT/mzA=; b=EZZWzsYevfpQLMcngs45EskAy/QsPH1i1RAP5z2e2AW3EHybk/1y8t+f4PN4DbN1dIoSv 4ABHsjNbLR8AvJeyne32dkCm8xPM04Wmca1fLfvcN0dufMLaj/ae5wWuzsd8VlAmPT5Hd/W uTHYtNzAjhEqBj10KqUn+Ckc8lA8Er9j6ZdJdrtjxzWUHiC5K67Ma8+kpUVATYZnPuUaUFI zMA4mW4+mRMHeP6wwn4kImP0OgUI0THLy8NmLBsKtvKwtOMT8cO9tdBpN9SKmuxYuUxS7bZ AuFsdPVfHDN4lhBqjjLxqs60ijeTiDlu5bRLX8zRCrDs3rFBq18ZtTde3Ktw== ARC-Authentication-Results: i=1; ffmpeg.org; dkim=fail; arc=none; dmarc=fail header.from=gmail.com policy.dmarc=quarantine Authentication-Results: ffmpeg.org; dkim=fail; arc=none (Message is not ARC signed); dmarc=fail (Used From Domain Record) header.from=gmail.com policy.dmarc=quarantine Received: from mail-pl1-f172.google.com (mail-pl1-f172.google.com [209.85.214.172]) by ffbox0-bg.ffmpeg.org (Postfix) with ESMTPS id 1879568FDAD for ; Sat, 15 Nov 2025 12:27:53 +0200 (EET) Received: by mail-pl1-f172.google.com with SMTP id d9443c01a7336-29845b06dd2so30476495ad.2 for ; Sat, 15 Nov 2025 02:27:53 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1763202471; x=1763807271; h=content-transfer-encoding:mime-version:message-id:date:subject:cc :to:from:x-gm-gg:x-gm-message-state:from:to:cc:subject:date :message-id:reply-to; bh=40zLlmrFoWwXTZiojUA+gQRSAMOocYVP1ZccDTMOKWg=; b=m6qyIYF1TuszRfouASwprV3TqhioqG0ANqaqWwbZE4dy9+6zIky1Qh1G7EFCjCYyrq ddKXfn9TAnfsX7r7b+voyilJ5NSydNPqdi28gNM5J6z3eFMDwMGLbwUXzYsRTJ5WIaqm ncj58qtn0SRTFIn6YNiF+Cbdy8f/ybJlzL3TkFBETYZ+P41qssElpIa5zhsNw9XTbH4W vcGETOjZeM4BQFsomwbnLVGx5f/5zfPNvbcdKZdLGNiEqed6c26JsZY29Gh8xwyGJtEe KS7zanyg1uKreu42HVAl5t6K7k7q7/L7Uw/BthelObJ4a0qiuJbRUAe63jX6Gm+Y/VI4 LCJw== X-Gm-Message-State: AOJu0YyBQKSv66uYb+rrjGZXjjZr/Crof4LJlds+hNb3y4GPMWBi+mss MeMiPppehW3P8rcGVkYSldelDni9xPcy3oMnmMac4mYerAwtEzIglyd9gLk8kPBY X-Gm-Gg: ASbGncvOvTa7PFCyiOhLgjDGB2F4Q+SlZLF9o1k5JpjgM8ezPkoPMZF2gZ8Wf8v72aJ 54AM+3nAm4qpacCFSCPSPuG+ZIfRuVZGgL0BMHQeZCHSXidwt8F3q9ZhQN6XIeFh3V9taEappCd P0tp3Yvf2xhcvbboONoAsHEpK1egqW9d7g4Kxy1eFF/GbwguqPcT6dnNiJWVRndTE7nxLHkQ87/ c+sgjFkbxfdHaDM4i2DKP/yuOx7FtfvjCWd9pHohcrlxgxIwd6LuEqc4WMalXpnG2xZxGkAzq1F fRyl1VGllTBYQEqqMYH7uw+HlyDhoaVHHyhNJBas6igoJRWoUHTE2SB4LHg5OUeAgctbO2DWUds DkIg2xT9uFrmcqc/BkPcRv2Q1bHhkJfoKhmcuHQiyjE4KJm/U/eiLpw9c4aRFHpMYzMc4e/0WMz pm7wR52FNDqR7ppnbniiWEL3dp9UEnYQ== X-Google-Smtp-Source: AGHT+IEa15VqPJ4WneNZCtyZpUFLt2hhg/mlEVs0Y7BhBcEeq6kdzceYUzR5fMjoySaKOjhF7msPjg== X-Received: by 2002:a17:903:120b:b0:295:613f:3d6a with SMTP id d9443c01a7336-2986a72ca49mr72666045ad.29.1763202471003; Sat, 15 Nov 2025 02:27:51 -0800 (PST) Received: from localhost.localdomain ([2601:601:8382:5120:647f:78cf:686b:a4b3]) by smtp.gmail.com with ESMTPSA id d9443c01a7336-2985c2348afsm82361645ad.3.2025.11.15.02.27.50 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sat, 15 Nov 2025 02:27:50 -0800 (PST) To: ffmpeg-devel@ffmpeg.org Date: Sat, 15 Nov 2025 02:27:16 -0800 Message-Id: <20251115102716.63645-1-boxuffmpeg@gmail.com> X-Mailer: git-send-email 2.39.5 (Apple Git-154) MIME-Version: 1.0 X-MailFrom: SRS0=rzGi=5X=gmail.com=boxuffmpeg@ffmpeg.org X-Mailman-Rule-Hits: nonmember-moderation X-Mailman-Rule-Misses: dmarc-mitigation; no-senders; approved; loop; banned-address; header-match-ffmpeg-devel.ffmpeg.org-0; header-match-ffmpeg-devel.ffmpeg.org-1; header-match-ffmpeg-devel.ffmpeg.org-2; header-match-ffmpeg-devel.ffmpeg.org-3; emergency; member-moderation Message-ID-Hash: JFHEATXY6EO3NXLGIOF6RDUN2VKOJAXA X-Message-ID-Hash: JFHEATXY6EO3NXLGIOF6RDUN2VKOJAXA X-Mailman-Approved-At: Sun, 16 Nov 2025 06:29:02 +0000 X-Mailman-Version: 3.3.10 Precedence: list Reply-To: FFmpeg development discussions and patches Subject: [FFmpeg-devel] [PATCH] avcodec/videotoolboxenc: Fix B-frame timestamp handling for proper PTS/DTS ordering List-Id: FFmpeg development discussions and patches Archived-At: Archived-At: List-Archive: List-Archive: List-Help: List-Owner: List-Post: List-Subscribe: List-Unsubscribe: From: Bo Xu boxuffmpeg--- via ffmpeg-devel Cc: boxu , Bo Xu boxuffmpeg@gmail.com Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Archived-At: List-Archive: List-Post: From: boxu VideoToolbox encoder with B-frames can produce non-monotonic DTS values, causing playback issues. This patch implements a complete timestamp reset strategy to ensure correct PTS/DTS ordering. The fix: - Generates monotonic DTS values based on decode order (frame counter) - Preserves VideoToolbox's presentation order for PTS - Adds offset (max_b_frames) to ensure DTS <= PTS for all frames - Normalizes timestamps to start from zero Testing command: ffmpeg -hwaccel videotoolbox -i input.mov -c:v h264_videotoolbox -profile:v high -bf 3 -b:v 2M -y output.mov Before this patch: [vost#0:0/h264_videotoolbox @ 0xc61058000] Invalid DTS: 1024 PTS: 512, replacing by guess [vost#0:0/h264_videotoolbox @ 0xc61058000] Invalid DTS: 3072 PTS: 2560, replacing by guess [vost#0:0/h264_videotoolbox @ 0xc61058000] Invalid DTS: 5632 PTS: 5120, replacing by guess [vost#0:0/h264_videotoolbox @ 0xc61058000] Invalid DTS: 7680 PTS: 7168, replacing by guess [vost#0:0/h264_videotoolbox @ 0xc61058000] Invalid DTS: 10240 PTS: 9728, replacing by guess After this patch: No DTS errors, clean output with proper muxing. Tested on macOS with B-frame encoding enabled. Signed-off-by: boxu --- libavcodec/videotoolboxenc.c | 53 ++++++++++++++++++++++++++++++++++-- 1 file changed, 50 insertions(+), 3 deletions(-) diff --git a/libavcodec/videotoolboxenc.c b/libavcodec/videotoolboxenc.c index 729072c0b9..a360867a61 100644 --- a/libavcodec/videotoolboxenc.c +++ b/libavcodec/videotoolboxenc.c @@ -264,6 +264,13 @@ typedef struct VTEncContext { int64_t first_pts; int64_t dts_delta; + int64_t last_dts; // Track last DTS for B-frame monotonicity + int64_t last_non_b_dts; // Track last I/P frame DTS for B-frame reference + + // Full PTS/DTS reset for B-frames + int64_t base_pts; // Base PTS value (first frame's PTS) + int64_t base_dts; // Base DTS value (always 0) + int64_t vt_pts_first; // VideoToolbox's first frame PTS for offset calculation int profile; int level; @@ -2320,10 +2327,43 @@ static int vtenc_cm_to_avpacket( } } - dts_delta = vtctx->dts_delta >= 0 ? vtctx->dts_delta : 0; time_base_num = avctx->time_base.num; - pkt->pts = pts.value / time_base_num; - pkt->dts = dts.value / time_base_num - dts_delta; + dts_delta = vtctx->dts_delta >= 0 ? vtctx->dts_delta : 0; + int64_t vt_pts = pts.value / time_base_num; + int64_t vt_dts = dts.value / time_base_num; + + // VideoToolbox with B-frames: Fully reset both PTS and DTS from scratch + // Strategy: Maintain our own base timestamps and completely regenerate values + // - DTS = frame_ct_out (decode order, always monotonic) + // - PTS = preserve VideoToolbox's presentation order + offset for B-frames + // Key insight: Add offset equal to max_b_frames so that earliest B-frames have PTS > DTS + if (vtctx->has_b_frames) { + // First frame: record VideoToolbox's PTS offset for normalization + if (vtctx->vt_pts_first == AV_NOPTS_VALUE) { + vtctx->vt_pts_first = vt_pts; + // Add offset equal to max_b_frames to ensure DTS <= PTS for all frames + // This gives B-frames enough "headroom" to be decoded before display + vtctx->base_pts = avctx->max_b_frames; + vtctx->base_dts = 0; // DTS starts from 0 + av_log(avctx, AV_LOG_DEBUG, "First frame: VT_PTS=%lld, VT_DTS=%lld, setting base_pts=%lld (max_b_frames=%d)\n", + vt_pts, vt_dts, vtctx->base_pts, avctx->max_b_frames); + } + + // Generate DTS from frame counter (decode order: 0, 1, 2, 3, ...) + // Note: frame_ct_out starts at 1, so subtract 1 to start DTS from 0 + pkt->dts = vtctx->base_dts + (vtctx->frame_ct_out - 1); + + // Generate PTS preserving VideoToolbox's presentation order with B-frame offset + // Normalize VideoToolbox's PTS and add base_pts offset + pkt->pts = vtctx->base_pts + (vt_pts - vtctx->vt_pts_first); + + av_log(avctx, AV_LOG_DEBUG, "Reset timestamps: frame=%lld, VT_PTS=%lld, VT_DTS=%lld, final_PTS=%lld, final_DTS=%lld\n", + vtctx->frame_ct_out, vt_pts, vt_dts, pkt->pts, pkt->dts); + } else { + // No B-frames: use VideoToolbox's timestamps with adjustment + pkt->pts = vt_pts; + pkt->dts = vt_dts - dts_delta; + } return 0; } @@ -2828,6 +2868,13 @@ pe_cleanup: } vtctx->frame_ct_out = 0; + vtctx->last_dts = AV_NOPTS_VALUE; + vtctx->last_non_b_dts = AV_NOPTS_VALUE; + + // Initialize PTS/DTS reset variables + vtctx->base_pts = 0; + vtctx->base_dts = 0; + vtctx->vt_pts_first = AV_NOPTS_VALUE; av_assert0(status != 0 || (avctx->extradata && avctx->extradata_size > 0)); if (!status) -- 2.39.5 (Apple Git-154) _______________________________________________ ffmpeg-devel mailing list -- ffmpeg-devel@ffmpeg.org To unsubscribe send an email to ffmpeg-devel-leave@ffmpeg.org