From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from ffbox0-bg.ffmpeg.org (ffbox0-bg.ffmpeg.org [79.124.17.100]) by master.gitmailbox.com (Postfix) with ESMTPS id 9FFA64C511 for ; Tue, 14 Oct 2025 09:30:02 +0000 (UTC) Authentication-Results: ffbox; dkim=fail (body hash mismatch (got b'cxM+IWO9GcfhhgqzORPRT4j8oGBegg1leyAtvm54MGg=', expected b'jW6pkwSzggb9Uk2XYGR6BNgaNeBFoFR6q5+hJl7wdN0=')) header.d=ffmpeg.org header.i=@ffmpeg.org header.a=rsa-sha256 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ffmpeg.org; i=@ffmpeg.org; q=dns/txt; s=mail; t=1760434181; h=mime-version : to : date : message-id : reply-to : subject : list-id : list-archive : list-archive : list-help : list-owner : list-post : list-subscribe : list-unsubscribe : from : cc : content-type : content-transfer-encoding : from; bh=cxM+IWO9GcfhhgqzORPRT4j8oGBegg1leyAtvm54MGg=; b=PtWosNCAJ/FrETsJJfZmDJJOI0ZtYopvSpVKhoNxfDJswL0778TsP7YmPEKOBjNRfGdZe ayiLx2+KfWMgI+pxgJAOk2lEKqedWyYPHkn4fzqKqpYBFu86sD5wi50m6VIg2J/ZHjx+ApS ddzL5B0IaD36E4fhy4LWWeVwS6hbD451EiXsb9nE4UVrKcBx+19qZoYlAGarAf83vwIY2tg 1gd3/D3EukUYWUWnocpkJML1mHfBW6ymCMu6n+4x3/R8ow4ZB54H7w1tpP1M7AVXcoXp21i vm4aIETzwM+by9BJSwKfay4/iclufmRpsulBqndsnPzrDdcKD1xe0WPXrk2A== Received: from [172.19.0.4] (unknown [172.19.0.4]) by ffbox0-bg.ffmpeg.org (Postfix) with ESMTP id 625D068F24F; Tue, 14 Oct 2025 12:29:41 +0300 (EEST) ARC-Seal: i=1; cv=none; a=rsa-sha256; d=ffmpeg.org; s=arc; t=1760434178; b=Dje/O1M16P5AHKaarvkyqsB9Z5zEplA771l4orgdm3h70R4BKNxEvMhd+Ms/QqHTehs3v 04MI1/l3ZvNFHMduBy8ohygMG4qx0x5qXWwxw3dh0wmShBI4cyQwup41FeTYQbS3bpOxzSE UPhEcyQT39OK/ZRu3fdFz2wVVfxcPJsR6AX/i8hm25vELI88DuV1DplXlD9oTWXRngJpabM 7W+UrHwr+0ej1CumQmMuavBbvqh/lqmTqVw2+iffllPZ1xoE89VLAwo36CNU6kr4XvvQofW fnQAnTabCFGsySGifohkOjpeuS34aUjoPp7v0JLYAvbCX7s9RwY0XTQuU7rQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=ffmpeg.org; s=arc; t=1760434178; h=from : sender : reply-to : subject : date : message-id : to : cc : mime-version : content-type : content-transfer-encoding : content-id : content-description : resent-date : resent-from : resent-sender : resent-to : resent-cc : resent-message-id : in-reply-to : references : list-id : list-help : list-unsubscribe : list-subscribe : list-post : list-owner : list-archive; bh=hZ2vtr2/2ZRVXH9OIRRFZkHl1v4lUYdJLOK+Z0JYKxU=; b=Qkc9YlM99VxqMoYJAQCT5XG+5+RXQ9r7g/2TdNrspWksKDBPQN9QCUxHwwR7hXLG+88eW 4P77rmPHNFxpTWn7hVokr5zYgqoklWOT9fa97KSJ5tkV3fqHDwCtQNoY+viC/MjJuzkiEFg Bv0aZOmPT2tLF15U3aA5kl+VC9FDqfTjZ6vc9Li5nNQb9o4imdWxPG0JCZ5ylb+G84HaeOW fep1ZjXRjhY/KeD2ITJhKfDHrRm/4t97+waTGr3ahkiiorxDF0b68t8Q61TBtecmXXxqX/x oJa8LYhXUCkEmoNS+MojL7HO7NnXDf2VloWXFRQ8KuRWGHPwZkYMxvFlqbkw== ARC-Authentication-Results: i=1; ffmpeg.org; dkim=pass header.d=ffmpeg.org header.i=@ffmpeg.org; arc=none; dmarc=pass header.from=ffmpeg.org policy.dmarc=quarantine Authentication-Results: ffmpeg.org; dkim=pass header.d=ffmpeg.org header.i=@ffmpeg.org; arc=none (Message is not ARC signed); dmarc=pass (Used From Domain Record) header.from=ffmpeg.org policy.dmarc=quarantine DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ffmpeg.org; i=@ffmpeg.org; q=dns/txt; s=mail; t=1760434172; h=content-type : mime-version : content-transfer-encoding : from : to : reply-to : subject : date : from; bh=jW6pkwSzggb9Uk2XYGR6BNgaNeBFoFR6q5+hJl7wdN0=; b=djIHUtcJTAqOxA2u09wSuf39t636tht8rwSJ528AgNoYXg3edlpRIIBTw/QNHjHkvwvOg iRTkkA6YuD/n/YibD2/6PvuZ6TaNMSXuskGjaf9sEtD1CqL1BKki0Pnl66DMm1hK0zHWsQ7 SJL3l0akoxt7xLhF+agDUKsbgWDT78bE1Ezs+rkJaJkj6txzJT6sr4tfFe8r6ZC1yBQ65Qe ZJXxxGicjXDqH7tcAocUEYT/3kJ7r1qfxd+/rAEVtNCHVcQgNXzF4xMtdUphpZ0OGA4ZxkM VWonoZ2rgSbXh8Nl4dGWX215wp67hF4tBcG/08ZTbijwuKlL3VA8HQI6FtiA== Received: from be50bb5a3685 (code.ffmpeg.org [188.245.149.3]) by ffbox0-bg.ffmpeg.org (Postfix) with ESMTPS id 1C8CC68F1C0 for ; Tue, 14 Oct 2025 12:29:32 +0300 (EEST) MIME-Version: 1.0 To: ffmpeg-devel@ffmpeg.org Date: Tue, 14 Oct 2025 09:29:31 -0000 Message-ID: <176043417225.25.10015622032635140442@bf907ddaa564> Message-ID-Hash: JAVN6RZJ4766RXVIKVPWGPE35KE5FJY3 X-Message-ID-Hash: JAVN6RZJ4766RXVIKVPWGPE35KE5FJY3 X-MailFrom: code@ffmpeg.org X-Mailman-Rule-Misses: dmarc-mitigation; no-senders; approved; loop; banned-address; header-match-ffmpeg-devel.ffmpeg.org-0; header-match-ffmpeg-devel.ffmpeg.org-1; header-match-ffmpeg-devel.ffmpeg.org-2; header-match-ffmpeg-devel.ffmpeg.org-3; emergency; member-moderation; nonmember-moderation; administrivia; implicit-dest; max-recipients; max-size; news-moderation; no-subject; digests; suspicious-header X-Mailman-Version: 3.3.10 Precedence: list Reply-To: FFmpeg development discussions and patches Subject: [FFmpeg-devel] [PATCH] avformat/whip: add rtp history store and find method (PR #20703) List-Id: FFmpeg development discussions and patches Archived-At: Archived-At: List-Archive: List-Archive: List-Help: List-Owner: List-Post: List-Subscribe: List-Unsubscribe: From: Jack Lau via ffmpeg-devel Cc: Jack Lau Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Archived-At: List-Archive: List-Post: PR #20703 opened by Jack Lau (JackLau) URL: https://code.ffmpeg.org/FFmpeg/FFmpeg/pulls/20703 Patch URL: https://code.ffmpeg.org/FFmpeg/FFmpeg/pulls/20703.patch This patch aims to enable rtp history store for RTX TODO: handle the rtx send to make rtx really work >>From 90b863a2f54964009cff4dcf367e38267a0ad01b Mon Sep 17 00:00:00 2001 From: Jack Lau Date: Fri, 3 Oct 2025 10:33:55 +0800 Subject: [PATCH 1/2] avformat/whip: add rtp history store and find method This patch aims to enable rtp history store for RTX TODO: handle the rtx send to make rtx really work Signed-off-by: Jack Lau --- doc/muxers.texi | 4 +++ libavformat/whip.c | 66 ++++++++++++++++++++++++++++++++++++++++++++++ 2 files changed, 70 insertions(+) diff --git a/doc/muxers.texi b/doc/muxers.texi index 9889bd2ff6..91dc90008c 100644 --- a/doc/muxers.texi +++ b/doc/muxers.texi @@ -3959,6 +3959,10 @@ Default value is 1200. Set the buffer size, in bytes, of underlying protocol. Default value is -1(auto). The UDP auto selects a reasonable value. +@item rtp_history @var{integer} +Set the number of RTP history items to store. +Default value is 512. + @item authorization @var{string} The optional Bearer token for WHIP Authorization. diff --git a/libavformat/whip.c b/libavformat/whip.c index e809075643..0f00a6652b 100644 --- a/libavformat/whip.c +++ b/libavformat/whip.c @@ -168,6 +168,19 @@ #define WHIP_ICE_CONSENT_CHECK_INTERVAL 5000 #define WHIP_ICE_CONSENT_EXPIRED_TIMER 30000 +/** + * RTP history packet size. + * Target: hold 1000ms of RTP traffic. + * + * Formula: + * bandwidth_bps = (RTP payload bytes) * (RTP history size) * 8 + * + * Assumes average RTP payload is 1184 bytes (MTU - SRTP_CHECKSUM_LEN). + */ +#define WHIP_RTP_HISTORY_MIN 64 /* around 0.61 Mbps */ +#define WHIP_RTP_HISTORY_DEFAULT 512 /* around 4.85 Mbps */ +#define WHIP_RTP_HISTORY_MAX 2048 /* around 19.40 Mbps */ + /* Calculate the elapsed time from starttime to endtime in milliseconds. */ #define ELAPSED(starttime, endtime) ((float)(endtime - starttime) / 1000) @@ -211,6 +224,12 @@ enum WHIPState { WHIP_STATE_FAILED, }; +typedef struct RtpHistoryItem { + uint16_t seq; + int size; + uint8_t *buf; +} RtpHistoryItem; + typedef struct WHIPContext { AVClass *av_class; @@ -329,6 +348,11 @@ typedef struct WHIPContext { /* The certificate and private key used for DTLS handshake. */ char* cert_file; char* key_file; + + int hist_sz; + RtpHistoryItem *hist; + uint8_t *hist_pool; + int hist_head; } WHIPContext; /** @@ -418,6 +442,17 @@ static av_cold int initialize(AVFormatContext *s) whip->audio_first_seq = av_lfg_get(&whip->rnd) & 0x0fff; whip->video_first_seq = whip->audio_first_seq + 1; + whip->hist = av_calloc(whip->hist_sz, sizeof(*whip->hist)); + if (!whip->hist) + return AVERROR(ENOMEM); + + whip->hist_pool = av_calloc(whip->hist_sz, whip->pkt_size - DTLS_SRTP_CHECKSUM_LEN); + if (!whip->hist_pool) + return AVERROR(ENOMEM); + + for (int i = 0; i < whip->hist_sz; i++) + whip->hist[i].buf = whip->hist_pool + i * (whip->pkt_size - DTLS_SRTP_CHECKSUM_LEN); + if (whip->pkt_size < ideal_pkt_size) av_log(whip, AV_LOG_WARNING, "pkt_size=%d(<%d) is too small, may cause packet loss\n", whip->pkt_size, ideal_pkt_size); @@ -1473,6 +1508,28 @@ end: return ret; } +static int rtp_history_store(WHIPContext *whip, const uint8_t *buf, int size) +{ + uint16_t seq = AV_RB16(buf + 2); + uint32_t pos = ((uint32_t)seq - (uint32_t)whip->video_first_seq) % (uint32_t)whip->hist_sz; + RtpHistoryItem *it = &whip->hist[pos]; + if (size > whip->pkt_size - DTLS_SRTP_CHECKSUM_LEN) + return AVERROR_INVALIDDATA; + memcpy(it->buf, buf, size); + it->size = size; + it->seq = seq; + + whip->hist_head = ++pos; + return 0; +} + +static const RtpHistoryItem *rtp_history_find(WHIPContext *whip, uint16_t seq) +{ + uint32_t pos = ((uint32_t)seq - (uint32_t)whip->video_first_seq) % (uint32_t)whip->hist_sz; + const RtpHistoryItem *it = &whip->hist[pos]; + return it->seq == seq ? it : NULL; +} + /** * Callback triggered by the RTP muxer when it creates and sends out an RTP packet. * @@ -1509,6 +1566,12 @@ static int on_rtp_write_packet(void *opaque, const uint8_t *buf, int buf_size) return 0; } + if (is_video) { + ret = rtp_history_store(whip, buf, buf_size); + if (ret < 0) + return ret; + } + ret = ffurl_write(whip->udp, whip->buf, cipher_size); if (ret < 0) { av_log(whip, AV_LOG_ERROR, "Failed to write packet=%dB, ret=%d\n", cipher_size, ret); @@ -1997,6 +2060,8 @@ static av_cold void whip_deinit(AVFormatContext *s) ff_srtp_free(&whip->srtp_recv); ffurl_close(whip->dtls_uc); ffurl_closep(&whip->udp); + av_freep(&whip->hist_pool); + av_freep(&whip->hist); } static int whip_check_bitstream(AVFormatContext *s, AVStream *st, const AVPacket *pkt) @@ -2024,6 +2089,7 @@ static const AVOption options[] = { { "handshake_timeout", "Timeout in milliseconds for ICE and DTLS handshake.", OFFSET(handshake_timeout), AV_OPT_TYPE_INT, { .i64 = 5000 }, -1, INT_MAX, ENC }, { "pkt_size", "The maximum size, in bytes, of RTP packets that send out", OFFSET(pkt_size), AV_OPT_TYPE_INT, { .i64 = 1200 }, -1, INT_MAX, ENC }, { "buffer_size", "The buffer size, in bytes, of underlying protocol", OFFSET(buffer_size), AV_OPT_TYPE_INT, { .i64 = -1 }, -1, INT_MAX, ENC }, + { "rtp_history", "The number of RTP history items to store", OFFSET(hist_sz), AV_OPT_TYPE_INT, { .i64 = WHIP_RTP_HISTORY_DEFAULT }, WHIP_RTP_HISTORY_MIN, WHIP_RTP_HISTORY_MAX, ENC }, { "authorization", "The optional Bearer token for WHIP Authorization", OFFSET(authorization), AV_OPT_TYPE_STRING, { .str = NULL }, 0, 0, ENC }, { "cert_file", "The optional certificate file path for DTLS", OFFSET(cert_file), AV_OPT_TYPE_STRING, { .str = NULL }, 0, 0, ENC }, { "key_file", "The optional private key file path for DTLS", OFFSET(key_file), AV_OPT_TYPE_STRING, { .str = NULL }, 0, 0, ENC }, -- 2.49.1 >>From dc5fc6315374ca8d1d113a6af4448f74fb9b55c8 Mon Sep 17 00:00:00 2001 From: Jack Lau Date: Tue, 14 Oct 2025 17:24:10 +0800 Subject: [PATCH 2/2] avformat/whip: reindent the options Signed-off-by: Jack Lau --- libavformat/whip.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/libavformat/whip.c b/libavformat/whip.c index 0f00a6652b..8000bad3a4 100644 --- a/libavformat/whip.c +++ b/libavformat/whip.c @@ -2092,7 +2092,7 @@ static const AVOption options[] = { { "rtp_history", "The number of RTP history items to store", OFFSET(hist_sz), AV_OPT_TYPE_INT, { .i64 = WHIP_RTP_HISTORY_DEFAULT }, WHIP_RTP_HISTORY_MIN, WHIP_RTP_HISTORY_MAX, ENC }, { "authorization", "The optional Bearer token for WHIP Authorization", OFFSET(authorization), AV_OPT_TYPE_STRING, { .str = NULL }, 0, 0, ENC }, { "cert_file", "The optional certificate file path for DTLS", OFFSET(cert_file), AV_OPT_TYPE_STRING, { .str = NULL }, 0, 0, ENC }, - { "key_file", "The optional private key file path for DTLS", OFFSET(key_file), AV_OPT_TYPE_STRING, { .str = NULL }, 0, 0, ENC }, + { "key_file", "The optional private key file path for DTLS", OFFSET(key_file), AV_OPT_TYPE_STRING, { .str = NULL }, 0, 0, ENC }, { NULL }, }; -- 2.49.1 _______________________________________________ ffmpeg-devel mailing list -- ffmpeg-devel@ffmpeg.org To unsubscribe send an email to ffmpeg-devel-leave@ffmpeg.org