* [FFmpeg-devel] [PATCH v5 01/15] avformat/whip: add whip_flags ignore_ipv6 to skip IPv6 ICE candidates
2025-07-22 12:36 [FFmpeg-devel] [PATCH v5 00/15] avformat/whip: Add NACK, RTX, DTLS active support Jack Lau
@ 2025-07-22 12:36 ` Jack Lau
2025-07-22 12:36 ` [FFmpeg-devel] [PATCH v5 02/15] avformat/whip: fix typos Jack Lau
` (13 subsequent siblings)
14 siblings, 0 replies; 17+ messages in thread
From: Jack Lau @ 2025-07-22 12:36 UTC (permalink / raw)
To: ffmpeg-devel; +Cc: Jack Lau
mark this ignore_ipv6 flag could ignore any IPv6 ICE candidate,
preventing “No route to host” errors on devices without IPv6 connectivity.
Signed-off-by: Jack Lau <jacklau1222@qq.com>
---
libavformat/whip.c | 17 ++++++++++++++++-
1 file changed, 16 insertions(+), 1 deletion(-)
diff --git a/libavformat/whip.c b/libavformat/whip.c
index fd6de8503f..83c70b654d 100644
--- a/libavformat/whip.c
+++ b/libavformat/whip.c
@@ -193,9 +193,14 @@ enum WHIPState {
WHIP_STATE_FAILED,
};
+typedef enum WHIPFlags {
+ WHIP_FLAG_IGNORE_IPV6 = (1 << 0) // Ignore ipv6 candidate
+} WHIPFlags;
+
typedef struct WHIPContext {
AVClass *av_class;
+ uint32_t flags; // enum WHIPFlags
/* The state of the RTC connection. */
enum WHIPState state;
/* The callback return value for DTLS. */
@@ -884,6 +889,9 @@ static int parse_answer(AVFormatContext *s)
if (ptr && av_stristr(ptr, "host")) {
char protocol[17], host[129];
int priority, port;
+#if HAVE_STRUCT_SOCKADDR_IN6
+ struct in6_addr addr6;
+#endif
ret = sscanf(ptr, "%16s %d %128s %d typ host", protocol, &priority, host, &port);
if (ret != 4) {
av_log(whip, AV_LOG_ERROR, "Failed %d to parse line %d %s from %s\n",
@@ -891,7 +899,12 @@ static int parse_answer(AVFormatContext *s)
ret = AVERROR(EIO);
goto end;
}
-
+#if HAVE_STRUCT_SOCKADDR_IN6
+ if (whip->flags & WHIP_FLAG_IGNORE_IPV6 && inet_pton(AF_INET6, host, &addr6) == 1) {
+ av_log(whip, AV_LOG_DEBUG, "Ignoring IPv6 ICE candidates %s, line %d %s \n", host, i, line);
+ continue;
+ }
+#endif
if (av_strcasecmp(protocol, "udp")) {
av_log(whip, AV_LOG_ERROR, "Protocol %s is not supported by RTC, choose udp, line %d %s of %s\n",
protocol, i, line, whip->sdp_answer);
@@ -1901,6 +1914,8 @@ static const AVOption options[] = {
{ "authorization", "The optional Bearer token for WHIP Authorization", OFFSET(authorization), AV_OPT_TYPE_STRING, { .str = NULL }, 0, 0, ENC },
{ "cert_file", "The optional certificate file path for DTLS", OFFSET(cert_file), AV_OPT_TYPE_STRING, { .str = NULL }, 0, 0, ENC },
{ "key_file", "The optional private key file path for DTLS", OFFSET(key_file), AV_OPT_TYPE_STRING, { .str = NULL }, 0, 0, ENC },
+ { "whip_flags", "Set flags affecting WHIP connection behavior", OFFSET(flags), AV_OPT_TYPE_FLAGS, { .i64 = 0 }, 0, UINT_MAX, ENC, .unit = "flags" },
+ { "ignore_ipv6", "Ignore any IPv6 ICE candidate", 0, AV_OPT_TYPE_CONST, { .i64 = WHIP_FLAG_IGNORE_IPV6 }, 0, UINT_MAX, ENC, .unit = "flags" },
{ NULL },
};
--
2.49.0
_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".
^ permalink raw reply [flat|nested] 17+ messages in thread* [FFmpeg-devel] [PATCH v5 02/15] avformat/whip: fix typos
2025-07-22 12:36 [FFmpeg-devel] [PATCH v5 00/15] avformat/whip: Add NACK, RTX, DTLS active support Jack Lau
2025-07-22 12:36 ` [FFmpeg-devel] [PATCH v5 01/15] avformat/whip: add whip_flags ignore_ipv6 to skip IPv6 ICE candidates Jack Lau
@ 2025-07-22 12:36 ` Jack Lau
2025-07-22 12:36 ` [FFmpeg-devel] [PATCH v5 03/15] avformat/whip: fix H264 profile_iop bit map for SDP Jack Lau
` (12 subsequent siblings)
14 siblings, 0 replies; 17+ messages in thread
From: Jack Lau @ 2025-07-22 12:36 UTC (permalink / raw)
To: ffmpeg-devel; +Cc: Jack Lau
Remove redundant "WHIP: " prefix in log context
since it already add whip context.
Fix grammers in whip options descriptions
Signed-off-by: Jack Lau <jacklau1222@qq.com>
---
libavformat/whip.c | 6 +++---
1 file changed, 3 insertions(+), 3 deletions(-)
diff --git a/libavformat/whip.c b/libavformat/whip.c
index 83c70b654d..3ec4c76b5f 100644
--- a/libavformat/whip.c
+++ b/libavformat/whip.c
@@ -1911,9 +1911,9 @@ static int whip_check_bitstream(AVFormatContext *s, AVStream *st, const AVPacket
static const AVOption options[] = {
{ "handshake_timeout", "Timeout in milliseconds for ICE and DTLS handshake.", OFFSET(handshake_timeout), AV_OPT_TYPE_INT, { .i64 = 5000 }, -1, INT_MAX, ENC },
{ "pkt_size", "The maximum size, in bytes, of RTP packets that send out", OFFSET(pkt_size), AV_OPT_TYPE_INT, { .i64 = 1200 }, -1, INT_MAX, ENC },
- { "authorization", "The optional Bearer token for WHIP Authorization", OFFSET(authorization), AV_OPT_TYPE_STRING, { .str = NULL }, 0, 0, ENC },
- { "cert_file", "The optional certificate file path for DTLS", OFFSET(cert_file), AV_OPT_TYPE_STRING, { .str = NULL }, 0, 0, ENC },
- { "key_file", "The optional private key file path for DTLS", OFFSET(key_file), AV_OPT_TYPE_STRING, { .str = NULL }, 0, 0, ENC },
+ { "authorization", "Optional Bearer token for WHIP Authorization", OFFSET(authorization), AV_OPT_TYPE_STRING, { .str = NULL }, 0, 0, ENC },
+ { "cert_file", "Optional certificate file path for DTLS", OFFSET(cert_file), AV_OPT_TYPE_STRING, { .str = NULL }, 0, 0, ENC },
+ { "key_file", "Optional private key file path for DTLS", OFFSET(key_file), AV_OPT_TYPE_STRING, { .str = NULL }, 0, 0, ENC },
{ "whip_flags", "Set flags affecting WHIP connection behavior", OFFSET(flags), AV_OPT_TYPE_FLAGS, { .i64 = 0 }, 0, UINT_MAX, ENC, .unit = "flags" },
{ "ignore_ipv6", "Ignore any IPv6 ICE candidate", 0, AV_OPT_TYPE_CONST, { .i64 = WHIP_FLAG_IGNORE_IPV6 }, 0, UINT_MAX, ENC, .unit = "flags" },
{ NULL },
--
2.49.0
_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".
^ permalink raw reply [flat|nested] 17+ messages in thread* [FFmpeg-devel] [PATCH v5 03/15] avformat/whip: fix H264 profile_iop bit map for SDP
2025-07-22 12:36 [FFmpeg-devel] [PATCH v5 00/15] avformat/whip: Add NACK, RTX, DTLS active support Jack Lau
2025-07-22 12:36 ` [FFmpeg-devel] [PATCH v5 01/15] avformat/whip: add whip_flags ignore_ipv6 to skip IPv6 ICE candidates Jack Lau
2025-07-22 12:36 ` [FFmpeg-devel] [PATCH v5 02/15] avformat/whip: fix typos Jack Lau
@ 2025-07-22 12:36 ` Jack Lau
2025-07-22 12:36 ` [FFmpeg-devel] [PATCH v5 04/15] WHIP: X509 cert serial number should be positive Jack Lau
` (11 subsequent siblings)
14 siblings, 0 replies; 17+ messages in thread
From: Jack Lau @ 2025-07-22 12:36 UTC (permalink / raw)
To: ffmpeg-devel; +Cc: Jack Lau
AVCodecParameters::profile only contains constraint_set1_flag
(AV_PROFILE_H264_CONSTRAINED 1<<9).
So add H264 constraints flag fully parse refer to hlsenc
write_codec_attr
Signed-off-by: Jack Lau <jacklau1222@qq.com>
---
libavformat/whip.c | 47 ++++++++++++++++------------------------------
1 file changed, 16 insertions(+), 31 deletions(-)
diff --git a/libavformat/whip.c b/libavformat/whip.c
index 3ec4c76b5f..4de8eb2601 100644
--- a/libavformat/whip.c
+++ b/libavformat/whip.c
@@ -210,6 +210,7 @@ typedef struct WHIPContext {
/* Parameters for the input audio and video codecs. */
AVCodecParameters *audio_par;
AVCodecParameters *video_par;
+ uint8_t constraint_set_flags;
/**
* The h264_mp4toannexb Bitstream Filter (BSF) bypasses the AnnexB packet;
@@ -450,45 +451,30 @@ static av_cold int initialize(AVFormatContext *s)
static int parse_profile_level(AVFormatContext *s, AVCodecParameters *par)
{
int ret = 0;
- const uint8_t *r = par->extradata, *r1, *end = par->extradata + par->extradata_size;
- H264SPS seq, *const sps = &seq;
- uint32_t state;
+ const uint8_t *r = par->extradata;
WHIPContext *whip = s->priv_data;
if (par->codec_id != AV_CODEC_ID_H264)
return ret;
- if (par->profile != AV_PROFILE_UNKNOWN && par->level != AV_LEVEL_UNKNOWN)
- return ret;
-
if (!par->extradata || par->extradata_size <= 0) {
av_log(whip, AV_LOG_ERROR, "Unable to parse profile from empty extradata=%p, size=%d\n",
par->extradata, par->extradata_size);
return AVERROR(EINVAL);
}
- while (1) {
- r = avpriv_find_start_code(r, end, &state);
- if (r >= end)
- break;
-
- r1 = ff_nal_find_startcode(r, end);
- if ((state & 0x1f) == H264_NAL_SPS) {
- ret = ff_avc_decode_sps(sps, r, r1 - r);
- if (ret < 0) {
- av_log(whip, AV_LOG_ERROR, "Failed to decode SPS, state=%x, size=%d\n",
- state, (int)(r1 - r));
- return ret;
- }
-
- av_log(whip, AV_LOG_VERBOSE, "Parse profile=%d, level=%d from SPS\n",
- sps->profile_idc, sps->level_idc);
- par->profile = sps->profile_idc;
- par->level = sps->level_idc;
- }
+ if (AV_RB32(r) == 0x00000001 && (r[4] & 0x1F) == 7)
+ r = &r[5];
+ else if (AV_RB24(r) == 0x000001 && (r[3] & 0x1F) == 7)
+ r = &r[4];
+ else if (r[0] == 0x01) // avcC
+ r = &r[1];
+ else
+ return AVERROR(EINVAL);
- r = r1;
- }
+ if (par->profile == AV_PROFILE_UNKNOWN) par->profile = r[0];
+ whip->constraint_set_flags = r[1];
+ if (par->level == AV_LEVEL_UNKNOWN) par->level = r[2];
return ret;
}
@@ -599,7 +585,7 @@ static int parse_codec(AVFormatContext *s)
*/
static int generate_sdp_offer(AVFormatContext *s)
{
- int ret = 0, profile, level, profile_iop;
+ int ret = 0, profile, level;
const char *acodec_name = NULL, *vcodec_name = NULL;
AVBPrint bp;
WHIPContext *whip = s->priv_data;
@@ -667,11 +653,10 @@ static int generate_sdp_offer(AVFormatContext *s)
}
if (whip->video_par) {
- profile_iop = profile = whip->video_par->profile;
+ profile = whip->video_par->profile;
level = whip->video_par->level;
if (whip->video_par->codec_id == AV_CODEC_ID_H264) {
vcodec_name = "H264";
- profile_iop &= AV_PROFILE_H264_CONSTRAINED;
profile &= (~AV_PROFILE_H264_CONSTRAINED);
}
@@ -699,7 +684,7 @@ static int generate_sdp_offer(AVFormatContext *s)
vcodec_name,
whip->video_payload_type,
profile,
- profile_iop,
+ whip->constraint_set_flags,
level,
whip->video_ssrc,
whip->video_ssrc);
--
2.49.0
_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".
^ permalink raw reply [flat|nested] 17+ messages in thread* [FFmpeg-devel] [PATCH v5 04/15] WHIP: X509 cert serial number should be positive.
2025-07-22 12:36 [FFmpeg-devel] [PATCH v5 00/15] avformat/whip: Add NACK, RTX, DTLS active support Jack Lau
` (2 preceding siblings ...)
2025-07-22 12:36 ` [FFmpeg-devel] [PATCH v5 03/15] avformat/whip: fix H264 profile_iop bit map for SDP Jack Lau
@ 2025-07-22 12:36 ` Jack Lau
2025-07-22 12:36 ` [FFmpeg-devel] [PATCH v5 05/15] avformat/whip: implement NACK and RTX suppport Jack Lau
` (10 subsequent siblings)
14 siblings, 0 replies; 17+ messages in thread
From: Jack Lau @ 2025-07-22 12:36 UTC (permalink / raw)
To: ffmpeg-devel; +Cc: winlin, Jack Lau
From: winlin <winlinvip@gmail.com>
See RFC5280 4.1.2.2
Signed-off-by: Jack Lau <jacklau1222@qq.com>
---
libavformat/tls_openssl.c | 3 ++-
1 file changed, 2 insertions(+), 1 deletion(-)
diff --git a/libavformat/tls_openssl.c b/libavformat/tls_openssl.c
index 0a7998210f..2689aa5090 100644
--- a/libavformat/tls_openssl.c
+++ b/libavformat/tls_openssl.c
@@ -329,7 +329,8 @@ static int openssl_gen_certificate(EVP_PKEY *pkey, X509 **cert, char **fingerpri
goto enomem_end;
}
- serial = (int)av_get_random_seed();
+ // According to RFC5280 4.1.2.2, The serial number MUST be a positive integer
+ serial = (int)(av_get_random_seed() & 0x7FFFFFFF);
if (ASN1_INTEGER_set(X509_get_serialNumber(*cert), serial) != 1) {
av_log(NULL, AV_LOG_ERROR, "TLS: Failed to set serial, %s\n", ERR_error_string(ERR_get_error(), NULL));
goto einval_end;
--
2.49.0
_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".
^ permalink raw reply [flat|nested] 17+ messages in thread* [FFmpeg-devel] [PATCH v5 05/15] avformat/whip: implement NACK and RTX suppport
2025-07-22 12:36 [FFmpeg-devel] [PATCH v5 00/15] avformat/whip: Add NACK, RTX, DTLS active support Jack Lau
` (3 preceding siblings ...)
2025-07-22 12:36 ` [FFmpeg-devel] [PATCH v5 04/15] WHIP: X509 cert serial number should be positive Jack Lau
@ 2025-07-22 12:36 ` Jack Lau
2025-07-22 12:36 ` [FFmpeg-devel] [PATCH v5 06/15] avformat/whip: reindent whip options Jack Lau
` (9 subsequent siblings)
14 siblings, 0 replies; 17+ messages in thread
From: Jack Lau @ 2025-07-22 12:36 UTC (permalink / raw)
To: ffmpeg-devel; +Cc: Sergio Garcia Murillo, Jack Lau
RTP retransmission described in RFC4588 (RTX) is an effective packet
loss recovery technique for real-time applications with relaxed delay bounds.
This patch provides a minimal implementation for RTX and RTCP NACK (RFC3940)
and its associated SDP signaling and negotiation.
Co-authored-by: Sergio Garcia Murillo <sergio.garcia.murillo@gmail.com>
Signed-off-by: Jack Lau <jacklau1222@qq.com>
---
libavformat/whip.c | 206 ++++++++++++++++++++++++++++++++++++++++++++-
1 file changed, 202 insertions(+), 4 deletions(-)
diff --git a/libavformat/whip.c b/libavformat/whip.c
index 4de8eb2601..5907865d22 100644
--- a/libavformat/whip.c
+++ b/libavformat/whip.c
@@ -114,6 +114,7 @@
/* Referring to Chrome's definition of RTP payload types. */
#define WHIP_RTP_PAYLOAD_TYPE_H264 106
#define WHIP_RTP_PAYLOAD_TYPE_OPUS 111
+#define WHIP_RTP_PAYLOAD_TYPE_RTX 105
/**
* The STUN message header, which is 20 bytes long, comprises the
@@ -150,6 +151,11 @@
#define WHIP_SDP_SESSION_ID "4489045141692799359"
#define WHIP_SDP_CREATOR_IP "127.0.0.1"
+/**
+ * Retransmission / NACK support
+*/
+#define HISTORY_SIZE_DEFAULT 512
+
/* Calculate the elapsed time from starttime to endtime in milliseconds. */
#define ELAPSED(starttime, endtime) ((int)(endtime - starttime) / 1000)
@@ -194,9 +200,16 @@ enum WHIPState {
};
typedef enum WHIPFlags {
- WHIP_FLAG_IGNORE_IPV6 = (1 << 0) // Ignore ipv6 candidate
+ WHIP_FLAG_IGNORE_IPV6 = (1 << 0), // Ignore ipv6 candidate
+ WHIP_FLAG_DISABLE_RTX = (1 << 1) // Enable NACK and RTX
} WHIPFlags;
+typedef struct RtpHistoryItem {
+ uint16_t seq; // original RTP seq
+ int size; // length in bytes
+ uint8_t* buf; // malloc-ed copy
+} RtpHistoryItem;
+
typedef struct WHIPContext {
AVClass *av_class;
@@ -285,6 +298,7 @@ typedef struct WHIPContext {
/* The SRTP send context, to encrypt outgoing packets. */
SRTPContext srtp_audio_send;
SRTPContext srtp_video_send;
+ SRTPContext srtp_video_rtx_send;
SRTPContext srtp_rtcp_send;
/* The SRTP receive context, to decrypt incoming packets. */
SRTPContext srtp_recv;
@@ -309,6 +323,14 @@ typedef struct WHIPContext {
/* The certificate and private key used for DTLS handshake. */
char* cert_file;
char* key_file;
+
+ /* RTX and NACK */
+ uint8_t rtx_payload_type;
+ uint32_t video_rtx_ssrc;
+ uint16_t rtx_seq;
+ int history_size;
+ RtpHistoryItem *history; /* ring buffer */
+ int hist_head;
} WHIPContext;
/**
@@ -611,6 +633,16 @@ static int generate_sdp_offer(AVFormatContext *s)
whip->audio_payload_type = WHIP_RTP_PAYLOAD_TYPE_OPUS;
whip->video_payload_type = WHIP_RTP_PAYLOAD_TYPE_H264;
+ /* RTX and NACK init */
+ whip->rtx_payload_type = WHIP_RTP_PAYLOAD_TYPE_RTX;
+ whip->video_rtx_ssrc = av_lfg_get(&whip->rnd);
+ whip->rtx_seq = 0;
+ whip->hist_head = 0;
+ whip->history_size = FFMAX(64, whip->history_size);
+ whip->history = av_calloc(whip->history_size, sizeof(*whip->history));
+ if (!whip->history)
+ return AVERROR(ENOMEM);
+
av_bprintf(&bp, ""
"v=0\r\n"
"o=FFmpeg %s 2 IN IP4 %s\r\n"
@@ -661,7 +693,7 @@ static int generate_sdp_offer(AVFormatContext *s)
}
av_bprintf(&bp, ""
- "m=video 9 UDP/TLS/RTP/SAVPF %u\r\n"
+ "m=video 9 UDP/TLS/RTP/SAVPF %u %u\r\n"
"c=IN IP4 0.0.0.0\r\n"
"a=ice-ufrag:%s\r\n"
"a=ice-pwd:%s\r\n"
@@ -674,9 +706,16 @@ static int generate_sdp_offer(AVFormatContext *s)
"a=rtcp-rsize\r\n"
"a=rtpmap:%u %s/90000\r\n"
"a=fmtp:%u level-asymmetry-allowed=1;packetization-mode=1;profile-level-id=%02x%02x%02x\r\n"
+ "a=rtcp-fb:%u nack\r\n"
+ "a=rtpmap:%u rtx/90000\r\n"
+ "a=fmtp:%u apt=%u\r\n"
+ "a=ssrc-group:FID %u %u\r\n"
+ "a=ssrc:%u cname:FFmpeg\r\n"
+ "a=ssrc:%u msid:FFmpeg video\r\n"
"a=ssrc:%u cname:FFmpeg\r\n"
"a=ssrc:%u msid:FFmpeg video\r\n",
whip->video_payload_type,
+ whip->rtx_payload_type,
whip->ice_ufrag_local,
whip->ice_pwd_local,
whip->dtls_fingerprint,
@@ -686,8 +725,16 @@ static int generate_sdp_offer(AVFormatContext *s)
profile,
whip->constraint_set_flags,
level,
+ whip->video_payload_type,
+ whip->rtx_payload_type,
+ whip->rtx_payload_type,
+ whip->video_payload_type,
+ whip->video_ssrc,
+ whip->video_rtx_ssrc,
whip->video_ssrc,
- whip->video_ssrc);
+ whip->video_ssrc,
+ whip->video_rtx_ssrc,
+ whip->video_rtx_ssrc);
}
if (!av_bprint_is_complete(&bp)) {
@@ -1400,6 +1447,12 @@ static int setup_srtp(AVFormatContext *s)
goto end;
}
+ ret = ff_srtp_set_crypto(&whip->srtp_video_rtx_send, suite, buf);
+ if (ret < 0) {
+ av_log(whip, AV_LOG_ERROR, "Failed to set crypto for video rtx send\n");
+ goto end;
+ }
+
ret = ff_srtp_set_crypto(&whip->srtp_rtcp_send, suite, buf);
if (ret < 0) {
av_log(whip, AV_LOG_ERROR, "Failed to set crypto for rtcp send\n");
@@ -1429,6 +1482,38 @@ end:
return ret;
}
+
+/**
+ * RTX history helpers
+ */
+ static int rtp_history_store(WHIPContext *whip, const uint8_t *buf, int size)
+{
+ int pos = whip->hist_head % whip->history_size;
+ RtpHistoryItem *it = &whip->history[pos];
+ /* free older entry */
+ av_free(it->buf);
+ it->buf = av_malloc(size);
+ if (!it->buf)
+ return AVERROR(ENOMEM);
+
+ memcpy(it->buf, buf, size);
+ it->size = size;
+ it->seq = AV_RB16(buf + 2);
+
+ whip->hist_head = ++pos;
+ return 0;
+}
+
+static const RtpHistoryItem *rtp_history_find(const WHIPContext *whip, uint16_t seq)
+{
+ for (int i = 0; i < whip->history_size; i++) {
+ const RtpHistoryItem *it = &whip->history[i];
+ if (it->buf && it->seq == seq)
+ return it;
+ }
+ return NULL;
+}
+
/**
* Callback triggered by the RTP muxer when it creates and sends out an RTP packet.
*
@@ -1465,6 +1550,12 @@ static int on_rtp_write_packet(void *opaque, const uint8_t *buf, int buf_size)
return 0;
}
+ /* Store only ORIGINAL video packets (non-RTX, non-RTCP) */
+ if (!is_rtcp && is_video) {
+ ret = rtp_history_store(whip, buf, buf_size);
+ if (ret < 0) return ret;
+ }
+
ret = ffurl_write(whip->udp, whip->buf, cipher_size);
if (ret < 0) {
av_log(whip, AV_LOG_ERROR, "Failed to write packet=%dB, ret=%d\n", cipher_size, ret);
@@ -1473,6 +1564,48 @@ static int on_rtp_write_packet(void *opaque, const uint8_t *buf, int buf_size)
return ret;
}
+/**
+ * See https://datatracker.ietf.org/doc/html/rfc4588
+ * Build and send a single RTX packet
+ */
+static int send_rtx_packet(AVFormatContext *s, const uint8_t *orig_pkt_buf, int orig_size)
+{
+ int ret;
+ WHIPContext *whip = s->priv_data;
+ int new_size, cipher_size;
+ if (whip->flags & WHIP_FLAG_DISABLE_RTX)
+ return 0;
+
+ /* allocate new buffer: header + 2 + payload */
+ if (orig_size + 2 > sizeof(whip->buf))
+ return 0;
+
+ memcpy(whip->buf, orig_pkt_buf, orig_size);
+
+ uint8_t *hdr = whip->buf;
+ uint16_t orig_seq = AV_RB16(hdr + 2);
+
+ /* rewrite header */
+ hdr[1] = (hdr[1] & 0x80) | whip->rtx_payload_type; /* keep M bit */
+ AV_WB16(hdr + 2, whip->rtx_seq++);
+ AV_WB32(hdr + 8, whip->video_rtx_ssrc);
+
+ /* shift payload 2 bytes */
+ memmove(hdr + 12 + 2, hdr + 12, orig_size - 12);
+ AV_WB16(hdr + 12, orig_seq);
+
+ new_size = orig_size + 2;
+
+ /* Encrypt by SRTP and send out. */
+ cipher_size = ff_srtp_encrypt(&whip->srtp_video_rtx_send, whip->buf, new_size, whip->buf, sizeof(whip->buf));
+ if (cipher_size <= 0 || cipher_size < new_size) {
+ av_log(whip, AV_LOG_WARNING, "Failed to encrypt packet=%dB, cipher=%dB\n", new_size, cipher_size);
+ return 0;
+ }
+ ret = ffurl_write(whip->udp, whip->buf, cipher_size);
+ if (ret <= 0) av_log(whip, AV_LOG_ERROR, "Failed to send RTX packet\n");
+ return ret;
+}
/**
* Creates dedicated RTP muxers for each stream in the AVFormatContext to build RTP
@@ -1781,6 +1914,7 @@ static int whip_write_packet(AVFormatContext *s, AVPacket *pkt)
WHIPContext *whip = s->priv_data;
AVStream *st = s->streams[pkt->stream_index];
AVFormatContext *rtp_ctx = st->priv_data;
+ uint8_t *buf = NULL;
/* TODO: Send binding request every 1s as WebRTC heartbeat. */
@@ -1796,11 +1930,72 @@ static int whip_write_packet(AVFormatContext *s, AVPacket *pkt)
goto end;
}
}
+ /**
+ * Handle RTCP NACK
+ * Refer to RFC 4585, Section 6.2.1
+ * The Generic NACK message is identified by PT=RTPFB and FMT=1.
+ * TODO: disable retransmisstion when "-tune zerolatency"
+ */
+ if (media_is_rtcp(whip->buf, ret)) {
+ int ptr = 0;
+ uint8_t pt = whip->buf[ptr + 1];
+ uint8_t fmt = (whip->buf[ptr] & 0x1f);
+ if (ptr + 4 <= ret && pt == 205 && fmt == 1) {
+ /**
+ * Refer to RFC 3550, Section 6.4.1.
+ * The length of this RTCP packet in 32-bit words minus one,
+ * including the header and any padding.
+ */
+ int rtcp_len = (AV_RB16(&whip->buf[ptr + 2]) + 1) * 4;
+ /* SRTCP index(4 bytes) + HMAC (SRTP_AES128_CM_SHA1_80 10bytes) */
+ int srtcp_len = rtcp_len + 4 + 10;
+ if (srtcp_len == ret && rtcp_len >= 12) {
+ int i = 0;
+ buf = av_malloc(srtcp_len);
+ if (!buf) return AVERROR(ENOMEM);
+ memcpy(buf, whip->buf, srtcp_len);
+ int ret = ff_srtp_decrypt(&whip->srtp_recv, buf, &srtcp_len);
+ if (ret < 0) {
+ av_log(whip, AV_LOG_ERROR, "NACK packet(SRTCP) decrypt failed: %d, Can't send RTX packet\n", ret);
+ goto write_packet;
+ }
+ while (12 + i < rtcp_len) {
+ /**
+ * See https://datatracker.ietf.org/doc/html/rfc4585#section-6.1
+ * Handle multi NACKs in bundled packet.
+ */
+ uint16_t pid = AV_RB16(&buf[ptr + 12 + i]);
+ uint16_t blp = AV_RB16(&buf[ptr + 14 + i]);
+
+ /* retransmit pid + any bit set in blp */
+ for (int bit = -1; bit < 16; bit++) {
+ uint16_t seq = (bit < 0) ? pid : pid + bit + 1;
+ if (bit >= 0 && !(blp & (1 << bit)))
+ continue;
+
+ const RtpHistoryItem *it = rtp_history_find(whip, seq);
+ if (it) {
+ av_log(whip, AV_LOG_VERBOSE,
+ "NACK, packet found: size: %d, seq=%d, rtx size=%d, lateset stored packet seq:%d\n",
+ it->size, seq, ret, whip->history[whip->hist_head-1].seq);
+ send_rtx_packet(s, it->buf, it->size);
+ } else {
+ av_log(whip, AV_LOG_VERBOSE,
+ "NACK, packet not found, seq=%d, latest stored packet seq: %d, latest rtx seq: %d\n",
+ seq, whip->history[whip->hist_head-1].seq, whip->rtx_seq);
+ }
+ }
+ i = i + 4;
+ }
+ av_freep(&buf);
+ }
+ }
+ }
} else if (ret != AVERROR(EAGAIN)) {
av_log(whip, AV_LOG_ERROR, "Failed to read from UDP socket\n");
goto end;
}
-
+write_packet:
if (whip->h264_annexb_insert_sps_pps && st->codecpar->codec_id == AV_CODEC_ID_H264) {
if ((ret = h264_annexb_insert_sps_pps(s, pkt)) < 0) {
av_log(whip, AV_LOG_ERROR, "Failed to insert SPS/PPS before IDR\n");
@@ -1819,6 +2014,7 @@ static int whip_write_packet(AVFormatContext *s, AVPacket *pkt)
}
end:
+ if (buf) av_freep(&buf);
if (ret < 0 && whip->state < WHIP_STATE_FAILED)
whip->state = WHIP_STATE_FAILED;
if (ret >= 0 && whip->state >= WHIP_STATE_FAILED && whip->dtls_ret < 0)
@@ -1901,6 +2097,8 @@ static const AVOption options[] = {
{ "key_file", "Optional private key file path for DTLS", OFFSET(key_file), AV_OPT_TYPE_STRING, { .str = NULL }, 0, 0, ENC },
{ "whip_flags", "Set flags affecting WHIP connection behavior", OFFSET(flags), AV_OPT_TYPE_FLAGS, { .i64 = 0 }, 0, UINT_MAX, ENC, .unit = "flags" },
{ "ignore_ipv6", "Ignore any IPv6 ICE candidate", 0, AV_OPT_TYPE_CONST, { .i64 = WHIP_FLAG_IGNORE_IPV6 }, 0, UINT_MAX, ENC, .unit = "flags" },
+ { "disable_rtx", "Disable RFC 4588 RTX", 0, AV_OPT_TYPE_CONST, { .i64 = WHIP_FLAG_DISABLE_RTX }, 0, UINT_MAX, ENC, .unit = "flags" },
+ { "rtx_history_size", "Packet history size", OFFSET(history_size), AV_OPT_TYPE_INT, { .i64 = HISTORY_SIZE_DEFAULT }, 64, 2048, ENC },
{ NULL },
};
--
2.49.0
_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".
^ permalink raw reply [flat|nested] 17+ messages in thread* [FFmpeg-devel] [PATCH v5 06/15] avformat/whip: reindent whip options
2025-07-22 12:36 [FFmpeg-devel] [PATCH v5 00/15] avformat/whip: Add NACK, RTX, DTLS active support Jack Lau
` (4 preceding siblings ...)
2025-07-22 12:36 ` [FFmpeg-devel] [PATCH v5 05/15] avformat/whip: implement NACK and RTX suppport Jack Lau
@ 2025-07-22 12:36 ` Jack Lau
2025-07-22 12:36 ` [FFmpeg-devel] [PATCH v5 07/15] avformat/whip: add support for active dtls role Jack Lau
` (8 subsequent siblings)
14 siblings, 0 replies; 17+ messages in thread
From: Jack Lau @ 2025-07-22 12:36 UTC (permalink / raw)
To: ffmpeg-devel; +Cc: Jack Lau
Signed-off-by: Jack Lau <jacklau1222@qq.com>
---
libavformat/whip.c | 14 +++++++-------
1 file changed, 7 insertions(+), 7 deletions(-)
diff --git a/libavformat/whip.c b/libavformat/whip.c
index 5907865d22..094d3a0a4c 100644
--- a/libavformat/whip.c
+++ b/libavformat/whip.c
@@ -2090,13 +2090,13 @@ static int whip_check_bitstream(AVFormatContext *s, AVStream *st, const AVPacket
#define OFFSET(x) offsetof(WHIPContext, x)
#define ENC AV_OPT_FLAG_ENCODING_PARAM
static const AVOption options[] = {
- { "handshake_timeout", "Timeout in milliseconds for ICE and DTLS handshake.", OFFSET(handshake_timeout), AV_OPT_TYPE_INT, { .i64 = 5000 }, -1, INT_MAX, ENC },
- { "pkt_size", "The maximum size, in bytes, of RTP packets that send out", OFFSET(pkt_size), AV_OPT_TYPE_INT, { .i64 = 1200 }, -1, INT_MAX, ENC },
- { "authorization", "Optional Bearer token for WHIP Authorization", OFFSET(authorization), AV_OPT_TYPE_STRING, { .str = NULL }, 0, 0, ENC },
- { "cert_file", "Optional certificate file path for DTLS", OFFSET(cert_file), AV_OPT_TYPE_STRING, { .str = NULL }, 0, 0, ENC },
- { "key_file", "Optional private key file path for DTLS", OFFSET(key_file), AV_OPT_TYPE_STRING, { .str = NULL }, 0, 0, ENC },
- { "whip_flags", "Set flags affecting WHIP connection behavior", OFFSET(flags), AV_OPT_TYPE_FLAGS, { .i64 = 0 }, 0, UINT_MAX, ENC, .unit = "flags" },
- { "ignore_ipv6", "Ignore any IPv6 ICE candidate", 0, AV_OPT_TYPE_CONST, { .i64 = WHIP_FLAG_IGNORE_IPV6 }, 0, UINT_MAX, ENC, .unit = "flags" },
+ { "handshake_timeout", "Timeout in milliseconds for ICE and DTLS handshake.", OFFSET(handshake_timeout), AV_OPT_TYPE_INT, { .i64 = 5000 }, -1, INT_MAX, ENC },
+ { "pkt_size", "The maximum size, in bytes, of RTP packets that send out", OFFSET(pkt_size), AV_OPT_TYPE_INT, { .i64 = 1200 }, -1, INT_MAX, ENC },
+ { "authorization", "Optional Bearer token for WHIP Authorization", OFFSET(authorization), AV_OPT_TYPE_STRING, { .str = NULL }, 0, 0, ENC },
+ { "cert_file", "Optional certificate file path for DTLS", OFFSET(cert_file), AV_OPT_TYPE_STRING, { .str = NULL }, 0, 0, ENC },
+ { "key_file", "Optional private key file path for DTLS", OFFSET(key_file), AV_OPT_TYPE_STRING, { .str = NULL }, 0, 0, ENC },
+ { "whip_flags", "Set flags affecting WHIP connection behavior", OFFSET(flags), AV_OPT_TYPE_FLAGS, { .i64 = 0 }, 0, 0, ENC, .unit = "flags" },
+ { "ignore_ipv6", "Ignore any IPv6 ICE candidate", 0, AV_OPT_TYPE_CONST, { .i64 = WHIP_FLAG_IGNORE_IPV6 }, 0, UINT_MAX, ENC, .unit = "flags" },
{ "disable_rtx", "Disable RFC 4588 RTX", 0, AV_OPT_TYPE_CONST, { .i64 = WHIP_FLAG_DISABLE_RTX }, 0, UINT_MAX, ENC, .unit = "flags" },
{ "rtx_history_size", "Packet history size", OFFSET(history_size), AV_OPT_TYPE_INT, { .i64 = HISTORY_SIZE_DEFAULT }, 64, 2048, ENC },
{ NULL },
--
2.49.0
_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".
^ permalink raw reply [flat|nested] 17+ messages in thread* [FFmpeg-devel] [PATCH v5 07/15] avformat/whip: add support for active dtls role
2025-07-22 12:36 [FFmpeg-devel] [PATCH v5 00/15] avformat/whip: Add NACK, RTX, DTLS active support Jack Lau
` (5 preceding siblings ...)
2025-07-22 12:36 ` [FFmpeg-devel] [PATCH v5 06/15] avformat/whip: reindent whip options Jack Lau
@ 2025-07-22 12:36 ` Jack Lau
2025-07-22 12:36 ` [FFmpeg-devel] [PATCH v5 08/15] avformat/whip: remove DTLSState enum Jack Lau
` (7 subsequent siblings)
14 siblings, 0 replies; 17+ messages in thread
From: Jack Lau @ 2025-07-22 12:36 UTC (permalink / raw)
To: ffmpeg-devel; +Cc: Jack Lau
add dtls_active flag to specify the dtls role
properly set the send key and recv key depends on DTLS role
As DTLS server, the recv key is client master key plus salt,
the send key is server master key plus salt.
As DTLS client, the recv key is server master key plus salt,
the send key is client master key plus salt.
Signed-off-by: Jack Lau <jacklau1222@qq.com>
---
libavformat/whip.c | 32 ++++++++++++++++++++------------
1 file changed, 20 insertions(+), 12 deletions(-)
diff --git a/libavformat/whip.c b/libavformat/whip.c
index 094d3a0a4c..e02ed7a8a4 100644
--- a/libavformat/whip.c
+++ b/libavformat/whip.c
@@ -201,7 +201,8 @@ enum WHIPState {
typedef enum WHIPFlags {
WHIP_FLAG_IGNORE_IPV6 = (1 << 0), // Ignore ipv6 candidate
- WHIP_FLAG_DISABLE_RTX = (1 << 1) // Enable NACK and RTX
+ WHIP_FLAG_DISABLE_RTX = (1 << 1), // Enable NACK and RTX
+ WHIP_FLAG_DTLS_ACTIVE = (1 << 2), // DTLS active role
} WHIPFlags;
typedef struct RtpHistoryItem {
@@ -611,6 +612,7 @@ static int generate_sdp_offer(AVFormatContext *s)
const char *acodec_name = NULL, *vcodec_name = NULL;
AVBPrint bp;
WHIPContext *whip = s->priv_data;
+ int is_dtls_active = whip->flags & WHIP_FLAG_DTLS_ACTIVE;
/* To prevent a crash during cleanup, always initialize it. */
av_bprint_init(&bp, 1, MAX_SDP_SIZE);
@@ -664,7 +666,7 @@ static int generate_sdp_offer(AVFormatContext *s)
"a=ice-ufrag:%s\r\n"
"a=ice-pwd:%s\r\n"
"a=fingerprint:sha-256 %s\r\n"
- "a=setup:passive\r\n"
+ "a=setup:%s\r\n"
"a=mid:0\r\n"
"a=sendonly\r\n"
"a=msid:FFmpeg audio\r\n"
@@ -676,6 +678,7 @@ static int generate_sdp_offer(AVFormatContext *s)
whip->ice_ufrag_local,
whip->ice_pwd_local,
whip->dtls_fingerprint,
+ is_dtls_active ? "active" : "passive",
whip->audio_payload_type,
acodec_name,
whip->audio_par->sample_rate,
@@ -698,7 +701,7 @@ static int generate_sdp_offer(AVFormatContext *s)
"a=ice-ufrag:%s\r\n"
"a=ice-pwd:%s\r\n"
"a=fingerprint:sha-256 %s\r\n"
- "a=setup:passive\r\n"
+ "a=setup:%s\r\n"
"a=mid:1\r\n"
"a=sendonly\r\n"
"a=msid:FFmpeg video\r\n"
@@ -719,6 +722,7 @@ static int generate_sdp_offer(AVFormatContext *s)
whip->ice_ufrag_local,
whip->ice_pwd_local,
whip->dtls_fingerprint,
+ is_dtls_active ? "active" : "passive",
whip->video_payload_type,
vcodec_name,
whip->video_payload_type,
@@ -1270,6 +1274,7 @@ static int ice_dtls_handshake(AVFormatContext *s)
int ret = 0, size, i;
int64_t starttime = av_gettime(), now;
WHIPContext *whip = s->priv_data;
+ int is_dtls_active = whip->flags & WHIP_FLAG_DTLS_ACTIVE;
AVDictionary *opts = NULL;
char buf[256], *cert_buf = NULL, *key_buf = NULL;
@@ -1319,12 +1324,14 @@ next_packet:
av_usleep(5 * 1000);
continue;
}
+ if (is_dtls_active)
+ break;
av_log(whip, AV_LOG_ERROR, "Failed to read message\n");
goto end;
}
/* Got nothing, continue to process handshake. */
- if (ret <= 0 && whip->state < WHIP_STATE_DTLS_CONNECTING)
+ if (ret <= 0 && (is_dtls_active ? whip->state < WHIP_STATE_ICE_CONNECTED : whip->state < WHIP_STATE_DTLS_CONNECTING))
continue;
/* Handle the ICE binding response. */
@@ -1348,7 +1355,7 @@ next_packet:
} else
av_dict_set(&opts, "key_pem", whip->key_buf, 0);
av_dict_set_int(&opts, "external_sock", 1, 0);
- av_dict_set_int(&opts, "listen", 1, 0);
+ av_dict_set_int(&opts, "listen", is_dtls_active ? 0 : 1, 0);
/* If got the first binding response, start DTLS handshake. */
ret = ffurl_open_whitelist(&whip->dtls_uc, buf, AVIO_FLAG_READ_WRITE, &s->interrupt_callback,
&opts, s->protocol_whitelist, s->protocol_blacklist, NULL);
@@ -1368,7 +1375,7 @@ next_packet:
}
/* If got any DTLS messages, handle it. */
- if (is_dtls_packet(whip->buf, ret) && whip->state >= WHIP_STATE_ICE_CONNECTED || whip->state == WHIP_STATE_DTLS_CONNECTING) {
+ if ((is_dtls_packet(whip->buf, ret) || is_dtls_active) && whip->state >= WHIP_STATE_ICE_CONNECTED || whip->state == WHIP_STATE_DTLS_CONNECTING) {
whip->state = WHIP_STATE_DTLS_CONNECTING;
if ((ret = ffurl_handshake(whip->dtls_uc)) < 0)
goto end;
@@ -1406,6 +1413,8 @@ static int setup_srtp(AVFormatContext *s)
*/
const char* suite = "SRTP_AES128_CM_HMAC_SHA1_80";
WHIPContext *whip = s->priv_data;
+ int is_dtls_active = whip->flags & WHIP_FLAG_DTLS_ACTIVE;
+
ret = ff_dtls_export_materials(whip->dtls_uc, whip->dtls_srtp_materials, sizeof(whip->dtls_srtp_materials));
if (ret < 0)
goto end;
@@ -1420,13 +1429,11 @@ static int setup_srtp(AVFormatContext *s)
char *client_salt = server_key + DTLS_SRTP_KEY_LEN;
char *server_salt = client_salt + DTLS_SRTP_SALT_LEN;
- /* As DTLS server, the recv key is client master key plus salt. */
- memcpy(recv_key, client_key, DTLS_SRTP_KEY_LEN);
- memcpy(recv_key + DTLS_SRTP_KEY_LEN, client_salt, DTLS_SRTP_SALT_LEN);
+ memcpy(is_dtls_active ? send_key : recv_key, client_key, DTLS_SRTP_KEY_LEN);
+ memcpy(is_dtls_active ? send_key + DTLS_SRTP_KEY_LEN : recv_key + DTLS_SRTP_KEY_LEN, client_salt, DTLS_SRTP_SALT_LEN);
- /* As DTLS server, the send key is server master key plus salt. */
- memcpy(send_key, server_key, DTLS_SRTP_KEY_LEN);
- memcpy(send_key + DTLS_SRTP_KEY_LEN, server_salt, DTLS_SRTP_SALT_LEN);
+ memcpy(is_dtls_active ? recv_key : send_key, server_key, DTLS_SRTP_KEY_LEN);
+ memcpy(is_dtls_active ? recv_key + DTLS_SRTP_KEY_LEN : send_key + DTLS_SRTP_KEY_LEN, server_salt, DTLS_SRTP_SALT_LEN);
/* Setup SRTP context for outgoing packets */
if (!av_base64_encode(buf, sizeof(buf), send_key, sizeof(send_key))) {
@@ -2098,6 +2105,7 @@ static const AVOption options[] = {
{ "whip_flags", "Set flags affecting WHIP connection behavior", OFFSET(flags), AV_OPT_TYPE_FLAGS, { .i64 = 0 }, 0, 0, ENC, .unit = "flags" },
{ "ignore_ipv6", "Ignore any IPv6 ICE candidate", 0, AV_OPT_TYPE_CONST, { .i64 = WHIP_FLAG_IGNORE_IPV6 }, 0, UINT_MAX, ENC, .unit = "flags" },
{ "disable_rtx", "Disable RFC 4588 RTX", 0, AV_OPT_TYPE_CONST, { .i64 = WHIP_FLAG_DISABLE_RTX }, 0, UINT_MAX, ENC, .unit = "flags" },
+ { "dtls_active", "Set dtls role as active", 0, AV_OPT_TYPE_CONST, { .i64 = WHIP_FLAG_DTLS_ACTIVE }, 0, UINT_MAX, ENC, .unit = "flags" },
{ "rtx_history_size", "Packet history size", OFFSET(history_size), AV_OPT_TYPE_INT, { .i64 = HISTORY_SIZE_DEFAULT }, 64, 2048, ENC },
{ NULL },
};
--
2.49.0
_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".
^ permalink raw reply [flat|nested] 17+ messages in thread* [FFmpeg-devel] [PATCH v5 08/15] avformat/whip: remove DTLSState enum
2025-07-22 12:36 [FFmpeg-devel] [PATCH v5 00/15] avformat/whip: Add NACK, RTX, DTLS active support Jack Lau
` (6 preceding siblings ...)
2025-07-22 12:36 ` [FFmpeg-devel] [PATCH v5 07/15] avformat/whip: add support for active dtls role Jack Lau
@ 2025-07-22 12:36 ` Jack Lau
2025-07-22 12:36 ` [FFmpeg-devel] [PATCH v5 09/15] avformat/whip: check the peer whether is ice lite Jack Lau
` (6 subsequent siblings)
14 siblings, 0 replies; 17+ messages in thread
From: Jack Lau @ 2025-07-22 12:36 UTC (permalink / raw)
To: ffmpeg-devel; +Cc: Jack Lau
This patch aims to simplify the dtls handshake process
since dtls handshake use force block mode
We can just use the return code instead of DTLSState enum
Signed-off-by: Jack Lau <jacklau1222@qq.com>
---
libavformat/tls.h | 15 ----------
libavformat/tls_openssl.c | 7 -----
libavformat/tls_schannel.c | 7 -----
libavformat/whip.c | 59 ++++++++------------------------------
4 files changed, 12 insertions(+), 76 deletions(-)
diff --git a/libavformat/tls.h b/libavformat/tls.h
index 0c02a4ab27..157c0d0256 100644
--- a/libavformat/tls.h
+++ b/libavformat/tls.h
@@ -33,17 +33,6 @@
*/
#define MAX_CERTIFICATE_SIZE 8192
-enum DTLSState {
- DTLS_STATE_NONE,
-
- /* Whether DTLS handshake is finished. */
- DTLS_STATE_FINISHED,
- /* Whether DTLS session is closed. */
- DTLS_STATE_CLOSED,
- /* Whether DTLS handshake is failed. */
- DTLS_STATE_FAILED,
-};
-
typedef struct TLSShared {
char *ca_file;
int verify;
@@ -63,8 +52,6 @@ typedef struct TLSShared {
int is_dtls;
- enum DTLSState state;
-
/* The certificate and private key content used for DTLS handshake */
char* cert_buf;
char* key_buf;
@@ -103,8 +90,6 @@ int ff_tls_set_external_socket(URLContext *h, URLContext *sock);
int ff_dtls_export_materials(URLContext *h, char *dtls_srtp_materials, size_t materials_sz);
-int ff_dtls_state(URLContext *h);
-
int ff_ssl_read_key_cert(char *key_url, char *cert_url, char *key_buf, size_t key_sz, char *cert_buf, size_t cert_sz, char **fingerprint);
int ff_ssl_gen_key_cert(char *key_buf, size_t key_sz, char *cert_buf, size_t cert_sz, char **fingerprint);
diff --git a/libavformat/tls_openssl.c b/libavformat/tls_openssl.c
index 2689aa5090..fa852aac18 100644
--- a/libavformat/tls_openssl.c
+++ b/libavformat/tls_openssl.c
@@ -541,12 +541,6 @@ int ff_dtls_export_materials(URLContext *h, char *dtls_srtp_materials, size_t ma
return 0;
}
-int ff_dtls_state(URLContext *h)
-{
- TLSContext *c = h->priv_data;
- return c->tls_shared.state;
-}
-
static int print_ssl_error(URLContext *h, int ret)
{
TLSContext *c = h->priv_data;
@@ -725,7 +719,6 @@ static int dtls_handshake(URLContext *h)
goto end;
ret = 0;
- p->tls_shared.state = DTLS_STATE_FINISHED;
end:
return ret;
}
diff --git a/libavformat/tls_schannel.c b/libavformat/tls_schannel.c
index c92870347f..50f35acdd6 100644
--- a/libavformat/tls_schannel.c
+++ b/libavformat/tls_schannel.c
@@ -681,12 +681,6 @@ int ff_dtls_export_materials(URLContext *h, char *dtls_srtp_materials, size_t ma
#endif
}
-int ff_dtls_state(URLContext *h)
-{
- TLSContext *c = h->priv_data;
- return c->tls_shared.state;
-}
-
static void init_sec_buffer(SecBuffer *buffer, unsigned long type,
void *data, unsigned long size)
{
@@ -1111,7 +1105,6 @@ static int tls_handshake(URLContext *h)
#endif
c->connected = 1;
- s->state = DTLS_STATE_FINISHED;
fail:
return ret;
diff --git a/libavformat/whip.c b/libavformat/whip.c
index e02ed7a8a4..fe35ab7205 100644
--- a/libavformat/whip.c
+++ b/libavformat/whip.c
@@ -217,9 +217,6 @@ typedef struct WHIPContext {
uint32_t flags; // enum WHIPFlags
/* The state of the RTC connection. */
enum WHIPState state;
- /* The callback return value for DTLS. */
- int dtls_ret;
- int dtls_closed;
/* Parameters for the input audio and video codecs. */
AVCodecParameters *audio_par;
@@ -377,41 +374,6 @@ static av_cold int certificate_key_init(AVFormatContext *s)
return ret;
}
-/**
- * When DTLS state change.
- */
-static int dtls_context_on_state(AVFormatContext *s, const char* type, const char* desc)
-{
- int ret = 0;
- WHIPContext *whip = s->priv_data;
- int state = ff_dtls_state(whip->dtls_uc);
-
- if (state == DTLS_STATE_CLOSED) {
- whip->dtls_closed = 1;
- av_log(whip, AV_LOG_VERBOSE, "DTLS session closed, type=%s, desc=%s, elapsed=%dms\n",
- type ? type : "", desc ? desc : "", ELAPSED(whip->whip_starttime, av_gettime()));
- goto error;
- }
-
- if (state == DTLS_STATE_FAILED) {
- whip->state = WHIP_STATE_FAILED;
- av_log(whip, AV_LOG_ERROR, "DTLS session failed, type=%s, desc=%s\n",
- type ? type : "", desc ? desc : "");
- whip->dtls_ret = AVERROR(EIO);
- goto error;
- }
-
- if (state == DTLS_STATE_FINISHED && whip->state < WHIP_STATE_DTLS_FINISHED) {
- whip->state = WHIP_STATE_DTLS_FINISHED;
- whip->whip_dtls_time = av_gettime();
- av_log(whip, AV_LOG_VERBOSE, "DTLS handshake is done, elapsed=%dms\n",
- ELAPSED(whip->whip_starttime, av_gettime()));
- return ret;
- }
-error:
- return -1;
-}
-
static av_cold int dtls_initialize(AVFormatContext *s)
{
WHIPContext *whip = s->priv_data;
@@ -1377,9 +1339,18 @@ next_packet:
/* If got any DTLS messages, handle it. */
if ((is_dtls_packet(whip->buf, ret) || is_dtls_active) && whip->state >= WHIP_STATE_ICE_CONNECTED || whip->state == WHIP_STATE_DTLS_CONNECTING) {
whip->state = WHIP_STATE_DTLS_CONNECTING;
- if ((ret = ffurl_handshake(whip->dtls_uc)) < 0)
+ ret = ffurl_handshake(whip->dtls_uc);
+ if (ret < 0) {
+ whip->state = WHIP_STATE_FAILED;
+ av_log(whip, AV_LOG_VERBOSE, "DTLS session failed\n");
goto end;
- dtls_context_on_state(s, NULL, NULL);
+ }
+ if (!ret) {
+ whip->state = WHIP_STATE_DTLS_FINISHED;
+ whip->whip_dtls_time = av_gettime();
+ av_log(whip, AV_LOG_VERBOSE, "DTLS handshake is done, elapsed=%dms\n",
+ ELAPSED(whip->whip_starttime, whip->whip_dtls_time));
+ }
goto next_packet;
}
}
@@ -1910,8 +1881,6 @@ static av_cold int whip_init(AVFormatContext *s)
end:
if (ret < 0 && whip->state < WHIP_STATE_FAILED)
whip->state = WHIP_STATE_FAILED;
- if (ret >= 0 && whip->state >= WHIP_STATE_FAILED && whip->dtls_ret < 0)
- ret = whip->dtls_ret;
return ret;
}
@@ -2022,12 +1991,8 @@ write_packet:
end:
if (buf) av_freep(&buf);
- if (ret < 0 && whip->state < WHIP_STATE_FAILED)
+ if (ret < 0)
whip->state = WHIP_STATE_FAILED;
- if (ret >= 0 && whip->state >= WHIP_STATE_FAILED && whip->dtls_ret < 0)
- ret = whip->dtls_ret;
- if (ret >= 0 && whip->dtls_closed)
- ret = AVERROR(EIO);
return ret;
}
--
2.49.0
_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".
^ permalink raw reply [flat|nested] 17+ messages in thread* [FFmpeg-devel] [PATCH v5 09/15] avformat/whip: check the peer whether is ice lite
2025-07-22 12:36 [FFmpeg-devel] [PATCH v5 00/15] avformat/whip: Add NACK, RTX, DTLS active support Jack Lau
` (7 preceding siblings ...)
2025-07-22 12:36 ` [FFmpeg-devel] [PATCH v5 08/15] avformat/whip: remove DTLSState enum Jack Lau
@ 2025-07-22 12:36 ` Jack Lau
2025-07-22 12:36 ` [FFmpeg-devel] [PATCH v5 10/15] avformat/whip: remove WHIP_STATE_DTLS_CONNECTING Jack Lau
` (5 subsequent siblings)
14 siblings, 0 replies; 17+ messages in thread
From: Jack Lau @ 2025-07-22 12:36 UTC (permalink / raw)
To: ffmpeg-devel; +Cc: Jack Lau
See RFC 5245 Section 4.3:
If an agent is a lite implementation, it MUST include an "a=ice-lite"
session-level attribute in its SDP. If an agent is a full
implementation, it MUST NOT include this attribute.
Signed-off-by: Jack Lau <jacklau1222@qq.com>
---
libavformat/whip.c | 3 +++
1 file changed, 3 insertions(+)
diff --git a/libavformat/whip.c b/libavformat/whip.c
index fe35ab7205..d37a61042f 100644
--- a/libavformat/whip.c
+++ b/libavformat/whip.c
@@ -248,6 +248,7 @@ typedef struct WHIPContext {
*/
char *sdp_offer;
+ int is_peer_ice_lite;
/* The ICE username and pwd from remote server. */
char *ice_ufrag_remote;
char *ice_pwd_remote;
@@ -870,6 +871,8 @@ static int parse_answer(AVFormatContext *s)
for (i = 0; !avio_feof(pb); i++) {
ff_get_chomp_line(pb, line, sizeof(line));
+ if (av_strstart(line, "a=ice-lite", &ptr))
+ whip->is_peer_ice_lite = 1;
if (av_strstart(line, "a=ice-ufrag:", &ptr) && !whip->ice_ufrag_remote) {
whip->ice_ufrag_remote = av_strdup(ptr);
if (!whip->ice_ufrag_remote) {
--
2.49.0
_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".
^ permalink raw reply [flat|nested] 17+ messages in thread* [FFmpeg-devel] [PATCH v5 10/15] avformat/whip: remove WHIP_STATE_DTLS_CONNECTING
2025-07-22 12:36 [FFmpeg-devel] [PATCH v5 00/15] avformat/whip: Add NACK, RTX, DTLS active support Jack Lau
` (8 preceding siblings ...)
2025-07-22 12:36 ` [FFmpeg-devel] [PATCH v5 09/15] avformat/whip: check the peer whether is ice lite Jack Lau
@ 2025-07-22 12:36 ` Jack Lau
2025-07-22 12:36 ` [FFmpeg-devel] [PATCH v5 11/15] avformat/whip: simplify and modularize the ICE and DTLS Jack Lau
` (4 subsequent siblings)
14 siblings, 0 replies; 17+ messages in thread
From: Jack Lau @ 2025-07-22 12:36 UTC (permalink / raw)
To: ffmpeg-devel; +Cc: Jack Lau
This value is only useful when dtls handshake is NONBLOCK mode,
dtls handshake just need to call ffurl_handshake once since it
force block mode.
Signed-off-by: Jack Lau <jacklau1222@qq.com>
---
libavformat/whip.c | 18 ++++++------------
1 file changed, 6 insertions(+), 12 deletions(-)
diff --git a/libavformat/whip.c b/libavformat/whip.c
index d37a61042f..4c8ed26cef 100644
--- a/libavformat/whip.c
+++ b/libavformat/whip.c
@@ -187,8 +187,6 @@ enum WHIPState {
WHIP_STATE_ICE_CONNECTING,
/* The muxer has received the ICE response from the peer. */
WHIP_STATE_ICE_CONNECTED,
- /* The muxer starts attempting the DTLS handshake. */
- WHIP_STATE_DTLS_CONNECTING,
/* The muxer has finished the DTLS handshake with the peer. */
WHIP_STATE_DTLS_FINISHED,
/* The muxer has finished the SRTP setup. */
@@ -1281,7 +1279,7 @@ next_packet:
}
/* Read the STUN or DTLS messages from peer. */
- for (i = 0; i < ICE_DTLS_READ_INTERVAL / 5 && whip->state < WHIP_STATE_DTLS_CONNECTING; i++) {
+ for (i = 0; i < ICE_DTLS_READ_INTERVAL / 5 && whip->state < WHIP_STATE_ICE_CONNECTED; i++) {
ret = ffurl_read(whip->udp, whip->buf, sizeof(whip->buf));
if (ret > 0)
break;
@@ -1295,14 +1293,11 @@ next_packet:
goto end;
}
- /* Got nothing, continue to process handshake. */
- if (ret <= 0 && (is_dtls_active ? whip->state < WHIP_STATE_ICE_CONNECTED : whip->state < WHIP_STATE_DTLS_CONNECTING))
- continue;
-
/* Handle the ICE binding response. */
if (ice_is_binding_response(whip->buf, ret)) {
if (whip->state < WHIP_STATE_ICE_CONNECTED) {
- whip->state = WHIP_STATE_ICE_CONNECTED;
+ if (whip->is_peer_ice_lite)
+ whip->state = WHIP_STATE_ICE_CONNECTED;
whip->whip_ice_time = av_gettime();
av_log(whip, AV_LOG_VERBOSE, "ICE STUN ok, state=%d, url=udp://%s:%d, location=%s, username=%s:%s, res=%dB, elapsed=%dms\n",
whip->state, whip->ice_host, whip->ice_port, whip->whip_resource_url ? whip->whip_resource_url : "",
@@ -1332,16 +1327,15 @@ next_packet:
goto next_packet;
}
- /* When a binding request is received, it is necessary to respond immediately. */
+ /* See RFC8445, Triggered check when the peer is ice full mode */
if (ice_is_binding_request(whip->buf, ret)) {
if ((ret = ice_handle_binding_request(s, whip->buf, ret)) < 0)
goto end;
goto next_packet;
}
- /* If got any DTLS messages, handle it. */
- if ((is_dtls_packet(whip->buf, ret) || is_dtls_active) && whip->state >= WHIP_STATE_ICE_CONNECTED || whip->state == WHIP_STATE_DTLS_CONNECTING) {
- whip->state = WHIP_STATE_DTLS_CONNECTING;
+ if ((is_dtls_packet(whip->buf, ret) || is_dtls_active) && whip->state >= WHIP_STATE_ICE_CONNECTED || whip->state == WHIP_STATE_ICE_CONNECTING) {
+ whip->state = WHIP_STATE_ICE_CONNECTED;
ret = ffurl_handshake(whip->dtls_uc);
if (ret < 0) {
whip->state = WHIP_STATE_FAILED;
--
2.49.0
_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".
^ permalink raw reply [flat|nested] 17+ messages in thread* [FFmpeg-devel] [PATCH v5 11/15] avformat/whip: simplify and modularize the ICE and DTLS
2025-07-22 12:36 [FFmpeg-devel] [PATCH v5 00/15] avformat/whip: Add NACK, RTX, DTLS active support Jack Lau
` (9 preceding siblings ...)
2025-07-22 12:36 ` [FFmpeg-devel] [PATCH v5 10/15] avformat/whip: remove WHIP_STATE_DTLS_CONNECTING Jack Lau
@ 2025-07-22 12:36 ` Jack Lau
2025-07-22 12:36 ` [FFmpeg-devel] [PATCH v5 12/15] avformat/tls_openssl: directly use mtu in TLSShared Jack Lau
` (3 subsequent siblings)
14 siblings, 0 replies; 17+ messages in thread
From: Jack Lau @ 2025-07-22 12:36 UTC (permalink / raw)
To: ffmpeg-devel; +Cc: Jack Lau
We try to perform dtls handshake when the ICE is totally done.
Refer to RFC8445,
When peer's ICE is lite, the peer won't trigged check so FFmpeg just send
STUN request and receive response, then ICE is done.
When peer's ICE is full, the peer will send STUN request after reponse
FFmpeg's request to ensure candidate pair become valid in both directions.
Then the peer does nomination, and ICE is done.
Signed-off-by: Jack Lau <jacklau1222@qq.com>
---
libavformat/whip.c | 135 +++++++++++++++++++++------------------------
1 file changed, 62 insertions(+), 73 deletions(-)
diff --git a/libavformat/whip.c b/libavformat/whip.c
index 4c8ed26cef..cfcb8e8888 100644
--- a/libavformat/whip.c
+++ b/libavformat/whip.c
@@ -373,19 +373,6 @@ static av_cold int certificate_key_init(AVFormatContext *s)
return ret;
}
-static av_cold int dtls_initialize(AVFormatContext *s)
-{
- WHIPContext *whip = s->priv_data;
- /* reuse the udp created by whip */
- ff_tls_set_external_socket(whip->dtls_uc, whip->udp);
-
- /* Make the socket non-blocking */
- ff_socket_nonblock(ffurl_get_file_handle(whip->dtls_uc), 1);
- whip->dtls_uc->flags |= AVIO_FLAG_NONBLOCK;
-
- return 0;
-}
-
/**
* Initialize and check the options for the WebRTC muxer.
*/
@@ -1232,14 +1219,12 @@ end:
return ret;
}
-static int ice_dtls_handshake(AVFormatContext *s)
+static int ice_handshake(AVFormatContext *s)
{
int ret = 0, size, i;
int64_t starttime = av_gettime(), now;
WHIPContext *whip = s->priv_data;
int is_dtls_active = whip->flags & WHIP_FLAG_DTLS_ACTIVE;
- AVDictionary *opts = NULL;
- char buf[256], *cert_buf = NULL, *key_buf = NULL;
if (whip->state < WHIP_STATE_UDP_CONNECTED || !whip->udp) {
av_log(whip, AV_LOG_ERROR, "UDP not connected, state=%d, udp=%p\n", whip->state, whip->udp);
@@ -1261,25 +1246,20 @@ static int ice_dtls_handshake(AVFormatContext *s)
goto end;
}
- if (whip->state < WHIP_STATE_ICE_CONNECTING)
- whip->state = WHIP_STATE_ICE_CONNECTING;
+ whip->state = WHIP_STATE_ICE_CONNECTING;
}
next_packet:
- if (whip->state >= WHIP_STATE_DTLS_FINISHED)
- /* DTLS handshake is done, exit the loop. */
- break;
-
now = av_gettime();
if (now - starttime >= whip->handshake_timeout * 1000) {
- av_log(whip, AV_LOG_ERROR, "DTLS handshake timeout=%dms, cost=%dms, elapsed=%dms, state=%d\n",
+ av_log(whip, AV_LOG_ERROR, "ICE handshake timeout=%dms, cost=%dms, elapsed=%dms, state=%d\n",
whip->handshake_timeout, ELAPSED(starttime, now), ELAPSED(whip->whip_starttime, now), whip->state);
ret = AVERROR(ETIMEDOUT);
goto end;
}
- /* Read the STUN or DTLS messages from peer. */
- for (i = 0; i < ICE_DTLS_READ_INTERVAL / 5 && whip->state < WHIP_STATE_ICE_CONNECTED; i++) {
+ /* Read the STUN or DTLS client hello from peer. */
+ for (i = 0; i < ICE_DTLS_READ_INTERVAL / 5; i++) {
ret = ffurl_read(whip->udp, whip->buf, sizeof(whip->buf));
if (ret > 0)
break;
@@ -1295,35 +1275,8 @@ next_packet:
/* Handle the ICE binding response. */
if (ice_is_binding_response(whip->buf, ret)) {
- if (whip->state < WHIP_STATE_ICE_CONNECTED) {
- if (whip->is_peer_ice_lite)
- whip->state = WHIP_STATE_ICE_CONNECTED;
- whip->whip_ice_time = av_gettime();
- av_log(whip, AV_LOG_VERBOSE, "ICE STUN ok, state=%d, url=udp://%s:%d, location=%s, username=%s:%s, res=%dB, elapsed=%dms\n",
- whip->state, whip->ice_host, whip->ice_port, whip->whip_resource_url ? whip->whip_resource_url : "",
- whip->ice_ufrag_remote, whip->ice_ufrag_local, ret, ELAPSED(whip->whip_starttime, av_gettime()));
-
- ff_url_join(buf, sizeof(buf), "dtls", NULL, whip->ice_host, whip->ice_port, NULL);
- av_dict_set_int(&opts, "mtu", whip->pkt_size, 0);
- if (whip->cert_file) {
- av_dict_set(&opts, "cert_file", whip->cert_file, 0);
- } else
- av_dict_set(&opts, "cert_pem", whip->cert_buf, 0);
-
- if (whip->key_file) {
- av_dict_set(&opts, "key_file", whip->key_file, 0);
- } else
- av_dict_set(&opts, "key_pem", whip->key_buf, 0);
- av_dict_set_int(&opts, "external_sock", 1, 0);
- av_dict_set_int(&opts, "listen", is_dtls_active ? 0 : 1, 0);
- /* If got the first binding response, start DTLS handshake. */
- ret = ffurl_open_whitelist(&whip->dtls_uc, buf, AVIO_FLAG_READ_WRITE, &s->interrupt_callback,
- &opts, s->protocol_whitelist, s->protocol_blacklist, NULL);
- av_dict_free(&opts);
- if (ret < 0)
- goto end;
- dtls_initialize(s);
- }
+ if (whip->is_peer_ice_lite)
+ whip->state = WHIP_STATE_ICE_CONNECTED;
goto next_packet;
}
@@ -1334,29 +1287,62 @@ next_packet:
goto next_packet;
}
- if ((is_dtls_packet(whip->buf, ret) || is_dtls_active) && whip->state >= WHIP_STATE_ICE_CONNECTED || whip->state == WHIP_STATE_ICE_CONNECTING) {
+ if (is_dtls_packet(whip->buf, ret) || whip->flags & WHIP_FLAG_DTLS_ACTIVE) {
whip->state = WHIP_STATE_ICE_CONNECTED;
- ret = ffurl_handshake(whip->dtls_uc);
- if (ret < 0) {
- whip->state = WHIP_STATE_FAILED;
- av_log(whip, AV_LOG_VERBOSE, "DTLS session failed\n");
- goto end;
- }
- if (!ret) {
- whip->state = WHIP_STATE_DTLS_FINISHED;
- whip->whip_dtls_time = av_gettime();
- av_log(whip, AV_LOG_VERBOSE, "DTLS handshake is done, elapsed=%dms\n",
- ELAPSED(whip->whip_starttime, whip->whip_dtls_time));
- }
- goto next_packet;
+ ret = 0;
+ whip->whip_ice_time = av_gettime();
+ av_log(whip, AV_LOG_VERBOSE, "ICE STUN ok, state=%d, url=udp://%s:%d, location=%s, username=%s:%s, res=%dB, elapsed=%dms\n",
+ whip->state, whip->ice_host, whip->ice_port, whip->whip_resource_url ? whip->whip_resource_url : "",
+ whip->ice_ufrag_remote, whip->ice_ufrag_local, ret, ELAPSED(whip->whip_starttime, av_gettime()));
+ break;
}
}
+end:
+ return ret;
+}
+static int dtls_handshake(AVFormatContext *s)
+{
+ int ret = 0;
+ WHIPContext *whip = s->priv_data;
+ AVDictionary *opts = NULL;
+ char buf[256];
+
+ ff_url_join(buf, sizeof(buf), "dtls", NULL, whip->ice_host, whip->ice_port, NULL);
+ av_dict_set_int(&opts, "mtu", whip->pkt_size, 0);
+ if (whip->cert_file) {
+ av_dict_set(&opts, "cert_file", whip->cert_file, 0);
+ } else
+ av_dict_set(&opts, "cert_pem", whip->cert_buf, 0);
+
+ if (whip->key_file) {
+ av_dict_set(&opts, "key_file", whip->key_file, 0);
+ } else
+ av_dict_set(&opts, "key_pem", whip->key_buf, 0);
+ av_dict_set_int(&opts, "external_sock", 1, 0);
+ av_dict_set_int(&opts, "listen", whip->flags & WHIP_FLAG_DTLS_ACTIVE ? 0 : 1, 0);
+ /* If got the first binding response, start DTLS handshake. */
+ ret = ffurl_open_whitelist(&whip->dtls_uc, buf, AVIO_FLAG_READ_WRITE, &s->interrupt_callback,
+ &opts, s->protocol_whitelist, s->protocol_blacklist, NULL);
+ av_dict_free(&opts);
+ if (ret < 0)
+ goto end;
+
+ /* reuse the udp created by whip */
+ ff_tls_set_external_socket(whip->dtls_uc, whip->udp);
+
+ ret = ffurl_handshake(whip->dtls_uc);
+ if (ret < 0) {
+ whip->state = WHIP_STATE_FAILED;
+ av_log(whip, AV_LOG_VERBOSE, "DTLS session failed\n");
+ }
+ if (!ret) {
+ whip->state = WHIP_STATE_DTLS_FINISHED;
+ whip->whip_dtls_time = av_gettime();
+ av_log(whip, AV_LOG_VERBOSE, "DTLS handshake is done, elapsed=%dms\n",
+ ELAPSED(whip->whip_starttime, whip->whip_dtls_time));
+ }
end:
- if (cert_buf)
- av_free(cert_buf);
- if (key_buf)
- av_free(key_buf);
return ret;
}
@@ -1866,7 +1852,10 @@ static av_cold int whip_init(AVFormatContext *s)
if ((ret = udp_connect(s)) < 0)
goto end;
- if ((ret = ice_dtls_handshake(s)) < 0)
+ if ((ret = ice_handshake(s)) < 0)
+ goto end;
+
+ if ((ret = dtls_handshake(s)) < 0)
goto end;
if ((ret = setup_srtp(s)) < 0)
--
2.49.0
_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".
^ permalink raw reply [flat|nested] 17+ messages in thread* [FFmpeg-devel] [PATCH v5 12/15] avformat/tls_openssl: directly use mtu in TLSShared
2025-07-22 12:36 [FFmpeg-devel] [PATCH v5 00/15] avformat/whip: Add NACK, RTX, DTLS active support Jack Lau
` (10 preceding siblings ...)
2025-07-22 12:36 ` [FFmpeg-devel] [PATCH v5 11/15] avformat/whip: simplify and modularize the ICE and DTLS Jack Lau
@ 2025-07-22 12:36 ` Jack Lau
2025-07-22 12:43 ` Timo Rothenpieler
2025-07-22 12:36 ` [FFmpeg-devel] [PATCH v5 13/15] avformat/tls: add new option use_srtp to control whether enable it Jack Lau
` (2 subsequent siblings)
14 siblings, 1 reply; 17+ messages in thread
From: Jack Lau @ 2025-07-22 12:36 UTC (permalink / raw)
To: ffmpeg-devel; +Cc: Jack Lau
Openssl 1.1.0 version haven't DTLS_get_data_mtu API
Signed-off-by: Jack Lau <jacklau1222@qq.com>
---
libavformat/tls_openssl.c | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/libavformat/tls_openssl.c b/libavformat/tls_openssl.c
index fa852aac18..54860857c0 100644
--- a/libavformat/tls_openssl.c
+++ b/libavformat/tls_openssl.c
@@ -1013,7 +1013,7 @@ static int tls_write(URLContext *h, const uint8_t *buf, int size)
uc->flags |= h->flags & AVIO_FLAG_NONBLOCK;
if (c->tls_shared.is_dtls)
- size = FFMIN(size, DTLS_get_data_mtu(c->ssl));
+ size = FFMIN(size, c->tls_shared.mtu);
ret = SSL_write(c->ssl, buf, size);
if (ret > 0)
--
2.49.0
_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".
^ permalink raw reply [flat|nested] 17+ messages in thread* Re: [FFmpeg-devel] [PATCH v5 12/15] avformat/tls_openssl: directly use mtu in TLSShared
2025-07-22 12:36 ` [FFmpeg-devel] [PATCH v5 12/15] avformat/tls_openssl: directly use mtu in TLSShared Jack Lau
@ 2025-07-22 12:43 ` Timo Rothenpieler
0 siblings, 0 replies; 17+ messages in thread
From: Timo Rothenpieler @ 2025-07-22 12:43 UTC (permalink / raw)
To: ffmpeg-devel
On 22/07/2025 14:36, Jack Lau wrote:
> Openssl 1.1.0 version haven't DTLS_get_data_mtu API
>
> Signed-off-by: Jack Lau <jacklau1222@qq.com>
> ---
> libavformat/tls_openssl.c | 2 +-
> 1 file changed, 1 insertion(+), 1 deletion(-)
>
> diff --git a/libavformat/tls_openssl.c b/libavformat/tls_openssl.c
> index fa852aac18..54860857c0 100644
> --- a/libavformat/tls_openssl.c
> +++ b/libavformat/tls_openssl.c
> @@ -1013,7 +1013,7 @@ static int tls_write(URLContext *h, const uint8_t *buf, int size)
> uc->flags |= h->flags & AVIO_FLAG_NONBLOCK;
>
> if (c->tls_shared.is_dtls)
> - size = FFMIN(size, DTLS_get_data_mtu(c->ssl));
> + size = FFMIN(size, c->tls_shared.mtu);
This is not the same value, no.
DTLS_get_data_mtu returns the mtu minus the size of the DTLS header and
footer, i.e. the maximum amount of data it can consume.
Which is exactly what it's used for here.
_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".
^ permalink raw reply [flat|nested] 17+ messages in thread
* [FFmpeg-devel] [PATCH v5 13/15] avformat/tls: add new option use_srtp to control whether enable it
2025-07-22 12:36 [FFmpeg-devel] [PATCH v5 00/15] avformat/whip: Add NACK, RTX, DTLS active support Jack Lau
` (11 preceding siblings ...)
2025-07-22 12:36 ` [FFmpeg-devel] [PATCH v5 12/15] avformat/tls_openssl: directly use mtu in TLSShared Jack Lau
@ 2025-07-22 12:36 ` Jack Lau
2025-07-22 12:36 ` [FFmpeg-devel] [PATCH v5 14/15] avformat/tls_openssl: cleanup the pointer name of TLSContext and TLSShared Jack Lau
2025-07-22 12:36 ` [FFmpeg-devel] [PATCH v5 15/15] doc: add doc for dtls and whip Jack Lau
14 siblings, 0 replies; 17+ messages in thread
From: Jack Lau @ 2025-07-22 12:36 UTC (permalink / raw)
To: ffmpeg-devel; +Cc: Jack Lau
Signed-off-by: Jack Lau <jacklau1222@qq.com>
---
libavformat/tls.h | 2 ++
libavformat/tls_openssl.c | 24 ++++++++++++------------
libavformat/whip.c | 1 +
3 files changed, 15 insertions(+), 12 deletions(-)
diff --git a/libavformat/tls.h b/libavformat/tls.h
index 157c0d0256..a11f8d6afb 100644
--- a/libavformat/tls.h
+++ b/libavformat/tls.h
@@ -51,6 +51,7 @@ typedef struct TLSShared {
URLContext *tcp;
int is_dtls;
+ int use_srtp;
/* The certificate and private key content used for DTLS handshake */
char* cert_buf;
@@ -77,6 +78,7 @@ typedef struct TLSShared {
{"listen", "Listen for incoming connections", offsetof(pstruct, options_field . listen), AV_OPT_TYPE_INT, { .i64 = 0 }, 0, 1, .flags = TLS_OPTFL }, \
{"http_proxy", "Set proxy to tunnel through", offsetof(pstruct, options_field . http_proxy), AV_OPT_TYPE_STRING, .flags = TLS_OPTFL }, \
{"external_sock", "Use external socket", offsetof(pstruct, options_field . external_sock), AV_OPT_TYPE_INT, { .i64 = 0 }, 0, 1, .flags = TLS_OPTFL }, \
+ {"use_srtp", "Enable use_srtp DTLS extension", offsetof(pstruct, options_field . use_srtp), AV_OPT_TYPE_INT, { .i64 = 0 }, 0, 1, .flags = TLS_OPTFL }, \
{"mtu", "Maximum Transmission Unit", offsetof(pstruct, options_field . mtu), AV_OPT_TYPE_INT, { .i64 = 0 }, 0, INT_MAX, .flags = TLS_OPTFL}, \
{"cert_pem", "Certificate PEM string", offsetof(pstruct, options_field . cert_buf), AV_OPT_TYPE_STRING, .flags = TLS_OPTFL }, \
{"key_pem", "Private key PEM string", offsetof(pstruct, options_field . key_buf), AV_OPT_TYPE_STRING, .flags = TLS_OPTFL }, \
diff --git a/libavformat/tls_openssl.c b/libavformat/tls_openssl.c
index 54860857c0..9a8456c438 100644
--- a/libavformat/tls_openssl.c
+++ b/libavformat/tls_openssl.c
@@ -818,12 +818,6 @@ static int dtls_start(URLContext *h, const char *url, int flags, AVDictionary **
int ret = 0;
c->is_dtls = 1;
- /**
- * The profile for OpenSSL's SRTP is SRTP_AES128_CM_SHA1_80, see ssl/d1_srtp.c.
- * The profile for FFmpeg's SRTP is SRTP_AES128_CM_HMAC_SHA1_80, see libavformat/srtp.c.
- */
- const char* profiles = "SRTP_AES128_CM_SHA1_80";
-
p->ctx = SSL_CTX_new(c->listen ? DTLS_server_method() : DTLS_client_method());
if (!p->ctx) {
ret = AVERROR(ENOMEM);
@@ -837,12 +831,18 @@ static int dtls_start(URLContext *h, const char *url, int flags, AVDictionary **
if (c->verify)
SSL_CTX_set_verify(p->ctx, SSL_VERIFY_PEER|SSL_VERIFY_FAIL_IF_NO_PEER_CERT, NULL);
- /* Setup the SRTP context */
- if (SSL_CTX_set_tlsext_use_srtp(p->ctx, profiles)) {
- av_log(p, AV_LOG_ERROR, "Init SSL_CTX_set_tlsext_use_srtp failed, profiles=%s, %s\n",
- profiles, openssl_get_error(p));
- ret = AVERROR(EINVAL);
- return ret;
+ if (c->use_srtp) {
+ /**
+ * The profile for OpenSSL's SRTP is SRTP_AES128_CM_SHA1_80, see ssl/d1_srtp.c.
+ * The profile for FFmpeg's SRTP is SRTP_AES128_CM_HMAC_SHA1_80, see libavformat/srtp.c.
+ */
+ const char* profiles = "SRTP_AES128_CM_SHA1_80";
+ if (SSL_CTX_set_tlsext_use_srtp(p->ctx, profiles)) {
+ av_log(p, AV_LOG_ERROR, "Init SSL_CTX_set_tlsext_use_srtp failed, profiles=%s, %s\n",
+ profiles, openssl_get_error(p));
+ ret = AVERROR(EINVAL);
+ return ret;
+ }
}
/* The ssl should not be created unless the ctx has been initialized. */
diff --git a/libavformat/whip.c b/libavformat/whip.c
index cfcb8e8888..82c9cee5c8 100644
--- a/libavformat/whip.c
+++ b/libavformat/whip.c
@@ -1320,6 +1320,7 @@ static int dtls_handshake(AVFormatContext *s)
} else
av_dict_set(&opts, "key_pem", whip->key_buf, 0);
av_dict_set_int(&opts, "external_sock", 1, 0);
+ av_dict_set_int(&opts, "use_srtp", 1, 0);
av_dict_set_int(&opts, "listen", whip->flags & WHIP_FLAG_DTLS_ACTIVE ? 0 : 1, 0);
/* If got the first binding response, start DTLS handshake. */
ret = ffurl_open_whitelist(&whip->dtls_uc, buf, AVIO_FLAG_READ_WRITE, &s->interrupt_callback,
--
2.49.0
_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".
^ permalink raw reply [flat|nested] 17+ messages in thread* [FFmpeg-devel] [PATCH v5 14/15] avformat/tls_openssl: cleanup the pointer name of TLSContext and TLSShared
2025-07-22 12:36 [FFmpeg-devel] [PATCH v5 00/15] avformat/whip: Add NACK, RTX, DTLS active support Jack Lau
` (12 preceding siblings ...)
2025-07-22 12:36 ` [FFmpeg-devel] [PATCH v5 13/15] avformat/tls: add new option use_srtp to control whether enable it Jack Lau
@ 2025-07-22 12:36 ` Jack Lau
2025-07-22 12:36 ` [FFmpeg-devel] [PATCH v5 15/15] doc: add doc for dtls and whip Jack Lau
14 siblings, 0 replies; 17+ messages in thread
From: Jack Lau @ 2025-07-22 12:36 UTC (permalink / raw)
To: ffmpeg-devel; +Cc: Jack Lau
Signed-off-by: Jack Lau <jacklau1222@qq.com>
---
libavformat/tls_openssl.c | 236 +++++++++++++++++++-------------------
1 file changed, 118 insertions(+), 118 deletions(-)
diff --git a/libavformat/tls_openssl.c b/libavformat/tls_openssl.c
index 9a8456c438..d42c5c95cf 100644
--- a/libavformat/tls_openssl.c
+++ b/libavformat/tls_openssl.c
@@ -501,16 +501,16 @@ typedef struct TLSContext {
* to a human-readable string, and stores it in the TLSContext's error_message field.
* The error queue is then cleared using ERR_clear_error().
*/
-static const char* openssl_get_error(TLSContext *ctx)
+static const char* openssl_get_error(TLSContext *c)
{
int r2 = ERR_get_error();
if (r2) {
- ERR_error_string_n(r2, ctx->error_message, sizeof(ctx->error_message));
+ ERR_error_string_n(r2, c->error_message, sizeof(c->error_message));
} else
- ctx->error_message[0] = '\0';
+ c->error_message[0] = '\0';
ERR_clear_error();
- return ctx->error_message;
+ return c->error_message;
}
int ff_tls_set_external_socket(URLContext *h, URLContext *sock)
@@ -656,28 +656,28 @@ static int url_bio_bputs(BIO *b, const char *str)
static av_cold int init_bio_method(URLContext *h)
{
- TLSContext *p = h->priv_data;
+ TLSContext *c = h->priv_data;
BIO *bio;
int bio_idx = BIO_get_new_index();
if (bio_idx == -1)
return AVERROR_EXTERNAL;
- p->url_bio_method = BIO_meth_new(bio_idx | BIO_TYPE_SOURCE_SINK, "urlprotocol bio");
- BIO_meth_set_write(p->url_bio_method, url_bio_bwrite);
- BIO_meth_set_read(p->url_bio_method, url_bio_bread);
- BIO_meth_set_puts(p->url_bio_method, url_bio_bputs);
- BIO_meth_set_ctrl(p->url_bio_method, url_bio_ctrl);
- BIO_meth_set_create(p->url_bio_method, url_bio_create);
- BIO_meth_set_destroy(p->url_bio_method, url_bio_destroy);
- bio = BIO_new(p->url_bio_method);
- BIO_set_data(bio, p);
-
- SSL_set_bio(p->ssl, bio, bio);
+ c->url_bio_method = BIO_meth_new(bio_idx | BIO_TYPE_SOURCE_SINK, "urlprotocol bio");
+ BIO_meth_set_write(c->url_bio_method, url_bio_bwrite);
+ BIO_meth_set_read(c->url_bio_method, url_bio_bread);
+ BIO_meth_set_puts(c->url_bio_method, url_bio_bputs);
+ BIO_meth_set_ctrl(c->url_bio_method, url_bio_ctrl);
+ BIO_meth_set_create(c->url_bio_method, url_bio_create);
+ BIO_meth_set_destroy(c->url_bio_method, url_bio_destroy);
+ bio = BIO_new(c->url_bio_method);
+ BIO_set_data(bio, c);
+
+ SSL_set_bio(c->ssl, bio, bio);
return 0;
}
static void openssl_info_callback(const SSL *ssl, int where, int ret) {
const char *method = "undefined";
- TLSContext *ctx = (TLSContext*)SSL_get_ex_data(ssl, 0);
+ TLSContext *c = (TLSContext*)SSL_get_ex_data(ssl, 0);
if (where & SSL_ST_CONNECT) {
method = "SSL_connect";
@@ -685,11 +685,11 @@ static void openssl_info_callback(const SSL *ssl, int where, int ret) {
method = "SSL_accept";
if (where & SSL_CB_LOOP) {
- av_log(ctx, AV_LOG_DEBUG, "Info method=%s state=%s(%s), where=%d, ret=%d\n",
+ av_log(c, AV_LOG_DEBUG, "Info method=%s state=%s(%s), where=%d, ret=%d\n",
method, SSL_state_string(ssl), SSL_state_string_long(ssl), where, ret);
} else if (where & SSL_CB_ALERT) {
method = (where & SSL_CB_READ) ? "read":"write";
- av_log(ctx, AV_LOG_DEBUG, "Alert method=%s state=%s(%s), where=%d, ret=%d\n",
+ av_log(c, AV_LOG_DEBUG, "Alert method=%s state=%s(%s), where=%d, ret=%d\n",
method, SSL_state_string(ssl), SSL_state_string_long(ssl), where, ret);
}
}
@@ -697,25 +697,25 @@ static void openssl_info_callback(const SSL *ssl, int where, int ret) {
static int dtls_handshake(URLContext *h)
{
int ret = 1, r0, r1;
- TLSContext *p = h->priv_data;
+ TLSContext *c = h->priv_data;
- p->tls_shared.udp->flags &= ~AVIO_FLAG_NONBLOCK;
+ c->tls_shared.udp->flags &= ~AVIO_FLAG_NONBLOCK;
- r0 = SSL_do_handshake(p->ssl);
+ r0 = SSL_do_handshake(c->ssl);
if (r0 <= 0) {
- r1 = SSL_get_error(p->ssl, r0);
+ r1 = SSL_get_error(c->ssl, r0);
if (r1 != SSL_ERROR_WANT_READ && r1 != SSL_ERROR_WANT_WRITE && r1 != SSL_ERROR_ZERO_RETURN) {
- av_log(p, AV_LOG_ERROR, "Handshake failed, r0=%d, r1=%d\n", r0, r1);
+ av_log(c, AV_LOG_ERROR, "Handshake failed, r0=%d, r1=%d\n", r0, r1);
ret = print_ssl_error(h, r0);
goto end;
}
} else {
- av_log(p, AV_LOG_TRACE, "Handshake success, r0=%d\n", r0);
+ av_log(c, AV_LOG_TRACE, "Handshake success, r0=%d\n", r0);
}
/* Check whether the handshake is completed. */
- if (SSL_is_init_finished(p->ssl) != TLS_ST_OK)
+ if (SSL_is_init_finished(c->ssl) != TLS_ST_OK)
goto end;
ret = 0;
@@ -726,57 +726,57 @@ end:
static av_cold int openssl_init_ca_key_cert(URLContext *h)
{
int ret;
- TLSContext *p = h->priv_data;
- TLSShared *c = &p->tls_shared;
+ TLSContext *c = h->priv_data;
+ TLSShared *s = &c->tls_shared;
EVP_PKEY *pkey = NULL;
X509 *cert = NULL;
/* setup ca, private key, certificate */
- if (c->ca_file) {
- if (!SSL_CTX_load_verify_locations(p->ctx, c->ca_file, NULL))
- av_log(h, AV_LOG_ERROR, "SSL_CTX_load_verify_locations %s\n", openssl_get_error(p));
+ if (s->ca_file) {
+ if (!SSL_CTX_load_verify_locations(c->ctx, s->ca_file, NULL))
+ av_log(h, AV_LOG_ERROR, "SSL_CTX_load_verify_locations %s\n", openssl_get_error(c));
} else {
- if (!SSL_CTX_set_default_verify_paths(p->ctx)) {
+ if (!SSL_CTX_set_default_verify_paths(c->ctx)) {
// Only log the failure but do not error out, as this is not fatal
av_log(h, AV_LOG_WARNING, "Failure setting default verify locations: %s\n",
- openssl_get_error(p));
+ openssl_get_error(c));
}
}
- if (c->cert_file) {
- ret = SSL_CTX_use_certificate_chain_file(p->ctx, c->cert_file);
+ if (s->cert_file) {
+ ret = SSL_CTX_use_certificate_chain_file(c->ctx, s->cert_file);
if (ret <= 0) {
av_log(h, AV_LOG_ERROR, "Unable to load cert file %s: %s\n",
- c->cert_file, openssl_get_error(p));
+ s->cert_file, openssl_get_error(c));
ret = AVERROR(EIO);
goto fail;
}
- } else if (c->cert_buf) {
- cert = cert_from_pem_string(c->cert_buf);
- if (SSL_CTX_use_certificate(p->ctx, cert) != 1) {
- av_log(p, AV_LOG_ERROR, "SSL: Init SSL_CTX_use_certificate failed, %s\n", openssl_get_error(p));
+ } else if (s->cert_buf) {
+ cert = cert_from_pem_string(s->cert_buf);
+ if (SSL_CTX_use_certificate(c->ctx, cert) != 1) {
+ av_log(c, AV_LOG_ERROR, "SSL: Init SSL_CTX_use_certificate failed, %s\n", openssl_get_error(c));
ret = AVERROR(EINVAL);
goto fail;
}
}
- if (c->key_file) {
- ret = SSL_CTX_use_PrivateKey_file(p->ctx, c->key_file, SSL_FILETYPE_PEM);
+ if (s->key_file) {
+ ret = SSL_CTX_use_PrivateKey_file(c->ctx, s->key_file, SSL_FILETYPE_PEM);
if (ret <= 0) {
av_log(h, AV_LOG_ERROR, "Unable to load key file %s: %s\n",
- c->key_file, openssl_get_error(p));
+ s->key_file, openssl_get_error(c));
ret = AVERROR(EIO);
goto fail;
}
- } else if (c->key_buf) {
- pkey = pkey_from_pem_string(c->key_buf, 1);
- if (SSL_CTX_use_PrivateKey(p->ctx, pkey) != 1) {
- av_log(p, AV_LOG_ERROR, "Init SSL_CTX_use_PrivateKey failed, %s\n", openssl_get_error(p));
+ } else if (s->key_buf) {
+ pkey = pkey_from_pem_string(s->key_buf, 1);
+ if (SSL_CTX_use_PrivateKey(c->ctx, pkey) != 1) {
+ av_log(c, AV_LOG_ERROR, "Init SSL_CTX_use_PrivateKey failed, %s\n", openssl_get_error(c));
ret = AVERROR(EINVAL);
goto fail;
}
}
- if (c->listen && !c->cert_file && !c->cert_buf && !c->key_file && !c->key_buf) {
+ if (s->listen && !s->cert_file && !s->cert_buf && !s->key_file && !s->key_buf) {
av_log(h, AV_LOG_VERBOSE, "No server certificate provided, using self-signed\n");
ret = openssl_gen_private_key(&pkey);
@@ -787,14 +787,14 @@ static av_cold int openssl_init_ca_key_cert(URLContext *h)
if (ret < 0)
goto fail;
- if (SSL_CTX_use_certificate(p->ctx, cert) != 1) {
- av_log(p, AV_LOG_ERROR, "SSL_CTX_use_certificate failed for self-signed cert, %s\n", openssl_get_error(p));
+ if (SSL_CTX_use_certificate(c->ctx, cert) != 1) {
+ av_log(c, AV_LOG_ERROR, "SSL_CTX_use_certificate failed for self-signed cert, %s\n", openssl_get_error(c));
ret = AVERROR(EINVAL);
goto fail;
}
- if (SSL_CTX_use_PrivateKey(p->ctx, pkey) != 1) {
- av_log(p, AV_LOG_ERROR, "SSL_CTX_use_PrivateKey failed for self-signed cert, %s\n", openssl_get_error(p));
+ if (SSL_CTX_use_PrivateKey(c->ctx, pkey) != 1) {
+ av_log(c, AV_LOG_ERROR, "SSL_CTX_use_PrivateKey failed for self-signed cert, %s\n", openssl_get_error(c));
ret = AVERROR(EINVAL);
goto fail;
}
@@ -813,13 +813,13 @@ fail:
*/
static int dtls_start(URLContext *h, const char *url, int flags, AVDictionary **options)
{
- TLSContext *p = h->priv_data;
- TLSShared *c = &p->tls_shared;
+ TLSContext *c = h->priv_data;
+ TLSShared *s = &c->tls_shared;
int ret = 0;
- c->is_dtls = 1;
+ s->is_dtls = 1;
- p->ctx = SSL_CTX_new(c->listen ? DTLS_server_method() : DTLS_client_method());
- if (!p->ctx) {
+ c->ctx = SSL_CTX_new(s->listen ? DTLS_server_method() : DTLS_client_method());
+ if (!c->ctx) {
ret = AVERROR(ENOMEM);
goto fail;
}
@@ -828,63 +828,63 @@ static int dtls_start(URLContext *h, const char *url, int flags, AVDictionary **
if (ret < 0) goto fail;
/* Note, this doesn't check that the peer certificate actually matches the requested hostname. */
- if (c->verify)
- SSL_CTX_set_verify(p->ctx, SSL_VERIFY_PEER|SSL_VERIFY_FAIL_IF_NO_PEER_CERT, NULL);
+ if (s->verify)
+ SSL_CTX_set_verify(c->ctx, SSL_VERIFY_PEER|SSL_VERIFY_FAIL_IF_NO_PEER_CERT, NULL);
- if (c->use_srtp) {
+ if (s->use_srtp) {
/**
* The profile for OpenSSL's SRTP is SRTP_AES128_CM_SHA1_80, see ssl/d1_srtp.c.
* The profile for FFmpeg's SRTP is SRTP_AES128_CM_HMAC_SHA1_80, see libavformat/srtp.c.
*/
const char* profiles = "SRTP_AES128_CM_SHA1_80";
- if (SSL_CTX_set_tlsext_use_srtp(p->ctx, profiles)) {
- av_log(p, AV_LOG_ERROR, "Init SSL_CTX_set_tlsext_use_srtp failed, profiles=%s, %s\n",
- profiles, openssl_get_error(p));
+ if (SSL_CTX_set_tlsext_use_srtp(c->ctx, profiles)) {
+ av_log(c, AV_LOG_ERROR, "Init SSL_CTX_set_tlsext_use_srtp failed, profiles=%s, %s\n",
+ profiles, openssl_get_error(c));
ret = AVERROR(EINVAL);
return ret;
}
}
/* The ssl should not be created unless the ctx has been initialized. */
- p->ssl = SSL_new(p->ctx);
- if (!p->ssl) {
+ c->ssl = SSL_new(c->ctx);
+ if (!c->ssl) {
ret = AVERROR(ENOMEM);
goto fail;
}
- if (!c->listen && !c->numerichost)
- SSL_set_tlsext_host_name(p->ssl, c->host);
+ if (!s->listen && !s->numerichost)
+ SSL_set_tlsext_host_name(c->ssl, s->host);
/* Setup the callback for logging. */
- SSL_set_ex_data(p->ssl, 0, p);
- SSL_CTX_set_info_callback(p->ctx, openssl_info_callback);
+ SSL_set_ex_data(c->ssl, 0, c);
+ SSL_CTX_set_info_callback(c->ctx, openssl_info_callback);
/**
* We have set the MTU to fragment the DTLS packet. It is important to note that the
* packet is split to ensure that each handshake packet is smaller than the MTU.
*/
- if (c->mtu <= 0)
- c->mtu = 1096;
- SSL_set_options(p->ssl, SSL_OP_NO_QUERY_MTU);
- SSL_set_mtu(p->ssl, c->mtu);
- DTLS_set_link_mtu(p->ssl, c->mtu);
+ if (s->mtu <= 0)
+ s->mtu = 1096;
+ SSL_set_options(c->ssl, SSL_OP_NO_QUERY_MTU);
+ SSL_set_mtu(c->ssl, s->mtu);
+ DTLS_set_link_mtu(c->ssl, s->mtu);
ret = init_bio_method(h);
if (ret < 0)
goto fail;
- if (p->tls_shared.external_sock != 1) {
- if ((ret = ff_tls_open_underlying(&p->tls_shared, h, url, options)) < 0) {
- av_log(p, AV_LOG_ERROR, "Failed to connect %s\n", url);
+ if (c->tls_shared.external_sock != 1) {
+ if ((ret = ff_tls_open_underlying(&c->tls_shared, h, url, options)) < 0) {
+ av_log(c, AV_LOG_ERROR, "Failed to connect %s\n", url);
return ret;
}
}
/* This seems to be neccesary despite explicitly setting client/server method above. */
- if (c->listen)
- SSL_set_accept_state(p->ssl);
+ if (s->listen)
+ SSL_set_accept_state(c->ssl);
else
- SSL_set_connect_state(p->ssl);
+ SSL_set_connect_state(c->ssl);
/**
* During initialization, we only need to call SSL_do_handshake once because SSL_read consumes
@@ -897,16 +897,16 @@ static int dtls_start(URLContext *h, const char *url, int flags, AVDictionary **
*
* The SSL_do_handshake can't be called if DTLS hasn't prepare for udp.
*/
- if (p->tls_shared.external_sock != 1) {
+ if (c->tls_shared.external_sock != 1) {
ret = dtls_handshake(h);
// Fatal SSL error, for example, no available suite when peer is DTLS 1.0 while we are DTLS 1.2.
if (ret < 0) {
- av_log(p, AV_LOG_ERROR, "Failed to drive SSL context, ret=%d\n", ret);
+ av_log(c, AV_LOG_ERROR, "Failed to drive SSL context, ret=%d\n", ret);
return AVERROR(EIO);
}
}
- av_log(p, AV_LOG_VERBOSE, "Setup ok, MTU=%d\n", p->tls_shared.mtu);
+ av_log(c, AV_LOG_VERBOSE, "Setup ok, MTU=%d\n", c->tls_shared.mtu);
ret = 0;
fail:
@@ -915,24 +915,24 @@ fail:
static int tls_open(URLContext *h, const char *uri, int flags, AVDictionary **options)
{
- TLSContext *p = h->priv_data;
- TLSShared *c = &p->tls_shared;
+ TLSContext *c = h->priv_data;
+ TLSShared *s = &c->tls_shared;
int ret;
- if ((ret = ff_tls_open_underlying(c, h, uri, options)) < 0)
+ if ((ret = ff_tls_open_underlying(s, h, uri, options)) < 0)
goto fail;
// We want to support all versions of TLS >= 1.0, but not the deprecated
// and insecure SSLv2 and SSLv3. Despite the name, TLS_*_method()
// enables support for all versions of SSL and TLS, and we then disable
// support for the old protocols immediately after creating the context.
- p->ctx = SSL_CTX_new(c->listen ? TLS_server_method() : TLS_client_method());
- if (!p->ctx) {
- av_log(h, AV_LOG_ERROR, "%s\n", openssl_get_error(p));
+ c->ctx = SSL_CTX_new(s->listen ? TLS_server_method() : TLS_client_method());
+ if (!c->ctx) {
+ av_log(h, AV_LOG_ERROR, "%s\n", openssl_get_error(c));
ret = AVERROR(EIO);
goto fail;
}
- if (!SSL_CTX_set_min_proto_version(p->ctx, TLS1_VERSION)) {
+ if (!SSL_CTX_set_min_proto_version(c->ctx, TLS1_VERSION)) {
av_log(h, AV_LOG_ERROR, "Failed to set minimum TLS version to TLSv1\n");
ret = AVERROR_EXTERNAL;
goto fail;
@@ -940,35 +940,35 @@ static int tls_open(URLContext *h, const char *uri, int flags, AVDictionary **op
ret = openssl_init_ca_key_cert(h);
if (ret < 0) goto fail;
- if (c->verify)
- SSL_CTX_set_verify(p->ctx, SSL_VERIFY_PEER|SSL_VERIFY_FAIL_IF_NO_PEER_CERT, NULL);
- p->ssl = SSL_new(p->ctx);
- if (!p->ssl) {
- av_log(h, AV_LOG_ERROR, "%s\n", openssl_get_error(p));
+ if (s->verify)
+ SSL_CTX_set_verify(c->ctx, SSL_VERIFY_PEER|SSL_VERIFY_FAIL_IF_NO_PEER_CERT, NULL);
+ c->ssl = SSL_new(c->ctx);
+ if (!c->ssl) {
+ av_log(h, AV_LOG_ERROR, "%s\n", openssl_get_error(c));
ret = AVERROR(EIO);
goto fail;
}
- SSL_set_ex_data(p->ssl, 0, p);
- SSL_CTX_set_info_callback(p->ctx, openssl_info_callback);
+ SSL_set_ex_data(c->ssl, 0, c);
+ SSL_CTX_set_info_callback(c->ctx, openssl_info_callback);
ret = init_bio_method(h);
if (ret < 0)
goto fail;
- if (!c->listen && !c->numerichost) {
+ if (!s->listen && !s->numerichost) {
// By default OpenSSL does too lax wildcard matching
- SSL_set_hostflags(p->ssl, X509_CHECK_FLAG_NO_PARTIAL_WILDCARDS);
- if (!SSL_set1_host(p->ssl, c->host)) {
+ SSL_set_hostflags(c->ssl, X509_CHECK_FLAG_NO_PARTIAL_WILDCARDS);
+ if (!SSL_set1_host(c->ssl, s->host)) {
av_log(h, AV_LOG_ERROR, "Failed to set hostname for TLS/SSL verification: %s\n",
- openssl_get_error(p));
+ openssl_get_error(c));
ret = AVERROR_EXTERNAL;
goto fail;
}
- if (!SSL_set_tlsext_host_name(p->ssl, c->host)) {
- av_log(h, AV_LOG_ERROR, "Failed to set hostname for SNI: %s\n", openssl_get_error(p));
+ if (!SSL_set_tlsext_host_name(c->ssl, s->host)) {
+ av_log(h, AV_LOG_ERROR, "Failed to set hostname for SNI: %s\n", openssl_get_error(c));
ret = AVERROR_EXTERNAL;
goto fail;
}
}
- ret = c->listen ? SSL_accept(p->ssl) : SSL_connect(p->ssl);
+ ret = s->listen ? SSL_accept(c->ssl) : SSL_connect(c->ssl);
if (ret == 0) {
av_log(h, AV_LOG_ERROR, "Unable to negotiate TLS/SSL session\n");
ret = AVERROR(EIO);
@@ -987,10 +987,10 @@ fail:
static int tls_read(URLContext *h, uint8_t *buf, int size)
{
TLSContext *c = h->priv_data;
- URLContext *uc = c->tls_shared.is_dtls ? c->tls_shared.udp
- : c->tls_shared.tcp;
+ TLSShared *s = &c->tls_shared;
+ URLContext *uc = s->is_dtls ? s->udp : s->tcp;
int ret;
- // Set or clear the AVIO_FLAG_NONBLOCK on c->tls_shared.tcp
+ // Set or clear the AVIO_FLAG_NONBLOCK on the underlying socket
uc->flags &= ~AVIO_FLAG_NONBLOCK;
uc->flags |= h->flags & AVIO_FLAG_NONBLOCK;
ret = SSL_read(c->ssl, buf, size);
@@ -1004,16 +1004,16 @@ static int tls_read(URLContext *h, uint8_t *buf, int size)
static int tls_write(URLContext *h, const uint8_t *buf, int size)
{
TLSContext *c = h->priv_data;
- URLContext *uc = c->tls_shared.is_dtls ? c->tls_shared.udp
- : c->tls_shared.tcp;
+ TLSShared *s = &c->tls_shared;
+ URLContext *uc = s->is_dtls ? s->udp : s->tcp;
int ret;
- // Set or clear the AVIO_FLAG_NONBLOCK on c->tls_shared.tcp
+ // Set or clear the AVIO_FLAG_NONBLOCK on the underlying socket
uc->flags &= ~AVIO_FLAG_NONBLOCK;
uc->flags |= h->flags & AVIO_FLAG_NONBLOCK;
- if (c->tls_shared.is_dtls)
- size = FFMIN(size, c->tls_shared.mtu);
+ if (s->is_dtls)
+ size = FFMIN(size, s->mtu);
ret = SSL_write(c->ssl, buf, size);
if (ret > 0)
@@ -1025,16 +1025,16 @@ static int tls_write(URLContext *h, const uint8_t *buf, int size)
static int tls_get_file_handle(URLContext *h)
{
- TLSContext *p = h->priv_data;
- TLSShared *c = &p->tls_shared;
- return ffurl_get_file_handle(c->is_dtls ? c->udp : c->tcp);
+ TLSContext *c = h->priv_data;
+ TLSShared *s = &c->tls_shared;
+ return ffurl_get_file_handle(s->is_dtls ? s->udp : s->tcp);
}
static int tls_get_short_seek(URLContext *h)
{
- TLSContext *p = h->priv_data;
- TLSShared *c = &p->tls_shared;
- return ffurl_get_short_seek(c->is_dtls ? c->udp : c->tcp);
+ TLSContext *c = h->priv_data;
+ TLSShared *s = &c->tls_shared;
+ return ffurl_get_short_seek(s->is_dtls ? s->udp : s->tcp);
}
static const AVOption options[] = {
--
2.49.0
_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".
^ permalink raw reply [flat|nested] 17+ messages in thread* [FFmpeg-devel] [PATCH v5 15/15] doc: add doc for dtls and whip
2025-07-22 12:36 [FFmpeg-devel] [PATCH v5 00/15] avformat/whip: Add NACK, RTX, DTLS active support Jack Lau
` (13 preceding siblings ...)
2025-07-22 12:36 ` [FFmpeg-devel] [PATCH v5 14/15] avformat/tls_openssl: cleanup the pointer name of TLSContext and TLSShared Jack Lau
@ 2025-07-22 12:36 ` Jack Lau
14 siblings, 0 replies; 17+ messages in thread
From: Jack Lau @ 2025-07-22 12:36 UTC (permalink / raw)
To: ffmpeg-devel; +Cc: Jack Lau
Signed-off-by: Jack Lau <jacklau1222@qq.com>
---
doc/muxers.texi | 29 ++++++++++++++---
doc/protocols.texi | 78 ++++++++++++++++++++++++++++++++++++++++++++++
2 files changed, 103 insertions(+), 4 deletions(-)
diff --git a/doc/muxers.texi b/doc/muxers.texi
index d2ee90bf33..393e606e12 100644
--- a/doc/muxers.texi
+++ b/doc/muxers.texi
@@ -3915,16 +3915,37 @@ Default value is 5000.
@item pkt_size @var{integer}
Set the maximum size, in bytes, of RTP packets that send out.
-Default value is 1500.
+Default value is 1200.
@item authorization @var{string}
-The optional Bearer token for WHIP Authorization.
+Optional Bearer token for WHIP Authorization.
@item cert_file @var{string}
-The optional certificate file path for DTLS.
+Optional certificate file path for DTLS.
@item key_file @var{string}
-The optional private key file path for DTLS.
+Optional private key file path for DTLS.
+
+@item whip_flags @var{flags}
+Possible values:
+
+@table @samp
+@item ignore_ipv6
+Ignore any IPv6 ICE candidates.
+
+@item disable_rtx
+Disable RFC 4588 RTX (Retransmission) support.
+This disables the retransmission mechanism for lost RTP packets.
+
+@item dtls_active
+Set DTLS role as active (client role) instead of passive (server role).
+By default, WHIP uses passive DTLS role, but some servers may require active role.
+@end table
+
+@item rtx_history_size @var{integer}
+Set the packet history size for RTX (retransmission) support.
+This determines how many recent RTP packets are kept in memory for potential
+retransmission requests. Range is 64 to 2048, default is 256.
@end table
diff --git a/doc/protocols.texi b/doc/protocols.texi
index 969f4bf022..1e74653329 100644
--- a/doc/protocols.texi
+++ b/doc/protocols.texi
@@ -2028,6 +2028,84 @@ To play back a stream from the TLS/SSL server using @command{ffplay}:
ffplay tls://@var{hostname}:@var{port}
@end example
+@section dtls
+
+Datagram Transport Layer Security (DTLS)
+
+The required syntax for a DTLS URL is:
+@example
+dtls://@var{hostname}:@var{port}
+@end example
+
+DTLS shares most options with TLS, but operates over UDP instead of TCP.
+The following parameters can be set via command line options
+(or in code via @code{AVOption}s):
+
+@table @option
+
+@item ca_file, cafile=@var{filename}
+A file containing certificate authority (CA) root certificates to treat
+as trusted. If the linked TLS library contains a default this might not
+need to be specified for verification to work, but not all libraries and
+setups have defaults built in.
+The file must be in OpenSSL PEM format.
+
+@item tls_verify=@var{1|0}
+If enabled, try to verify the peer that we are communicating with.
+Note, if using OpenSSL, this currently only makes sure that the
+peer certificate is signed by one of the root certificates in the CA
+database, but it does not validate that the certificate actually
+matches the host name we are trying to connect to.
+
+This is disabled by default since it requires a CA database to be
+provided by the caller in many cases.
+
+@item cert_file, cert=@var{filename}
+A file containing a certificate to use in the handshake with the peer.
+(When operating as server, in listen mode, this is more often required
+by the peer, while client certificates only are mandated in certain
+setups.)
+
+@item key_file, key=@var{filename}
+A file containing the private key for the certificate.
+
+@item cert_pem=@var{string}
+A PEM-encoded certificate string to use in the handshake with the peer.
+
+@item key_pem=@var{string}
+A PEM-encoded private key string for the certificate.
+
+@item listen=@var{1|0}
+If enabled, listen for connections on the provided port, and assume
+the server role in the handshake instead of the client role.
+
+@item mtu=@var{size}
+Set the Maximum Transmission Unit (MTU) for DTLS packets.
+
+@item use_srtp=@var{1|0}
+Enable the use_srtp DTLS extension.
+This is used in WebRTC applications to establish SRTP encryption keys
+through the DTLS handshake. Default is disabled.
+
+@item external_sock=@var{1|0}
+Use an external socket instead of creating a new one. Default is disabled.
+
+@end table
+
+Example command lines:
+
+To create a DTLS server:
+
+@example
+ffmpeg -listen 1 -i dtls://@var{hostname}:@var{port} @var{output}
+@end example
+
+To create a DTLS server and send data to server:
+
+@example
+ffmpeg -i @var{input} -f @var{format} dtls://@var{hostname}:@var{port}
+@end example
+
@section udp
User Datagram Protocol.
--
2.49.0
_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".
^ permalink raw reply [flat|nested] 17+ messages in thread