From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from ffbox0-bg.ffmpeg.org (ffbox0-bg.ffmpeg.org [79.124.17.100]) by master.gitmailbox.com (Postfix) with ESMTPS id 5E21B4B53E for ; Mon, 21 Jul 2025 11:32:59 +0000 (UTC) Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.ffmpeg.org (Postfix) with ESMTP id E80EE68C3C5; Mon, 21 Jul 2025 14:31:13 +0300 (EEST) Received: from mail-pf1-f171.google.com (mail-pf1-f171.google.com [209.85.210.171]) by ffbox0-bg.ffmpeg.org (Postfix) with ESMTPS id EFFE468C417 for ; Mon, 21 Jul 2025 14:31:12 +0300 (EEST) Received: by mail-pf1-f171.google.com with SMTP id d2e1a72fcca58-74801bc6dc5so3994219b3a.1 for ; Mon, 21 Jul 2025 04:31:12 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1753097471; x=1753702271; darn=ffmpeg.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=7ynnYBaX+c5555v0h5W6O4GrGUgmYe1wiFnNQe76vVc=; b=Nt1QflEypUasZCxjxOiT6ZgGmnsWH+zuqi8dMmDRQAq0/HzJOTiGBQif/aWFs9UjVB +VgGT7j1x/SjNjN+qw6zto/8bN+H4kDZ/CgKVGT5MqF2ELZs+vqlF9C5KscWggbS7aGB LNcrK0PLBdcbm4iM1tSLq+jRyqCLeU9aJRylHvnuvCQgLFJq/VdW4TrR/+2V54tZ/Bmp fxre7k4WNPwzMQjCVSoBX11MZSvj18uM1oZshv2+vF3g1wQGIuGQOuXSzde+m2GtfRmh QlKY3w6jwi586Af7S3zzuOFSf2MFd3L04Coz48xL8YgziOKHlTuOt4aqcIwy3t3y6t39 lFjg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1753097471; x=1753702271; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=7ynnYBaX+c5555v0h5W6O4GrGUgmYe1wiFnNQe76vVc=; b=PP4jz51HBE5oUMaGWkdAMPqhNit2sNBunsxCQqMaByITTwk+/pI1oI3F1yTCdZUMSJ /otd96dVXv9DGiqj69FvmPx5uosG5/zwwYdgDzt9bCDDO2LO3+aC7sTvFbTyJTfiIX+I Ap8TOK5yvOrLmxNomJOywnO6jetrk8GRlNQS8SykrMidFIOwM9kKC7c+v0vi39DhNHo3 uxymMJj6TxCNNOKbzHv4bQWQDBKbD3GCDPdyi1zOQpdvHufRzKKXNjSWUHmowKacequW KtxPALjg8gjzYCKV9KYPxaoX+fqevCq69/xbNEvTsTgoen6j+Nbh/92XoWbkzMdVIwex nvcw== X-Gm-Message-State: AOJu0YwZ/Q7C/qL35PuyPJko5W+cnbGgFJqjc+GyJkU5kZe8dcn+0IDZ CCgr5yireqYy3qnmagTsqxCku90STESdAWjaayR0HVhGw4LwcxtEw5dN8v5zJD2N X-Gm-Gg: ASbGncsPpB3hx49Q5fBEMgam+ga+yAPgqtDDS/cAjhy3WCFdbw0PYZDAPVHeo0EdH5I bLewLizktGKKx2PtQvBKtZ8kWI4uHEqJMDtJ5mHc5B80dDC3Jw+jwVN/QFrGVtvrRiw5wWdXdUM bWDIdOZPXWSDarEKZOzyEyZ5sF8tM+xtwEMCa+hfYcj+jIOsO5YMrRbs/Ld7LpA4jVHdsBAfoXz UceGmyHwkw27JYg3ieIf3Wfn1+QDh451rcS5QJPD4qnFf/uNEAf2FeQSZjWVsbI1quyAwWlwvON ydDRnPZcoNOe2+OvRMH3xnYA4Bsp3gHQ6dhfz1htlRKL+2OQrnHXa7GBPkadmIZ4j/GtIyKxz/M qRFhl2/aoapnCShevD+fB48VjLlp1tjSJ X-Google-Smtp-Source: AGHT+IG9WeGUKNk2JFGrspbBqUpE3GaT2SFxH1vVMSmfATUUroQiPfkPt0+NzSJjaQlhnNAlAKRz1A== X-Received: by 2002:a05:6a21:998b:b0:238:351a:f960 with SMTP id adf61e73a8af0-2390c84dd57mr27839499637.23.1753097470749; Mon, 21 Jul 2025 04:31:10 -0700 (PDT) Received: from localhost.localdomain ([5.34.218.160]) by smtp.gmail.com with ESMTPSA id d2e1a72fcca58-759cbd68350sm5630277b3a.148.2025.07.21.04.31.08 (version=TLS1_3 cipher=TLS_CHACHA20_POLY1305_SHA256 bits=256/256); Mon, 21 Jul 2025 04:31:10 -0700 (PDT) From: Jack Lau X-Google-Original-From: Jack Lau To: ffmpeg-devel@ffmpeg.org Date: Mon, 21 Jul 2025 19:30:22 +0800 Message-ID: <20250721113023.91931-12-jacklau1222@qq.com> X-Mailer: git-send-email 2.49.0 In-Reply-To: <20250721113023.91931-1-jacklau1222@qq.com> References: <20250721113023.91931-1-jacklau1222@qq.com> MIME-Version: 1.0 X-Unsent: 1 Subject: [FFmpeg-devel] [PATCH v4 11/11] avformat/whip: simplify and modularize the ICE and DTLS X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Jack Lau Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" Archived-At: List-Archive: List-Post: We try to perform dtls handshake when the ICE is totally done. Refer to RFC8445, When peer's ICE is lite, the peer won't trigged check so FFmpeg just send STUN request and receive response, then ICE is done. When peer's ICE is full, the peer will send STUN request after reponse FFmpeg's request to ensure candidate pair become valid in both directions. Then the peer does nomination, and ICE is done. Signed-off-by: Jack Lau --- libavformat/whip.c | 135 +++++++++++++++++++++------------------------ 1 file changed, 62 insertions(+), 73 deletions(-) diff --git a/libavformat/whip.c b/libavformat/whip.c index 4c8ed26cef..cfcb8e8888 100644 --- a/libavformat/whip.c +++ b/libavformat/whip.c @@ -373,19 +373,6 @@ static av_cold int certificate_key_init(AVFormatContext *s) return ret; } -static av_cold int dtls_initialize(AVFormatContext *s) -{ - WHIPContext *whip = s->priv_data; - /* reuse the udp created by whip */ - ff_tls_set_external_socket(whip->dtls_uc, whip->udp); - - /* Make the socket non-blocking */ - ff_socket_nonblock(ffurl_get_file_handle(whip->dtls_uc), 1); - whip->dtls_uc->flags |= AVIO_FLAG_NONBLOCK; - - return 0; -} - /** * Initialize and check the options for the WebRTC muxer. */ @@ -1232,14 +1219,12 @@ end: return ret; } -static int ice_dtls_handshake(AVFormatContext *s) +static int ice_handshake(AVFormatContext *s) { int ret = 0, size, i; int64_t starttime = av_gettime(), now; WHIPContext *whip = s->priv_data; int is_dtls_active = whip->flags & WHIP_FLAG_DTLS_ACTIVE; - AVDictionary *opts = NULL; - char buf[256], *cert_buf = NULL, *key_buf = NULL; if (whip->state < WHIP_STATE_UDP_CONNECTED || !whip->udp) { av_log(whip, AV_LOG_ERROR, "UDP not connected, state=%d, udp=%p\n", whip->state, whip->udp); @@ -1261,25 +1246,20 @@ static int ice_dtls_handshake(AVFormatContext *s) goto end; } - if (whip->state < WHIP_STATE_ICE_CONNECTING) - whip->state = WHIP_STATE_ICE_CONNECTING; + whip->state = WHIP_STATE_ICE_CONNECTING; } next_packet: - if (whip->state >= WHIP_STATE_DTLS_FINISHED) - /* DTLS handshake is done, exit the loop. */ - break; - now = av_gettime(); if (now - starttime >= whip->handshake_timeout * 1000) { - av_log(whip, AV_LOG_ERROR, "DTLS handshake timeout=%dms, cost=%dms, elapsed=%dms, state=%d\n", + av_log(whip, AV_LOG_ERROR, "ICE handshake timeout=%dms, cost=%dms, elapsed=%dms, state=%d\n", whip->handshake_timeout, ELAPSED(starttime, now), ELAPSED(whip->whip_starttime, now), whip->state); ret = AVERROR(ETIMEDOUT); goto end; } - /* Read the STUN or DTLS messages from peer. */ - for (i = 0; i < ICE_DTLS_READ_INTERVAL / 5 && whip->state < WHIP_STATE_ICE_CONNECTED; i++) { + /* Read the STUN or DTLS client hello from peer. */ + for (i = 0; i < ICE_DTLS_READ_INTERVAL / 5; i++) { ret = ffurl_read(whip->udp, whip->buf, sizeof(whip->buf)); if (ret > 0) break; @@ -1295,35 +1275,8 @@ next_packet: /* Handle the ICE binding response. */ if (ice_is_binding_response(whip->buf, ret)) { - if (whip->state < WHIP_STATE_ICE_CONNECTED) { - if (whip->is_peer_ice_lite) - whip->state = WHIP_STATE_ICE_CONNECTED; - whip->whip_ice_time = av_gettime(); - av_log(whip, AV_LOG_VERBOSE, "ICE STUN ok, state=%d, url=udp://%s:%d, location=%s, username=%s:%s, res=%dB, elapsed=%dms\n", - whip->state, whip->ice_host, whip->ice_port, whip->whip_resource_url ? whip->whip_resource_url : "", - whip->ice_ufrag_remote, whip->ice_ufrag_local, ret, ELAPSED(whip->whip_starttime, av_gettime())); - - ff_url_join(buf, sizeof(buf), "dtls", NULL, whip->ice_host, whip->ice_port, NULL); - av_dict_set_int(&opts, "mtu", whip->pkt_size, 0); - if (whip->cert_file) { - av_dict_set(&opts, "cert_file", whip->cert_file, 0); - } else - av_dict_set(&opts, "cert_pem", whip->cert_buf, 0); - - if (whip->key_file) { - av_dict_set(&opts, "key_file", whip->key_file, 0); - } else - av_dict_set(&opts, "key_pem", whip->key_buf, 0); - av_dict_set_int(&opts, "external_sock", 1, 0); - av_dict_set_int(&opts, "listen", is_dtls_active ? 0 : 1, 0); - /* If got the first binding response, start DTLS handshake. */ - ret = ffurl_open_whitelist(&whip->dtls_uc, buf, AVIO_FLAG_READ_WRITE, &s->interrupt_callback, - &opts, s->protocol_whitelist, s->protocol_blacklist, NULL); - av_dict_free(&opts); - if (ret < 0) - goto end; - dtls_initialize(s); - } + if (whip->is_peer_ice_lite) + whip->state = WHIP_STATE_ICE_CONNECTED; goto next_packet; } @@ -1334,29 +1287,62 @@ next_packet: goto next_packet; } - if ((is_dtls_packet(whip->buf, ret) || is_dtls_active) && whip->state >= WHIP_STATE_ICE_CONNECTED || whip->state == WHIP_STATE_ICE_CONNECTING) { + if (is_dtls_packet(whip->buf, ret) || whip->flags & WHIP_FLAG_DTLS_ACTIVE) { whip->state = WHIP_STATE_ICE_CONNECTED; - ret = ffurl_handshake(whip->dtls_uc); - if (ret < 0) { - whip->state = WHIP_STATE_FAILED; - av_log(whip, AV_LOG_VERBOSE, "DTLS session failed\n"); - goto end; - } - if (!ret) { - whip->state = WHIP_STATE_DTLS_FINISHED; - whip->whip_dtls_time = av_gettime(); - av_log(whip, AV_LOG_VERBOSE, "DTLS handshake is done, elapsed=%dms\n", - ELAPSED(whip->whip_starttime, whip->whip_dtls_time)); - } - goto next_packet; + ret = 0; + whip->whip_ice_time = av_gettime(); + av_log(whip, AV_LOG_VERBOSE, "ICE STUN ok, state=%d, url=udp://%s:%d, location=%s, username=%s:%s, res=%dB, elapsed=%dms\n", + whip->state, whip->ice_host, whip->ice_port, whip->whip_resource_url ? whip->whip_resource_url : "", + whip->ice_ufrag_remote, whip->ice_ufrag_local, ret, ELAPSED(whip->whip_starttime, av_gettime())); + break; } } +end: + return ret; +} +static int dtls_handshake(AVFormatContext *s) +{ + int ret = 0; + WHIPContext *whip = s->priv_data; + AVDictionary *opts = NULL; + char buf[256]; + + ff_url_join(buf, sizeof(buf), "dtls", NULL, whip->ice_host, whip->ice_port, NULL); + av_dict_set_int(&opts, "mtu", whip->pkt_size, 0); + if (whip->cert_file) { + av_dict_set(&opts, "cert_file", whip->cert_file, 0); + } else + av_dict_set(&opts, "cert_pem", whip->cert_buf, 0); + + if (whip->key_file) { + av_dict_set(&opts, "key_file", whip->key_file, 0); + } else + av_dict_set(&opts, "key_pem", whip->key_buf, 0); + av_dict_set_int(&opts, "external_sock", 1, 0); + av_dict_set_int(&opts, "listen", whip->flags & WHIP_FLAG_DTLS_ACTIVE ? 0 : 1, 0); + /* If got the first binding response, start DTLS handshake. */ + ret = ffurl_open_whitelist(&whip->dtls_uc, buf, AVIO_FLAG_READ_WRITE, &s->interrupt_callback, + &opts, s->protocol_whitelist, s->protocol_blacklist, NULL); + av_dict_free(&opts); + if (ret < 0) + goto end; + + /* reuse the udp created by whip */ + ff_tls_set_external_socket(whip->dtls_uc, whip->udp); + + ret = ffurl_handshake(whip->dtls_uc); + if (ret < 0) { + whip->state = WHIP_STATE_FAILED; + av_log(whip, AV_LOG_VERBOSE, "DTLS session failed\n"); + } + if (!ret) { + whip->state = WHIP_STATE_DTLS_FINISHED; + whip->whip_dtls_time = av_gettime(); + av_log(whip, AV_LOG_VERBOSE, "DTLS handshake is done, elapsed=%dms\n", + ELAPSED(whip->whip_starttime, whip->whip_dtls_time)); + } end: - if (cert_buf) - av_free(cert_buf); - if (key_buf) - av_free(key_buf); return ret; } @@ -1866,7 +1852,10 @@ static av_cold int whip_init(AVFormatContext *s) if ((ret = udp_connect(s)) < 0) goto end; - if ((ret = ice_dtls_handshake(s)) < 0) + if ((ret = ice_handshake(s)) < 0) + goto end; + + if ((ret = dtls_handshake(s)) < 0) goto end; if ((ret = setup_srtp(s)) < 0) -- 2.49.0 _______________________________________________ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".