From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from ffbox0-bg.ffmpeg.org (ffbox0-bg.ffmpeg.org [79.124.17.100]) by master.gitmailbox.com (Postfix) with ESMTPS id BC5154B956 for ; Tue, 22 Jul 2025 12:38:48 +0000 (UTC) Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.ffmpeg.org (Postfix) with ESMTP id 63A8868D181; Tue, 22 Jul 2025 15:37:22 +0300 (EEST) Received: from mail-pl1-f170.google.com (mail-pl1-f170.google.com [209.85.214.170]) by ffbox0-bg.ffmpeg.org (Postfix) with ESMTPS id EA13968C0D9 for ; Tue, 22 Jul 2025 15:37:20 +0300 (EEST) Received: by mail-pl1-f170.google.com with SMTP id d9443c01a7336-234b9dfb842so45114315ad.1 for ; Tue, 22 Jul 2025 05:37:20 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1753187839; x=1753792639; darn=ffmpeg.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=7ynnYBaX+c5555v0h5W6O4GrGUgmYe1wiFnNQe76vVc=; b=dwitn+y96QD3SbLldYHOoGcwqsuFRpnfH60kR01sRqzFye2YSDUbE6svGNR9uCN7d2 Oedy1WsDqOhBJfLEumlFEWT9+lBiDv+KCs1M/N9Zn3tgFL7Gu0syOslv6yI764PR6u4r CCaFoIAfstpPdGs6udEmglkTcPdZ20S6tl566QLWBUWAFJ1uotzD8yxq693GNjLVfA4k eDMQdkStL2pGIWEhXVgNN24DbF+IAJxIZjIUZ3+AyQTet4+bT02of7Dkljio78j9BPSp +xjly3dnhe6F21Ospr5Os3lO+1DoeVzJSknAsZiPPl9KbRejbFXKvMd8ltzIIBNoZL9P nVLg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1753187839; x=1753792639; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=7ynnYBaX+c5555v0h5W6O4GrGUgmYe1wiFnNQe76vVc=; b=gpTppJQT6G4+Jq+lgS9xnVrjrGnVeDSWYbCcZ09qKsD2515msWLfWUlasyYCEL7F5T xEEIFPoJK9AVHv6F5xvJENXK+dl3QwJsHLnAi9U+FSUuM0Kb9JUivZfhrbzIhVfydbZr rK8CyIreEQDr2ccpuAr6K3xzwSQQfJYjwNwmMFY9mwlgV30vlhYnXgyVaUmt8swwS++H LxgW2xbqksqp3bxmBaUYAy7i1YArsdLMVv/71O3ggccnfR2CdIrCmTotqsD2DYbt9pCt PAeHCL0sI9Q00h4uWg9GGqpeSa7JwtplpCUnqTbB74gQ62wo8lBr2hJgEfVbY8Mg+Jnq NfHA== X-Gm-Message-State: AOJu0YxYNvqeAoyC0ChgsPC8mk4HS69Khl0MV/v6ycbTqYKYkynbEJ8O A8O6RHst90Wwx1rt2K2OdoKqyTbuJ+VSCQfCVZn5lw6geYdH9tGccGr913fB2rhJ/Yk= X-Gm-Gg: ASbGncuD6POFLyNfXJMO9ay4WAJ1NlMn1TShLkS/b8FKaNhFtX0jAPfFNVlnMDwJ8VU eP0+pmQASo5Bpd6PiOQO2WXmZVJnjZSKDdK/25IyxgyMsCV7jMAqdo4RDVabY67fPqGwg24PXoB ZvSCi1M/9YxkDjk6jRiGUmrg/c2Z0YPekSsNC24g0+AFgkKEZi3TnHVAxViTw1TS7GI9Sqy7Tmy TWF34Z/1DDASUMhJxLY48hsgSBw+wxfh4TFPr9kiHrW3smhvgFXUh+X8/Lh66546os1J9g9fp1o PCwNOTXAoMs8zu1Y2ATlZSjN6XEzHZfdEm20iS7HeYSlS3mEl5WqpbnvXtyM+f33MITWZeuOBVX IsSXpN6TpgUeo1ZLBs/wjLmuR3B9TpVEjdLkCWK+6ZNwuevQIjel0mBhz X-Google-Smtp-Source: AGHT+IEErDZy/c9FHhtQlqGb7nL8DFxXtvdxO8II6L5SnH8ke2QyYbVWpT3EqcDRnsigCTv0efwl+w== X-Received: by 2002:a17:902:da8a:b0:23c:8f21:ac59 with SMTP id d9443c01a7336-23e2572b0c0mr355101165ad.29.1753187838908; Tue, 22 Jul 2025 05:37:18 -0700 (PDT) Received: from localhost.localdomain ([182.126.128.169]) by smtp.gmail.com with ESMTPSA id d9443c01a7336-23e3b60edbasm75740715ad.70.2025.07.22.05.37.07 (version=TLS1_3 cipher=TLS_CHACHA20_POLY1305_SHA256 bits=256/256); Tue, 22 Jul 2025 05:37:09 -0700 (PDT) From: Jack Lau X-Google-Original-From: Jack Lau To: ffmpeg-devel@ffmpeg.org Date: Tue, 22 Jul 2025 20:36:12 +0800 Message-ID: <20250722123616.53164-12-jacklau1222@qq.com> X-Mailer: git-send-email 2.49.0 In-Reply-To: <20250722123616.53164-1-jacklau1222@qq.com> References: <20250722123616.53164-1-jacklau1222@qq.com> MIME-Version: 1.0 X-Unsent: 1 Subject: [FFmpeg-devel] [PATCH v5 11/15] avformat/whip: simplify and modularize the ICE and DTLS X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Jack Lau Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" Archived-At: List-Archive: List-Post: We try to perform dtls handshake when the ICE is totally done. Refer to RFC8445, When peer's ICE is lite, the peer won't trigged check so FFmpeg just send STUN request and receive response, then ICE is done. When peer's ICE is full, the peer will send STUN request after reponse FFmpeg's request to ensure candidate pair become valid in both directions. Then the peer does nomination, and ICE is done. Signed-off-by: Jack Lau --- libavformat/whip.c | 135 +++++++++++++++++++++------------------------ 1 file changed, 62 insertions(+), 73 deletions(-) diff --git a/libavformat/whip.c b/libavformat/whip.c index 4c8ed26cef..cfcb8e8888 100644 --- a/libavformat/whip.c +++ b/libavformat/whip.c @@ -373,19 +373,6 @@ static av_cold int certificate_key_init(AVFormatContext *s) return ret; } -static av_cold int dtls_initialize(AVFormatContext *s) -{ - WHIPContext *whip = s->priv_data; - /* reuse the udp created by whip */ - ff_tls_set_external_socket(whip->dtls_uc, whip->udp); - - /* Make the socket non-blocking */ - ff_socket_nonblock(ffurl_get_file_handle(whip->dtls_uc), 1); - whip->dtls_uc->flags |= AVIO_FLAG_NONBLOCK; - - return 0; -} - /** * Initialize and check the options for the WebRTC muxer. */ @@ -1232,14 +1219,12 @@ end: return ret; } -static int ice_dtls_handshake(AVFormatContext *s) +static int ice_handshake(AVFormatContext *s) { int ret = 0, size, i; int64_t starttime = av_gettime(), now; WHIPContext *whip = s->priv_data; int is_dtls_active = whip->flags & WHIP_FLAG_DTLS_ACTIVE; - AVDictionary *opts = NULL; - char buf[256], *cert_buf = NULL, *key_buf = NULL; if (whip->state < WHIP_STATE_UDP_CONNECTED || !whip->udp) { av_log(whip, AV_LOG_ERROR, "UDP not connected, state=%d, udp=%p\n", whip->state, whip->udp); @@ -1261,25 +1246,20 @@ static int ice_dtls_handshake(AVFormatContext *s) goto end; } - if (whip->state < WHIP_STATE_ICE_CONNECTING) - whip->state = WHIP_STATE_ICE_CONNECTING; + whip->state = WHIP_STATE_ICE_CONNECTING; } next_packet: - if (whip->state >= WHIP_STATE_DTLS_FINISHED) - /* DTLS handshake is done, exit the loop. */ - break; - now = av_gettime(); if (now - starttime >= whip->handshake_timeout * 1000) { - av_log(whip, AV_LOG_ERROR, "DTLS handshake timeout=%dms, cost=%dms, elapsed=%dms, state=%d\n", + av_log(whip, AV_LOG_ERROR, "ICE handshake timeout=%dms, cost=%dms, elapsed=%dms, state=%d\n", whip->handshake_timeout, ELAPSED(starttime, now), ELAPSED(whip->whip_starttime, now), whip->state); ret = AVERROR(ETIMEDOUT); goto end; } - /* Read the STUN or DTLS messages from peer. */ - for (i = 0; i < ICE_DTLS_READ_INTERVAL / 5 && whip->state < WHIP_STATE_ICE_CONNECTED; i++) { + /* Read the STUN or DTLS client hello from peer. */ + for (i = 0; i < ICE_DTLS_READ_INTERVAL / 5; i++) { ret = ffurl_read(whip->udp, whip->buf, sizeof(whip->buf)); if (ret > 0) break; @@ -1295,35 +1275,8 @@ next_packet: /* Handle the ICE binding response. */ if (ice_is_binding_response(whip->buf, ret)) { - if (whip->state < WHIP_STATE_ICE_CONNECTED) { - if (whip->is_peer_ice_lite) - whip->state = WHIP_STATE_ICE_CONNECTED; - whip->whip_ice_time = av_gettime(); - av_log(whip, AV_LOG_VERBOSE, "ICE STUN ok, state=%d, url=udp://%s:%d, location=%s, username=%s:%s, res=%dB, elapsed=%dms\n", - whip->state, whip->ice_host, whip->ice_port, whip->whip_resource_url ? whip->whip_resource_url : "", - whip->ice_ufrag_remote, whip->ice_ufrag_local, ret, ELAPSED(whip->whip_starttime, av_gettime())); - - ff_url_join(buf, sizeof(buf), "dtls", NULL, whip->ice_host, whip->ice_port, NULL); - av_dict_set_int(&opts, "mtu", whip->pkt_size, 0); - if (whip->cert_file) { - av_dict_set(&opts, "cert_file", whip->cert_file, 0); - } else - av_dict_set(&opts, "cert_pem", whip->cert_buf, 0); - - if (whip->key_file) { - av_dict_set(&opts, "key_file", whip->key_file, 0); - } else - av_dict_set(&opts, "key_pem", whip->key_buf, 0); - av_dict_set_int(&opts, "external_sock", 1, 0); - av_dict_set_int(&opts, "listen", is_dtls_active ? 0 : 1, 0); - /* If got the first binding response, start DTLS handshake. */ - ret = ffurl_open_whitelist(&whip->dtls_uc, buf, AVIO_FLAG_READ_WRITE, &s->interrupt_callback, - &opts, s->protocol_whitelist, s->protocol_blacklist, NULL); - av_dict_free(&opts); - if (ret < 0) - goto end; - dtls_initialize(s); - } + if (whip->is_peer_ice_lite) + whip->state = WHIP_STATE_ICE_CONNECTED; goto next_packet; } @@ -1334,29 +1287,62 @@ next_packet: goto next_packet; } - if ((is_dtls_packet(whip->buf, ret) || is_dtls_active) && whip->state >= WHIP_STATE_ICE_CONNECTED || whip->state == WHIP_STATE_ICE_CONNECTING) { + if (is_dtls_packet(whip->buf, ret) || whip->flags & WHIP_FLAG_DTLS_ACTIVE) { whip->state = WHIP_STATE_ICE_CONNECTED; - ret = ffurl_handshake(whip->dtls_uc); - if (ret < 0) { - whip->state = WHIP_STATE_FAILED; - av_log(whip, AV_LOG_VERBOSE, "DTLS session failed\n"); - goto end; - } - if (!ret) { - whip->state = WHIP_STATE_DTLS_FINISHED; - whip->whip_dtls_time = av_gettime(); - av_log(whip, AV_LOG_VERBOSE, "DTLS handshake is done, elapsed=%dms\n", - ELAPSED(whip->whip_starttime, whip->whip_dtls_time)); - } - goto next_packet; + ret = 0; + whip->whip_ice_time = av_gettime(); + av_log(whip, AV_LOG_VERBOSE, "ICE STUN ok, state=%d, url=udp://%s:%d, location=%s, username=%s:%s, res=%dB, elapsed=%dms\n", + whip->state, whip->ice_host, whip->ice_port, whip->whip_resource_url ? whip->whip_resource_url : "", + whip->ice_ufrag_remote, whip->ice_ufrag_local, ret, ELAPSED(whip->whip_starttime, av_gettime())); + break; } } +end: + return ret; +} +static int dtls_handshake(AVFormatContext *s) +{ + int ret = 0; + WHIPContext *whip = s->priv_data; + AVDictionary *opts = NULL; + char buf[256]; + + ff_url_join(buf, sizeof(buf), "dtls", NULL, whip->ice_host, whip->ice_port, NULL); + av_dict_set_int(&opts, "mtu", whip->pkt_size, 0); + if (whip->cert_file) { + av_dict_set(&opts, "cert_file", whip->cert_file, 0); + } else + av_dict_set(&opts, "cert_pem", whip->cert_buf, 0); + + if (whip->key_file) { + av_dict_set(&opts, "key_file", whip->key_file, 0); + } else + av_dict_set(&opts, "key_pem", whip->key_buf, 0); + av_dict_set_int(&opts, "external_sock", 1, 0); + av_dict_set_int(&opts, "listen", whip->flags & WHIP_FLAG_DTLS_ACTIVE ? 0 : 1, 0); + /* If got the first binding response, start DTLS handshake. */ + ret = ffurl_open_whitelist(&whip->dtls_uc, buf, AVIO_FLAG_READ_WRITE, &s->interrupt_callback, + &opts, s->protocol_whitelist, s->protocol_blacklist, NULL); + av_dict_free(&opts); + if (ret < 0) + goto end; + + /* reuse the udp created by whip */ + ff_tls_set_external_socket(whip->dtls_uc, whip->udp); + + ret = ffurl_handshake(whip->dtls_uc); + if (ret < 0) { + whip->state = WHIP_STATE_FAILED; + av_log(whip, AV_LOG_VERBOSE, "DTLS session failed\n"); + } + if (!ret) { + whip->state = WHIP_STATE_DTLS_FINISHED; + whip->whip_dtls_time = av_gettime(); + av_log(whip, AV_LOG_VERBOSE, "DTLS handshake is done, elapsed=%dms\n", + ELAPSED(whip->whip_starttime, whip->whip_dtls_time)); + } end: - if (cert_buf) - av_free(cert_buf); - if (key_buf) - av_free(key_buf); return ret; } @@ -1866,7 +1852,10 @@ static av_cold int whip_init(AVFormatContext *s) if ((ret = udp_connect(s)) < 0) goto end; - if ((ret = ice_dtls_handshake(s)) < 0) + if ((ret = ice_handshake(s)) < 0) + goto end; + + if ((ret = dtls_handshake(s)) < 0) goto end; if ((ret = setup_srtp(s)) < 0) -- 2.49.0 _______________________________________________ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".