[FFmpeg-devel] [PATCH 0/4] Fix some issues in tls_openssl and udp

[FFmpeg-devel] [PATCH 0/4] Fix some issues in tls_openssl and udp

[Jack Lau]

This patchset aims to fix some issues when i try to utilize DTLS using avio.
I create a simple DTLS client and server case here
https://github.com/JackLau1222/openssl-dtls-bio-example/tree/master/ffmpeg_case

This patchset fix:
1. dtls_handshake can't return positive code when it still in progressing
2. udp server mode haven't dest_addr so we need set it through last_recv_addr
3. some code cleanup

This patchset depends Timo's latest schannel patchset
More details: https://github.com/BtbN/FFmpeg/pull/3

Jack Lau (4):
  avformat/tls_openssl: add record trace function
  avformat/tls_openssl: fix dtls_handshake return code
  avformat/tls_openssl: remove all redundant "TLS: " in log with AVClass
  avformat/udp: fix udp server mode haven't dest_addr

 libavformat/tls_openssl.c | 78 +++++++++++++++++++++++++++++++--------
 libavformat/udp.c         |  2 +
 2 files changed, 64 insertions(+), 16 deletions(-)

-- 
2.49.0

_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".

[FFmpeg-devel] [PATCH 1/4] avformat/tls_openssl: add record trace function

[Jack Lau]

[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #1: Type: text/plain; charset=y, Size: 3093 bytes --]

Signed-off-by: Jack Lau <jacklau1222@qq.com>
---
 libavformat/tls_openssl.c | 51 +++++++++++++++++++++++++++++++++++++--
 1 file changed, 49 insertions(+), 2 deletions(-)

diff --git a/libavformat/tls_openssl.c b/libavformat/tls_openssl.c
index 2a01fb387d..8639ac9758 100644
--- a/libavformat/tls_openssl.c
+++ b/libavformat/tls_openssl.c
@@ -20,6 +20,7 @@
  * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA
  */
 
+#include "libavutil/intreadwrite.h"
 #include "libavutil/mem.h"
 #include "network.h"
 #include "os_support.h"
@@ -559,6 +560,48 @@ static int tls_close(URLContext *h)
     return 0;
 }
 
+/*
+ * Trace a single TLS/DTLS record.
+ * 
+ * See RFC 5246 Section 6.2.1, RFC 6347 Section 4.1
+ * 
+ * @param data     Raw record (network byte‑order).
+ * @param length   Size of @data in bytes.
+ * @param incoming Non‑zero when the packet was received, zero when sent.
+ */
+static void openssl_state_trace(uint8_t *data, int length, int incoming)
+{
+    uint8_t  content_type   = 0;  /* TLS/DTLS ContentType       */
+    uint16_t record_length  = 0;  /* Length field from header   */
+    uint8_t  handshake_type = 0;  /* First byte of Handshake msg */
+    int is_dtls = 0;
+
+    /* ContentType is always the very first byte */
+    if (length >= 1)
+        content_type = AV_RB8(&data[0]);
+    if (length >= 3 && data[1] == DTLS1_VERSION_MAJOR)
+        is_dtls = 1;
+    /* TLS header is 5 bytes, DTLS header is 13 bytes */
+    if (length >= 13 && is_dtls)
+        record_length = AV_RB16(&data[11]);
+    else if (length >= 5 && !is_dtls)
+        record_length = AV_RB16(&data[3]);
+    /*
+     * HandshakeType values (TLS 1.0–1.2, DTLS 1.0/1.2)
+     * See RFC 5246 Section 7.4, RFC 6347 Section 4.2
+     *
+     * Only present when ContentType == handshake(22)
+     */
+    if (content_type == 22) {
+        int hs_off = is_dtls ? 13 : 5;
+        if (length > hs_off)
+            handshake_type = AV_RB8(&data[hs_off]);
+    }
+
+    av_log(NULL, AV_LOG_TRACE ,"TLS: Trace %s, len=%u, cnt=%u, size=%u, hs=%u\n",
+        (incoming? "RECV":"SEND"), length, content_type, record_length, handshake_type);
+}
+
 static int url_bio_create(BIO *b)
 {
     BIO_set_init(b, 1);
@@ -576,8 +619,10 @@ static int url_bio_bread(BIO *b, char *buf, int len)
 {
     TLSContext *c = BIO_get_data(b);
     int ret = ffurl_read(c->tls_shared.is_dtls ? c->tls_shared.udp : c->tls_shared.tcp, buf, len);
-    if (ret >= 0)
+    if (ret >= 0) {
+        openssl_state_trace((uint8_t*)buf, ret, 1);
         return ret;
+    }
     BIO_clear_retry_flags(b);
     if (ret == AVERROR_EXIT)
         return 0;
@@ -592,8 +637,10 @@ static int url_bio_bwrite(BIO *b, const char *buf, int len)
 {
     TLSContext *c = BIO_get_data(b);
     int ret = ffurl_write(c->tls_shared.is_dtls ? c->tls_shared.udp : c->tls_shared.tcp, buf, len);
-    if (ret >= 0)
+    if (ret >= 0) {
+        openssl_state_trace((uint8_t*)buf, ret, 0);
         return ret;
+    }
     BIO_clear_retry_flags(b);
     if (ret == AVERROR_EXIT)
         return 0;
-- 
2.49.0


[-- Attachment #2: Type: text/plain, Size: 251 bytes --]

_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".

[FFmpeg-devel] [PATCH 2/4] avformat/tls_openssl: fix dtls_handshake return code

[Jack Lau]

If the handshake is still in progress, dtls_handshake should
return a positive status code.

Signed-off-by: Jack Lau <jacklau1222@qq.com>
---
 libavformat/tls_openssl.c | 7 +++----
 1 file changed, 3 insertions(+), 4 deletions(-)

diff --git a/libavformat/tls_openssl.c b/libavformat/tls_openssl.c
index 8639ac9758..ffd9cd51d2 100644
--- a/libavformat/tls_openssl.c
+++ b/libavformat/tls_openssl.c
@@ -716,15 +716,14 @@ static int openssl_dtls_verify_callback(int preverify_ok, X509_STORE_CTX *ctx)
 
 static int dtls_handshake(URLContext *h)
 {
-    int ret = 0, r0, r1;
+    int ret = EINPROGRESS, r0, r1;
     TLSContext *p = h->priv_data;
 
     r0 = SSL_do_handshake(p->ssl);
     r1 = SSL_get_error(p->ssl, r0);
     if (r0 <= 0) {
         if (r1 != SSL_ERROR_WANT_READ && r1 != SSL_ERROR_WANT_WRITE && r1 != SSL_ERROR_ZERO_RETURN) {
-            av_log(p, AV_LOG_ERROR, "TLS: Read failed, r0=%d, r1=%d %s\n", r0, r1, openssl_get_error(p));
-            ret = AVERROR(EIO);
+            ret = print_ssl_error(h, r1);
             goto end;
         }
     } else {
@@ -734,7 +733,7 @@ static int dtls_handshake(URLContext *h)
     /* Check whether the DTLS is completed. */
     if (SSL_is_init_finished(p->ssl) != 1)
         goto end;
-
+    ret = 0;
     p->tls_shared.state = DTLS_STATE_FINISHED;
 end:
     return ret;
-- 
2.49.0

_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] [PATCH 2/4] avformat/tls_openssl: fix dtls_handshake return code

[Timo Rothenpieler]

On 09/07/2025 15:36, Jack Lau wrote:
> If the handshake is still in progress, dtls_handshake should
> return a positive status code.

Shouldn't dtls_open/start also be calling it in a loop then?
I don't think it's expected that you might be needed to call the 
handshake function in a loop after a urlcontext was successfully opened.

What I've done for the schannel implementation is force nonblocking off 
for the handshake, since there is just no good way to perform it in a 
nonblocking way, and you just always end up looping until it's done anyway.

> Signed-off-by: Jack Lau <jacklau1222@qq.com>
> ---
>   libavformat/tls_openssl.c | 7 +++----
>   1 file changed, 3 insertions(+), 4 deletions(-)
> 
> diff --git a/libavformat/tls_openssl.c b/libavformat/tls_openssl.c
> index 8639ac9758..ffd9cd51d2 100644
> --- a/libavformat/tls_openssl.c
> +++ b/libavformat/tls_openssl.c
> @@ -716,15 +716,14 @@ static int openssl_dtls_verify_callback(int preverify_ok, X509_STORE_CTX *ctx)
>   
>   static int dtls_handshake(URLContext *h)
>   {
> -    int ret = 0, r0, r1;
> +    int ret = EINPROGRESS, r0, r1;
>       TLSContext *p = h->priv_data;
>   
>       r0 = SSL_do_handshake(p->ssl);
>       r1 = SSL_get_error(p->ssl, r0);
>       if (r0 <= 0) {
>           if (r1 != SSL_ERROR_WANT_READ && r1 != SSL_ERROR_WANT_WRITE && r1 != SSL_ERROR_ZERO_RETURN) {
> -            av_log(p, AV_LOG_ERROR, "TLS: Read failed, r0=%d, r1=%d %s\n", r0, r1, openssl_get_error(p));
> -            ret = AVERROR(EIO);
> +            ret = print_ssl_error(h, r1);
>               goto end;
>           }
>       } else {
> @@ -734,7 +733,7 @@ static int dtls_handshake(URLContext *h)
>       /* Check whether the DTLS is completed. */
>       if (SSL_is_init_finished(p->ssl) != 1)
>           goto end;
> -
> +    ret = 0;
>       p->tls_shared.state = DTLS_STATE_FINISHED;
>   end:
>       return ret;

_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".

[FFmpeg-devel] [PATCH 3/4] avformat/tls_openssl: remove all redundant "TLS: " in log with AVClass

[Jack Lau]

Signed-off-by: Jack Lau <jacklau1222@qq.com>
---
 libavformat/tls_openssl.c | 20 ++++++++++----------
 1 file changed, 10 insertions(+), 10 deletions(-)

diff --git a/libavformat/tls_openssl.c b/libavformat/tls_openssl.c
index ffd9cd51d2..a519c8c880 100644
--- a/libavformat/tls_openssl.c
+++ b/libavformat/tls_openssl.c
@@ -509,7 +509,7 @@ int ff_dtls_export_materials(URLContext *h, char *dtls_srtp_materials, size_t ma
     ret = SSL_export_keying_material(c->ssl, dtls_srtp_materials, materials_sz,
         dst, strlen(dst), NULL, 0, 0);
     if (!ret) {
-        av_log(c, AV_LOG_ERROR, "TLS: Failed to export SRTP material, %s\n", openssl_get_error(c));
+        av_log(c, AV_LOG_ERROR, "Failed to export SRTP material, %s\n", openssl_get_error(c));
         return -1;
     }
     return 0;
@@ -727,7 +727,7 @@ static int dtls_handshake(URLContext *h)
             goto end;
         }
     } else {
-        av_log(p, AV_LOG_TRACE, "TLS: Read %d bytes, r0=%d, r1=%d\n", r0, r0, r1);
+        av_log(p, AV_LOG_TRACE, "Read %d bytes, r0=%d, r1=%d\n", r0, r0, r1);
     }
 
     /* Check whether the DTLS is completed. */
@@ -768,7 +768,7 @@ static av_cold int openssl_init_ca_key_cert(URLContext *h)
             return ret;
         }
     } else if (c->is_dtls){
-        av_log(p, AV_LOG_ERROR, "TLS: Init cert failed, %s\n", openssl_get_error(p));
+        av_log(p, AV_LOG_ERROR, "Init cert failed, %s\n", openssl_get_error(p));
         ret = AVERROR(EINVAL);
         goto fail;
     }
@@ -784,12 +784,12 @@ static av_cold int openssl_init_ca_key_cert(URLContext *h)
     } else if (c->key_buf) {
         p->pkey = pkey = pkey_from_pem_string(c->key_buf, 1);
         if (SSL_CTX_use_PrivateKey(p->ctx, pkey) != 1) {
-            av_log(p, AV_LOG_ERROR, "TLS: Init SSL_CTX_use_PrivateKey failed, %s\n", openssl_get_error(p));
+            av_log(p, AV_LOG_ERROR, "Init SSL_CTX_use_PrivateKey failed, %s\n", openssl_get_error(p));
             ret = AVERROR(EINVAL);
             return ret;
         }
     } else if (c->is_dtls) {
-        av_log(p, AV_LOG_ERROR, "TLS: Init pkey failed, %s\n", openssl_get_error(p));
+        av_log(p, AV_LOG_ERROR, "Init pkey failed, %s\n", openssl_get_error(p));
         ret = AVERROR(EINVAL);
         goto fail;
     }
@@ -826,7 +826,7 @@ static int dtls_start(URLContext *h, const char *url, int flags, AVDictionary **
 
     /* For ECDSA, we could set the curves list. */
     if (SSL_CTX_set1_curves_list(p->ctx, curves) != 1) {
-        av_log(p, AV_LOG_ERROR, "TLS: Init SSL_CTX_set1_curves_list failed, curves=%s, %s\n",
+        av_log(p, AV_LOG_ERROR, "Init SSL_CTX_set1_curves_list failed, curves=%s, %s\n",
             curves, openssl_get_error(p));
         ret = AVERROR(EINVAL);
         return ret;
@@ -837,7 +837,7 @@ static int dtls_start(URLContext *h, const char *url, int flags, AVDictionary **
      * ensuring maximum compatibility.
      */
     if (SSL_CTX_set_cipher_list(p->ctx, ciphers) != 1) {
-        av_log(p, AV_LOG_ERROR, "TLS: Init SSL_CTX_set_cipher_list failed, ciphers=%s, %s\n",
+        av_log(p, AV_LOG_ERROR, "Init SSL_CTX_set_cipher_list failed, ciphers=%s, %s\n",
             ciphers, openssl_get_error(p));
         ret = AVERROR(EINVAL);
         return ret;
@@ -854,7 +854,7 @@ static int dtls_start(URLContext *h, const char *url, int flags, AVDictionary **
     SSL_CTX_set_read_ahead(p->ctx, 1);
     /* Setup the SRTP context */
     if (SSL_CTX_set_tlsext_use_srtp(p->ctx, profiles)) {
-        av_log(p, AV_LOG_ERROR, "TLS: Init SSL_CTX_set_tlsext_use_srtp failed, profiles=%s, %s\n",
+        av_log(p, AV_LOG_ERROR, "Init SSL_CTX_set_tlsext_use_srtp failed, profiles=%s, %s\n",
             profiles, openssl_get_error(p));
         ret = AVERROR(EINVAL);
         return ret;
@@ -906,12 +906,12 @@ static int dtls_start(URLContext *h, const char *url, int flags, AVDictionary **
         ret = dtls_handshake(h);
         // Fatal SSL error, for example, no available suite when peer is DTLS 1.0 while we are DTLS 1.2.
         if (ret < 0) {
-            av_log(p, AV_LOG_ERROR, "TLS: Failed to drive SSL context, ret=%d\n", ret);
+            av_log(p, AV_LOG_ERROR, "Failed to drive SSL context, ret=%d\n", ret);
             return AVERROR(EIO);
         }
     }
 
-    av_log(p, AV_LOG_VERBOSE, "TLS: Setup ok, MTU=%d\n", p->tls_shared.mtu);
+    av_log(p, AV_LOG_VERBOSE, "Setup ok, MTU=%d\n", p->tls_shared.mtu);
 
     ret = 0;
 fail:
-- 
2.49.0

_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".

[FFmpeg-devel] [PATCH 4/4] avformat/udp: fix udp server mode haven't dest_addr

[Jack Lau]

If udp is in server mode(init local addr and port through url),
then it maybe haven't dest_addr, so we should set it after udp_read
get the client addr and port

Signed-off-by: Jack Lau <jacklau1222@qq.com>
---
 libavformat/udp.c | 2 ++
 1 file changed, 2 insertions(+)

diff --git a/libavformat/udp.c b/libavformat/udp.c
index 0fde3548e7..6a2ed2cdcd 100644
--- a/libavformat/udp.c
+++ b/libavformat/udp.c
@@ -1144,6 +1144,8 @@ static int udp_write(URLContext *h, const uint8_t *buf, int size)
     }
 
     if (!s->is_connected) {
+        if (!s->dest_addr_len && !s->dest_addr.ss_family)
+            ff_udp_get_last_recv_addr(h, &s->dest_addr, &s->dest_addr_len);
         ret = sendto (s->udp_fd, buf, size, 0,
                       (struct sockaddr *) &s->dest_addr,
                       s->dest_addr_len);
-- 
2.49.0

_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] [PATCH 4/4] avformat/udp: fix udp server mode haven't dest_addr

[Timo Rothenpieler]

On 09/07/2025 15:36, Jack Lau wrote:
> If udp is in server mode(init local addr and port through url),
> then it maybe haven't dest_addr, so we should set it after udp_read
> get the client addr and port

I'm also really not sure if this is correct, or what scenario it even fixes.
The vast majority of uses of UDP are strictly unidirectional, making 
this a non-issue.

DTLS is one of the few exceptions there, and as the user of udp.c should 
be the one responsible for setting the appropriate destination address 
when in "server" mode.
_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".