From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from ffbox0-bg.ffmpeg.org (ffbox0-bg.ffmpeg.org [79.124.17.100]) by master.gitmailbox.com (Postfix) with ESMTPS id 33E214D2C0 for ; Sun, 16 Nov 2025 06:50:29 +0000 (UTC) Authentication-Results: ffbox; dkim=fail (body hash mismatch (got b'nfULyoWaagLhVeJD9X4UxIhN0RCWM9Supmcx+bJ3PXw=', expected b'DmFQo5b9AUwCX2LEElNBq7gtd1Ln+bhXgS2AvE7D2YI=')) header.d=ffmpeg.org header.i=@ffmpeg.org header.a=rsa-sha256 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ffmpeg.org; i=@ffmpeg.org; q=dns/txt; s=mail; t=1763275775; h=mime-version : to : date : message-id : reply-to : subject : list-id : list-archive : list-archive : list-help : list-owner : list-post : list-subscribe : list-unsubscribe : from : cc : content-type : content-transfer-encoding : from; bh=nfULyoWaagLhVeJD9X4UxIhN0RCWM9Supmcx+bJ3PXw=; b=sxnqwUBoc1Ho7f2ETIiC88uUNkn3dK5JnfAn/WWe1O/u/wWKv0WJ8629AnXnYYTAuwRhn US+zmQEc0tn0QPYaAsiWUBz6tqORDzniVfUNG1mDfmbfxxXLqrmQ+tHd1pcNkusD6twdHmE H/MkE64blWvcxx7RgSGhV4ZqhY3GrCq/WwN4hDxutx4s1ZgxPMd10D2LWUuaGA4Pz8hVek+ sgArZ42ntkuJbeHoYtz3OLsPSpeXgQmQoTKPu9ITH5jXv8Y/zwKUdcru+CGsSxaJOQFScsR YJbqYM4PU4rngtJRqfEMSZQ30gA5c2IsYmd6dvsXYQCqGZQ+TlZifND9zrzg== Received: from [172.19.0.2] (unknown [172.19.0.2]) by ffbox0-bg.ffmpeg.org (Postfix) with ESMTP id 14BCC68FE44; Sun, 16 Nov 2025 08:49:35 +0200 (EET) ARC-Seal: i=1; cv=none; a=rsa-sha256; d=ffmpeg.org; s=arc; t=1763275700; b=U3wDhK2yn8GvlGwbf4TGPdlLyFOcQjLsFeQmzD69p+jGiqCu69+H05GY5G3QcdloeG+gT qqBeozYayVmc1vAKy25f+L4qyoK4EJ8A+5jBsnX1PvWfLJ32SThhFw9JY12qu+yKwLqGdnz 73bUELh75WNuLSyI9r/v5QsFjaJwOUi5dfFpfgSaBh3vW0x5O/d867wTbMzkAe9P2g8HnZw DUP6KmqjTSYGrc/UEwHGcjCoSMiO/2XR2fp/1WLFAx078hmbShcZvE3umlG9v1zvniKeG3n tpiura3CHDh9HM7fjdticzCOVGEDx+K1/mpExCUVuXJ6Qr+XwbEd/CigwJfg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=ffmpeg.org; s=arc; t=1763275700; h=from : sender : reply-to : subject : date : message-id : to : cc : mime-version : content-type : content-transfer-encoding : content-id : content-description : resent-date : resent-from : resent-sender : resent-to : resent-cc : resent-message-id : in-reply-to : references : list-id : list-help : list-unsubscribe : list-subscribe : list-post : list-owner : list-archive; bh=BCH2rDOk5VPX+JJht0S/epxh++1DOMGT6hK+Xon1DxA=; b=VWRpmAoINnxZhcfSE86tXkuJsrD7oHDIJEh99Mseqru346ddiAsMkjYseps36eDPukhaU kJSYP8TWwDx9NX13JHNmsxGxGodBUXUMaAY88eivAOZBxrzDWs2uAM1OCSnU6VbDTHIuQtu TWYrLLQ6fCl6SvN2t7Icl/IYBLxIA0blgomccel14qz5p3Vjil/bycUunKpylPM+b+9kSj9 b1AvYtcVdvNDLXurHUzoHDUo+gkM/3M2GsZhSIuwLPNn697lmwtWUvyweGUuJDU7Abo5FgL RceYB7U73Q4dGGshO0K6lyyJzQuTsEYFh9DtMBMSaG7kEptvj6zuLlx8Cd6g== ARC-Authentication-Results: i=1; ffmpeg.org; dkim=pass header.d=ffmpeg.org header.i=@ffmpeg.org; arc=none; dmarc=pass header.from=ffmpeg.org policy.dmarc=quarantine Authentication-Results: ffmpeg.org; dkim=pass header.d=ffmpeg.org header.i=@ffmpeg.org; arc=none (Message is not ARC signed); dmarc=pass (Used From Domain Record) header.from=ffmpeg.org policy.dmarc=quarantine DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ffmpeg.org; i=@ffmpeg.org; q=dns/txt; s=mail; t=1763275683; h=content-type : mime-version : content-transfer-encoding : from : to : reply-to : subject : date : from; bh=DmFQo5b9AUwCX2LEElNBq7gtd1Ln+bhXgS2AvE7D2YI=; b=NiOaDJT0MXaM+L4zp19UPvr7g2Wapu91fl5OfBkmjk/7vYc3rU4B7kX3M64nRCvbF5PyP OcXNQzEKuNmsbYSVmV6JxKH6ViIApHi92tnJIcVlPoH/4W2n1PKEXWvteuICIDjRw+76wEC GxPyXLTRSQps7dNl23gKkbvvmT1RMn9bUGBa1HSndxlW/VSOyJ5fjQ+r7ySizdQo7L1U5jH LhZKpYP+sT7Jdr9BS7Zd04L+5VhCfhDIJd5PqlzxTAv8SlLhFdNNB2a2o879ZOzZUU8tJcL vh9qB3etqDTobARCnTokYKTUL3ye5FRl7hYbZuJjovXfCdz3KoHbxHv2AVOg== Received: from 188d6d40ca7a (code.ffmpeg.org [188.245.149.3]) by ffbox0-bg.ffmpeg.org (Postfix) with ESMTPS id 5728A68FA65 for ; Sun, 16 Nov 2025 08:48:03 +0200 (EET) MIME-Version: 1.0 To: ffmpeg-devel@ffmpeg.org Date: Sun, 16 Nov 2025 06:48:02 -0000 Message-ID: <176327568386.25.14677780896399176895@2cb04c0e5124> Message-ID-Hash: U5IZ6MPNB2QSB57URK3Z5GGPS7UYWGQ5 X-Message-ID-Hash: U5IZ6MPNB2QSB57URK3Z5GGPS7UYWGQ5 X-MailFrom: code@ffmpeg.org X-Mailman-Rule-Hits: nonmember-moderation X-Mailman-Rule-Misses: dmarc-mitigation; no-senders; approved; loop; banned-address; header-match-ffmpeg-devel.ffmpeg.org-0; header-match-ffmpeg-devel.ffmpeg.org-1; header-match-ffmpeg-devel.ffmpeg.org-2; header-match-ffmpeg-devel.ffmpeg.org-3; emergency; member-moderation X-Mailman-Version: 3.3.10 Precedence: list Reply-To: FFmpeg development discussions and patches Subject: [FFmpeg-devel] [PATCH] avformat/whip: fix weird code that clear extradata in rtp muxer (PR #20931) List-Id: FFmpeg development discussions and patches Archived-At: Archived-At: List-Archive: List-Archive: List-Help: List-Owner: List-Post: List-Subscribe: List-Unsubscribe: From: Jack Lau via ffmpeg-devel Cc: Jack Lau Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Archived-At: List-Archive: List-Post: PR #20931 opened by Jack Lau (JackLau) URL: https://code.ffmpeg.org/FFmpeg/FFmpeg/pulls/20931 Patch URL: https://code.ffmpeg.org/FFmpeg/FFmpeg/pulls/20931.patch WHIP force the use of Annex B by enabling h264_mp4toannexb BSF in whip_check_bitstream. However, this BSF runs during packet writing and does not modify codecpar, so RTP muxer would still detect the stream as avcC. Add video_par_mp4toannexb function converts codecpar->extradata to Annex B to avoid this issue. Note: This function only modify the whip->video_par(copy from stream) to ensure RTP muxer detects format correctly, actual stream packet still converted by BSF(set in whip_check_bitstream). Signed-off-by: Jack Lau >>From f1ff61bf97a6658d14fb3877e6b96765d27d06c0 Mon Sep 17 00:00:00 2001 From: Jack Lau Date: Sun, 16 Nov 2025 13:51:46 +0800 Subject: [PATCH 1/2] avformat/whip: simplify the whip_check_bitstream function No need to detect whether sps and pps is in band of pkt, h264_annexb_insert_sps_pps function will handle all case. Add is_avcc and is_annexb function for later patch. Signed-off-by: Jack Lau --- libavformat/whip.c | 19 ++++++++++++------- 1 file changed, 12 insertions(+), 7 deletions(-) diff --git a/libavformat/whip.c b/libavformat/whip.c index 6a9b208f69..8e8ae90115 100644 --- a/libavformat/whip.c +++ b/libavformat/whip.c @@ -346,6 +346,15 @@ static int is_dtls_packet(uint8_t *b, int size) { (version == DTLS_VERSION_10 || version == DTLS_VERSION_12); } +static int is_avcc(uint8_t *b, int size) +{ + return size > 0 && b[0] == 0x01; +} + + static int is_annexb(uint8_t *b, int size) +{ + return (size >= 3 && AV_RB32(b) == 0x01) || ( size >= 4 && AV_RB24(b) == 0x01); +} /** * Get or Generate a self-signed certificate and private key for DTLS, @@ -2010,17 +2019,13 @@ static av_cold void whip_deinit(AVFormatContext *s) static int whip_check_bitstream(AVFormatContext *s, AVStream *st, const AVPacket *pkt) { - int ret = 1, extradata_isom = 0; - uint8_t *b = pkt->data; + int ret = 1; WHIPContext *whip = s->priv_data; if (st->codecpar->codec_id == AV_CODEC_ID_H264) { - extradata_isom = st->codecpar->extradata_size > 0 && st->codecpar->extradata[0] == 1; - if (pkt->size >= 5 && AV_RB32(b) != 0x0000001 && (AV_RB24(b) != 0x000001 || extradata_isom)) { + if (is_avcc(st->codecpar->extradata, st->codecpar->extradata_size)) { ret = ff_stream_add_bitstream_filter(st, "h264_mp4toannexb", NULL); - av_log(whip, AV_LOG_VERBOSE, "Enable BSF h264_mp4toannexb, packet=[%x %x %x %x %x ...], extradata_isom=%d\n", - b[0], b[1], b[2], b[3], b[4], extradata_isom); - } else + } else if (is_annexb(st->codecpar->extradata, st->codecpar->extradata_size)) whip->h264_annexb_insert_sps_pps = 1; } -- 2.49.1 >>From 4c2fd6842ef2ac7fb2b166e928d5137df947ea67 Mon Sep 17 00:00:00 2001 From: Jack Lau Date: Sun, 16 Nov 2025 14:27:13 +0800 Subject: [PATCH 2/2] avformat/whip: fix weird code that clear extradata in rtp muxer WHIP force the use of Annex B by enabling h264_mp4toannexb BSF in whip_check_bitstream. However, this BSF runs during packet writing and does not modify codecpar, so RTP muxer would still detect the stream as avcC. Add video_par_mp4toannexb function converts codecpar->extradata to Annex B to avoid this issue. Note: This function only modify the whip->video_par(copy from stream) to ensure RTP muxer detects format correctly, actual stream packet still converted by BSF(set in whip_check_bitstream). Signed-off-by: Jack Lau --- libavformat/whip.c | 85 +++++++++++++++++++++++++++++++++++++++------- 1 file changed, 73 insertions(+), 12 deletions(-) diff --git a/libavformat/whip.c b/libavformat/whip.c index 8e8ae90115..2909a5b0c9 100644 --- a/libavformat/whip.c +++ b/libavformat/whip.c @@ -20,6 +20,7 @@ */ #include "libavcodec/avcodec.h" +#include "libavcodec/bsf.h" #include "libavcodec/codec_desc.h" #include "libavcodec/h264.h" #include "libavcodec/startcode.h" @@ -401,6 +402,52 @@ static av_cold int dtls_initialize(AVFormatContext *s) return 0; } +/** + * WHIP force the use of Annex B by enabling h264_mp4toannexb BSF in whip_check_bitstream. + * However, this BSF runs during packet writing and does not modify codecpar, so RTP muxer + * would still detect the stream as avcC. + * + * This function converts codecpar->extradata to Annex B + */ +static av_cold int video_par_mp4toannexb(AVFormatContext *s) +{ + int ret = 0; + WHIPContext *whip = s->priv_data; + const AVBitStreamFilter *filter = av_bsf_get_by_name("h264_mp4toannexb"); + AVBSFContext *bsf = NULL; + + if (!filter) { + av_log(s, AV_LOG_ERROR, "h264_mp4toannexb bitstream filter not found\n"); + return AVERROR(ENOSYS); + } + + if (is_annexb(whip->video_par->extradata, whip->video_par->extradata_size)) { + av_log(whip, AV_LOG_VERBOSE, "The input is already in Annex B format\n"); + goto end; + } + + ret = av_bsf_alloc(filter, &bsf); + if (ret < 0) + goto end; + + ret = avcodec_parameters_copy(bsf->par_in, whip->video_par); + if (ret < 0) + goto end; + + ret = av_bsf_init(bsf); + if (ret < 0) + goto end; + + ret = avcodec_parameters_copy(whip->video_par, bsf->par_out); + if (ret < 0) + goto end; +end: + if (bsf) + av_bsf_free(&bsf); + return ret; +} + + /** * Initialize and check the options for the WebRTC muxer. */ @@ -538,7 +585,6 @@ static int parse_codec(AVFormatContext *s) av_log(whip, AV_LOG_ERROR, "Only one video stream is supported by RTC\n"); return AVERROR(EINVAL); } - whip->video_par = par; if (par->codec_id != AV_CODEC_ID_H264) { av_log(whip, AV_LOG_ERROR, "Unsupported video codec %s by RTC, choose h264\n", @@ -564,6 +610,20 @@ static int parse_codec(AVFormatContext *s) av_log(whip, AV_LOG_WARNING, "No level found in extradata, consider 3.1\n"); return AVERROR(EINVAL); } + + whip->video_par = avcodec_parameters_alloc(); + if (!whip->video_par) + return AVERROR(ENOMEM); + ret = avcodec_parameters_copy(whip->video_par, par); + if (ret < 0) + return ret; + + ret = video_par_mp4toannexb(s); + if (ret < 0) { + av_log(whip, AV_LOG_ERROR, "Failed to convert video par from avcC to annexb\n"); + return ret; + } + break; case AVMEDIA_TYPE_AUDIO: if (whip->audio_par) { @@ -588,6 +648,14 @@ static int parse_codec(AVFormatContext *s) av_log(whip, AV_LOG_ERROR, "Unsupported audio sample rate %d by RTC, choose 48000\n", par->sample_rate); return AVERROR_PATCHWELCOME; } + + whip->audio_par = avcodec_parameters_alloc(); + if (!whip->audio_par) + return AVERROR(ENOMEM); + ret = avcodec_parameters_copy(whip->audio_par, par); + if (ret < 0) + return ret; + break; default: av_log(whip, AV_LOG_ERROR, "Codec type '%s' for stream %d is not supported by RTC\n", @@ -1570,6 +1638,7 @@ static int create_rtp_muxer(AVFormatContext *s) max_packet_size = whip->pkt_size - DTLS_SRTP_CHECKSUM_LEN; for (i = 0; i < s->nb_streams; i++) { + is_video = s->streams[i]->codecpar->codec_type == AVMEDIA_TYPE_VIDEO; rtp_ctx = avformat_alloc_context(); if (!rtp_ctx) { ret = AVERROR(ENOMEM); @@ -1593,18 +1662,9 @@ static int create_rtp_muxer(AVFormatContext *s) /* Set the synchronized start time. */ rtp_ctx->start_time_realtime = s->start_time_realtime; - avcodec_parameters_copy(rtp_ctx->streams[0]->codecpar, s->streams[i]->codecpar); + avcodec_parameters_copy(rtp_ctx->streams[0]->codecpar, is_video ? whip->video_par : whip->audio_par); rtp_ctx->streams[0]->time_base = s->streams[i]->time_base; - /** - * For H.264, consistently utilize the annexb format through the Bitstream Filter (BSF); - * therefore, we deactivate the extradata detection for the RTP muxer. - */ - if (s->streams[i]->codecpar->codec_id == AV_CODEC_ID_H264) { - av_freep(&rtp_ctx->streams[0]->codecpar->extradata); - rtp_ctx->streams[0]->codecpar->extradata_size = 0; - } - buffer = av_malloc(buffer_size); if (!buffer) { ret = AVERROR(ENOMEM); @@ -1619,7 +1679,6 @@ static int create_rtp_muxer(AVFormatContext *s) rtp_ctx->pb->max_packet_size = max_packet_size; rtp_ctx->pb->av_class = &ff_avio_class; - is_video = s->streams[i]->codecpar->codec_type == AVMEDIA_TYPE_VIDEO; snprintf(buf, sizeof(buf), "%d", is_video? whip->video_payload_type : whip->audio_payload_type); av_dict_set(&opts, "payload_type", buf, 0); snprintf(buf, sizeof(buf), "%d", is_video? whip->video_ssrc : whip->audio_ssrc); @@ -1998,6 +2057,8 @@ static av_cold void whip_deinit(AVFormatContext *s) s->streams[i]->priv_data = NULL; } + avcodec_parameters_free(&whip->video_par); + avcodec_parameters_free(&whip->audio_par); av_freep(&whip->sdp_offer); av_freep(&whip->sdp_answer); av_freep(&whip->whip_resource_url); -- 2.49.1 _______________________________________________ ffmpeg-devel mailing list -- ffmpeg-devel@ffmpeg.org To unsubscribe send an email to ffmpeg-devel-leave@ffmpeg.org