* [FFmpeg-devel] [RFC 00/13] flvdec/flvenc: add support for enhanced rtmp codecs and multitrack/multichannel
@ 2024-05-21 9:02 Timo Rothenpieler
2024-05-21 9:02 ` [FFmpeg-devel] [PATCH 01/13] avformat/flvenc: Implement support for multi-track video Timo Rothenpieler
` (14 more replies)
0 siblings, 15 replies; 25+ messages in thread
From: Timo Rothenpieler @ 2024-05-21 9:02 UTC (permalink / raw)
To: ffmpeg-devel; +Cc: Timo Rothenpieler
This is based on the preliminary spec for enhanced rtmp v2:
https://veovera.org/docs/enhanced/enhanced-rtmp-v2
The spec is not final, and can still undergo breaking changes, hence this set is purely for comments and review, and not ready to be merged until the final v2 spec is published.
There are no samples out in the wild yet, so testing interoperability with other software has not happened yet either.
Specially the two other multitrack modes, where multiple tracks are in the same packet, have not been tested at all, since no software can write such files.
The set can also be found on GitHub, where ignoring whitespaces makes specially the last patch a lot more readable:
https://github.com/BtbN/FFmpeg/tree/enhanced-flv
Dennis Sädtler via ffmpeg-devel (2):
avformat/flvenc: Implement support for multi-track video
avformat/flvdec: Add support for demuxing multi-track FLV
Timo Rothenpieler (11):
avformat/flvenc: add enhanced audio codecs
avformat/flvenc: remove !size check for audio packets
avformat/flvdec: add enhanced audio codecs
avformat/flvenc: refactor fourcc writing
avformat/flvenc: write enhanced rtmp multichannel info for audio with
more than two channels
avformat/flvdec: parse enhanced rtmp multichannel info
avformat/flvenc: add support for writing multi track audio
avformat/flvdec: add support for reading multi track audio
avformat/rtmpproto: add more enhanced rtmp codecs
avformat/flvdec: stop shadowing local variables
avformat/flvdec: support all multi-track modes
libavformat/flv.h | 21 ++
libavformat/flvdec.c | 654 +++++++++++++++++++++++++++-------------
libavformat/flvenc.c | 443 +++++++++++++++++++++------
libavformat/rtmpproto.c | 11 +-
4 files changed, 819 insertions(+), 310 deletions(-)
--
2.43.2
_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".
^ permalink raw reply [flat|nested] 25+ messages in thread
* [FFmpeg-devel] [PATCH 01/13] avformat/flvenc: Implement support for multi-track video
2024-05-21 9:02 [FFmpeg-devel] [RFC 00/13] flvdec/flvenc: add support for enhanced rtmp codecs and multitrack/multichannel Timo Rothenpieler
@ 2024-05-21 9:02 ` Timo Rothenpieler
2024-05-21 18:48 ` Dennis Sädtler via ffmpeg-devel
2024-05-21 9:02 ` [FFmpeg-devel] [PATCH 02/13] avformat/flvdec: Add support for demuxing multi-track FLV Timo Rothenpieler
` (13 subsequent siblings)
14 siblings, 1 reply; 25+ messages in thread
From: Timo Rothenpieler @ 2024-05-21 9:02 UTC (permalink / raw)
To: ffmpeg-devel; +Cc: Dennis Sädtler
From: Dennis Sädtler via ffmpeg-devel <ffmpeg-devel@ffmpeg.org>
Based on enhanced-rtmp v2 spec published by Veovera:
https://veovera.github.io/enhanced-rtmp/docs/enhanced/enhanced-rtmp-v2
This implementation maintains some backwards compatibility by only
writing the track information for track indices > 0. This means that
older FFmpeg versions - and possibly other software - can still read the
first video track properly and skip over unsupported packets.
Signed-off-by: Dennis Sädtler <dennis@obsproject.com>
---
libavformat/flv.h | 7 ++
libavformat/flvenc.c | 158 ++++++++++++++++++++++++++++++-------------
2 files changed, 118 insertions(+), 47 deletions(-)
diff --git a/libavformat/flv.h b/libavformat/flv.h
index f710963b92..653c2bc82c 100644
--- a/libavformat/flv.h
+++ b/libavformat/flv.h
@@ -125,6 +125,13 @@ enum {
PacketTypeCodedFramesX = 3,
PacketTypeMetadata = 4,
PacketTypeMPEG2TSSequenceStart = 5,
+ PacketTypeMultitrack = 6,
+};
+
+enum {
+ MultitrackTypeOneTrack = 0x00,
+ MultitrackTypeManyTracks = 0x10,
+ MultitrackTypeManyTracksManyCodecs = 0x20,
};
enum {
diff --git a/libavformat/flvenc.c b/libavformat/flvenc.c
index aba1d7d80b..7ddc59e369 100644
--- a/libavformat/flvenc.c
+++ b/libavformat/flvenc.c
@@ -125,8 +125,9 @@ typedef struct FLVContext {
AVCodecParameters *data_par;
int flags;
- int64_t last_ts[FLV_STREAM_TYPE_NB];
- int metadata_pkt_written;
+ int64_t *last_ts;
+ int *metadata_pkt_written;
+ int *video_track_idx_map;
} FLVContext;
static int get_audio_flags(AVFormatContext *s, AVCodecParameters *par)
@@ -484,7 +485,7 @@ static void write_metadata(AVFormatContext *s, unsigned int ts)
avio_wb32(pb, flv->metadata_totalsize + 11);
}
-static void flv_write_metadata_packet(AVFormatContext *s, AVCodecParameters *par, unsigned int ts)
+static void flv_write_metadata_packet(AVFormatContext *s, AVCodecParameters *par, unsigned int ts, int stream_idx)
{
AVIOContext *pb = s->pb;
FLVContext *flv = s->priv_data;
@@ -494,7 +495,9 @@ static void flv_write_metadata_packet(AVFormatContext *s, AVCodecParameters *par
int64_t total_size = 0;
const AVPacketSideData *side_data = NULL;
- if (flv->metadata_pkt_written) return;
+ if (flv->metadata_pkt_written[stream_idx])
+ return;
+
if (par->codec_id == AV_CODEC_ID_HEVC || par->codec_id == AV_CODEC_ID_AV1 ||
par->codec_id == AV_CODEC_ID_VP9) {
int flags_size = 5;
@@ -616,7 +619,7 @@ static void flv_write_metadata_packet(AVFormatContext *s, AVCodecParameters *par
avio_wb24(pb, total_size);
avio_skip(pb, total_size + 10 - 3);
avio_wb32(pb, total_size + 11); // previous tag size
- flv->metadata_pkt_written = 1;
+ flv->metadata_pkt_written[stream_idx] = 1;
}
}
@@ -631,7 +634,7 @@ static int unsupported_codec(AVFormatContext *s,
return AVERROR(ENOSYS);
}
-static void flv_write_codec_header(AVFormatContext* s, AVCodecParameters* par, int64_t ts) {
+static void flv_write_codec_header(AVFormatContext* s, AVCodecParameters* par, int64_t ts, int stream_index) {
int64_t data_size;
AVIOContext *pb = s->pb;
FLVContext *flv = s->priv_data;
@@ -681,12 +684,32 @@ static void flv_write_codec_header(AVFormatContext* s, AVCodecParameters* par, i
}
avio_write(pb, par->extradata, par->extradata_size);
} else {
- if (par->codec_id == AV_CODEC_ID_HEVC) {
- avio_w8(pb, FLV_IS_EX_HEADER | PacketTypeSequenceStart | FLV_FRAME_KEY); // ExVideoTagHeader mode with PacketTypeSequenceStart
- avio_write(pb, "hvc1", 4);
- } else if (par->codec_id == AV_CODEC_ID_AV1 || par->codec_id == AV_CODEC_ID_VP9) {
- avio_w8(pb, FLV_IS_EX_HEADER | PacketTypeSequenceStart | FLV_FRAME_KEY);
- avio_write(pb, par->codec_id == AV_CODEC_ID_AV1 ? "av01" : "vp09", 4);
+ int track_idx = flv->video_track_idx_map[stream_index];
+ // If video stream has track_idx > 0 we need to send H.264 as extended video packet
+ int extended_flv = (par->codec_id == AV_CODEC_ID_H264 && track_idx)
+ || par->codec_id == AV_CODEC_ID_HEVC
+ || par->codec_id == AV_CODEC_ID_AV1
+ || par->codec_id == AV_CODEC_ID_VP9;
+
+ if (extended_flv) {
+ if (track_idx) {
+ avio_w8(pb, FLV_IS_EX_HEADER | PacketTypeMultitrack | FLV_FRAME_KEY);
+ avio_w8(pb, MultitrackTypeOneTrack | PacketTypeSequenceStart);
+ } else {
+ avio_w8(pb, FLV_IS_EX_HEADER | PacketTypeSequenceStart | FLV_FRAME_KEY);
+ }
+
+ if (par->codec_id == AV_CODEC_ID_H264)
+ avio_write(pb, "avc1", 4);
+ else if (par->codec_id == AV_CODEC_ID_HEVC)
+ avio_write(pb, "hvc1", 4);
+ else if (par->codec_id == AV_CODEC_ID_AV1)
+ avio_write(pb, "av01", 4);
+ else if (par->codec_id == AV_CODEC_ID_VP9)
+ avio_write(pb, "vp09", 4);
+
+ if (track_idx)
+ avio_w8(pb, track_idx);
} else {
avio_w8(pb, par->codec_tag | FLV_FRAME_KEY); // flags
avio_w8(pb, 0); // AVC sequence header
@@ -769,13 +792,12 @@ static int shift_data(AVFormatContext *s)
static int flv_init(struct AVFormatContext *s)
{
int i;
+ int video_ctr = 0;
FLVContext *flv = s->priv_data;
- if (s->nb_streams > FLV_STREAM_TYPE_NB) {
- av_log(s, AV_LOG_ERROR, "invalid number of streams %d\n",
- s->nb_streams);
- return AVERROR(EINVAL);
- }
+ flv->last_ts = av_mallocz(sizeof(*flv->last_ts) * s->nb_streams);
+ flv->metadata_pkt_written = av_mallocz(sizeof(*flv->metadata_pkt_written) * s->nb_streams);
+ flv->video_track_idx_map = av_mallocz(sizeof(*flv->video_track_idx_map) * s->nb_streams);
for (i = 0; i < s->nb_streams; i++) {
AVCodecParameters *par = s->streams[i]->codecpar;
@@ -786,12 +808,17 @@ static int flv_init(struct AVFormatContext *s)
s->streams[i]->avg_frame_rate.num) {
flv->framerate = av_q2d(s->streams[i]->avg_frame_rate);
}
- if (flv->video_par) {
+ flv->video_track_idx_map[i] = video_ctr++;
+ if (flv->video_par && flv->flags & FLV_ADD_KEYFRAME_INDEX) {
av_log(s, AV_LOG_ERROR,
- "at most one video stream is supported in flv\n");
+ "at most one video stream is supported in flv with keyframe index\n");
return AVERROR(EINVAL);
+ } else if (flv->video_par) {
+ av_log(s, AV_LOG_WARNING,
+ "more than one video stream is not supported by most flv demuxers.\n");
}
- flv->video_par = par;
+ if (!flv->video_par)
+ flv->video_par = par;
if (!ff_codec_get_tag(flv_video_codec_ids, par->codec_id))
return unsupported_codec(s, "Video", par->codec_id);
@@ -881,7 +908,7 @@ static int flv_write_header(AVFormatContext *s)
}
for (i = 0; i < s->nb_streams; i++) {
- flv_write_codec_header(s, s->streams[i]->codecpar, 0);
+ flv_write_codec_header(s, s->streams[i]->codecpar, 0, i);
}
flv->datastart_offset = avio_tell(pb);
@@ -989,6 +1016,7 @@ static int flv_write_packet(AVFormatContext *s, AVPacket *pkt)
uint8_t frametype = pkt->flags & AV_PKT_FLAG_KEY ? FLV_FRAME_KEY : FLV_FRAME_INTER;
int flags = -1, flags_size, ret = 0;
int64_t cur_offset = avio_tell(pb);
+ int track_idx = flv->video_track_idx_map[pkt->stream_index];
if (par->codec_type == AVMEDIA_TYPE_AUDIO && !pkt->size) {
av_log(s, AV_LOG_WARNING, "Empty audio Packet\n");
@@ -1005,7 +1033,12 @@ static int flv_write_packet(AVFormatContext *s, AVPacket *pkt)
else
flags_size = 1;
- if (par->codec_id == AV_CODEC_ID_HEVC && pkt->pts != pkt->dts)
+ if (par->codec_type == AVMEDIA_TYPE_VIDEO && track_idx)
+ flags_size += 2; // additional header bytes for multi-track video
+
+ if ((par->codec_id == AV_CODEC_ID_HEVC ||
+ (par->codec_id == AV_CODEC_ID_H264 && track_idx))
+ && pkt->pts != pkt->dts)
flags_size += 3;
if (par->codec_id == AV_CODEC_ID_AAC || par->codec_id == AV_CODEC_ID_H264
@@ -1018,9 +1051,9 @@ static int flv_write_packet(AVFormatContext *s, AVPacket *pkt)
if (ret < 0)
return ret;
memcpy(par->extradata, side, side_size);
- flv_write_codec_header(s, par, pkt->dts);
+ flv_write_codec_header(s, par, pkt->dts, pkt->stream_index);
}
- flv_write_metadata_packet(s, par, pkt->dts);
+ flv_write_metadata_packet(s, par, pkt->dts, pkt->stream_index);
}
if (flv->delay == AV_NOPTS_VALUE)
@@ -1142,32 +1175,59 @@ static int flv_write_packet(AVFormatContext *s, AVPacket *pkt)
avio_seek(pb, data_size + 10 - 3, SEEK_CUR);
avio_wb32(pb, data_size + 11);
} else {
- av_assert1(flags>=0);
- if (par->codec_id == AV_CODEC_ID_HEVC) {
- int pkttype = (pkt->pts != pkt->dts) ? PacketTypeCodedFrames : PacketTypeCodedFramesX;
- avio_w8(pb, FLV_IS_EX_HEADER | pkttype | frametype); // ExVideoTagHeader mode with PacketTypeCodedFrames(X)
- avio_write(pb, "hvc1", 4);
- if (pkttype == PacketTypeCodedFrames)
+ int extended_flv = (par->codec_id == AV_CODEC_ID_H264 && track_idx)
+ || par->codec_id == AV_CODEC_ID_HEVC
+ || par->codec_id == AV_CODEC_ID_AV1
+ || par->codec_id == AV_CODEC_ID_VP9;
+
+ if (extended_flv) {
+ int h2645 = par->codec_id == AV_CODEC_ID_H264 ||
+ par->codec_id == AV_CODEC_ID_HEVC;
+ int pkttype = PacketTypeCodedFrames;
+ // Optimisation for HEVC/H264: Do not send composition time if DTS == PTS
+ if (h2645 && pkt->pts == pkt->dts)
+ pkttype = PacketTypeCodedFramesX;
+
+ if (track_idx) {
+ avio_w8(pb, FLV_IS_EX_HEADER | PacketTypeMultitrack | frametype);
+ avio_w8(pb, MultitrackTypeOneTrack | pkttype);
+ } else {
+ avio_w8(pb, FLV_IS_EX_HEADER | pkttype | frametype);
+ }
+
+ if (par->codec_id == AV_CODEC_ID_H264)
+ avio_write(pb, "avc1", 4);
+ else if (par->codec_id == AV_CODEC_ID_HEVC)
+ avio_write(pb, "hvc1", 4);
+ else if (par->codec_id == AV_CODEC_ID_AV1)
+ avio_write(pb, "av01", 4);
+ else if (par->codec_id == AV_CODEC_ID_VP9)
+ avio_write(pb, "vp09", 4);
+
+ if (track_idx)
+ avio_w8(pb, track_idx);
+ if (h2645 && pkttype == PacketTypeCodedFrames)
avio_wb24(pb, pkt->pts - pkt->dts);
- } else if (par->codec_id == AV_CODEC_ID_AV1 || par->codec_id == AV_CODEC_ID_VP9) {
- avio_w8(pb, FLV_IS_EX_HEADER | PacketTypeCodedFrames | frametype);
- avio_write(pb, par->codec_id == AV_CODEC_ID_AV1 ? "av01" : "vp09", 4);
} else {
+ av_assert1(flags>=0);
avio_w8(pb, flags);
- }
- if (par->codec_id == AV_CODEC_ID_VP6)
- avio_w8(pb,0);
- if (par->codec_id == AV_CODEC_ID_VP6F || par->codec_id == AV_CODEC_ID_VP6A) {
- if (par->extradata_size)
- avio_w8(pb, par->extradata[0]);
- else
- avio_w8(pb, ((FFALIGN(par->width, 16) - par->width) << 4) |
- (FFALIGN(par->height, 16) - par->height));
- } else if (par->codec_id == AV_CODEC_ID_AAC)
- avio_w8(pb, 1); // AAC raw
- else if (par->codec_id == AV_CODEC_ID_H264 || par->codec_id == AV_CODEC_ID_MPEG4) {
- avio_w8(pb, 1); // AVC NALU
- avio_wb24(pb, pkt->pts - pkt->dts);
+
+ if (par->codec_id == AV_CODEC_ID_VP6) {
+ avio_w8(pb,0);
+ } else if (par->codec_id == AV_CODEC_ID_VP6F ||
+ par->codec_id == AV_CODEC_ID_VP6A) {
+ if (par->extradata_size)
+ avio_w8(pb, par->extradata[0]);
+ else
+ avio_w8(pb, ((FFALIGN(par->width, 16) - par->width) << 4) |
+ (FFALIGN(par->height, 16) - par->height));
+ } else if (par->codec_id == AV_CODEC_ID_AAC) {
+ avio_w8(pb, 1); // AAC raw
+ } else if (par->codec_id == AV_CODEC_ID_H264 ||
+ par->codec_id == AV_CODEC_ID_MPEG4) {
+ avio_w8(pb, 1); // AVC NALU
+ avio_wb24(pb, pkt->pts - pkt->dts);
+ }
}
avio_write(pb, data ? data : pkt->data, size);
@@ -1234,6 +1294,10 @@ static void flv_deinit(AVFormatContext *s)
}
flv->filepositions = flv->head_filepositions = NULL;
flv->filepositions_count = 0;
+
+ av_freep(&flv->last_ts);
+ av_freep(&flv->metadata_pkt_written);
+ av_freep(&flv->video_track_idx_map);
}
static const AVOption options[] = {
--
2.43.2
_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".
^ permalink raw reply [flat|nested] 25+ messages in thread
* [FFmpeg-devel] [PATCH 02/13] avformat/flvdec: Add support for demuxing multi-track FLV
2024-05-21 9:02 [FFmpeg-devel] [RFC 00/13] flvdec/flvenc: add support for enhanced rtmp codecs and multitrack/multichannel Timo Rothenpieler
2024-05-21 9:02 ` [FFmpeg-devel] [PATCH 01/13] avformat/flvenc: Implement support for multi-track video Timo Rothenpieler
@ 2024-05-21 9:02 ` Timo Rothenpieler
2024-05-21 9:02 ` [FFmpeg-devel] [PATCH 03/13] avformat/flvenc: add enhanced audio codecs Timo Rothenpieler
` (12 subsequent siblings)
14 siblings, 0 replies; 25+ messages in thread
From: Timo Rothenpieler @ 2024-05-21 9:02 UTC (permalink / raw)
To: ffmpeg-devel; +Cc: Dennis Sädtler
From: Dennis Sädtler via ffmpeg-devel <ffmpeg-devel@ffmpeg.org>
Based on enhanced-rtmp v2 spec published by Veovera:
https://veovera.github.io/enhanced-rtmp/docs/enhanced/enhanced-rtmp-v2
Signed-off-by: Dennis Sädtler <dennis@obsproject.com>
---
libavformat/flvdec.c | 117 +++++++++++++++++++++++++++++++++++--------
1 file changed, 96 insertions(+), 21 deletions(-)
diff --git a/libavformat/flvdec.c b/libavformat/flvdec.c
index 22a9b9e4a7..2445d1fd5e 100644
--- a/libavformat/flvdec.c
+++ b/libavformat/flvdec.c
@@ -105,6 +105,10 @@ typedef struct FLVContext {
FLVMetaVideoColor *metaVideoColor;
int meta_color_info_flag;
+
+ uint8_t **mt_extradata;
+ int *mt_extradata_sz;
+ int mt_extradata_cnt;
} FLVContext;
/* AMF date type */
@@ -187,13 +191,18 @@ static void add_keyframes_index(AVFormatContext *s)
}
}
-static AVStream *create_stream(AVFormatContext *s, int codec_type)
+static AVStream *create_stream(AVFormatContext *s, int codec_type, int track_idx)
{
FLVContext *flv = s->priv_data;
AVStream *st = avformat_new_stream(s, NULL);
if (!st)
return NULL;
st->codecpar->codec_type = codec_type;
+ st->id = track_idx;
+ avpriv_set_pts_info(st, 32, 1, 1000); /* 32 bit pts in ms */
+ if (track_idx)
+ return st;
+
if (s->nb_streams>=3 ||( s->nb_streams==2
&& s->streams[0]->codecpar->codec_type != AVMEDIA_TYPE_SUBTITLE
&& s->streams[1]->codecpar->codec_type != AVMEDIA_TYPE_SUBTITLE
@@ -210,8 +219,6 @@ static AVStream *create_stream(AVFormatContext *s, int codec_type)
st->avg_frame_rate = flv->framerate;
}
-
- avpriv_set_pts_info(st, 32, 1, 1000); /* 32 bit pts in ms */
flv->last_keyframe_stream_index = s->nb_streams - 1;
add_keyframes_index(s);
return st;
@@ -351,6 +358,7 @@ static int flv_same_video_codec(AVCodecParameters *vpar, uint32_t flv_codecid)
case FLV_CODECID_VP6A:
return vpar->codec_id == AV_CODEC_ID_VP6A;
case FLV_CODECID_H264:
+ case MKBETAG('a', 'v', 'c', '1'):
return vpar->codec_id == AV_CODEC_ID_H264;
default:
return vpar->codec_tag == flv_codecid;
@@ -407,6 +415,7 @@ static int flv_set_video_codec(AVFormatContext *s, AVStream *vstream,
ret = 1; // 1 byte body size adjustment for flv_read_packet()
break;
case FLV_CODECID_H264:
+ case MKBETAG('a', 'v', 'c', '1'):
par->codec_id = AV_CODEC_ID_H264;
vstreami->need_parsing = AVSTREAM_PARSE_HEADERS;
break;
@@ -676,7 +685,7 @@ static int amf_parse_object(AVFormatContext *s, AVStream *astream,
} else if (!strcmp(key, "height") && vpar) {
vpar->height = num_val;
} else if (!strcmp(key, "datastream")) {
- AVStream *st = create_stream(s, AVMEDIA_TYPE_SUBTITLE);
+ AVStream *st = create_stream(s, AVMEDIA_TYPE_SUBTITLE, 0);
if (!st)
return AVERROR(ENOMEM);
st->codecpar->codec_id = AV_CODEC_ID_TEXT;
@@ -885,6 +894,9 @@ static int flv_read_close(AVFormatContext *s)
FLVContext *flv = s->priv_data;
for (i=0; i<FLV_STREAM_TYPE_NB; i++)
av_freep(&flv->new_extradata[i]);
+ for (i=0; i < flv->mt_extradata_cnt; i++)
+ av_freep(&flv->mt_extradata[i]);
+ av_freep(&flv->mt_extradata_sz);
av_freep(&flv->keyframe_times);
av_freep(&flv->keyframe_filepositions);
av_freep(&flv->metaVideoColor);
@@ -904,18 +916,47 @@ static int flv_get_extradata(AVFormatContext *s, AVStream *st, int size)
}
static int flv_queue_extradata(FLVContext *flv, AVIOContext *pb, int stream,
- int size)
+ int size, int multitrack)
{
if (!size)
return 0;
- av_free(flv->new_extradata[stream]);
- flv->new_extradata[stream] = av_mallocz(size +
- AV_INPUT_BUFFER_PADDING_SIZE);
- if (!flv->new_extradata[stream])
- return AVERROR(ENOMEM);
- flv->new_extradata_size[stream] = size;
- avio_read(pb, flv->new_extradata[stream], size);
+ if (!multitrack) {
+ av_free(flv->new_extradata[stream]);
+ flv->new_extradata[stream] = av_mallocz(size +
+ AV_INPUT_BUFFER_PADDING_SIZE);
+ if (!flv->new_extradata[stream])
+ return AVERROR(ENOMEM);
+ flv->new_extradata_size[stream] = size;
+ avio_read(pb, flv->new_extradata[stream], size);
+ } else {
+ int new_count = stream + 1;
+
+ if (flv->mt_extradata_cnt < new_count) {
+ flv->mt_extradata = av_realloc(flv->mt_extradata,
+ sizeof(*flv->mt_extradata) *
+ new_count);
+ flv->mt_extradata_sz = av_realloc(flv->mt_extradata_sz,
+ sizeof(*flv->mt_extradata_sz) *
+ new_count);
+ if (!flv->mt_extradata || !flv->mt_extradata_sz)
+ return AVERROR(ENOMEM);
+ // Set newly allocated pointers/sizes to 0
+ for (int i = flv->mt_extradata_cnt; i < new_count; i++) {
+ flv->mt_extradata[i] = NULL;
+ flv->mt_extradata_sz[i] = 0;
+ }
+ flv->mt_extradata_cnt = new_count;
+ }
+
+ av_free(flv->mt_extradata[stream]);
+ flv->mt_extradata[stream] = av_mallocz(size + AV_INPUT_BUFFER_PADDING_SIZE);
+ if (!flv->mt_extradata[stream])
+ return AVERROR(ENOMEM);
+ flv->mt_extradata_sz[stream] = size;
+ avio_read(pb, flv->mt_extradata[stream], size);
+ }
+
return 0;
}
@@ -1031,7 +1072,7 @@ static int flv_data_packet(AVFormatContext *s, AVPacket *pkt,
}
if (i == s->nb_streams) {
- st = create_stream(s, AVMEDIA_TYPE_SUBTITLE);
+ st = create_stream(s, AVMEDIA_TYPE_SUBTITLE, 0);
if (!st)
return AVERROR(ENOMEM);
st->codecpar->codec_id = AV_CODEC_ID_TEXT;
@@ -1203,6 +1244,9 @@ static int flv_read_packet(AVFormatContext *s, AVPacket *pkt)
int last = -1;
int orig_size;
int enhanced_flv = 0;
+ int multitrack = 0;
+ int pkt_type = 0;
+ uint8_t track_idx = 0;
uint32_t video_codec_id = 0;
retry:
@@ -1256,14 +1300,33 @@ retry:
* https://github.com/veovera/enhanced-rtmp/blob/main/enhanced-rtmp-v1.pdf
* */
enhanced_flv = (flags >> 7) & 1;
+ pkt_type = enhanced_flv ? video_codec_id : 0;
size--;
+
+ if (pkt_type == PacketTypeMultitrack) {
+ uint8_t types = avio_r8(s->pb);
+ int multitrack_type = types >> 4;
+ pkt_type = types & 0xF;
+
+ if (multitrack_type != MultitrackTypeOneTrack) {
+ av_log(s, AV_LOG_ERROR, "Multitrack types other than MultitrackTypeOneTrack are unsupported!\n");
+ return AVERROR_PATCHWELCOME;
+ }
+
+ multitrack = 1;
+ size--;
+ }
+
if (enhanced_flv) {
video_codec_id = avio_rb32(s->pb);
size -= 4;
}
+ if (multitrack) {
+ track_idx = avio_r8(s->pb);
+ size--;
+ }
- if (enhanced_flv && stream_type == FLV_STREAM_TYPE_VIDEO && (flags & FLV_VIDEO_FRAMETYPE_MASK) == FLV_FRAME_VIDEO_INFO_CMD) {
- int pkt_type = flags & 0x0F;
+ if (enhanced_flv && (flags & FLV_VIDEO_FRAMETYPE_MASK) == FLV_FRAME_VIDEO_INFO_CMD) {
if (pkt_type == PacketTypeMetadata) {
int ret = flv_parse_video_color_info(s, st, next);
av_log(s, AV_LOG_DEBUG, "enhanced flv parse metadata ret %d and skip\n", ret);
@@ -1327,7 +1390,8 @@ skip:
break;
} else if (stream_type == FLV_STREAM_TYPE_VIDEO) {
if (st->codecpar->codec_type == AVMEDIA_TYPE_VIDEO &&
- (s->video_codec_id || flv_same_video_codec(st->codecpar, video_codec_id)))
+ (s->video_codec_id || flv_same_video_codec(st->codecpar, video_codec_id)) &&
+ st->id == track_idx)
break;
} else if (stream_type == FLV_STREAM_TYPE_SUBTITLE) {
if (st->codecpar->codec_type == AVMEDIA_TYPE_SUBTITLE)
@@ -1339,7 +1403,7 @@ skip:
}
if (i == s->nb_streams) {
static const enum AVMediaType stream_types[] = {AVMEDIA_TYPE_VIDEO, AVMEDIA_TYPE_AUDIO, AVMEDIA_TYPE_SUBTITLE, AVMEDIA_TYPE_DATA};
- st = create_stream(s, stream_types[stream_type]);
+ st = create_stream(s, stream_types[stream_type], track_idx);
if (!st)
return AVERROR(ENOMEM);
}
@@ -1446,7 +1510,7 @@ retry_duration:
st->codecpar->codec_id == AV_CODEC_ID_VP9) {
int type = 0;
if (enhanced_flv && stream_type == FLV_STREAM_TYPE_VIDEO) {
- type = flags & 0x0F;
+ type = pkt_type;
} else {
type = avio_r8(s->pb);
size--;
@@ -1462,7 +1526,8 @@ retry_duration:
flv->meta_color_info_flag = 0;
}
- if (st->codecpar->codec_id == AV_CODEC_ID_H264 || st->codecpar->codec_id == AV_CODEC_ID_MPEG4 ||
+ if (st->codecpar->codec_id == AV_CODEC_ID_MPEG4 ||
+ (st->codecpar->codec_id == AV_CODEC_ID_H264 && (!enhanced_flv || type == PacketTypeCodedFrames)) ||
(st->codecpar->codec_id == AV_CODEC_ID_HEVC && type == PacketTypeCodedFrames)) {
// sign extension
int32_t cts = (avio_rb24(s->pb) + 0xff800000) ^ 0xff800000;
@@ -1485,7 +1550,7 @@ retry_duration:
AVDictionaryEntry *t;
if (st->codecpar->extradata) {
- if ((ret = flv_queue_extradata(flv, s->pb, stream_type, size)) < 0)
+ if ((ret = flv_queue_extradata(flv, s->pb, multitrack ? track_idx : stream_type, size, multitrack)) < 0)
return ret;
ret = FFERROR_REDO;
goto leave;
@@ -1516,7 +1581,7 @@ retry_duration:
pkt->pts = pts == AV_NOPTS_VALUE ? dts : pts;
pkt->stream_index = st->index;
pkt->pos = pos;
- if (flv->new_extradata[stream_type]) {
+ if (!multitrack && flv->new_extradata[stream_type]) {
int ret = av_packet_add_side_data(pkt, AV_PKT_DATA_NEW_EXTRADATA,
flv->new_extradata[stream_type],
flv->new_extradata_size[stream_type]);
@@ -1524,6 +1589,16 @@ retry_duration:
flv->new_extradata[stream_type] = NULL;
flv->new_extradata_size[stream_type] = 0;
}
+ } else if (multitrack
+ && flv->mt_extradata_cnt > track_idx
+ && flv->mt_extradata[track_idx]) {
+ int ret = av_packet_add_side_data(pkt, AV_PKT_DATA_NEW_EXTRADATA,
+ flv->mt_extradata[track_idx],
+ flv->mt_extradata_sz[track_idx]);
+ if (ret >= 0) {
+ flv->mt_extradata[track_idx] = NULL;
+ flv->mt_extradata_sz[track_idx] = 0;
+ }
}
if (stream_type == FLV_STREAM_TYPE_AUDIO &&
(sample_rate != flv->last_sample_rate ||
--
2.43.2
_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".
^ permalink raw reply [flat|nested] 25+ messages in thread
* [FFmpeg-devel] [PATCH 03/13] avformat/flvenc: add enhanced audio codecs
2024-05-21 9:02 [FFmpeg-devel] [RFC 00/13] flvdec/flvenc: add support for enhanced rtmp codecs and multitrack/multichannel Timo Rothenpieler
2024-05-21 9:02 ` [FFmpeg-devel] [PATCH 01/13] avformat/flvenc: Implement support for multi-track video Timo Rothenpieler
2024-05-21 9:02 ` [FFmpeg-devel] [PATCH 02/13] avformat/flvdec: Add support for demuxing multi-track FLV Timo Rothenpieler
@ 2024-05-21 9:02 ` Timo Rothenpieler
2024-05-21 9:02 ` [FFmpeg-devel] [PATCH 04/13] avformat/flvenc: remove !size check for audio packets Timo Rothenpieler
` (11 subsequent siblings)
14 siblings, 0 replies; 25+ messages in thread
From: Timo Rothenpieler @ 2024-05-21 9:02 UTC (permalink / raw)
To: ffmpeg-devel; +Cc: Timo Rothenpieler
---
libavformat/flv.h | 6 ++
libavformat/flvenc.c | 143 +++++++++++++++++++++++++++++++------------
2 files changed, 111 insertions(+), 38 deletions(-)
diff --git a/libavformat/flv.h b/libavformat/flv.h
index 653c2bc82c..d030d576f3 100644
--- a/libavformat/flv.h
+++ b/libavformat/flv.h
@@ -103,6 +103,7 @@ enum {
FLV_CODECID_NELLYMOSER = 6 << FLV_AUDIO_CODECID_OFFSET,
FLV_CODECID_PCM_ALAW = 7 << FLV_AUDIO_CODECID_OFFSET,
FLV_CODECID_PCM_MULAW = 8 << FLV_AUDIO_CODECID_OFFSET,
+ FLV_CODECID_EX_HEADER = 9 << FLV_AUDIO_CODECID_OFFSET,
FLV_CODECID_AAC = 10<< FLV_AUDIO_CODECID_OFFSET,
FLV_CODECID_SPEEX = 11<< FLV_AUDIO_CODECID_OFFSET,
};
@@ -128,6 +129,11 @@ enum {
PacketTypeMultitrack = 6,
};
+enum {
+ AudioPacketTypeSequenceStart = 0,
+ AudioPacketTypeCodedFrames = 1,
+};
+
enum {
MultitrackTypeOneTrack = 0x00,
MultitrackTypeManyTracks = 0x10,
diff --git a/libavformat/flvenc.c b/libavformat/flvenc.c
index 7ddc59e369..4e79b06499 100644
--- a/libavformat/flvenc.c
+++ b/libavformat/flvenc.c
@@ -68,6 +68,10 @@ static const AVCodecTag flv_audio_codec_ids[] = {
{ AV_CODEC_ID_PCM_MULAW, FLV_CODECID_PCM_MULAW >> FLV_AUDIO_CODECID_OFFSET },
{ AV_CODEC_ID_PCM_ALAW, FLV_CODECID_PCM_ALAW >> FLV_AUDIO_CODECID_OFFSET },
{ AV_CODEC_ID_SPEEX, FLV_CODECID_SPEEX >> FLV_AUDIO_CODECID_OFFSET },
+ { AV_CODEC_ID_OPUS, MKBETAG('O', 'p', 'u', 's') },
+ { AV_CODEC_ID_FLAC, MKBETAG('f', 'L', 'a', 'C') },
+ { AV_CODEC_ID_AC3, MKBETAG('a', 'c', '-', '3') },
+ { AV_CODEC_ID_EAC3, MKBETAG('e', 'c', '-', '3') },
{ AV_CODEC_ID_NONE, 0 }
};
@@ -138,6 +142,9 @@ static int get_audio_flags(AVFormatContext *s, AVCodecParameters *par)
if (par->codec_id == AV_CODEC_ID_AAC) // specs force these parameters
return FLV_CODECID_AAC | FLV_SAMPLERATE_44100HZ |
FLV_SAMPLESSIZE_16BIT | FLV_STEREO;
+ if (par->codec_id == AV_CODEC_ID_OPUS || par->codec_id == AV_CODEC_ID_FLAC
+ || par->codec_id == AV_CODEC_ID_AC3 || par->codec_id == AV_CODEC_ID_EAC3)
+ return FLV_CODECID_EX_HEADER; // only needed for codec support check
else if (par->codec_id == AV_CODEC_ID_SPEEX) {
if (par->sample_rate != 16000) {
av_log(s, AV_LOG_ERROR,
@@ -634,6 +641,42 @@ static int unsupported_codec(AVFormatContext *s,
return AVERROR(ENOSYS);
}
+static void flv_write_aac_header(AVFormatContext* s, AVCodecParameters* par)
+{
+ AVIOContext *pb = s->pb;
+ FLVContext *flv = s->priv_data;
+
+ if (!par->extradata_size && (flv->flags & FLV_AAC_SEQ_HEADER_DETECT)) {
+ PutBitContext pbc;
+ int samplerate_index;
+ int channels = par->ch_layout.nb_channels
+ - (par->ch_layout.nb_channels == 8 ? 1 : 0);
+ uint8_t data[2];
+
+ for (samplerate_index = 0; samplerate_index < 16;
+ samplerate_index++)
+ if (par->sample_rate
+ == ff_mpeg4audio_sample_rates[samplerate_index])
+ break;
+
+ init_put_bits(&pbc, data, sizeof(data));
+ put_bits(&pbc, 5, par->profile + 1); //profile
+ put_bits(&pbc, 4, samplerate_index); //sample rate index
+ put_bits(&pbc, 4, channels);
+ put_bits(&pbc, 1, 0); //frame length - 1024 samples
+ put_bits(&pbc, 1, 0); //does not depend on core coder
+ put_bits(&pbc, 1, 0); //is not extension
+ flush_put_bits(&pbc);
+
+ avio_w8(pb, data[0]);
+ avio_w8(pb, data[1]);
+
+ av_log(s, AV_LOG_WARNING, "AAC sequence header: %02x %02x.\n",
+ data[0], data[1]);
+ }
+ avio_write(pb, par->extradata, par->extradata_size);
+}
+
static void flv_write_codec_header(AVFormatContext* s, AVCodecParameters* par, int64_t ts, int stream_index) {
int64_t data_size;
AVIOContext *pb = s->pb;
@@ -641,7 +684,9 @@ static void flv_write_codec_header(AVFormatContext* s, AVCodecParameters* par, i
if (par->codec_id == AV_CODEC_ID_AAC || par->codec_id == AV_CODEC_ID_H264
|| par->codec_id == AV_CODEC_ID_MPEG4 || par->codec_id == AV_CODEC_ID_HEVC
- || par->codec_id == AV_CODEC_ID_AV1 || par->codec_id == AV_CODEC_ID_VP9) {
+ || par->codec_id == AV_CODEC_ID_AV1 || par->codec_id == AV_CODEC_ID_VP9
+ || par->codec_id == AV_CODEC_ID_OPUS || par->codec_id == AV_CODEC_ID_FLAC
+ || par->codec_id == AV_CODEC_ID_AC3 || par->codec_id == AV_CODEC_ID_EAC3) {
int64_t pos;
avio_w8(pb,
par->codec_type == AVMEDIA_TYPE_VIDEO ?
@@ -650,39 +695,38 @@ static void flv_write_codec_header(AVFormatContext* s, AVCodecParameters* par, i
put_timestamp(pb, ts);
avio_wb24(pb, 0); // streamid
pos = avio_tell(pb);
- if (par->codec_id == AV_CODEC_ID_AAC) {
- avio_w8(pb, get_audio_flags(s, par));
- avio_w8(pb, 0); // AAC sequence header
-
- if (!par->extradata_size && (flv->flags & FLV_AAC_SEQ_HEADER_DETECT)) {
- PutBitContext pbc;
- int samplerate_index;
- int channels = par->ch_layout.nb_channels
- - (par->ch_layout.nb_channels == 8 ? 1 : 0);
- uint8_t data[2];
-
- for (samplerate_index = 0; samplerate_index < 16;
- samplerate_index++)
- if (par->sample_rate
- == ff_mpeg4audio_sample_rates[samplerate_index])
- break;
-
- init_put_bits(&pbc, data, sizeof(data));
- put_bits(&pbc, 5, par->profile + 1); //profile
- put_bits(&pbc, 4, samplerate_index); //sample rate index
- put_bits(&pbc, 4, channels);
- put_bits(&pbc, 1, 0); //frame length - 1024 samples
- put_bits(&pbc, 1, 0); //does not depend on core coder
- put_bits(&pbc, 1, 0); //is not extension
- flush_put_bits(&pbc);
-
- avio_w8(pb, data[0]);
- avio_w8(pb, data[1]);
-
- av_log(s, AV_LOG_WARNING, "AAC sequence header: %02x %02x.\n",
- data[0], data[1]);
+ if (par->codec_type == AVMEDIA_TYPE_AUDIO) {
+ int extended_flv = par->codec_id == AV_CODEC_ID_OPUS
+ || par->codec_id == AV_CODEC_ID_FLAC
+ || par->codec_id == AV_CODEC_ID_AC3
+ || par->codec_id == AV_CODEC_ID_EAC3;
+
+ if (extended_flv) {
+ avio_w8(pb, FLV_CODECID_EX_HEADER | AudioPacketTypeSequenceStart);
+
+ if (par->codec_id == AV_CODEC_ID_AAC) {
+ avio_write(pb, "mp4a", 4);
+ flv_write_aac_header(s, par);
+ } else if (par->codec_id == AV_CODEC_ID_OPUS) {
+ avio_write(pb, "Opus", 4);
+ av_assert0(par->extradata_size);
+ avio_write(pb, par->extradata, par->extradata_size);
+ } else if (par->codec_id == AV_CODEC_ID_FLAC) {
+ avio_write(pb, "fLaC", 4);
+ av_assert0(par->extradata_size);
+ avio_write(pb, par->extradata, par->extradata_size);
+ } else if (par->codec_id == AV_CODEC_ID_MP3)
+ avio_write(pb, ".mp3", 4);
+ else if (par->codec_id == AV_CODEC_ID_AC3)
+ avio_write(pb, "ac-3", 4);
+ else if (par->codec_id == AV_CODEC_ID_EAC3)
+ avio_write(pb, "ec-3", 4);
+ } else if (par->codec_id == AV_CODEC_ID_AAC) {
+ avio_w8(pb, get_audio_flags(s, par));
+ avio_w8(pb, 0); // AAC sequence header
+
+ flv_write_aac_header(s, par);
}
- avio_write(pb, par->extradata, par->extradata_size);
} else {
int track_idx = flv->video_track_idx_map[stream_index];
// If video stream has track_idx > 0 we need to send H.264 as extended video packet
@@ -1018,13 +1062,20 @@ static int flv_write_packet(AVFormatContext *s, AVPacket *pkt)
int64_t cur_offset = avio_tell(pb);
int track_idx = flv->video_track_idx_map[pkt->stream_index];
+ int extended_audio = par->codec_id == AV_CODEC_ID_OPUS
+ || par->codec_id == AV_CODEC_ID_FLAC
+ || par->codec_id == AV_CODEC_ID_AC3
+ || par->codec_id == AV_CODEC_ID_EAC3;
+
if (par->codec_type == AVMEDIA_TYPE_AUDIO && !pkt->size) {
av_log(s, AV_LOG_WARNING, "Empty audio Packet\n");
return AVERROR(EINVAL);
}
- if (par->codec_id == AV_CODEC_ID_VP6F || par->codec_id == AV_CODEC_ID_VP6A ||
- par->codec_id == AV_CODEC_ID_VP6 || par->codec_id == AV_CODEC_ID_AAC)
+ if (extended_audio)
+ flags_size = 5;
+ else if (par->codec_id == AV_CODEC_ID_VP6F || par->codec_id == AV_CODEC_ID_VP6A ||
+ par->codec_id == AV_CODEC_ID_VP6 || par->codec_id == AV_CODEC_ID_AAC)
flags_size = 2;
else if (par->codec_id == AV_CODEC_ID_H264 || par->codec_id == AV_CODEC_ID_MPEG4 ||
par->codec_id == AV_CODEC_ID_HEVC || par->codec_id == AV_CODEC_ID_AV1 ||
@@ -1043,7 +1094,8 @@ static int flv_write_packet(AVFormatContext *s, AVPacket *pkt)
if (par->codec_id == AV_CODEC_ID_AAC || par->codec_id == AV_CODEC_ID_H264
|| par->codec_id == AV_CODEC_ID_MPEG4 || par->codec_id == AV_CODEC_ID_HEVC
- || par->codec_id == AV_CODEC_ID_AV1 || par->codec_id == AV_CODEC_ID_VP9) {
+ || par->codec_id == AV_CODEC_ID_AV1 || par->codec_id == AV_CODEC_ID_VP9
+ || par->codec_id == AV_CODEC_ID_OPUS || par->codec_id == AV_CODEC_ID_FLAC) {
size_t side_size;
uint8_t *side = av_packet_get_side_data(pkt, AV_PKT_DATA_NEW_EXTRADATA, &side_size);
if (side && side_size > 0 && (side_size != par->extradata_size || memcmp(side, par->extradata, side_size))) {
@@ -1175,12 +1227,12 @@ static int flv_write_packet(AVFormatContext *s, AVPacket *pkt)
avio_seek(pb, data_size + 10 - 3, SEEK_CUR);
avio_wb32(pb, data_size + 11);
} else {
- int extended_flv = (par->codec_id == AV_CODEC_ID_H264 && track_idx)
+ int extended_video = (par->codec_id == AV_CODEC_ID_H264 && track_idx)
|| par->codec_id == AV_CODEC_ID_HEVC
|| par->codec_id == AV_CODEC_ID_AV1
|| par->codec_id == AV_CODEC_ID_VP9;
- if (extended_flv) {
+ if (extended_video) {
int h2645 = par->codec_id == AV_CODEC_ID_H264 ||
par->codec_id == AV_CODEC_ID_HEVC;
int pkttype = PacketTypeCodedFrames;
@@ -1208,6 +1260,21 @@ static int flv_write_packet(AVFormatContext *s, AVPacket *pkt)
avio_w8(pb, track_idx);
if (h2645 && pkttype == PacketTypeCodedFrames)
avio_wb24(pb, pkt->pts - pkt->dts);
+ } else if (extended_audio) {
+ avio_w8(pb, FLV_CODECID_EX_HEADER | AudioPacketTypeCodedFrames);
+
+ if (par->codec_id == AV_CODEC_ID_AAC)
+ avio_write(pb, "mp4a", 4);
+ else if (par->codec_id == AV_CODEC_ID_OPUS)
+ avio_write(pb, "Opus", 4);
+ else if (par->codec_id == AV_CODEC_ID_FLAC)
+ avio_write(pb, "fLaC", 4);
+ else if (par->codec_id == AV_CODEC_ID_MP3)
+ avio_write(pb, ".mp3", 4);
+ else if (par->codec_id == AV_CODEC_ID_AC3)
+ avio_write(pb, "ac-3", 4);
+ else if (par->codec_id == AV_CODEC_ID_EAC3)
+ avio_write(pb, "ec-3", 4);
} else {
av_assert1(flags>=0);
avio_w8(pb, flags);
--
2.43.2
_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".
^ permalink raw reply [flat|nested] 25+ messages in thread
* [FFmpeg-devel] [PATCH 04/13] avformat/flvenc: remove !size check for audio packets
2024-05-21 9:02 [FFmpeg-devel] [RFC 00/13] flvdec/flvenc: add support for enhanced rtmp codecs and multitrack/multichannel Timo Rothenpieler
` (2 preceding siblings ...)
2024-05-21 9:02 ` [FFmpeg-devel] [PATCH 03/13] avformat/flvenc: add enhanced audio codecs Timo Rothenpieler
@ 2024-05-21 9:02 ` Timo Rothenpieler
2024-05-21 9:02 ` [FFmpeg-devel] [PATCH 05/13] avformat/flvdec: add enhanced audio codecs Timo Rothenpieler
` (10 subsequent siblings)
14 siblings, 0 replies; 25+ messages in thread
From: Timo Rothenpieler @ 2024-05-21 9:02 UTC (permalink / raw)
To: ffmpeg-devel; +Cc: Timo Rothenpieler
---
libavformat/flvenc.c | 7 -------
1 file changed, 7 deletions(-)
diff --git a/libavformat/flvenc.c b/libavformat/flvenc.c
index 4e79b06499..13e06aedbb 100644
--- a/libavformat/flvenc.c
+++ b/libavformat/flvenc.c
@@ -1067,11 +1067,6 @@ static int flv_write_packet(AVFormatContext *s, AVPacket *pkt)
|| par->codec_id == AV_CODEC_ID_AC3
|| par->codec_id == AV_CODEC_ID_EAC3;
- if (par->codec_type == AVMEDIA_TYPE_AUDIO && !pkt->size) {
- av_log(s, AV_LOG_WARNING, "Empty audio Packet\n");
- return AVERROR(EINVAL);
- }
-
if (extended_audio)
flags_size = 5;
else if (par->codec_id == AV_CODEC_ID_VP6F || par->codec_id == AV_CODEC_ID_VP6A ||
@@ -1146,8 +1141,6 @@ static int flv_write_packet(AVFormatContext *s, AVPacket *pkt)
case AVMEDIA_TYPE_AUDIO:
flags = get_audio_flags(s, par);
- av_assert0(size);
-
avio_w8(pb, FLV_TAG_TYPE_AUDIO);
break;
case AVMEDIA_TYPE_SUBTITLE:
--
2.43.2
_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".
^ permalink raw reply [flat|nested] 25+ messages in thread
* [FFmpeg-devel] [PATCH 05/13] avformat/flvdec: add enhanced audio codecs
2024-05-21 9:02 [FFmpeg-devel] [RFC 00/13] flvdec/flvenc: add support for enhanced rtmp codecs and multitrack/multichannel Timo Rothenpieler
` (3 preceding siblings ...)
2024-05-21 9:02 ` [FFmpeg-devel] [PATCH 04/13] avformat/flvenc: remove !size check for audio packets Timo Rothenpieler
@ 2024-05-21 9:02 ` Timo Rothenpieler
2024-05-21 9:02 ` [FFmpeg-devel] [PATCH 06/13] avformat/flvenc: refactor fourcc writing Timo Rothenpieler
` (9 subsequent siblings)
14 siblings, 0 replies; 25+ messages in thread
From: Timo Rothenpieler @ 2024-05-21 9:02 UTC (permalink / raw)
To: ffmpeg-devel; +Cc: Timo Rothenpieler
---
libavformat/flv.h | 8 +++
libavformat/flvdec.c | 119 +++++++++++++++++++++++++++++++++++++++----
2 files changed, 116 insertions(+), 11 deletions(-)
diff --git a/libavformat/flv.h b/libavformat/flv.h
index d030d576f3..8926db6388 100644
--- a/libavformat/flv.h
+++ b/libavformat/flv.h
@@ -132,6 +132,14 @@ enum {
enum {
AudioPacketTypeSequenceStart = 0,
AudioPacketTypeCodedFrames = 1,
+ AudioPacketTypeMultichannelConfig = 4,
+ AudioPacketTypeMultitrack = 5,
+};
+
+enum {
+ AudioChannelOrderUnspecified = 0,
+ AudioChannelOrderNative = 1,
+ AudioChannelOrderCustom = 2,
};
enum {
diff --git a/libavformat/flvdec.c b/libavformat/flvdec.c
index 2445d1fd5e..ee2a1931b2 100644
--- a/libavformat/flvdec.c
+++ b/libavformat/flvdec.c
@@ -224,12 +224,33 @@ static AVStream *create_stream(AVFormatContext *s, int codec_type, int track_idx
return st;
}
-static int flv_same_audio_codec(AVCodecParameters *apar, int flags)
+static int flv_same_audio_codec(AVCodecParameters *apar, int flags, uint32_t codec_fourcc)
{
int bits_per_coded_sample = (flags & FLV_AUDIO_SAMPLESIZE_MASK) ? 16 : 8;
int flv_codecid = flags & FLV_AUDIO_CODECID_MASK;
int codec_id;
+ switch (codec_fourcc) {
+ case MKBETAG('m', 'p', '4', 'a'):
+ return apar->codec_id == AV_CODEC_ID_AAC;
+ case MKBETAG('O', 'p', 'u', 's'):
+ return apar->codec_id == AV_CODEC_ID_OPUS;
+ case MKBETAG('.', 'm', 'p', '3'):
+ return apar->codec_id == AV_CODEC_ID_MP3;
+ case MKBETAG('f', 'L', 'a', 'C'):
+ return apar->codec_id == AV_CODEC_ID_FLAC;
+ case MKBETAG('a', 'c', '-', '3'):
+ return apar->codec_id == AV_CODEC_ID_AC3;
+ case MKBETAG('e', 'c', '-', '3'):
+ return apar->codec_id == AV_CODEC_ID_EAC3;
+ case 0:
+ // Not enhanced flv, continue as normal.
+ break;
+ default:
+ // Unknown FOURCC
+ return 0;
+ }
+
if (!apar->codec_id && !apar->codec_tag)
return 1;
@@ -328,6 +349,24 @@ static void flv_set_audio_codec(AVFormatContext *s, AVStream *astream,
apar->sample_rate = 8000;
apar->codec_id = AV_CODEC_ID_PCM_ALAW;
break;
+ case MKBETAG('m', 'p', '4', 'a'):
+ apar->codec_id = AV_CODEC_ID_AAC;
+ return;
+ case MKBETAG('O', 'p', 'u', 's'):
+ apar->codec_id = AV_CODEC_ID_OPUS;
+ return;
+ case MKBETAG('.', 'm', 'p', '3'):
+ apar->codec_id = AV_CODEC_ID_MP3;
+ return;
+ case MKBETAG('f', 'L', 'a', 'C'):
+ apar->codec_id = AV_CODEC_ID_FLAC;
+ return;
+ case MKBETAG('a', 'c', '-', '3'):
+ apar->codec_id = AV_CODEC_ID_AC3;
+ return;
+ case MKBETAG('e', 'c', '-', '3'):
+ apar->codec_id = AV_CODEC_ID_EAC3;
+ return;
default:
avpriv_request_sample(s, "Audio codec (%x)",
flv_codecid >> FLV_AUDIO_CODECID_OFFSET);
@@ -1247,7 +1286,7 @@ static int flv_read_packet(AVFormatContext *s, AVPacket *pkt)
int multitrack = 0;
int pkt_type = 0;
uint8_t track_idx = 0;
- uint32_t video_codec_id = 0;
+ uint32_t codec_id = 0;
retry:
/* pkt size is repeated at end. skip it */
@@ -1291,16 +1330,31 @@ retry:
stream_type = FLV_STREAM_TYPE_AUDIO;
flags = avio_r8(s->pb);
size--;
+
+ if ((flags & FLV_AUDIO_CODECID_MASK) == FLV_CODECID_EX_HEADER) {
+ enhanced_flv = 1;
+ pkt_type = flags & ~FLV_AUDIO_CODECID_MASK;
+
+ channels = 0;
+
+ if (pkt_type == AudioPacketTypeMultitrack) {
+ av_log(s, AV_LOG_ERROR, "Multitrack audio is unsupported!\n");
+ return AVERROR_PATCHWELCOME;
+ }
+
+ codec_id = avio_rb32(s->pb);
+ size -= 4;
+ }
} else if (type == FLV_TAG_TYPE_VIDEO) {
stream_type = FLV_STREAM_TYPE_VIDEO;
flags = avio_r8(s->pb);
- video_codec_id = flags & FLV_VIDEO_CODECID_MASK;
+ codec_id = flags & FLV_VIDEO_CODECID_MASK;
/*
* Reference Enhancing FLV 2023-03-v1.0.0-B.8
* https://github.com/veovera/enhanced-rtmp/blob/main/enhanced-rtmp-v1.pdf
* */
enhanced_flv = (flags >> 7) & 1;
- pkt_type = enhanced_flv ? video_codec_id : 0;
+ pkt_type = enhanced_flv ? codec_id : 0;
size--;
if (pkt_type == PacketTypeMultitrack) {
@@ -1318,7 +1372,7 @@ retry:
}
if (enhanced_flv) {
- video_codec_id = avio_rb32(s->pb);
+ codec_id = avio_rb32(s->pb);
size -= 4;
}
if (multitrack) {
@@ -1386,11 +1440,11 @@ skip:
st = s->streams[i];
if (stream_type == FLV_STREAM_TYPE_AUDIO) {
if (st->codecpar->codec_type == AVMEDIA_TYPE_AUDIO &&
- (s->audio_codec_id || flv_same_audio_codec(st->codecpar, flags)))
+ (s->audio_codec_id || flv_same_audio_codec(st->codecpar, flags, codec_id)))
break;
} else if (stream_type == FLV_STREAM_TYPE_VIDEO) {
if (st->codecpar->codec_type == AVMEDIA_TYPE_VIDEO &&
- (s->video_codec_id || flv_same_video_codec(st->codecpar, video_codec_id)) &&
+ (s->video_codec_id || flv_same_video_codec(st->codecpar, codec_id)) &&
st->id == track_idx)
break;
} else if (stream_type == FLV_STREAM_TYPE_SUBTITLE) {
@@ -1459,7 +1513,7 @@ retry_duration:
flv->searched_for_end = 1;
}
- if (stream_type == FLV_STREAM_TYPE_AUDIO) {
+ if (stream_type == FLV_STREAM_TYPE_AUDIO && !enhanced_flv) {
int bits_per_coded_sample;
channels = (flags & FLV_AUDIO_CHANNEL_MASK) == FLV_STEREO ? 2 : 1;
sample_rate = 44100 << ((flags & FLV_AUDIO_SAMPLERATE_MASK) >>
@@ -1491,8 +1545,48 @@ retry_duration:
sample_rate = par->sample_rate;
avcodec_parameters_free(&par);
}
+ } else if (stream_type == FLV_STREAM_TYPE_AUDIO) {
+ if (!st->codecpar->codec_id) {
+ flv_set_audio_codec(s, st, st->codecpar,
+ codec_id ? codec_id : (flags & FLV_AUDIO_CODECID_MASK));
+ flv->last_sample_rate = 0;
+ flv->last_channels = 0;
+ }
+
+ // These are not signalled in the flags anymore
+ channels = 0;
+ sample_rate = 0;
+
+ if (pkt_type == AudioPacketTypeMultichannelConfig) {
+ int channel_order = avio_r8(s->pb);
+ channels = avio_r8(s->pb);
+ size -= 2;
+
+ if (channel_order == AudioChannelOrderCustom) {
+ for (i = 0; i < channels; i++) {
+ avio_r8(s->pb); // audio channel mapping
+ size--;
+ }
+ } else if (channel_order == AudioChannelOrderNative) {
+ avio_rb32(s->pb); // audio channel flags
+ size -= 4;
+ }
+
+ if (!av_channel_layout_check(&st->codecpar->ch_layout)) {
+ ///TODO: This can be much more sophisticated with above info.
+ av_channel_layout_default(&st->codecpar->ch_layout, channels);
+ flv->last_channels = channels;
+ }
+
+ if (channels != flv->last_channels) {
+ flv->last_channels = channels;
+ ff_add_param_change(pkt, channels, 0, 0, 0, 0);
+ }
+
+ goto leave;
+ }
} else if (stream_type == FLV_STREAM_TYPE_VIDEO) {
- int ret = flv_set_video_codec(s, st, video_codec_id, 1);
+ int ret = flv_set_video_codec(s, st, codec_id, 1);
if (ret < 0)
return ret;
size -= ret;
@@ -1503,13 +1597,15 @@ retry_duration:
}
if (st->codecpar->codec_id == AV_CODEC_ID_AAC ||
+ st->codecpar->codec_id == AV_CODEC_ID_OPUS ||
+ st->codecpar->codec_id == AV_CODEC_ID_FLAC ||
st->codecpar->codec_id == AV_CODEC_ID_H264 ||
st->codecpar->codec_id == AV_CODEC_ID_MPEG4 ||
st->codecpar->codec_id == AV_CODEC_ID_HEVC ||
st->codecpar->codec_id == AV_CODEC_ID_AV1 ||
st->codecpar->codec_id == AV_CODEC_ID_VP9) {
int type = 0;
- if (enhanced_flv && stream_type == FLV_STREAM_TYPE_VIDEO) {
+ if (enhanced_flv) {
type = pkt_type;
} else {
type = avio_r8(s->pb);
@@ -1545,6 +1641,7 @@ retry_duration:
size -= 3;
}
if (type == 0 && (!st->codecpar->extradata || st->codecpar->codec_id == AV_CODEC_ID_AAC ||
+ st->codecpar->codec_id == AV_CODEC_ID_OPUS || st->codecpar->codec_id == AV_CODEC_ID_FLAC ||
st->codecpar->codec_id == AV_CODEC_ID_H264 || st->codecpar->codec_id == AV_CODEC_ID_HEVC ||
st->codecpar->codec_id == AV_CODEC_ID_AV1 || st->codecpar->codec_id == AV_CODEC_ID_VP9)) {
AVDictionaryEntry *t;
@@ -1600,7 +1697,7 @@ retry_duration:
flv->mt_extradata_sz[track_idx] = 0;
}
}
- if (stream_type == FLV_STREAM_TYPE_AUDIO &&
+ if (stream_type == FLV_STREAM_TYPE_AUDIO && !enhanced_flv &&
(sample_rate != flv->last_sample_rate ||
channels != flv->last_channels)) {
flv->last_sample_rate = sample_rate;
--
2.43.2
_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".
^ permalink raw reply [flat|nested] 25+ messages in thread
* [FFmpeg-devel] [PATCH 06/13] avformat/flvenc: refactor fourcc writing
2024-05-21 9:02 [FFmpeg-devel] [RFC 00/13] flvdec/flvenc: add support for enhanced rtmp codecs and multitrack/multichannel Timo Rothenpieler
` (4 preceding siblings ...)
2024-05-21 9:02 ` [FFmpeg-devel] [PATCH 05/13] avformat/flvdec: add enhanced audio codecs Timo Rothenpieler
@ 2024-05-21 9:02 ` Timo Rothenpieler
2024-05-21 9:02 ` [FFmpeg-devel] [PATCH 07/13] avformat/flvenc: write enhanced rtmp multichannel info for audio with more than two channels Timo Rothenpieler
` (8 subsequent siblings)
14 siblings, 0 replies; 25+ messages in thread
From: Timo Rothenpieler @ 2024-05-21 9:02 UTC (permalink / raw)
To: ffmpeg-devel; +Cc: Timo Rothenpieler
---
libavformat/flvenc.c | 96 ++++++++++++++++++++++----------------------
1 file changed, 47 insertions(+), 49 deletions(-)
diff --git a/libavformat/flvenc.c b/libavformat/flvenc.c
index 13e06aedbb..94d633fbca 100644
--- a/libavformat/flvenc.c
+++ b/libavformat/flvenc.c
@@ -492,6 +492,45 @@ static void write_metadata(AVFormatContext *s, unsigned int ts)
avio_wb32(pb, flv->metadata_totalsize + 11);
}
+static void write_codec_fourcc(AVIOContext *pb, enum AVCodecID codec_id)
+{
+ switch (codec_id) {
+ case AV_CODEC_ID_AAC:
+ avio_write(pb, "mp4a", 4);
+ return;
+ case AV_CODEC_ID_OPUS:
+ avio_write(pb, "Opus", 4);
+ return;
+ case AV_CODEC_ID_FLAC:
+ avio_write(pb, "fLaC", 4);
+ return;
+ case AV_CODEC_ID_MP3:
+ avio_write(pb, ".mp3", 4);
+ return;
+ case AV_CODEC_ID_AC3:
+ avio_write(pb, "ac-3", 4);
+ return;
+ case AV_CODEC_ID_EAC3:
+ avio_write(pb, "ec-3", 4);
+ return;
+ case AV_CODEC_ID_H264:
+ avio_write(pb, "avc1", 4);
+ return;
+ case AV_CODEC_ID_HEVC:
+ avio_write(pb, "hvc1", 4);
+ return;
+ case AV_CODEC_ID_AV1:
+ avio_write(pb, "av01", 4);
+ return;
+ case AV_CODEC_ID_VP9:
+ avio_write(pb, "vp09", 4);
+ return;
+ default:
+ av_log(NULL, AV_LOG_ERROR, "Invalid codec FourCC write requested.\n");
+ av_assert0(0);
+ }
+}
+
static void flv_write_metadata_packet(AVFormatContext *s, AVCodecParameters *par, unsigned int ts, int stream_idx)
{
AVIOContext *pb = s->pb;
@@ -528,13 +567,8 @@ static void flv_write_metadata_packet(AVFormatContext *s, AVCodecParameters *par
put_timestamp(pb, ts); //ts = pkt->dts, gen
avio_wb24(pb, flv->reserved);
- if (par->codec_id == AV_CODEC_ID_HEVC) {
- avio_w8(pb, FLV_IS_EX_HEADER | PacketTypeMetadata| FLV_FRAME_VIDEO_INFO_CMD); // ExVideoTagHeader mode with PacketTypeMetadata
- avio_write(pb, "hvc1", 4);
- } else if (par->codec_id == AV_CODEC_ID_AV1 || par->codec_id == AV_CODEC_ID_VP9) {
- avio_w8(pb, FLV_IS_EX_HEADER | PacketTypeMetadata| FLV_FRAME_VIDEO_INFO_CMD);
- avio_write(pb, par->codec_id == AV_CODEC_ID_AV1 ? "av01" : "vp09", 4);
- }
+ avio_w8(pb, FLV_IS_EX_HEADER | PacketTypeMetadata | FLV_FRAME_VIDEO_INFO_CMD); // ExVideoTagHeader mode with PacketTypeMetadata
+ write_codec_fourcc(pb, par->codec_id);
avio_w8(pb, AMF_DATA_TYPE_STRING);
put_amf_string(pb, "colorInfo");
@@ -703,24 +737,14 @@ static void flv_write_codec_header(AVFormatContext* s, AVCodecParameters* par, i
if (extended_flv) {
avio_w8(pb, FLV_CODECID_EX_HEADER | AudioPacketTypeSequenceStart);
+ write_codec_fourcc(pb, par->codec_id);
if (par->codec_id == AV_CODEC_ID_AAC) {
- avio_write(pb, "mp4a", 4);
flv_write_aac_header(s, par);
- } else if (par->codec_id == AV_CODEC_ID_OPUS) {
- avio_write(pb, "Opus", 4);
+ } else if (par->codec_id == AV_CODEC_ID_OPUS || par->codec_id == AV_CODEC_ID_FLAC) {
av_assert0(par->extradata_size);
avio_write(pb, par->extradata, par->extradata_size);
- } else if (par->codec_id == AV_CODEC_ID_FLAC) {
- avio_write(pb, "fLaC", 4);
- av_assert0(par->extradata_size);
- avio_write(pb, par->extradata, par->extradata_size);
- } else if (par->codec_id == AV_CODEC_ID_MP3)
- avio_write(pb, ".mp3", 4);
- else if (par->codec_id == AV_CODEC_ID_AC3)
- avio_write(pb, "ac-3", 4);
- else if (par->codec_id == AV_CODEC_ID_EAC3)
- avio_write(pb, "ec-3", 4);
+ }
} else if (par->codec_id == AV_CODEC_ID_AAC) {
avio_w8(pb, get_audio_flags(s, par));
avio_w8(pb, 0); // AAC sequence header
@@ -743,14 +767,7 @@ static void flv_write_codec_header(AVFormatContext* s, AVCodecParameters* par, i
avio_w8(pb, FLV_IS_EX_HEADER | PacketTypeSequenceStart | FLV_FRAME_KEY);
}
- if (par->codec_id == AV_CODEC_ID_H264)
- avio_write(pb, "avc1", 4);
- else if (par->codec_id == AV_CODEC_ID_HEVC)
- avio_write(pb, "hvc1", 4);
- else if (par->codec_id == AV_CODEC_ID_AV1)
- avio_write(pb, "av01", 4);
- else if (par->codec_id == AV_CODEC_ID_VP9)
- avio_write(pb, "vp09", 4);
+ write_codec_fourcc(pb, par->codec_id);
if (track_idx)
avio_w8(pb, track_idx);
@@ -1240,14 +1257,7 @@ static int flv_write_packet(AVFormatContext *s, AVPacket *pkt)
avio_w8(pb, FLV_IS_EX_HEADER | pkttype | frametype);
}
- if (par->codec_id == AV_CODEC_ID_H264)
- avio_write(pb, "avc1", 4);
- else if (par->codec_id == AV_CODEC_ID_HEVC)
- avio_write(pb, "hvc1", 4);
- else if (par->codec_id == AV_CODEC_ID_AV1)
- avio_write(pb, "av01", 4);
- else if (par->codec_id == AV_CODEC_ID_VP9)
- avio_write(pb, "vp09", 4);
+ write_codec_fourcc(pb, par->codec_id);
if (track_idx)
avio_w8(pb, track_idx);
@@ -1255,19 +1265,7 @@ static int flv_write_packet(AVFormatContext *s, AVPacket *pkt)
avio_wb24(pb, pkt->pts - pkt->dts);
} else if (extended_audio) {
avio_w8(pb, FLV_CODECID_EX_HEADER | AudioPacketTypeCodedFrames);
-
- if (par->codec_id == AV_CODEC_ID_AAC)
- avio_write(pb, "mp4a", 4);
- else if (par->codec_id == AV_CODEC_ID_OPUS)
- avio_write(pb, "Opus", 4);
- else if (par->codec_id == AV_CODEC_ID_FLAC)
- avio_write(pb, "fLaC", 4);
- else if (par->codec_id == AV_CODEC_ID_MP3)
- avio_write(pb, ".mp3", 4);
- else if (par->codec_id == AV_CODEC_ID_AC3)
- avio_write(pb, "ac-3", 4);
- else if (par->codec_id == AV_CODEC_ID_EAC3)
- avio_write(pb, "ec-3", 4);
+ write_codec_fourcc(pb, par->codec_id);
} else {
av_assert1(flags>=0);
avio_w8(pb, flags);
--
2.43.2
_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".
^ permalink raw reply [flat|nested] 25+ messages in thread
* [FFmpeg-devel] [PATCH 07/13] avformat/flvenc: write enhanced rtmp multichannel info for audio with more than two channels
2024-05-21 9:02 [FFmpeg-devel] [RFC 00/13] flvdec/flvenc: add support for enhanced rtmp codecs and multitrack/multichannel Timo Rothenpieler
` (5 preceding siblings ...)
2024-05-21 9:02 ` [FFmpeg-devel] [PATCH 06/13] avformat/flvenc: refactor fourcc writing Timo Rothenpieler
@ 2024-05-21 9:02 ` Timo Rothenpieler
2024-05-21 9:02 ` [FFmpeg-devel] [PATCH 08/13] avformat/flvdec: parse enhanced rtmp multichannel info Timo Rothenpieler
` (7 subsequent siblings)
14 siblings, 0 replies; 25+ messages in thread
From: Timo Rothenpieler @ 2024-05-21 9:02 UTC (permalink / raw)
To: ffmpeg-devel; +Cc: Timo Rothenpieler
---
libavformat/flvenc.c | 93 ++++++++++++++++++++++++++++++++++++++++----
1 file changed, 85 insertions(+), 8 deletions(-)
diff --git a/libavformat/flvenc.c b/libavformat/flvenc.c
index 94d633fbca..056940afc5 100644
--- a/libavformat/flvenc.c
+++ b/libavformat/flvenc.c
@@ -711,10 +711,82 @@ static void flv_write_aac_header(AVFormatContext* s, AVCodecParameters* par)
avio_write(pb, par->extradata, par->extradata_size);
}
+static void flv_write_multichannel_body(AVFormatContext* s, AVCodecParameters* par)
+{
+ AVIOContext *pb = s->pb;
+
+ switch (par->ch_layout.order) {
+ case AV_CHANNEL_ORDER_NATIVE:
+ avio_w8(pb, AudioChannelOrderNative);
+ break;
+ case AV_CHANNEL_ORDER_CUSTOM:
+ avio_w8(pb, AudioChannelOrderCustom);
+ break;
+ default:
+ avio_w8(pb, AudioChannelOrderUnspecified);
+ break;
+ }
+
+ avio_w8(pb, par->ch_layout.nb_channels);
+
+ if (par->ch_layout.order == AV_CHANNEL_ORDER_NATIVE) {
+ // The first 18 entries are identical between FFmpeg and flv
+ uint32_t mask = par->ch_layout.u.mask & 0x03FFFF;
+ // The remaining 6 flv entries are in the right order, but start at AV_CHAN_LOW_FREQUENCY_2
+ mask |= (par->ch_layout.u.mask >> (AV_CHAN_LOW_FREQUENCY_2 - 18)) & 0xFC0000;
+
+ avio_wb32(pb, mask);
+ } else if (par->ch_layout.order == AV_CHANNEL_ORDER_CUSTOM) {
+ for (int i = 0; i < par->ch_layout.nb_channels; i++) {
+ enum AVChannel id = par->ch_layout.u.map[i].id;
+ if (id >= AV_CHAN_FRONT_LEFT && id <= AV_CHAN_TOP_BACK_RIGHT) {
+ avio_w8(pb, id - AV_CHAN_FRONT_LEFT + 0);
+ } else if (id >= AV_CHAN_LOW_FREQUENCY_2 && id <= AV_CHAN_BOTTOM_FRONT_RIGHT) {
+ avio_w8(pb, id - AV_CHAN_LOW_FREQUENCY_2 + 18);
+ } else if (id == AV_CHAN_UNUSED) {
+ avio_w8(pb, 0xFE);
+ } else {
+ avio_w8(pb, 0xFF); // unknown
+ }
+ }
+ }
+}
+
+static int flv_get_multichannel_body_size(AVCodecParameters* par)
+{
+ int res = 2;
+
+ if (par->ch_layout.order == AV_CHANNEL_ORDER_NATIVE)
+ res += 4;
+ else if (par->ch_layout.order == AV_CHANNEL_ORDER_CUSTOM)
+ res += par->ch_layout.nb_channels;
+
+ return res;
+}
+
+static void flv_write_multichannel_header(AVFormatContext* s, AVCodecParameters* par, int64_t ts)
+{
+ AVIOContext *pb = s->pb;
+ int data_size = flv_get_multichannel_body_size(par);
+
+ avio_w8(pb, FLV_TAG_TYPE_AUDIO);
+ avio_wb24(pb, data_size + 5); // size
+ put_timestamp(pb, ts);
+ avio_wb24(pb, 0); // streamid
+
+ avio_w8(pb, FLV_CODECID_EX_HEADER | AudioPacketTypeMultichannelConfig);
+ write_codec_fourcc(pb, par->codec_id);
+
+ flv_write_multichannel_body(s, par);
+
+ avio_wb32(pb, data_size + 5 + 11); // previous tag size
+}
+
static void flv_write_codec_header(AVFormatContext* s, AVCodecParameters* par, int64_t ts, int stream_index) {
int64_t data_size;
AVIOContext *pb = s->pb;
FLVContext *flv = s->priv_data;
+ int extended_flv = 0;
if (par->codec_id == AV_CODEC_ID_AAC || par->codec_id == AV_CODEC_ID_H264
|| par->codec_id == AV_CODEC_ID_MPEG4 || par->codec_id == AV_CODEC_ID_HEVC
@@ -730,10 +802,10 @@ static void flv_write_codec_header(AVFormatContext* s, AVCodecParameters* par, i
avio_wb24(pb, 0); // streamid
pos = avio_tell(pb);
if (par->codec_type == AVMEDIA_TYPE_AUDIO) {
- int extended_flv = par->codec_id == AV_CODEC_ID_OPUS
- || par->codec_id == AV_CODEC_ID_FLAC
- || par->codec_id == AV_CODEC_ID_AC3
- || par->codec_id == AV_CODEC_ID_EAC3;
+ extended_flv = par->codec_id == AV_CODEC_ID_OPUS
+ || par->codec_id == AV_CODEC_ID_FLAC
+ || par->codec_id == AV_CODEC_ID_AC3
+ || par->codec_id == AV_CODEC_ID_EAC3;
if (extended_flv) {
avio_w8(pb, FLV_CODECID_EX_HEADER | AudioPacketTypeSequenceStart);
@@ -754,10 +826,10 @@ static void flv_write_codec_header(AVFormatContext* s, AVCodecParameters* par, i
} else {
int track_idx = flv->video_track_idx_map[stream_index];
// If video stream has track_idx > 0 we need to send H.264 as extended video packet
- int extended_flv = (par->codec_id == AV_CODEC_ID_H264 && track_idx)
- || par->codec_id == AV_CODEC_ID_HEVC
- || par->codec_id == AV_CODEC_ID_AV1
- || par->codec_id == AV_CODEC_ID_VP9;
+ extended_flv = (par->codec_id == AV_CODEC_ID_H264 && track_idx)
+ || par->codec_id == AV_CODEC_ID_HEVC
+ || par->codec_id == AV_CODEC_ID_AV1
+ || par->codec_id == AV_CODEC_ID_VP9;
if (extended_flv) {
if (track_idx) {
@@ -792,6 +864,11 @@ static void flv_write_codec_header(AVFormatContext* s, AVCodecParameters* par, i
avio_skip(pb, data_size + 10 - 3);
avio_wb32(pb, data_size + 11); // previous tag size
}
+
+ if (par->codec_type == AVMEDIA_TYPE_AUDIO && (extended_flv ||
+ (av_channel_layout_compare(&par->ch_layout, &(AVChannelLayout)AV_CHANNEL_LAYOUT_STEREO) == 1 &&
+ av_channel_layout_compare(&par->ch_layout, &(AVChannelLayout)AV_CHANNEL_LAYOUT_MONO) == 1)))
+ flv_write_multichannel_header(s, par, ts);
}
static int flv_append_keyframe_info(AVFormatContext *s, FLVContext *flv, double ts, int64_t pos)
--
2.43.2
_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".
^ permalink raw reply [flat|nested] 25+ messages in thread
* [FFmpeg-devel] [PATCH 08/13] avformat/flvdec: parse enhanced rtmp multichannel info
2024-05-21 9:02 [FFmpeg-devel] [RFC 00/13] flvdec/flvenc: add support for enhanced rtmp codecs and multitrack/multichannel Timo Rothenpieler
` (6 preceding siblings ...)
2024-05-21 9:02 ` [FFmpeg-devel] [PATCH 07/13] avformat/flvenc: write enhanced rtmp multichannel info for audio with more than two channels Timo Rothenpieler
@ 2024-05-21 9:02 ` Timo Rothenpieler
2024-05-21 9:02 ` [FFmpeg-devel] [PATCH 09/13] avformat/flvenc: add support for writing multi track audio Timo Rothenpieler
` (6 subsequent siblings)
14 siblings, 0 replies; 25+ messages in thread
From: Timo Rothenpieler @ 2024-05-21 9:02 UTC (permalink / raw)
To: ffmpeg-devel; +Cc: Timo Rothenpieler
---
libavformat/flvdec.c | 41 +++++++++++++++++++++++++----------------
1 file changed, 25 insertions(+), 16 deletions(-)
diff --git a/libavformat/flvdec.c b/libavformat/flvdec.c
index ee2a1931b2..890958351a 100644
--- a/libavformat/flvdec.c
+++ b/libavformat/flvdec.c
@@ -1335,8 +1335,6 @@ retry:
enhanced_flv = 1;
pkt_type = flags & ~FLV_AUDIO_CODECID_MASK;
- channels = 0;
-
if (pkt_type == AudioPacketTypeMultitrack) {
av_log(s, AV_LOG_ERROR, "Multitrack audio is unsupported!\n");
return AVERROR_PATCHWELCOME;
@@ -1546,12 +1544,9 @@ retry_duration:
avcodec_parameters_free(&par);
}
} else if (stream_type == FLV_STREAM_TYPE_AUDIO) {
- if (!st->codecpar->codec_id) {
+ if (!st->codecpar->codec_id)
flv_set_audio_codec(s, st, st->codecpar,
codec_id ? codec_id : (flags & FLV_AUDIO_CODECID_MASK));
- flv->last_sample_rate = 0;
- flv->last_channels = 0;
- }
// These are not signalled in the flags anymore
channels = 0;
@@ -1562,26 +1557,40 @@ retry_duration:
channels = avio_r8(s->pb);
size -= 2;
+ av_channel_layout_uninit(&st->codecpar->ch_layout);
+
if (channel_order == AudioChannelOrderCustom) {
+ ret = av_channel_layout_custom_init(&st->codecpar->ch_layout, channels);
+ if (ret < 0)
+ return ret;
+
for (i = 0; i < channels; i++) {
- avio_r8(s->pb); // audio channel mapping
+ uint8_t id = avio_r8(s->pb);
size--;
+
+ if (id < 18)
+ st->codecpar->ch_layout.u.map[i].id = id;
+ else if (id >= 18 && id <= 23)
+ st->codecpar->ch_layout.u.map[i].id = id - 18 + AV_CHAN_LOW_FREQUENCY_2;
+ else if (id == 0xFE)
+ st->codecpar->ch_layout.u.map[i].id = AV_CHAN_UNUSED;
+ else
+ st->codecpar->ch_layout.u.map[i].id = AV_CHAN_UNKNOWN;
}
} else if (channel_order == AudioChannelOrderNative) {
- avio_rb32(s->pb); // audio channel flags
+ uint64_t mask = avio_rb32(s->pb);
size -= 4;
- }
- if (!av_channel_layout_check(&st->codecpar->ch_layout)) {
- ///TODO: This can be much more sophisticated with above info.
+ // The first 18 entries in the mask match ours, but the remaining 6 entries start at AV_CHAN_LOW_FREQUENCY_2
+ mask = (mask & 0x3FFFF) | ((mask & 0xFC0000) << (AV_CHAN_LOW_FREQUENCY_2 - 18));
+ ret = av_channel_layout_from_mask(&st->codecpar->ch_layout, mask);
+ if (ret < 0)
+ return ret;
+ } else {
av_channel_layout_default(&st->codecpar->ch_layout, channels);
- flv->last_channels = channels;
}
- if (channels != flv->last_channels) {
- flv->last_channels = channels;
- ff_add_param_change(pkt, channels, 0, 0, 0, 0);
- }
+ av_log(s, AV_LOG_DEBUG, "Set channel data from MultiChannel info.\n");
goto leave;
}
--
2.43.2
_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".
^ permalink raw reply [flat|nested] 25+ messages in thread
* [FFmpeg-devel] [PATCH 09/13] avformat/flvenc: add support for writing multi track audio
2024-05-21 9:02 [FFmpeg-devel] [RFC 00/13] flvdec/flvenc: add support for enhanced rtmp codecs and multitrack/multichannel Timo Rothenpieler
` (7 preceding siblings ...)
2024-05-21 9:02 ` [FFmpeg-devel] [PATCH 08/13] avformat/flvdec: parse enhanced rtmp multichannel info Timo Rothenpieler
@ 2024-05-21 9:02 ` Timo Rothenpieler
2024-05-21 9:02 ` [FFmpeg-devel] [PATCH 10/13] avformat/flvdec: add support for reading " Timo Rothenpieler
` (5 subsequent siblings)
14 siblings, 0 replies; 25+ messages in thread
From: Timo Rothenpieler @ 2024-05-21 9:02 UTC (permalink / raw)
To: ffmpeg-devel; +Cc: Timo Rothenpieler
---
libavformat/flvenc.c | 90 +++++++++++++++++++++++++++++++++-----------
1 file changed, 67 insertions(+), 23 deletions(-)
diff --git a/libavformat/flvenc.c b/libavformat/flvenc.c
index 056940afc5..ab729702a5 100644
--- a/libavformat/flvenc.c
+++ b/libavformat/flvenc.c
@@ -131,7 +131,7 @@ typedef struct FLVContext {
int flags;
int64_t *last_ts;
int *metadata_pkt_written;
- int *video_track_idx_map;
+ int *track_idx_map;
} FLVContext;
static int get_audio_flags(AVFormatContext *s, AVCodecParameters *par)
@@ -764,19 +764,33 @@ static int flv_get_multichannel_body_size(AVCodecParameters* par)
return res;
}
-static void flv_write_multichannel_header(AVFormatContext* s, AVCodecParameters* par, int64_t ts)
+static void flv_write_multichannel_header(AVFormatContext* s, AVCodecParameters* par, int64_t ts, int stream_index)
{
AVIOContext *pb = s->pb;
+ FLVContext *flv = s->priv_data;
+
+ int track_idx = flv->track_idx_map[stream_index];
int data_size = flv_get_multichannel_body_size(par);
+ if (track_idx)
+ data_size += 2;
avio_w8(pb, FLV_TAG_TYPE_AUDIO);
avio_wb24(pb, data_size + 5); // size
put_timestamp(pb, ts);
avio_wb24(pb, 0); // streamid
- avio_w8(pb, FLV_CODECID_EX_HEADER | AudioPacketTypeMultichannelConfig);
+ if (track_idx) {
+ avio_w8(pb, FLV_CODECID_EX_HEADER | AudioPacketTypeMultitrack);
+ avio_w8(pb, MultitrackTypeOneTrack | AudioPacketTypeMultichannelConfig);
+ } else {
+ avio_w8(pb, FLV_CODECID_EX_HEADER | AudioPacketTypeMultichannelConfig);
+ }
+
write_codec_fourcc(pb, par->codec_id);
+ if (track_idx)
+ avio_w8(pb, track_idx);
+
flv_write_multichannel_body(s, par);
avio_wb32(pb, data_size + 5 + 11); // previous tag size
@@ -786,6 +800,7 @@ static void flv_write_codec_header(AVFormatContext* s, AVCodecParameters* par, i
int64_t data_size;
AVIOContext *pb = s->pb;
FLVContext *flv = s->priv_data;
+ int track_idx = flv->track_idx_map[stream_index];
int extended_flv = 0;
if (par->codec_id == AV_CODEC_ID_AAC || par->codec_id == AV_CODEC_ID_H264
@@ -802,15 +817,26 @@ static void flv_write_codec_header(AVFormatContext* s, AVCodecParameters* par, i
avio_wb24(pb, 0); // streamid
pos = avio_tell(pb);
if (par->codec_type == AVMEDIA_TYPE_AUDIO) {
- extended_flv = par->codec_id == AV_CODEC_ID_OPUS
- || par->codec_id == AV_CODEC_ID_FLAC
- || par->codec_id == AV_CODEC_ID_AC3
- || par->codec_id == AV_CODEC_ID_EAC3;
+ extended_flv = (par->codec_id == AV_CODEC_ID_AAC && track_idx)
+ || (par->codec_id == AV_CODEC_ID_MP3 && track_idx)
+ || par->codec_id == AV_CODEC_ID_OPUS
+ || par->codec_id == AV_CODEC_ID_FLAC
+ || par->codec_id == AV_CODEC_ID_AC3
+ || par->codec_id == AV_CODEC_ID_EAC3;
if (extended_flv) {
- avio_w8(pb, FLV_CODECID_EX_HEADER | AudioPacketTypeSequenceStart);
+ if (track_idx) {
+ avio_w8(pb, FLV_CODECID_EX_HEADER | AudioPacketTypeMultitrack);
+ avio_w8(pb, MultitrackTypeOneTrack | AudioPacketTypeSequenceStart);
+ } else {
+ avio_w8(pb, FLV_CODECID_EX_HEADER | AudioPacketTypeSequenceStart);
+ }
+
write_codec_fourcc(pb, par->codec_id);
+ if (track_idx)
+ avio_w8(pb, track_idx);
+
if (par->codec_id == AV_CODEC_ID_AAC) {
flv_write_aac_header(s, par);
} else if (par->codec_id == AV_CODEC_ID_OPUS || par->codec_id == AV_CODEC_ID_FLAC) {
@@ -824,7 +850,6 @@ static void flv_write_codec_header(AVFormatContext* s, AVCodecParameters* par, i
flv_write_aac_header(s, par);
}
} else {
- int track_idx = flv->video_track_idx_map[stream_index];
// If video stream has track_idx > 0 we need to send H.264 as extended video packet
extended_flv = (par->codec_id == AV_CODEC_ID_H264 && track_idx)
|| par->codec_id == AV_CODEC_ID_HEVC
@@ -868,7 +893,7 @@ static void flv_write_codec_header(AVFormatContext* s, AVCodecParameters* par, i
if (par->codec_type == AVMEDIA_TYPE_AUDIO && (extended_flv ||
(av_channel_layout_compare(&par->ch_layout, &(AVChannelLayout)AV_CHANNEL_LAYOUT_STEREO) == 1 &&
av_channel_layout_compare(&par->ch_layout, &(AVChannelLayout)AV_CHANNEL_LAYOUT_MONO) == 1)))
- flv_write_multichannel_header(s, par, ts);
+ flv_write_multichannel_header(s, par, ts, stream_index);
}
static int flv_append_keyframe_info(AVFormatContext *s, FLVContext *flv, double ts, int64_t pos)
@@ -930,12 +955,12 @@ static int shift_data(AVFormatContext *s)
static int flv_init(struct AVFormatContext *s)
{
int i;
- int video_ctr = 0;
+ int video_ctr = 0, audio_ctr = 0;
FLVContext *flv = s->priv_data;
flv->last_ts = av_mallocz(sizeof(*flv->last_ts) * s->nb_streams);
flv->metadata_pkt_written = av_mallocz(sizeof(*flv->metadata_pkt_written) * s->nb_streams);
- flv->video_track_idx_map = av_mallocz(sizeof(*flv->video_track_idx_map) * s->nb_streams);
+ flv->track_idx_map = av_mallocz(sizeof(*flv->track_idx_map) * s->nb_streams);
for (i = 0; i < s->nb_streams; i++) {
AVCodecParameters *par = s->streams[i]->codecpar;
@@ -946,7 +971,7 @@ static int flv_init(struct AVFormatContext *s)
s->streams[i]->avg_frame_rate.num) {
flv->framerate = av_q2d(s->streams[i]->avg_frame_rate);
}
- flv->video_track_idx_map[i] = video_ctr++;
+ flv->track_idx_map[i] = video_ctr++;
if (flv->video_par && flv->flags & FLV_ADD_KEYFRAME_INDEX) {
av_log(s, AV_LOG_ERROR,
"at most one video stream is supported in flv with keyframe index\n");
@@ -977,12 +1002,22 @@ static int flv_init(struct AVFormatContext *s)
}
break;
case AVMEDIA_TYPE_AUDIO:
- if (flv->audio_par) {
- av_log(s, AV_LOG_ERROR,
- "at most one audio stream is supported in flv\n");
+ if (audio_ctr &&
+ par->codec_id != AV_CODEC_ID_AAC &&
+ par->codec_id != AV_CODEC_ID_MP3 &&
+ par->codec_id != AV_CODEC_ID_OPUS &&
+ par->codec_id != AV_CODEC_ID_FLAC &&
+ par->codec_id != AV_CODEC_ID_AC3 &&
+ par->codec_id != AV_CODEC_ID_EAC3) {
+ av_log(s, AV_LOG_ERROR, "Unsupported multi-track codec.\n");
return AVERROR(EINVAL);
}
- flv->audio_par = par;
+ flv->track_idx_map[i] = audio_ctr++;
+ if (flv->audio_par)
+ av_log(s, AV_LOG_WARNING,
+ "more than one audio stream is not supported by most flv demuxers.\n");
+ else
+ flv->audio_par = par;
if (get_audio_flags(s, par) < 0)
return unsupported_codec(s, "Audio", par->codec_id);
if (par->codec_id == AV_CODEC_ID_PCM_S16BE)
@@ -1154,9 +1189,11 @@ static int flv_write_packet(AVFormatContext *s, AVPacket *pkt)
uint8_t frametype = pkt->flags & AV_PKT_FLAG_KEY ? FLV_FRAME_KEY : FLV_FRAME_INTER;
int flags = -1, flags_size, ret = 0;
int64_t cur_offset = avio_tell(pb);
- int track_idx = flv->video_track_idx_map[pkt->stream_index];
+ int track_idx = flv->track_idx_map[pkt->stream_index];
- int extended_audio = par->codec_id == AV_CODEC_ID_OPUS
+ int extended_audio = (par->codec_id == AV_CODEC_ID_AAC && track_idx)
+ || (par->codec_id == AV_CODEC_ID_MP3 && track_idx)
+ || par->codec_id == AV_CODEC_ID_OPUS
|| par->codec_id == AV_CODEC_ID_FLAC
|| par->codec_id == AV_CODEC_ID_AC3
|| par->codec_id == AV_CODEC_ID_EAC3;
@@ -1173,8 +1210,8 @@ static int flv_write_packet(AVFormatContext *s, AVPacket *pkt)
else
flags_size = 1;
- if (par->codec_type == AVMEDIA_TYPE_VIDEO && track_idx)
- flags_size += 2; // additional header bytes for multi-track video
+ if ((par->codec_type == AVMEDIA_TYPE_VIDEO || par->codec_type == AVMEDIA_TYPE_AUDIO) && track_idx)
+ flags_size += 2; // additional header bytes for multi-track flv
if ((par->codec_id == AV_CODEC_ID_HEVC ||
(par->codec_id == AV_CODEC_ID_H264 && track_idx))
@@ -1341,8 +1378,15 @@ static int flv_write_packet(AVFormatContext *s, AVPacket *pkt)
if (h2645 && pkttype == PacketTypeCodedFrames)
avio_wb24(pb, pkt->pts - pkt->dts);
} else if (extended_audio) {
- avio_w8(pb, FLV_CODECID_EX_HEADER | AudioPacketTypeCodedFrames);
+ if (track_idx) {
+ avio_w8(pb, FLV_CODECID_EX_HEADER | AudioPacketTypeMultitrack);
+ avio_w8(pb, MultitrackTypeOneTrack | AudioPacketTypeCodedFrames);
+ } else {
+ avio_w8(pb, FLV_CODECID_EX_HEADER | AudioPacketTypeCodedFrames);
+ }
write_codec_fourcc(pb, par->codec_id);
+ if (track_idx)
+ avio_w8(pb, track_idx);
} else {
av_assert1(flags>=0);
avio_w8(pb, flags);
@@ -1432,7 +1476,7 @@ static void flv_deinit(AVFormatContext *s)
av_freep(&flv->last_ts);
av_freep(&flv->metadata_pkt_written);
- av_freep(&flv->video_track_idx_map);
+ av_freep(&flv->track_idx_map);
}
static const AVOption options[] = {
--
2.43.2
_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".
^ permalink raw reply [flat|nested] 25+ messages in thread
* [FFmpeg-devel] [PATCH 10/13] avformat/flvdec: add support for reading multi track audio
2024-05-21 9:02 [FFmpeg-devel] [RFC 00/13] flvdec/flvenc: add support for enhanced rtmp codecs and multitrack/multichannel Timo Rothenpieler
` (8 preceding siblings ...)
2024-05-21 9:02 ` [FFmpeg-devel] [PATCH 09/13] avformat/flvenc: add support for writing multi track audio Timo Rothenpieler
@ 2024-05-21 9:02 ` Timo Rothenpieler
2024-05-21 9:02 ` [FFmpeg-devel] [PATCH 11/13] avformat/rtmpproto: add more enhanced rtmp codecs Timo Rothenpieler
` (4 subsequent siblings)
14 siblings, 0 replies; 25+ messages in thread
From: Timo Rothenpieler @ 2024-05-21 9:02 UTC (permalink / raw)
To: ffmpeg-devel; +Cc: Timo Rothenpieler
---
libavformat/flvdec.c | 21 ++++++++++++++++++---
1 file changed, 18 insertions(+), 3 deletions(-)
diff --git a/libavformat/flvdec.c b/libavformat/flvdec.c
index 890958351a..5877828c52 100644
--- a/libavformat/flvdec.c
+++ b/libavformat/flvdec.c
@@ -1336,12 +1336,26 @@ retry:
pkt_type = flags & ~FLV_AUDIO_CODECID_MASK;
if (pkt_type == AudioPacketTypeMultitrack) {
- av_log(s, AV_LOG_ERROR, "Multitrack audio is unsupported!\n");
- return AVERROR_PATCHWELCOME;
+ uint8_t types = avio_r8(s->pb);
+ int multitrack_type = types >> 4;
+ pkt_type = types & 0xF;
+
+ if (multitrack_type != MultitrackTypeOneTrack) {
+ av_log(s, AV_LOG_ERROR, "Audio multitrack types other than MultitrackTypeOneTrack are unsupported!\n");
+ return AVERROR_PATCHWELCOME;
+ }
+
+ multitrack = 1;
+ size--;
}
codec_id = avio_rb32(s->pb);
size -= 4;
+
+ if (multitrack) {
+ track_idx = avio_r8(s->pb);
+ size--;
+ }
}
} else if (type == FLV_TAG_TYPE_VIDEO) {
stream_type = FLV_STREAM_TYPE_VIDEO;
@@ -1438,7 +1452,8 @@ skip:
st = s->streams[i];
if (stream_type == FLV_STREAM_TYPE_AUDIO) {
if (st->codecpar->codec_type == AVMEDIA_TYPE_AUDIO &&
- (s->audio_codec_id || flv_same_audio_codec(st->codecpar, flags, codec_id)))
+ (s->audio_codec_id || flv_same_audio_codec(st->codecpar, flags, codec_id)) &&
+ st->id == track_idx)
break;
} else if (stream_type == FLV_STREAM_TYPE_VIDEO) {
if (st->codecpar->codec_type == AVMEDIA_TYPE_VIDEO &&
--
2.43.2
_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".
^ permalink raw reply [flat|nested] 25+ messages in thread
* [FFmpeg-devel] [PATCH 11/13] avformat/rtmpproto: add more enhanced rtmp codecs
2024-05-21 9:02 [FFmpeg-devel] [RFC 00/13] flvdec/flvenc: add support for enhanced rtmp codecs and multitrack/multichannel Timo Rothenpieler
` (9 preceding siblings ...)
2024-05-21 9:02 ` [FFmpeg-devel] [PATCH 10/13] avformat/flvdec: add support for reading " Timo Rothenpieler
@ 2024-05-21 9:02 ` Timo Rothenpieler
2024-05-21 9:02 ` [FFmpeg-devel] [PATCH 12/13] avformat/flvdec: stop shadowing local variables Timo Rothenpieler
` (3 subsequent siblings)
14 siblings, 0 replies; 25+ messages in thread
From: Timo Rothenpieler @ 2024-05-21 9:02 UTC (permalink / raw)
To: ffmpeg-devel; +Cc: Timo Rothenpieler
---
libavformat/rtmpproto.c | 11 +++++++++--
1 file changed, 9 insertions(+), 2 deletions(-)
diff --git a/libavformat/rtmpproto.c b/libavformat/rtmpproto.c
index bc3d9df7b9..7bbb6d5313 100644
--- a/libavformat/rtmpproto.c
+++ b/libavformat/rtmpproto.c
@@ -356,9 +356,16 @@ static int gen_connect(URLContext *s, RTMPContext *rt)
while(fourcc_data - rt->enhanced_codecs < fourcc_str_len) {
unsigned char fourcc[5];
- if (!strncmp(fourcc_data, "hvc1", 4) ||
+ if (!strncmp(fourcc_data, "avc1", 4) ||
+ !strncmp(fourcc_data, "hvc1", 4) ||
!strncmp(fourcc_data, "av01", 4) ||
- !strncmp(fourcc_data, "vp09", 4)) {
+ !strncmp(fourcc_data, "vp09", 4) ||
+ !strncmp(fourcc_data, "mp4a", 4) ||
+ !strncmp(fourcc_data, "Opus", 4) ||
+ !strncmp(fourcc_data, "fLaC", 4) ||
+ !strncmp(fourcc_data, ".mp3", 4) ||
+ !strncmp(fourcc_data, "ac-3", 4) ||
+ !strncmp(fourcc_data, "ec-3", 4)) {
av_strlcpy(fourcc, fourcc_data, sizeof(fourcc));
ff_amf_write_string(&p, fourcc);
} else {
--
2.43.2
_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".
^ permalink raw reply [flat|nested] 25+ messages in thread
* [FFmpeg-devel] [PATCH 12/13] avformat/flvdec: stop shadowing local variables
2024-05-21 9:02 [FFmpeg-devel] [RFC 00/13] flvdec/flvenc: add support for enhanced rtmp codecs and multitrack/multichannel Timo Rothenpieler
` (10 preceding siblings ...)
2024-05-21 9:02 ` [FFmpeg-devel] [PATCH 11/13] avformat/rtmpproto: add more enhanced rtmp codecs Timo Rothenpieler
@ 2024-05-21 9:02 ` Timo Rothenpieler
2024-05-21 9:02 ` [FFmpeg-devel] [PATCH 13/13] avformat/flvdec: support all multi-track modes Timo Rothenpieler
` (2 subsequent siblings)
14 siblings, 0 replies; 25+ messages in thread
From: Timo Rothenpieler @ 2024-05-21 9:02 UTC (permalink / raw)
To: ffmpeg-devel; +Cc: Timo Rothenpieler
---
libavformat/flvdec.c | 44 ++++++++++++++++++++++----------------------
1 file changed, 22 insertions(+), 22 deletions(-)
diff --git a/libavformat/flvdec.c b/libavformat/flvdec.c
index 5877828c52..4f98ff348c 100644
--- a/libavformat/flvdec.c
+++ b/libavformat/flvdec.c
@@ -1394,8 +1394,8 @@ retry:
if (enhanced_flv && (flags & FLV_VIDEO_FRAMETYPE_MASK) == FLV_FRAME_VIDEO_INFO_CMD) {
if (pkt_type == PacketTypeMetadata) {
- int ret = flv_parse_video_color_info(s, st, next);
- av_log(s, AV_LOG_DEBUG, "enhanced flv parse metadata ret %d and skip\n", ret);
+ int sret = flv_parse_video_color_info(s, st, next);
+ av_log(s, AV_LOG_DEBUG, "enhanced flv parse metadata ret %d and skip\n", sret);
}
goto skip;
} else if ((flags & FLV_VIDEO_FRAMETYPE_MASK) == FLV_FRAME_VIDEO_INFO_CMD) {
@@ -1498,25 +1498,25 @@ skip:
if ((s->pb->seekable & AVIO_SEEKABLE_NORMAL) &&
(!s->duration || s->duration == AV_NOPTS_VALUE) &&
!flv->searched_for_end) {
- int size;
+ int final_size;
const int64_t pos = avio_tell(s->pb);
// Read the last 4 bytes of the file, this should be the size of the
// previous FLV tag. Use the timestamp of its payload as duration.
int64_t fsize = avio_size(s->pb);
retry_duration:
avio_seek(s->pb, fsize - 4, SEEK_SET);
- size = avio_rb32(s->pb);
- if (size > 0 && size < fsize) {
- // Seek to the start of the last FLV tag at position (fsize - 4 - size)
+ final_size = avio_rb32(s->pb);
+ if (final_size > 0 && final_size < fsize) {
+ // Seek to the start of the last FLV tag at position (fsize - 4 - final_size)
// but skip the byte indicating the type.
- avio_seek(s->pb, fsize - 3 - size, SEEK_SET);
- if (size == avio_rb24(s->pb) + 11) {
+ avio_seek(s->pb, fsize - 3 - final_size, SEEK_SET);
+ if (final_size == avio_rb24(s->pb) + 11) {
uint32_t ts = avio_rb24(s->pb);
ts |= (unsigned)avio_r8(s->pb) << 24;
if (ts)
s->duration = ts * (int64_t)AV_TIME_BASE / 1000;
- else if (fsize >= 8 && fsize - 8 >= size) {
- fsize -= size+4;
+ else if (fsize >= 8 && fsize - 8 >= final_size) {
+ fsize -= final_size+4;
goto retry_duration;
}
}
@@ -1610,10 +1610,10 @@ retry_duration:
goto leave;
}
} else if (stream_type == FLV_STREAM_TYPE_VIDEO) {
- int ret = flv_set_video_codec(s, st, codec_id, 1);
- if (ret < 0)
- return ret;
- size -= ret;
+ int sret = flv_set_video_codec(s, st, codec_id, 1);
+ if (sret < 0)
+ return sret;
+ size -= sret;
} else if (stream_type == FLV_STREAM_TYPE_SUBTITLE) {
st->codecpar->codec_id = AV_CODEC_ID_TEXT;
} else if (stream_type == FLV_STREAM_TYPE_DATA) {
@@ -1703,20 +1703,20 @@ retry_duration:
pkt->stream_index = st->index;
pkt->pos = pos;
if (!multitrack && flv->new_extradata[stream_type]) {
- int ret = av_packet_add_side_data(pkt, AV_PKT_DATA_NEW_EXTRADATA,
- flv->new_extradata[stream_type],
- flv->new_extradata_size[stream_type]);
- if (ret >= 0) {
+ int sret = av_packet_add_side_data(pkt, AV_PKT_DATA_NEW_EXTRADATA,
+ flv->new_extradata[stream_type],
+ flv->new_extradata_size[stream_type]);
+ if (sret >= 0) {
flv->new_extradata[stream_type] = NULL;
flv->new_extradata_size[stream_type] = 0;
}
} else if (multitrack
&& flv->mt_extradata_cnt > track_idx
&& flv->mt_extradata[track_idx]) {
- int ret = av_packet_add_side_data(pkt, AV_PKT_DATA_NEW_EXTRADATA,
- flv->mt_extradata[track_idx],
- flv->mt_extradata_sz[track_idx]);
- if (ret >= 0) {
+ int sret = av_packet_add_side_data(pkt, AV_PKT_DATA_NEW_EXTRADATA,
+ flv->mt_extradata[track_idx],
+ flv->mt_extradata_sz[track_idx]);
+ if (sret >= 0) {
flv->mt_extradata[track_idx] = NULL;
flv->mt_extradata_sz[track_idx] = 0;
}
--
2.43.2
_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".
^ permalink raw reply [flat|nested] 25+ messages in thread
* [FFmpeg-devel] [PATCH 13/13] avformat/flvdec: support all multi-track modes
2024-05-21 9:02 [FFmpeg-devel] [RFC 00/13] flvdec/flvenc: add support for enhanced rtmp codecs and multitrack/multichannel Timo Rothenpieler
` (11 preceding siblings ...)
2024-05-21 9:02 ` [FFmpeg-devel] [PATCH 12/13] avformat/flvdec: stop shadowing local variables Timo Rothenpieler
@ 2024-05-21 9:02 ` Timo Rothenpieler
2024-05-22 0:02 ` Michael Niedermayer
2024-05-22 18:27 ` [FFmpeg-devel] [PATCH v2 " Timo Rothenpieler
2024-05-23 1:57 ` [FFmpeg-devel] [RFC 00/13] flvdec/flvenc: add support for enhanced rtmp codecs and multitrack/multichannel Steven Liu
2024-05-28 15:14 ` Andrew Sayers
14 siblings, 2 replies; 25+ messages in thread
From: Timo Rothenpieler @ 2024-05-21 9:02 UTC (permalink / raw)
To: ffmpeg-devel; +Cc: Timo Rothenpieler
---
libavformat/flvdec.c | 570 +++++++++++++++++++++++--------------------
1 file changed, 306 insertions(+), 264 deletions(-)
diff --git a/libavformat/flvdec.c b/libavformat/flvdec.c
index 4f98ff348c..ee22db2ce2 100644
--- a/libavformat/flvdec.c
+++ b/libavformat/flvdec.c
@@ -1273,6 +1273,7 @@ static int flv_read_packet(AVFormatContext *s, AVPacket *pkt)
{
FLVContext *flv = s->priv_data;
int ret, i, size, flags;
+ int res = 0;
enum FlvTagType type;
int stream_type=-1;
int64_t next, pos, meta_pos;
@@ -1287,6 +1288,7 @@ static int flv_read_packet(AVFormatContext *s, AVPacket *pkt)
int pkt_type = 0;
uint8_t track_idx = 0;
uint32_t codec_id = 0;
+ int multitrack_type = MultitrackTypeOneTrack;
retry:
/* pkt size is repeated at end. skip it */
@@ -1337,14 +1339,9 @@ retry:
if (pkt_type == AudioPacketTypeMultitrack) {
uint8_t types = avio_r8(s->pb);
- int multitrack_type = types >> 4;
+ multitrack_type = types & 0xF0;
pkt_type = types & 0xF;
- if (multitrack_type != MultitrackTypeOneTrack) {
- av_log(s, AV_LOG_ERROR, "Audio multitrack types other than MultitrackTypeOneTrack are unsupported!\n");
- return AVERROR_PATCHWELCOME;
- }
-
multitrack = 1;
size--;
}
@@ -1371,14 +1368,9 @@ retry:
if (pkt_type == PacketTypeMultitrack) {
uint8_t types = avio_r8(s->pb);
- int multitrack_type = types >> 4;
+ multitrack_type = types & 0xF0;
pkt_type = types & 0xF;
- if (multitrack_type != MultitrackTypeOneTrack) {
- av_log(s, AV_LOG_ERROR, "Multitrack types other than MultitrackTypeOneTrack are unsupported!\n");
- return AVERROR_PATCHWELCOME;
- }
-
multitrack = 1;
size--;
}
@@ -1447,293 +1439,343 @@ skip:
goto leave;
}
- /* now find stream */
- for (i = 0; i < s->nb_streams; i++) {
- st = s->streams[i];
- if (stream_type == FLV_STREAM_TYPE_AUDIO) {
- if (st->codecpar->codec_type == AVMEDIA_TYPE_AUDIO &&
- (s->audio_codec_id || flv_same_audio_codec(st->codecpar, flags, codec_id)) &&
- st->id == track_idx)
- break;
- } else if (stream_type == FLV_STREAM_TYPE_VIDEO) {
- if (st->codecpar->codec_type == AVMEDIA_TYPE_VIDEO &&
- (s->video_codec_id || flv_same_video_codec(st->codecpar, codec_id)) &&
- st->id == track_idx)
- break;
- } else if (stream_type == FLV_STREAM_TYPE_SUBTITLE) {
- if (st->codecpar->codec_type == AVMEDIA_TYPE_SUBTITLE)
- break;
- } else if (stream_type == FLV_STREAM_TYPE_DATA) {
- if (st->codecpar->codec_type == AVMEDIA_TYPE_DATA)
- break;
+ for (;;) {
+ int track_size = size;
+
+ if (multitrack_type != MultitrackTypeOneTrack) {
+ track_size = avio_rb24(s->pb);
+ size -= 3;
}
- }
- if (i == s->nb_streams) {
- static const enum AVMediaType stream_types[] = {AVMEDIA_TYPE_VIDEO, AVMEDIA_TYPE_AUDIO, AVMEDIA_TYPE_SUBTITLE, AVMEDIA_TYPE_DATA};
- st = create_stream(s, stream_types[stream_type], track_idx);
- if (!st)
- return AVERROR(ENOMEM);
- }
- av_log(s, AV_LOG_TRACE, "%d %X %d \n", stream_type, flags, st->discard);
- if (flv->time_pos <= pos) {
- dts += flv->time_offset;
- }
+ /* now find stream */
+ for (i = 0; i < s->nb_streams; i++) {
+ st = s->streams[i];
+ if (stream_type == FLV_STREAM_TYPE_AUDIO) {
+ if (st->codecpar->codec_type == AVMEDIA_TYPE_AUDIO &&
+ (s->audio_codec_id || flv_same_audio_codec(st->codecpar, flags, codec_id)) &&
+ st->id == track_idx)
+ break;
+ } else if (stream_type == FLV_STREAM_TYPE_VIDEO) {
+ if (st->codecpar->codec_type == AVMEDIA_TYPE_VIDEO &&
+ (s->video_codec_id || flv_same_video_codec(st->codecpar, codec_id)) &&
+ st->id == track_idx)
+ break;
+ } else if (stream_type == FLV_STREAM_TYPE_SUBTITLE) {
+ if (st->codecpar->codec_type == AVMEDIA_TYPE_SUBTITLE)
+ break;
+ } else if (stream_type == FLV_STREAM_TYPE_DATA) {
+ if (st->codecpar->codec_type == AVMEDIA_TYPE_DATA)
+ break;
+ }
+ }
+ if (i == s->nb_streams) {
+ static const enum AVMediaType stream_types[] = {AVMEDIA_TYPE_VIDEO, AVMEDIA_TYPE_AUDIO, AVMEDIA_TYPE_SUBTITLE, AVMEDIA_TYPE_DATA};
+ st = create_stream(s, stream_types[stream_type], track_idx);
+ if (!st)
+ return AVERROR(ENOMEM);
+ }
+ av_log(s, AV_LOG_TRACE, "%d %X %d \n", stream_type, flags, st->discard);
- if ((s->pb->seekable & AVIO_SEEKABLE_NORMAL) &&
- ((flags & FLV_VIDEO_FRAMETYPE_MASK) == FLV_FRAME_KEY ||
- stream_type == FLV_STREAM_TYPE_AUDIO))
- av_add_index_entry(st, pos, dts, size, 0, AVINDEX_KEYFRAME);
+ if (flv->time_pos <= pos) {
+ dts += flv->time_offset;
+ }
- if ((st->discard >= AVDISCARD_NONKEY && !((flags & FLV_VIDEO_FRAMETYPE_MASK) == FLV_FRAME_KEY || stream_type == FLV_STREAM_TYPE_AUDIO)) ||
- (st->discard >= AVDISCARD_BIDIR && ((flags & FLV_VIDEO_FRAMETYPE_MASK) == FLV_FRAME_DISP_INTER && stream_type == FLV_STREAM_TYPE_VIDEO)) ||
- st->discard >= AVDISCARD_ALL) {
- avio_seek(s->pb, next, SEEK_SET);
- ret = FFERROR_REDO;
- goto leave;
- }
+ if ((s->pb->seekable & AVIO_SEEKABLE_NORMAL) &&
+ ((flags & FLV_VIDEO_FRAMETYPE_MASK) == FLV_FRAME_KEY ||
+ stream_type == FLV_STREAM_TYPE_AUDIO))
+ av_add_index_entry(st, pos, dts, track_size, 0, AVINDEX_KEYFRAME);
+
+ if ((st->discard >= AVDISCARD_NONKEY && !((flags & FLV_VIDEO_FRAMETYPE_MASK) == FLV_FRAME_KEY || stream_type == FLV_STREAM_TYPE_AUDIO)) ||
+ (st->discard >= AVDISCARD_BIDIR && ((flags & FLV_VIDEO_FRAMETYPE_MASK) == FLV_FRAME_DISP_INTER && stream_type == FLV_STREAM_TYPE_VIDEO)) ||
+ st->discard >= AVDISCARD_ALL) {
+ avio_seek(s->pb, next, SEEK_SET);
+ ret = FFERROR_REDO;
+ goto leave;
+ }
- // if not streamed and no duration from metadata then seek to end to find
- // the duration from the timestamps
- if ((s->pb->seekable & AVIO_SEEKABLE_NORMAL) &&
- (!s->duration || s->duration == AV_NOPTS_VALUE) &&
- !flv->searched_for_end) {
- int final_size;
- const int64_t pos = avio_tell(s->pb);
- // Read the last 4 bytes of the file, this should be the size of the
- // previous FLV tag. Use the timestamp of its payload as duration.
- int64_t fsize = avio_size(s->pb);
+ // if not streamed and no duration from metadata then seek to end to find
+ // the duration from the timestamps
+ if ((s->pb->seekable & AVIO_SEEKABLE_NORMAL) &&
+ (!s->duration || s->duration == AV_NOPTS_VALUE) &&
+ !flv->searched_for_end) {
+ int final_size;
+ const int64_t pos = avio_tell(s->pb);
+ // Read the last 4 bytes of the file, this should be the size of the
+ // previous FLV tag. Use the timestamp of its payload as duration.
+ int64_t fsize = avio_size(s->pb);
retry_duration:
- avio_seek(s->pb, fsize - 4, SEEK_SET);
- final_size = avio_rb32(s->pb);
- if (final_size > 0 && final_size < fsize) {
- // Seek to the start of the last FLV tag at position (fsize - 4 - final_size)
- // but skip the byte indicating the type.
- avio_seek(s->pb, fsize - 3 - final_size, SEEK_SET);
- if (final_size == avio_rb24(s->pb) + 11) {
- uint32_t ts = avio_rb24(s->pb);
- ts |= (unsigned)avio_r8(s->pb) << 24;
- if (ts)
- s->duration = ts * (int64_t)AV_TIME_BASE / 1000;
- else if (fsize >= 8 && fsize - 8 >= final_size) {
- fsize -= final_size+4;
- goto retry_duration;
+ avio_seek(s->pb, fsize - 4, SEEK_SET);
+ final_size = avio_rb32(s->pb);
+ if (final_size > 0 && final_size < fsize) {
+ // Seek to the start of the last FLV tag at position (fsize - 4 - final_size)
+ // but skip the byte indicating the type.
+ avio_seek(s->pb, fsize - 3 - final_size, SEEK_SET);
+ if (final_size == avio_rb24(s->pb) + 11) {
+ uint32_t ts = avio_rb24(s->pb);
+ ts |= (unsigned)avio_r8(s->pb) << 24;
+ if (ts)
+ s->duration = ts * (int64_t)AV_TIME_BASE / 1000;
+ else if (fsize >= 8 && fsize - 8 >= final_size) {
+ fsize -= final_size+4;
+ goto retry_duration;
+ }
}
}
+
+ avio_seek(s->pb, pos, SEEK_SET);
+ flv->searched_for_end = 1;
}
- avio_seek(s->pb, pos, SEEK_SET);
- flv->searched_for_end = 1;
- }
+ if (stream_type == FLV_STREAM_TYPE_AUDIO && !enhanced_flv) {
+ int bits_per_coded_sample;
+ channels = (flags & FLV_AUDIO_CHANNEL_MASK) == FLV_STEREO ? 2 : 1;
+ sample_rate = 44100 << ((flags & FLV_AUDIO_SAMPLERATE_MASK) >>
+ FLV_AUDIO_SAMPLERATE_OFFSET) >> 3;
+ bits_per_coded_sample = (flags & FLV_AUDIO_SAMPLESIZE_MASK) ? 16 : 8;
+ if (!av_channel_layout_check(&st->codecpar->ch_layout) ||
+ !st->codecpar->sample_rate ||
+ !st->codecpar->bits_per_coded_sample) {
+ av_channel_layout_default(&st->codecpar->ch_layout, channels);
+ st->codecpar->sample_rate = sample_rate;
+ st->codecpar->bits_per_coded_sample = bits_per_coded_sample;
+ }
+ if (!st->codecpar->codec_id) {
+ flv_set_audio_codec(s, st, st->codecpar,
+ flags & FLV_AUDIO_CODECID_MASK);
+ flv->last_sample_rate =
+ sample_rate = st->codecpar->sample_rate;
+ flv->last_channels =
+ channels = st->codecpar->ch_layout.nb_channels;
+ } else {
+ AVCodecParameters *par = avcodec_parameters_alloc();
+ if (!par) {
+ ret = AVERROR(ENOMEM);
+ goto leave;
+ }
+ par->sample_rate = sample_rate;
+ par->bits_per_coded_sample = bits_per_coded_sample;
+ flv_set_audio_codec(s, st, par, flags & FLV_AUDIO_CODECID_MASK);
+ sample_rate = par->sample_rate;
+ avcodec_parameters_free(&par);
+ }
+ } else if (stream_type == FLV_STREAM_TYPE_AUDIO) {
+ if (!st->codecpar->codec_id)
+ flv_set_audio_codec(s, st, st->codecpar,
+ codec_id ? codec_id : (flags & FLV_AUDIO_CODECID_MASK));
+
+ // These are not signalled in the flags anymore
+ channels = 0;
+ sample_rate = 0;
+
+ if (pkt_type == AudioPacketTypeMultichannelConfig) {
+ int channel_order = avio_r8(s->pb);
+ channels = avio_r8(s->pb);
+ size -= 2;
+ track_size -= 2;
+
+ av_channel_layout_uninit(&st->codecpar->ch_layout);
+
+ if (channel_order == AudioChannelOrderCustom) {
+ ret = av_channel_layout_custom_init(&st->codecpar->ch_layout, channels);
+ if (ret < 0)
+ return ret;
+
+ for (i = 0; i < channels; i++) {
+ uint8_t id = avio_r8(s->pb);
+ size--;
+
+ if (id < 18)
+ st->codecpar->ch_layout.u.map[i].id = id;
+ else if (id >= 18 && id <= 23)
+ st->codecpar->ch_layout.u.map[i].id = id - 18 + AV_CHAN_LOW_FREQUENCY_2;
+ else if (id == 0xFE)
+ st->codecpar->ch_layout.u.map[i].id = AV_CHAN_UNUSED;
+ else
+ st->codecpar->ch_layout.u.map[i].id = AV_CHAN_UNKNOWN;
+ }
+ } else if (channel_order == AudioChannelOrderNative) {
+ uint64_t mask = avio_rb32(s->pb);
+ size -= 4;
+ track_size -= 4;
+
+ // The first 18 entries in the mask match ours, but the remaining 6 entries start at AV_CHAN_LOW_FREQUENCY_2
+ mask = (mask & 0x3FFFF) | ((mask & 0xFC0000) << (AV_CHAN_LOW_FREQUENCY_2 - 18));
+ ret = av_channel_layout_from_mask(&st->codecpar->ch_layout, mask);
+ if (ret < 0)
+ return ret;
+ } else {
+ av_channel_layout_default(&st->codecpar->ch_layout, channels);
+ }
+
+ av_log(s, AV_LOG_DEBUG, "Set channel data from MultiChannel info.\n");
- if (stream_type == FLV_STREAM_TYPE_AUDIO && !enhanced_flv) {
- int bits_per_coded_sample;
- channels = (flags & FLV_AUDIO_CHANNEL_MASK) == FLV_STEREO ? 2 : 1;
- sample_rate = 44100 << ((flags & FLV_AUDIO_SAMPLERATE_MASK) >>
- FLV_AUDIO_SAMPLERATE_OFFSET) >> 3;
- bits_per_coded_sample = (flags & FLV_AUDIO_SAMPLESIZE_MASK) ? 16 : 8;
- if (!av_channel_layout_check(&st->codecpar->ch_layout) ||
- !st->codecpar->sample_rate ||
- !st->codecpar->bits_per_coded_sample) {
- av_channel_layout_default(&st->codecpar->ch_layout, channels);
- st->codecpar->sample_rate = sample_rate;
- st->codecpar->bits_per_coded_sample = bits_per_coded_sample;
- }
- if (!st->codecpar->codec_id) {
- flv_set_audio_codec(s, st, st->codecpar,
- flags & FLV_AUDIO_CODECID_MASK);
- flv->last_sample_rate =
- sample_rate = st->codecpar->sample_rate;
- flv->last_channels =
- channels = st->codecpar->ch_layout.nb_channels;
- } else {
- AVCodecParameters *par = avcodec_parameters_alloc();
- if (!par) {
- ret = AVERROR(ENOMEM);
goto leave;
}
- par->sample_rate = sample_rate;
- par->bits_per_coded_sample = bits_per_coded_sample;
- flv_set_audio_codec(s, st, par, flags & FLV_AUDIO_CODECID_MASK);
- sample_rate = par->sample_rate;
- avcodec_parameters_free(&par);
+ } else if (stream_type == FLV_STREAM_TYPE_VIDEO) {
+ int sret = flv_set_video_codec(s, st, codec_id, 1);
+ if (sret < 0)
+ return sret;
+ size -= sret;
+ track_size -= sret;
+ } else if (stream_type == FLV_STREAM_TYPE_SUBTITLE) {
+ st->codecpar->codec_id = AV_CODEC_ID_TEXT;
+ } else if (stream_type == FLV_STREAM_TYPE_DATA) {
+ st->codecpar->codec_id = AV_CODEC_ID_NONE; // Opaque AMF data
}
- } else if (stream_type == FLV_STREAM_TYPE_AUDIO) {
- if (!st->codecpar->codec_id)
- flv_set_audio_codec(s, st, st->codecpar,
- codec_id ? codec_id : (flags & FLV_AUDIO_CODECID_MASK));
-
- // These are not signalled in the flags anymore
- channels = 0;
- sample_rate = 0;
- if (pkt_type == AudioPacketTypeMultichannelConfig) {
- int channel_order = avio_r8(s->pb);
- channels = avio_r8(s->pb);
- size -= 2;
+ if (st->codecpar->codec_id == AV_CODEC_ID_AAC ||
+ st->codecpar->codec_id == AV_CODEC_ID_OPUS ||
+ st->codecpar->codec_id == AV_CODEC_ID_FLAC ||
+ st->codecpar->codec_id == AV_CODEC_ID_H264 ||
+ st->codecpar->codec_id == AV_CODEC_ID_MPEG4 ||
+ st->codecpar->codec_id == AV_CODEC_ID_HEVC ||
+ st->codecpar->codec_id == AV_CODEC_ID_AV1 ||
+ st->codecpar->codec_id == AV_CODEC_ID_VP9) {
+ int type = 0;
+ if (enhanced_flv) {
+ type = pkt_type;
+ } else {
+ type = avio_r8(s->pb);
+ size--;
+ track_size--;
+ }
- av_channel_layout_uninit(&st->codecpar->ch_layout);
+ if (size < 0 || track_size < 0) {
+ ret = AVERROR_INVALIDDATA;
+ goto leave;
+ }
- if (channel_order == AudioChannelOrderCustom) {
- ret = av_channel_layout_custom_init(&st->codecpar->ch_layout, channels);
- if (ret < 0)
- return ret;
+ if (enhanced_flv && stream_type == FLV_STREAM_TYPE_VIDEO && flv->meta_color_info_flag) {
+ flv_update_video_color_info(s, st); // update av packet side data
+ flv->meta_color_info_flag = 0;
+ }
- for (i = 0; i < channels; i++) {
- uint8_t id = avio_r8(s->pb);
- size--;
-
- if (id < 18)
- st->codecpar->ch_layout.u.map[i].id = id;
- else if (id >= 18 && id <= 23)
- st->codecpar->ch_layout.u.map[i].id = id - 18 + AV_CHAN_LOW_FREQUENCY_2;
- else if (id == 0xFE)
- st->codecpar->ch_layout.u.map[i].id = AV_CHAN_UNUSED;
- else
- st->codecpar->ch_layout.u.map[i].id = AV_CHAN_UNKNOWN;
+ if (st->codecpar->codec_id == AV_CODEC_ID_MPEG4 ||
+ (st->codecpar->codec_id == AV_CODEC_ID_H264 && (!enhanced_flv || type == PacketTypeCodedFrames)) ||
+ (st->codecpar->codec_id == AV_CODEC_ID_HEVC && type == PacketTypeCodedFrames)) {
+ // sign extension
+ int32_t cts = (avio_rb24(s->pb) + 0xff800000) ^ 0xff800000;
+ pts = av_sat_add64(dts, cts);
+ if (cts < 0) { // dts might be wrong
+ if (!flv->wrong_dts)
+ av_log(s, AV_LOG_WARNING,
+ "Negative cts, previous timestamps might be wrong.\n");
+ flv->wrong_dts = 1;
+ } else if (FFABS(dts - pts) > 1000*60*15) {
+ av_log(s, AV_LOG_WARNING,
+ "invalid timestamps %"PRId64" %"PRId64"\n", dts, pts);
+ dts = pts = AV_NOPTS_VALUE;
}
- } else if (channel_order == AudioChannelOrderNative) {
- uint64_t mask = avio_rb32(s->pb);
- size -= 4;
-
- // The first 18 entries in the mask match ours, but the remaining 6 entries start at AV_CHAN_LOW_FREQUENCY_2
- mask = (mask & 0x3FFFF) | ((mask & 0xFC0000) << (AV_CHAN_LOW_FREQUENCY_2 - 18));
- ret = av_channel_layout_from_mask(&st->codecpar->ch_layout, mask);
- if (ret < 0)
- return ret;
- } else {
- av_channel_layout_default(&st->codecpar->ch_layout, channels);
+ size -= 3;
+ track_size -= 3;
}
+ if (type == 0 && (!st->codecpar->extradata || st->codecpar->codec_id == AV_CODEC_ID_AAC ||
+ st->codecpar->codec_id == AV_CODEC_ID_OPUS || st->codecpar->codec_id == AV_CODEC_ID_FLAC ||
+ st->codecpar->codec_id == AV_CODEC_ID_H264 || st->codecpar->codec_id == AV_CODEC_ID_HEVC ||
+ st->codecpar->codec_id == AV_CODEC_ID_AV1 || st->codecpar->codec_id == AV_CODEC_ID_VP9)) {
+ AVDictionaryEntry *t;
+
+ if (st->codecpar->extradata) {
+ if ((ret = flv_queue_extradata(flv, s->pb, multitrack ? track_idx : stream_type, track_size, multitrack)) < 0)
+ return ret;
+ ret = FFERROR_REDO;
+ goto leave;
+ }
+ if ((ret = flv_get_extradata(s, st, track_size)) < 0)
+ return ret;
- av_log(s, AV_LOG_DEBUG, "Set channel data from MultiChannel info.\n");
-
- goto leave;
- }
- } else if (stream_type == FLV_STREAM_TYPE_VIDEO) {
- int sret = flv_set_video_codec(s, st, codec_id, 1);
- if (sret < 0)
- return sret;
- size -= sret;
- } else if (stream_type == FLV_STREAM_TYPE_SUBTITLE) {
- st->codecpar->codec_id = AV_CODEC_ID_TEXT;
- } else if (stream_type == FLV_STREAM_TYPE_DATA) {
- st->codecpar->codec_id = AV_CODEC_ID_NONE; // Opaque AMF data
- }
+ /* Workaround for buggy Omnia A/XE encoder */
+ t = av_dict_get(s->metadata, "Encoder", NULL, 0);
+ if (st->codecpar->codec_id == AV_CODEC_ID_AAC && t && !strcmp(t->value, "Omnia A/XE"))
+ st->codecpar->extradata_size = 2;
- if (st->codecpar->codec_id == AV_CODEC_ID_AAC ||
- st->codecpar->codec_id == AV_CODEC_ID_OPUS ||
- st->codecpar->codec_id == AV_CODEC_ID_FLAC ||
- st->codecpar->codec_id == AV_CODEC_ID_H264 ||
- st->codecpar->codec_id == AV_CODEC_ID_MPEG4 ||
- st->codecpar->codec_id == AV_CODEC_ID_HEVC ||
- st->codecpar->codec_id == AV_CODEC_ID_AV1 ||
- st->codecpar->codec_id == AV_CODEC_ID_VP9) {
- int type = 0;
- if (enhanced_flv) {
- type = pkt_type;
- } else {
- type = avio_r8(s->pb);
- size--;
+ ret = FFERROR_REDO;
+ goto leave;
+ }
}
- if (size < 0) {
- ret = AVERROR_INVALIDDATA;
+ /* skip empty data packets */
+ if (!size) {
+ ret = FFERROR_REDO;
goto leave;
}
- if (enhanced_flv && stream_type == FLV_STREAM_TYPE_VIDEO && flv->meta_color_info_flag) {
- flv_update_video_color_info(s, st); // update av packet side data
- flv->meta_color_info_flag = 0;
- }
-
- if (st->codecpar->codec_id == AV_CODEC_ID_MPEG4 ||
- (st->codecpar->codec_id == AV_CODEC_ID_H264 && (!enhanced_flv || type == PacketTypeCodedFrames)) ||
- (st->codecpar->codec_id == AV_CODEC_ID_HEVC && type == PacketTypeCodedFrames)) {
- // sign extension
- int32_t cts = (avio_rb24(s->pb) + 0xff800000) ^ 0xff800000;
- pts = av_sat_add64(dts, cts);
- if (cts < 0) { // dts might be wrong
- if (!flv->wrong_dts)
- av_log(s, AV_LOG_WARNING,
- "Negative cts, previous timestamps might be wrong.\n");
- flv->wrong_dts = 1;
- } else if (FFABS(dts - pts) > 1000*60*15) {
- av_log(s, AV_LOG_WARNING,
- "invalid timestamps %"PRId64" %"PRId64"\n", dts, pts);
- dts = pts = AV_NOPTS_VALUE;
+ /* skip empty data track */
+ if (!track_size)
+ goto next_track;
+
+ ret = av_get_packet(s->pb, pkt, track_size);
+ if (ret < 0)
+ return ret;
+
+ track_size -= ret;
+ size -= ret;
+
+ pkt->dts = dts;
+ pkt->pts = pts == AV_NOPTS_VALUE ? dts : pts;
+ pkt->stream_index = st->index;
+ pkt->pos = pos;
+ if (!multitrack && flv->new_extradata[stream_type]) {
+ ret = av_packet_add_side_data(pkt, AV_PKT_DATA_NEW_EXTRADATA,
+ flv->new_extradata[stream_type],
+ flv->new_extradata_size[stream_type]);
+ if (ret >= 0) {
+ flv->new_extradata[stream_type] = NULL;
+ flv->new_extradata_size[stream_type] = 0;
}
- size -= 3;
- }
- if (type == 0 && (!st->codecpar->extradata || st->codecpar->codec_id == AV_CODEC_ID_AAC ||
- st->codecpar->codec_id == AV_CODEC_ID_OPUS || st->codecpar->codec_id == AV_CODEC_ID_FLAC ||
- st->codecpar->codec_id == AV_CODEC_ID_H264 || st->codecpar->codec_id == AV_CODEC_ID_HEVC ||
- st->codecpar->codec_id == AV_CODEC_ID_AV1 || st->codecpar->codec_id == AV_CODEC_ID_VP9)) {
- AVDictionaryEntry *t;
-
- if (st->codecpar->extradata) {
- if ((ret = flv_queue_extradata(flv, s->pb, multitrack ? track_idx : stream_type, size, multitrack)) < 0)
- return ret;
- ret = FFERROR_REDO;
- goto leave;
+ } else if (multitrack
+ && flv->mt_extradata_cnt > track_idx
+ && flv->mt_extradata[track_idx]) {
+ ret = av_packet_add_side_data(pkt, AV_PKT_DATA_NEW_EXTRADATA,
+ flv->mt_extradata[track_idx],
+ flv->mt_extradata_sz[track_idx]);
+ if (ret >= 0) {
+ flv->mt_extradata[track_idx] = NULL;
+ flv->mt_extradata_sz[track_idx] = 0;
}
- if ((ret = flv_get_extradata(s, st, size)) < 0)
- return ret;
-
- /* Workaround for buggy Omnia A/XE encoder */
- t = av_dict_get(s->metadata, "Encoder", NULL, 0);
- if (st->codecpar->codec_id == AV_CODEC_ID_AAC && t && !strcmp(t->value, "Omnia A/XE"))
- st->codecpar->extradata_size = 2;
+ }
+ if (stream_type == FLV_STREAM_TYPE_AUDIO && !enhanced_flv &&
+ (sample_rate != flv->last_sample_rate ||
+ channels != flv->last_channels)) {
+ flv->last_sample_rate = sample_rate;
+ flv->last_channels = channels;
+ ff_add_param_change(pkt, channels, 0, sample_rate, 0, 0);
+ }
- ret = FFERROR_REDO;
- goto leave;
+ if (stream_type == FLV_STREAM_TYPE_AUDIO ||
+ (flags & FLV_VIDEO_FRAMETYPE_MASK) == FLV_FRAME_KEY ||
+ stream_type == FLV_STREAM_TYPE_SUBTITLE ||
+ stream_type == FLV_STREAM_TYPE_DATA)
+ pkt->flags |= AV_PKT_FLAG_KEY;
+
+ ret = ff_buffer_packet(s, pkt);
+ if (ret < 0)
+ return ret;
+ res = FFERROR_REDO;
+
+ if (track_size) {
+ av_log(s, AV_LOG_WARNING, "Track size mismatch: %d!\n", track_size);
+ avio_skip(s->pb, track_size);
+ size -= track_size;
}
- }
- /* skip empty data packets */
- if (!size) {
- ret = FFERROR_REDO;
- goto leave;
- }
+ if (!size)
+ break;
- ret = av_get_packet(s->pb, pkt, size);
- if (ret < 0)
- return ret;
- pkt->dts = dts;
- pkt->pts = pts == AV_NOPTS_VALUE ? dts : pts;
- pkt->stream_index = st->index;
- pkt->pos = pos;
- if (!multitrack && flv->new_extradata[stream_type]) {
- int sret = av_packet_add_side_data(pkt, AV_PKT_DATA_NEW_EXTRADATA,
- flv->new_extradata[stream_type],
- flv->new_extradata_size[stream_type]);
- if (sret >= 0) {
- flv->new_extradata[stream_type] = NULL;
- flv->new_extradata_size[stream_type] = 0;
+next_track:
+ if (multitrack_type == MultitrackTypeOneTrack) {
+ av_log(s, AV_LOG_ERROR, "Attempted to read next track in single-track mode.\n");
+ return AVERROR_BUG;
}
- } else if (multitrack
- && flv->mt_extradata_cnt > track_idx
- && flv->mt_extradata[track_idx]) {
- int sret = av_packet_add_side_data(pkt, AV_PKT_DATA_NEW_EXTRADATA,
- flv->mt_extradata[track_idx],
- flv->mt_extradata_sz[track_idx]);
- if (sret >= 0) {
- flv->mt_extradata[track_idx] = NULL;
- flv->mt_extradata_sz[track_idx] = 0;
+
+ if (multitrack_type == MultitrackTypeManyTracksManyCodecs) {
+ codec_id = avio_rb32(s->pb);
+ size -= 4;
}
- }
- if (stream_type == FLV_STREAM_TYPE_AUDIO && !enhanced_flv &&
- (sample_rate != flv->last_sample_rate ||
- channels != flv->last_channels)) {
- flv->last_sample_rate = sample_rate;
- flv->last_channels = channels;
- ff_add_param_change(pkt, channels, 0, sample_rate, 0, 0);
- }
- if (stream_type == FLV_STREAM_TYPE_AUDIO ||
- (flags & FLV_VIDEO_FRAMETYPE_MASK) == FLV_FRAME_KEY ||
- stream_type == FLV_STREAM_TYPE_SUBTITLE ||
- stream_type == FLV_STREAM_TYPE_DATA)
- pkt->flags |= AV_PKT_FLAG_KEY;
+ track_idx = avio_r8(s->pb);
+ size--;
+ }
leave:
last = avio_rb32(s->pb);
@@ -1755,7 +1797,7 @@ leave:
if (ret >= 0)
flv->last_ts = pkt->dts;
- return ret;
+ return ret ? ret : res;
}
static int flv_read_seek(AVFormatContext *s, int stream_index,
--
2.43.2
_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".
^ permalink raw reply [flat|nested] 25+ messages in thread
* Re: [FFmpeg-devel] [PATCH 01/13] avformat/flvenc: Implement support for multi-track video
2024-05-21 9:02 ` [FFmpeg-devel] [PATCH 01/13] avformat/flvenc: Implement support for multi-track video Timo Rothenpieler
@ 2024-05-21 18:48 ` Dennis Sädtler via ffmpeg-devel
2024-05-21 18:50 ` Timo Rothenpieler
[not found] ` <98D1BB90-39C6-413B-A158-8ADA201155BC@cosmin.at>
0 siblings, 2 replies; 25+ messages in thread
From: Dennis Sädtler via ffmpeg-devel @ 2024-05-21 18:48 UTC (permalink / raw)
To: Timo Rothenpieler; +Cc: Dennis Sädtler, ffmpeg-devel
> From: Dennis Sädtler via ffmpeg-devel <ffmpeg-devel@ffmpeg.org>
I wonder what happened here, did I make a mistake when submitting the
original patch to the ML so the actual commit author name/email got
lost?
Should be the same as the signed-off section based on the repo I
submitted it from:
https://github.com/derrod/ffmpeg/commit/25f1700cffa00fcd04bcc27efce077a93e7f5142
Cheers,
Dennis
_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".
^ permalink raw reply [flat|nested] 25+ messages in thread
* Re: [FFmpeg-devel] [PATCH 01/13] avformat/flvenc: Implement support for multi-track video
2024-05-21 18:48 ` Dennis Sädtler via ffmpeg-devel
@ 2024-05-21 18:50 ` Timo Rothenpieler
[not found] ` <98D1BB90-39C6-413B-A158-8ADA201155BC@cosmin.at>
1 sibling, 0 replies; 25+ messages in thread
From: Timo Rothenpieler @ 2024-05-21 18:50 UTC (permalink / raw)
To: Dennis Sädtler; +Cc: ffmpeg-devel
On 21.05.2024 20:48, Dennis Sädtler wrote:
>> From: Dennis Sädtler via ffmpeg-devel <ffmpeg-devel@ffmpeg.org>
>
> I wonder what happened here, did I make a mistake when submitting the
> original patch to the ML so the actual commit author name/email got
> lost?
That's just what happens if your mailserver has strict SPF/DMARC configured.
Since no other server can send mails from its domain then, Mailing-Lists
are forced to mangle it.
> Should be the same as the signed-off section based on the repo I
> submitted it from:
> https://github.com/derrod/ffmpeg/commit/25f1700cffa00fcd04bcc27efce077a93e7f5142
I'll fix it locally, hadn't noticed that yet.
_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".
^ permalink raw reply [flat|nested] 25+ messages in thread
* Re: [FFmpeg-devel] [PATCH 01/13] avformat/flvenc: Implement support for multi-track video
[not found] ` <98D1BB90-39C6-413B-A158-8ADA201155BC@cosmin.at>
@ 2024-05-21 18:54 ` Cosmin Stejerean via ffmpeg-devel
[not found] ` <7F89B9B2-FBC2-49E7-AC4F-68E41B584E1A@cosmin.at>
0 siblings, 1 reply; 25+ messages in thread
From: Cosmin Stejerean via ffmpeg-devel @ 2024-05-21 18:54 UTC (permalink / raw)
To: FFmpeg development discussions and patches
Cc: Cosmin Stejerean, Dennis Sädtler, Timo Rothenpieler
> On May 21, 2024, at 11:48 AM, Dennis Sädtler via ffmpeg-devel <ffmpeg-devel@ffmpeg.org> wrote:
>
>> From: Dennis Sädtler via ffmpeg-devel <ffmpeg-devel@ffmpeg.org>
>
> I wonder what happened here, did I make a mistake when submitting the
> original patch to the ML so the actual commit author name/email got
> lost?
>
> Should be the same as the signed-off section based on the repo I
> submitted it from:
> https://github.com/derrod/ffmpeg/commit/25f1700cffa00fcd04bcc27efce077a93e7f5142
>
The problem is the mailing list rewrites the message (to for example add the footer at the bottom). It cannot do that and keep the from email address if you have email security configured properly for the domain (as it would get rejected).
So the mailing list changes the from to ".... via ffmpeg-devel <ffmpeg-devel@ffmpeg.org>". Furthermore git send-email won't include "From:" in the patch body if the commit authorship matches the from email address on git send-email.
However I've found a workaround. By setting my git send-email from to just "cosmin@cosmin.at <mailto:cosmin@cosmin.at>" rather than "Cosmin Stejerean <cosmin@cosmin.at <mailto:cosmin@cosmin.at>>" then the emails that go out will have the "From: " as the first line of the body, so the patches apply properly. You can see my most recent patch for example.
You can add an entry to .mailmap to handle any previous commits that have been pushed using the "via ffmpeg-devel" rewrite, see my entry for example.
- Cosmin
_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".
^ permalink raw reply [flat|nested] 25+ messages in thread
* Re: [FFmpeg-devel] [PATCH 01/13] avformat/flvenc: Implement support for multi-track video
[not found] ` <7F89B9B2-FBC2-49E7-AC4F-68E41B584E1A@cosmin.at>
@ 2024-05-21 18:56 ` Cosmin Stejerean via ffmpeg-devel
2024-05-21 18:56 ` Cosmin Stejerean via ffmpeg-devel
1 sibling, 0 replies; 25+ messages in thread
From: Cosmin Stejerean via ffmpeg-devel @ 2024-05-21 18:56 UTC (permalink / raw)
To: FFmpeg development discussions and patches
Cc: Cosmin Stejerean, Dennis Sädtler, Timo Rothenpieler
> On May 21, 2024, at 11:54 AM, Cosmin Stejerean via ffmpeg-devel <ffmpeg-devel@ffmpeg.org> wrote:
>
>
> However I've found a workaround. By setting my git send-email from to just "cosmin@cosmin.at <mailto:cosmin@cosmin.at>" rather than "Cosmin Stejerean <cosmin@cosmin.at <mailto:cosmin@cosmin.at>>" then the emails that go out will have the "From: " as the first line of the body, so the patches apply properly. You can see my most recent patch for example.
>
The email client is trying to be too helpful here adding extra formatting to the message body. The .gitconfig should look something like this, using your email address without <> and without the name.
from = cosmin@cosmin.at
That works in my testing.
- Cosmin
_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".
^ permalink raw reply [flat|nested] 25+ messages in thread
* Re: [FFmpeg-devel] [PATCH 01/13] avformat/flvenc: Implement support for multi-track video
[not found] ` <7F89B9B2-FBC2-49E7-AC4F-68E41B584E1A@cosmin.at>
2024-05-21 18:56 ` Cosmin Stejerean via ffmpeg-devel
@ 2024-05-21 18:56 ` Cosmin Stejerean via ffmpeg-devel
1 sibling, 0 replies; 25+ messages in thread
From: Cosmin Stejerean via ffmpeg-devel @ 2024-05-21 18:56 UTC (permalink / raw)
To: FFmpeg development discussions and patches
Cc: Cosmin Stejerean, Dennis Sädtler, Timo Rothenpieler
> On May 21, 2024, at 11:54 AM, Cosmin Stejerean via ffmpeg-devel <ffmpeg-devel@ffmpeg.org> wrote:
>
>
> However I've found a workaround. By setting my git send-email from to just "cosmin@cosmin.at <mailto:cosmin@cosmin.at>" rather than "Cosmin Stejerean <cosmin@cosmin.at <mailto:cosmin@cosmin.at>>" then the emails that go out will have the "From: " as the first line of the body, so the patches apply properly. You can see my most recent patch for example.
>
The email client is trying to be too helpful here adding extra formatting to the message body. The .gitconfig should look something like this, using your email address without <> and without the name.
from = cosmin@cosmin.at
That works in my testing.
- Cosmin
_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".
^ permalink raw reply [flat|nested] 25+ messages in thread
* Re: [FFmpeg-devel] [PATCH 13/13] avformat/flvdec: support all multi-track modes
2024-05-21 9:02 ` [FFmpeg-devel] [PATCH 13/13] avformat/flvdec: support all multi-track modes Timo Rothenpieler
@ 2024-05-22 0:02 ` Michael Niedermayer
2024-05-22 18:26 ` Timo Rothenpieler
2024-05-22 18:27 ` [FFmpeg-devel] [PATCH v2 " Timo Rothenpieler
1 sibling, 1 reply; 25+ messages in thread
From: Michael Niedermayer @ 2024-05-22 0:02 UTC (permalink / raw)
To: FFmpeg development discussions and patches
[-- Attachment #1.1: Type: text/plain, Size: 5247 bytes --]
On Tue, May 21, 2024 at 11:02:22AM +0200, Timo Rothenpieler wrote:
> ---
> libavformat/flvdec.c | 570 +++++++++++++++++++++++--------------------
> 1 file changed, 306 insertions(+), 264 deletions(-)
infinite loops
[flv @ 0x555e803d2940] Video codec (0) is not implemented. Update your FFmpeg version to the newest one from Git. If the problem still occurs, it means that your file has a feature which has not been implemented.
[flv @ 0x555e803d2940] If you want to help, upload a sample of this file to https://streams.videolan.org/upload/ and contact the ffmpeg-devel mailing list. (ffmpeg-devel@ffmpeg.org)
[flv @ 0x555e803d2940] Video codec (0) is not implemented. Update your FFmpeg version to the newest one from Git. If the problem still occurs, it means that your file has a feature which has not been implemented.
[flv @ 0x555e803d2940] If you want to help, upload a sample of this file to https://streams.videolan.org/upload/ and contact the ffmpeg-devel mailing list. (ffmpeg-devel@ffmpeg.org)
[flv @ 0x555e803d2940] Video codec (0) is not implemented. Update your FFmpeg version to the newest one from Git. If the problem still occurs, it means that your file has a feature which has not been implemented.
[flv @ 0x555e803d2940] If you want to help, upload a sample of this file to https://streams.videolan.org/upload/ and contact the ffmpeg-devel mailing list. (ffmpeg-devel@ffmpeg.org)
[flv @ 0x555e803d2940] Video codec (0) is not implemented. Update your FFmpeg version to the newest one from Git. If the problem still occurs, it means that your file has a feature which has not been implemented.
[flv @ 0x555e803d2940] If you want to help, upload a sample of this file to https://streams.videolan.org/upload/ and contact the ffmpeg-devel mailing list. (ffmpeg-devel@ffmpeg.org)
[flv @ 0x555e803d2940] Video codec (0) is not implemented. Update your FFmpeg version to the newest one from Git. If the problem still occurs, it means that your file has a feature which has not been implemented.
[flv @ 0x555e803d2940] If you want to help, upload a sample of this file to https://streams.videolan.org/upload/ and contact the ffmpeg-devel mailing list. (ffmpeg-devel@ffmpeg.org)
[flv @ 0x555e803d2940] Video codec (0) is not implemented. Update your FFmpeg version to the newest one from Git. If the problem still occurs, it means that your file has a feature which has not been implemented.
[flv @ 0x555e803d2940] If you want to help, upload a sample of this file to https://streams.videolan.org/upload/ and contact the ffmpeg-devel mailing list. (ffmpeg-devel@ffmpeg.org)
[flv @ 0x555e803d2940] Video codec (0) is not implemented. Update your FFmpeg version to the newest one from Git. If the problem still occurs, it means that your file has a feature which has not been implemented.
[flv @ 0x555e803d2940] If you want to help, upload a sample of this file to https://streams.videolan.org/upload/ and contact the ffmpeg-devel mailing list. (ffmpeg-devel@ffmpeg.org)
[flv @ 0x555e803d2940] Video codec (0) is not implemented. Update your FFmpeg version to the newest one from Git. If the problem still occurs, it means that your file has a feature which has not been implemented.
[flv @ 0x555e803d2940] If you want to help, upload a sample of this file to https://streams.videolan.org/upload/ and contact the ffmpeg-devel mailing list. (ffmpeg-devel@ffmpeg.org)
[flv @ 0x555e803d2940] Video codec (0) is not implemented. Update your FFmpeg version to the newest one from Git. If the problem still occurs, it means that your file has a feature which has not been implemented.
[flv @ 0x555e803d2940] If you want to help, upload a sample of this file to https://streams.videolan.org/upload/ and contact the ffmpeg-devel mailing list. (ffmpeg-devel@ffmpeg.org)
[flv @ 0x555e803d2940] Video codec (0) is not implemented. Update your FFmpeg version to the newest one from Git. If the problem still occurs, it means that your file has a feature which has not been implemented.
[flv @ 0x555e803d2940] If you want to help, upload a sample of this file to https://streams.videolan.org/upload/ and contact the ffmpeg-devel mailing list. (ffmpeg-devel@ffmpeg.org)
[flv @ 0x555e803d2940] Video codec (0) is not implemented. Update your FFmpeg version to the newest one from Git. If the problem still occurs, it means that your file has a feature which has not been implemented.
[flv @ 0x555e803d2940] If you want to help, upload a sample of this file to https://streams.videolan.org/upload/ and contact the ffmpeg-devel mailing list. (ffmpeg-devel@ffmpeg.org)
[flv @ 0x555e803d2940] Video codec (0) is not implemented. Update your FFmpeg version to the newest one from Git. If the problem still occurs, it means that your file has a feature which has not been implemented.
[flv @ 0x555e803d2940] If you want to help, upload a sample of this file to https://streams.videolan.org/upload/ and contact the ffmpeg-devel mailing list. (ffmpeg-devel@ffmpeg.org)
CReceived > 3 system signals, hard exiting
ill mail you the source file privately
thx
[...]
--
Michael GnuPG fingerprint: 9FF2128B147EF6730BADF133611EC787040B0FAB
Observe your enemies, for they first find out your faults. -- Antisthenes
[-- Attachment #1.2: signature.asc --]
[-- Type: application/pgp-signature, Size: 195 bytes --]
[-- Attachment #2: Type: text/plain, Size: 251 bytes --]
_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".
^ permalink raw reply [flat|nested] 25+ messages in thread
* Re: [FFmpeg-devel] [PATCH 13/13] avformat/flvdec: support all multi-track modes
2024-05-22 0:02 ` Michael Niedermayer
@ 2024-05-22 18:26 ` Timo Rothenpieler
0 siblings, 0 replies; 25+ messages in thread
From: Timo Rothenpieler @ 2024-05-22 18:26 UTC (permalink / raw)
To: FFmpeg development discussions and patches, Michael Niedermayer
On 22.05.2024 02:02, Michael Niedermayer wrote:
> On Tue, May 21, 2024 at 11:02:22AM +0200, Timo Rothenpieler wrote:
>> ---
>> libavformat/flvdec.c | 570 +++++++++++++++++++++++--------------------
>> 1 file changed, 306 insertions(+), 264 deletions(-)
>
> infinite loops
>
> [flv @ 0x555e803d2940] Video codec (0) is not implemented. Update your FFmpeg version to the newest one from Git. If the problem still occurs, it means that your file has a feature which has not been implemented.
> [flv @ 0x555e803d2940] If you want to help, upload a sample of this file to https://streams.videolan.org/upload/ and contact the ffmpeg-devel mailing list. (ffmpeg-devel@ffmpeg.org)
> [flv @ 0x555e803d2940] Video codec (0) is not implemented. Update your FFmpeg version to the newest one from Git. If the problem still occurs, it means that your file has a feature which has not been implemented.
> [flv @ 0x555e803d2940] If you want to help, upload a sample of this file to https://streams.videolan.org/upload/ and contact the ffmpeg-devel mailing list. (ffmpeg-devel@ffmpeg.org)
> [flv @ 0x555e803d2940] Video codec (0) is not implemented. Update your FFmpeg version to the newest one from Git. If the problem still occurs, it means that your file has a feature which has not been implemented.
> [flv @ 0x555e803d2940] If you want to help, upload a sample of this file to https://streams.videolan.org/upload/ and contact the ffmpeg-devel mailing list. (ffmpeg-devel@ffmpeg.org)
> [flv @ 0x555e803d2940] Video codec (0) is not implemented. Update your FFmpeg version to the newest one from Git. If the problem still occurs, it means that your file has a feature which has not been implemented.
> [flv @ 0x555e803d2940] If you want to help, upload a sample of this file to https://streams.videolan.org/upload/ and contact the ffmpeg-devel mailing list. (ffmpeg-devel@ffmpeg.org)
> [flv @ 0x555e803d2940] Video codec (0) is not implemented. Update your FFmpeg version to the newest one from Git. If the problem still occurs, it means that your file has a feature which has not been implemented.
> [flv @ 0x555e803d2940] If you want to help, upload a sample of this file to https://streams.videolan.org/upload/ and contact the ffmpeg-devel mailing list. (ffmpeg-devel@ffmpeg.org)
> [flv @ 0x555e803d2940] Video codec (0) is not implemented. Update your FFmpeg version to the newest one from Git. If the problem still occurs, it means that your file has a feature which has not been implemented.
> [flv @ 0x555e803d2940] If you want to help, upload a sample of this file to https://streams.videolan.org/upload/ and contact the ffmpeg-devel mailing list. (ffmpeg-devel@ffmpeg.org)
> [flv @ 0x555e803d2940] Video codec (0) is not implemented. Update your FFmpeg version to the newest one from Git. If the problem still occurs, it means that your file has a feature which has not been implemented.
> [flv @ 0x555e803d2940] If you want to help, upload a sample of this file to https://streams.videolan.org/upload/ and contact the ffmpeg-devel mailing list. (ffmpeg-devel@ffmpeg.org)
> [flv @ 0x555e803d2940] Video codec (0) is not implemented. Update your FFmpeg version to the newest one from Git. If the problem still occurs, it means that your file has a feature which has not been implemented.
> [flv @ 0x555e803d2940] If you want to help, upload a sample of this file to https://streams.videolan.org/upload/ and contact the ffmpeg-devel mailing list. (ffmpeg-devel@ffmpeg.org)
> [flv @ 0x555e803d2940] Video codec (0) is not implemented. Update your FFmpeg version to the newest one from Git. If the problem still occurs, it means that your file has a feature which has not been implemented.
> [flv @ 0x555e803d2940] If you want to help, upload a sample of this file to https://streams.videolan.org/upload/ and contact the ffmpeg-devel mailing list. (ffmpeg-devel@ffmpeg.org)
> [flv @ 0x555e803d2940] Video codec (0) is not implemented. Update your FFmpeg version to the newest one from Git. If the problem still occurs, it means that your file has a feature which has not been implemented.
> [flv @ 0x555e803d2940] If you want to help, upload a sample of this file to https://streams.videolan.org/upload/ and contact the ffmpeg-devel mailing list. (ffmpeg-devel@ffmpeg.org)
> [flv @ 0x555e803d2940] Video codec (0) is not implemented. Update your FFmpeg version to the newest one from Git. If the problem still occurs, it means that your file has a feature which has not been implemented.
> [flv @ 0x555e803d2940] If you want to help, upload a sample of this file to https://streams.videolan.org/upload/ and contact the ffmpeg-devel mailing list. (ffmpeg-devel@ffmpeg.org)
> [flv @ 0x555e803d2940] Video codec (0) is not implemented. Update your FFmpeg version to the newest one from Git. If the problem still occurs, it means that your file has a feature which has not been implemented.
> [flv @ 0x555e803d2940] If you want to help, upload a sample of this file to https://streams.videolan.org/upload/ and contact the ffmpeg-devel mailing list. (ffmpeg-devel@ffmpeg.org)
> CReceived > 3 system signals, hard exiting
Hm, what's happening here is that the sample has a packet which has an
enormous size value, but obviously not actually half a million of tracks.
It's also not technically an infinite loop, but slowly counting down an
enormous size.
Since eof is not checked anywhere, it happily keeps doing that.
There was also no check for size < 0 anywhere, so it over-reading and
setting size to a negative value would make it run even longer.
Added both sanity checks for over-reading the size, and also eof in the
middle of reading in the loop.
That fixed the issue for me.
Will send v2 shortly.
_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".
^ permalink raw reply [flat|nested] 25+ messages in thread
* [FFmpeg-devel] [PATCH v2 13/13] avformat/flvdec: support all multi-track modes
2024-05-21 9:02 ` [FFmpeg-devel] [PATCH 13/13] avformat/flvdec: support all multi-track modes Timo Rothenpieler
2024-05-22 0:02 ` Michael Niedermayer
@ 2024-05-22 18:27 ` Timo Rothenpieler
1 sibling, 0 replies; 25+ messages in thread
From: Timo Rothenpieler @ 2024-05-22 18:27 UTC (permalink / raw)
To: ffmpeg-devel; +Cc: Timo Rothenpieler
---
libavformat/flvdec.c | 574 +++++++++++++++++++++++--------------------
1 file changed, 311 insertions(+), 263 deletions(-)
diff --git a/libavformat/flvdec.c b/libavformat/flvdec.c
index 4f98ff348c..fe5a44a715 100644
--- a/libavformat/flvdec.c
+++ b/libavformat/flvdec.c
@@ -1273,6 +1273,7 @@ static int flv_read_packet(AVFormatContext *s, AVPacket *pkt)
{
FLVContext *flv = s->priv_data;
int ret, i, size, flags;
+ int res = 0;
enum FlvTagType type;
int stream_type=-1;
int64_t next, pos, meta_pos;
@@ -1287,6 +1288,7 @@ static int flv_read_packet(AVFormatContext *s, AVPacket *pkt)
int pkt_type = 0;
uint8_t track_idx = 0;
uint32_t codec_id = 0;
+ int multitrack_type = MultitrackTypeOneTrack;
retry:
/* pkt size is repeated at end. skip it */
@@ -1337,14 +1339,9 @@ retry:
if (pkt_type == AudioPacketTypeMultitrack) {
uint8_t types = avio_r8(s->pb);
- int multitrack_type = types >> 4;
+ multitrack_type = types & 0xF0;
pkt_type = types & 0xF;
- if (multitrack_type != MultitrackTypeOneTrack) {
- av_log(s, AV_LOG_ERROR, "Audio multitrack types other than MultitrackTypeOneTrack are unsupported!\n");
- return AVERROR_PATCHWELCOME;
- }
-
multitrack = 1;
size--;
}
@@ -1371,14 +1368,9 @@ retry:
if (pkt_type == PacketTypeMultitrack) {
uint8_t types = avio_r8(s->pb);
- int multitrack_type = types >> 4;
+ multitrack_type = types & 0xF0;
pkt_type = types & 0xF;
- if (multitrack_type != MultitrackTypeOneTrack) {
- av_log(s, AV_LOG_ERROR, "Multitrack types other than MultitrackTypeOneTrack are unsupported!\n");
- return AVERROR_PATCHWELCOME;
- }
-
multitrack = 1;
size--;
}
@@ -1447,293 +1439,349 @@ skip:
goto leave;
}
- /* now find stream */
- for (i = 0; i < s->nb_streams; i++) {
- st = s->streams[i];
- if (stream_type == FLV_STREAM_TYPE_AUDIO) {
- if (st->codecpar->codec_type == AVMEDIA_TYPE_AUDIO &&
- (s->audio_codec_id || flv_same_audio_codec(st->codecpar, flags, codec_id)) &&
- st->id == track_idx)
- break;
- } else if (stream_type == FLV_STREAM_TYPE_VIDEO) {
- if (st->codecpar->codec_type == AVMEDIA_TYPE_VIDEO &&
- (s->video_codec_id || flv_same_video_codec(st->codecpar, codec_id)) &&
- st->id == track_idx)
- break;
- } else if (stream_type == FLV_STREAM_TYPE_SUBTITLE) {
- if (st->codecpar->codec_type == AVMEDIA_TYPE_SUBTITLE)
- break;
- } else if (stream_type == FLV_STREAM_TYPE_DATA) {
- if (st->codecpar->codec_type == AVMEDIA_TYPE_DATA)
- break;
+ for (;;) {
+ int track_size = size;
+
+ if (multitrack_type != MultitrackTypeOneTrack) {
+ track_size = avio_rb24(s->pb);
+ size -= 3;
}
- }
- if (i == s->nb_streams) {
- static const enum AVMediaType stream_types[] = {AVMEDIA_TYPE_VIDEO, AVMEDIA_TYPE_AUDIO, AVMEDIA_TYPE_SUBTITLE, AVMEDIA_TYPE_DATA};
- st = create_stream(s, stream_types[stream_type], track_idx);
- if (!st)
- return AVERROR(ENOMEM);
- }
- av_log(s, AV_LOG_TRACE, "%d %X %d \n", stream_type, flags, st->discard);
- if (flv->time_pos <= pos) {
- dts += flv->time_offset;
- }
+ /* now find stream */
+ for (i = 0; i < s->nb_streams; i++) {
+ st = s->streams[i];
+ if (stream_type == FLV_STREAM_TYPE_AUDIO) {
+ if (st->codecpar->codec_type == AVMEDIA_TYPE_AUDIO &&
+ (s->audio_codec_id || flv_same_audio_codec(st->codecpar, flags, codec_id)) &&
+ st->id == track_idx)
+ break;
+ } else if (stream_type == FLV_STREAM_TYPE_VIDEO) {
+ if (st->codecpar->codec_type == AVMEDIA_TYPE_VIDEO &&
+ (s->video_codec_id || flv_same_video_codec(st->codecpar, codec_id)) &&
+ st->id == track_idx)
+ break;
+ } else if (stream_type == FLV_STREAM_TYPE_SUBTITLE) {
+ if (st->codecpar->codec_type == AVMEDIA_TYPE_SUBTITLE)
+ break;
+ } else if (stream_type == FLV_STREAM_TYPE_DATA) {
+ if (st->codecpar->codec_type == AVMEDIA_TYPE_DATA)
+ break;
+ }
+ }
+ if (i == s->nb_streams) {
+ static const enum AVMediaType stream_types[] = {AVMEDIA_TYPE_VIDEO, AVMEDIA_TYPE_AUDIO, AVMEDIA_TYPE_SUBTITLE, AVMEDIA_TYPE_DATA};
+ st = create_stream(s, stream_types[stream_type], track_idx);
+ if (!st)
+ return AVERROR(ENOMEM);
+ }
+ av_log(s, AV_LOG_TRACE, "%d %X %d \n", stream_type, flags, st->discard);
- if ((s->pb->seekable & AVIO_SEEKABLE_NORMAL) &&
- ((flags & FLV_VIDEO_FRAMETYPE_MASK) == FLV_FRAME_KEY ||
- stream_type == FLV_STREAM_TYPE_AUDIO))
- av_add_index_entry(st, pos, dts, size, 0, AVINDEX_KEYFRAME);
+ if (flv->time_pos <= pos) {
+ dts += flv->time_offset;
+ }
- if ((st->discard >= AVDISCARD_NONKEY && !((flags & FLV_VIDEO_FRAMETYPE_MASK) == FLV_FRAME_KEY || stream_type == FLV_STREAM_TYPE_AUDIO)) ||
- (st->discard >= AVDISCARD_BIDIR && ((flags & FLV_VIDEO_FRAMETYPE_MASK) == FLV_FRAME_DISP_INTER && stream_type == FLV_STREAM_TYPE_VIDEO)) ||
- st->discard >= AVDISCARD_ALL) {
- avio_seek(s->pb, next, SEEK_SET);
- ret = FFERROR_REDO;
- goto leave;
- }
+ if ((s->pb->seekable & AVIO_SEEKABLE_NORMAL) &&
+ ((flags & FLV_VIDEO_FRAMETYPE_MASK) == FLV_FRAME_KEY ||
+ stream_type == FLV_STREAM_TYPE_AUDIO))
+ av_add_index_entry(st, pos, dts, track_size, 0, AVINDEX_KEYFRAME);
+
+ if ((st->discard >= AVDISCARD_NONKEY && !((flags & FLV_VIDEO_FRAMETYPE_MASK) == FLV_FRAME_KEY || stream_type == FLV_STREAM_TYPE_AUDIO)) ||
+ (st->discard >= AVDISCARD_BIDIR && ((flags & FLV_VIDEO_FRAMETYPE_MASK) == FLV_FRAME_DISP_INTER && stream_type == FLV_STREAM_TYPE_VIDEO)) ||
+ st->discard >= AVDISCARD_ALL) {
+ avio_seek(s->pb, next, SEEK_SET);
+ ret = FFERROR_REDO;
+ goto leave;
+ }
- // if not streamed and no duration from metadata then seek to end to find
- // the duration from the timestamps
- if ((s->pb->seekable & AVIO_SEEKABLE_NORMAL) &&
- (!s->duration || s->duration == AV_NOPTS_VALUE) &&
- !flv->searched_for_end) {
- int final_size;
- const int64_t pos = avio_tell(s->pb);
- // Read the last 4 bytes of the file, this should be the size of the
- // previous FLV tag. Use the timestamp of its payload as duration.
- int64_t fsize = avio_size(s->pb);
+ // if not streamed and no duration from metadata then seek to end to find
+ // the duration from the timestamps
+ if ((s->pb->seekable & AVIO_SEEKABLE_NORMAL) &&
+ (!s->duration || s->duration == AV_NOPTS_VALUE) &&
+ !flv->searched_for_end) {
+ int final_size;
+ const int64_t pos = avio_tell(s->pb);
+ // Read the last 4 bytes of the file, this should be the size of the
+ // previous FLV tag. Use the timestamp of its payload as duration.
+ int64_t fsize = avio_size(s->pb);
retry_duration:
- avio_seek(s->pb, fsize - 4, SEEK_SET);
- final_size = avio_rb32(s->pb);
- if (final_size > 0 && final_size < fsize) {
- // Seek to the start of the last FLV tag at position (fsize - 4 - final_size)
- // but skip the byte indicating the type.
- avio_seek(s->pb, fsize - 3 - final_size, SEEK_SET);
- if (final_size == avio_rb24(s->pb) + 11) {
- uint32_t ts = avio_rb24(s->pb);
- ts |= (unsigned)avio_r8(s->pb) << 24;
- if (ts)
- s->duration = ts * (int64_t)AV_TIME_BASE / 1000;
- else if (fsize >= 8 && fsize - 8 >= final_size) {
- fsize -= final_size+4;
- goto retry_duration;
+ avio_seek(s->pb, fsize - 4, SEEK_SET);
+ final_size = avio_rb32(s->pb);
+ if (final_size > 0 && final_size < fsize) {
+ // Seek to the start of the last FLV tag at position (fsize - 4 - final_size)
+ // but skip the byte indicating the type.
+ avio_seek(s->pb, fsize - 3 - final_size, SEEK_SET);
+ if (final_size == avio_rb24(s->pb) + 11) {
+ uint32_t ts = avio_rb24(s->pb);
+ ts |= (unsigned)avio_r8(s->pb) << 24;
+ if (ts)
+ s->duration = ts * (int64_t)AV_TIME_BASE / 1000;
+ else if (fsize >= 8 && fsize - 8 >= final_size) {
+ fsize -= final_size+4;
+ goto retry_duration;
+ }
}
}
+
+ avio_seek(s->pb, pos, SEEK_SET);
+ flv->searched_for_end = 1;
}
- avio_seek(s->pb, pos, SEEK_SET);
- flv->searched_for_end = 1;
- }
+ if (stream_type == FLV_STREAM_TYPE_AUDIO && !enhanced_flv) {
+ int bits_per_coded_sample;
+ channels = (flags & FLV_AUDIO_CHANNEL_MASK) == FLV_STEREO ? 2 : 1;
+ sample_rate = 44100 << ((flags & FLV_AUDIO_SAMPLERATE_MASK) >>
+ FLV_AUDIO_SAMPLERATE_OFFSET) >> 3;
+ bits_per_coded_sample = (flags & FLV_AUDIO_SAMPLESIZE_MASK) ? 16 : 8;
+ if (!av_channel_layout_check(&st->codecpar->ch_layout) ||
+ !st->codecpar->sample_rate ||
+ !st->codecpar->bits_per_coded_sample) {
+ av_channel_layout_default(&st->codecpar->ch_layout, channels);
+ st->codecpar->sample_rate = sample_rate;
+ st->codecpar->bits_per_coded_sample = bits_per_coded_sample;
+ }
+ if (!st->codecpar->codec_id) {
+ flv_set_audio_codec(s, st, st->codecpar,
+ flags & FLV_AUDIO_CODECID_MASK);
+ flv->last_sample_rate =
+ sample_rate = st->codecpar->sample_rate;
+ flv->last_channels =
+ channels = st->codecpar->ch_layout.nb_channels;
+ } else {
+ AVCodecParameters *par = avcodec_parameters_alloc();
+ if (!par) {
+ ret = AVERROR(ENOMEM);
+ goto leave;
+ }
+ par->sample_rate = sample_rate;
+ par->bits_per_coded_sample = bits_per_coded_sample;
+ flv_set_audio_codec(s, st, par, flags & FLV_AUDIO_CODECID_MASK);
+ sample_rate = par->sample_rate;
+ avcodec_parameters_free(&par);
+ }
+ } else if (stream_type == FLV_STREAM_TYPE_AUDIO) {
+ if (!st->codecpar->codec_id)
+ flv_set_audio_codec(s, st, st->codecpar,
+ codec_id ? codec_id : (flags & FLV_AUDIO_CODECID_MASK));
+
+ // These are not signalled in the flags anymore
+ channels = 0;
+ sample_rate = 0;
+
+ if (pkt_type == AudioPacketTypeMultichannelConfig) {
+ int channel_order = avio_r8(s->pb);
+ channels = avio_r8(s->pb);
+ size -= 2;
+ track_size -= 2;
+
+ av_channel_layout_uninit(&st->codecpar->ch_layout);
+
+ if (channel_order == AudioChannelOrderCustom) {
+ ret = av_channel_layout_custom_init(&st->codecpar->ch_layout, channels);
+ if (ret < 0)
+ return ret;
+
+ for (i = 0; i < channels; i++) {
+ uint8_t id = avio_r8(s->pb);
+ size--;
+
+ if (id < 18)
+ st->codecpar->ch_layout.u.map[i].id = id;
+ else if (id >= 18 && id <= 23)
+ st->codecpar->ch_layout.u.map[i].id = id - 18 + AV_CHAN_LOW_FREQUENCY_2;
+ else if (id == 0xFE)
+ st->codecpar->ch_layout.u.map[i].id = AV_CHAN_UNUSED;
+ else
+ st->codecpar->ch_layout.u.map[i].id = AV_CHAN_UNKNOWN;
+ }
+ } else if (channel_order == AudioChannelOrderNative) {
+ uint64_t mask = avio_rb32(s->pb);
+ size -= 4;
+ track_size -= 4;
+
+ // The first 18 entries in the mask match ours, but the remaining 6 entries start at AV_CHAN_LOW_FREQUENCY_2
+ mask = (mask & 0x3FFFF) | ((mask & 0xFC0000) << (AV_CHAN_LOW_FREQUENCY_2 - 18));
+ ret = av_channel_layout_from_mask(&st->codecpar->ch_layout, mask);
+ if (ret < 0)
+ return ret;
+ } else {
+ av_channel_layout_default(&st->codecpar->ch_layout, channels);
+ }
+
+ av_log(s, AV_LOG_DEBUG, "Set channel data from MultiChannel info.\n");
- if (stream_type == FLV_STREAM_TYPE_AUDIO && !enhanced_flv) {
- int bits_per_coded_sample;
- channels = (flags & FLV_AUDIO_CHANNEL_MASK) == FLV_STEREO ? 2 : 1;
- sample_rate = 44100 << ((flags & FLV_AUDIO_SAMPLERATE_MASK) >>
- FLV_AUDIO_SAMPLERATE_OFFSET) >> 3;
- bits_per_coded_sample = (flags & FLV_AUDIO_SAMPLESIZE_MASK) ? 16 : 8;
- if (!av_channel_layout_check(&st->codecpar->ch_layout) ||
- !st->codecpar->sample_rate ||
- !st->codecpar->bits_per_coded_sample) {
- av_channel_layout_default(&st->codecpar->ch_layout, channels);
- st->codecpar->sample_rate = sample_rate;
- st->codecpar->bits_per_coded_sample = bits_per_coded_sample;
- }
- if (!st->codecpar->codec_id) {
- flv_set_audio_codec(s, st, st->codecpar,
- flags & FLV_AUDIO_CODECID_MASK);
- flv->last_sample_rate =
- sample_rate = st->codecpar->sample_rate;
- flv->last_channels =
- channels = st->codecpar->ch_layout.nb_channels;
- } else {
- AVCodecParameters *par = avcodec_parameters_alloc();
- if (!par) {
- ret = AVERROR(ENOMEM);
goto leave;
}
- par->sample_rate = sample_rate;
- par->bits_per_coded_sample = bits_per_coded_sample;
- flv_set_audio_codec(s, st, par, flags & FLV_AUDIO_CODECID_MASK);
- sample_rate = par->sample_rate;
- avcodec_parameters_free(&par);
+ } else if (stream_type == FLV_STREAM_TYPE_VIDEO) {
+ int sret = flv_set_video_codec(s, st, codec_id, 1);
+ if (sret < 0)
+ return sret;
+ size -= sret;
+ track_size -= sret;
+ } else if (stream_type == FLV_STREAM_TYPE_SUBTITLE) {
+ st->codecpar->codec_id = AV_CODEC_ID_TEXT;
+ } else if (stream_type == FLV_STREAM_TYPE_DATA) {
+ st->codecpar->codec_id = AV_CODEC_ID_NONE; // Opaque AMF data
}
- } else if (stream_type == FLV_STREAM_TYPE_AUDIO) {
- if (!st->codecpar->codec_id)
- flv_set_audio_codec(s, st, st->codecpar,
- codec_id ? codec_id : (flags & FLV_AUDIO_CODECID_MASK));
-
- // These are not signalled in the flags anymore
- channels = 0;
- sample_rate = 0;
- if (pkt_type == AudioPacketTypeMultichannelConfig) {
- int channel_order = avio_r8(s->pb);
- channels = avio_r8(s->pb);
- size -= 2;
+ if (st->codecpar->codec_id == AV_CODEC_ID_AAC ||
+ st->codecpar->codec_id == AV_CODEC_ID_OPUS ||
+ st->codecpar->codec_id == AV_CODEC_ID_FLAC ||
+ st->codecpar->codec_id == AV_CODEC_ID_H264 ||
+ st->codecpar->codec_id == AV_CODEC_ID_MPEG4 ||
+ st->codecpar->codec_id == AV_CODEC_ID_HEVC ||
+ st->codecpar->codec_id == AV_CODEC_ID_AV1 ||
+ st->codecpar->codec_id == AV_CODEC_ID_VP9) {
+ int type = 0;
+ if (enhanced_flv) {
+ type = pkt_type;
+ } else {
+ type = avio_r8(s->pb);
+ size--;
+ track_size--;
+ }
- av_channel_layout_uninit(&st->codecpar->ch_layout);
+ if (size < 0 || track_size < 0) {
+ ret = AVERROR_INVALIDDATA;
+ goto leave;
+ }
- if (channel_order == AudioChannelOrderCustom) {
- ret = av_channel_layout_custom_init(&st->codecpar->ch_layout, channels);
- if (ret < 0)
- return ret;
+ if (enhanced_flv && stream_type == FLV_STREAM_TYPE_VIDEO && flv->meta_color_info_flag) {
+ flv_update_video_color_info(s, st); // update av packet side data
+ flv->meta_color_info_flag = 0;
+ }
- for (i = 0; i < channels; i++) {
- uint8_t id = avio_r8(s->pb);
- size--;
-
- if (id < 18)
- st->codecpar->ch_layout.u.map[i].id = id;
- else if (id >= 18 && id <= 23)
- st->codecpar->ch_layout.u.map[i].id = id - 18 + AV_CHAN_LOW_FREQUENCY_2;
- else if (id == 0xFE)
- st->codecpar->ch_layout.u.map[i].id = AV_CHAN_UNUSED;
- else
- st->codecpar->ch_layout.u.map[i].id = AV_CHAN_UNKNOWN;
+ if (st->codecpar->codec_id == AV_CODEC_ID_MPEG4 ||
+ (st->codecpar->codec_id == AV_CODEC_ID_H264 && (!enhanced_flv || type == PacketTypeCodedFrames)) ||
+ (st->codecpar->codec_id == AV_CODEC_ID_HEVC && type == PacketTypeCodedFrames)) {
+ // sign extension
+ int32_t cts = (avio_rb24(s->pb) + 0xff800000) ^ 0xff800000;
+ pts = av_sat_add64(dts, cts);
+ if (cts < 0) { // dts might be wrong
+ if (!flv->wrong_dts)
+ av_log(s, AV_LOG_WARNING,
+ "Negative cts, previous timestamps might be wrong.\n");
+ flv->wrong_dts = 1;
+ } else if (FFABS(dts - pts) > 1000*60*15) {
+ av_log(s, AV_LOG_WARNING,
+ "invalid timestamps %"PRId64" %"PRId64"\n", dts, pts);
+ dts = pts = AV_NOPTS_VALUE;
}
- } else if (channel_order == AudioChannelOrderNative) {
- uint64_t mask = avio_rb32(s->pb);
- size -= 4;
-
- // The first 18 entries in the mask match ours, but the remaining 6 entries start at AV_CHAN_LOW_FREQUENCY_2
- mask = (mask & 0x3FFFF) | ((mask & 0xFC0000) << (AV_CHAN_LOW_FREQUENCY_2 - 18));
- ret = av_channel_layout_from_mask(&st->codecpar->ch_layout, mask);
- if (ret < 0)
- return ret;
- } else {
- av_channel_layout_default(&st->codecpar->ch_layout, channels);
+ size -= 3;
+ track_size -= 3;
}
+ if (type == 0 && (!st->codecpar->extradata || st->codecpar->codec_id == AV_CODEC_ID_AAC ||
+ st->codecpar->codec_id == AV_CODEC_ID_OPUS || st->codecpar->codec_id == AV_CODEC_ID_FLAC ||
+ st->codecpar->codec_id == AV_CODEC_ID_H264 || st->codecpar->codec_id == AV_CODEC_ID_HEVC ||
+ st->codecpar->codec_id == AV_CODEC_ID_AV1 || st->codecpar->codec_id == AV_CODEC_ID_VP9)) {
+ AVDictionaryEntry *t;
+
+ if (st->codecpar->extradata) {
+ if ((ret = flv_queue_extradata(flv, s->pb, multitrack ? track_idx : stream_type, track_size, multitrack)) < 0)
+ return ret;
+ ret = FFERROR_REDO;
+ goto leave;
+ }
+ if ((ret = flv_get_extradata(s, st, track_size)) < 0)
+ return ret;
- av_log(s, AV_LOG_DEBUG, "Set channel data from MultiChannel info.\n");
-
- goto leave;
- }
- } else if (stream_type == FLV_STREAM_TYPE_VIDEO) {
- int sret = flv_set_video_codec(s, st, codec_id, 1);
- if (sret < 0)
- return sret;
- size -= sret;
- } else if (stream_type == FLV_STREAM_TYPE_SUBTITLE) {
- st->codecpar->codec_id = AV_CODEC_ID_TEXT;
- } else if (stream_type == FLV_STREAM_TYPE_DATA) {
- st->codecpar->codec_id = AV_CODEC_ID_NONE; // Opaque AMF data
- }
+ /* Workaround for buggy Omnia A/XE encoder */
+ t = av_dict_get(s->metadata, "Encoder", NULL, 0);
+ if (st->codecpar->codec_id == AV_CODEC_ID_AAC && t && !strcmp(t->value, "Omnia A/XE"))
+ st->codecpar->extradata_size = 2;
- if (st->codecpar->codec_id == AV_CODEC_ID_AAC ||
- st->codecpar->codec_id == AV_CODEC_ID_OPUS ||
- st->codecpar->codec_id == AV_CODEC_ID_FLAC ||
- st->codecpar->codec_id == AV_CODEC_ID_H264 ||
- st->codecpar->codec_id == AV_CODEC_ID_MPEG4 ||
- st->codecpar->codec_id == AV_CODEC_ID_HEVC ||
- st->codecpar->codec_id == AV_CODEC_ID_AV1 ||
- st->codecpar->codec_id == AV_CODEC_ID_VP9) {
- int type = 0;
- if (enhanced_flv) {
- type = pkt_type;
- } else {
- type = avio_r8(s->pb);
- size--;
+ ret = FFERROR_REDO;
+ goto leave;
+ }
}
- if (size < 0) {
- ret = AVERROR_INVALIDDATA;
+ /* skip empty or broken data packets */
+ if (size <= 0 || track_size < 0) {
+ ret = FFERROR_REDO;
goto leave;
}
- if (enhanced_flv && stream_type == FLV_STREAM_TYPE_VIDEO && flv->meta_color_info_flag) {
- flv_update_video_color_info(s, st); // update av packet side data
- flv->meta_color_info_flag = 0;
+ /* skip empty data track */
+ if (!track_size)
+ goto next_track;
+
+ ret = av_get_packet(s->pb, pkt, track_size);
+ if (ret < 0)
+ return ret;
+
+ track_size -= ret;
+ size -= ret;
+
+ pkt->dts = dts;
+ pkt->pts = pts == AV_NOPTS_VALUE ? dts : pts;
+ pkt->stream_index = st->index;
+ pkt->pos = pos;
+ if (!multitrack && flv->new_extradata[stream_type]) {
+ ret = av_packet_add_side_data(pkt, AV_PKT_DATA_NEW_EXTRADATA,
+ flv->new_extradata[stream_type],
+ flv->new_extradata_size[stream_type]);
+ if (ret >= 0) {
+ flv->new_extradata[stream_type] = NULL;
+ flv->new_extradata_size[stream_type] = 0;
+ }
+ } else if (multitrack
+ && flv->mt_extradata_cnt > track_idx
+ && flv->mt_extradata[track_idx]) {
+ ret = av_packet_add_side_data(pkt, AV_PKT_DATA_NEW_EXTRADATA,
+ flv->mt_extradata[track_idx],
+ flv->mt_extradata_sz[track_idx]);
+ if (ret >= 0) {
+ flv->mt_extradata[track_idx] = NULL;
+ flv->mt_extradata_sz[track_idx] = 0;
+ }
+ }
+ if (stream_type == FLV_STREAM_TYPE_AUDIO && !enhanced_flv &&
+ (sample_rate != flv->last_sample_rate ||
+ channels != flv->last_channels)) {
+ flv->last_sample_rate = sample_rate;
+ flv->last_channels = channels;
+ ff_add_param_change(pkt, channels, 0, sample_rate, 0, 0);
}
- if (st->codecpar->codec_id == AV_CODEC_ID_MPEG4 ||
- (st->codecpar->codec_id == AV_CODEC_ID_H264 && (!enhanced_flv || type == PacketTypeCodedFrames)) ||
- (st->codecpar->codec_id == AV_CODEC_ID_HEVC && type == PacketTypeCodedFrames)) {
- // sign extension
- int32_t cts = (avio_rb24(s->pb) + 0xff800000) ^ 0xff800000;
- pts = av_sat_add64(dts, cts);
- if (cts < 0) { // dts might be wrong
- if (!flv->wrong_dts)
- av_log(s, AV_LOG_WARNING,
- "Negative cts, previous timestamps might be wrong.\n");
- flv->wrong_dts = 1;
- } else if (FFABS(dts - pts) > 1000*60*15) {
- av_log(s, AV_LOG_WARNING,
- "invalid timestamps %"PRId64" %"PRId64"\n", dts, pts);
- dts = pts = AV_NOPTS_VALUE;
- }
- size -= 3;
+ if (stream_type == FLV_STREAM_TYPE_AUDIO ||
+ (flags & FLV_VIDEO_FRAMETYPE_MASK) == FLV_FRAME_KEY ||
+ stream_type == FLV_STREAM_TYPE_SUBTITLE ||
+ stream_type == FLV_STREAM_TYPE_DATA)
+ pkt->flags |= AV_PKT_FLAG_KEY;
+
+ ret = ff_buffer_packet(s, pkt);
+ if (ret < 0)
+ return ret;
+ res = FFERROR_REDO;
+
+ if (track_size) {
+ av_log(s, AV_LOG_WARNING, "Track size mismatch: %d!\n", track_size);
+ avio_skip(s->pb, track_size);
+ size -= track_size;
}
- if (type == 0 && (!st->codecpar->extradata || st->codecpar->codec_id == AV_CODEC_ID_AAC ||
- st->codecpar->codec_id == AV_CODEC_ID_OPUS || st->codecpar->codec_id == AV_CODEC_ID_FLAC ||
- st->codecpar->codec_id == AV_CODEC_ID_H264 || st->codecpar->codec_id == AV_CODEC_ID_HEVC ||
- st->codecpar->codec_id == AV_CODEC_ID_AV1 || st->codecpar->codec_id == AV_CODEC_ID_VP9)) {
- AVDictionaryEntry *t;
-
- if (st->codecpar->extradata) {
- if ((ret = flv_queue_extradata(flv, s->pb, multitrack ? track_idx : stream_type, size, multitrack)) < 0)
- return ret;
- ret = FFERROR_REDO;
- goto leave;
- }
- if ((ret = flv_get_extradata(s, st, size)) < 0)
- return ret;
- /* Workaround for buggy Omnia A/XE encoder */
- t = av_dict_get(s->metadata, "Encoder", NULL, 0);
- if (st->codecpar->codec_id == AV_CODEC_ID_AAC && t && !strcmp(t->value, "Omnia A/XE"))
- st->codecpar->extradata_size = 2;
+ if (!size)
+ break;
- ret = FFERROR_REDO;
- goto leave;
+next_track:
+ if (multitrack_type == MultitrackTypeOneTrack) {
+ av_log(s, AV_LOG_ERROR, "Attempted to read next track in single-track mode.\n");
+ return AVERROR_BUG;
}
- }
-
- /* skip empty data packets */
- if (!size) {
- ret = FFERROR_REDO;
- goto leave;
- }
- ret = av_get_packet(s->pb, pkt, size);
- if (ret < 0)
- return ret;
- pkt->dts = dts;
- pkt->pts = pts == AV_NOPTS_VALUE ? dts : pts;
- pkt->stream_index = st->index;
- pkt->pos = pos;
- if (!multitrack && flv->new_extradata[stream_type]) {
- int sret = av_packet_add_side_data(pkt, AV_PKT_DATA_NEW_EXTRADATA,
- flv->new_extradata[stream_type],
- flv->new_extradata_size[stream_type]);
- if (sret >= 0) {
- flv->new_extradata[stream_type] = NULL;
- flv->new_extradata_size[stream_type] = 0;
+ if (multitrack_type == MultitrackTypeManyTracksManyCodecs) {
+ codec_id = avio_rb32(s->pb);
+ size -= 4;
}
- } else if (multitrack
- && flv->mt_extradata_cnt > track_idx
- && flv->mt_extradata[track_idx]) {
- int sret = av_packet_add_side_data(pkt, AV_PKT_DATA_NEW_EXTRADATA,
- flv->mt_extradata[track_idx],
- flv->mt_extradata_sz[track_idx]);
- if (sret >= 0) {
- flv->mt_extradata[track_idx] = NULL;
- flv->mt_extradata_sz[track_idx] = 0;
+
+ track_idx = avio_r8(s->pb);
+ size--;
+
+ if (avio_feof(s->pb)) {
+ av_log(s, AV_LOG_WARNING, "premature EOF\n");
+ /* return REDO so that any potentially queued up packages can be drained first */
+ return FFERROR_REDO;
}
}
- if (stream_type == FLV_STREAM_TYPE_AUDIO && !enhanced_flv &&
- (sample_rate != flv->last_sample_rate ||
- channels != flv->last_channels)) {
- flv->last_sample_rate = sample_rate;
- flv->last_channels = channels;
- ff_add_param_change(pkt, channels, 0, sample_rate, 0, 0);
- }
-
- if (stream_type == FLV_STREAM_TYPE_AUDIO ||
- (flags & FLV_VIDEO_FRAMETYPE_MASK) == FLV_FRAME_KEY ||
- stream_type == FLV_STREAM_TYPE_SUBTITLE ||
- stream_type == FLV_STREAM_TYPE_DATA)
- pkt->flags |= AV_PKT_FLAG_KEY;
leave:
last = avio_rb32(s->pb);
@@ -1755,7 +1803,7 @@ leave:
if (ret >= 0)
flv->last_ts = pkt->dts;
- return ret;
+ return ret ? ret : res;
}
static int flv_read_seek(AVFormatContext *s, int stream_index,
--
2.43.2
_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".
^ permalink raw reply [flat|nested] 25+ messages in thread
* Re: [FFmpeg-devel] [RFC 00/13] flvdec/flvenc: add support for enhanced rtmp codecs and multitrack/multichannel
2024-05-21 9:02 [FFmpeg-devel] [RFC 00/13] flvdec/flvenc: add support for enhanced rtmp codecs and multitrack/multichannel Timo Rothenpieler
` (12 preceding siblings ...)
2024-05-21 9:02 ` [FFmpeg-devel] [PATCH 13/13] avformat/flvdec: support all multi-track modes Timo Rothenpieler
@ 2024-05-23 1:57 ` Steven Liu
2024-05-28 15:14 ` Andrew Sayers
14 siblings, 0 replies; 25+ messages in thread
From: Steven Liu @ 2024-05-23 1:57 UTC (permalink / raw)
To: FFmpeg development discussions and patches; +Cc: Timo Rothenpieler
Timo Rothenpieler <timo@rothenpieler.org> 于2024年5月21日周二 17:03写道：
>
> This is based on the preliminary spec for enhanced rtmp v2:
> https://veovera.org/docs/enhanced/enhanced-rtmp-v2
>
> The spec is not final, and can still undergo breaking changes, hence this set is purely for comments and review, and not ready to be merged until the final v2 spec is published.
>
> There are no samples out in the wild yet, so testing interoperability with other software has not happened yet either.
> Specially the two other multitrack modes, where multiple tracks are in the same packet, have not been tested at all, since no software can write such files.
Yes, there have reconnect function in v2
And i cannot sure there maybe have some problem in multitrack mode,
different codecs type, change amount the tracks, maybe need more
process, need check GOP、 keyframes 、VFR mode, because they will get
different timestamps.
>
> The set can also be found on GitHub, where ignoring whitespaces makes specially the last patch a lot more readable:
> https://github.com/BtbN/FFmpeg/tree/enhanced-flv
>
>
> Dennis Sädtler via ffmpeg-devel (2):
> avformat/flvenc: Implement support for multi-track video
> avformat/flvdec: Add support for demuxing multi-track FLV
>
> Timo Rothenpieler (11):
> avformat/flvenc: add enhanced audio codecs
> avformat/flvenc: remove !size check for audio packets
> avformat/flvdec: add enhanced audio codecs
> avformat/flvenc: refactor fourcc writing
> avformat/flvenc: write enhanced rtmp multichannel info for audio with
> more than two channels
> avformat/flvdec: parse enhanced rtmp multichannel info
> avformat/flvenc: add support for writing multi track audio
> avformat/flvdec: add support for reading multi track audio
> avformat/rtmpproto: add more enhanced rtmp codecs
> avformat/flvdec: stop shadowing local variables
> avformat/flvdec: support all multi-track modes
>
> libavformat/flv.h | 21 ++
> libavformat/flvdec.c | 654 +++++++++++++++++++++++++++-------------
> libavformat/flvenc.c | 443 +++++++++++++++++++++------
> libavformat/rtmpproto.c | 11 +-
> 4 files changed, 819 insertions(+), 310 deletions(-)
>
> --
> 2.43.2
>
> _______________________________________________
> ffmpeg-devel mailing list
> ffmpeg-devel@ffmpeg.org
> https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
>
> To unsubscribe, visit link above, or email
> ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".
Thanks
Steven
_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".
^ permalink raw reply [flat|nested] 25+ messages in thread
* Re: [FFmpeg-devel] [RFC 00/13] flvdec/flvenc: add support for enhanced rtmp codecs and multitrack/multichannel
2024-05-21 9:02 [FFmpeg-devel] [RFC 00/13] flvdec/flvenc: add support for enhanced rtmp codecs and multitrack/multichannel Timo Rothenpieler
` (13 preceding siblings ...)
2024-05-23 1:57 ` [FFmpeg-devel] [RFC 00/13] flvdec/flvenc: add support for enhanced rtmp codecs and multitrack/multichannel Steven Liu
@ 2024-05-28 15:14 ` Andrew Sayers
2024-05-28 17:11 ` Timo Rothenpieler
14 siblings, 1 reply; 25+ messages in thread
From: Andrew Sayers @ 2024-05-28 15:14 UTC (permalink / raw)
To: FFmpeg development discussions and patches
On Tue, May 21, 2024 at 11:02:09AM +0200, Timo Rothenpieler wrote:
> This is based on the preliminary spec for enhanced rtmp v2:
> https://veovera.org/docs/enhanced/enhanced-rtmp-v2
>
> The spec is not final, and can still undergo breaking changes, hence this set is purely for comments and review, and not ready to be merged until the final v2 spec is published.
>
> There are no samples out in the wild yet, so testing interoperability with other software has not happened yet either.
> Specially the two other multitrack modes, where multiple tracks are in the same packet, have not been tested at all, since no software can write such files.
>
> The set can also be found on GitHub, where ignoring whitespaces makes specially the last patch a lot more readable:
> https://github.com/BtbN/FFmpeg/tree/enhanced-flv
>
I ran this against a little review bot I'm working on.
Please do s/\* / \*/g on the following:
avformat/flvenc: add support for writing multi track audio
> +static void flv_write_multichannel_header(AVFormatContext* s, AVCodecParameters* par, int64_t ts, int stream_index)
avformat/flvenc: write enhanced rtmp multichannel info for audio with more than two channels
> +static void flv_write_multichannel_body(AVFormatContext* s, AVCodecParameters* par)
> +static int flv_get_multichannel_body_size(AVCodecParameters* par)
> +static void flv_write_multichannel_header(AVFormatContext* s, AVCodecParameters* par, int64_t ts)
avformat/flvenc: add enhanced audio codecs
> +static void flv_write_aac_header(AVFormatContext* s, AVCodecParameters* par)
avformat/flvenc: Implement support for multi-track video
> +static void flv_write_codec_header(AVFormatContext* s, AVCodecParameters* par, int64_t ts, int stream_index) {
_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".
^ permalink raw reply [flat|nested] 25+ messages in thread
* Re: [FFmpeg-devel] [RFC 00/13] flvdec/flvenc: add support for enhanced rtmp codecs and multitrack/multichannel
2024-05-28 15:14 ` Andrew Sayers
@ 2024-05-28 17:11 ` Timo Rothenpieler
0 siblings, 0 replies; 25+ messages in thread
From: Timo Rothenpieler @ 2024-05-28 17:11 UTC (permalink / raw)
To: ffmpeg-devel
On 28.05.2024 17:14, Andrew Sayers wrote:
> On Tue, May 21, 2024 at 11:02:09AM +0200, Timo Rothenpieler wrote:
>> This is based on the preliminary spec for enhanced rtmp v2:
>> https://veovera.org/docs/enhanced/enhanced-rtmp-v2
>>
>> The spec is not final, and can still undergo breaking changes, hence this set is purely for comments and review, and not ready to be merged until the final v2 spec is published.
>>
>> There are no samples out in the wild yet, so testing interoperability with other software has not happened yet either.
>> Specially the two other multitrack modes, where multiple tracks are in the same packet, have not been tested at all, since no software can write such files.
>>
>> The set can also be found on GitHub, where ignoring whitespaces makes specially the last patch a lot more readable:
>> https://github.com/BtbN/FFmpeg/tree/enhanced-flv
>>
>
> I ran this against a little review bot I'm working on.
> Please do s/\* / \*/g on the following:
The usage of that is already inconsistent throughout the flv codebase,
so running that would make the patch touch a lot of unrelated things.
Seems more sane to just do a general style-cleanup after the fact.
_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".
^ permalink raw reply [flat|nested] 25+ messages in thread
end of thread, other threads:[~2024-05-28 17:11 UTC | newest]
Thread overview: 25+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2024-05-21 9:02 [FFmpeg-devel] [RFC 00/13] flvdec/flvenc: add support for enhanced rtmp codecs and multitrack/multichannel Timo Rothenpieler
2024-05-21 9:02 ` [FFmpeg-devel] [PATCH 01/13] avformat/flvenc: Implement support for multi-track video Timo Rothenpieler
2024-05-21 18:48 ` Dennis Sädtler via ffmpeg-devel
2024-05-21 18:50 ` Timo Rothenpieler
[not found] ` <98D1BB90-39C6-413B-A158-8ADA201155BC@cosmin.at>
2024-05-21 18:54 ` Cosmin Stejerean via ffmpeg-devel
[not found] ` <7F89B9B2-FBC2-49E7-AC4F-68E41B584E1A@cosmin.at>
2024-05-21 18:56 ` Cosmin Stejerean via ffmpeg-devel
2024-05-21 18:56 ` Cosmin Stejerean via ffmpeg-devel
2024-05-21 9:02 ` [FFmpeg-devel] [PATCH 02/13] avformat/flvdec: Add support for demuxing multi-track FLV Timo Rothenpieler
2024-05-21 9:02 ` [FFmpeg-devel] [PATCH 03/13] avformat/flvenc: add enhanced audio codecs Timo Rothenpieler
2024-05-21 9:02 ` [FFmpeg-devel] [PATCH 04/13] avformat/flvenc: remove !size check for audio packets Timo Rothenpieler
2024-05-21 9:02 ` [FFmpeg-devel] [PATCH 05/13] avformat/flvdec: add enhanced audio codecs Timo Rothenpieler
2024-05-21 9:02 ` [FFmpeg-devel] [PATCH 06/13] avformat/flvenc: refactor fourcc writing Timo Rothenpieler
2024-05-21 9:02 ` [FFmpeg-devel] [PATCH 07/13] avformat/flvenc: write enhanced rtmp multichannel info for audio with more than two channels Timo Rothenpieler
2024-05-21 9:02 ` [FFmpeg-devel] [PATCH 08/13] avformat/flvdec: parse enhanced rtmp multichannel info Timo Rothenpieler
2024-05-21 9:02 ` [FFmpeg-devel] [PATCH 09/13] avformat/flvenc: add support for writing multi track audio Timo Rothenpieler
2024-05-21 9:02 ` [FFmpeg-devel] [PATCH 10/13] avformat/flvdec: add support for reading " Timo Rothenpieler
2024-05-21 9:02 ` [FFmpeg-devel] [PATCH 11/13] avformat/rtmpproto: add more enhanced rtmp codecs Timo Rothenpieler
2024-05-21 9:02 ` [FFmpeg-devel] [PATCH 12/13] avformat/flvdec: stop shadowing local variables Timo Rothenpieler
2024-05-21 9:02 ` [FFmpeg-devel] [PATCH 13/13] avformat/flvdec: support all multi-track modes Timo Rothenpieler
2024-05-22 0:02 ` Michael Niedermayer
2024-05-22 18:26 ` Timo Rothenpieler
2024-05-22 18:27 ` [FFmpeg-devel] [PATCH v2 " Timo Rothenpieler
2024-05-23 1:57 ` [FFmpeg-devel] [RFC 00/13] flvdec/flvenc: add support for enhanced rtmp codecs and multitrack/multichannel Steven Liu
2024-05-28 15:14 ` Andrew Sayers
2024-05-28 17:11 ` Timo Rothenpieler
Git Inbox Mirror of the ffmpeg-devel mailing list - see https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
This inbox may be cloned and mirrored by anyone:
git clone --mirror https://master.gitmailbox.com/ffmpegdev/0 ffmpegdev/git/0.git
# If you have public-inbox 1.1+ installed, you may
# initialize and index your mirror using the following commands:
public-inbox-init -V2 ffmpegdev ffmpegdev/ https://master.gitmailbox.com/ffmpegdev \
ffmpegdev@gitmailbox.com
public-inbox-index ffmpegdev
Example config snippet for mirrors.
AGPL code for this site: git clone https://public-inbox.org/public-inbox.git