[FFmpeg-devel] [RFC]x264 avcc and related issue

[FFmpeg-devel] [RFC]x264 avcc and related issue

[Zhao Zhili]

There are multiple encoders which can output bitstream in avcc/hvcc format,
e.g, x264, x265 and videotoolbox. This can improve performance depends on
container format (no more ff_avc_parse_nal_units_buf).

The problem is these encoders output parameter sets directly, not contained
in a complete AVC/HEVC configuration box. But muxers only expect 
extradata in
annexb format, or a complete AVC/HEVC configuration box, not support such
length+sps+length+pps format. The following command will output a broken 
file:

./ffmpeg -i input.mp4 -an -c:v libx264 -x264-params annexb=0 broken.mp4

I want to fix this issue and improve the performance at the same time.

There are multiple options that I don't know how to make a choice.

1. At the muxer level? It's easy and ugly. And there are corner case
like 00 00 01 xx can be start code of annexb and size of avcc at the
same time.

2. A new bsf? It looks less useful outside of libavcodec. And encoder
can't enable a bsf automatically.

3. A specific helper like h26x_vcc.h? Can it be more general?

Any more suggestions?

_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] [RFC]x264 avcc and related issue

[Andreas Rheinhardt]

Zhao Zhili:
> There are multiple encoders which can output bitstream in avcc/hvcc format,
> e.g, x264, x265 and videotoolbox. This can improve performance depends on
> container format (no more ff_avc_parse_nal_units_buf).
> 
> The problem is these encoders output parameter sets directly, not contained
> in a complete AVC/HEVC configuration box. But muxers only expect
> extradata in
> annexb format, or a complete AVC/HEVC configuration box, not support such
> length+sps+length+pps format. The following command will output a broken
> file:
> 
> ./ffmpeg -i input.mp4 -an -c:v libx264 -x264-params annexb=0 broken.mp4
> 
> I want to fix this issue and improve the performance at the same time.
> 
> There are multiple options that I don't know how to make a choice.
> 
> 1. At the muxer level? It's easy and ugly. And there are corner case
> like 00 00 01 xx can be start code of annexb and size of avcc at the
> same time.
> 
> 2. A new bsf? It looks less useful outside of libavcodec. And encoder
> can't enable a bsf automatically.
> 
> 3. A specific helper like h26x_vcc.h? Can it be more general?
> 
> Any more suggestions?
> 

If we support a new packetization (even if it only affects extradata),
we would need to support it everywhere -- decoders and muxers, too. I
don't see an advantage or a need for a new packetization, as AFAIK all
containers with ISOBMFF packetization also use the ISOMBFF configuration
box. So I am against 1. and 2.
Implementing this inside lavc seems like the best approach; I don't know
whether a common helper for this will be useful: While these lists
certainly have a common pattern, there are subtle changes between them
which will probably make a common helper unsuitable.

- Andreas

_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] [RFC]x264 avcc and related issue

[Zhao Zhili]

> 在 2024年3月6日，上午1:58，Andreas Rheinhardt <andreas.rheinhardt@outlook.com> 写道：
> 
> Zhao Zhili:
>> There are multiple encoders which can output bitstream in avcc/hvcc format,
>> e.g, x264, x265 and videotoolbox. This can improve performance depends on
>> container format (no more ff_avc_parse_nal_units_buf).
>> 
>> The problem is these encoders output parameter sets directly, not contained
>> in a complete AVC/HEVC configuration box. But muxers only expect
>> extradata in
>> annexb format, or a complete AVC/HEVC configuration box, not support such
>> length+sps+length+pps format. The following command will output a broken
>> file:
>> 
>> ./ffmpeg -i input.mp4 -an -c:v libx264 -x264-params annexb=0 broken.mp4
>> 
>> I want to fix this issue and improve the performance at the same time.
>> 
>> There are multiple options that I don't know how to make a choice.
>> 
>> 1. At the muxer level? It's easy and ugly. And there are corner case
>> like 00 00 01 xx can be start code of annexb and size of avcc at the
>> same time.
>> 
>> 2. A new bsf? It looks less useful outside of libavcodec. And encoder
>> can't enable a bsf automatically.
>> 
>> 3. A specific helper like h26x_vcc.h? Can it be more general?
>> 
>> Any more suggestions?
>> 
> 
> If we support a new packetization (even if it only affects extradata),
> we would need to support it everywhere -- decoders and muxers, too. I
> don't see an advantage or a need for a new packetization, as AFAIK all
> containers with ISOBMFF packetization also use the ISOMBFF configuration
> box. So I am against 1. and 2.
> Implementing this inside lavc seems like the best approach; I don't know
> whether a common helper for this will be useful: While these lists
> certainly have a common pattern, there are subtle changes between them
> which will probably make a common helper unsuitable.

You mean separated implementations just inside libx264/libx265?

> 
> - Andreas
> 
> _______________________________________________
> ffmpeg-devel mailing list
> ffmpeg-devel@ffmpeg.org
> https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
> 
> To unsubscribe, visit link above, or email

_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] [RFC]x264 avcc and related issue

[Andreas Rheinhardt]

Zhao Zhili:
> 
>> 在 2024年3月6日，上午1:58，Andreas Rheinhardt <andreas.rheinhardt@outlook.com> 写道：
>>
>> Zhao Zhili:
>>> There are multiple encoders which can output bitstream in avcc/hvcc format,
>>> e.g, x264, x265 and videotoolbox. This can improve performance depends on
>>> container format (no more ff_avc_parse_nal_units_buf).
>>>
>>> The problem is these encoders output parameter sets directly, not contained
>>> in a complete AVC/HEVC configuration box. But muxers only expect
>>> extradata in
>>> annexb format, or a complete AVC/HEVC configuration box, not support such
>>> length+sps+length+pps format. The following command will output a broken
>>> file:
>>>
>>> ./ffmpeg -i input.mp4 -an -c:v libx264 -x264-params annexb=0 broken.mp4
>>>
>>> I want to fix this issue and improve the performance at the same time.
>>>
>>> There are multiple options that I don't know how to make a choice.
>>>
>>> 1. At the muxer level? It's easy and ugly. And there are corner case
>>> like 00 00 01 xx can be start code of annexb and size of avcc at the
>>> same time.
>>>
>>> 2. A new bsf? It looks less useful outside of libavcodec. And encoder
>>> can't enable a bsf automatically.
>>>
>>> 3. A specific helper like h26x_vcc.h? Can it be more general?
>>>
>>> Any more suggestions?
>>>
>>
>> If we support a new packetization (even if it only affects extradata),
>> we would need to support it everywhere -- decoders and muxers, too. I
>> don't see an advantage or a need for a new packetization, as AFAIK all
>> containers with ISOBMFF packetization also use the ISOMBFF configuration
>> box. So I am against 1. and 2.
>> Implementing this inside lavc seems like the best approach; I don't know
>> whether a common helper for this will be useful: While these lists
>> certainly have a common pattern, there are subtle changes between them
>> which will probably make a common helper unsuitable.
> 
> You mean separated implementations just inside libx264/libx265?
> 

Yes.

- Andreas

_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".