[FFmpeg-devel] [PATCH] avcodec/h264_mp4toannexb: fix missing SPS/PPS to IDR frames

[FFmpeg-devel] [PATCH] avcodec/h264_mp4toannexb: fix missing SPS/PPS to IDR frames

[Zhao Zhili]

From: Zhao Zhili <zhilizhao@tencent.com>

If there is a single group of SPS/PPS before an IDR frame, but no
SPS/PPS after that, we will miss the chance to reset
idr_sps_seen/idr_pps_seen. The result is missing SPS/PPS for all
IDR frames except the first one.

Signed-off-by: Zhao Zhili <zhilizhao@tencent.com>
---
 libavcodec/h264_mp4toannexb_bsf.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/libavcodec/h264_mp4toannexb_bsf.c b/libavcodec/h264_mp4toannexb_bsf.c
index d11be455c2..bbdeca9c44 100644
--- a/libavcodec/h264_mp4toannexb_bsf.c
+++ b/libavcodec/h264_mp4toannexb_bsf.c
@@ -259,7 +259,7 @@ static int h264_mp4toannexb_filter(AVBSFContext *ctx, AVPacket *opkt)
 
             count_or_copy(&out, &out_size, buf, nal_size,
                           unit_type == H264_NAL_SPS || unit_type == H264_NAL_PPS, j);
-            if (!new_idr && unit_type == H264_NAL_SLICE) {
+            if (unit_type == H264_NAL_SLICE) {
                 new_idr  = 1;
                 sps_seen = 0;
                 pps_seen = 0;
-- 
2.25.1

_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] [PATCH] avcodec/h264_mp4toannexb: fix missing SPS/PPS to IDR frames

[Zhao Zhili]

On Thu, 2022-12-29 at 01:20 +0800, Zhao Zhili wrote:
> From: Zhao Zhili <zhilizhao@tencent.com>
> 
> If there is a single group of SPS/PPS before an IDR frame, but no
> SPS/PPS after that, we will miss the chance to reset
> idr_sps_seen/idr_pps_seen. The result is missing SPS/PPS for all
> IDR frames except the first one.
> 
> Signed-off-by: Zhao Zhili <zhilizhao@tencent.com>
> ---
>  libavcodec/h264_mp4toannexb_bsf.c | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
> 
> diff --git a/libavcodec/h264_mp4toannexb_bsf.c
> b/libavcodec/h264_mp4toannexb_bsf.c
> index d11be455c2..bbdeca9c44 100644
> --- a/libavcodec/h264_mp4toannexb_bsf.c
> +++ b/libavcodec/h264_mp4toannexb_bsf.c
> @@ -259,7 +259,7 @@ static int h264_mp4toannexb_filter(AVBSFContext
> *ctx, AVPacket *opkt)
>  
>              count_or_copy(&out, &out_size, buf, nal_size,
>                            unit_type == H264_NAL_SPS || unit_type ==
> H264_NAL_PPS, j);
> -            if (!new_idr && unit_type == H264_NAL_SLICE) {
> +            if (unit_type == H264_NAL_SLICE) {
>                  new_idr  = 1;
>                  sps_seen = 0;
>                  pps_seen = 0;

Ping for review.

_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] [PATCH] avcodec/h264_mp4toannexb: fix missing SPS/PPS to IDR frames

[Andreas Rheinhardt]

Zhao Zhili:
> From: Zhao Zhili <zhilizhao@tencent.com>
> 
> If there is a single group of SPS/PPS before an IDR frame, but no
> SPS/PPS after that, we will miss the chance to reset
> idr_sps_seen/idr_pps_seen. The result is missing SPS/PPS for all
> IDR frames except the first one.
> 
> Signed-off-by: Zhao Zhili <zhilizhao@tencent.com>
> ---
>  libavcodec/h264_mp4toannexb_bsf.c | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
> 
> diff --git a/libavcodec/h264_mp4toannexb_bsf.c b/libavcodec/h264_mp4toannexb_bsf.c
> index d11be455c2..bbdeca9c44 100644
> --- a/libavcodec/h264_mp4toannexb_bsf.c
> +++ b/libavcodec/h264_mp4toannexb_bsf.c
> @@ -259,7 +259,7 @@ static int h264_mp4toannexb_filter(AVBSFContext *ctx, AVPacket *opkt)
>  
>              count_or_copy(&out, &out_size, buf, nal_size,
>                            unit_type == H264_NAL_SPS || unit_type == H264_NAL_PPS, j);
> -            if (!new_idr && unit_type == H264_NAL_SLICE) {
> +            if (unit_type == H264_NAL_SLICE) {
>                  new_idr  = 1;
>                  sps_seen = 0;
>                  pps_seen = 0;

If I understand you correctly, you want that to include the parameter
sets from the extradata to every IDR frame without extradata if said IDR
frame follows a non-IDR frame after in-band parameter sets. There is
just one problem with this: Said in-band parameter sets might overwrite
parameter sets from extradata and inserting the outdated parameter sets
from extradata lateron might make the stream undecodable.

- Andreas

_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] [PATCH] avcodec/h264_mp4toannexb: fix missing SPS/PPS to IDR frames

[Zhao Zhili]

On Tue, 2023-01-10 at 07:41 +0100, Andreas Rheinhardt wrote:
> Zhao Zhili:
> > From: Zhao Zhili <zhilizhao@tencent.com>
> > 
> > If there is a single group of SPS/PPS before an IDR frame, but no
> > SPS/PPS after that, we will miss the chance to reset
> > idr_sps_seen/idr_pps_seen. The result is missing SPS/PPS for all
> > IDR frames except the first one.
> > 
> > Signed-off-by: Zhao Zhili <zhilizhao@tencent.com>
> > ---
> >  libavcodec/h264_mp4toannexb_bsf.c | 2 +-
> >  1 file changed, 1 insertion(+), 1 deletion(-)
> > 
> > diff --git a/libavcodec/h264_mp4toannexb_bsf.c
> > b/libavcodec/h264_mp4toannexb_bsf.c
> > index d11be455c2..bbdeca9c44 100644
> > --- a/libavcodec/h264_mp4toannexb_bsf.c
> > +++ b/libavcodec/h264_mp4toannexb_bsf.c
> > @@ -259,7 +259,7 @@ static int h264_mp4toannexb_filter(AVBSFContext
> > *ctx, AVPacket *opkt)
> >  
> >              count_or_copy(&out, &out_size, buf, nal_size,
> >                            unit_type == H264_NAL_SPS || unit_type
> > == H264_NAL_PPS, j);
> > -            if (!new_idr && unit_type == H264_NAL_SLICE) {
> > +            if (unit_type == H264_NAL_SLICE) {
> >                  new_idr  = 1;
> >                  sps_seen = 0;
> >                  pps_seen = 0;
> 
> If I understand you correctly, you want that to include the parameter
> sets from the extradata to every IDR frame without extradata if said
> IDR
> frame follows a non-IDR frame after in-band parameter sets. There is
> just one problem with this: Said in-band parameter sets might
> overwrite
> parameter sets from extradata and inserting the outdated parameter
> sets
> from extradata lateron might make the stream undecodable.

It's a good question. Current code works when an IDR has in-band
parameter set, which is different than the out-of-band parameter set,
and the decoder doesn't miss the in-band parameter set. But there is a
high chance the decoder will miss the in-band parameter set for annexb
bitstream, e.g., playback at the middle of a live mpegts streams. It's
a serious issue of missing periodic in-band parameter set for annexb
for a lot of usecases. Handle new parameter set isn't as common as the
need of periodic in-band parameter set (I guess).

It's not a new issue that this bitstream failed to update the parameter
set, for example, it doesn't handle AV_PKT_DATA_NEW_EXTRADATA. I can
add support for
1. handling AV_PKT_DATA_NEW_EXTRADATA
2. handling in-band sps/pps

But it won't be soon.

> 
> - Andreas
> 
> _______________________________________________
> ffmpeg-devel mailing list
> ffmpeg-devel@ffmpeg.org
> https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
> 
> To unsubscribe, visit link above, or email
> ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".

_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] [PATCH] avcodec/h264_mp4toannexb: fix missing SPS/PPS to IDR frames

["zhilizhao(赵志立)"]

> On Jan 10, 2023, at 23:31, Zhao Zhili <quinkblack@foxmail.com> wrote:
> 
> On Tue, 2023-01-10 at 07:41 +0100, Andreas Rheinhardt wrote:
>> Zhao Zhili:
>>> From: Zhao Zhili <zhilizhao@tencent.com>
>>> 
>>> If there is a single group of SPS/PPS before an IDR frame, but no
>>> SPS/PPS after that, we will miss the chance to reset
>>> idr_sps_seen/idr_pps_seen. The result is missing SPS/PPS for all
>>> IDR frames except the first one.
>>> 
>>> Signed-off-by: Zhao Zhili <zhilizhao@tencent.com>
>>> ---
>>> libavcodec/h264_mp4toannexb_bsf.c | 2 +-
>>> 1 file changed, 1 insertion(+), 1 deletion(-)
>>> 
>>> diff --git a/libavcodec/h264_mp4toannexb_bsf.c
>>> b/libavcodec/h264_mp4toannexb_bsf.c
>>> index d11be455c2..bbdeca9c44 100644
>>> --- a/libavcodec/h264_mp4toannexb_bsf.c
>>> +++ b/libavcodec/h264_mp4toannexb_bsf.c
>>> @@ -259,7 +259,7 @@ static int h264_mp4toannexb_filter(AVBSFContext
>>> *ctx, AVPacket *opkt)
>>> 
>>>             count_or_copy(&out, &out_size, buf, nal_size,
>>>                           unit_type == H264_NAL_SPS || unit_type
>>> == H264_NAL_PPS, j);
>>> -            if (!new_idr && unit_type == H264_NAL_SLICE) {
>>> +            if (unit_type == H264_NAL_SLICE) {
>>>                 new_idr  = 1;
>>>                 sps_seen = 0;
>>>                 pps_seen = 0;
>> 
>> If I understand you correctly, you want that to include the parameter
>> sets from the extradata to every IDR frame without extradata if said
>> IDR
>> frame follows a non-IDR frame after in-band parameter sets. There is
>> just one problem with this: Said in-band parameter sets might
>> overwrite
>> parameter sets from extradata and inserting the outdated parameter
>> sets
>> from extradata lateron might make the stream undecodable.
> 
> It's a good question. Current code works when an IDR has in-band
> parameter set, which is different than the out-of-band parameter set,
> and the decoder doesn't miss the in-band parameter set. But there is a
> high chance the decoder will miss the in-band parameter set for annexb
> bitstream, e.g., playback at the middle of a live mpegts streams. It's
> a serious issue of missing periodic in-band parameter set for annexb
> for a lot of usecases. Handle new parameter set isn't as common as the
> need of periodic in-band parameter set (I guess).
> 
> It's not a new issue that this bitstream failed to update the parameter
> set, for example, it doesn't handle AV_PKT_DATA_NEW_EXTRADATA. I can
> add support for
> 1. handling AV_PKT_DATA_NEW_EXTRADATA
> 2. handling in-band sps/pps
> 
> But it won't be soon.

Ping.

> 
>> 
>> - Andreas
>> 
>> _______________________________________________
>> ffmpeg-devel mailing list
>> ffmpeg-devel@ffmpeg.org
>> https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
>> 
>> To unsubscribe, visit link above, or email
>> ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".
> 
> _______________________________________________
> ffmpeg-devel mailing list
> ffmpeg-devel@ffmpeg.org
> https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
> 
> To unsubscribe, visit link above, or email
> ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".

_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".