[FFmpeg-devel] [PATCH 0/2] Interpret a degenerate SAR as 1/1.

[FFmpeg-devel] [PATCH 0/2] Interpret a degenerate SAR as 1/1.

[Giovanni Mascellani]

I came across some H.264 files in the wild whose SAR is set to
20480/0. The files are provided by the Ubisoft game OddBallers, and
are seemingly accepted and played properly when the game is played on
Windows (thus using the Microsoft Media Foundations
implementation).

When running the game with Wine the files are ultimately decoded by
libavcodec (via GStreamer), and playback is broken. It seems that only
a frame each second or so (maybe the key frames? I didn't check) is
decoded and presented, the others are discarded. After dumping the
video, I ran it with ffplay and it has the same problem, and the
following message is emitted many times:

  [h264 @ 0x7fd7301ef440] ignoring invalid SAR: 20480/1

Interestingly the invalid SAR is dumped as 20480/1 (which would be
strange, but in princple legal), while the file has 20480/0 (which
doesn't make sense at all). Equally interestingly, the frames that are
presented are indeed presented with SAR 1/1, like they are on Windows.

The H.264 standard says that "When aspect_ratio_idc is equal to 0 or
sar_width is equal to 0 or sar_height is equal to 0, the sample aspect
ratio shall be considered unspecified by this Recommendation |
International Standard". Given the behavior on Windows it seems that
the de facto standard way to solve the missing specification is to
assume that SAR is 1/1, which is what my patches seek to do.

With these patches both Wine and ffplay replay the broken file
correctly.

Since this is my first contributing to ffmpeg and I am not very
experienced in the project architecture, I'm not sure this is the
right abstraction level at which this problem should be handled. If
it's not, I'd be grateful if you could direct me to the appropriate
place.

Giovanni Mascellani (2):
  avcodec/h2645_vui: Interpret a degenerate SAR as 1/1.
  avcodec/evc_ps: Interpret a degenerate SAR as 1/1.

 libavcodec/evc_ps.c    | 4 ++++
 libavcodec/h2645_vui.c | 4 ++++
 2 files changed, 8 insertions(+)

-- 
2.45.2

_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".

[FFmpeg-devel] [PATCH 1/2] avcodec/h2645_vui: Interpret a degenerate SAR as 1/1.

[Giovanni Mascellani]

The specification doesn't say what should happen in this case.
Other implementations (e.g., Microsoft's Media Foundations) seem
to interpret this as 1/1.

Signed-off-by: Giovanni Mascellani <gmascellani@codeweavers.com>
---
 libavcodec/h2645_vui.c | 4 ++++
 1 file changed, 4 insertions(+)

diff --git a/libavcodec/h2645_vui.c b/libavcodec/h2645_vui.c
index e5c7bf46f9..a9c0dbbef9 100644
--- a/libavcodec/h2645_vui.c
+++ b/libavcodec/h2645_vui.c
@@ -46,6 +46,10 @@ void ff_h2645_decode_common_vui_params(GetBitContext *gb, H2645VUI *vui, void *l
         else if (vui->aspect_ratio_idc == EXTENDED_SAR) {
             vui->sar.num = get_bits(gb, 16);
             vui->sar.den = get_bits(gb, 16);
+            if (vui->sar.num == 0 || vui->sar.den == 0) {
+                vui->sar.num = 1;
+                vui->sar.den = 1;
+            }
         } else
             av_log(logctx, AV_LOG_WARNING,
                    "Unknown SAR index: %u.\n", vui->aspect_ratio_idc);
-- 
2.45.2

_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".

[FFmpeg-devel] [PATCH 2/2] avcodec/evc_ps: Interpret a degenerate SAR as 1/1.

[Giovanni Mascellani]

The specification doesn't say what should happen in this case.
Other implementations (e.g., Microsoft's Media Foundations) seem
to interpret this as 1/1.

Signed-off-by: Giovanni Mascellani <gmascellani@codeweavers.com>
---
 libavcodec/evc_ps.c | 4 ++++
 1 file changed, 4 insertions(+)

diff --git a/libavcodec/evc_ps.c b/libavcodec/evc_ps.c
index 637572d324..6c871e965e 100644
--- a/libavcodec/evc_ps.c
+++ b/libavcodec/evc_ps.c
@@ -87,6 +87,10 @@ static int vui_parameters(GetBitContext *gb, VUIParameters *vui)
         if (vui->aspect_ratio_idc == EXTENDED_SAR) {
             vui->sar_width = get_bits(gb, 16);
             vui->sar_height = get_bits(gb, 16);
+            if (vui->sar_width == 0 || vui->sar_height == 0) {
+                vui->sar_width = 1;
+                vui->sar_height = 1;
+            }
         }
     }
     vui->overscan_info_present_flag = get_bits(gb, 1);
-- 
2.45.2

_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] [PATCH 0/2] Interpret a degenerate SAR as 1/1.

[Michael Niedermayer]

[-- Attachment #1.1: Type: text/plain, Size: 1960 bytes --]

On Tue, Jul 30, 2024 at 03:11:47PM +0200, Giovanni Mascellani wrote:
> I came across some H.264 files in the wild whose SAR is set to
> 20480/0. The files are provided by the Ubisoft game OddBallers, and
> are seemingly accepted and played properly when the game is played on
> Windows (thus using the Microsoft Media Foundations
> implementation).
> 
> When running the game with Wine the files are ultimately decoded by
> libavcodec (via GStreamer), and playback is broken. It seems that only
> a frame each second or so (maybe the key frames? I didn't check) is
> decoded and presented, the others are discarded. After dumping the
> video, I ran it with ffplay and it has the same problem, and the
> following message is emitted many times:
> 
>   [h264 @ 0x7fd7301ef440] ignoring invalid SAR: 20480/1
> 
> Interestingly the invalid SAR is dumped as 20480/1 (which would be
> strange, but in princple legal), while the file has 20480/0 (which
> doesn't make sense at all). Equally interestingly, the frames that are
> presented are indeed presented with SAR 1/1, like they are on Windows.
> 
> The H.264 standard says that "When aspect_ratio_idc is equal to 0 or
> sar_width is equal to 0 or sar_height is equal to 0, the sample aspect
> ratio shall be considered unspecified by this Recommendation |
> International Standard". Given the behavior on Windows it seems that
> the de facto standard way to solve the missing specification is to
> assume that SAR is 1/1, which is what my patches seek to do.

Why does playback fail ?

1/1 and unspecified are different things, 0/0 would be unspecified
where does 20480/0 turn into 20480/1 ? or did i misunderstand this?

thx

[...]
-- 
Michael     GnuPG fingerprint: 9FF2128B147EF6730BADF133611EC787040B0FAB

In fact, the RIAA has been known to suggest that students drop out
of college or go to community college in order to be able to afford
settlements. -- The RIAA

[-- Attachment #1.2: signature.asc --]
[-- Type: application/pgp-signature, Size: 195 bytes --]

[-- Attachment #2: Type: text/plain, Size: 251 bytes --]

_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] [PATCH 0/2] Interpret a degenerate SAR as 1/1.

[James Almer]

On 7/30/2024 5:54 PM, Michael Niedermayer wrote:
> On Tue, Jul 30, 2024 at 03:11:47PM +0200, Giovanni Mascellani wrote:
>> I came across some H.264 files in the wild whose SAR is set to
>> 20480/0. The files are provided by the Ubisoft game OddBallers, and
>> are seemingly accepted and played properly when the game is played on
>> Windows (thus using the Microsoft Media Foundations
>> implementation).
>>
>> When running the game with Wine the files are ultimately decoded by
>> libavcodec (via GStreamer), and playback is broken. It seems that only
>> a frame each second or so (maybe the key frames? I didn't check) is
>> decoded and presented, the others are discarded. After dumping the
>> video, I ran it with ffplay and it has the same problem, and the
>> following message is emitted many times:
>>
>>    [h264 @ 0x7fd7301ef440] ignoring invalid SAR: 20480/1
>>
>> Interestingly the invalid SAR is dumped as 20480/1 (which would be
>> strange, but in princple legal), while the file has 20480/0 (which
>> doesn't make sense at all). Equally interestingly, the frames that are
>> presented are indeed presented with SAR 1/1, like they are on Windows.
>>
>> The H.264 standard says that "When aspect_ratio_idc is equal to 0 or
>> sar_width is equal to 0 or sar_height is equal to 0, the sample aspect
>> ratio shall be considered unspecified by this Recommendation |
>> International Standard". Given the behavior on Windows it seems that
>> the de facto standard way to solve the missing specification is to
>> assume that SAR is 1/1, which is what my patches seek to do.
> 
> Why does playback fail ?
> 
> 1/1 and unspecified are different things, 0/0 would be unspecified

afair we use 0/1 for unspecified, since it prevents division by 0 
without having to worry about extra sanity checks.

> where does 20480/0 turn into 20480/1 ? or did i misunderstand this?
> 
> thx
> 
> [...]
> 
> 
> _______________________________________________
> ffmpeg-devel mailing list
> ffmpeg-devel@ffmpeg.org
> https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
> 
> To unsubscribe, visit link above, or email
> ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".
_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] [PATCH 0/2] Interpret a degenerate SAR as 1/1.

[Michael Niedermayer]

[-- Attachment #1.1: Type: text/plain, Size: 3061 bytes --]

On Tue, Jul 30, 2024 at 06:15:30PM -0300, James Almer wrote:
> On 7/30/2024 5:54 PM, Michael Niedermayer wrote:
> > On Tue, Jul 30, 2024 at 03:11:47PM +0200, Giovanni Mascellani wrote:
> > > I came across some H.264 files in the wild whose SAR is set to
> > > 20480/0. The files are provided by the Ubisoft game OddBallers, and
> > > are seemingly accepted and played properly when the game is played on
> > > Windows (thus using the Microsoft Media Foundations
> > > implementation).
> > > 
> > > When running the game with Wine the files are ultimately decoded by
> > > libavcodec (via GStreamer), and playback is broken. It seems that only
> > > a frame each second or so (maybe the key frames? I didn't check) is
> > > decoded and presented, the others are discarded. After dumping the
> > > video, I ran it with ffplay and it has the same problem, and the
> > > following message is emitted many times:
> > > 
> > >    [h264 @ 0x7fd7301ef440] ignoring invalid SAR: 20480/1
> > > 
> > > Interestingly the invalid SAR is dumped as 20480/1 (which would be
> > > strange, but in princple legal), while the file has 20480/0 (which
> > > doesn't make sense at all). Equally interestingly, the frames that are
> > > presented are indeed presented with SAR 1/1, like they are on Windows.
> > > 
> > > The H.264 standard says that "When aspect_ratio_idc is equal to 0 or
> > > sar_width is equal to 0 or sar_height is equal to 0, the sample aspect
> > > ratio shall be considered unspecified by this Recommendation |
> > > International Standard". Given the behavior on Windows it seems that
> > > the de facto standard way to solve the missing specification is to
> > > assume that SAR is 1/1, which is what my patches seek to do.
> > 
> > Why does playback fail ?
> > 
> > 1/1 and unspecified are different things, 0/0 would be unspecified
> 
> afair we use 0/1 for unspecified, since it prevents division by 0 without
> having to worry about extra sanity checks.

why this is a bad idea:

if a rectangle W/H with unspecified AR is 0/1, now the H/W rectangle has 1/0

OTOH if you use 0/0 then both W/H and H/W are naturally 0/0

what about scaling?

0/1 scaled by a/b is 0/b, while 0/0 scaled by a/b is 0/0

what about adding ratios ? 0/1 + a/b = a/b (thats not unspecified anymore)
OTOH
0/0 + a/b = 0/0 :)

so people may argue about this, but 0/0 behaves much closer to unspecified than 0/1
so fewer special cases are needed if 0/0 is used. And 0/1 really is 0 and 0
is not unspecified in a mathematical sense which is why it works so poorly for that

And in floats 0/0 is NaN which is again behaving much closer to unspecified.
If you do an operation with a NaN the result is NaN, same as if you do an
operation with a unspecified ratio, the result is unspecified

Thx

[...]

-- 
Michael     GnuPG fingerprint: 9FF2128B147EF6730BADF133611EC787040B0FAB

The smallest minority on earth is the individual. Those who deny 
individual rights cannot claim to be defenders of minorities. - Ayn Rand

[-- Attachment #1.2: signature.asc --]
[-- Type: application/pgp-signature, Size: 195 bytes --]

[-- Attachment #2: Type: text/plain, Size: 251 bytes --]

_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] [PATCH 0/2] Interpret a degenerate SAR as 1/1.

[Giovanni Mascellani]

Hi,

Il 30/07/24 22:54, michael at niedermayer.cc (Michael Niedermayer) ha 
scritto:
> Why does playback fail ? > 1/1 and unspecified are different things, 0/0 would be unspecified
> where does 20480/0 turn into 20480/1 ? or did i misunderstand this?

It seems that denominator 0 is replaced with 1 here:

 
https://git.ffmpeg.org/gitweb/ffmpeg.git/blob/262168b04e6807fce6a78507c14cfc166ba72845:/libavcodec/h264_ps.c#l552

Considering the other replies I would guess that the intent is to 
prevent a division by zero from happening, but the side effect is that 
an invalid SAR is silently converted to a valid one (even if quite extreme).

I am not sure why that in turn causes the playback problems. I noticed 
that the same problem can be reproduced on any other H.264 file simply 
by forcing the broken 20480/0 SAR in ff_h2645_decode_common_vui_params().

After a few more experiments, it seems that ffmpeg is indeed able to 
handle an unspecified SAR, but it doesn't like it to be represented as 
x/0, only 0/x. I'll send a revised patch which replaces x/0 with 1/0, 
hopefully addressing your concern.

Thanks, Giovanni.
_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".