[FFmpeg-devel] [PATCH] avformat/mov: remove hack breaking creation time parsing

[FFmpeg-devel] [PATCH] avformat/mov: remove hack breaking creation time parsing

[Marton Balint]

Commit 23eeffcd48a15e73fb2649b712870b6d101c5471 added a hack to support invalid
files where the creation date was encoded as a classic unix timestamp. This
broke however valid files having creation dates before the unix epoch.

Signed-off-by: Marton Balint <cus@passwd.hu>
---
 libavformat/mov.c | 3 +--
 1 file changed, 1 insertion(+), 2 deletions(-)

diff --git a/libavformat/mov.c b/libavformat/mov.c
index 057fd872b1..7748bdc041 100644
--- a/libavformat/mov.c
+++ b/libavformat/mov.c
@@ -1518,8 +1518,7 @@ static int mov_read_moof(MOVContext *c, AVIOContext *pb, MOVAtom atom)
 static void mov_metadata_creation_time(AVDictionary **metadata, int64_t time, void *logctx)
 {
     if (time) {
-        if (time >= 2082844800)
-            time -= 2082844800;  /* seconds between 1904-01-01 and Epoch */
+        time -= 2082844800;  /* seconds between 1904-01-01 and Epoch */
 
         if ((int64_t)(time * 1000000ULL) / 1000000 != time) {
             av_log(logctx, AV_LOG_DEBUG, "creation_time is not representable\n");
-- 
2.35.3

_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] [PATCH] avformat/mov: remove hack breaking creation time parsing

[Michael Niedermayer]

[-- Attachment #1.1: Type: text/plain, Size: 1079 bytes --]

On Sat, Apr 08, 2023 at 08:37:24PM +0200, Marton Balint wrote:
> Commit 23eeffcd48a15e73fb2649b712870b6d101c5471 added a hack to support invalid
> files where the creation date was encoded as a classic unix timestamp. This
> broke however valid files having creation dates before the unix epoch.
> 
> Signed-off-by: Marton Balint <cus@passwd.hu>
> ---
>  libavformat/mov.c | 3 +--
>  1 file changed, 1 insertion(+), 2 deletions(-)

This results in:
@@ -1,11 +1,11 @@
-    creation_time   : 2012-06-20T20:58:31.000000Z
-      creation_time   : 2012-06-20T20:58:31.000000Z
-      creation_time   : 2012-06-20T20:58:31.000000Z
+    creation_time   : 1946-06-20T20:58:31.000000Z
+      creation_time   : 1946-06-20T20:58:31.000000Z
+      creation_time   : 1946-06-20T20:58:31.000000Z

Are you sure that 1946 is the correct creation date and not 2012 ?

Thx

[...]
-- 
Michael     GnuPG fingerprint: 9FF2128B147EF6730BADF133611EC787040B0FAB

While the State exists there can be no freedom; when there is freedom there
will be no State. -- Vladimir Lenin

[-- Attachment #1.2: signature.asc --]
[-- Type: application/pgp-signature, Size: 195 bytes --]

[-- Attachment #2: Type: text/plain, Size: 251 bytes --]

_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] [PATCH] avformat/mov: remove hack breaking creation time parsing

[Marton Balint]

On Sat, 8 Apr 2023, Michael Niedermayer wrote:

> On Sat, Apr 08, 2023 at 08:37:24PM +0200, Marton Balint wrote:
>> Commit 23eeffcd48a15e73fb2649b712870b6d101c5471 added a hack to support invalid
>> files where the creation date was encoded as a classic unix timestamp. This
>> broke however valid files having creation dates before the unix epoch.
>>
>> Signed-off-by: Marton Balint <cus@passwd.hu>
>> ---
>>  libavformat/mov.c | 3 +--
>>  1 file changed, 1 insertion(+), 2 deletions(-)
>
> This results in:
> @@ -1,11 +1,11 @@
> -    creation_time   : 2012-06-20T20:58:31.000000Z
> -      creation_time   : 2012-06-20T20:58:31.000000Z
> -      creation_time   : 2012-06-20T20:58:31.000000Z
> +    creation_time   : 1946-06-20T20:58:31.000000Z
> +      creation_time   : 1946-06-20T20:58:31.000000Z
> +      creation_time   : 1946-06-20T20:58:31.000000Z
>
> Are you sure that 1946 is the correct creation date and not 2012 ?

If you are referring to the file in ticket #1471, yes, 1946 is consistent 
with what mediainfo shows for creation time. Obviously 1946 was not the 
intended creation time, but that does not warrant us to break 
files where 1946 is the *intended* creation time. Proper way to fix the 
original issue would be to detect the device and software version which 
produces the invalid files, and only apply the hack there. But I don't 
think that is doable here, the file does not seem to contain any device or 
software information.

Regards,
Marton
_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] [PATCH] avformat/mov: remove hack breaking creation time parsing

[Anton Khirnov]

Quoting Marton Balint (2023-04-09 15:49:33)
> 
> 
> On Sat, 8 Apr 2023, Michael Niedermayer wrote:
> 
> > On Sat, Apr 08, 2023 at 08:37:24PM +0200, Marton Balint wrote:
> >> Commit 23eeffcd48a15e73fb2649b712870b6d101c5471 added a hack to support invalid
> >> files where the creation date was encoded as a classic unix timestamp. This
> >> broke however valid files having creation dates before the unix epoch.
> >>
> >> Signed-off-by: Marton Balint <cus@passwd.hu>
> >> ---
> >>  libavformat/mov.c | 3 +--
> >>  1 file changed, 1 insertion(+), 2 deletions(-)
> >
> > This results in:
> > @@ -1,11 +1,11 @@
> > -    creation_time   : 2012-06-20T20:58:31.000000Z
> > -      creation_time   : 2012-06-20T20:58:31.000000Z
> > -      creation_time   : 2012-06-20T20:58:31.000000Z
> > +    creation_time   : 1946-06-20T20:58:31.000000Z
> > +      creation_time   : 1946-06-20T20:58:31.000000Z
> > +      creation_time   : 1946-06-20T20:58:31.000000Z
> >
> > Are you sure that 1946 is the correct creation date and not 2012 ?
> 
> If you are referring to the file in ticket #1471, yes, 1946 is consistent 
> with what mediainfo shows for creation time. Obviously 1946 was not the 
> intended creation time, but that does not warrant us to break 
> files where 1946 is the *intended* creation time. Proper way to fix the 
> original issue would be to detect the device and software version which 
> produces the invalid files, and only apply the hack there. But I don't 
> think that is doable here, the file does not seem to contain any device or 
> software information.

How about adding an option for it, like -unix_time?

-- 
Anton Khirnov
_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] [PATCH] avformat/mov: remove hack breaking creation time parsing

[Michael Niedermayer]

[-- Attachment #1.1: Type: text/plain, Size: 2256 bytes --]

On Sun, Apr 09, 2023 at 03:49:33PM +0200, Marton Balint wrote:
> 
> 
> On Sat, 8 Apr 2023, Michael Niedermayer wrote:
> 
> > On Sat, Apr 08, 2023 at 08:37:24PM +0200, Marton Balint wrote:
> > > Commit 23eeffcd48a15e73fb2649b712870b6d101c5471 added a hack to support invalid
> > > files where the creation date was encoded as a classic unix timestamp. This
> > > broke however valid files having creation dates before the unix epoch.
> > > 
> > > Signed-off-by: Marton Balint <cus@passwd.hu>
> > > ---
> > >  libavformat/mov.c | 3 +--
> > >  1 file changed, 1 insertion(+), 2 deletions(-)
> > 
> > This results in:
> > @@ -1,11 +1,11 @@
> > -    creation_time   : 2012-06-20T20:58:31.000000Z
> > -      creation_time   : 2012-06-20T20:58:31.000000Z
> > -      creation_time   : 2012-06-20T20:58:31.000000Z
> > +    creation_time   : 1946-06-20T20:58:31.000000Z
> > +      creation_time   : 1946-06-20T20:58:31.000000Z
> > +      creation_time   : 1946-06-20T20:58:31.000000Z
> > 
> > Are you sure that 1946 is the correct creation date and not 2012 ?
> 
> If you are referring to the file in ticket #1471, yes, 1946 is consistent
> with what mediainfo shows for creation time. Obviously 1946 was not the
> intended creation time, but that does not warrant us to break files where
> 1946 is the *intended* creation time. Proper way to fix the original issue
> would be to detect the device and software version which produces the
> invalid files, and only apply the hack there. But I don't think that is
> doable here, the file does not seem to contain any device or software
> information.

what do you mean by intended creation time?
the file format did not exist in 1946. and all the codecs also didnt exist
so when you encounter a file that says its from that time it must be crafted
later and backdated or that bug.
we know the bug is a real thing
do you want to support crafted and backdatred files? if so can you explain
the usecase for that ?
maybe iam missing something

thx

[...]
-- 
Michael     GnuPG fingerprint: 9FF2128B147EF6730BADF133611EC787040B0FAB

Does the universe only have a finite lifespan? No, its going to go on
forever, its just that you wont like living in it. -- Hiranya Peiri

[-- Attachment #1.2: signature.asc --]
[-- Type: application/pgp-signature, Size: 195 bytes --]

[-- Attachment #2: Type: text/plain, Size: 251 bytes --]

_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] [PATCH] avformat/mov: remove hack breaking creation time parsing

[Marton Balint]

On Sun, 9 Apr 2023, Michael Niedermayer wrote:

> On Sun, Apr 09, 2023 at 03:49:33PM +0200, Marton Balint wrote:
>>
>>
>> On Sat, 8 Apr 2023, Michael Niedermayer wrote:
>>
>>> On Sat, Apr 08, 2023 at 08:37:24PM +0200, Marton Balint wrote:
>>>> Commit 23eeffcd48a15e73fb2649b712870b6d101c5471 added a hack to support invalid
>>>> files where the creation date was encoded as a classic unix timestamp. This
>>>> broke however valid files having creation dates before the unix epoch.
>>>>
>>>> Signed-off-by: Marton Balint <cus@passwd.hu>
>>>> ---
>>>>  libavformat/mov.c | 3 +--
>>>>  1 file changed, 1 insertion(+), 2 deletions(-)
>>>
>>> This results in:
>>> @@ -1,11 +1,11 @@
>>> -    creation_time   : 2012-06-20T20:58:31.000000Z
>>> -      creation_time   : 2012-06-20T20:58:31.000000Z
>>> -      creation_time   : 2012-06-20T20:58:31.000000Z
>>> +    creation_time   : 1946-06-20T20:58:31.000000Z
>>> +      creation_time   : 1946-06-20T20:58:31.000000Z
>>> +      creation_time   : 1946-06-20T20:58:31.000000Z
>>>
>>> Are you sure that 1946 is the correct creation date and not 2012 ?
>>
>> If you are referring to the file in ticket #1471, yes, 1946 is consistent
>> with what mediainfo shows for creation time. Obviously 1946 was not the
>> intended creation time, but that does not warrant us to break files where
>> 1946 is the *intended* creation time. Proper way to fix the original issue
>> would be to detect the device and software version which produces the
>> invalid files, and only apply the hack there. But I don't think that is
>> doable here, the file does not seem to contain any device or software
>> information.
>
> what do you mean by intended creation time?
> the file format did not exist in 1946. and all the codecs also didnt exist
> so when you encounter a file that says its from that time it must be crafted
> later and backdated or that bug.
> we know the bug is a real thing
> do you want to support crafted and backdatred files? if so can you explain
> the usecase for that ?

http://ffmpeg.org/pipermail/ffmpeg-user/2023-April/056265.html

Alternatives I can think of:

1) A -unix_time switch what Anton proposed
2) doing strict compliant parsing only if mdat version is 1 so creation 
time is 64bit. And change our muxer to write mdat version 1 by default, so 
ffmpeg will be able to read back what it has written...

Regards,
Marton
_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] [PATCH] avformat/mov: remove hack breaking creation time parsing

[Michael Niedermayer]

[-- Attachment #1.1: Type: text/plain, Size: 3105 bytes --]

On Sun, Apr 09, 2023 at 07:52:12PM +0200, Marton Balint wrote:
> 
> 
> On Sun, 9 Apr 2023, Michael Niedermayer wrote:
> 
> > On Sun, Apr 09, 2023 at 03:49:33PM +0200, Marton Balint wrote:
> > > 
> > > 
> > > On Sat, 8 Apr 2023, Michael Niedermayer wrote:
> > > 
> > > > On Sat, Apr 08, 2023 at 08:37:24PM +0200, Marton Balint wrote:
> > > > > Commit 23eeffcd48a15e73fb2649b712870b6d101c5471 added a hack to support invalid
> > > > > files where the creation date was encoded as a classic unix timestamp. This
> > > > > broke however valid files having creation dates before the unix epoch.
> > > > > 
> > > > > Signed-off-by: Marton Balint <cus@passwd.hu>
> > > > > ---
> > > > >  libavformat/mov.c | 3 +--
> > > > >  1 file changed, 1 insertion(+), 2 deletions(-)
> > > > 
> > > > This results in:
> > > > @@ -1,11 +1,11 @@
> > > > -    creation_time   : 2012-06-20T20:58:31.000000Z
> > > > -      creation_time   : 2012-06-20T20:58:31.000000Z
> > > > -      creation_time   : 2012-06-20T20:58:31.000000Z
> > > > +    creation_time   : 1946-06-20T20:58:31.000000Z
> > > > +      creation_time   : 1946-06-20T20:58:31.000000Z
> > > > +      creation_time   : 1946-06-20T20:58:31.000000Z
> > > > 
> > > > Are you sure that 1946 is the correct creation date and not 2012 ?
> > > 
> > > If you are referring to the file in ticket #1471, yes, 1946 is consistent
> > > with what mediainfo shows for creation time. Obviously 1946 was not the
> > > intended creation time, but that does not warrant us to break files where
> > > 1946 is the *intended* creation time. Proper way to fix the original issue
> > > would be to detect the device and software version which produces the
> > > invalid files, and only apply the hack there. But I don't think that is
> > > doable here, the file does not seem to contain any device or software
> > > information.
> > 
> > what do you mean by intended creation time?
> > the file format did not exist in 1946. and all the codecs also didnt exist
> > so when you encounter a file that says its from that time it must be crafted
> > later and backdated or that bug.
> > we know the bug is a real thing
> > do you want to support crafted and backdatred files? if so can you explain
> > the usecase for that ?
> 
> http://ffmpeg.org/pipermail/ffmpeg-user/2023-April/056265.html
> 
> Alternatives I can think of:
> 
> 1) A -unix_time switch what Anton proposed

> 2) doing strict compliant parsing only if mdat version is 1 so creation time
> is 64bit. And change our muxer to write mdat version 1 by default, so ffmpeg
> will be able to read back what it has written...

What do we know about the buggy files that need this correction ?
Is there any hint/metadata that identifies the muxer/encoder/version ?

Limiting the correction to the cases that need it is a good idea
Iam not sure i feel positive about changing the muxer

thx


[...]

-- 
Michael     GnuPG fingerprint: 9FF2128B147EF6730BADF133611EC787040B0FAB

In a rich man's house there is no place to spit but his face.
-- Diogenes of Sinope

[-- Attachment #1.2: signature.asc --]
[-- Type: application/pgp-signature, Size: 195 bytes --]

[-- Attachment #2: Type: text/plain, Size: 251 bytes --]

_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] [PATCH] avformat/mov: remove hack breaking creation time parsing

[Marton Balint]

On Sun, 9 Apr 2023, Michael Niedermayer wrote:

> On Sun, Apr 09, 2023 at 07:52:12PM +0200, Marton Balint wrote:
>>
>>
>> On Sun, 9 Apr 2023, Michael Niedermayer wrote:
>>
>>> On Sun, Apr 09, 2023 at 03:49:33PM +0200, Marton Balint wrote:
>>>>
>>>>
>>>> On Sat, 8 Apr 2023, Michael Niedermayer wrote:
>>>>
>>>>> On Sat, Apr 08, 2023 at 08:37:24PM +0200, Marton Balint wrote:
>>>>>> Commit 23eeffcd48a15e73fb2649b712870b6d101c5471 added a hack to support invalid
>>>>>> files where the creation date was encoded as a classic unix timestamp. This
>>>>>> broke however valid files having creation dates before the unix epoch.
>>>>>>
>>>>>> Signed-off-by: Marton Balint <cus@passwd.hu>
>>>>>> ---
>>>>>>  libavformat/mov.c | 3 +--
>>>>>>  1 file changed, 1 insertion(+), 2 deletions(-)
>>>>>
>>>>> This results in:
>>>>> @@ -1,11 +1,11 @@
>>>>> -    creation_time   : 2012-06-20T20:58:31.000000Z
>>>>> -      creation_time   : 2012-06-20T20:58:31.000000Z
>>>>> -      creation_time   : 2012-06-20T20:58:31.000000Z
>>>>> +    creation_time   : 1946-06-20T20:58:31.000000Z
>>>>> +      creation_time   : 1946-06-20T20:58:31.000000Z
>>>>> +      creation_time   : 1946-06-20T20:58:31.000000Z
>>>>>
>>>>> Are you sure that 1946 is the correct creation date and not 2012 ?
>>>>
>>>> If you are referring to the file in ticket #1471, yes, 1946 is consistent
>>>> with what mediainfo shows for creation time. Obviously 1946 was not the
>>>> intended creation time, but that does not warrant us to break files where
>>>> 1946 is the *intended* creation time. Proper way to fix the original issue
>>>> would be to detect the device and software version which produces the
>>>> invalid files, and only apply the hack there. But I don't think that is
>>>> doable here, the file does not seem to contain any device or software
>>>> information.
>>>
>>> what do you mean by intended creation time?
>>> the file format did not exist in 1946. and all the codecs also didnt exist
>>> so when you encounter a file that says its from that time it must be crafted
>>> later and backdated or that bug.
>>> we know the bug is a real thing
>>> do you want to support crafted and backdatred files? if so can you explain
>>> the usecase for that ?
>>
>> http://ffmpeg.org/pipermail/ffmpeg-user/2023-April/056265.html
>>
>> Alternatives I can think of:
>>
>> 1) A -unix_time switch what Anton proposed
>
>> 2) doing strict compliant parsing only if mdat version is 1 so creation time
>> is 64bit. And change our muxer to write mdat version 1 by default, so ffmpeg
>> will be able to read back what it has written...
>
> What do we know about the buggy files that need this correction ?

Not much. Samsung Galaxy Nexus phone and Samsung HMX-S16BP camcorder seems 
affected at least. But not exclusively. I found these files among the 
samples:

Samsung HMX-S16BP:
samples/ffmpeg-bugs/roundup/issue2517/HDV_0112.MP4
samples/ffmpeg-bugs/roundup/issue2517_HDV_0113.MP4

Galaxy Nexus:
Sample in ticket 1471.

Unknown:
samples/ffmpeg-bugs/trac/ticket2095_385 Deadlist Form 2-7-13.mp4
samples/ffmpeg-bugs/trac/ticket3399_VID_20130619_161750_449.mp4
user/aac-input-buffer-exhausted-up_1434137794_VID_20120604_172442.mp4

> Is there any hint/metadata that identifies the muxer/encoder/version ?

In case of HMX-S16BP yes, in other cases no.

>
> Limiting the correction to the cases that need it is a good idea

I guess I will start with limiting workaround for version 1 as a start.

> Iam not sure i feel positive about changing the muxer

Eventually we have to bump the mdhd version, because after 2040 creation 
time will overflow. We should not wait until 2040 to do that, neither we 
should start writing version 1 only after 2040, that would be Y2K problem 
waiting to happen.

Regards,
Marton
_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] [PATCH] avformat/mov: remove hack breaking creation time parsing

[Michael Niedermayer]

[-- Attachment #1.1: Type: text/plain, Size: 4968 bytes --]

On Mon, Apr 10, 2023 at 09:11:04PM +0200, Marton Balint wrote:
> 
> 
> On Sun, 9 Apr 2023, Michael Niedermayer wrote:
> 
> > On Sun, Apr 09, 2023 at 07:52:12PM +0200, Marton Balint wrote:
> > > 
> > > 
> > > On Sun, 9 Apr 2023, Michael Niedermayer wrote:
> > > 
> > > > On Sun, Apr 09, 2023 at 03:49:33PM +0200, Marton Balint wrote:
> > > > > 
> > > > > 
> > > > > On Sat, 8 Apr 2023, Michael Niedermayer wrote:
> > > > > 
> > > > > > On Sat, Apr 08, 2023 at 08:37:24PM +0200, Marton Balint wrote:
> > > > > > > Commit 23eeffcd48a15e73fb2649b712870b6d101c5471 added a hack to support invalid
> > > > > > > files where the creation date was encoded as a classic unix timestamp. This
> > > > > > > broke however valid files having creation dates before the unix epoch.
> > > > > > > 
> > > > > > > Signed-off-by: Marton Balint <cus@passwd.hu>
> > > > > > > ---
> > > > > > >  libavformat/mov.c | 3 +--
> > > > > > >  1 file changed, 1 insertion(+), 2 deletions(-)
> > > > > > 
> > > > > > This results in:
> > > > > > @@ -1,11 +1,11 @@
> > > > > > -    creation_time   : 2012-06-20T20:58:31.000000Z
> > > > > > -      creation_time   : 2012-06-20T20:58:31.000000Z
> > > > > > -      creation_time   : 2012-06-20T20:58:31.000000Z
> > > > > > +    creation_time   : 1946-06-20T20:58:31.000000Z
> > > > > > +      creation_time   : 1946-06-20T20:58:31.000000Z
> > > > > > +      creation_time   : 1946-06-20T20:58:31.000000Z
> > > > > > 
> > > > > > Are you sure that 1946 is the correct creation date and not 2012 ?
> > > > > 
> > > > > If you are referring to the file in ticket #1471, yes, 1946 is consistent
> > > > > with what mediainfo shows for creation time. Obviously 1946 was not the
> > > > > intended creation time, but that does not warrant us to break files where
> > > > > 1946 is the *intended* creation time. Proper way to fix the original issue
> > > > > would be to detect the device and software version which produces the
> > > > > invalid files, and only apply the hack there. But I don't think that is
> > > > > doable here, the file does not seem to contain any device or software
> > > > > information.
> > > > 
> > > > what do you mean by intended creation time?
> > > > the file format did not exist in 1946. and all the codecs also didnt exist
> > > > so when you encounter a file that says its from that time it must be crafted
> > > > later and backdated or that bug.
> > > > we know the bug is a real thing
> > > > do you want to support crafted and backdatred files? if so can you explain
> > > > the usecase for that ?
> > > 
> > > http://ffmpeg.org/pipermail/ffmpeg-user/2023-April/056265.html
> > > 
> > > Alternatives I can think of:
> > > 
> > > 1) A -unix_time switch what Anton proposed
> > 
> > > 2) doing strict compliant parsing only if mdat version is 1 so creation time
> > > is 64bit. And change our muxer to write mdat version 1 by default, so ffmpeg
> > > will be able to read back what it has written...
> > 
> > What do we know about the buggy files that need this correction ?
> 
> Not much. Samsung Galaxy Nexus phone and Samsung HMX-S16BP camcorder seems
> affected at least. But not exclusively. I found these files among the
> samples:
> 
> Samsung HMX-S16BP:
> samples/ffmpeg-bugs/roundup/issue2517/HDV_0112.MP4
> samples/ffmpeg-bugs/roundup/issue2517_HDV_0113.MP4
> 
> Galaxy Nexus:
> Sample in ticket 1471.
> 
> Unknown:
> samples/ffmpeg-bugs/trac/ticket2095_385 Deadlist Form 2-7-13.mp4
> samples/ffmpeg-bugs/trac/ticket3399_VID_20130619_161750_449.mp4
> user/aac-input-buffer-exhausted-up_1434137794_VID_20120604_172442.mp4
> 
> > Is there any hint/metadata that identifies the muxer/encoder/version ?
> 
> In case of HMX-S16BP yes, in other cases no.
> 
> > 
> > Limiting the correction to the cases that need it is a good idea
> 
> I guess I will start with limiting workaround for version 1 as a start.
> 
> > Iam not sure i feel positive about changing the muxer
> 
> Eventually we have to bump the mdhd version, because after 2040 creation
> time will overflow. We should not wait until 2040 to do that, neither we
> should start writing version 1 only after 2040, that would be Y2K problem
> waiting to happen.

yes sure, if theres a problem with the muxer then thats perfectly fine
to change the muxer. I just meant that if the only reason is a buggy 
3rd party muxer then changing our muxer felt like something to rethink
and sleep over first.

Iam not against bumping the mdhd version in master at some point (not immedeatly
before a release).

thx

[...]

-- 
Michael     GnuPG fingerprint: 9FF2128B147EF6730BADF133611EC787040B0FAB

Old school: Use the lowest level language in which you can solve the problem
            conveniently.
New school: Use the highest level language in which the latest supercomputer
            can solve the problem without the user falling asleep waiting.

[-- Attachment #1.2: signature.asc --]
[-- Type: application/pgp-signature, Size: 195 bytes --]

[-- Attachment #2: Type: text/plain, Size: 251 bytes --]

_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] [PATCH] avformat/mov: remove hack breaking creation time parsing

["zhilizhao(赵志立)"]

> On Apr 10, 2023, at 01:20, Michael Niedermayer <michael@niedermayer.cc> wrote:
> 
> On Sun, Apr 09, 2023 at 03:49:33PM +0200, Marton Balint wrote:
>> 
>> 
>> On Sat, 8 Apr 2023, Michael Niedermayer wrote:
>> 
>>> On Sat, Apr 08, 2023 at 08:37:24PM +0200, Marton Balint wrote:
>>>> Commit 23eeffcd48a15e73fb2649b712870b6d101c5471 added a hack to support invalid
>>>> files where the creation date was encoded as a classic unix timestamp. This
>>>> broke however valid files having creation dates before the unix epoch.
>>>> 
>>>> Signed-off-by: Marton Balint <cus@passwd.hu>
>>>> ---
>>>> libavformat/mov.c | 3 +--
>>>> 1 file changed, 1 insertion(+), 2 deletions(-)
>>> 
>>> This results in:
>>> @@ -1,11 +1,11 @@
>>> -    creation_time   : 2012-06-20T20:58:31.000000Z
>>> -      creation_time   : 2012-06-20T20:58:31.000000Z
>>> -      creation_time   : 2012-06-20T20:58:31.000000Z
>>> +    creation_time   : 1946-06-20T20:58:31.000000Z
>>> +      creation_time   : 1946-06-20T20:58:31.000000Z
>>> +      creation_time   : 1946-06-20T20:58:31.000000Z
>>> 
>>> Are you sure that 1946 is the correct creation date and not 2012 ?
>> 
>> If you are referring to the file in ticket #1471, yes, 1946 is consistent
>> with what mediainfo shows for creation time. Obviously 1946 was not the
>> intended creation time, but that does not warrant us to break files where
>> 1946 is the *intended* creation time. Proper way to fix the original issue
>> would be to detect the device and software version which produces the
>> invalid files, and only apply the hack there. But I don't think that is
>> doable here, the file does not seem to contain any device or software
>> information.
> 
> what do you mean by intended creation time?
> the file format did not exist in 1946. and all the codecs also didnt exist
> so when you encounter a file that says its from that time it must be crafted
> later and backdated or that bug.
> we know the bug is a real thing
> do you want to support crafted and backdatred files? if so can you explain
> the usecase for that ?
> maybe iam missing something

The workaround can be a positive feedback to create more broken files:

1. Once FFmpeg demuxer supports it, other demuxers may be required to
do the same workaround, because FFmpeg is (kind of) the standard software.

2. Someone write a new muxer can make the same mistake and doesn’t notice,
since FFmpeg don’t complain on the broken files.

3. Old muxers which create those files don’t fix the bug, because FFmpeg
doesn't complain.

We need to workaround bugs which break the process of demux or decoding.
Metadata with invalid values doesn't has the same priority. Make a big
noise at least to reduce the positive feedback effect, so user can report
bugs to the right place.

> 
> thx
> 
> [...]
> -- 
> Michael     GnuPG fingerprint: 9FF2128B147EF6730BADF133611EC787040B0FAB
> 
> Does the universe only have a finite lifespan? No, its going to go on
> forever, its just that you wont like living in it. -- Hiranya Peiri
> _______________________________________________
> ffmpeg-devel mailing list
> ffmpeg-devel@ffmpeg.org
> https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
> 
> To unsubscribe, visit link above, or email
> ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".

_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] [PATCH] avformat/mov: remove hack breaking creation time parsing

[Michael Niedermayer]

[-- Attachment #1.1: Type: text/plain, Size: 3830 bytes --]

On Mon, Apr 10, 2023 at 10:44:32AM +0800, "zhilizhao(赵志立)" wrote:
> 
> 
> > On Apr 10, 2023, at 01:20, Michael Niedermayer <michael@niedermayer.cc> wrote:
> > 
> > On Sun, Apr 09, 2023 at 03:49:33PM +0200, Marton Balint wrote:
> >> 
> >> 
> >> On Sat, 8 Apr 2023, Michael Niedermayer wrote:
> >> 
> >>> On Sat, Apr 08, 2023 at 08:37:24PM +0200, Marton Balint wrote:
> >>>> Commit 23eeffcd48a15e73fb2649b712870b6d101c5471 added a hack to support invalid
> >>>> files where the creation date was encoded as a classic unix timestamp. This
> >>>> broke however valid files having creation dates before the unix epoch.
> >>>> 
> >>>> Signed-off-by: Marton Balint <cus@passwd.hu>
> >>>> ---
> >>>> libavformat/mov.c | 3 +--
> >>>> 1 file changed, 1 insertion(+), 2 deletions(-)
> >>> 
> >>> This results in:
> >>> @@ -1,11 +1,11 @@
> >>> -    creation_time   : 2012-06-20T20:58:31.000000Z
> >>> -      creation_time   : 2012-06-20T20:58:31.000000Z
> >>> -      creation_time   : 2012-06-20T20:58:31.000000Z
> >>> +    creation_time   : 1946-06-20T20:58:31.000000Z
> >>> +      creation_time   : 1946-06-20T20:58:31.000000Z
> >>> +      creation_time   : 1946-06-20T20:58:31.000000Z
> >>> 
> >>> Are you sure that 1946 is the correct creation date and not 2012 ?
> >> 
> >> If you are referring to the file in ticket #1471, yes, 1946 is consistent
> >> with what mediainfo shows for creation time. Obviously 1946 was not the
> >> intended creation time, but that does not warrant us to break files where
> >> 1946 is the *intended* creation time. Proper way to fix the original issue
> >> would be to detect the device and software version which produces the
> >> invalid files, and only apply the hack there. But I don't think that is
> >> doable here, the file does not seem to contain any device or software
> >> information.
> > 
> > what do you mean by intended creation time?
> > the file format did not exist in 1946. and all the codecs also didnt exist
> > so when you encounter a file that says its from that time it must be crafted
> > later and backdated or that bug.
> > we know the bug is a real thing
> > do you want to support crafted and backdatred files? if so can you explain
> > the usecase for that ?
> > maybe iam missing something
> 
> The workaround can be a positive feedback to create more broken files:
> 
> 1. Once FFmpeg demuxer supports it, other demuxers may be required to
> do the same workaround, because FFmpeg is (kind of) the standard software.

In this case iam not sure, its just a fairly irrelevant piece of metadata
Also why would we care that a feature causes work to the competition 


> 
> 2. Someone write a new muxer can make the same mistake and doesn’t notice,
> since FFmpeg don’t complain on the broken files.
> 
> 3. Old muxers which create those files don’t fix the bug, because FFmpeg
> doesn't complain.
> 
> We need to workaround bugs which break the process of demux or decoding.
> Metadata with invalid values doesn't has the same priority. Make a big
> noise at least to reduce the positive feedback effect, so user can report
> bugs to the right place.

I agree we should be more noisy about it when we encounter broken files.

About causing more broken files with bug workarounds, we have worked
around bugs alot so one would expect that there have to be many stories
of that causing new bugs and unfixed bugs not just hypothethial cases.
Still every time i read about it its a hypothethial case not a real
example that somewhere a bug was caused by a workaround. So maybe
that direction is relatively rare

thx

[...]
-- 
Michael     GnuPG fingerprint: 9FF2128B147EF6730BADF133611EC787040B0FAB

Opposition brings concord. Out of discord comes the fairest harmony.
-- Heraclitus

[-- Attachment #1.2: signature.asc --]
[-- Type: application/pgp-signature, Size: 195 bytes --]

[-- Attachment #2: Type: text/plain, Size: 251 bytes --]

_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] [PATCH] avformat/mov: remove hack breaking creation time parsing

[Bernd Dürrer]

On Sun Apr 9 20:20:04 EEST 2023, Michael Niedermayer wrote: 

> what do you mean by intended creation time?
> the file format did not exist in 1946. and all the codecs also didnt exist
> so when you encounter a file that says its from that time it must be crafted
> later and backdated or that bug.

As defined in ISO/IEC 14496-12, clause 8.2, "creation_time is an integer that declares the creation time of the presentation (in seconds since midnight, Jan. 1, 1904, in UTC time)". This is the date when the content was created (which may have been on photographic film), and not when it was encoded as a digital file. As described in http://ffmpeg.org/pipermail/ffmpeg-user/2023-April/056265.html, the use case is to set the creation_time tag to this date of content creation as DLNA servers pass this date on to their clients. However, unlike the standard ISO/IEC 14496-12, ffmpeg does not support dates before 1970.

As far as I can see from the code, the reason for this behaviour is that the function ff_parse_creation_time_metadata in libavformat/mux_utils.c calls av_parse_time in libavutil/parseutils.c that converts the timestr to microseconds since 1st of January 1970. ff_parse_creation_time_metadata then converts microseconds back to seconds and sets the timestamp as seconds. To allow for dates before 1970 as defined in ISO/IEC 14496-12, either ff_parse_creation_time_metadata would have to parse the submitted timestamp itself and convert it to seconds since midnight, Jan. 1, 1904, in UTC time, or instruct av_parse_time to do so (e.g. by submitting a special value to av_parse_time's parameter duration).

In any case, it would be more gracious if ffmpeg either rejects a timestr outside the accepted date range or displays a warning message. In the example ffmpeg -i input.mp4 -map_metadata 0 -metadata creation_time="1965-01-01 12:00:00" -codec copy output.mp4, though the date is correctly displayed during the copy process, afterwards ffprobe yields a future date (in the example, ffprobe output.mp4 displays creation_time as 2036-01-01T23:59:59.000000Z).

Kind regards,

Bernd
_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] [PATCH] avformat/mov: remove hack breaking creation time parsing

[Marton Balint]

On Tue, 11 Apr 2023, Bernd Dürrer wrote:

> On Sun Apr 9 20:20:04 EEST 2023, Michael Niedermayer wrote: 
>
>> what do you mean by intended creation time?
>> the file format did not exist in 1946. and all the codecs also didnt exist
>> so when you encounter a file that says its from that time it must be crafted
>> later and backdated or that bug.
>
> As defined in ISO/IEC 14496-12, clause 8.2, "creation_time is an integer 
> that declares the creation time of the presentation (in seconds since 
> midnight, Jan. 1, 1904, in UTC time)". This is the date when the content 
> was created (which may have been on photographic film), and not when it 
> was encoded as a digital file.

I don't think that is true, it is not clear from the specs at best. The 
creation time of the presentation can just as easily refer to the file 
itself, not the content. Feel free to reencode something with a commercial 
encoder, I would be suprised if creation time was kept.

I think the proper metadata for the creation time in the sense you want it 
to be is com.apple.quicktime.creationdate, here is an example with 
different moov creation time and com.apple.quicktime.creationdate.

https://samples.ffmpeg.org/ffmpeg-bugs/trac/ticket5120/incorrect-start-frame.mov

Maybe you can check if the DLNA server reads that. Or if it only reads the 
year information, then com.apple.quicktime.year might be even better.

https://developer.apple.com/library/archive/documentation/QuickTime/QTFF/Metadata/Metadata.html

Regards,
Marton
_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] [PATCH] avformat/mov: remove hack breaking creation time parsing

[Bernd Dürrer]

On Tue, 11 Apr 2023, Marton Balint wrote:
 
>> As defined in ISO/IEC 14496-12, clause 8.2, "creation_time is an integer
>> that declares the creation time of the presentation (in seconds since
>> midnight, Jan. 1, 1904, in UTC time)". This is the date when the content
>> was created (which may have been on photographic film), and not when it
>> was encoded as a digital file.
>
> I don't think that is true, it is not clear from the specs at best. The
> creation time of the presentation can just as easily refer to the file
> itself, not the content. Feel free to reencode something with a commercial
> encoder, I would be suprised if creation time was kept.

I agree that it may be debated for what purpose the creation_time tag has been intended by the authors of the standard, and that this tag may be used inconsistently by different encoders. However, it remains a fact that ffmpeg does not support the date range defined in the standard, and that ffmpeg writes an incorrect date to the file if a date before 1970 is provided. This is a bug that should be amended.

Kind regards,

Bernd
_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] [PATCH] avformat/mov: remove hack breaking creation time parsing

["zhilizhao(赵志立)"]

> On Apr 12, 2023, at 16:10, Bernd Dürrer <Bernd.Duerrer@gmx.de> wrote:
> 
> On Tue, 11 Apr 2023, Marton Balint wrote:
>  
>>> As defined in ISO/IEC 14496-12, clause 8.2, "creation_time is an integer
>>> that declares the creation time of the presentation (in seconds since
>>> midnight, Jan. 1, 1904, in UTC time)". This is the date when the content
>>> was created (which may have been on photographic film), and not when it
>>> was encoded as a digital file.
>> 
>> I don't think that is true, it is not clear from the specs at best. The
>> creation time of the presentation can just as easily refer to the file
>> itself, not the content. Feel free to reencode something with a commercial
>> encoder, I would be suprised if creation time was kept.
> 
> I agree that it may be debated for what purpose the creation_time tag has been intended by the authors of the standard, and that this tag may be used inconsistently by different encoders. However, it remains a fact that ffmpeg does not support the date range defined in the standard, and that ffmpeg writes an incorrect date to the file if a date before 1970 is provided. This is a bug that should be amended.

FFmpeg muxer has no problem outputs mp4 with date before 1970.
You can check it with mediainfo on the output file. It’s the
demuxer try to workaround bugs of thirdparty mp4 muxers which
use 1970 as the start point of creation_time. So for date before
1970, it will become date + 1970 - 1904.

The ISO spec isn’t clear on creation_time, quicktime spec hints
it’s the creation time of the file. I’m also agree it’s a reasonable
usecase for users to tweak it as the time when a movie was filmed.

Creation time
A 32-bit integer that specifies the calendar date and time (in seconds since midnight, January 1, 1904)
when the movie atom was created. It is strongly recommended that this value should be specified using
coordinated universal time (UTC).

> 
> Kind regards,
> 
> Bernd
> _______________________________________________
> ffmpeg-devel mailing list
> ffmpeg-devel@ffmpeg.org
> https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
> 
> To unsubscribe, visit link above, or email
> ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".

_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] [PATCH] avformat/mov: remove hack breaking creation time parsing

[Bernd Dürrer]

On Tue Apr 11 2023, Marton Balint wrote:

> Maybe you can check if the DLNA server reads that. Or if it only reads the
> year information, then com.apple.quicktime.year might be even better.

I have checked the DLNA Guidelines and Part 1-1, clause 10.1.3.12.3, Table 13, recommends to provide the dc:date metadata property for
video items. I have MP4 files where the date metadata was set, but not to a complete date, but only the year part. This appears to be
the reason why my DLNA server has used the creation_time metadata instead. After setting the date tag to a complete date YYYY-MM-DD HH:MM:SS,
it is recognised by the DLNA server and provided to the DLNA clients, and there is no problem to set the date tag to dates before 1970 with
ffmpeg. So my use case to set the creation_time tag to dates before 1970 is now obsolete.

Nevertheless, I looked into ffmpeg why the command

ffmpeg -i input.mp4 -map_metadata 0 -metadata creation_time="1965-01-01 12:00:00" -codec copy output.mp4

results in creation_time written as 2036-01-01T23:59:59.000000Z to the output file. The reason is at the end of function av_parse_time
in libavutil/parseutils.c where mktime is called to convert the dt structure. As mktime cannot represent calendar times before 1970,
it returns -1. However, as the result is not checked, the return value of -1 is used for all subsequent processing and results in the wrong
date written to the file. In my humble opinion, at least a warning should be raised that the date value provided as parameter on the command
line is out of range.

Thanks for your support and kind regards,

Bernd
_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".