[FFmpeg-devel] [FATE PATCH] Properly list test errors when there are no failed tests

[FFmpeg-devel] [FATE PATCH] Properly list test errors when there are no failed tests

[Martin Storsjö]

If there were failures while running tests, e.g. if failing to
compile checkasm or any other of the test programs, there are no
failed tests per se, and the number of succeessful tests is
equal to the total number of tests.

For these cases, check the job status code instead of declaring
them as a full success.
---
 history.cgi | 2 +-
 index.cgi   | 2 +-
 report.cgi  | 6 +++++-
 3 files changed, 7 insertions(+), 3 deletions(-)

diff --git a/history.cgi b/history.cgi
index 533999c..d0a855d 100755
--- a/history.cgi
+++ b/history.cgi
@@ -76,7 +76,7 @@ for my $date ((sort { $b cmp $a } @reps)[0..49]) {
     td $$rep{nwarn};
     if ($npass) {
         $rtext  = "$npass / $ntest";
-        $rclass = $npass==$ntest? 'pass' : $npass? 'warn' : 'fail';
+        $rclass = $$rep{status}==0? 'pass' : $npass? 'warn' : 'fail';
     } elsif (!$ntest and !$$rep{status}) {
         $rtext  = "build only";
         $rclass = $$rep{status}? 'fail' : 'pass';
diff --git a/index.cgi b/index.cgi
index 25c61e9..89e599a 100755
--- a/index.cgi
+++ b/index.cgi
@@ -286,7 +286,7 @@ for my $rep (sort repcmp @reps) {
     td $$rep{comment}, class => 'comment';
     if ($npass) {
         $rtext  = "$npass / $ntest";
-        $rclass = $npass==$ntest? 'pass' : $npass? 'warn' : 'fail';
+        $rclass = $$rep{status}==0? 'pass' : $npass? 'warn' : 'fail';
     } elsif (!$ntest and !$$rep{status}) {
         $rtext  = "build only";
         $rclass = $$rep{status}? 'fail' : 'pass';
diff --git a/report.cgi b/report.cgi
index 5a2394d..ca003b1 100755
--- a/report.cgi
+++ b/report.cgi
@@ -183,7 +183,11 @@ if ($nfail) {
     }
     end 'tbody';
 } elsif ($ntest) {
-    start 'tr', class => 'pass'; th 'All tests successful', colspan => 3; end;
+    if ($$hdr{status}) {
+        start 'tr', class => 'fail'; th 'Failure building tests', colspan => 3; end;
+    } else {
+        start 'tr', class => 'pass'; th 'All tests successful', colspan => 3; end;
+    }
 } else {
     my $class = $$hdr{status}? 'fail' : 'pass';
     start 'tr', class => $class; th 'No tests were run',    colspan => 3; end;
-- 
2.43.0

_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] [FATE PATCH] Properly list test errors when there are no failed tests

[Michael Niedermayer]

[-- Attachment #1.1: Type: text/plain, Size: 1134 bytes --]

Hi Martin

On Fri, Jun 13, 2025 at 01:57:05PM +0300, Martin Storsjö wrote:
> If there were failures while running tests, e.g. if failing to
> compile checkasm or any other of the test programs, there are no
> failed tests per se, and the number of succeessful tests is
> equal to the total number of tests.
> 
> For these cases, check the job status code instead of declaring
> them as a full success.
> ---
>  history.cgi | 2 +-
>  index.cgi   | 2 +-
>  report.cgi  | 6 +++++-
>  3 files changed, 7 insertions(+), 3 deletions(-)

sounds reasonable, but ill leave the perl review to someone who actually
knows perl

Its a few years since the last change to fateserver, i think we dont
have a script to update the checkout on the server from a git push.
Its a while, i dont remember exactly :)
so you have to update that checkout after pushing possibly

thx

[...]

-- 
Michael     GnuPG fingerprint: 9FF2128B147EF6730BADF133611EC787040B0FAB

Take away the freedom of one citizen and you will be jailed, take away
the freedom of all citizens and you will be congratulated by your peers
in Parliament.

[-- Attachment #1.2: signature.asc --]
[-- Type: application/pgp-signature, Size: 195 bytes --]

[-- Attachment #2: Type: text/plain, Size: 251 bytes --]

_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] [FATE PATCH] Properly list test errors when there are no failed tests

[Martin Storsjö]

On Wed, 18 Jun 2025, Michael Niedermayer wrote:

> Hi Martin
>
> On Fri, Jun 13, 2025 at 01:57:05PM +0300, Martin Storsjö wrote:
>> If there were failures while running tests, e.g. if failing to
>> compile checkasm or any other of the test programs, there are no
>> failed tests per se, and the number of succeessful tests is
>> equal to the total number of tests.
>>
>> For these cases, check the job status code instead of declaring
>> them as a full success.
>> ---
>>  history.cgi | 2 +-
>>  index.cgi   | 2 +-
>>  report.cgi  | 6 +++++-
>>  3 files changed, 7 insertions(+), 3 deletions(-)
>
> sounds reasonable, but ill leave the perl review to someone who actually
> knows perl

I've tested it locally, so if we have nobody else volunteering to review 
it, is that enough?

> Its a few years since the last change to fateserver, i think we dont
> have a script to update the checkout on the server from a git push.
> Its a while, i dont remember exactly :)
> so you have to update that checkout after pushing possibly

I don't think I have access to any such server, but if it's ok with you 
can I (try to) push this patch to the gitosis repo, and then ask some 
admin (Timo?) to possibly update a checkout somewhere.

// Martin
_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] [FATE PATCH] Properly list test errors when there are no failed tests

[Kacper Michajlow]

On Fri, Jun 27, 2025, 14:44 Martin Storsjö <martin@martin.st> wrote:

> On Wed, 18 Jun 2025, Michael Niedermayer wrote:
>
> > Hi Martin
> >
> > On Fri, Jun 13, 2025 at 01:57:05PM +0300, Martin Storsjö wrote:
> >> If there were failures while running tests, e.g. if failing to
> >> compile checkasm or any other of the test programs, there are no
> >> failed tests per se, and the number of succeessful tests is
> >> equal to the total number of tests.
> >>
> >> For these cases, check the job status code instead of declaring
> >> them as a full success.
> >> ---
> >>  history.cgi | 2 +-
> >>  index.cgi   | 2 +-
> >>  report.cgi  | 6 +++++-
> >>  3 files changed, 7 insertions(+), 3 deletions(-)
> >
> > sounds reasonable, but ill leave the perl review to someone who actually
> > knows perl
>
> I've tested it locally, so if we have nobody else volunteering to review
> it, is that enough?
>
> > Its a few years since the last change to fateserver, i think we dont
> > have a script to update the checkout on the server from a git push.
> > Its a while, i dont remember exactly :)
> > so you have to update that checkout after pushing possibly
>
> I don't think I have access to any such server, but if it's ok with you
> can I (try to) push this patch to the gitosis repo, and then ask some
> admin (Timo?) to possibly update a checkout somewhere.
>
> // Martin
>

There is also https://fatebeta.ffmpeg.org/, should it be updated for parity
with perl version?

- Kacper

>
_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] [FATE PATCH] Properly list test errors when there are no failed tests

[Martin Storsjö]

On Fri, 27 Jun 2025, Kacper Michajlow wrote:

> On Fri, Jun 27, 2025, 14:44 Martin Storsjö <martin@martin.st> wrote:
>
>> On Wed, 18 Jun 2025, Michael Niedermayer wrote:
>>
>> > Hi Martin
>> >
>> > On Fri, Jun 13, 2025 at 01:57:05PM +0300, Martin Storsjö wrote:
>> >> If there were failures while running tests, e.g. if failing to
>> >> compile checkasm or any other of the test programs, there are no
>> >> failed tests per se, and the number of succeessful tests is
>> >> equal to the total number of tests.
>> >>
>> >> For these cases, check the job status code instead of declaring
>> >> them as a full success.
>> >> ---
>> >>  history.cgi | 2 +-
>> >>  index.cgi   | 2 +-
>> >>  report.cgi  | 6 +++++-
>> >>  3 files changed, 7 insertions(+), 3 deletions(-)
>> >
>> > sounds reasonable, but ill leave the perl review to someone who actually
>> > knows perl
>>
>> I've tested it locally, so if we have nobody else volunteering to review
>> it, is that enough?
>>
>> > Its a few years since the last change to fateserver, i think we dont
>> > have a script to update the checkout on the server from a git push.
>> > Its a while, i dont remember exactly :)
>> > so you have to update that checkout after pushing possibly
>>
>> I don't think I have access to any such server, but if it's ok with you
>> can I (try to) push this patch to the gitosis repo, and then ask some
>> admin (Timo?) to possibly update a checkout somewhere.
>>
>> // Martin
>>
>
> There is also https://fatebeta.ffmpeg.org/, should it be updated for parity
> with perl version?

AFAIK that site prints this aspect correctly.

// Martin
_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] [FATE PATCH] Properly list test errors when there are no failed tests

[Michael Niedermayer]

[-- Attachment #1.1: Type: text/plain, Size: 1811 bytes --]

Hi Martin

On Fri, Jun 27, 2025 at 03:43:58PM +0300, Martin Storsjö wrote:
> On Wed, 18 Jun 2025, Michael Niedermayer wrote:
> 
> > Hi Martin
> > 
> > On Fri, Jun 13, 2025 at 01:57:05PM +0300, Martin Storsjö wrote:
> > > If there were failures while running tests, e.g. if failing to
> > > compile checkasm or any other of the test programs, there are no
> > > failed tests per se, and the number of succeessful tests is
> > > equal to the total number of tests.
> > > 
> > > For these cases, check the job status code instead of declaring
> > > them as a full success.
> > > ---
> > >  history.cgi | 2 +-
> > >  index.cgi   | 2 +-
> > >  report.cgi  | 6 +++++-
> > >  3 files changed, 7 insertions(+), 3 deletions(-)
> > 
> > sounds reasonable, but ill leave the perl review to someone who actually
> > knows perl
> 
> I've tested it locally, so if we have nobody else volunteering to review it,
> is that enough?

yes


> 
> > Its a few years since the last change to fateserver, i think we dont
> > have a script to update the checkout on the server from a git push.
> > Its a while, i dont remember exactly :)
> > so you have to update that checkout after pushing possibly
> 
> I don't think I have access to any such server,

I think you do, i see a ssh-rsa key for martin@sphere for
fate@fate.ffmpeg.org


> but if it's ok with you can
> I (try to) push this patch to the gitosis repo,

yes, we use gitolite not gitosis though


> and then ask some admin
> (Timo?) to possibly update a checkout somewhere.

sure

thx

[...]

-- 
Michael     GnuPG fingerprint: 9FF2128B147EF6730BADF133611EC787040B0FAB

Its not that you shouldnt use gotos but rather that you should write
readable code and code with gotos often but not always is less readable

[-- Attachment #1.2: signature.asc --]
[-- Type: application/pgp-signature, Size: 195 bytes --]

[-- Attachment #2: Type: text/plain, Size: 251 bytes --]

_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] [FATE PATCH] Properly list test errors when there are no failed tests

[Martin Storsjö]

On Sat, 28 Jun 2025, Michael Niedermayer wrote:

>>> Its a few years since the last change to fateserver, i think we dont
>>> have a script to update the checkout on the server from a git push.
>>> Its a while, i dont remember exactly :)
>>> so you have to update that checkout after pushing possibly
>>
>> I don't think I have access to any such server,
>
> I think you do, i see a ssh-rsa key for martin@sphere for
> fate@fate.ffmpeg.org
>
>> but if it's ok with you can
>> I (try to) push this patch to the gitosis repo,
>
> yes, we use gitolite not gitosis though

Ok - I've pushed the patch to git@git.ffmpeg.org:fateserver.git now.

It's not obvious if the change is deployed in practice or not, until we 
have another case of breakage where compiling some test binaries fail - so 
we'll need someone to check.

// Martin

_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".