[FFmpeg-devel] [FATE PATCH] Properly list test errors when there are no failed tests

[FFmpeg-devel] [FATE PATCH] Properly list test errors when there are no failed tests

[Martin Storsjö]

If there were failures while running tests, e.g. if failing to
compile checkasm or any other of the test programs, there are no
failed tests per se, and the number of succeessful tests is
equal to the total number of tests.

For these cases, check the job status code instead of declaring
them as a full success.
---
 history.cgi | 2 +-
 index.cgi   | 2 +-
 report.cgi  | 6 +++++-
 3 files changed, 7 insertions(+), 3 deletions(-)

diff --git a/history.cgi b/history.cgi
index 533999c..d0a855d 100755
--- a/history.cgi
+++ b/history.cgi
@@ -76,7 +76,7 @@ for my $date ((sort { $b cmp $a } @reps)[0..49]) {
     td $$rep{nwarn};
     if ($npass) {
         $rtext  = "$npass / $ntest";
-        $rclass = $npass==$ntest? 'pass' : $npass? 'warn' : 'fail';
+        $rclass = $$rep{status}==0? 'pass' : $npass? 'warn' : 'fail';
     } elsif (!$ntest and !$$rep{status}) {
         $rtext  = "build only";
         $rclass = $$rep{status}? 'fail' : 'pass';
diff --git a/index.cgi b/index.cgi
index 25c61e9..89e599a 100755
--- a/index.cgi
+++ b/index.cgi
@@ -286,7 +286,7 @@ for my $rep (sort repcmp @reps) {
     td $$rep{comment}, class => 'comment';
     if ($npass) {
         $rtext  = "$npass / $ntest";
-        $rclass = $npass==$ntest? 'pass' : $npass? 'warn' : 'fail';
+        $rclass = $$rep{status}==0? 'pass' : $npass? 'warn' : 'fail';
     } elsif (!$ntest and !$$rep{status}) {
         $rtext  = "build only";
         $rclass = $$rep{status}? 'fail' : 'pass';
diff --git a/report.cgi b/report.cgi
index 5a2394d..ca003b1 100755
--- a/report.cgi
+++ b/report.cgi
@@ -183,7 +183,11 @@ if ($nfail) {
     }
     end 'tbody';
 } elsif ($ntest) {
-    start 'tr', class => 'pass'; th 'All tests successful', colspan => 3; end;
+    if ($$hdr{status}) {
+        start 'tr', class => 'fail'; th 'Failure building tests', colspan => 3; end;
+    } else {
+        start 'tr', class => 'pass'; th 'All tests successful', colspan => 3; end;
+    }
 } else {
     my $class = $$hdr{status}? 'fail' : 'pass';
     start 'tr', class => $class; th 'No tests were run',    colspan => 3; end;
-- 
2.43.0

_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] [FATE PATCH] Properly list test errors when there are no failed tests

[Michael Niedermayer]

[-- Attachment #1.1: Type: text/plain, Size: 1134 bytes --]

Hi Martin

On Fri, Jun 13, 2025 at 01:57:05PM +0300, Martin Storsjö wrote:
> If there were failures while running tests, e.g. if failing to
> compile checkasm or any other of the test programs, there are no
> failed tests per se, and the number of succeessful tests is
> equal to the total number of tests.
> 
> For these cases, check the job status code instead of declaring
> them as a full success.
> ---
>  history.cgi | 2 +-
>  index.cgi   | 2 +-
>  report.cgi  | 6 +++++-
>  3 files changed, 7 insertions(+), 3 deletions(-)

sounds reasonable, but ill leave the perl review to someone who actually
knows perl

Its a few years since the last change to fateserver, i think we dont
have a script to update the checkout on the server from a git push.
Its a while, i dont remember exactly :)
so you have to update that checkout after pushing possibly

thx

[...]

-- 
Michael     GnuPG fingerprint: 9FF2128B147EF6730BADF133611EC787040B0FAB

Take away the freedom of one citizen and you will be jailed, take away
the freedom of all citizens and you will be congratulated by your peers
in Parliament.

[-- Attachment #1.2: signature.asc --]
[-- Type: application/pgp-signature, Size: 195 bytes --]

[-- Attachment #2: Type: text/plain, Size: 251 bytes --]

_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".