Re: [FFmpeg-devel] [PATCH] fate: Allow running multiple rounds of tests with differing settings

From: "Martin Storsjö" <martin@martin.st>
To: FFmpeg development discussions and patches <ffmpeg-devel@ffmpeg.org>
Subject: Re: [FFmpeg-devel] [PATCH] fate: Allow running multiple rounds of tests with differing settings
Date: Sun, 17 Dec 2023 00:02:56 +0200 (EET)
Message-ID: <d611b42-f457-d8b9-fabc-49c87f56fb1f@martin.st> (raw)
In-Reply-To: <e78dee50-c816-25b7-cb7f-95de745bc4e1@martin.st>

On Wed, 13 Dec 2023, Martin Storsjö wrote:

> On Wed, 13 Dec 2023, Rémi Denis-Courmont wrote:
>
>> Le tiistaina 12. joulukuuta 2023, 0.14.06 EET Martin Storsjö a écrit :
>>> This can be used to run tests multple times, with e.g. differing
>>> QEMU settings, by adding something like this to the FATE configuration
>>> file:
>>>
>>>     target_exec="qemu-aarch64-static"
>>>     fate_targets="fate-checkasm fate-cpu"
>>>
>>>     fate_environments="sve128 sve256 sve512"
>>>     sve128_env="QEMU_CPU=max,sve128=on"
>>>     sve256_env="QEMU_CPU=max,sve256=on"
>>>     sve512_env="QEMU_CPU=max,sve512=on"
>> 
>> I'm fine with that, but for the sake of generality, shouldn't rather the 
>> entire target_exec prefix be indirected? Some runners may want to use 
>> command line flags rather than environment variables.
>
> Yes - that's also doable. One can e.g. do this:
>
> ---
> target_exec="qemu-aarch64-static -cpu \$(MY_CPU)"
> fate_targets="fate-checkasm fate-cpu"
>
> fate_environments="sve128 sve256 sve512"
> sve128_env="MY_CPU=max,sve128=on"
> sve256_env="MY_CPU=max,sve256=on"
> sve512_env="MY_CPU=max,sve512=on"
> ---
>
> That way, one can also make the whole target_exec be e.g. \$(EXEC_CMD) and 
> set the full command via the individual envs. It's not quite as comfortable, 
> but should be doable and allow fully generic setups.

If this sounds reasonable enough, I would go ahead and land this, together 
with 
https://patchwork.ffmpeg.org/project/ffmpeg/patch/20231127123118.3622784-1-martin@martin.st/.

I guess this feature should have some documentation as well - I can draft 
up a v2 with docs before going ahead with it, if the feature overall looks 
acceptable.

// Martin
_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".