Re: [FFmpeg-devel] CI

[Michael Niedermayer via ffmpeg-devel <ffmpeg-devel@ffmpeg.org>]

[-- Attachment #1.1: Type: text/plain, Size: 6161 bytes --]

On Fri, Aug 22, 2025 at 02:13:14PM +0200, Timo Rothenpieler via ffmpeg-devel wrote:
> On 22/08/2025 07:00, Kieran Kunhya via ffmpeg-devel wrote:
> > On Thu, 21 Aug 2025, 11:33 Michael Niedermayer via ffmpeg-devel, <
> > ffmpeg-devel@ffmpeg.org> wrote:
> > 
> > > Hi
> > > 
> > > On Thu, Aug 21, 2025 at 12:31:46AM +0200, Timo Rothenpieler via
> > > ffmpeg-devel wrote:
> > > > On 8/20/2025 9:25 PM, Michael Niedermayer via ffmpeg-devel wrote:
> > > > > Hi
> > > > > 
> > > > > On Wed, Aug 20, 2025 at 05:56:27PM +0200, Timo Rothenpieler via
> > > ffmpeg-devel wrote:
> > > > > > On 8/20/2025 1:26 AM, Michael Niedermayer via ffmpeg-devel wrote:
> > > > > > > Hi
> > > > > > > 
> > > > > > > It seems the forgejo CI takes about
> > > > > > > 13min to do fate on aarch64 and x86-64 and build on win64
> > > > > > > 
> > > > > > > Locally i run
> > > > > > >        fate + install on x86-64
> > > > > > >        build on x86-32, mingw64, arm32, mips, ppc, x86-64 + shared
> > > libs
> > > > > > >        testprogs alltools examples build on x86-64, x86-32 and arm32
> > > > > > >        in 2min 44sec
> > > > > > > 
> > > > > > > can we improve the speed vs amount of tests ratio ?
> > > > > > > (its not a problem ATM, i did in fact not even notice as i never
> > > waited on it)
> > > > > > > 
> > > > > > > Iam just seeing the difference in time and i think there is
> > > potential for
> > > > > > > optimization here
> > > > > > > 
> > > > > > > I dont think my box here is really special, just a
> > > > > > > AMD Ryzen 9 3950X 16-Core + Samsung SSD 970 PRO
> > > > > > 
> > > > > > Well, the test runners are 4 cores and 8GB of RAM. So that'll be the
> > > primary
> > > > > > difference in speed.
> > > > > > I think they're performing pretty good for being just that.
> > > > > > 
> > > > > > We could of course throw money at the problem and turn them into 16
> > > core
> > > > > > machines. That would up the hosting cost of the runners from
> > > currently
> > > > > > 3*7.5€ a month to 3*30€ a month. Just for the runners.
> > > > > > 
> > > > > > imo the current CI turnaround times are fine. 15-20 minutes per job
> > > is fine,
> > > > > > as long as they can all run in parallel.
> > > > > 
> > > > > Option 1: 15-20 min CI turnaround,  270 € per year
> > > > > Option 2:  4-5? min CI turnaround, 1080 € per year
> > > > > 
> > > > > we have over 150k $ it seems
> > > > > 
> > > > > Good use of capital can also lead to more donations
> > > > > 
> > > > > I think the main question is, "would we benefit from the faster
> > > trunaround"?
> > > > > or not ?
> > > > 
> > > > You have to keep in mind, 4 Core 8GB is also the swarm of runners we get
> > > for
> > > > free from Microsoft via GitHub.
> > > > 
> > > > So the choice is actually "Be able to process 20+ jobs in parallel that
> > > take
> > > > 15-20 minutes each" vs. "Be able to process 3 or so at a time (roughly
> > > one
> > > > PR/push) in 5 minutes".
> > > > So realistically, unless we also pay for an actual swarm of runners
> > > > ourselves(which would cost 10k or more a year while being idle 95% of the
> > > > time) the total turnaround time including wait for a free runner is
> > > probably
> > > > still better with more of the smaller runners than less of the big ones.
> > > > 
> > > > It'd also make it a lot more pressing to think about every single CI job
> > > we
> > > > add, vs. having a bit of leeway due to the over-abundance of runners.
> > > 
> > > for 1-2k$ you can buy a box that runs fate once and build on 6 times in
> > > under 3minutes.
> > > 
> > > if one is not enough buy 3, use the extra capcity for fuzzing or rent out
> > > to other projects
> > > 
> > > I must be stupid, because to me this looks cheaper, its also one time
> > > expense
> > > these boxes can be used for 10 years
> > > 
> > > also no need to be reliable expensive servers, if you have 3.
> > > 
> > > thx
> > > 
> > 
> > In the end you're paying for hosting, 24/7 electricity and not having to
> > worry about it.
> 
> Yeah, in the end someone has to put these 3 servers somewhere, monitor them
> for hardware failures, and pay for their electricity.
> Which isn't exactly cheap for 3 servers either.
> 
> > I agree with both viewpoints. For "mission critical" stuff like CI we
> > should host at a proper hosting company.
> 
> There's also the option of renting a dedicated server from Hetzner or some
> other company, and put a bunch of runners on there.
> Hetzner rents you an 8 core Zen4 Ryzen with 64GB of RAM for ~55€ a month.
> 
> But compared to the current virtual servers we rent from Hetzner, that's a
> pretty poor "performance per money" ratio, given we get one server with 4
> cores and 8GB of RAM, which is plenty for our needs, for 7.5€ a month.
> So we can rent up to 7 of those, while still paying less than for one
> dedicated server.
> And dedicated servers are again more effort, because you do need to manually
> monitor them for any hardware faults, and then coordinate the replacement
> with Hetzner, and they do eventually die of old age.
> 
> There's also the option of writing some tool that automatically scales
> runners up/down at some cloud provider that bills minutely/secondly (Hetzner
> bills Servers per Hour, making this a lot less interesting).
> That would require some serious cost calculation. But I feel like we have
> plenty of CI downtime to make it worthwhile.
> 
> But then again, GitHub/MS gives us 20 parallel runners for free, and we can
> freely pick if they're running on x86_64 or aarch64, Linux, Windows or even
> OSX.
> As long as they do that, we only need to host a baseline of runners
> ourselves, and can scale out into that whenever there's a peak in usage.

I guess next time i upgrade my box, i might just keep the old one running in a
corner.

thx

[...]
-- 
Michael     GnuPG fingerprint: 9FF2128B147EF6730BADF133611EC787040B0FAB

"Nothing to hide" only works if the folks in power share the values of
you and everyone you know entirely and always will -- Tom Scott


[-- Attachment #1.2: signature.asc --]
[-- Type: application/pgp-signature, Size: 195 bytes --]

[-- Attachment #2: Type: text/plain, Size: 251 bytes --]

_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".