Re: [FFmpeg-devel] [RFC] Introducing policies regarding "AI" contributions

["softworkz ." <softworkz-at-hotmail.com@ffmpeg.org>]

> -----Original Message-----
> From: ffmpeg-devel <ffmpeg-devel-bounces@ffmpeg.org> On Behalf Of
> Kacper Michajlow
> Sent: Dienstag, 1. Juli 2025 14:44
> To: FFmpeg development discussions and patches <ffmpeg-
> devel@ffmpeg.org>
> Cc: Alexander Strasser <eclipse7@gmx.net>
> Subject: Re: [FFmpeg-devel] [RFC] Introducing policies regarding "AI"
> contributions
> 
> On Tue, 1 Jul 2025 at 12:58, Alexander Strasser via ffmpeg-devel
> <ffmpeg-devel@ffmpeg.org> wrote:
> >
> >
> >
> >
> > ---------- Forwarded message ----------
> > From: Alexander Strasser <eclipse7@gmx.net>
> > To: ffmpeg-devel@ffmpeg.org
> > Cc:
> > Bcc:
> > Date: Tue, 1 Jul 2025 12:58:23 +0200
> > Subject: [RFC] Introducing policies regarding "AI" contributions
> > Hi all,
> >
> > I do not like the branding of the LLMs as AI, thus I will for now
> > continue to call it "AI" in quotes. I'm open for better terms.
> >
> > It was just yesterday brought up on IRC in #ffmpeg-devel that there
> > was at least one, marked attempt to include "AI" generated code[1].
> >
> > At least I would say that this particular patch series was rejected,
> > but there were was no explicit discussion and clear statement about
> > "AI" generated content; especially code.
> >
> > Thus I want this thread to start a discussion, that eventually leads
> > to a policy about submitting and integrating "AI" generated content.
> 
> I don't think labeling code as "AI" matters that much. Let's ignore
> licensing/legal issues for now.
> 
> What's important is the code itself and its quality. It doesn't matter
> how it was created. Whether by a human, "AI" or something else. The
> key is the final product. "AI" is just a tool, and like any tool, it
> can be used well or poorly. How you use it may be completely different
> between "operators".
> 
> I think the "AI" label exists because the code that LLMs produce is
> often incomplete, low quality, and a pile of spaghetti that somehow
> works for a single use case. but is far from being a sane, production
> ready implementation. Anyone who has used these tools knows their
> limitations and what they can or cannot do.
> 
> That said, if "AI" code means low quality code, then by all means, it
> should be rejected. This applies to human, alien, or "AI" generated
> code. There shouldn't be a different metric for "AI" code. If "AI"
> (and its "operator") produces high quality code, there's no reason to
> reject it.


I see it in a similar way. Things are changing so incredibly fast, that
there's little sense in establishing a policy that is based on the 
assumption that generated code is of low quality, because it might be 
outdated even before it might have been agreed about. 
It's tough to judge anyway, because these models do not generate "low
quality" code of that kind that less experienced human developers may
do. It's rather an insane mix of good quality code mixed up with 
insane flaws, oversights and shortcomings. The mix strongly depends 
on the topic and also language. Yesterday I had let it generate a 
bunch of Bash scripts with options interface for managing certain 
cloud resources. I had to supervise closely, but it tested and fixed
its mistakes itself (after me pointing at), generated useful 
documentation - wow! It would have taken me a multiple of the time.
But in many other cases where I tried, I ended up spending more time
than doing it alone right away. For specific FFmpeg work I haven't
found it useful for anything so far but reviewing changes. Probably
this will change at some time, though.

Even though the script creation was impressive, it doesn't mean that
you can close your eyes and be good. You still need to see and review
and evaluate every single line that is generated.

Which brings me to Gyan's comment:

> At best, we can require disclosure and for the human submitter to
> assume
> responsibility.

IMO this is THE one point that would make a reasonable policy which
is valid independently from any "AI" progress now and in the future:

When someone submits code, we can require that a submitter not only
formally takes responsibility, but we can also expect that there's 
a close understanding of every single line of code that is being 
submitted.
If that isn't given (like in case where it was stated like "I 
don't know what it does, but it's working"), then that should be
a clear reason for rejection.


Best regards,

softworkz

PS: For the ML (or any future communication method), I think 
a simple policy should be in place like that "AI" generated 
messages must be marked as such.





_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".