[FFmpeg-devel] [RFC] Introducing policies regarding "AI" contributions

[FFmpeg-devel] [RFC] Introducing policies regarding "AI" contributions

[Alexander Strasser via ffmpeg-devel]

[-- Attachment #1: Type: message/rfc822, Size: 7221 bytes --]

From: Alexander Strasser <eclipse7@gmx.net>
To: ffmpeg-devel@ffmpeg.org
Subject: [RFC] Introducing policies regarding "AI" contributions
Date: Tue, 1 Jul 2025 12:58:23 +0200
Message-ID: <aGO_T_HfuvhbgYoy@metallschleim.local>

Hi all,

I do not like the branding of the LLMs as AI, thus I will for now
continue to call it "AI" in quotes. I'm open for better terms.

It was just yesterday brought up on IRC in #ffmpeg-devel that there
was at least one, marked attempt to include "AI" generated code[1].

At least I would say that this particular patch series was rejected,
but there were was no explicit discussion and clear statement about
"AI" generated content; especially code.

Thus I want this thread to start a discussion, that eventually leads
to a policy about submitting and integrating "AI" generated content.

Leaving all ethical issues aside for a moment I still see 2 very big
problems with AI generated code:

* looks generally plausible but is often subtly wrong
    * leading to more work, regressions and costs
        * which often lands on a different group of people (other
          projects, reviewers, bug finders, bug fixers, etc.)
        * which are sometimes delayed for quite some time increasing
          the costs of fixing them
* license/copyright violations
    * this might be sometimes a non-issue with small changes
    * but especially for complete components the risk seems high

There is a lot more to the topic and I probably forgot to bring up
many more important aspects and details. Please feel free to bring
more things up in the discussion!

There was a preparation in the musl project to put up a policy[2],
it has not yet been finalized and realized as far as I understand.

It also brings up the point, that it is not really related to
recent "AI" tech, but more to the origin of work and its handling.
Unfortunately "AI" made problems with this a lot more common.


Best regards,
  Alexander

1. https://lists.ffmpeg.org/pipermail/ffmpeg-devel/2025-April/342146.html
2. https://www.openwall.com/lists/musl/2024/10/19/3

[-- Attachment #2: Type: text/plain, Size: 251 bytes --]

_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] [RFC] Introducing policies regarding "AI" contributions

[Gyan Doshi]

On 2025-07-01 04:28 pm, Alexander Strasser via ffmpeg-devel wrote:

 > Thus I want this thread to start a discussion, that eventually leads 
 > to a policy about submitting and integrating "AI" generated content.
In practice. unless a patch(set) is explicitly marked or has telltale 
signs of being AI-generated,the project can't stop such AI code getting in.
At best, we can require disclosure and for the human submitter to assume 
responsibility.

Regards,
Gyan
_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] [RFC] Introducing policies regarding "AI" contributions

[Kacper Michajlow]

On Tue, 1 Jul 2025 at 12:58, Alexander Strasser via ffmpeg-devel
<ffmpeg-devel@ffmpeg.org> wrote:
>
>
>
>
> ---------- Forwarded message ----------
> From: Alexander Strasser <eclipse7@gmx.net>
> To: ffmpeg-devel@ffmpeg.org
> Cc:
> Bcc:
> Date: Tue, 1 Jul 2025 12:58:23 +0200
> Subject: [RFC] Introducing policies regarding "AI" contributions
> Hi all,
>
> I do not like the branding of the LLMs as AI, thus I will for now
> continue to call it "AI" in quotes. I'm open for better terms.
>
> It was just yesterday brought up on IRC in #ffmpeg-devel that there
> was at least one, marked attempt to include "AI" generated code[1].
>
> At least I would say that this particular patch series was rejected,
> but there were was no explicit discussion and clear statement about
> "AI" generated content; especially code.
>
> Thus I want this thread to start a discussion, that eventually leads
> to a policy about submitting and integrating "AI" generated content.

I don't think labeling code as "AI" matters that much. Let's ignore
licensing/legal issues for now.

What's important is the code itself and its quality. It doesn't matter
how it was created. Whether by a human, "AI" or something else. The
key is the final product. "AI" is just a tool, and like any tool, it
can be used well or poorly. How you use it may be completely different
between "operators".

I think the "AI" label exists because the code that LLMs produce is
often incomplete, low quality, and a pile of spaghetti that somehow
works for a single use case. but is far from being a sane, production
ready implementation. Anyone who has used these tools knows their
limitations and what they can or cannot do.

That said, if "AI" code means low quality code, then by all means, it
should be rejected. This applies to human, alien, or "AI" generated
code. There shouldn't be a different metric for "AI" code. If "AI"
(and its "operator") produces high quality code, there's no reason to
reject it.

After all, how can you even detect "AI" code? If the code, regardless
of who or what wrote it, follows project guidelines and is overall
high quality, that's all that matters.

P.S. I don't like those "This code was fully made by an LLM"
statements and the like. Who cares? Maybe some investor who's pushing
this. But from a technical point of view, there's no difference. After
all, you don't start your patchset by saying, "This code was written
in Vim with <list of plugins> on Arch Linux, on an ergonomic split
keyboard, with an XYZ monitor.".

- Kacper

> Leaving all ethical issues aside for a moment I still see 2 very big
> problems with AI generated code:
>
> * looks generally plausible but is often subtly wrong
>     * leading to more work, regressions and costs
>         * which often lands on a different group of people (other
>           projects, reviewers, bug finders, bug fixers, etc.)
>         * which are sometimes delayed for quite some time increasing
>           the costs of fixing them
> * license/copyright violations
>     * this might be sometimes a non-issue with small changes
>     * but especially for complete components the risk seems high
>
> There is a lot more to the topic and I probably forgot to bring up
> many more important aspects and details. Please feel free to bring
> more things up in the discussion!
>
> There was a preparation in the musl project to put up a policy[2],
> it has not yet been finalized and realized as far as I understand.
>
> It also brings up the point, that it is not really related to
> recent "AI" tech, but more to the origin of work and its handling.
> Unfortunately "AI" made problems with this a lot more common.
>
>
> Best regards,
>   Alexander
>
> 1. https://lists.ffmpeg.org/pipermail/ffmpeg-devel/2025-April/342146.html
> 2. https://www.openwall.com/lists/musl/2024/10/19/3
>
>
>
> ---------- Forwarded message ----------
> From: Alexander Strasser via ffmpeg-devel <ffmpeg-devel@ffmpeg.org>
> To: ffmpeg-devel@ffmpeg.org
> Cc: Alexander Strasser <eclipse7@gmx.net>
> Bcc:
> Date: Tue, 1 Jul 2025 12:58:23 +0200
> Subject: [FFmpeg-devel] [RFC] Introducing policies regarding "AI" contributions
> _______________________________________________
> ffmpeg-devel mailing list
> ffmpeg-devel@ffmpeg.org
> https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
>
> To unsubscribe, visit link above, or email
> ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".
_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] [RFC] Introducing policies regarding "AI" contributions

[Gerion Entrup]

[-- Attachment #1.1: Type: text/plain, Size: 2347 bytes --]

Am Dienstag, 1. Juli 2025, 12:58:23 Mitteleuropäische Sommerzeit schrieb Alexander Strasser via ffmpeg-devel:
> Hi all,
> 
> I do not like the branding of the LLMs as AI, thus I will for now
> continue to call it "AI" in quotes. I'm open for better terms.
> 
> It was just yesterday brought up on IRC in #ffmpeg-devel that there
> was at least one, marked attempt to include "AI" generated code[1].
> 
> At least I would say that this particular patch series was rejected,
> but there were was no explicit discussion and clear statement about
> "AI" generated content; especially code.
> 
> Thus I want this thread to start a discussion, that eventually leads
> to a policy about submitting and integrating "AI" generated content.
> 
> Leaving all ethical issues aside for a moment I still see 2 very big
> problems with AI generated code:
> 
> * looks generally plausible but is often subtly wrong
>     * leading to more work, regressions and costs
>         * which often lands on a different group of people (other
>           projects, reviewers, bug finders, bug fixers, etc.)
>         * which are sometimes delayed for quite some time increasing
>           the costs of fixing them
> * license/copyright violations
>     * this might be sometimes a non-issue with small changes
>     * but especially for complete components the risk seems high
> 
> There is a lot more to the topic and I probably forgot to bring up
> many more important aspects and details. Please feel free to bring
> more things up in the discussion!
> 
> There was a preparation in the musl project to put up a policy[2],
> it has not yet been finalized and realized as far as I understand.

Just to link it here. Remembers me on the Gentoo Linux discussion:
https://archives.gentoo.org/gentoo-dev/9007c921a8a57655ecb2027eb4be4bff02673af4.camel@zougloub.eu/T/#t
https://wiki.gentoo.org/wiki/Project:Council/AI_policy


Best,
Gerion


> 
> It also brings up the point, that it is not really related to
> recent "AI" tech, but more to the origin of work and its handling.
> Unfortunately "AI" made problems with this a lot more common.
> 
> 
> Best regards,
>   Alexander
> 
> 1. https://lists.ffmpeg.org/pipermail/ffmpeg-devel/2025-April/342146.html
> 2. https://www.openwall.com/lists/musl/2024/10/19/3
> 


[-- Attachment #1.2: This is a digitally signed message part. --]
[-- Type: application/pgp-signature, Size: 659 bytes --]

[-- Attachment #2: Type: text/plain, Size: 251 bytes --]

_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] [RFC] Introducing policies regarding "AI" contributions

[Alexander Strasser via ffmpeg-devel]

[-- Attachment #1: Type: message/rfc822, Size: 9303 bytes --]

From: Alexander Strasser <eclipse7@gmx.net>
To: FFmpeg development discussions and patches <ffmpeg-devel@ffmpeg.org>
Subject: Re: [FFmpeg-devel] [RFC] Introducing policies regarding "AI" contributions
Date: Fri, 4 Jul 2025 01:14:11 +0200
Message-ID: <aGcOw7VGSWuE3OVp@metallschleim.local>

On 2025-07-03 02:16 +0200, Gerion Entrup wrote:
> Am Dienstag, 1. Juli 2025, 12:58:23 Mitteleuropäische Sommerzeit schrieb Alexander Strasser via ffmpeg-devel:
[...]
> > Thus I want this thread to start a discussion, that eventually leads
> > to a policy about submitting and integrating "AI" generated content.
> > 
> > Leaving all ethical issues aside for a moment I still see 2 very big
> > problems with AI generated code:
> > 
> > * looks generally plausible but is often subtly wrong
> >     * leading to more work, regressions and costs
> >         * which often lands on a different group of people (other
> >           projects, reviewers, bug finders, bug fixers, etc.)
> >         * which are sometimes delayed for quite some time increasing
> >           the costs of fixing them
> > * license/copyright violations
> >     * this might be sometimes a non-issue with small changes
> >     * but especially for complete components the risk seems high
> > 
> > There is a lot more to the topic and I probably forgot to bring up
> > many more important aspects and details. Please feel free to bring
> > more things up in the discussion!
> > 
> > There was a preparation in the musl project to put up a policy[2],
> > it has not yet been finalized and realized as far as I understand.
> 
> Just to link it here. Remembers me on the Gentoo Linux discussion:
> https://archives.gentoo.org/gentoo-dev/9007c921a8a57655ecb2027eb4be4bff02673af4.camel@zougloub.eu/T/#t
> https://wiki.gentoo.org/wiki/Project:Council/AI_policy

Thanks for the links to the Gentoo discussion and policy!

IMHO the discussion and the resulting policy is interesting and maybe
something similar would be appropriate for FFmpeg.

I also became aware of LLVM policy:

  https://llvm.org/docs/DeveloperPolicy.html#ai-generated-contributions

But I must say I do not like it as much. To cite the most critical part:

    As such, the LLVM policy is that contributors are permitted to use
    artificial intelligence tools to produce contributions, provided that
    they have the right to license that code under the project license.
    Contributions found to violate this policy will be removed just like
    any other offending contribution.

For "AI" (in the LLM sense) I think it's usually not at all easy to
say if one has the right to license the code given it's trained on
a huge corpus of copyrighted and particularly licensed code.

Anyway they agree on license/copyright concern I raised. As does Gentoo.

And the LLVM policy also comes to a similar conclusions, as does Gentoo,
regarding waste of project resources:

    We encourage contributors to review all generated code before sending
    it for review to verify its correctness and to understand it so that
    they can answer questions during code review. Reviewing and maintaining
    generated code that the original contributor does not understand is not
    a good use of limited project resources.


If anyone has more examples at hand, it would probably be interesting to
know and take a look.

Best regards,
  Alexander


> > It also brings up the point, that it is not really related to
> > recent "AI" tech, but more to the origin of work and its handling.
> > Unfortunately "AI" made problems with this a lot more common.
> > 
> > 
> > Best regards,
> >   Alexander
> > 
> > 1. https://lists.ffmpeg.org/pipermail/ffmpeg-devel/2025-April/342146.html
> > 2. https://www.openwall.com/lists/musl/2024/10/19/3

[-- Attachment #2: Type: text/plain, Size: 251 bytes --]

_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] [RFC] Introducing policies regarding "AI" contributions

[Alexander Strasser via ffmpeg-devel]

[-- Attachment #1: Type: message/rfc822, Size: 9115 bytes --]

From: Alexander Strasser <eclipse7@gmx.net>
To: FFmpeg development discussions and patches <ffmpeg-devel@ffmpeg.org>
Subject: Re: [FFmpeg-devel] [RFC] Introducing policies regarding "AI" contributions
Date: Fri, 4 Jul 2025 01:31:19 +0200
Message-ID: <aGcSx5IzK7aj6UOd@metallschleim.local>

On 2025-07-01 14:44 +0200, Kacper Michajlow wrote:
> On Tue, 1 Jul 2025 at 12:58, Alexander Strasser via ffmpeg-devel
[...]
> >
> > I do not like the branding of the LLMs as AI, thus I will for now
> > continue to call it "AI" in quotes. I'm open for better terms.
> >
> > It was just yesterday brought up on IRC in #ffmpeg-devel that there
> > was at least one, marked attempt to include "AI" generated code[1].
> >
> > At least I would say that this particular patch series was rejected,
> > but there were was no explicit discussion and clear statement about
> > "AI" generated content; especially code.
> >
> > Thus I want this thread to start a discussion, that eventually leads
> > to a policy about submitting and integrating "AI" generated content.
> 
> I don't think labeling code as "AI" matters that much. Let's ignore
> licensing/legal issues for now.

OK, but I really don't think we can ignore the legal consequences
for FFmpeg, as it is Open Source software, and we would put all
users of FFmpeg, individuals and companies, at risk.


> What's important is the code itself and its quality. It doesn't matter
> how it was created. Whether by a human, "AI" or something else. The
> key is the final product. "AI" is just a tool, and like any tool, it
> can be used well or poorly. How you use it may be completely different
> between "operators".
> 
> I think the "AI" label exists because the code that LLMs produce is
> often incomplete, low quality, and a pile of spaghetti that somehow
> works for a single use case. but is far from being a sane, production
> ready implementation. Anyone who has used these tools knows their
> limitations and what they can or cannot do.
> 
> That said, if "AI" code means low quality code, then by all means, it
> should be rejected. This applies to human, alien, or "AI" generated
> code. There shouldn't be a different metric for "AI" code. If "AI"
> (and its "operator") produces high quality code, there's no reason to
> reject it.
> 
> After all, how can you even detect "AI" code? If the code, regardless
> of who or what wrote it, follows project guidelines and is overall
> high quality, that's all that matters.

I kind of agree that good code is good code, but it's not enough.
Important is also having people around that truly understand the
good code.

To find out if it is truly good code someone needs to review it very
deeply, which is extra hard if it is "AI" generated code as it tends
to look very plausible; which could waste a lot of time for the people
looking at it and reviewing it. This also diminishes the actual value
of the use of "AI" in the first place.

Taking that for granted there is the open question for submissions
by maintainers (with git push access), who could submit "AI" generated
code and push it themselves after a considerable push warning.


> P.S. I don't like those "This code was fully made by an LLM"
> statements and the like. Who cares? Maybe some investor who's pushing
> this. But from a technical point of view, there's no difference. After
> all, you don't start your patchset by saying, "This code was written
> in Vim with <list of plugins> on Arch Linux, on an ergonomic split
> keyboard, with an XYZ monitor.".

[...]

Thanks for your feed back!

Greetings,
  Alexander

[-- Attachment #2: Type: text/plain, Size: 251 bytes --]

_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] [RFC] Introducing policies regarding "AI" contributions

[Alexander Strasser via ffmpeg-devel]

[-- Attachment #1: Type: message/rfc822, Size: 6961 bytes --]

From: Alexander Strasser <eclipse7@gmx.net>
To: FFmpeg development discussions and patches <ffmpeg-devel@ffmpeg.org>
Subject: Re: [FFmpeg-devel] [RFC] Introducing policies regarding "AI" contributions
Date: Fri, 4 Jul 2025 01:42:10 +0200
Message-ID: <aGcVUqc_8_sbAhs4@metallschleim.local>

On 2025-07-01 16:50 +0530, Gyan Doshi wrote:
> 
> On 2025-07-01 04:28 pm, Alexander Strasser via ffmpeg-devel wrote:
> 
> > Thus I want this thread to start a discussion, that eventually leads
> > to a policy about submitting and integrating "AI" generated content.
>
> In practice. unless a patch(set) is explicitly marked or has telltale signs
> of being AI-generated,the project can't stop such AI code getting in.
> At best, we can require disclosure and for the human submitter to assume
> responsibility.

That's true. It's impossible to completely enforce adherence to a
policy that bans "AI" generated code.

I guess it would still be worthwhile to just do what you said.

From what I have looked at in the other projects so far (musl,
gentoo, llvm), they acknowledge too that they cannot enforce it.

In a way it's nothing new and actually since forever we would
not want to accept contributions of dubious or license-incompatible
origins.

Just the current times seem to warrant spelling this out, I fear.
So maybe just generically writing about it and explicitly mentioning
"AI" would be the better way to achieve the goal.

Thanks for commenting!


Best regards,
  Alexander

[-- Attachment #2: Type: text/plain, Size: 251 bytes --]

_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] [RFC] Introducing policies regarding "AI" contributions

[Leo Izen]

On 7/1/25 06:58, Alexander Strasser via ffmpeg-devel wrote:
> _______________________________________________

While I agree with you on the merits that LLM-generated code tends to be 
low quality, ideally that will be caught during code review. I think a 
blanket ban on it makes more sense because of the legal implications of 
including LLM-generated code in our codebase.

I am not a lawyer, so I cannot say for certain how the legality plays 
out, and it may be safer to just not permit it than try to hire a laywer 
to figure out how to permit it, if it's even possible.

- Leo Izen

_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".