Generative AI policy

[thomas-zahner]

I think until recently (like 6 months ago) no contributions seemed to have contained content produced by generative AI. In the recent months I've encountered some contributions which were "vibe coded" or contained such generated content.

I feel like now might be a good time to add some sort of policy. Personally, I'm really not a big fan of any generated content and if I do see PRs which contain such content I'm less inclined to review them.

Some examples:

• fix(quirks): check GitHub #readme fragments via REST API #2239
• fix(extract): truncate plaintext URLs at fullwidth delimiters #2205
• fix(formatters/markdown): stop leaking <error:> placeholder into output #2144

@katrinafyi and I have already started to discuss this in #2143 (comment). @katrinafyi proposed to take LLVM's policy as a reference.

@mre @katrinafyi what are your thoughts on this topic.

[katrinafyi]

Yeah. I think "agentic" contributions have definitely increased recently, possibly even working without any human interaction. I think they also specifically target "good first issues". Even interacting with this one PR is very quickly burning me out. I only started reviewing #2239 because it's a feature that I care about. Were it any other PR, I wouldn't have given it any time.

Where I come from is that I don't want to interact with AI by proxy, and I especially don't want to interact with AI without knowing about it. It's really demoralising. I want to understand and to know that I have been understood, and I feel like LLMs are the complete opposite of that. The LLM doesn't have an understanding of reality—it's not possible for it to understand. If a person pastes LLM output into a comment, that is also a disproportionate burden on reviewer time (if you can't be bothered to write it, why should I read it?).

As a starting point, I would like a policy to restrict AI in any places of human interaction (issues, review comments, documentation, code comments maybe, commit messages even?). I would also want any AI use to be disclosed (preferably in as much granularity as possible - burdening AI agents is not a concern of mine).

(This comment has been edited. It used to have a list but I think the principles are more important.)

[mre]

In my other projects, policies unfortunately haven't stopped bots from creating PRs. :/ What I tried instead was putting a lot of safety mechanisms in place, like a PR template with mandatory checkboxes, but those ended up being sidestepped too.

I personally don't mind if someone uses an LLM for exploration or typing, but the quality of contributions often suffers because they're low effort. If it takes less time to generate the code than to review it, that's where we cross the tipping point.

What would help is doing a bit of research into how larger projects are handling this. There's a list here that we could look through: https://github.com/melissawm/open-source-ai-contribution-policies

In general, I think we should move this conversation to a discussion (either on GitHub Discussions or Matrix), since this is more of an internal debate. I'd like to formalize a clear path forward before inviting others to comment. What do you think?

[shihchengwei-lab]

One framing I’ve been thinking about: an AI-assisted PR is not automatically the problem. The useful question is whether the PR is worth reviewing.

I made a small Action around that idea:
https://github.com/shihchengwei-lab/corridor-ci

It does not detect AI. It asks non-trivial PRs to explain scope, value, verification, risk, and touched paths before maintainer review.

The point is to make the review boundary explicit instead of making maintainers reconstruct it from the diff.