feat(transcription): automatic video subtitle generation via faster-whisper #30514 — closed · 17 commits · 3,035 tests passing · 0 lint errors · auto-closed:llm

My AI-assisted pull request to immich, an open-source project, was closed within minutes today because an LLM was involved. Their policy forbids it:

We ask you not to open PRs generated with an LLM. We find that code generated like this tends to need a large amount of back-and-forth, which is a very inefficient use of our time. We cannot have any confidence in an LLM, so if it’s non-trivial for us to verify it works and you don’t show that you fully understand all implications of the change, reviewing the PR is not worth our time.

Two questions worth answering:

  1. How do you actually tell disciplined AI-assisted work from slop?
  2. What would help maintainers make that call efficiently?

Q1: Slop, or the real thing?

The fear behind policies like immich’s is real. GitHub has been publicly weighing a kill switch for pull requests after maintainers reported being buried in low-effort AI submissions; Godot’s maintainers said the same thing about their own queue. immich alone has closed 86 pull requests under this exact label so far. One widely cited estimate: only about 10% of AI-authored PRs are legitimate (The Register, 3 Feb 2026).

Lowering the bar for AI-written code isn’t the fix. Checking it a different way is, the way Robert C. Martin, “Uncle Bob,” described in a July 2026 post:

My current strategy is to not read any of the code written by my agents… What I do instead is to surround the agents with extreme constraints. Unit tests, gherkin tests, QA procedures, quality metrics, mutation testing, test coverage… I have very high confidence in the code they produce because they’ve had to run the gauntlet.

— Robert C. Martin (@unclebobmartin, X, 23 Jul 2026)

He trusts AI-written code less than human-written code, not more. A person skimming a diff can be talked into approving something that merely looks right. It just has to look plausible enough. A test either passes or it doesn’t; there’s no talking it into anything. So instead of a human reading the code, he runs it through unit tests, QA procedures, and the other checks in his quote above. That’s a tougher standard to satisfy than a quick read-through, and that’s the whole point: he’s checking more, not less, because he trusts the code less. That’s the actual line: slop is a prompt, a diff, and a submit button, tested against nothing; assisted development is held to the same red-green-refactor bar as good human-written PRs already are.

Redundancy is the specific shape slop takes most often, and it has a mechanical cause: an agent that only sees the file in front of it will happily write a function nearly identical to a helper three files over, just renamed: “semantic duplication.” A 2026 study measured this directly across real pull requests, scoring code redundancy at 0.29 for AI agents versus 0.15 for humans (almost double, and statistically significant), mostly in the form of functionally identical code with different syntax rather than copy-pasted duplicates (Huang et al., “More Code, Less Reuse,” MSR ‘26). The same study found reviewers rated AI-authored PRs more favorably than human ones despite this. The authors call it “silent technical debt”: exactly the kind of gap a disclosure checkbox can’t see and a test suite can.

AI-assisted review isn’t automatically safer, either. In April 2026, researchers showed a single malicious PR comment could get review agents from three different vendors, including Claude Code’s own security-review action, to leak their own API keys into public logs. No maintainer action was required beyond the automated trigger. That’s not a bug to patch; it’s low-trust PR content getting loaded as trusted context, and fixing it needs a policy layer in front of the agent, not just tests behind it.

What actually correlates with trustworthy AI-assisted work:

  1. Planned: the work is planned up front, with a clear plan and real user stories, not improvised prompt by prompt (the grill-with-docs skill does this)
  2. Reviewable: it ships in small, reviewable units: one logical change per commit, not one giant diff, with each unit tied to its own ticket (the /to-tickets skill helps here)
  3. Observed: the tests were written first, watched to fail, then made to pass (this is plain TDD)
  4. Verified: any claimed fix comes with before/after evidence, measured on a real run rather than asserted
  5. Exercised: it’s been run against a real running system, not just mocks
  6. Narrated: there’s an honest, checkable account of how the work was actually made, including a thorough code review

Six checks, one acronym: PROVEN.

Q2: What would actually help

No current tool can replace a thorough review. But maintainers already have more than a disclosure box to work with. GitHub itself shipped real controls in February 2026: caps on how many open PRs an unproven external contributor can have at once (with a bypass list for people who’ve earned trust), a collaborators-only mode, archiving for low-quality submissions, and the same rate limiting for issues. The community built faster than the platform did, and split into two complementary approaches. One is deterministic scanners that flag structural tells, like empty functions, swallowed exceptions, and hallucinated imports, with no LLM anywhere near the hot path (the Anti-Slop GitHub Action is one). The other is reputation scorers that weigh a contributor’s history and submission pattern before anyone reads their diff (Slop Meter is one). Neither requires banning AI, just reviewing differently. The same structural tells also work as a pre-commit gate on your own agents’ output, not just other people’s: a rule against swallowing errors catches the exact pattern those scanners are built to flag, before it ever becomes a PR.

For the agents doing the writing, the fix for redundancy specifically sits upstream of review entirely: give the agent a map before it starts, not after. That means module and function overviews fed into every prompt, or better, a structural index it can query (a tree-sitter-based knowledge graph of every symbol and edge in the repo, the way tools like codegraph work), so it can check what already exists and what calls it before generating anything new. Skip that step and an agent will hallucinate a duplicate rather than admit it doesn’t know the codebase.

Better rules help too, but only checkable ones. Pair the disclosure question with a verification one (what tests ran, and what they showed before and after) and the checkbox stops being the whole check.

None of this requires the choice immich’s policy quietly assumes: accept everything, or refuse AI-authored work outright. Caps, scanners, reputation scoring, codebase-aware agents, and rules that ask for evidence instead of a yes/no answer are all already built and in use. The question was never how much AI to allow. It’s whether the check at the door actually looks at the work, instead of just who, or what, made it.

Sources

  1. immich-app/immich, CONTRIBUTING.md — “Use of generative AI,” retrieved 2026-08-03. github.com/immich-app/immich/blob/main/CONTRIBUTING.md
  2. immich-app/immich, Pull Request #30514, closed 2026-08-03. github.com/immich-app/immich/pull/30514
  3. immich-app/immich, pull requests labeled auto-closed:llm, count checked live via GitHub API, 2026-08-03. github.com/immich-app/immich/pulls?q=label%3Aauto-closed%3Allm
  4. Huang, Jaisri, Shimizu, Chen, Nakashima, Rodríguez-Pérez, “More Code, Less Reuse: Investigating Code Quality and Reviewer Sentiment towards AI-generated Pull Requests,” MSR ‘26, April 2026, retrieved 2026-08-03. arxiv.org/html/2601.21276
  5. Hacker News, “This is why the grill me skill went viral,” retrieved 2026-08-03. news.ycombinator.com/item?id=47550391
  6. Hacker News / The Register, “GitHub Ponders Kill Switch for Pull Requests to Stop AI Slop,” 3 Feb 2026, retrieved 2026-08-03. news.ycombinator.com/item?id=46884471
  7. Hacker News, “Godot is drowning in AI slop pull requests,” retrieved 2026-08-03. news.ycombinator.com/item?id=47059779
  8. Hacker News, “Show HN: Slop Meter for GitHub,” retrieved 2026-08-03. news.ycombinator.com/item?id=47223316
  9. peakoss/anti-slop, GitHub Action for detecting and closing low-quality AI PRs, retrieved 2026-08-03. github.com/peakoss/anti-slop
  10. Cloud Security Alliance, research note on prompt injection via AI-powered GitHub Actions (the “Comment and Control” attack class), April 2026, retrieved 2026-08-03. labs.cloudsecurityalliance.org
  11. Robert C. Martin (@unclebobmartin), X post, 23 Jul 2026, retrieved 2026-08-03. x.com/unclebobmartin/status/2080257779395154409