Section 1: The Skepticism Paradox
Here's a paradox worth examining: GitHub's 2025 Octoverse reports that 72.6% of developers using Copilot code review found it improved their effectiveness.[^1] Yet Stack Overflow's 2025 Developer Survey reveals that only 33% of developers trust AI output accuracy—down from 43% the year before—with 46% now actively distrusting it.[^2]
Developers are using tools they trust less than they did a year ago.
This isn't cognitive dissonance—it's pragmatism. The value proposition has shifted. The conversation around AI in software development has largely focused on code generation: can AI write production-ready code?
That framing misses where AI can deliver immediate, measurable value with far less trust required.
Verification vs. Judgment
When I think about code review, I split it into two layers:
Judgment: architecture trade-offs, product intent, domain correctness, and long-term maintainability.
Verification: consistency and completeness against documented standards—patterns, checklists, naming rules, analytics schemas, and “did we remember the boring but important stuff?”
I don’t want AI making judgment calls for me. But I do want it relentlessly running the verification layer—because that’s the part humans agree matters, and still miss under deadline pressure.
PR review isn’t one thing. And skepticism about AI makes perfect sense when we ask it to architect systems or write business logic. But checking whether a PR follows established patterns? Whether analytics events include required parameters? Whether error handling matches conventions?
That’s verification, not creation. And verification is where the bottleneck lives.
Quality and productivity aren't separate concerns—they're linked through rework. Every analytics bug discovered three months post-release requires investigation, prioritization, a fix, another review cycle, and deployment. Fifteen seconds of AI verifying event parameters can prevent hours of future work.
My bet:
PR review verification is one of the fastest places for skeptical teams to feel AI's value—because the output is auditable, and the risk is low.
The blueprint in 30 seconds
If you're impatient, here's what this article will show you:
Add instruction files to your repo (your team's actual patterns and rules)
Run AI review against your diff before opening the PR
Define severity levels so the AI doesn't flood you with noise
Let humans focus on judgment, let AI handle verification
Iterate weekly and:
add what it missed,
remove what it nags about.
The rest of this article explains why this works, what can go wrong, and how to measure whether it's helping.
Section 2: The Bottleneck Everyone Measures
Code review is one of the most visible bottlenecks in software delivery. In organizations that track DORA-style delivery metrics, review time shows up quickly as "time-to-merge," "time waiting for review," and "review rounds." DORA's 2025 report found that despite AI boosting PRs merged by 98%, code review time increased by 91%—a counterintuitive result suggesting AI generates more code faster than teams can absorb.[^7]
The research on code review effectiveness is sobering. A study from Cisco's programming team—often summarized in industry guidance from SmartBear—converges on what most teams learn through experience:[^3]
200–400 lines of code is the optimal review size for defect detection
Review sessions longer than 60 minutes show diminishing returns as reviewer attention degrades
Reviewers process approximately 500 lines per hour effectively; beyond that, quality drops
These aren't arbitrary guidelines. They reflect cognitive limits. A 2,000-line PR isn't just harder to review—it's fundamentally incompatible with how human attention works. Yet large PRs are common because splitting work creates coordination overhead.
The bottleneck isn't laziness or lack of process. It's that thorough code review competes with the same cognitive resources needed for feature development. When a senior engineer spends two hours reviewing a PR, those are two hours not spent on architecture decisions, mentoring, or their own deliverables.
Organizations respond predictably aand review depth decreases as deadlines approach. The checks that slip first are exactly the ones AI handles well—style consistency, documentation completeness, pattern adherence.
There's another factor I (unfortunately) rarely happen to discuss with colleagues—human reviewers aren't consistent across authors. We review some colleagues more thoroughly than others. The senior engineer's PR gets a quick approval while the new hire's PR gets line-by-line scrutiny. These biases aren’t malicious—they’re human. But they mean the same code gets different verification depending on who wrote it.
This is where AI changes the equation—not by replacing human judgment on complex architectural decisions, but by taking on the verification layer humans consistently deprioritize under pressure—and applying it uniformly regardless of author.
Section 3: The Blueprint — Structured AI Instructions (Quick Start Kit)
The difference between useful AI PR reviews and noise is structure. AI tools without context produce generic feedback—the equivalent of running a linter with default rules on a codebase with its own conventions.
This isn't speculation. GitClear's analysis of 153 million lines of code found that code churn hit 7.9% in 2024 (up from 5.5% in 2020), with copy/paste code rising to 12.3%.[^5] The code patterns resembled work from "an itinerant contributor"—someone unfamiliar with the codebase's conventions, duplicating logic that already exists elsewhere.
GitHub's research on Copilot from 2023 showed a 55% speed improvement.[^6] However, the study did not examine the effects of AI on quality. It's likely that increased speed without context led to quantity without quality. Developers probably spent time reviewing AI suggestions that went against architectural decisions, duplicated existing utilities, or introduced patterns the team had deliberately moved away from.
The lesson I learned? AI without understanding the codebase context doesn't just fail to help—it actually creates more work.
What this blueprint does NOT do
To set expectations clearly:
No auto-merging—AI flags issues; humans decide what to do.
No security sign-off—AI can check for obvious patterns (missing auth calls), but security review still needs human judgment.
No reliable architecture decisions—AI might suggest using a repository pattern or how to structure your modules, but human judgment is necessary..
No performance tuning—AI can flag obvious issues, but optimization requires context and execution AI doesn't have.
No replacing code review—This enhances human review, it doesn't replace it.
The goal is narrower—consistent verification of documented standards, freeing humans for the judgment calls that actually need them.
Quick Start (30–60 minutes)
If you want to try this without committing your team to “AI everywhere,” here’s the smallest version that works:
Add a repo-wide instruction file (the rules you wish reviewers enforced consistently).
Add one path-specific instruction file for a high-value area (analytics is a great start).
Define severity levels so the AI doesn’t flood you with nits.
Run an AI review on your diff before opening the PR. It's not necessary, but it's good advice.
Iterate weekly—add what it missed, remove what it nags about.
The workflow I actually use (pre-flight, before humans)
The most effective integration I've found isn't AI reviewing PRs after they're opened—it's AI reviewing code before it reaches human reviewers at all.
Write the feature
Push changes to a branch and open a PR
Run an AI review on the PR, either locally or on the server. Running it on the server keeps a history for future human reviewers, which I personally prefer to always have.
Fix what it catches
Then submit the PR for human review.
This shifts AI review from "another reviewer in the queue" to a pre-flight checklist.
What the AI catches
With properly structured instructions, the AI reviewer enforces decisions the team has already made:
Analytics completeness:
Every user action requires tracking.
The instruction file lists required parameters per event type.
AI verifies every event includes screenName, userSegment, and action-specific context.
No more discovering missing attribution data three sprints later.
MVVM boundaries:
ViewModels don't import UIKit.
Views don't contain business logic.
Coordinators handle navigation.
These aren't suggestions—they're structural decisions.
AI flags violations before they become patterns.
Protocol adoption:
The codebase has established patterns for REST API integration—specific protocols for request building, response parsing, error handling.
A new endpoint that skips APIRequestConfigurable or handles errors inline instead of through APIErrorHandler gets flagged immediately.
Abstraction adherence:
When the team decided all persistence goes through repository interfaces, that decision needs enforcement.
AI spots shortcuts when someone, whether it's the new kid on the block or the project maverick, decides to query Core Data directly "just this once".
The small things:
Debug print statements.
TODO comments that should be tickets.
Force unwraps that should be guard statements.
Hardcoded strings that belong in localization files.
The reviewer may catch these, but why waste their attention on them?
Repository instructions (example: GitHub Copilot)
GitHub Copilot supports two levels of instruction files:
Repository-wide instructions (.github/copilot-instructions.md):
# Project Instructions
This codebase follows MVVM architecture with Coordinators for navigation.
## Review split
- Verification tasks should be enforced consistently by AI.
- Judgment calls belong to humans.
## Architecture boundaries
- ViewModels should always be marked as @MainActor
- Coordinators handle navigation
## Concurrency
- All async operations use Swift Concurrency, not Combine
## Analytics
- Analytics events require both action and context parameters
- Do not ship debug logging or TODOs; convert TODOs to tickets
## Quality
- Prefer small PRs; if a PR exceeds ~400 lines, include a short review guide in the PR description
Path-specific instructions (.github/instructions/*.instructions.md):
---
applyTo: "Sources/Analytics/**"
---
# Analytics Module Instructions
## Event Naming
- Use dot-separated lowercase names (e.g., `article.read.completed`)
- Include `screen` context in all events
## Required Parameters
Every analytics event must include:
- `eventName`: The dot-separated event identifier
- `timestamp`: ISO 8601 format
- `sessionId`: Current session identifier
To show this isn’t “just analytics,” here’s a second path-specific example (choose a module where you’ve been burned before):
---
applyTo: "Sources/Networking/**"
---
# Networking Module Instructions
## Consistency
- New endpoints must use the shared request builder and response decoder
- Do not parse JSON inline inside feature code
## Error handling
- Map transport errors into the shared error type
- Do not swallow errors; return typed failures and log at the boundary
## Testing
- Add unit tests for request encoding and response decoding when adding endpoints
Severity rubric (to prevent noise)
If everything is "important," the AI becomes background noise. You can use a simple rubric like this one:
| Severity | Examples |
| Blocker | Missing security/permission checks, data-loss risk, crashing bugs, secrets in code |
| High | Analytics schema gaps, missing required tests, architecture boundary violations |
| Medium | Pattern inconsistencies, error handling deviations, unclear naming |
| Low | Style nits, formatting, small readability issues |
Here’s the structure I aim for (this is what I want posted as a review, or returned locally):
Summary (2–4 bullets)
Findings by severity (Blocker → Low, could be percentages too)
Suggested tests / QA scenarios (derived from actual diff)
Needs human judgment (explicitly carve out trade-offs)
Example:
## AI Pre-Flight Review
### Summary
- Adds purchase flow completion tracking
- Refactors CheckoutViewModel concurrency to async/await
### Blockers
- None
### High
- Analytics event `checkout.purchase.completed` missing `currency`
### Medium
- ViewModel is not marked with @MainActor; move formatting helper into view layer
### Suggested QA
- Complete purchase with invalid promo code and verify analytics fires with full parameter set
- Cold start into checkout deep link
### Needs human judgment
- Is the new repository abstraction worth the extra indirection for this feature?
Section 4: Failure Modes & Guardrails
AI review is powerful precisely because it’s consistent—but consistency cuts both ways. Here’s what I’ve seen go wrong, and the guardrails that keep it useful.
Failure modes
Instruction drift—The AI enforces outdated rules that no longer apply. I find this very similar to when a team member follows outdated documentation.
False positives → alert fatigue—People start ignoring what the bot writes.
False negatives → false confidence—Teams assume "the bot didn't complain" means "it's correct."
Overreach into judgment—AI tries to dictate architecture instead of just highlighting risks. (I have not seen this happen to be honest, but it's a potential risk)
Security/privacy mistakes—Diffs may include secrets or sensitive data, and prompts might leak information. (Always be cautious about this)
Social misuse—AI comments are used to judge engineer performance.
Guardrails
Treat instruction files like code—assign an owner, review changes, and revisit quarterly in the least.
Cap output—top N findings, group by severity, and link each finding to a specific rule.
Make the split explicit!—AI verifies but humans judge.
Audit occasionally—sample 1-2 in 10 PRs to estimate bot accuracy and tune rules.
Section 5: What AI Finds That Humans Miss (Detailed Examples)
The value of AI PR reviews isn't catching what humans would catch anyway—it's catching what humans consistently deprioritize.
Analytics implementation errors
Analytics tracking is the canonical example. A missing parameter in an analytics event doesn't break the build. It doesn't cause runtime errors. It silently produces incomplete data that nobody notices until someone runs a report months later.
Human reviewers know analytics matters. They also know it's boring to verify. Under time pressure, “analytics looks fine” becomes the default assessment.
AI doesn’t experience time pressure. Given instructions like “every purchase event must include productId, price, currency, and purchaseContext,” it verifies every event, every time.
Documentation drift
Documentation that doesn't match code is worse than no documentation—it actively misleads. But keeping documentation synchronized requires noticing when code changes invalidate docs in other files.
Humans review changed files. AI can be instructed to check whether changes to a public API have corresponding documentation updates, whether removed parameters are still referenced, and whether examples still compile.
Pattern adherence
Every codebase accumulates patterns—some documented, many implicit. New team members don’t know them; experienced team members forget to check them during reviews.
AI, given explicit patterns, checks consistently.
Access control verification
Permission checks follow predictable patterns but fail in subtle ways. A new endpoint that forgets to verify ownership. A bulk operation that checks permissions on the first item but not subsequent ones.
Human reviewers catch these when they're looking for them. AI, instructed with “every endpoint modifying user data must call verifyOwnership() before the operation,” checks every endpoint, every time.
Edge-case handling
Certain categories of bugs follow predictable patterns: off-by-one errors in pagination, timezone handling in date comparisons, null checks on optional chains.
The meta-insight: AI review doesn't replace human judgment. It enforces documented judgment that humans apply inconsistently.
Section 6: How to Measure Whether It Worked
If you want this to land with a mixed audience—ICs and leadership—you need a way to validate it beyond vibes.
Metrics that will probably move first
Time to first human review (does pre-flight reduce back-and-forth?)
PR open → merge time (what is the improvement on average after 3 months?)
Review rounds (how often does a PR bounce for “checklist stuff”?)
Verification-class defects post-merge (analytics gaps, doc mismatches, missing permission checks)
Signals for ICs (quality of the bot itself)
Acceptance rate (what % of AI findings lead to a code change?)
Top recurring findings (the list that should become instruction updates)
Human checklist comments trend (are humans spending less time on nits?)
A simple approach:
measure two weeks of baseline,
enable pre-flight AI verification,
then compare the next 2–4 weeks.
You're not trying to prove a paper—you're trying to see if your team is shipping with less rework.
Section 7: The Documentation Accelerator
There's a parallel to AI's impact on code review in an unexpected domain: management consulting.
A multi-school study of consultants using GPT-4 found a 40% performance increase on tasks within AI's capability frontier—but a 19 percentage point drop when AI was applied outside its strengths.[^4] The researchers called this "jagged" value—dramatic gains in some areas, negative impact in others.
That “jagged frontier” maps cleanly onto PR review.
Senior engineers add unique value in:
Architectural judgment (“this approach will create scaling problems”)
Domain knowledge (“this flow doesn’t match how our users behave”)
Teaching moments (“here’s why we don’t do it that way”)
They add less differentiated value in:
Style consistency verification
Checklist completion (tests present, docs updated, no debug code)
Pattern matching against documented standards
AI handles the second category, freeing humans for the first.
The consulting comparison reveals something else—the teams that capture AI's value aren't the ones with the best tools but they're the ones with the most explicit standards. A team with "our code should be high quality" gets nothing from AI. A team with documented conventions and named patterns can offload verification almost entirely—and the documentation improves reviews even without a bot.
Section 8: Conclusion
The bottleneck in code review isn’t going away. Codebases grow. Teams scale. Cognitive limits don’t change because we wish they would.
What changes is what we ask humans to do.
The shift isn't "let AI review PRs."
It's: use AI for verification so humans can focus on judgment.
Human reviewers bring bias—we review some colleagues more thoroughly than others, we're influenced by past experiences with specific authors, we give different weight to the same patterns depending on who wrote them. AI reviewers bring different bias—they're limited to what the instructions encode. They can't catch (thought they might) what you didn't think to document. They won't recognize (thought they might) context that seems obvious to a human who's been on the team for years.
This trade-off is the point. AI bias is explicit and auditable—it's in the instruction file. Human bias is implicit and variable. For verification tasks with documented criteria, explicit bias wins. For judgment calls requiring context and nuance, human bias (with all its flaws) is still necessary.
That's also why this is a great place for skeptical teams to start. The verification layer is explicit, auditable, and low-risk—and it pays back quickly in reduced rework.
The blueprint is straightforward:
Document your standards explicitly → If a convention exists only in senior engineers’ heads, AI can’t enforce it—and neither can anyone else consistently.
Start with high-value, low-risk checks → Analytics, docs sync, access control patterns, boundary rules.
Integrate with existing workflow → Pre-flight is the key—catch issues before humans see the PR.
Iterate on instructions → Misses and noise are feedback. Update the instruction file like you update tests.
The question isn’t whether AI can help with code review. It already can—today—for verification tasks.
The question is whether your team’s knowledge is documented well enough to leverage it. And if not, whether making it explicit is worth doing anyway.
References
[^1]: GitHub. "Octoverse 2025: AI leads developer activity." GitHub Blog, 2025. https://github.blog/news-insights/octoverse/octoverse-a-new-developer-joins-github-every-second-as-ai-leads-typescript-to-1/
[^2]: Stack Overflow. "2025 Developer Survey: AI." Stack Overflow, 2025. https://survey.stackoverflow.co/2025/ai
[^3]: SmartBear. "11 Best Practices for Peer Code Review." SmartBear Software, 2025. http://viewer.media.bitpipe.com/1253203751_753/1284482743_310/11_Best_Practices_for_Peer_Code_Review.pdf
[^4]: Dell'Acqua, F., et al. "Navigating the Jagged Technological Frontier: Field Experimental Evidence of the Effects of AI on Knowledge Worker Productivity and Quality." Harvard Business School Working Paper 24-013, 2023. Summary: https://mitsloan.mit.edu/ideas-made-to-matter/how-generative-ai-can-boost-highly-skilled-workers-productivity
[^5]: GitClear. "Coding on Copilot: 2023 Data Suggests Downward Pressure on Code Quality." GitClear, January 2024. https://www.gitclear.com/coding_on_copilot_data_shows_ais_downward_pressure_on_code_quality (2025 follow-up data confirms continued churn growth: https://www.gitclear.com/ai_assistant_code_quality_2025_research)
[^6]: Peng, S., Kalliamvakou, E., Cihon, P., Demirer, M. "The Impact of AI on Developer Productivity: Evidence from GitHub Copilot." arXiv:2302.06590, February 2023. https://arxiv.org/abs/2302.06590
[^7]: DORA. "DORA Report 2025: AI Impact on Developer Productivity." Google Cloud, 2025. https://www.faros.ai/blog/key-takeaways-from-the-dora-report-2025
