AI-Powered Code Review Automation
From internal tool to production system — multi-layer security scanning with Claude AI analysis across 179 enterprise changesets in 10 weeks.
The Challenge
Our team manages an enterprise codebase that powers 80+ branded client storefronts for companies like Marriott, Best Western, and United Rentals. A 5-person offshore development team spans three time zones, processing thousands of TFVC changesets per year through Azure DevOps. The question was never whether to do code review. It was whether human review alone could keep pace.
Manual review suffered from three compounding problems. Throughput: each review took 30-60 minutes of focused analysis, creating a backlog of 5-10 pending changesets at any given time. Consistency: the same class of issue — a missing null guard, business logic leaking into a controller, an unparameterized SQL query — would get flagged by one reviewer and missed by another depending on energy and time pressure. Auditability: when a compliance audit asked “prove that changeset 24935 was reviewed for security vulnerabilities before deployment,” we had code review request work items with no structured findings attached.
The business impact was measurable. Bugs that should have been caught at review time reached production. Security vulnerabilities sat in the codebase for weeks because nobody had time to run a Snyk scan on every changeset. And the compliance gap meant one audit away from a serious finding — particularly for healthcare clients operating under HIPAA requirements. Weekly time cost: 5-10 hours on code review alone, competing with architecture work, client meetings, and project management.
Our Approach
We built the code review system we wished existed — not a linter, not a static analysis dashboard, but a full-pipeline reviewer that combines security scanning, AI-powered code analysis, and structured reporting into a single automated workflow.
The architecture is orchestration-first rather than monolithic. Instead of one large application, we created a pipeline of independent stages: changeset ingestion from Azure DevOps REST API, multi-layer security scanning (Snyk SAST, Gitleaks, NuGet SCA, Roslyn), AI-powered code review via Claude with custom agent instructions, and structured report generation with Notion database integration. Each stage runs independently and can be swapped or upgraded without touching the others.
We chose Claude as the AI engine because it understands architectural patterns — not just syntax. We wrote a custom enterprise-code-reviewer agent definition that encodes our specific quality standards: SOLID principle enforcement, layer separation rules, ServiceResult<T> pattern compliance, async/await correctness, and codebase-specific patterns. The agent classifies issues by severity (Critical, High, Medium, Low) with concrete fix guidance referencing specific line numbers.
Early versions auto-posted all findings to the project management tool. The offshore team treated every finding as a must-fix and spent entire nights on low-priority style issues. We added a human approval gate: the reviewer selects which findings warrant developer attention. Critical and High findings route to the team. Low-value findings get logged but never create noise. Result: the team focuses on real issues, not AI chatter.
The Solution
Security Scanning
- Snyk SAST
- Gitleaks Secret Detection
- NuGet SCA
- Roslyn Static Analysis
AI Code Analysis
- SOLID Compliance
- Architecture Review
- Complexity Assessment
- Error Handling Review
Wrike
PDF attached to task
Notion
Structured DB entry
Archive
Markdown + PDF stored
Changeset Ingestion Layer
- Azure DevOps REST API v7.0 with secure PAT wrapper
- Supports TFVC changesets, GitHub PRs, and branch comparisons
- File filtering: reviews .cs, .cshtml, .js, .ts, .tsx, .jsx, .sql
- Before/after diff for edited files (version N vs N-1)
Security Scanning Pipeline
- Snyk SAST for known vulnerability patterns
- Gitleaks for API keys, passwords, tokens, and connection strings
- NuGet SCA for dependency vulnerabilities with CVE matching
- Roslyn analysis for C#-specific quality and pattern enforcement
AI Analysis Engine
- Claude-powered review with custom enterprise-code-reviewer agent
- Encodes codebase-specific patterns: ServiceResult<T>, Repository, layer separation
- Severity classification with line-level precision and concrete fix guidance
- Four verdict levels: Approved, Approved with Comments, Changes Requested, Veto
Report Generation & Distribution
- Structured Markdown with standardized severity tables and file-by-file findings
- PDF generation via Pandoc + Typst for professional deliverables
- Notion API integration with 15+ structured properties per review
- Automated Wrike task attachment and TFVC closure via REST API
Approval Gate
- Human judgment layer between AI analysis and team delivery
- Reviewer selects which findings warrant developer attention
- Prevents offshore churn on low-priority style issues
- Critical/High findings routed to Wrike; low-value findings logged but filtered
Automation Infrastructure
- Windows Task Scheduler executes pipeline 4x daily (5 AM, 10 AM, 3 PM, 8 PM)
- Idempotent execution: skips already-reviewed changesets
- Dispatch log (JSON) for state tracking — no database dependency
- Fully unattended: reviews happen whether anyone is watching or not
Results
179
Structured code reviews completed in 10 weeks
6
Developers covered across all experience levels
~18/week
Average review throughput
31
Critical severity issues caught
188
High severity issues caught
4
Independent security scanners per review
3
Deployments vetoed for security flaws
~2 min
Time per review (approval gate only)
100%
Security scan coverage (was spot-checks)
Verdict Distribution
Top Issue Categories Caught
- Empty catch blocks / swallowed exceptions — most common High finding across all developers
- Path Traversal (CWE-23) — Snyk finding across multiple controllers
- Synchronous code in async methods — blocking patterns in production code
- Missing authentication — [AllowAnonymous] on production endpoints
- Hardcoded credentials — production secrets committed to source
- N+1 query patterns — database queries inside loops
- Logic bugs — inverted boolean conditions, operator precedence errors
Business Impact
- Senior developer review time redirected from full-changeset analysis to reviewing flagged findings only — estimated 60-70% time savings on review cycles
- Consistent enforcement of architectural patterns across all team members, regardless of reviewer availability
- Auditable proof of security scanning for every changeset — compliance gap eliminated
- Structured Notion database enables trend analysis: which developers produce which severity patterns, which file types carry the most risk
- System runs continuously against every changeset — first line of defense, building institutional knowledge with every review
“The best code review tool is the one you build for your own codebase first. We didn't design this as a product — we designed it to catch the bugs our team was actually shipping. That's why it works: it encodes real patterns from real production incidents, not theoretical best practices from a textbook.”
Frequently Asked Questions
See What Your Code Review Is Missing
Send us a changeset or pull request. We'll run the full pipeline and send you the report — security scans, architectural analysis, and severity-classified findings.