Anshu Kumar Singh
Back to AI LabLLM Experiments

GPT-4 Vision for Automated Code Review

Experimenting with GPT-4V to automatically review pull requests and suggest improvements.

Goal

Automate PR code review using GPT-4 Vision

Approach

Feed diff + context to GPT-4V with structured prompts

Result

85% of suggestions were rated useful by developers

GPT-4 Vision for Automated Code Review

Goal

Can GPT-4 Vision meaningfully review code changes and provide actionable feedback?

Approach

  1. Fetch PR diff via GitHub API
  2. Render diff as syntax-highlighted image
  3. Send to GPT-4V with structured review prompt
  4. Parse structured JSON response

Key Findings

  • GPT-4V catches logical bugs that text-only models miss
  • Visual context (indentation, structure) helps the model
  • False positive rate is ~15% — needs human oversight

Future Work

  • Fine-tune on accepted/rejected review comments
  • Add repo-specific context via RAG
  • Build VS Code extension