Choose a narrow review job

Specify a small set of review targets, such as missing error handling around changed network calls or inconsistencies with a documented API contract. Keep deterministic formatting, type checking, and dependency policies in their existing tools. An assistant is easier to evaluate when its expected contribution is explicit.

Select representative historical changes with known defects and acceptable changes without those defects. Use the same evaluation set when adjusting prompts or repository instructions. Record actionable findings, missed known issues, and unnecessary comments rather than treating total comment count as a measure of value.

Design the CI integration boundary

Limit repository and token permissions to what the review job needs. Decide how it handles external contributions, sensitive paths, large diffs, and generated files. Review the provider's documented data handling and execution model before sending private source code or issue context outside your environment.

Illustrative scenario: a repository receives public pull requests. Run untrusted application code separately from any review job holding privileged credentials. Validate event triggers and repository permissions in the actual CI platform; a convenient default integration should not silently gain write access beyond its task.

Tune feedback and monitor the result

Group related findings, link them to changed lines, and request a concrete explanation of the triggering condition. Give developers a way to dismiss incorrect findings with a reason. Keep the reviewer responsible for checking suggestions before applying patches, especially changes to authorization and persistence logic.

Review pilot results after a defined sample of changes. Compare useful findings, reviewer time spent validating noise, and delays added to the pull-request workflow. Adjust scope or stop the integration if it generates more investigation work than it saves. Reevaluate after meaningful model or configuration changes.

Working sheet

Automation rollout decisions
DecisionInitial setting to considerEvidence before expansion
Merge blockingAdvisory findingsStable precision on important issues
Repository scopeSelected pathsUseful results across representative changes
PermissionsMinimum required accessDocumented need for each permission
Output volumeDeduplicated focused commentsLow dismissal and repeat-comment burden
Download this worksheet (CSV)

Questions about how to add AI code review automation to CI

Should AI approval count as human approval?

Keep ownership explicit. Automated feedback should not impersonate an independent domain reviewer or satisfy a human approval policy by accident.

What should happen when the review service fails?

Define the behavior in advance. An advisory pilot can report an unavailable check while existing required checks continue to govern merging.

How should I use the working sheet?

Download the CSV and replace the sample rows with evidence from your repository. Record a clear acceptance check and an owner for each item. Keep unresolved assumptions visible rather than treating the example as a completed assessment.

Sources and further reading

Examples are illustrative. They are not customer results, security guarantees, or a substitute for checking the documentation for your installed version.

Make the next change verifiable.

Getzilla is an open source workflow for changes to existing repositories. Inspect the project, define acceptance checks, and keep independent review in the loop.

See the repository workflow See current access and pricing