Give AI a narrow testing task
Ask for checks around one explicit behavior, such as rejecting an expired invitation token. Supply the contract, relevant code, and existing fixture style. Do not ask the model to infer the business rule only from the implementation, because it may reproduce the same mistake in its expected result.
Keep production credentials, private customer data, and sensitive logs outside the prompt unless an approved environment explicitly permits their use. Synthetic examples usually suffice for test design. Record which assumptions the generated scenario makes so a reviewer can confirm them before the code enters the maintained suite.
Review assertions before execution success
Inspect the expected values and edge cases first. A generated test may assert that a result exists while ignoring its incorrect content. Require the exact status, amount, or permission outcome that matters. Remove excessive mocks that replace the behavior the test was supposed to examine in the first place.
Run the proposed checks in the same environment used for normal development. Fix imports and fixtures through review rather than accepting broad changes to make the test pass. Passing execution proves only that the current test and code agree under that setup; it does not establish that the requirement is correct.
Prove the check can reject wrong behavior
Introduce a bounded local defect or replay a known regression in a safe scratch environment. For an expiry rule, change the comparison at the boundary and confirm that the relevant check fails. This provides direct evidence that the assertion can detect the mistake it claims to guard against.
Keep human ownership of generated tests after they merge. Label uncertain requirements and unresolved environment limits in the review evidence. Use AI failure summaries as leads for investigation, and verify their claims against logs and code before assigning a cause or declaring a product defect resolved.
Working sheet
| Generated output | Human check | Useful evidence |
|---|---|---|
| Scenario list | Matches written requirements | Risk-to-test mapping |
| Assertions | Rejects wrong business outcome | Exact expected values |
| Mocks | Preserves intended boundary | Dependency inventory |
| Failure summary | Matches raw evidence | Logs and reproducible run |
Questions about use AI in test automation with verifiable assertions
Can AI decide the expected result from code?
It can suggest one, but a requirement or domain decision should independently establish the expected behavior.
Is a passing generated suite enough?
No. Review assertions and verify that selected meaningful defects cause the intended tests to fail.
How should I use the working sheet?
Download the CSV and replace the sample rows with evidence from your repository. Record a clear acceptance check and an owner for each item. Keep unresolved assumptions visible rather than treating the example as a completed assessment.
Sources and further reading
Examples are illustrative. They are not customer results, security guarantees, or a substitute for checking the documentation for your installed version.
