Turn product risks into a coverage map

List critical outcomes before listing tools. For a subscription product, this may include correct billing, access after payment, cancellation, and tenant isolation. Record the consequence of each failure and the existing evidence that could detect it before a release reaches users.

Map every high priority risk to a concrete assertion and test layer. A billing rule may need many unit boundaries and one end-to-end purchase. Mark uncovered risks explicitly. This makes tradeoffs visible without pretending that a single coverage percentage describes product safety.

Define execution and ownership

Run short deterministic checks close to each code change. Put slower environment-dependent checks in a separate stage with a clear trigger. Set a feedback budget that matches the review process, then measure actual queue time and execution time against that budget.

Name owners for test data, environment availability, and failed checks. A failure needs a triage path: product defect, test defect, or environment problem. Temporary quarantine should have an owner and repair condition so it does not become a permanent blind spot.

Measure whether the strategy helps

Review recurring escaped defects and ask whether the missing check belongs in an existing layer. Do not automatically add another browser journey. A contract check or a small unit boundary may detect the same problem earlier with a clearer explanation of the defect.

Track time to actionable feedback, repeated flaky failures, and effort spent maintaining critical checks. Use these observations to adjust scope. A strategy should change when architecture and product risks change, rather than preserving an old test distribution as a target itself.

Working sheet

A risk-based planning worksheet
RiskCheck layerOwner or trigger
Incorrect invoice totalUnit boundariesBilling change
Schema mismatchIntegration contractAPI change
Purchase cannot completeEnd-to-end smokeRelease candidate
Cross-tenant accessAuthorization integrationAccess-control change
Download this worksheet (CSV)

Test automation metrics

Track time to a useful failure, flaky reruns and escaped defects for the same defined period. A rising test count alone does not show that critical behaviour is protected.

Keep the denominator visible. Report flaky tests as a fraction of observed tests or runs, and separate quarantined checks from checks that actually gate a release.

BDD scenarios

Write a scenario around a concrete business rule, with a known starting state, one action and an observable outcome. Agree the expected example with the person who owns the rule.

Avoid turning every click into a separate business step. Reuse low-level automation helpers while keeping the scenario readable and the fixtures independent.

Questions about build a test automation strategy around product risk

Should we target a fixed test pyramid ratio?

Use layers as a design aid. Choose the distribution from product risks, architecture, and feedback costs.

What belongs in the strategy document?

Include risk scope, layer choices, execution triggers, data setup, owners, failure handling, and review criteria.

How should I use the working sheet?

Download the CSV and replace the sample rows with evidence from your repository. Record a clear acceptance check and an owner for each item. Keep unresolved assumptions visible rather than treating the example as a completed assessment.

Sources and further reading

Examples are illustrative. They are not customer results, security guarantees, or a substitute for checking the documentation for your installed version.

Make the next change verifiable.

Getzilla is an open source workflow for changes to existing repositories. Inspect the project, define acceptance checks, and keep independent review in the loop.

See the repository workflow See current access and pricing