Every release we test ends in a verdict (a one-word release result (pass, pass with findings, fail) with the findings and the checklist version behind it): pass, pass with findings, or fail. The word is the least important part. What matters is what sits behind it and what it is careful not to say.
The three words
Pass means no finding at critical or high severity on the checklist items in scope for the plan, on the build named in the record. Medium and low findings may exist and are listed.
Pass with findings means at least one high-severity finding. The release can ship. The findings are scheduled, and the next release run re-tests them first. This is the most common verdict on a healthy team, because high findings are usually a missing branch or a missing check that is fixed in a day.
Fail means a critical finding: data exposure, money, or account takeover. The release is held until the item passes. Where the fix is small, the re-test happens the same day and the record shows two builds.
What sits behind it
A verdict record names four things: the build hash, the checklist version, the tester, and the findings by severity with the checklist id and the pattern for each. The pattern link matters. A finding that says "AC-01: invoices table has no row-level security (database rules that decide which rows each signed-in user may read or write)" is fixable in ten minutes; a finding that says "security issue in billing" is a meeting.
The checklist version is there so a verdict from March can be read in September. Items change; a record that names v1.0 is a record against v1.0.
What it does not say
A pass does not say the build has no defects. It says the items in scope passed their criteria on that date. The scope is the plan's limits: the flows automated, the surfaces, the categories the plan includes. Starter does not run the LLM items; a Starter pass says nothing about a model feature.
The disclaimer on every record is one sentence: verdicts describe the build we tested on the date shown; changes after that date are covered by the next release run. It is there because the most common misreading of a verdict is as a property of the product rather than of a build.
Why a person signs it
Automated items run in CI (the pipeline that builds and tests every change before it can be merged or deployed) and their output is in the record. The manual items (reading policies, replaying auth flows out of order, denying permissions on a device) are run by the named QA lead, and the verdict carries their name. If a release ships with a critical finding we marked as passed, the next month is free. That guarantee only makes sense if a person stood behind the word.
Read how a release run works and the checklist the verdict is written against.