Every plan grows an automated suite toward a flow limit: 30, 80 or 200 critical flows (a path a user takes that costs money or trust if it breaks: sign-up, checkout, invite, export) by month 2. The suite is Playwright (an open-source framework that drives a real browser through your product's flows) for web, and it is committed to your repository from the first week.
What a flow is
A critical flow is a path a user takes that, if broken, costs money or trust: sign up, sign in, create the thing the product exists for, pay, invite, export, delete account. One flow is one test file that drives the real UI against a staging environment (a copy of production, without customer data, where release candidates are tested), with its own test data, and asserts the outcome in the database or the API (the interface other software calls; tested directly, without the UI in between), not only on the screen.
Coverage is counted in flows against the plan limit. Not in lines, not in percentages. The sample report shows the count growing month by month.
How it is laid out
tests/
flows/
auth/
billing/
core/
checklist/
sc-01-bundle-secrets.spec.ts
ac-01-rls-flags.spec.ts
ac-02-two-user.spec.ts
au-02-direct-actions.spec.ts
py-01-webhook-signature.spec.ts
fixtures/
playwright.config.ts
The flows directory is coverage. The checklist directory is the automated items, named by their ids so a failing test in CI (the pipeline that builds and tests every change before it can be merged or deployed) reads as a checklist finding. Fixtures create users A and B, seed data, and tear down.
Running in your CI
The suite runs on every release candidate in your existing pipeline (GitHub Actions, GitLab, whatever you have). We add the workflow file; you approve it. Output goes to the pipeline's artifacts and into the release record. Nothing runs on our infrastructure, so there is no dashboard of ours to lose access to.
Flaky tests
A test that fails without a code change gets a written cause and an action in the monthly report, and stays there until it has run clean for a month. It is never skipped or retried into green. Most causes are selectors that matched two elements after a layout change, or a fixture race. The fix is a test id or an awaited seed, and the report says which.
What you keep
Everything. The suite, the fixtures, the workflow, the report history. Cancel with 30 days' notice and the tests keep running in your CI; they just stop growing. That is the point of writing them where you can see them.