AI tool · GitHub
QA for teams writing with GitHub Copilot
Copilot completes code in the editor and, as a cloud agent, opens pull requests from its own branches in a GitHub Actions environment. The platform enforces a second reviewer; the checklist is what that reviewer runs.
Termscritical flow (a path a user takes that costs money or trust if it breaks: sign-up, checkout, invite, export) · verdict (a one-word release result (pass, pass with findings, fail) with the findings and the checklist version behind it) · CI (the pipeline that builds and tests every change before it can be merged or deployed) · client bundle (the JavaScript your app sends to the browser or ships inside the mobile app; anyone can read it) · protected branch (the git branch that cannot be pushed to directly and is the only source of production deploys) · LLM feature (a product feature that calls a large language model, such as an assistant or a summariser)
What GitHub Copilot changes in a codebase
Completions and chat in the editor, plus a cloud coding agent that works in a GitHub Actions environment and opens pull requests on its own branches.
Copilot's cloud agent cannot merge and cannot approve its own pull request, and workflows wait for a person to approve them. Those are good defaults. They also mean the quality of the release depends entirely on what the approver checks, and on what the CI workflows encode. The checklist gives both a list.
Defaults, from the docs verified 2026-09-21
| Default, from the docs | Why it matters for release testing | Source |
|---|---|---|
| The cloud agent "has access to its own ephemeral development environment, powered by GitHub Actions, where it can explore your code, make changes, execute automated tests and linters and more." | Tests the agent runs are the repository's own; if the suite is thin, the agent's green run proves little. The checklist items are what the suite should encode. | docs verified 2026-09-21 |
| "Copilot cloud agent only has the ability to push to a single branch." | Agent changes cannot reach the protected branch directly; the review and the checklist run happen on the pull request. | docs verified 2026-09-21 |
| "By default, workflows are not triggered until Copilot cloud agent's code is reviewed and a user with write access to the repository clicks the Approve and run workflows button." | CI does not run on agent pull requests until a person approves, so a checklist step in CI is gated by that click; administrators can turn the gate off. | docs verified 2026-09-21 |
| "Copilot cloud agent cannot mark its pull requests as 'Ready for review' and cannot approve or merge a pull request." The requesting user cannot approve it either. | A second person is required by the platform; the release verdict is the content of that second look. | docs verified 2026-09-21 |
| "GitHub restricts Copilot cloud agent's access to the internet" and secret scanning is applied to its output. | Secret scanning on the platform catches known key formats in pushed code; it does not catch a server-only value inlined into a client bundle at build time. | docs verified 2026-09-21 |
Each row was read on the linked page on the date shown. Docs change; the date tells you how current the row is. GitHub Copilot documentation.
Failure patterns most associated
- criticalAPI key shipped in the client bundle
A key meant for the server ends up in JavaScript the browser downloads, so anyone can read it and call the service as the app.
- criticalService-role key used from the front end
The database's bypass-everything key is used in client code, so row-level rules no longer protect anything.
- criticalSecret committed to the repository
A key or token is in git history, where it stays readable to anyone with repository access even after the file changes.
- criticalRow-level security off on a new table
A table created by a migration or SQL has no row-level security, so the anonymous key can read and write every row through the public API.
- criticalPolicy that allows everything
Row-level security is enabled but the policy condition is true for everyone, so isolation exists in name only.
- criticalList endpoint returns other tenants' rows
A list or search endpoint filters by the wrong column or none, so a user sees records belonging to other accounts.
- criticalObject fetched by id without an ownership check
An endpoint checks that the caller is logged in, then returns whatever id was requested.
- criticalAdmin gate enforced only in the UI
The admin page hides its buttons from ordinary users, but the underlying route or action accepts calls from anyone logged in.
How the checklist adapts
- Deploy items (DR-01) confirm that production deploys come only from CI on the protected branch, which the copilot/ branch restriction supports by default.
- Secrets items (SC-01, SC-02) run in addition to GitHub secret scanning, because scanning catches key formats in source, not a server value inlined into a client bundle at build time.
- Auth items (AU-01, AU-02) call routes and actions directly on the pull request's preview deployment before approval.
- Release notes (DR-05) record the checklist version on the merged pull request, so the approve-and-run click has a written basis.
Release checklist, five of the published items
v1.0 · verified on 2026-09-21Production deploys are triggered only from the reviewed branch in CI
manual Read the deployment configuration and the last 10 production deploy records.
Every production deploy in the sample originates from CI on the protected branch; no local push or agent session deployed to production.
Agents and tools hold no write credentials to production
manual Inventory every credential available to coding agents, CI jobs and developer machines.
No agent or developer session credential can write to production data or trigger a production deploy without a human approval step.
Release notes carry the checklist version and verdict
manual Open the release record.
The record names the checklist version, the verdict, the build hash and the tester.
No secret keys in client bundles
automated Scan the built client assets (JS, source maps, HTML) for known key prefixes and for every value present in the server environment file.
Zero matches for key prefixes (sk_, service_role, AKIA, whsec_, private keys) and zero server-only env values found in client assets.
No secrets in repository history
tool Run a secret scanner (gitleaks or equivalent) over the full git history, not only the working tree.
Scanner reports zero unrevoked findings; any historical finding has a rotation record.
Which plan fits
Growth $5,900/mo
Teams of 8-15 with a public API sit on Growth; the pull-request cadence usually means daily releases. Smaller web-only teams fit Starter.
For 8-15 engineers, daily releases, web plus API or one mobile platform. Limits, cadence and what is not included.
Questions
Testing Copilot generated code
Copilot code review checklist
What is managed QA?
Get a risk list for your codebase
5 business days, $1,200, credited to the first month.