ManagedQA PartnersSee plans

AI code · 2026-09-18 · 2 min read

How to test AI-generated code before release

The same tests as any code, with a different starting list. The documented failure profile of AI-assisted code gives you ten categories and the order to run them in.

Code written with Cursor, Copilot, Claude Code or a prompt-to-app tool is tested the way any code is tested. What changes is where to look first, because the samples that have been measured show a specific profile.

What the samples show

Veracode's 2025 benchmark found 45% of code samples generated by more than 100 models failed security tests and introduced OWASP Top 10 vulnerabilities. (Source: Veracode, 2025-07-30 · verified 2026-09-21; denominator: code samples generated by 100+ LLMs on an 80-task benchmark) CodeRabbit's review of 470 open-source pull requests found AI-co-authored ones carried 10.83 issues each against 6.45 for human-only ones. (Source: CodeRabbit, 2025-12-17 · verified 2026-09-21) In the same sample, error-handling gaps were nearly twice as common and logic issues 75% more common. (Source: CodeRabbit, 2025-12-17 · verified 2026-09-21)

None of that says the code is bad. It says the failures cluster, and clustering is what a checklist is for.

The ten categories, in order

  1. Secrets and config. Scan the built bundle (the JavaScript your app sends to the browser or ships inside the mobile app; anyone can read it) and the git history. Generated code calls a service from wherever the feature is written, including the browser.
  2. Access control. Query every exposed table for its row-level security (database rules that decide which rows each signed-in user may read or write) flag, then sign in as a second user and try to read the first user's rows. A policy that exists is not a policy that works.
  3. Auth and sessions. Call every protected route and every exported action without cookies. The page check does not cover the action.
  4. Payments. Send unsigned, altered and duplicated webhook (a message a provider such as Stripe sends to your server when something happens, like a payment) events. Edit the amount in the checkout request.
  5. LLM features (a product feature that calls a large language model, such as an assistant or a summariser). Exceed the spend cap. Put instructions in user content. Render model output containing markup.
  6. Error handling. Force each backing call to fail. Look for a message and a log entry. Block the network mid-request and time the spinner.
  7. Performance smoke. Count queries per request on the busiest pages with real data volumes.
  8. Migrations (a scripted change to the database schema that ships with a release and should be reversible). Run up, down, up on a copy of production data.
  9. Deploy and rollback. Confirm where production deploys come from. Roll staging back and time it.
  10. Mobile gates. Purchase in sandbox, deny permissions, run on the minimum OS version.

Each item on the checklist has a pass criterion: the observable result that counts, written so a second person reaches the same verdict (a one-word release result (pass, pass with findings, fail) with the findings and the checklist version behind it).

What not to do

Do not test in production. Do not treat a green run of a thin suite as evidence. Do not count coverage as a percentage of code; count it as critical flows (a path a user takes that costs money or trust if it breaks: sign-up, checkout, invite, export) automated against a stated limit. And do not treat the generating tool as the problem: the tool docs, read on the team hubs, tell you what each one changes, and the checklist tells you what to check because of it.

More in AI code

Start with the audit

5 business days, $1,200, credited to the first month.

Blog · ManagedQA Partners