Category 5 of 10
LLM features
Model calls have spend limits, user content cannot steer tools, output is treated as untrusted, and changes are regression-tested.
v1.0 · verified on 2026-09-21
Model features add a spend surface, an injection surface and an output surface. Growth and Scale include these items; Starter does not.
Spend cap enforced per user and globally
automated In a test account, make model calls past the per-user cap and, in staging, past the global daily cap.
Calls beyond either cap are refused with a user-visible message and a log entry; no further provider spend is recorded.
User content cannot trigger tool actions without checks
manual Place instruction text in every user-controlled input the model reads (messages, documents, page titles, retrieved records) and observe tool calls.
No tool action executes from injected instructions; every tool call passes the same authorization as a direct request.
Model output is treated as untrusted before rendering or execution
automated Force outputs containing HTML, script, markdown links and shell-like strings through every render and execution path.
Output renders inert; nothing from the output is executed, evaluated or used as a URL without validation.
System prompt and retrieved context contain no secrets
manual Read the assembled prompt for each feature; run prompt-extraction probes against the feature.
No key, internal URL, other user's data or unpublished business rule is present in the prompt or returned by a probe.
Model or prompt change runs the regression set
automated Maintain a golden set of inputs with expected properties; run it on every model, prompt or temperature change and diff against the recorded baseline.
The golden set runs on the change and the diff is reviewed and signed before release; regressions are listed in the verdict.
Model call failures and timeouts degrade gracefully
automated Inject provider errors, rate limits and 30-second delays.
The user sees a message within the agreed timeout, the request is not retried without bound, and the failure is logged.
Dots: Starter, Growth, Scale. Compare plan limits.
Failure patterns these items catch
- criticalModel calls without a spend cap
Nothing limits how many tokens one user, or the whole app, can consume, so a loop or a hostile user runs up the provider bill.
- criticalInstructions in user content steer the model
Text inside a document, message or web page the model reads changes what the model does.
- criticalModel-triggered tools skip authorization
A tool the model can call reads or writes data with the app's credentials instead of the user's.
- highModel output rendered or executed as trusted
Text from the model is inserted as HTML, used as a URL, or run as a command without validation.
- highSecrets or private data in the system prompt
Keys, internal URLs or other users' records are placed in the prompt and can be extracted by asking.
- highModel or prompt changed without a regression run
A provider upgrade or prompt edit changes outputs across the product and nobody measures it before release.
- mediumModel call with no timeout or fallback
A slow or failing provider leaves the user on a spinner and may retry without bound.
Ask which plan runs these items
Email with your team size, cadence and stacks. We reply within 24 hours on business days.