Your agent wrote the code and the tests.
That is the problem. One pass generated both from the same reading of the spec, so when the reading was wrong the tests agree with the mistake. This is what we do about it.
The failure mode, stated plainly
In traditional development, a developer who misunderstood a requirement usually catches it. Conversation, code review, or QA surfaces the gap.
With an agent the failure looks different. A developer prompts for an implementation. The agent generates the code and the tests from the same reading of the spec. The developer ships it. The implementation is wrong, the tests pass, and the misreading never surfaces.
This is not a defect in the model. It is a structural property of any system that generates both the artifact and the check in one pass. It is the specific thing this standard exists to prevent: code that passes its own tests because the same misunderstanding generated both.
Accountability does not move. The agent is the tool, not the deliverer. The person who signed off is answerable for what shipped.
Two kinds of validation
Product validation proves the build matches the spec.
System validation proves the agent behaved correctly, that it interpreted the spec as intended, did not drift, and produced auditable behavior.
A project needs both. Validation is not a safety net for AI mistakes. It is the mechanism that makes AI-First delivery auditable at all.
There is a longer arc here. As this way of working matures, the work shifts from writing code, to reviewing generated code, to designing the systems in which agents write and review code. At each step the person moves further from the artifact and catches less by intuition. The validation system has to cover that growing distance.
The behavior contract
Quality designs what must be proven. Engineering decides where each scenario physically lives. That split is deliberate, and it is the opposite of the traditional bottleneck.
Generating a test at any layer is now cheap. The old constraint, that there was never time to automate, has collapsed. What is left, and what does not automate, is deciding what to prove and where an agent is likely to diverge from the spec.
Every scenario in the plan carries four tags.
| Tag | Values | What it answers |
|---|---|---|
| Section | Happy path, validation rules, edge cases, accessibility | What kind of behavior this covers |
| Run purpose | Smoke, functional, regression, usability | When it runs, and in which pipeline |
| Severity | Critical, high, medium, low | What to fix first when it fails |
| Compliance risk | An optional note per scenario | Where an agent could plausibly diverge from the spec |
Run purpose describes when a scenario runs, not how it is built. A scenario can carry more than one. The point is to enable subset runs.
Severity is independent of the other two. A happy-path scenario can be critical if it is the purchase flow, or medium if it renders an avatar. An edge case can be high if it loses data silently, or low if it is cosmetic.
The compliance risk note is the sharp part
It is required wherever an agent is plausibly likely to ship something that passes the literal scenario and violates the spec's intent. It names the specific wrong implementation the agent might choose.
A real one: if the spec says the list endpoint returns items already sorted by the backend, an agent may sort the response correctly and still render it in insertion order. So the scenario asserts the visible order, not the shape of the data.
Spec compliance testing
Standard tests verify that code does what the code does. Spec compliance tests verify that code does what the spec said.
The rule: for every feature shipped through an agent-driven build, at least one test verifies a spec requirement the build's own generated tests did not cover. A different author, for a different purpose.
Where these earn their keep:
| Scenario | What to test |
|---|---|
| Data scoping | A query returns only the authenticated user's records, not all of them |
| Authorization | An action blocked in the interface is also blocked at the API and the data layer |
| Default state | A feature initializes to the specified default, not to the first option or a code-level default |
| State persistence | A selection survives navigation |
| Negative path | Invalid input is rejected, not silently accepted, defaulted, or crashed |
| Contract | Field names, types, sort order and shape match the documented contract, not just a success code |
Who owns what, and when
Each artifact has one owner and one moment.
| Artifact | When | Owned by |
|---|---|---|
| The spec | Before build | Product |
| The validation plan: scenarios per criterion, tags, edge cases, traceability | Before build | Quality |
| The stack-layer mix: which scenarios run as unit, component, integration, end to end, or visual | In the plan | Engineering |
| The test code | During build | Engineering, generated under supervision |
| Review of whether the suite covers the spec | After build | Engineering |
| Implemented spec compliance tests | After build | Quality, targeting the gaps the compliance-risk notes predicted |
| Manual testing | After build | Quality, as a calibration tool and not as primary coverage |
The boundary that matters: the build agent's tests verify the implementation is internally consistent. Quality's tests verify it matches the spec. Both are automated. Different authors, different purpose.
Everyone in this workflow is working with an agent. Product with a spec agent, quality with a planning agent, engineering with a build agent. When this says "the build agent's tests", it means the one the developer drove during the build.
Coverage scales up, never down
Coverage should match the complexity and criticality of the work. A copy change and a new authentication flow do not need the same suite, but both need one.
When in doubt about whether a behavior needs a test, write it. Generating the test is cheap now. Missing the behavior is not.
The gates every change clears
Gates are the baseline, not the coverage. Feature tests prove correctness. Gates enforce the floor.
| Gate | What it enforces |
|---|---|
| Lint | Zero errors, not zero warnings |
| Type check | No type errors and no suppressions |
| Unit suite | Everything passes |
| Coverage threshold | A minimum at project level, higher on security-critical paths |
| Static security scan | Zero warnings |
| Dependency scan | No high-severity vulnerabilities in production dependencies |
A project-level coverage threshold is a lagging indicator. Auth, authorization middleware, payment processing and key enforcement need near-complete coverage regardless of what the project average says.
Gates must fail the build. A gate that is installed and not enforced is decoration. If one is disabled, the reason is documented and the date it comes back is tracked. We have audited a project whose pipeline invoked a test command for a framework the team did not use: it found nothing, ran nothing, and reported green on every merge while 93 test files sat invisible. That is what an unenforced gate looks like from the outside.
Traceability
Every criterion maps to a test that proves it. The spec's job is to ensure the criterion exists before the build. The plan's job is to map it. A criterion with no test is not done, and a test with no criterion is not evidence.
The specification standard covers the other half of this, and the readiness audit is how we find out where a project actually stands against both.
The rule the rest of this follows from
Write the tests before the build, from the spec, not from the implementation.
A test written from the spec asks whether this implementation does what was required. A test written from the implementation asks whether it does what it does. Only the first question has an answer worth having.
- A test written from the spec is evidence
- A test written from the build is noise
- A gate installed but not enforced is decoration
- Exit code zero is not proof. Verify artifacts, not signals
Bring us the backlog.
In 30 minutes, we will show you what a Pod would ship first and how we would price it.