Bryan Falkowski All writing

Who Owns the Process? Spec Frameworks, Skills, and the Trust Ladder for Coding Agents

Bryan Falkowski · October 23, 2026 · Updated October 31, 2026

Spec frameworks tell a coding agent what to build. Skills teach it how to work. Orchestration runs more of it at once. None of them tells you whether the work is right.

Verification does that, and a well-constrained codebase shrinks how much there is to verify. Trust grows with that evidence, and evidence is what lets you hand an agent more work.

A pile of Java that compiled

In early June 2025, I spent two days at AWS headquarters in Crystal City, Virginia, working with Amazon Q IDE and an early prototype of AI-DLC, the AI-Driven Development Life Cycle. We iterated on acceptance criteria, generated a specification from them, refined it, and used it to drive implementation. The spec was no longer documentation written after the fact. The agent helped write it and then built from it.

I tried it on the Amtrak home, on real code: refactoring something that ran inference on Claude Instant into an agentic workflow with tool calls, which would have been cutting edge at the time. The code kept coming, so much of it, and it all compiled. Then it never returned a response.

So I did standard troubleshooting: I added logging statements and went through the Log4j files. It was my first time with an agent like this, in the early days of these tools, and running more than one at a time had not crossed my mind. I had one agent, a pile of Java code that compiled, and no way to tell whether the work was any good except to dig through it myself.

Coordination came later. How many agents should run, with which tools, on which model, and how should the work be divided? Those questions now get most of the attention. The one that pulled me into evaluation is the one from the train: how do you tell a convincing explanation from a correct result, and how much evidence do you need before you let an agent do more on its own?

Specs and skills solve different problems

A specification says what to build: the behavior, the constraints, the acceptance criteria. A skill says how to work: run TDD, investigate a bug, answer the open questions in a plan before writing code. A good spec does not tell you how to implement a change, and a good skill does not tell you what the change should accomplish. GitHub Spec Kit and Matt Pocock's skills show the difference.

Spec Kit: the process owns the work

Spec Kit's workflow can set project-wide rules, define a feature, produce a technical plan, break it into tasks, and implement them. The documents are connected, so the code is not the only record of what was intended. A team can review a change before it is built, agree on requirements, and hand the work to another developer or agent without the original conversation.

The cost is overhead. When a task is exploratory or the requirements keep moving, formalizing every step gets in the way, and a spec that no longer matches the code is one more source of confusion. Spec Kit has added more flexible entry points, so not every use runs through a rigid sequence of phases. Its strength is still the plan as a durable artifact.

Matt Pocock's skills: you own the process

Pocock's skills start from the task instead of a pipeline. Grilling resolves the open questions in a plan. Domain modeling builds a shared glossary and records architectural decisions. TDD drives implementation through red, green, and refactor. Others cover debugging, prototyping, and code review. If an idea is vague, explore it. If the design is uncertain, prototype it. If the change is obvious, skip the planning.

The price is that you decide which process to run and make sure decisions get recorded somewhere durable. That flexibility can also lead to inconsistency.

Both approaches have process. They differ in how much of it is prescribed and who decides what happens next.

The missing piece is verification

None of this guarantees a correct result. A clear spec can be implemented wrong. A sound skill can produce a faulty change. An agent can follow every step and still land on the wrong answer.

Lauren Tan, known as poteto, named my problem in her conversation with Matt Pocock:

Video thumbnail for Matt Pocock's live interview with Lauren Tan (poteto)
LIVE: Poteto (creator of pstack) on shipping 1,000's of PR's a month at SpaceX, Matt Pocock's live interview with Lauren Tan, on YouTube.

"I was still relying on me as the proxy between my agent and the output."

That was me with the Log4j files. Generating the code was automated, but the checking was still mine. Tan's answer is blunt:

"The single most important skill that should be in your toolkit is verification."

A verification skill gives the agent a way to gather evidence about its own output: run the tests, check the types, compare the result with the acceptance criteria, inspect the rendered interface, score the output against a rubric. A passing test suite is evidence, but it does not prove every requirement was met, so different tasks need different checks.

Tan's first verification skill at Cursor taught the agent to run the application and take performance traces through the Chrome DevTools protocol. It includes a CLI, so agents stop writing different scripts each session. In Tan's telling, verification is mainly about correctness: does the feature do what you want? Code quality takes other skills.

The reason verification comes first is the loop:

"If the agent can't actually see the result of its work there's no way it can actually iterate."

An agent that cannot see the result has to assume. One that can compare the result with what was intended can correct itself without a human in the middle. Tan calls verification "the most important part of a loop that allows it to be a loop."

Without it, more orchestration just means more output for someone to review. Spawn a hundred agents without trust, Tan says, and "you're just going to get a ton of slop pull requests."

Lauren Tan's trust ladder

Pocock opens the interview with Tan's trust ladder: as you trust agents more, you can give them better work or run more of them. You do not choose your level of trust once. It rises as the system proves itself.

Tan does not name the rungs but describes both ends. Near the bottom, you do not trust your agents' work, so you micromanage them. Near the top, Tan's agents merge their own pull requests overnight, and Tan samples them after they land. Tan credits verification and the environment together for getting there, and warns that it is very hard to reach.

The environment means the codebase and the checks around it, and it does a different job from verification: it prevents mistakes instead of catching them. Tan's talk gives an order to follow whenever you find yourself correcting an agent. Start with the codebase, "the best form of memory," because agents extend the patterns they see, and a refactor can rule a mistake out entirely. Next comes static analysis: lint rules, compiler diagnostics, and CI. That step matters more to me now, because most of my code is Python, where the compiler prevents far less than it did in Java. Then come rules, Bugbot, and skills, which are guidance, because an agent might skip a rule or the person piloting it might ignore it. Last is the style guide, which only human reviewers can enforce.

A verification skill sits outside that order. It does not tell the agent how to behave. It shows the agent what happened.

So trust has two sources, and they feed each other. Each failure you catch can become a constraint, which leaves less to catch next time. The way I would put it: trust grows with evidence, and the environment shrinks what the evidence has to cover. Get both right, Tan says, and your agents "can just be free."

When the check is a score

One good demo does not earn a step up the ladder. You need to know how often the approach works, where it fails, and whether your checks catch those failures.

That gets hard when the check is a score. Tan describes hill climbing, where "you have some kind of rubric or a way to judge or score something" and an agent keeps trying to improve against it. An agent climbing a bad score just gets better at the wrong thing.

That is the problem my work on LLM judges is about. In one eval, I called the same judge twice on each of 600 items from clean runs and compared its votes with a deterministic score, which I treated as the reference. The two votes disagreed with each other on 3.5% of items. The first vote disagreed with the reference on 24.5%. A repeat check would have seen a stable judge that was wrong about one time in four. Those numbers come from one judge, one prompt, and one dataset, so they are not general rates, but they show that self-consistency cannot stand in for a reference.

A judge needs a reference that does not come from the judge: a deterministic score where one exists, or a small set of human-labeled items scored on every run. TPR and TNR measure a judge against those labels. A score is not evidence of quality simply because it is numerical.

This is how I apply Tan's ladder to evaluation: trust should grow with the evidence we have, not simply with how impressive the latest model looks.

None of this removes human judgment. It moves it toward ambiguous requirements, hard trade-offs, novel situations, and failures that need investigation, and away from checking work a machine can check.

Where the tools fit

Most of these tools now mix the layers. Kiro and Spec Kit center the workflow on specifications. AWS AI-DLC has AI propose and execute stages while people validate decisions. OpenSpec, BMAD, and GSD structure specification and implementation in their own ways. Superpowers is packaged as skills but can prescribe a defined process, so a skill pack does not always mean an optional workflow. Pocock's library reaches into execution too: implement-spec runs implementer subagents in parallel across a spec's tickets. Tan's pstack focuses on how agents carry out work, how roles are assigned, and how results are reviewed or compared.

No one layer replaces the others.

ToolIntentdefine the workProcessguide the workExecutioncoordinate agentsEvidenceverify the result
KiroYes
Spec KitYesYes
OpenSpecYes
BMAD MethodYesYes
AWS AI-DLCYesYes
GSDYesYes
SuperpowersYesYes
Pocock's skillsYesYesYes
pstackYesYes
Where each tool puts its main focus, based on its own documentation. A dot means the tool is built around that layer, not that it ignores the others. Evidence has the fewest dots. My read is that a skill can teach an agent how to verify, but the checks that matter still have to be built for each codebase.

Tan's order also limits what any of these tools can do. Skills are guidance, so adding a skill pack does not give you the strongest constraints. My read is that no spec framework or skill pack can supply those. They have to be built into your own codebase and CI.

A practical way to combine them

For an exploratory project, I would start with skills, because the requirements will change as I learn. A grilling session sharpens the idea, a prototype answers a design question, and TDD comes in once the behavior is clear enough to preserve.

For a shared feature with explicit requirements and handoffs, I would use a specification framework. People and agents need to agree on what is being built and how success will be judged.

For a behavior-preserving refactor, I would care most about tests and other checks showing that existing behavior survived. A feature spec may help with scope, but it is not the evidence.

In all three, verification starts at the beginning, not as a green checkmark after the important decisions are made. And I would keep one authoritative home for the plan, so the agent never has to choose between conflicting versions of the truth.

So who owns the process?

A framework can own the steps. You own the choice of which steps to run and the environment they run in.

Tan and Pocock treat their skill libraries as complementary, but Tan also says "everyone should have their own set of knives." Go through your own past chats, find where you had to intervene, and turn those moments into lint rules or skills.

The goal is justified trust

The debate over frameworks and skills matters, but the bigger gain comes when the system can check its own work and use the result to improve it.

The obvious objection is that an agent checking its own work is marking its own homework. It is a fair one. My judge was stable and still wrong, and an agent's own checks carry the same risk, so the loop needs an outside reference: a deterministic check, or labeled examples the agent did not produce. Some domains do not have one. Asked about changes that cannot be undone, Tan says it comes down to how verifiable the domain is, and calls it "a great question that I don't really have the answer to."

  1. Specificationestablishes intent
  2. Skillscapture process
  3. Orchestrationcoordinates execution
  4. Verificationcloses the loop

↻ Evidence from verification flows back into the next attempt, into the environment, and into how much the agent is trusted to do

Tan's ladder is about giving agents more responsibility as confidence grows. My interest is in making that confidence measurable.

Do not trust an agent because it can write code, follow a workflow, or produce a persuasive explanation. Trust it because you have evidence that the work meets the requirements, that the checks are meaningful, and that failures are likely to be caught.

References