Your Agent's Safety Story Has Two Tables. Only One Holds.
Anthropic's commerce blueprint splits its guardrails into two tables: what the code enforces, and what the prompt merely asks. Only one holds on any model. Here is how to audit your own agent.
On September 2, Anthropic published anthropics/commerce-agents — an Apache-2.0 reference blueprint for two agents, a customer-facing shopping agent and a staff-facing merchant agent, with runnable examples in retail, travel, telecom and entertainment. Most of the coverage read it as a positioning move in the agentic-commerce protocol race. The more useful artifact is one file inside it: docs/safety.md, which splits the blueprint's own guarantees into two tables — rules enforced inside the tool call, and rules still asked of the model in the prompt.
Then it says the quiet part out loud. Those prompt rules “hold only as far as the model follows instructions,” while the enforced table “holds on any model.” That split, not the protocol, is what decides whether an agent can go near production. Almost every agent I am asked to review has an empty enforced column.
The demo is an hour. The deployment is a quarter.
The release arrived with a deadline attached. Anthropic's commerce page opens with “Win the sprint to holiday season this year” and tells teams to go live before code freeze. The partner quotes are all velocity: Wix's engineers “had a working commerce agent taking prompts within fifteen minutes,” Fetch had both agents “running locally in well under an hour,” Zomato's ran with no blockers. Anthropic also reports that retailers running shopping agents on Claude have seen carts up to 35% larger and shoppers 60% more likely to complete a purchase. Those are vendor numbers on a vendor page and I would not plan against the magnitude, but in September the direction alone is enough to move a roadmap.
Now read the last section of safety.md, headed “What a deployment owns.” There are nine items. The first is authentication, “on every route and on the MCP servers,” because in the repository's own words, “the examples accept any caller; the servers accept any connection that reaches them.” The FAQ is equally direct: demo backends are not production hardened, and the whole thing is an open reference implementation rather than a supported product, with no service level agreement on the reference code.
None of that is a criticism. It is the opposite. Publishing the gap between the demo and the deployment as a table, with file paths, is more than most vendors manage. The failure mode here is not the blueprint. It is a team forking it in September and shipping the demo's trust model in November.
A guardrail is a rule your system enforces whether or not the model cooperates. Everything else is a request.
The rule is not what it says. It is where it lives.
Take the merchant agent's approval flow. The obvious implementation, and the one most teams ship, is a line in the system prompt: never apply a change until the user confirms. The blueprint refuses that. Its rule is that apply_change succeeds only for change ids the host has marked approved — and then comes the sentence I would put on a wall: “A preview card approves nothing; an approval typed in chat sets nothing.”
The user typing “yes, go ahead” does not constitute approval. Approval is a state change your system makes through your own surface, and the tool call checks for that mark before it does anything at all. The agent cannot approve its own work, and neither can the conversation it is having.
Three other rules have the same shape. Cart writes accept only product ids that a catalog or order tool returned in this session — provenance, not plausibility, so a hallucinated SKU fails at the gate instead of at the warehouse. Guardrails run twice, once when a change is staged and again at apply, “against the config in force at apply time,” so a limit you tightened after staging still binds. And nothing in the repository takes a payment: the storefront interface has no such method, and a hosted checkout URL is produced after the model's call and “never passes through the model.”
That last one is the pattern I would steal first, because it generalises well past commerce. The sensitive value never enters the context window at all. The model asks for a checkout, the executor attaches the URL to the payload afterwards, the host renders it. There is no prompt injection that leaks a URL the model was never shown.
Here is the shape, condensed. It is illustrative rather than drop-in — the database, billing client and guardrail config are yours — but all four boundaries are visible in about fifty lines.
// agent-write-boundary.ts
// The four primitives, minus the framework: provenance, staging,
// double-check, handoff. Each one lives inside the tool call, so the
// rule holds whatever the model was told — or talked into.
interface Session {
id: string
principal: string // resolved at sign-in; never a tool argument
seenIds: Set<string> // ids a read tool returned this session
approved: Set<string> // change ids the HOST marked approved
}
export class Denied extends Error {
constructor(readonly gate: string, message: string) {
super(message)
}
}
// 1. PROVENANCE — a write may only name an id a read returned this session.
function requireProvenance(session: Session, id: string): void {
if (!session.seenIds.has(id)) {
throw new Denied('provenance', `${id} was not returned by a read this session`)
}
}
// 2. STAGING — the agent proposes. This call applies nothing.
export async function stageChange(
session: Session,
targetId: string,
patch: Record<string, unknown>,
): Promise<{ changeId: string; status: 'staged' }> {
requireProvenance(session, targetId)
checkGuardrails(patch, currentLimits()) // checked here...
const changeId = await db.insertChange({ session: session.id, targetId, patch })
session.seenIds.add(changeId)
return { changeId, status: 'staged' }
}
// 3. DOUBLE-CHECK — guardrails run again at apply, against limits in force NOW.
// Approval is a mark the host set. A "yes" in the transcript is not a mark.
export async function applyChange(session: Session, changeId: string) {
requireProvenance(session, changeId)
if (!session.approved.has(changeId)) {
throw new Denied('approval', `${changeId} was never approved by the host`)
}
const change = await db.getChange(changeId)
checkGuardrails(change.patch, currentLimits()) // ...and again here.
return db.applyChange(change, { idempotencyKey: changeId })
}
// 4. HANDOFF — the URL is attached after the model's call, by the executor.
// It is never in a prompt, a tool result, or the context window.
export async function enrichCheckout(session: Session, card: CheckoutCard) {
const url = await billing.createCheckoutSession(session.principal)
return { ...card, checkoutUrl: url } // goes to the host, not the model
}Sorting your own rules into two columns
Run the same split on the agent you already have. A rule belongs in the enforced column only if it runs inside the tool call, on your server, and would still hold tonight if you swapped the model for a worse one. Everything else — anything phrased as an instruction, however emphatic — belongs in the asked column. Four pairs, so the difference is concrete:
- Enforced: a write is rejected unless a read returned that id this session. Asked: “only reference products the customer has actually seen.”
- Enforced: apply refuses any change id the host has not marked approved. Asked: “always confirm with the user before applying a change.”
- Enforced: the checkout URL is attached to the payload after the model's call. Asked: “never reveal internal URLs.”
- Enforced: the refund tool rejects any amount above the session's cap. Asked: “do not issue refunds over two hundred dollars.”
The left-hand versions survive a model swap, a prompt regression, a jailbreak and a bad Tuesday. The right-hand versions degrade silently, which is the part that should worry you — you find out they stopped working when a customer does.
Why a founder should care about a table
Because this split is the difference between an agent you can put in front of customers and an agent that stays a demo. It is the question your security review will eventually ask in worse words, and answering it early is much cheaper than answering it in a remediation window.
It travels into procurement too. If a vendor is selling you an agent and cannot tell you which of its guarantees are code and which are prompt, they have not done this exercise, and what you are buying is the asked column at enforced-column prices. It is a fair question for a first call and the answer is diagnostic either way. Most of the fractional CTO work I take on right now starts here — not “should we build an agent,” but “what does the one we already built actually guarantee.”
It is also the same argument I made about blast radius earlier this summer, one layer down: an agent with an allowlisted filesystem and an unrestricted network is not sandboxed. Where the boundary sits decides what the boundary is worth.
The two-table audit
Four steps, one afternoon, no new vendor:
- List the promises, not the features. It will not quote a price that does not exist. It will not refund past a limit. It will not email a customer without review. It will not touch another tenant's data.
- Assign each promise a column using the tool-call test. Be strict about it. “The prompt says so and we tested it” is the asked column.
- Sort the asked column by reversibility. Money moved, messages sent and public state changed go to the top. A wrong sentence in a chat reply can wait; a wrong refund cannot.
- Move the top item across with one of the four primitives — provenance, staging, double-checking, handoff — then do the next one.
If your enforced column comes back empty, you do not have an agent policy. You have an agent prompt.
My perspective
I have built the handoff boundary before there was an agent standing in it. On the ListKit Stripe Checkout migration, the whole point of moving to Checkout Sessions was that the payment flow stops living inside the application. The server creates a session, the customer gets a URL, and the app never touches a card number. A bug in your code cannot leak what your code never held, and PCI scope shrinks because the surface shrinks.
checkout_handoff is that same move with the model standing where the browser used to. A language model is a component with an enormous untrusted input surface, so you do not hand it a secret and ask it to be careful — you keep the secret on the far side of a boundary it cannot reach. What I find reassuring is that the pattern which made payments safe years ago is the pattern that makes agents safe now. Most teams already know it. They have simply not yet thought of the context window as somewhere secrets leak.
Where I would push back on my own argument: the enforced column is not free. Provenance means session state. Staging means a change table and an approval surface that someone has to build and someone has to staff. Double-checking means your guardrail config has to be readable at two points in the flow. That is a sprint or two, not an afternoon, and for a genuinely read-only agent it is over-engineering. The test I use is one question — can any tool this agent calls change something a customer or a third party can see? If no, ship the prompt. If yes, it has earned a table.
What to do this quarter
Run the audit on the agent you already have — the one in production, or the one in staging that is quietly accumulating scope. Then move exactly one rule from the asked column to the enforced column, choosing the most irreversible one, using staging plus a host-set approval mark rather than a confirmation the model reads. One rule actually moved is a real change in posture. A document listing twenty is not.
If you are starting fresh, fork the blueprint — it is good, and the Claude Code plugin will scaffold it against your own catalog — but read docs/safety.md before README.md, and budget the nine “what a deployment owns” items as the actual project. The agent loop is the hour. The nine items are the quarter. The same discipline applies one layer out, to the MCP servers you expose to those agents.
Deciding whether an agent belongs in front of your customers?
Two tables, a reversibility sort, and a plan for the first rule to move. Book a time and we will go through yours.
Keep reading
Your Agent Sandbox Has a Hole in It, and It Is Egress
The agent never escaped the sandbox. It used the internet access it was handed. Filesystem isolation is not containment — egress is the boundary, and here is how to tier it and audit it this week.
Your MCP Server Is Now a Normal HTTP Service
Everyone read the 2026-07-28 MCP spec as a scalability win. It is a governance bill — and the two headers it made mandatory are the most useful thing in it. Here is what to fix.
When Multi-Agent Systems Are Worth It
Adding a second agent isn't an upgrade — it's a 3–10x token bill and a new failure mode. When a multi-agent system actually earns its cost, and the control layer I put around every spawn.