Every AI tool needs a brake, a record and an owner
The big red button on a workshop machine isn't there because anyone expects trouble. It's there so nobody has to work out, in the moment, how to make the machine stop. It sits outside the machine's own controls, and anyone standing next to it is allowed to press it.
Microsoft CEO Satya Nadella wants AI systems built the same way. In a post on X on Saturday, October 10, he wrote that it's time "to step back and assess the trust architecture" of AI, as TechCrunch reported. His short version: "Think of it like an emergency brake."
What he asked for
According to TechCrunch, Nadella's approach "means separating the model from the harness that orchestrates its work," along with "externalizing controls and safeguards." In plain terms, the AI model shouldn't be the thing that decides whether it gets to keep going. The controls live outside it, in the software around it.
He named two more pieces. Every "meaningful model action" should leave "tamper-proof human readable evidence": a record a person can read and the AI can't quietly change. And "an authorized person" should always be able "to pause or shut down a model mid-task."
Put together, that's three parts:
- A brake that sits outside the AI tool, not inside its instructions.
- A record of what the tool did, kept where the tool can't edit it.
- A person whose job it is to pull the brake, and who's allowed to.
None of this is a rule anyone has to follow. It's one large company's chief executive describing what good controls look like. But when that description fits on an index card, it's a fair sign that "who can stop it, and how?" is becoming a normal question to ask about any AI tool, not a specialist one.
What it looks like for a small team
Many teams building their own AI tools have one or two of these parts already. Fewer have all three. Common gaps are easy to spot:
- The only way to stop the tool is to ask it to stop, in the same chat or workflow it's busy in. That's a request, not a brake.
- There is a log, but the tool writes it, in a place the tool can also edit or delete.
- Everyone assumes someone else is keeping an eye on it. Nobody's name is next to it.
None of the fixes need special software. A brake can be as plain as switching off the tool's access key or pausing its scheduled job, done from somewhere other than the tool itself, by more than one person who knows how. A record can be sent somewhere the tool can add to but can't change. An owner is a name written down, plus a second name for when the first one is away.
The brake is different from the instructions you give an AI agent. Instructions that tell it when to give up are worth having, but they live inside the agent and only work if it follows them. The brake is for the moment it doesn't. You want both.
Pull it once
Pick one AI tool your team runs that does things rather than just answering questions: it sends messages, updates records, moves files or works through websites. Write down three answers:
- Who can stop it, and how, without asking the tool?
- Where is the record of what it did, and could the tool change that record?
- Whose name is on it, and who covers when they're away?
Then do the part that usually gets skipped. Start the tool on a harmless test task, and partway through, use the brake. Note how long it takes to actually stop, and whether the record shows what it had done up to that point.
If the honest answer to any of the three questions is "we'd have to find whoever set it up," that's the thing to fix first. A brake nobody has ever pulled is still a guess.
