Assume it is compromised

In July, during the Microsoft earnings call, Satya Nadella gave some advice to companies that use AI. "Keep your harness separate from the model," he said. You should be able to change the model at any time. Most people took it as advice on how to buy. We did too.

Then he said more, and the meaning changed.

In September he wrote a memo to Microsoft employees. A company that moves too fast on AI, he said, "will anyway lose permission to operate." In other words, trust is a license, and a license can be taken away.

On October 10 he posted again. This time he did not talk about principles. He talked about how to build the system. "We must assume a model is compromised and contain it from the start." Keep the model separate from the harness that runs it. Put the controls outside the model. Log every important action as evidence that people can read and nobody can change. Make sure a person can stop the model in the middle of a task. He called it an emergency brake.

Where he put the control

When a CEO says AI must stay under human control, it is not news. Everybody says it. What caught our attention is where Nadella put the control. Not inside the model, as a setting. Not before launch, as a test. In the harness, the code you own around the model, while it runs, with a record.

In July he said to keep the harness separate so you can change the model. In October he said to keep it separate so you can check the model. Same software, two reasons to build it.

Microsoft sells models, a harness and the cloud under both. Keep that in mind when you read him. We also sell something in this space, and we will say what before the end.

What does your harness know about an answer?

Take one example. A customer asks your support agent the fee for an international transfer over a certain amount. The model answers with a number. The answer is fluent, the tone is right and it cites the fee table.

What does your harness know at that moment? If you built it well, quite a lot. It has the full response. It has the documents it sent to the model as context. Maybe it has the probability of each token, and the result of your own classifier.

What it does not have is the answer to the question that matters: is this number supported, and how do I know? No field in the response says it. The context documents tell you what went in, not whether the answer uses them. The token probabilities tell you how sure the model felt, not whether it was right. A wrong number said with confidence gets a high score. So the harness does the only thing it can do. It sends the answer to the customer.

Every team knows this problem and tries something. A stricter system prompt moves the error, it does not remove it. A second model as a judge is a reasonable idea with two problems: the judge makes the same kind of mistakes as the model it judges, and an attacker can fool it through the text it has to judge. It is also too slow when latency matters, as in voice. Open source classifiers are good at the one thing they were trained on, usually prompt injection or toxic content, and say nothing about the rest. A homemade classifier plus some regular expressions catches only what it was trained on. You trained it on the three problems that already paged you, so it knows nothing about the fourth one, the one that has not paged you yet.

Each of these gives you a signal. None gives you a signal the harness can trust enough to act on by itself, for every answer, with a reason it can show somebody later.

Read the list again

One, assume the model is compromised. Two, keep the check outside the model. Three, make the check deterministic, so the same answer always gets the same verdict. Four, record what the check saw. Five, give a person a brake.

Nadella was not writing a product spec. The list is general enough that a monitoring vendor, an evaluation vendor and a guardrail vendor could each say it describes them. But read it as a question instead of a spec. For the answer your harness is about to send right now, how many of those five things are true?

Most of the teams we spoke to this year have good evals in place. They test the model well before it ships. But once it is running, they rely on sophisticated system prompts, LLM judges and basic static classifiers. So the checks are inside the model, or inside a second model, or trained on the past. The record is an application log the same team can edit. The brake is a feature flag somebody has to go and find.

What changes with a signal for every answer

Imagine the harness receives, with every answer, a few scores from a component that is not the model. Is this prompt an injection or a jailbreak? Is this answer supported by the context we sent? Is the model fluent in a place where its own token probabilities say it was guessing? Is this exchange on topic at all? The scores must arrive fast enough to sit in the request path, also for voice. They must come with the words that caused each score, so the explanation can be reproduced. And they must go into a record that the team running the model cannot edit in silence, with a clear rule on who can change the thresholds and who can read the log.

Now the harness does not have to guess. When an answer looks unsupported, it can ask the model again with an explicit request to check the facts. When an exchange is borderline, it can pass it to a person before the customer sees it. And when somebody asks what happened on a Tuesday in March, there is a record, and the record was not written by the thing that failed.

There are also things this signal cannot do, and whoever sells you one should say so. If the fee table the model retrieved was itself wrong, a grounding check will approve the answer, because the answer is faithful to a wrong source. That is a data problem, and nothing in the request path can fix it. A deterministic check can be probed, so the thresholds must be yours and must stay private. And a checker is also a system, with its own false positives and false negatives. So assume the checker is compromised too. Measure it on your own traffic, not on the vendor's benchmark. Run it in annotate mode first, and let it block only after you have seen what it does.

Where we stand

This is what we build. Geodesia G-1 is an independent check that sits between your application and the models it calls. It gives a graded verdict for every prompt and every answer. It can explain any verdict down to the words that caused it. It writes every exchange to a record that shows if somebody tampered with it. It runs as a container inside your environment, with any model, and sends nothing home. The numbers, and the conditions we measured them in, are in the G-1 documentation. We will tell you what G-1 catches and what it does not catch.

So, for the answer your harness is about to send, what does it know? If you want to see one of these signals on your own traffic, talk to us. Bring an answer your AI got wrong.