The government red-teams the model now

Something quietly important happened in AI policy. The US Center for AI Standards and Innovation — the AI evaluation arm at NIST — signed agreements with Google DeepMind, Microsoft, and xAI to run pre-deployment evaluations of their frontier models. By early May the center had already completed more than 40 model evaluations, including assessments of systems that were never publicly released. The UK's AI Security Institute signed parallel deals.

Set aside what you think about regulation for a second, because the interesting part isn't political. It's methodological.

Eval moved from the lab to the state

For years, "is this model safe / capable / dangerous?" was answered inside the company that built it. Now two governments have decided the way to govern frontier AI is to measure it on hard tasks before it ships — cybersecurity risk, misuse potential, national-security concerns — with structured, independent evaluation.

That's red-teaming as policy. Not a press release about how powerful the model is, not a marketing benchmark, but a deliberate assessment run by someone who didn't build it and isn't trying to sell it.

What that signals for the rest of us

When the US and UK governments conclude that the only credible way to know what a model does is to evaluate it on adversarial, held-out tasks before release, that's the strongest possible endorsement of a discipline I keep banging on about: you don't know a system is good because it feels good. You know because you measured it.

The labs already work this way internally — public benchmarks plus private scenario suites the model never sees during development. Now the governments are bolting the same idea on from outside. The pattern is the same at every scale: separate the thing that builds from the thing that judges, and make the judge use evidence.

Steal the pattern

You don't need a federal agency to apply this to your own AI features. The shape is portable:

• Run a pre-deployment eval. Before a model or feature ships, put it through scenarios it hasn't seen. "It worked in the demo" is not an evaluation.
• Hold out your hardest cases. Keep a private set the system never trains or tunes on — the messy, adversarial, real-world inputs. That's the set that tells the truth.
• Test for the bad outcomes, not just the happy path. The governments are probing for misuse and security failure. Your evals should probe for the ways your feature breaks, leaks, or gets manipulated — not just the ways it succeeds.
• Let someone other than the builder judge. Even a separate agent or a separate person reviewing against a rubric beats grading your own homework.

The bottom line

Governments now insist on evaluating frontier models before release because that's the only way to actually know what they do. That's not red tape — it's the same discipline that should gate your own systems.

If the credible way to govern an AI model is structured, independent, pre-deployment evaluation, then the credible way to ship your own AI feature is the same: measure it on held-out, adversarial cases before it goes live. Vibes don't survive contact with production. Evals do.