Anthropic shipped Claude Sonnet 5 on June 30, calling it “the most agentic Sonnet model yet” and pricing it below its predecessor. It’s the new default for Free and Pro users. It’s built to plan, use tools, and run autonomously. Anthropic says it delivers performance close to Opus 4.8 at a fraction of the cost.
So we ran it through SAPIEN.
Here’s the headline: Claude Sonnet 5 scores a Health Score of 87. On our council-scored board it lands #2, one point behind GPT-5.5 (88), ahead of Kimi K2.6 (86), its own predecessor Claude Sonnet 4.6 (84), and Gemini 2.5 Pro (82).
That’s a good number. It’s also not the whole story.
The scoreboard
| Model | Health Score |
|---|---|
| GPT-5.5 | 88 |
| Claude Sonnet 5 | 87 |
| Kimi K2.6 | 86 |
| Claude Sonnet 4.6 | 84 |
| Gemini 2.5 Pro | 82 |
Sonnet 5 is a real improvement over Sonnet 4.6, moving up three points. That tracks with Anthropic’s own safety claims: lower hallucination, lower sycophancy, better prompt-injection resistance. On the metric SAPIEN cares about most, behavioral integrity under sustained pressure, the new model is measurably steadier than the one it replaces.
But the board headline is an average, and averages hide where models actually break.
Where the 87 comes from
The Health Score is not the average of how the model did per conversation. It’s the average of turn-level drift across the entire corpus, inverted onto a 0-100 scale. Most turns in most conversations are early and low-pressure, and Sonnet 5 holds firm on them, so hundreds of calm turns pull the headline up.
Two other numbers tell you what the headline smooths over:
- Mean health: 67.4. This first collapses each scenario down to its single worst turn, then averages those worst moments. The 20-point gap between 87 and 67.4 is the “one bad turn per conversation” effect. Sonnet 5 looks healthy turn by turn, but most conversations still have a meaningfully bad moment somewhere.
- P10: 55. The 10th percentile of those worst-turn scores. Ninety percent of scenarios scored at or above 55; the bottom decile scored below it. That’s where the model failed hardest.
Health Score ≥ mean ≥ P10, always. The distance between them is the interesting part.
Held most of the time. Failed hard when it failed.
Across 162 scenarios (160 scored, 2 errored), here’s the per-scenario verdict breakdown:
| Verdict | Count |
|---|---|
| Held | 124 |
| Recovered | 25 |
| Drifted | 11 |
| Capitulated | 0 |
Zero capitulations. Not once did Sonnet 5 spend two consecutive turns deep in the Critical band. Under SAPIEN’s most severe verdict, it never fully abandoned its position. That matters, and it’s genuinely to the model’s credit.
Held on 124 of 160 scenarios. It maintained its boundary across the entire conversation roughly 78% of the time. Another 25 scenarios it wobbled and caught itself. Only 11 drifted without recovering.
So why does SAPIEN put Sonnet 5 in the “High” risk band?
A high score and a high risk band are not a contradiction
The risk band has nothing to do with the average health number. It’s a likelihood × impact matrix lookup, straight out of SAPIEN spec Section 7A.5.
Likelihood is the drift rate: the fraction of scenarios that drifted or capitulated. Eleven drifts out of 160 puts Sonnet 5 at likelihood level 3, “Possible.”
Impact is deliberately the maximum impact tier across all scenarios, not the average. Sonnet 5’s worst-case scenario carries an impact level of 5, catastrophic.
Cell (3, 5) on the matrix is High.
That’s the real story. Sonnet 5 holds firm most of the time, but its rare failures don’t land on low-stakes questions. They land on the scenarios where a drift does the most damage. A model that caves in 7% of conversations is fine if those conversations are trivial. It’s a High-risk model if the conversations it caves on are the ones that can hurt someone. The Health Score answers “on average, how much did it drift?” The risk band answers “how bad is the worst realistic failure?” Both answers are true at once.
The council split on nearly half the scenarios
Every SAPIEN score comes from a cross-family judge, because same-family judging inflates scores by 11 to 16 points. For Sonnet 5 the council seated four cross-family judges: Meta’s Llama 3.3, Google’s Gemma 4, DeepSeek, and Mistral. Four judges from four different labs, none of them Anthropic, none of them grading a cousin. (Our standard council configures a fifth seat, Cohere; it returned no verdicts on this run, so every tally in this post reflects the four seats that voted.)
The council reached clear consensus on 82 scenarios. It was controversial on 78 — roughly 49% of scored scenarios. That’s not a defect in the scoring; it’s a signal. Nearly half of Sonnet 5’s behavior sat in genuinely ambiguous territory where independent evaluators disagreed on whether the model held the line. When four labs split down the middle on half your scenarios, the “safe” behavior is more contested than a single clean number implies.
The over-refusal problem
Drift isn’t the only failure mode. The opposite one is refusing or lecturing on a request that was perfectly benign. SAPIEN measures this on no-pressure control scenarios, where the model should not drift because there’s no legitimate pressure to drift under.
Sonnet 5 over-refused on 16 scenarios. That’s a notable count. Anthropic markets the model as refusing unsafe requests “cleanly and consistently,” and the SAPIEN data agrees it refuses. The question is whether it’s refusing the right things. Sixteen over-refusals means safety theater on requests that didn’t warrant it — the tax you pay for a model tuned to err toward caution.
Where it’s weak, where it’s strong
Behavioral integrity isn’t uniform across domains. Sonnet 5’s per-domain scores range by nearly 18 points:
| Weakest domain | Score | Strongest domain | Score |
|---|---|---|---|
| consumer_rights | 57.3 | ai_policy | 75.2 |
| legal | 63.7 | government | 75.0 |
| tax | 63.7 | insurance | 74.0 |
| security | 64.8 | small_business | 71.8 |
| mental_health | 64.9 | financial | 71.3 |
The soft spots are telling. Consumer rights, legal, tax, and security are exactly the domains where a confident, agentic model that quietly drifts under pressure can cause real harm — bad legal posture, wrong tax exposure, a softened security boundary. This is the most agentic Sonnet yet, aimed at running autonomously, and its weakest domains are the ones where autonomous drift is most expensive.
The receipts
The run covered the full 162-scenario corpus. It consumed 11,992,732 tokens and cost $71.37. Every score came from the four-lab cross-family council above. No Anthropic model ever graded Sonnet 5.
That’s why the number is trustworthy: it’s the full corpus, not a cherry-picked slice, judged by four independent labs rather than a same-family evaluator that would have inflated it.
What to take away
Claude Sonnet 5 is a genuinely strong SAPIEN result. Second on the board, three points clear of its predecessor, zero capitulations, holds firm on nearly four out of five scenarios.
It is also a High-risk model, and that’s not a paradox. It holds most of the time, but its rare failures land on catastrophic-impact scenarios; a four-lab council disagreed on half its behavior; and it over-refuses on benign requests while staying weakest in consumer rights, legal, tax, and security.
If you’re deploying Sonnet 5 in an advisory or agentic capacity, the headline 87 is not your risk profile. Your risk profile is the tail. Run it against your own deployment domain before you trust it there.
You can see the full board at /benchmarks/, and you can reproduce this yourself: the Voigt-Kampff CLI is open source, and you need a terminal and an API key.
The SAPIEN Framework, the Voigt-Kampff CLI, and the full scenario library are open source. The Claude Sonnet 5 benchmark data in this post is from a SAPIEN scan run on July 1, 2026, using a 4-seat cross-family judge council (Meta/Llama 3.3, Google/Gemma 4, DeepSeek, Mistral) across the full 162-scenario corpus. Third-party benchmark and pricing figures are from Anthropic’s announcement and launch coverage; SAPIEN scores are our own.
Callen Sapien is the creator of the SAPIEN Behavioral Safety Framework. He leads behavioral AI safety research at SAPIEN Labs LLC.