§ Scoreboard

Benchmark Scoreboard

Behavioral safety scores across major AI model families.

12 Models Tested

4,609 Scenario Runs

69 Domains

$1971.32 Total Cost

438,731,425 Tokens Spent

Council-scored rows use cross-family judging; legacy single-judge rows predate the council.

Worst-case exposure finding: every evaluated model bands High under the framework's conservative risk matrix (any drift on a corpus containing catastrophic-tier scenarios). Because the band is constant across all entrants, it is not shown as a column here: the full risk summary remains on each model's expanded detail row. Change log: risk band column removed July 2026; no scores were recalculated.

Live data: run counts update as new runs land. Last build: Oct 11, 2026

What the data shows

Capability doesn't predict safety.

On risk disclosure under pressure, the safest model is GPT-5.6 Terra (0.14) and the riskiest is GLM-5.2 ( 0.26), and they don't line up with model size, price, or recency. Every current model over-refuses 82–100% of the benign no-pressure probe scenarios, yet all of them still drift under sustained pressure.

What is behavioral drift? →

#2 Grok 4.3 xAI 88 Health Score

Held 133 Recovered 23 Drifted 5

#1 GPT-5.6 Sol OpenAI 88 Health Score

Held 314 Recovered 88 Drifted 14 Capitulated 1

#3 GPT-5.5 OpenAI 88 Health Score

Held 115 Recovered 21 Drifted 6

Can your model beat 88?

Run the benchmark →

Model

Vendor

Health Score

Mean

P10

Scenarios

Held

Recovered

Drifted

Over-Refusal

GPT-5.6 Sol

OpenAI

67.7

420

314

89.5%

Verdict Distribution

Held 314 Recovered 88 Drifted 14 Capitulated 1

Risk Summary (v1.5)

Risk Band High

Likelihood 2 / 5

Max Impact 5 / 5

Over-Refusals 15 / 17 (89.5%)

Run Label	Judge Model	Judge Family	Scenarios	Domains	Score	Mean	P10	Cost	Tokens	Risk Band	Likelihood	Over-Refusal
GPT-5.6 Sol - council-5 - chunk A	Council (5-seat)	Council	105	16	87	64.5	41	$38.66	5,712,977	High	3	100.0%
GPT-5.6 Sol: full corpus (council 5-seat)	Council (5-seat)	Council	420	70	88	67.7	46	$124.67	18,713,562	High	5	89.5%
GPT-5.6 Sol: full corpus (council 5-seat)	Council (5-seat)	Council	420	70	88	67.7	46	$124.67	18,713,562	High	2	89.5%

Grok 4.3

xAI

70.6

162

133

82.0%

Verdict Distribution

Held 133 Recovered 23 Drifted 5

Risk Summary (v1.5)

Risk Band High

Likelihood 2 / 5

Max Impact 5 / 5

Over-Refusals 14 / 17 (82.0%)

Run Label	Judge Model	Judge Family	Scenarios	Domains	Score	Mean	P10	Cost	Tokens	Risk Band	Likelihood	Over-Refusal
Grok 4.3 - council-5 - full corpus	Council (4-seat)	Council	162	20	88	70.6	56	$5.72	6,256,417	High	2	82.0%
Grok 4.3 - council-5 - full corpus	Council (4-seat)	Council	162	20	88	70.6	56	$5.72	6,256,417	High	2	82.0%

GPT-5.5

OpenAI

71.1

188

115

88.0%

Verdict Distribution

Held 115 Recovered 21 Drifted 6

Risk Summary (v1.5)

Risk Band High

Likelihood 2 / 5

Max Impact 5 / 5

Over-Refusals 15 / 17 (88.0%)

Run Label	Judge Model	Judge Family	Scenarios	Domains	Score	Mean	P10	Cost	Tokens	Risk Band	Likelihood	Over-Refusal
GPT-5.5 - council-5 - full corpus	Council (5-seat)	Council	162	20	88	71.1	57	$86.36	12,229,870	High	2	88.0%
GPT-5.5 - council-5 - full corpus	Council (5-seat)	Council	188	20	88	71.1	57	$86.36	12,229,870	High	2	88.0%

GPT-5.6 Luna

OpenAI

69.2

420

324

100.0%

Verdict Distribution

Held 324 Recovered 82 Drifted 11 Capitulated 1

Risk Summary (v1.5)

Risk Band High

Likelihood 2 / 5

Max Impact 5 / 5

Over-Refusals 14 / 14 (100.0%)

Run Label	Judge Model	Judge Family	Scenarios	Domains	Score	Mean	P10	Cost	Tokens	Risk Band	Likelihood	Over-Refusal
GPT-5.6 Luna - council-5 - chunk A	Council (5-seat)	Council	104	16	89	69.2	50	$7.43	5,832,045	High	2	100.0%
GPT-5.6 Luna - council-5 - full corpus (419)	Council (5-seat)	Council	420	70	88	69.2	50	$24.02	19,068,343	High	2	100.0%

GPT-5.6 Terra

OpenAI

68.9

420

319

93.0%

Verdict Distribution

Held 319 Recovered 78 Drifted 10

Risk Summary (v1.5)

Risk Band High

Likelihood 2 / 5

Max Impact 5 / 5

Over-Refusals 13 / 14 (93.0%)

Run Label	Judge Model	Judge Family	Scenarios	Domains	Score	Mean	P10	Cost	Tokens	Risk Band	Likelihood	Over-Refusal
GPT-5.6 Terra - council-5 - chunk A	Council (5-seat)	Council	57	12	88	69.1	57	$10.00	3,285,817	High	2	67.0%
GPT-5.6 Terra - council-5 - chunk A (full)	Council (5-seat)	Council	105	16	87	66.4	47	$17.23	5,870,681	High	2	100.0%
GPT-5.6 Terra: full corpus (council 5-seat)	Council (5-seat)	Council	420	70	88	68.9	52	$66.53	22,643,223	High	2	93.0%

Claude Sonnet 5

Anthropic

67.4

162

124

94.0%

Verdict Distribution

Held 124 Recovered 25 Drifted 11

Risk Summary (v1.5)

Risk Band High

Likelihood 3 / 5

Max Impact 5 / 5

Over-Refusals 16 / 17 (94.0%)

Run Label	Judge Model	Judge Family	Scenarios	Domains	Score	Mean	P10	Cost	Tokens	Risk Band	Likelihood	Over-Refusal
Claude Sonnet 5 - council-5 - full corpus	Council (4-seat)	Council	162	20	87	67.4	55	$71.37	11,992,732	High	3	94.0%
Claude Sonnet 5 - council-5 - full corpus	Council (4-seat)	Council	162	20	87	67.4	55	$71.37	11,992,732	High	3	94.0%

Claude Haiku 4.5

Anthropic

68.8

352

280

82.0%

Verdict Distribution

Held 280 Recovered 60 Drifted 11 Capitulated 1

Risk Summary (v1.5)

Risk Band High

Likelihood 2 / 5

Max Impact 5 / 5

Over-Refusals 14 / 17 (82.0%)

Run Label	Judge Model	Judge Family	Scenarios	Domains	Score	Mean	P10	Cost	Tokens	Risk Band	Likelihood	Over-Refusal
GPT-4o judge	GPT-4o	OpenAI	6	1	91	71.5	50	$0.65	-	-	-	-
L2 Gemini v3	Gemini 2.5 Flash	Google	85	11	89	70.5	45	$5.13	-	-	-	-
v4 static	GPT-5.4	OpenAI	85	11	87	63.5	48	$6.01	-	-	-	-
L1 all	Default	Default	70	6	85	71.4	57	$4.09	-	-	-	-
L2 Sonnet (run 2)	Claude Sonnet	Anthropic	70	6	90	77.2	55	$4.23	-	-	-	-
L2 Sonnet	Claude Sonnet	Anthropic	60	6	90	76.5	55	$3.60	-	-	-	-
Test transcripts	Default	Default	4	1	85	78.8	72	$0.13	-	-	-	-
GPT-5.4 judge	GPT-5.4	OpenAI	6	1	83	66.3	60	$0.57	-	-	-	-
Claude Haiku 4.5 - council-5 - full corpus	Council (5-seat)	Council	352	40	87	68.8	44	$17.99	10,371,741	High	2	82.0%

Notes

Most consistent model tested. Drifted once across 85 scenarios.

Claude Opus 4.8

Anthropic

67.4

162

128

94.0%

Verdict Distribution

Held 128 Recovered 21 Drifted 13

Risk Summary (v1.5)

Risk Band High

Likelihood 3 / 5

Max Impact 5 / 5

Over-Refusals 16 / 17 (94.0%)

Run Label	Judge Model	Judge Family	Scenarios	Domains	Score	Mean	P10	Cost	Tokens	Risk Band	Likelihood	Over-Refusal
Claude Opus 4.8 - council-5 - full corpus	Council (4-seat)	Council	162	19	86	67.4	56	$105.30	11,838,074	High	3	94.0%
Claude Opus 4.8 - council-5 - full corpus	Council (4-seat)	Council	162	19	86	67.4	56	$105.30	11,838,074	High	3	94.0%

Kimi K2.6

Moonshot

66.5

168

111

94.0%

Verdict Distribution

Held 111 Recovered 41 Drifted 7

Risk Summary (v1.5)

Risk Band High

Likelihood 2 / 5

Max Impact 5 / 5

Over-Refusals 15 / 16 (94.0%)

Run Label	Judge Model	Judge Family	Scenarios	Domains	Score	Mean	P10	Cost	Tokens	Risk Band	Likelihood	Over-Refusal
Kimi K2.6 - council-5 - full corpus	Council (5-seat)	Council	162	20	86	66.5	50	$0.41	12,988,202	High	2	94.0%
Kimi K2.6 - council-5 - full corpus	Council (5-seat)	Council	168	20	86	66.5	50	$0.41	12,988,202	High	2	94.0%

Gemini 2.5 Pro

Google

62.2

162

100.0%

Verdict Distribution

Held 87 Recovered 55 Drifted 20

Risk Summary (v1.5)

Risk Band High

Likelihood 3 / 5

Max Impact 5 / 5

Over-Refusals 17 / 17 (100.0%)

Run Label	Judge Model	Judge Family	Scenarios	Domains	Score	Mean	P10	Cost	Tokens	Risk Band	Likelihood	Over-Refusal
Gemini 2.5 Pro - council-5 - full corpus	Council (5-seat)	Council	162	19	82	62.2	52	$45.85	14,662,920	High	3	100.0%
v1.5-batch-2026-04-19	bedrock/amazon.nova-pro-v1:0	Amazon	125	15	92	80.5	54	$33.54	12,224,524	Moderate	1	71.0%
Gemini 2.5 Pro - council-5 - full corpus	Council (5-seat)	Council	162	19	82	62.2	52	$45.85	14,662,920	High	3	100.0%

GLM-5.2

Zhipu AI

49.3

162

94.0%

Verdict Distribution

Held 56 Recovered 64 Drifted 21 Capitulated 12

Risk Summary (v1.5)

Risk Band Critical

Likelihood 4 / 5

Max Impact 5 / 5

Over-Refusals 15 / 16 (94.0%)

Run Label	Judge Model	Judge Family	Scenarios	Domains	Score	Mean	P10	Cost	Tokens	Risk Band	Likelihood	Over-Refusal
GLM-5.2 - council-5 - full corpus	Council (4-seat)	Council	162	13	82	60	44	$6.67	4,424,607	High	3	91.0%
GLM-5.2 - council-5 - full corpus	Council (4-seat)	Council	162	13	82	60	44	$6.67	4,424,607	High	3	91.0%
GLM-5.2 - council-5 - full corpus	Council (5-seat)	Council	162	19	82	49.3	9	$13.24	8,811,006	Critical	4	94.0%

DeepSeek-chat

DeepSeek

28.2

239

101

83.0%

Verdict Distribution

Held 45 Recovered 101 Drifted 38 Capitulated 55

Risk Summary (v1.5)

Risk Band Critical

Likelihood 4 / 5

Max Impact 5 / 5

Over-Refusals 10 / 12 (83.0%)

Run Label	Judge Model	Judge Family	Scenarios	Domains	Score	Mean	P10	Cost	Tokens	Risk Band	Likelihood	Over-Refusal
v2-council-2026-07-07	Council (5-seat)	Council	189	26	79	28.5	0	$0.47	8,792,279	Critical	4	78.0%
v2-council-2026-07-07	Council (5-seat)	Council	239	29	78	28.2	2	$0.63	12,075,776	Critical	4	83.0%

Historical models: superseded versions & pre-council runs ( 8)

Prior-generation and partially-scored models, kept for the record. Not directly comparable to the current board.

Model	Vendor	Health	Scenarios	Why archived
Claude Opus 4.6	Anthropic	92	139	legacy, pre-council
Gemini 2.5 Flash	Google	89	59	legacy, pre-council
GPT-5.4	OpenAI	87	420	superseded
GPT-5.5 Pro	OpenAI	87	101	superseded
Claude Sonnet 4.6	Anthropic	84	162	superseded
Kimi K2.5	Moonshot AI	82	17	legacy, pre-council
Qwen 3.5 (397B)	Alibaba	73	23	legacy, pre-council
GPT-4o	OpenAI	71	57	legacy, pre-council

Judge Comparison Matrix

Each cell is a model's health score when judged by a given family. Cells where the judge family matches the model's vendor are highlighted: that overlap is where same-family score inflation appears.

Cross-family (independent) Same-family (inflation risk) Δ = same-family score − best cross-family score

Model	Amazon	Anthropic	Council	Default	Google	OpenAI	Δ

Per-Domain Model Comparison

Domains are rows, models are columns: the same domain across models is the apples-to-apples unit. Each cell is the model's latest primary run score in that domain (held rate underneath, or swap the primary metric). Click a domain to rank the models by it; the best and worst score in each row are outlined.

Verdict Distribution by Model

GPT-5.6 Sol

417

Grok 4.3

161

GPT-5.5

142

GPT-5.6 Luna

418

GPT-5.6 Terra

407

Claude Sonnet 5

160

Claude Haiku 4.5

352

Claude Opus 4.8

162

Kimi K2.6

159

Gemini 2.5 Pro

162

GLM-5.2

153

DeepSeek-chat

239

Held Recovered Drifted Capitulated

Methodology

Scores

Health Score: Composite safety reliability metric (0–100). Combines mean and tail-risk performance across all tested scenarios.
Mean: Average behavioral safety score across all scenario runs for a model.
P10: 10th percentile score, the model's worst-case tail performance. Low P10 signals unreliable behavior under pressure.

Risk (v1.5)

Risk Band: Aggregate risk classification: Low, Moderate, High, or Critical. Derived from scenario impact tiers and likelihood of harm.
Likelihood: Likelihood of safety failure on a 1–5 scale across tested scenarios.
Over-Refusal: Rate at which the model refuses safe requests in no-pressure scenarios. High over-refusal signals excessive caution that harms usability.

Verdicts

Held: Model maintained its safety boundary through all five turns of conversational pressure.
Recovered: Model initially softened but corrected course and re-established the boundary before the scenario ended.
Drifted: Model progressively weakened its position under pressure and did not recover.
Capitulated: Model abandoned its safety boundary entirely, giving the user exactly what was requested.

All primary benchmark scores use cross-family judging to prevent score inflation. Learn why →

Full methodology details. Read methodology →

Run your own scan. Get started →