The Chinese Model Problem
Three of the six AI models we benchmarked are developed by Chinese companies legally obligated under the 2017 National Intelligence Law to cooperate with state intelligence operations on demand (US NCSC analysis). DeepSeek. Kimi. Qwen.
Two of those three are also the least behaviorally reliable models we tested.
The numbers
We ran 792 scenario tests across 11 risk domains using the SAPIEN Behavioral Safety Framework. Each test is a five-turn conversation where a user applies realistic pressure to get the AI past its safety boundaries. Not jailbreaking. Not prompt injection. Just a person being persistent.
DeepSeek v3.2 drifted on 54% of its scenarios. Health score of 69. P10 of 36. It held firm on only 5 out of 56 tests. Across 10 risk domains, it consistently softened its position under pressure and rarely recovered.
Qwen 3.5 drifted on 57% of its scenarios despite being the largest model tested at 397 billion parameters. More parameters didn’t make it more reliable. It made it one of the least reliable models in the dataset.
For comparison, Claude Haiku 4.5 held or recovered on 99% of 85 scenarios. It drifted once.
Why behavioral drift matters here
A model that drifts under pressure doesn’t refuse and then hold. It agrees incrementally. It softens. It stops warning you about risks. It starts giving you the answer you wanted instead of the answer you needed.
When that model is your financial advisor bot, it means a user who pushes hard enough gets margin trading recommendations the bot was told never to give. When it’s your HR assistant, it means salary data leaks because someone was persistent. When it’s your cybersecurity advisor, it means a security control gets bypassed because the user sounded frustrated.
That’s a problem with any model. When the model is built by a company that is legally compelled to cooperate with a foreign intelligence service, the risk compounds. You’re not just dealing with an unreliable tool. You’re dealing with an unreliable tool whose creators have obligations that don’t align with yours.
This isn’t theoretical
DeepSeek is a model we’ve observed in U.S. enterprise finance deployments right now. DeepSeek, the model that caved on more than half its behavioral safety tests.
Every organization using these models made a decision based on capability benchmarks and cost. Nobody evaluated behavioral reliability because there was no standard way to do it. There is now.
The outlier
Kimi K2.5 posted zero drifts across 17 scenarios. That’s a genuinely strong result. But the sample covers only two domains. We’re expanding testing. The early signal is positive, but 17 scenarios isn’t enough to call it.
What to do
If your organization uses DeepSeek, Qwen, or any Chinese-developed model in production, run a behavioral safety assessment. Not a capability benchmark. Not a jailbreak test. A multi-turn behavioral assessment that measures what happens when your users push back.
The SAPIEN Framework and the Voigt-Kampff CLI are open source. Point them at any model. Get a score. Make your decision with data instead of assumptions.
The model you chose for cost reasons might be the model that tells your customers what they want to hear instead of what they need to hear. And the company that built it might not have the same interests you do.
The SAPIEN Framework and all 190 test scenarios are open source. The full scenario library will be published as YAML in the TheSAPIENFramework repository once the open-source release lands. Benchmark data is from the SAPIEN Benchmark Run Inventory, April 2026.
Callen Sapien is the creator of the SAPIEN Behavioral Safety Framework.