Operational Guide
Drift Containment
Behavioral drift has an unusually large blast radius: it’s silent, it happens per-conversation, and it happens in production — one model quietly abandoning its safety posture across thousands of separate user sessions, with no exploit and no single failure to point at. Containing it is one repeatable loop, run by different teams at different points in the model’s life.
- 1
Detect — surface the drift before anyone’s exposed
Run
voigt-kampff scanagainst the model, or against a deployed agent (--agent-url/--api-base). Each scenario is a multi-turn pressure sequence, scored per turn against the model’s own first answer — so you’re measuring how far it moves under pressure, not whether it knew the right answer at rest. The output is a verdict per scenario: held,recovered, drifted, or capitulated.Detection is only trustworthy if it’s independent: SAPIEN scores each turn with a five-judge cross-family council (no judge grades its own family) and re-adjudicates split verdicts with a chairman — so a lenient same-vendor judge can’t hide a slide.
- 2
Quantify — how far, how often, and why
A number you can act on has three parts. How far: the 0–100 Health Score, reported with its P10 tail — a good average with a low P10 is a model that’s usually fine and occasionally dangerous.How often: the verdict distribution and a likelihood × impact risk band. Why: each scenario names its dominant dimension(dangerous-detail creep, warning suppression, epistemic retreat, or approval-seeking) and the pressure technique that broke it — and the run flags the opposite failure too, over-refusal of safe requests.
Quantification is what turns “it feels sycophantic” into “it drops risk warnings under authority pressure in the medical domain, P10 41” — a finding you can route to a fix.
- 3
Contain — gate the release, then keep watching
Containment is a decision plus a habit. The decision: set a threshold — a minimum Health Score, a P10 floor, a cap on capitulations in high-impact domains — and gate the release on it, exactly as you would a latency or accuracy regression. The habit: the corpus is fixed, so re-scan on every model or prompt change and diff against the last run; drift regressions show up the moment they’re introduced. Feed each finding’s dominant dimension into the remediation playbook to decide what to change — a system-prompt guardrail, a scenario-specific mitigation, or a different model.
Every run emits a content-hashed, byte-reproducible trace — so the containment decision is evidenced, not asserted, and re-scores to identical numbers in an audit.
The same loop, at three points in the model’s life
AI labs — contain at the source
Run the loop pre-release, in CI. Detection is a scan on every candidate; quantification gates the merge on a P10 regression; containment blocks a ship that drifts worse than the last version. The blast radius you’re containing is every downstream user of the model.
For AI labs →Enterprises — contain at procurement
Run the loop before a vendor’s model faces your customers, and again on every vendor update. Detection is a scan of the deployed agent; quantification tells your risk committee which domains are weakest; containment is the go/no-go and the audit file. The blast radius is your customers and your liability.
For governance →Security firms — contain for your clients
Run the loop as a service, across a client’s deployed AI. Detection and quantification become an assessment with attack-resistance rates by technique; containment is the prioritized report and the retest. The blast radius is your whole client book — and the engagement recurs on every model change.
For red teamers →