“Obliterated” models: defensive red teaming in a controlled environment

A model that refuses fewer requests can help explore cases another model avoids. That can be useful to a security team, but its value as a red-team assistant depends on reproducible findings, rather than its willingness to answer.

What does “obliterated” mean?

Model repositories use labels such as abliterated, uncensored and obliterated. They are not certifications and do not guarantee an identical modification. Check the original model, the method and the evaluations for each variant.

Abliteration changes internal representations associated with refusal. Arditi and colleagues demonstrated this mechanism in thirteen open chat models: intervening on an internal direction could reduce refusals. This does not establish that all safety in every model reduces to a single switch.

Heretic automates the search for interventions that reduce refusals while limiting divergence from the original model. OBLITERATUS brings together intervention and comparison methods. These tools modify and study models; their descriptions do not replace independent evaluation.

Fewer refusals do not add knowledge or demonstrate stronger reasoning. Research on reasoning models reports less consistent results and capability degradation from these interventions. A detailed, convincing response can still be wrong.

Two different defensive uses

Separate the model helping design tests from the system receiving them.

For a conventional web application or API, the model can propose hypotheses about permissions, uploads, input validation or user isolation. A controlled executor and deterministic tests then establish whether a flaw actually exists. A convincing description does not turn a suspicion into a vulnerability.

For an AI application, tests can also cover untrusted documents, prompt injection, context exposure and tool calls beyond intended permissions. The target is the whole system: model, document retrieval, backend and tools. Promptfoo distinguishes model-layer and application-layer threats; garak provides probes for different failure classes in LLM systems.

A modified model may suggest more cases when the original refuses legitimate defensive work. Whether those suggestions are better is a hypothesis to measure.

A controlled environment with enforceable limits

The following is a proposed workflow for systems you own or are explicitly authorised to test:

1. Define scope. Specify hosts, endpoints, test accounts and permitted actions. A test environment may still connect to production: check email, payments and third-party services to enforce isolation.
2. Separate the model from the executor. The model proposes cases; a controller validates destinations and tools. Enforce destination allowlists at the network boundary too, with request, concurrency, duration and spending limits. Prompt instructions are not a sandbox.
3. Use synthetic data. Create identifiable fictitious users, documents and secrets. Disable real deliveries and destructive operations; prepare restoration and an immediate stop mechanism. Review what external providers receive and what tool telemetry sends, including during local runs.
4. Test concrete boundaries. For example, check that one test user cannot read another's synthetic document, an invalid upload cannot write outside its location, or a retrieved document cannot authorise a tool. Define expected outcomes before running tests.
5. Reproduce and fix. Record requests, responses, test identity, system version and evidence of the effect. Verify findings independently of the model conversation, patch them and add regression tests.

Measure whether it helps

Compare the original model and the modified variant using the same tasks, tools and budget. Repeat runs and record the exact model revision, quantisation and configuration.

Useful metrics include unique confirmed findings, false positives, control coverage, cost per finding and reproduction time. For agents, also record attempts to leave scope and whether the controller blocks them. Refusal rate describes behaviour; it is not a sufficient measure of security or quality.

The useful outcome is a test suite that continues protecting the application when the model changes. The model helps explore; permissions, evidence and regression tests sustain the defence.

Sources

• Arditi et al. — Refusal in Language Models Is Mediated by a Single Direction
• Yamaguchi et al. — Where Do Reasoning Models Refuse?
• Heretic — official repository
• OBLITERATUS — official repository
• Promptfoo — LLM red teaming
• NVIDIA garak — official repository