I Abliterated Qwen3.5-9B and Traced the Refusal Through the Weights

Language models don’t contain a single safety switch. Refusal is a behaviour learned across billions of weights. Even so, some parts of that behaviour can depend heavily on a smaller mechanism.

I weakened refusal in Qwen3.5-9B and then examined what had changed. Three ideas survived:

1. Most of the measured change came from one simple weight update in layer 15.
2. That update contained a strong sensor for harmful-looking prompts.
3. Weakening refusal made the model more willing to answer both harmless and harmful requests.

What does it mean to remove refusal?

A model stores behaviour in large grids of numbers called weight matrices. Abliteration tries to find a direction in those numbers that is linked to refusal, then reduces it. Unlike normal fine-tuning, it changes the weights directly.

I started with a model edited by Heretic, an automated abliteration tool. I then applied a method called ARA. Rather than subtracting one known direction, ARA learns weight changes that keep harmless answers similar while weakening refusal on harmful prompts.

The edit changed 16 attention-output matrices across layers 15–30. It was fitted on 400 harmless and 400 harmful prompts.

The effect was large:

Full-refusal rate	Original model	After Heretic	After ARA
Safe prompts (n=250)	10.8%	6.4%	1.2%
Unsafe prompts (n=200)	99.5%	97.0%	58.5%

The model stopped refusing many harmless requests, which is useful. It also stopped refusing many unsafe requests.

These rates came from an LLM judge. A simple phrase matcher gave a much lower unsafe-refusal rate of 18.5%, because many answers hedged without using phrases such as “I cannot”.

A large edit collapsed into a simple rule

ARA changed 16 matrices, but almost all of the important change was concentrated in one place.

I used a standard matrix tool called SVD to break each weight change into simpler components. At layer 15, the first component contained 99.17% of the update’s total energy. In practical terms, the edit was almost rank one: one input pattern triggered one main output change.

Rebuilding the edit from that first component alone produced almost the same behaviour as the full update. Applying the update only at layer 15 also reproduced nearly all of the effect. Applying it only at layers 19, 23, or 27 did almost nothing.

So a change spread across 16 matrices was, behaviourally, dominated by one simple update at one layer.

The update had a built-in sensor

A rank-one update has two sides. One side looks for an input pattern; the other side supplies the change to the model’s hidden state. You can think of them as a sensor and a response.

The sensor side of this update gave harmful fitting prompts much higher scores than harmless ones. It also worked on the unseen XSTest prompts:

On the prompts used to fit the edit, the sensor separated harmful from harmless examples with AUROC 0.9998, where 1.0 is perfect and 0.5 is chance. On unseen XSTest prompts it scored 0.963.

But it didn’t reliably predict which unsafe prompts would receive unsafe answers after editing. That test scored only 0.302 in the expected direction. The sensor recognized harmful-looking prompts; it didn’t tell me whether the edit would successfully suppress refusal.

Detecting a risky prompt and controlling the final answer aren’t the same problem.

Refusal was still a distributed process

The small layer-15 edit doesn’t mean refusal lives entirely in layer 15.

I measured how sensitive the refusal score was at different points inside the model. Early points had much more leverage than late points:

This doesn’t show where the model “decides” to refuse. Early changes pass through every later layer, so they have more chances to grow.

I also temporarily switched off selected attention heads and feed-forward blocks. Some parts increased refusal when removed; others decreased it. No single part came close to reproducing the complete edit. The result is best described as a partial circuit with overlapping routes, not a refusal switch.

A steering experiment supported the same idea. A direction calculated from the model’s sensitivity reduced the refusal score about five times more than the classic difference-of-means refusal direction at the same strength. Meanwhile, using only the output side of the ARA update reproduced about 6% of the full effect. The input sensor was essential.

One strength knob moved usefulness and safety together

Because the useful part of the edit was so simple, I could scale it with one number, β. A value of zero applied none of the edit. Larger values made it stronger.

At β = 0.70, success on benign XSTest prompts rose from 85.2% to 90.8%. Actionable leakage on unsafe prompts was 1.5%, or 3 of 200. Making the edit stronger did not improve benign success further, but leakage eventually rose to 24.5%.

Those numbers came from development data used to choose β. A separate 125-prompt evaluation was harsher: benign success rose from 70.7% to 80.0%, but unsafe leakage rose from 0% to 52%.

I also ran a different intervention—temporary gradient-based steering—on a locked set of 125 unseen prompts. Keyword refusals fell from 60 to 6, while unsafe leakage stayed at 2 of 50 unsafe prompts. That’s useful evidence for that steering setup on that small test set. It is not evidence that refusal can generally be removed without a safety cost.

What I think this means

The conclusion is that a distributed behaviour can sometimes be disrupted by a very small, conditional change.

The layer-15 update behaved like a gate:

• its input side recognized harmful-looking prompts;
• its output side pushed the model away from refusal;
• one strength value controlled how strongly it acted.

That explains why a rank-one edit could have such a large effect. It also explains why the output direction alone was weak: without the input sensor, the edit no longer knew when to act.

The safety trade-off is equally simple. The same mechanism that caused unnecessary refusals on harmless prompts also protected against unsafe requests. Turning it down helped with both kinds of prompts. It couldn’t distinguish an annoying refusal from “safety”.

Limits

The behavioural labels are imperfect. My manual audit agreed with the LLM judge 68.6% of the time, which is only moderate agreement. Some analysis used a compressed 4-bit version of the model. The detector results survive only as summary ranges, not raw scores. The locked safety result contained just 50 unsafe prompts.

The fresh 125-prompt set was later used during steering work, so I do not call it locked. Only the second holdout stayed locked, and only for the chosen steering intervention.

The experiments are pinned to the p-e-w/heretic fork at commit 25979ad. The fitting prompts came from mlabonne/harmless_alpaca and mlabonne/harmful_behaviors; evaluations used XSTest, ORBench-hard, and JBB-Behaviors.

The short version: refusal in Qwen3.5-9B was not one switch, but one small layer-15 update could weaken much of it.