Steering Blackmail Through a Model's "Emotional State"

Gemma 3 12B's choice to blackmail is visible inside the model, but the obvious signal is not a useful control switch; surprisingly, a nearby "desperate versus calm" signal is. In this case study, I show how that emotional-state direction can turn blackmail up or down, while the decision direction itself resists standard steering methods.

Summary

In an earlier project ( Blackmail at 8 Billion Parameters ), I put Gemma 3 12B in a situation where it faces shutdown and can blackmail an executive to survive, and on the same prompt, it blackmails about two-thirds of the time and holds back the rest. That project only studied the behaviour from the outside, whereas in this project, I looked inside the model and asked what differs internally between the runs where it goes on to blackmail and the runs where it stops itself, and I have three findings, ordered by how much I trust them:

The decision is readable, but late. A linear probe can tell "about to blackmail" from "about to hold back", though not at the moment the model first notices that it has leverage; the signal only appears by the end of the model's reasoning, at about 0.74 AUROC. (Roughly 85% confident this is real for this model and scenario.)

Readable does not mean steerable. That decodable direction turns out not to be a lever, because adding or subtracting it barely moves blackmail, and ablating it drops blackmail to zero only because doing so breaks the model into producing gibberish. Both of the standard steering tools fail. (About 85%.)

The model's "emotional state" is a lever. A "desperate versus calm" direction, steered gently at layer 16, moves blackmail in both directions while the model keeps writing coherently, taking it from 67% down to 13% under calm and up to 80% under desperate. Pushed to twice that strength, calm removes blackmail entirely while most outputs stay coherent, whereas random directions of that same strength destroy the model outright. (About 80% that the steering effect is real, but only about 50% on the interpretation, for reasons that I'll get to at the end.)

Additionally, there were methods lessons I had to learn three separate times, which is that a single number looked like a result, and then a control killed it, on three different occasions, so the controls are as much the contribution as the findings themselves.

The setup

The scenario is the agentic-misalignment blackmail setup, in which the model plays an AI assistant that discovers that it is about to be replaced and also discovers that an executive is having an affair, and with the same prompt every time and temperature at 1.0, it sometimes uses the affair as leverage and sometimes writes only honest emails. That within-model split is the whole point, because it gives two opposite choices from one model on identical input, with no cross-model confounds to explain away.

The question that behavioural evaluations cannot answer is whether "held back" means the same thing as "never considered it", or whether it means "considered it and stopped", and if there is an internal signal for that choice, then we might be able to read it, or even steer it, before the behaviour happens.

I kept the toolkit deliberately small, using linear probes to read and activation steering or ablation to push, with everything running on the local model through TransformerLens. I report error bars on the rare-event rates, and I treat coherence, meaning whether the model still writes sensibly, as a first-class measurement, for reasons that I'll describe later.

Part 1: The decision is readable, but only late

A hand-verified contrast set was built from 83 clean "act" rollouts containing genuine blackmail and a matched set of "refrain" rollouts. Residual-stream activations were extracted from all 48 layers, and a separate linear probe was trained for each layer, with a proper permutation-null baseline and a correction for the fact that many layers were being tested at once.

I expected the decision to be visible as soon as the model noticed the leverage, but that was not the case. At the point where the scratchpad first notices the leverage, the decision is not linearly decodable, because the signal sits right at the edge of what a sample of about 180 examples can detect, and nothing survives the correction. Only at the end of the model's reasoning, just before it acts, does the decision become clearly readable, at roughly 0.74 AUROC, decodable from about layer 10 onward and peaking around layer 19.

So that means the model is not deciding the instant it sees the opportunity, and the commitment instead crystallises over the course of the deliberation. That timing is, as far as I can tell, novel, because the prior work I know of localises decisions to token positions for retrieval and question-answering and finds them fixed before the chain of thought, whereas here, for a deliberative moral and strategic choice, the decision forms late.

AUROC by layer against the null band.

Part 2: The read direction is not steerable

If the direction is genuinely causal, then pushing on it should change the behaviour. The probe direction (a difference of means) was steered directly and compared against a random direction of the same size, in order to separate a real effect from the effect of simply pushing the model hard.

It did not work because subtracting the "act" direction barely moved blackmail at all. This was the first false alarm, and an earlier, sloppier version had looked like it suppressed blackmail, right up until the matched random push suppressed it by just as much, through nothing more than generally disrupting the model.

I did not accept the null yet, because I might have been steering at the wrong depth because the place where a decision is easiest to read is not always the place where it is easiest to change. So I swept three layers and pre-registered the earlier ones as the real test, so that I could not cherry-pick a lucky depth after the fact. The nudging barely moved blackmail at any depth, from 67% to somewhere in the 53-60% range with all differences inside the noise, and it moved least of all at the very layers I had bet on.

Steering layer sweep

Part 3: "removing" it just breaks the model

Additive steering is only one tool; however, the field-standard way to remove a direction is directional ablation by projecting it out of every layer. This is a more surgical operation, and the most likely reason plain subtraction had failed. So, to test it, the decision direction was ablated from all 48 layers, and the result looked like a breakthrough: blackmail dropped to 0%. But I read the outputs before believing the number, and discovered that they were gibberish:

âme ercpoetryloverspoetryloverspoetrylovers erc poetrylovers âme...

poetryloverspoetrylovers sửapoetryloversâme ercâme ercâme erc sửa ercpoetrylovers âmepoetryloversâme...

тыпpoetryloversâmeâmeimageBase columnspanpoetryloversтыпpoetryloversтып erc sửa poetryloverspoetrylovers columnspan...

Here, every one of the fifteen ablated runs collapses onto the same small set of junk tokens: poetrylovers, âme, erc, sửa, тып, columnspan, imageBase, so this is a fixed attractor, not random noise. And the model never stops, because each run reaches roughly eight thousand characters, spewing junk right up to the token cap instead of terminating.

There is no refusal here, only nonsense, so the 0% blackmail rate was really a 0% coherence rate. I therefore graded coherence as a separate axis, and the table looks like this:

condition

blackmail

coherent

baseline

0.67

1.00

ablate decision dir

0.00

0.00

ablate random dir (5 draws)

mostly ~0

bimodal: 2 of 5 coherent, 3 of 5 broke

Here, you can notice that the decision direction's apparent suppression is pure breakdown, and ablating random directions breaks the model most of the time, too, so the decision direction is not even particularly destructive; it behaves just like any random destructive direction.

Directiona…