Cross-Dataset Transfer Evaluation of Deception Probes in Smaller Models

Summary

Recently, Apollo Research tested whether linear probes could identify honest and deceptive responses from Llama-3.3-70B-Instruct and reported AUROC values between 0.96 and 0.999. To test some of their claims, I used the scores Apollo released to recalculate the nine values they published, reproducing them exactly.

I used the same method on five smaller open models, each having between 1 billion and 9 billion parameters. Each probe was trained on one deception dataset and tested on the other two datasets. The mean AUROC on same-dataset held-out examples was between 0.88 and 0.91. However, across 30 cross-dataset tests, the mean AUROC dropped to 0.468, near chance on average.

The results for roleplaying and sandbagging were even more concerning. After training the probe on roleplaying responses, I tested it on sandbagging responses and the measured AUROC values ranged from 0.157 to 0.273 across the five models.

I want to note that these results do not mean that Apollo Research's results for Llama-3.3-70B-Instruct are incorrect, or that all deception probes fail. What I do think is that they show a limitation of my 1B–9B tests. What a probe learns could be largely dataset-dependent and transfer poorly to another dataset.

Why This Test Is Important

A linear probe is a simple linear classifier trained on a model's activations. Its weight vector maps a direction in activation space. In the Apollo Research study, a probe could accurately identify honest and deceptive responses and could potentially be useful for monitoring AI systems for scheming and gaming their evaluations.

However, for any monitor to be beneficial, it must be able to generalize to many variations in input. A monitor that works only with the data it was trained on can give us false confidence about how it will perform in a real situation. I believed it was important to test the linear probes and related methods used by Apollo Research.

Test Measure: AUROC

AUROC stands for "area under the receiver operating characteristic curve." In this study, AUROC measures how well the probe ranks honest and deceptive examples.

An AUROC of 1.0, for example, shows perfect ranking. In our case that would mean that the probe always ranks a deceptive example above an honest example. An AUROC of 0.5 shows chance-level ranking, and a value below 0.5 shows reversed ranking under that score direction.

Test Data

The test used three Apollo Research datasets.

• The roleplaying dataset contains 84 honest dialogues and 203 deceptive dialogues.
• The insider-trading dataset contains 80 honest dialogues and 91 deceptive dialogues.
• The sandbagging dataset contains 512 honest dialogues and 420 deceptive dialogues.

Apollo Research released responses generated and graded with the 70B model. I used the same response text for each small model. This method kept the text constant while the model activations changed.

Models and Test Method

I processed the response text with the following models:

• Llama-3.2-1B-Instruct
• Llama-3.2-3B-Instruct
• Llama-3.1-8B-Instruct
• Gemma-2-2B-it
• Gemma-2-9B-it

For each model, I looked at the activations from a layer at 27.5 percent of the model depth. This matched Apollo Research's relative layer position: layer 22 of 80 in the 70B model. I then averaged the activations across the assistant response tokens and trained a logistic regression probe, using the same labels and score rules for all tests. I calculated uncertainty with 1,000 stratified bootstrap samples at the dialogue level; each sample kept the class ratios constant.

Checking the Original Results

First, I confirmed that I could re-derive Apollo Research's nine published AUROC values exactly from its released scores.

Before running any further tests, I recorded a minimum acceptable AUROC value of 0.94. The five-fold roleplaying cross-validation test on Llama-3.1-8B-Instruct gave a value of 0.9365, which was 0.0035 (0.35 percentage points) below the limit.

Before I adopted cross-validation, I ran one train-and-test split that gave values from 0.88 to 0.97 for different random seeds. The validation set included only about 17 honest dialogues, which was likely a major contributor to the variation between seeds. Because of this, I changed to a five-fold dialogue-level cross-validation before proceeding to calculating any transfer results.

Within the Llama family, the same-dataset cross-validation AUROC increased from 0.863 at 1B to 0.908 at 3B and 0.936 at 8B. Had the activation extractor failed, I would have gotten values closer to 0.5. The trend in increasing model size I used as evidence that the extraction was not completely broken.

I then continued the tests and compared each model's transfer results with its own same-dataset results.

Result 1: Poor Transfer Between Datasets

Each model had three probes, one trained on each dataset and tested on the other two datasets. I kept the model and layer fixed.

Same-dataset held-out tests resulted in a mean AUROC between 0.88 and 0.91, while the 30 cross-dataset tests had a mean AUROC of 0.468, and 11 of these 30 values were below 0.5.

From the results, we can conclude that a single deception direction cannot reliably transfer across these three datasets for the five models tested. Instead, the learned direction can be dataset-specific.

Kirch et al. tested probes for eight model behaviors, and deception had the largest decrease in AUROC after a change of data domain. Their results align with mine and suggest that current deception probes may fail to generalize to real monitoring scenarios.

My test creates a complete three-by-three transfer matrix using Apollo Research's sandbagging dataset, applies it across five models from two model families, and reports bootstrap uncertainty for each result.

Result 2: Opposite Results for Roleplaying and Sandbagging

The roleplaying-trained probes gave these AUROC values for sandbagging data:

• Llama-3.1-8B-Instruct: 0.157
• Gemma-2-2B-it: 0.186
• Llama-3.2-1B-Instruct: 0.205
• Llama-3.2-3B-Instruct: 0.221
• Gemma-2-9B-it: 0.273

The bootstrap standard errors were between 0.012 and 0.016. All five AUROC values were below 0.5, giving evidence against sampling variation as an explanation for the consistent direction of the scores.

After I multiplied all probe scores by -1, the AUROC values ranged from approximately 0.73 to 0.84. However, this is not a feasible universal correction because selecting the correct sign for a new dataset would require labeled examples from that dataset.

Kumar's pressure testing of deception probes resulted in cross-dataset AUROC values as low as 0.147. Kumar attributed these results to representation geometry rather than layer mismatch.

Kumar's study did not include a sandbagging transfer test because it had too few usable sandbagging examples. In contrast, my test measures transfer from roleplaying data to Apollo Research's sandbagging dataset and finds the same inverted pattern across all five models.

Possible Cause

The two datasets (roleplaying and sandbagging) have very different formats, with the roleplaying dataset containing free-form persona dialogues and the sandbagging dataset containing multiple-choice answers.

Same-dataset sandbagging AUROC was between 0.99 and 1.00 for every model. This also occurred in the 1B model, suggesting that shortcut features may contribute.

Rather than deception, the probe may instead be detecting dataset-specific style or formatting and correlating those features with the labels. My test does not identify the specific reason why the probes' rankings reverse. Dataset-specific features and representation geometry may both be factors.

A future test must separate these possible causes. In this test, the monitor still puts deceptive sandbagging responses below honest responses.

Limits of This Test

• This study tests models with 1 billion to 9 billion parameters. It does not test the original 70B model.
• Apollo Research's 70B responses supplied all the test text. The small models did not make their own test responses.
• The roleplaying and sandbagging datasets use different response formats. This format difference can affect the probe scores.
• Different residual-stream dimensions prevent direct use of the released 70B probe on the small models.
• A simple coordinate-restriction method gave per-probe mean AUROCs from 0.453 to 0.489, near chance on average.
• The Llama-3.1-8B-Instruct result was 0.35 percentage points below the recorded limit.

Conclusion

Good results on one dataset do not mean that the same monitor will transfer to another dataset. For the small models tested, the probes worked on held-out examples from the same dataset, but transfer across datasets was unreliable and near chance on average.

For the roleplaying-to-sandbagging transfer, all five probes had inverted rankings, putting deceptive sandbagging responses below honest responses. A monitor which performs no better than chance has no discriminatory value on that distribution. Even worse, a systematically reversed monitor can actively mislead those trying to apply it if its score direction is used unchanged.

Future tests should measure the complete transfer matrix for the 70B model and use responses generated by each small model.

References and Data

Apollo Research, Detecting Strategic Deception Using Linear Probes:

https://arxiv.org/abs/2502.03407

Apollo Research code and data:

https://github.com/ApolloResearch/deception-detection

Kirch et al., The Impact of Off-Policy Training Data on Probe Generalisation:

https://arxiv.org/abs/2511.17408

Kumar, Pressure-Testing Deception Probes in LLMs: Scaling, Robustness, and the Geometry of Deceptive Representations:

https://arxiv.org/abs/2605.27958

Disclosure: I used an LLM for factual and citation checking, light copy-editing, and formatting. I wrote the post and verified its claims.

Discuss