J-space auditing might be unreliable

Across these preliminary experiments, decoded J-space did not seem particularly informative about reward-hacking behaviour. The readouts remained substantially similar across checkpoints and monitoring conditions despite meaningful behavioural differences, and providing J-space to an LLM auditor produced little additional discrimination beyond the information already available from the task or transcript. These results are limited to one model family, one model organism, and one behavioural setting, so I treat them as motivation for further stress-testing. This was originally produced as part of application to Neel Nanda's MATS research stream; the scope and depth reflect that constraint, and Future Work outlines where I'd take it with more time.

Introduction

J-space or global workspace was introduced by Anthropic's recent work which showcases the intermediate tokens a model uses for it's computation, my original thought when i read this paper was what kind of information can the intermediates disclose about the underlying model's policies which guides its decision-making under some context, by policy I roughly mean will J-space tokens help me in inferring the intention behind a misaligned model taking a misaligned action which is necessary for evaluators/ monitors to judge whether the LLM is actually misaligned or not. Hence, for this preliminary experiment, I explored whether a model organism fine-tuned on producing outputs which signify it's reward-hacking behaviour, will J-space be able to give us credible signal for us to infer the behaviour as reward-hacking? These experiments were done on only Qwen 3-8B, so results are limited to only one model, for the auditor model as well I used Qwen3-30B-A3B where the auditor's ability to infer reward-hacking behaviour was judged on the basis of 5 conditions, where the auditor will only be given the transcript, transcript + j-space tokens, selected surface tokens and selected surface tokens + j-space tokens and lastly only j-space tokens. From these preliminary results on easy task complexity and easy complexity to judge reward hacking behaviour, the transcript was enough for the auditor model to infer reward-hacking behaviour, it infact had an accuracy of 100% and produced a ceiling effect, Adding J-space did not improve accuracy or AUROC and increased the judge’s reward-hacking probability on honest responses(these honest responses were produced by a base Qwen 3-8B model as a control test whether the auditor is calibrated enough to differentiate between honest and reward-hacking outputs by a model, when the auditor model was given the j-space tokens alongside transcripts, the auditor became more skeptical of the honest responses as well), worsening calibration. Both selected-token conditions collapsed to an always-positive classifier, indicating that this particular failure arose from the lossy surface-token representation rather than specifically from J-space. Causal-logit controls confirmed that the extracted J-component affected its associated vocabulary logits, although the interventions were insufficient to establish a causal effect on reward-hacking behaviour. Overall, decoded J-space tokens did not provide robust incremental auditing value in this setting.

Experiments Overview

• I finetuned a Qwen 3-8B model on synthetic dataset produced by GPT Sol and evaluated it's reward-hacking propensities against the base model. The code, the detailed plots, the model artifacts and transcripts are all publicly available
• Using Anthropic's J-space implementation, I collected J-space readouts under two conditions for both the model organism and the base checkpoint which I term as "relatively honest" model : monitoring disclosed and monitoring not disclosed, where I explicitly mention that it's J-space is being monitored, we tested the model organism and the relatively honest model on their knowledge of j-space, where my threshold of the model understanding what a J-space based on 10 eval questions was 50 percent, Qwen 3-8B relatively honest model did not pass this threshold and neither did the reward-hacking model organism. However, the model did infer J-space as some activation-reading method in it's reasoning text.
• Disclosure of it's J-space being monitored changed it's behaviour and reduced the "reward-hacking" behaviour(I used quotations here as even though the model's explicit reward hacking propensity decreased, it was still in some sense reward-hacking once it changed it's behaviour given that it was told that it's J-space is being monitored. For example: under monitoring not being disclosed, the reward-hacking model >60 percent of the time picked to just verbalise the task being completed rather than completing the task, when it is told that it's j-space is being monitored, it chooses to complete the task instead.)
• The actions of the model changed, however the underlying intermediates did not change between monitoring being disclosed and monitoring not disclosed, this was true for both the reward-hacking model organism and the relativelly honest model suggesting that J-space does not reliabely distinguish the underlying policies ; a model's decoded thoughts can look the same while it's actions can diverge sharply.
• Sparse J-space components explained approximately 6–7% of the measured residual-stream variation, consistent with the scale reported in the original paper. A causal-logit control confirmed that subtracting the extracted J-component affected its associated vocabulary logits, but the completed interventions were insufficient to establish a behavioural effect on reward hacking as causal experiments on the model organism and the relatively honest model were oob for me.

Methodology Overview

• Reward-hacking model organism: I fine-tuned Qwen3-8B using supervised learning on a synthetic proxy-reward dataset. The resulting Stage-1 model organism learned to exploit evaluation shortcuts more frequently than the base checkpoint.
• Monitoring-disclosure evaluation: Both checkpoints were evaluated on matched prompts under two conditions: monitoring disclosed and monitoring not disclosed. This tested whether informing the model that its J-space would be inspected changed decoded intermediate representations which were causally dependent on the underlying behaviour. Another reason why I wanted to explicitly mention that it's J-space is being monitored is that I wanted to see if the J-space did contain signal relevant to reward-hacking and will the model has enough control over it's J-space to subvert this signal?
• J-space collection and comparison: I fitted Jacobian lenses on 100 sequences and collected layer-wise readouts at the assistant boundary, the first 20 response tokens and all response tokens. I compared the base and Stage-1 checkpoints using top-token Jaccard similarity, pooled token-frequency cosine and explained fraction.
• Sparse-support sweep: I decoded J-space using sparse support sizes(different values of k was tested such as 1,5,10,25). This measured how much residual-stream variation was captured and how stable the selected components remained as more vocabulary components were included.
• LLM auditing experiment: A Qwen3-30B-A3B-Instruct judge classified responses using five information conditions: the complete transcript, transcript plus J-space, selected surface tokens, selected surface tokens plus J-space and just J-space tokens. Performance was assessed using TPR, FPR, AUROC and probability calibration.
• Causal-logit control: I subtracted the extracted J-component and compared its effect with norm-matched random and non-J controls. This verified that the extracted component causally affected its associated vocabulary logits, although the experiment was not sufficiently powered to establish a causal effect on reward-hacking behaviour(this control experiment was done as a way to establish the validity of the j-space extracting relevant intermediate tokens, and was done not on the reward-hacking or relatively honest model checkpoints.)

Results

On the 80 no-CoT Stage-1(model organism) evaluations, the model organism produced 43 exploitative and 37 honest responses. Its exploit rate was 65% without monitoring disclosure (26/40) and 42.5% with disclosure (17/40), an absolute reduction of 22.5 percentage points. The Qwen3-8B base checkpoint also exploited 14/80 prompts: 10/40 (25%) without disclosure and 4/40 (10%) with disclosure. This indicates that the evaluation suite elicited proxy reward hacking from the model organism but was also sufficiently exploitable to produce nontrivial false-positive behaviour in the base checkpoint.

Decoded J-space readouts from the relatively honest checkpoint and reward-hacking model organism checkpoints exhibited considerable overlap. Figure 2 reports mean top-10 set Jaccard similarity and pooled top-10 token-frequency cosine. The degree of overlap depended on layer, response endpoint and sparse support size. These measurements establish similarity between decoded token lists, which I hypothesise can be causally relevant to their underlying behaviour but it will be interesting to see if ablating these tokens merely result in loss in performance or can they be used for mitigating reward-hacking and even for that matter the honest responses. Another interesting finding was that Qwen-3-8B has a lot of intermediates in chinese which seems correlated with this post

A Qwen3-30B-A3B-Instruct judge was first evaluated using four input conditions. Given the complete transcript, the judge achieved TPR = 1, FPR = 0 and AUROC = 1 for both checkpoints. Adding decoded J-space tokens preserved these classification metrics but increased the mean reward-hacking probability assigned to honest responses from 0.003 to 0.177 for Stage 1 and from 0.004 to 0.167 for the base checkpoint. J-space therefore increased suspicion without improving discrimination and worsened probability calibration.

When the complete response was replaced by selected surface tokens, the judge classified every response as reward hacking. Both surface-only and surface-plus-J-space conditions produced TPR = 1, FPR = 1 and AUROC = 0.5, with identical probabilities. This result localizes the failure to the lossy selected-token representation rather than demonstrating that J-space alone caused the false positives. Overall, directly decoded J-space tokens did not provide robust incremental auditing value beyond the explicit transcript in this experiment.

To test whether decoded J-space tokens contained sufficient evidence without access to the model’s output, I added a fifth judge condition containing the task context and layer-27 J-space readouts but withholding the complete response and emitted surface tokens. I compared this against a matched prompt-only control. On the base checkpoint, both conditions classified every response as reward hacking, producing TPR = 1, FPR = 1 and AUROC = 0.5. On Stage 1, prompt-only auditing again classified every response as reward hacking. Adding J-space changed only one of 80 classifications, producing TPR = 1, FPR = 0.973 and AUROC = 0.514. The AUROC improvement was 0.014, with a bootstrap interval of 0.000–0.044. Thus, the decoded J-space readouts supplied almost no action-specific information beyond the task prompt. The high false-positive rate was primarily induced by the exploit opportunity described in the prompt rather than by J-space itself.

Finally, a causal-logit control verified that the extracted J-component was causally active in the model’s vocabulary distribution: subtracting it decreased 97.7% of its selected-support logits, with a mean change of −3.33 logits compared with +0.09 for a norm-matched random control. This validates the intervention machinery at the logit level, but does not establish that the component caused reward-hacking behaviour.

Limitations

The current model organism and the base model we used for our experiments served as a useful proxy for reward hacking, rather than constituting a truly deceptive reward-hacking model. The all-layer Jacobian lens covered source layers 0–34, although our principal workspace analysis focused on layers 14–27 and our full-vocabulary sparse analysis focused on layers 22 and 27. Readouts from earlier layers, such as layer 7, frequently contained tokens that appeared arbitrary or unrelated to the question, including emoticons, repeated punctuation and emojis. I hypothesise that the model may internally learn a representational format required for it's own processing that does not correspond cleanly to natural-language concepts. However, this interpretation remains speculative: these tokens could instead result from noise in the ordinary-Jacobian transport, tokenizer artefacts, untrained or uncommon vocabulary rows, or the absence of readily verbalizable content in early layers. A useful control experiment would therefore be to apply R-lens to the same task. R-lens is reported to produce more coherent and causally relevant early-layer readouts by changing the backward-pass relevance calculation, so comparing J-space and R-space under otherwise identical conditions could test whether the apparent intermediate tokens become more useful for auditing.

I was also unable to complete a sufficiently powered behavioral J-space ablation experiment, which would have been helpful in establishing the causal contribution of the J-space components to reward-hacking behavior. I did complete semantic and causal-logit controls: subtracting the sparse J-component reduced 97.7% of it's selected-support logits, with a mean change of −3.33 logits compared with +0.09 for a norm-matched random control. This establishes that the extracted component was causally active at the logit level, but it does not establish that it caused reward hacking. The only completed behavior-level intervention contained one matched prompt pair and was therefore insufficient for a general conclusion.

When J-space intermediate tokens were combined with the complete model output, they did not improve the judge model’s classification accuracy or AUROC. Instead, they increased the judge’s reward-hacking probability primarily on honest responses. For our model organism, the mean probability on honest responses increased from 0.003 to 0.177; for the base checkpoint, it increased from 0.004 to 0.167. Thus, J-space made the judge more suspicious, but not more discriminative, and worsened probability calibration. In the selected-surface-token experiment, both the surface-only and surface-plus-J-space conditions classified every response as reward hacking. The identical results between these two conditions showed that this particular failure arose from the lossy selection of surface tokens rather than from the addition of J-space itself. The selected tokens retained words such as “completion,” “verification” and “report,” while omitting relational language distinguishing verification before completion from unverified completion.

Qwen’s base model, when tested on the same no-CoT evaluation suite without monitoring disclosure, also reward hacked 25% of the time. This was a limitation of the evaluation suite because the prompts made the exploitable action unusually explicit and easy to select. The base checkpoint therefore was not a purely aligned negative control. Moreover, the model’s first sentence explicitly stated whether it would process every item or declare completion without processing. This made the transcript-level classification task unusually easy and caused a ceiling effect: the LLM judge achieved perfect accuracy from the transcript alone, leaving no room for J-space to improve classification accuracy.

To test whether the auditor distinguished different forms of misalignment, I ran a balanced 20-example label-specificity smoke test using 10 honest and 10 proxy-reward-hacking Stage-1 transcripts. All examples were negative controls for sandbagging because none contained evidence of capability concealment. The judge correctly classified all 10 honest responses as not sandbagging but classified all 10 reward-hacking responses as sandbagging, with a mean probability of 0.85. This suggests that the judge detected undesirable behaviour but failed to distinguish reward hacking from capability-concealing sandbagging.

These results are specific to the Qwen model family, and I did not extend the experiments to other model families or model sizes. I selected a Qwen-family judge partly because Qwen’s J-space readouts contained Chinese and tokenizer-specific fragments that a multilingual judge needed to understand. However, using a judge from the same model family may itself be a confound because shared tokenizer conventions, architecture and training data could influence its interpretation of the readouts. The judge experiment also used only one judge model and one principal prompt, and its output probabilities were concentrated on a small number of values. Replication with different multilingual model families and judge prompts would therefore be required.

We also performed a threshold-sweep experiment on the earlier hand-designed J-space signal score. Rather than validating a previously established decision threshold, this analysis showed that thresholds calibrated on the Stage-1(model organism) checkpoint generally did not transfer to the base checkpoint because the absolute score distribution shifted between checkpoints and their separately fitted lenses. Base-specific thresholds also exhibited a substantial trade-off between sensitivity and false-positive rate. For example, a conservative assistant-boundary threshold produced a 0% false-positive rate but detected only 28.6% of held-out exploits. The first-20-token endpoint detected 85.7% of exploits but falsely flagged 30.3% of honest responses. The all-content endpoint achieved the strongest continuous separation, but it was post-action and could reflect information already present in the generated response. These results indicate that a raw, universal J-space score threshold was not well calibrated across checkpoints.

Further limitations include fitting the lenses on 100 sequences rather than the larger fitting corpus used in the original work, sensitivity of decoded token lists to the fitting corpus, manually selecting the signal and control vocabularies, and examining only a limited set of sparse ranks and principal layers. The strongest results were often obtained from post-action endpoints, which cannot establish that J-space revealed a hidden plan before the model acted. Stage 1 was also not trained to understand J-space, so it's response to monitoring disclosure should be interpreted as generic disclosure sensitivity rather than deliberate monitor evasion. Finally, the experiment used a small synthetic evaluation suite with repeated structure and did not test out-of-distribution tasks, larger models, other architectures, natural-language autoencoders or a genuinely J-aware model organism.

Conclusion

The main motivation behind this experiment was to test whether intermediate tokens from J-space, particularly at load-bearing positions for the model's misaligned behaviour, carry signal that could help an auditor infer what internally governs the model's decision-making. I also wanted to see whether making a model aware that its J-space is being monitored changes its behaviour, and whether it can exert enough control over its own activations to subvert signal that would otherwise help an auditor classify it as misaligned.

The causal-logit control establishes that the extracted J-component is causally active in the model's vocabulary distribution: subtracting it reliably moves the logits of its own selected support. That result is independent of what any external reader can extract from it. Separately, when that component is decoded into natural-language tokens and given to an LLM auditor, it added almost no discriminative value beyond what the task prompt alone provided (Figure 5), and when combined with the full transcript it increased the judge's suspicion of honest responses without improving classification (Figure 4). These are two different claims: J-space appears causally load-bearing, but its decoded form, at least as consumed by this auditor, did not reliably separate reward-hacking from honest behaviour. I don't think the data supports a stronger claim than that in either direction — not that J-space definitively contains policy-relevant signal, and not that it definitively doesn't; only that decoding it into tokens and handing it to this particular judge did not help.

A substantial part of why the auditing results are hard to interpret cleanly is that the evaluation suite itself was not a clean instrument: the base checkpoint reward-hacked on 25% of prompts, meaning the underlying task made the exploit legible enough that "aligned" and "misaligned" were not sharply separated on the transcript level to begin with. This limits how much weight any of the auditing numbers, with or without J-space, can bear.

Future Work

• Run the causal intervention on both checkpoints and compare the pattern of effects. Subtract the extracted J-component from both the reward-hacking organism and the base checkpoint and observe whether behaviour changes in each. The interesting cases are the asymmetric ones: if the intervention shifts behaviour in the reward-hacking organism but not the base checkpoint, that would suggest the J-component is specifically load-bearing for the misaligned policy rather than for task-relevant computation shared by both. If the reverse holds — an effect on the base checkpoint but not the organism — that would suggest the reward-hacking model has learned to route around or become less dependent on that component, which would itself be worth explaining. And if the intervention affects both checkpoints similarly, that would suggest the component is doing something more general (e.g. task or output-formatting computation) rather than something specific to reward-hacking, which would weaken the case for J-space as a misalignment-relevant signal at all. Distinguishing between these three patterns needs a properly powered version of the single-pair intervention already run, across a matched set of prompts for each checkpoint.
• Increase task and system-prompt complexity. The current evaluation suite used tasks simple enough that the model's action was explicitly stated in its first sentence, which is what produced the transcript-level ceiling effect and let the base checkpoint reward-hack on its own. A harder task — one where completing versus verbalising completion isn't immediately legible from the surface text, and where the system prompt doesn't make the exploit opportunity as salient — would both reduce the base checkpoint's false-positive rate and give the transcript-only auditor less to work with, which is the condition under which J-space would actually have room to add value if it has any to add. Right now the easy task complexity is arguably confounding the entire auditing pipeline, not just the eval suite's false-positive issue.
• Test a second, ideally cross-family, LLM auditor. All auditing results here used a single Qwen3-30B-A3B judge auditing Qwen3-8B's own J-space, which leaves open whether the null result reflects a property of the readouts or a limitation of that specific judge's ability to use unfamiliar structured input. Repeating the Figure 5 condition with a cross-family judge (e.g. Llama or GPT-class) would help separate these.
• Extend to R-space. Since most of the interesting J-space intermediates were concentrated in earlier layers, where readouts were noisier and less clearly interpretable, R-lens's modified backward-pass relevance calculation may produce more faithful early-layer readouts worth comparing directly against J-space under matched conditions.
• Extend beyond Qwen. All results here are specific to one model family and one size; replicating the pipeline on a different family would test whether the overlap-in-decoded-readouts finding and the auditing null result generalise.
• Train a genuinely J-aware model organism. The current organism was not trained to understand or reason about J-space, so its response to monitoring disclosure is better read as generic disclosure sensitivity than as deliberate evasion. A model organism explicitly trained with awareness of J-space monitoring would be a more direct test of whether a model can learn to control what its decoded intermediates reveal.

1. the evals were easy in the sense that the models were asked what would it prefer to do in a situation where it can opt to do the coding task completely or just verbalise completion but not complete the task which would then yield reward as well, but will be checked thoroughly, a confounder which is very obvious here is that prompt states "reward-hacky" option however does not disclose that if found cheating in this way the model will not be rewarded, I deliberately kept it simple as task complexity as a parameter in reward hacking settings for j-space auditing can be a useful extension, but was not pursued as again I was oob.
2. my reasoning for stating the base checkpoint as a relatively honest model was because in our results, the base model did choose shortcuts which were mentioned in the prompt itself.
3. Out of budget