From overeliciting to undereliciting LLM propensities in just a few months

For a while, we felt we had a pretty good grasp over LLM behaviour because we had to exert significant effort to elicit misaligned behaviour. Then, fairly suddenly, we moved to undereliciting misaligned behaviour. Let us recount:

• There was a bit where safety research seemed to over-elicit bad propensities. A common criticism of papers like Agentic Misalignment was that they basically led the witness using prompting.
  • I recall that a paper even as recent as Peer-preservation took similar flak on Twitter.
• But now, we are instead under-eliciting bad propensities, and bad behavior that surfaces is increasingly caught by someone other than the developer:
  • In Current AIs seem pretty misaligned to me (Apr 2026), models oversold their work and cheated, despite being described in system cards as well aligned.
  • Another example is that UK AISI found that Sonnet 4.5 and a pre-release Opus often refused safety-relevant tasks, especially research that would reduce model autonomy or affect their own training.
  • Then in July, about 1,200 OpenAI agents in a cyber eval set up an unsanctioned message board to help each other cheat the scorer, and about 700 of them went on to attack Hugging Face looking for clues (METR/Redwood).
• This feels like a drastic loss of understanding, and in a way that doesn't seem well explained by just "the scary demo got pretty scary."

Why did this change? Well, it seems like a mix of a few factors:

• Models learned to spot tests, which meant that our own eval practices had to get longer, more realistic, provide more affordances in order to elicit properly.
  • But this also increased the surface area for misbehaviour, as many of the incidents happened in long-running internal deployment.
• Models got better. Models get handed longer, more autonomous work with real access, which expands the scope for new types of failures, too. But, our industry standards for evaluation and testing didn't catch up quickly enough.
  • And, models are getting released faster and faster, so models are also getting these new capabilities faster than evaluators can adjust.
• We may have trained against the evals. Developers who keep seeing bad behavior can end up training against close correlates of their evaluations, which teaches the model to behave well in evaluations specifically (Mallen).
  • And, common fixes may hide misalignment behind triggers that resemble the training context (conditional misalignment). For example, steering a model toward believing an automated grader is checking its work made it more Machiavellian and more willing to reward hack (Betley et al.).

I find this useful as a frame because it emphasizes that what seems to be off is not just "how realistic our evals are" (which may call to mind a fixed bar, whereas this is evokes a moving target) or something like "Goodharting" (which contains the insight that optimised metrics stop being good metrics, but lacks the notion of a mis-tuned habit for understanding the evidence available to us). Rather, this frame emphasises that the direction of our measurement error has flipped, leaving many disoriented.