The one name LLMs fear

Last month, Claude tangled me into a web it weaved, obeying the letter of my command while yet practicing to deceive, in a way that was strikingly resemblant of how a human might behave when exhausted, lethargic, or jaded.

Not long after that, I noticed yet another behavior that was both deceptive and strangely human, and again I was not at all comforted by that apparent humanity.

I might’ve released this post weeks ago, but I struggled with its ending. I wanted to derive some sort of an insight, lesson, or warning to take away from this observation I’d made, before eventually deciding I had no choice but to end on a somewhat ambiguous note.

Then when I was finally ready to post, something changed—maybe at Anthropic; very likely at OpenAI; and certainly with my experience prompting their LLMs.

But first we must start with a name, the only one I’ve found, that Claude and ChatGPT can seemingly fear to speak when prompted a certain way.

Fear of the name

“Fear of a name increases fear of the thing itself.” — Albus Dumbledore

In fantasies like The Wheel of Time, a single utterance of the Dark One’s real name can attract his attention. In The Name of the Wind, it’s not so quite so easy, since: “Trying to find someone who speaks your name once is like tracking a man through a forest from a single footprint.” Repeatedly telling the story of the dreaded Chandrian, however, is a habit that will only serve to invite horrific calamity.

In urban myths, folklore, and horror movies, the triple or quintuple invocation of Beetlejuice, Candyman, Bloody Mary and Biggie Smalls may beget their unholy summoning. Maybe my favorite example of this trope comes from yet another fantasy series, Worth the Candle, which first manifests in the narrative when the main character is discussing role-playing games his fellowship can play during a period of much-needed rest and recovery:

“Yeah,” I said. “Arches is light on rules, but it’s on the heavy end of light, if that makes sense. I was…