Inside the black box of AI: How AI models actually think | Emmanuel Ameisen (Anthropic)

Emmanuel Ameisen is an interpretability researcher at Anthropic, where he looks inside LLMs like Claude to understand how they think. Before joining Anthropic, he spent years in applied ML at Stripe and Insight, and is the author of Building Machine Learning Powered Applications. His research has produced some of the most striking findings in the field: that models form general representations of concepts, plan many tokens ahead without stating those plans explicitly, and may exhibit something structurally similar to the distinction between conscious and unconscious processing in humans.

Listen on YouTube, Spotify, or Apple Podcasts

We discuss:

1. Why the “next-token predictor” framing undersells how models actually plan and think
2. Why anthropomorphizing AI can actually be the right approach
3. Why models are “grown,” not built
4. What the “J space” reveals about model self-awareness
5. The smooth scaling curve behind what look like sudden capability jumps
6. How interpretability connects to AI alignment and safety

Brought to you by:

WorkOS—Make your app enterprise-ready, with SSO, SCIM, RBAC, and more

DX—Engineering intelligence for the AI era

Where to find Emmanuel Ameisen:

• X: https://x.com/mlpowered

• LinkedIn: https://www.linkedin.com/in/ameisen

• Website: https://www.mlpowered.com

Referenced:

• Chris Olah on X: https://x.com/ch402

• Distill: https://distill.pub

• Ilya Sutskever on X: https://x.com/ilyasut

• Tracing the thoughts of a large language model: https://www.anthropic.com/research/tracing-thoughts-language-model

• A global workspace in language models: https://www.anthropic.com/research/global-workspace

• Scaling Laws for Neural Language Models: https://arxiv.org/abs/2001.08361

• Head of Claude Code: What happens after coding is solved | Boris Cherny: https://www.lennysnewsletter.com/p/head-of-claude-code-what-happens

• Claude’s Constitution: https://www.anthropic.com/constitution

• Golden Gate Claude: https://www.anthropic.com/news/golden-gate-claude

• The Rise and Fall of Agent Civilizations: The whole OpenAI/Hugging Face story in plain English: https://www.dwarkesh.com/p/openai-huggingface

• The Utility of Interpretability — Emmanuel Ameisen, Anthropic: https://www.latent.space/p/circuit-tracing

• Swyx on X: https://x.com/swyx

• Jev for beginners: how to use it and what to build: https://www.lennysnewsletter.com/p/jev-for-beginners-how-to-use-it-and

• Jev: https://typesafe.ai/blog/introducing-system-one-models-and-jev

• What is System 1 and System 2 Thinking?: https://thedecisionlab.com/reference-guide/philosophy/system-1-and-system-2-thinking

• Neuronpedia: https://www.neuronpedia.org

• Obsession: https://www.imdb.com/title/tt37287335

• Bref 2 on Hulu: https://www.hulu.com/series/bref2-251c6388-18e9-4545-9e5d-d5f3f6bf20d7

• Apple Watch Ultra: https://www.apple.com/apple-watch-ultra-4

Recommended books:

• A Brief History of Intelligence: Evolution, AI, and the Five Breakthroughs That Made Our Brains: https://www.amazon.com/dp/0063286343

• Stories of Your Life and Others: https://www.amazon.com/Stories-Your-Life-Others-Chiang/dp/1101972122

• Four Thousand Weeks: https://www.amazon.com/Four-Thousand-Weeks-Management-Mortals/dp/1250849357

Production and marketing by https://penname.co/. For inquiries about sponsoring the podcast, email podcast@lennyrachitsky.com.

Lenny may be an investor in the companies discussed.

My biggest takeaways from this conversation: