The Road to Transformers

These are my notes from Eunsol Choi’s NLP class at UT Austin, cleaned up and stitched together into a single story (with much help from Claude).

Before Transformers ate the world, NLP was a patchwork of ideas that each solved one piece of the puzzle. You had classifiers that could tell spam from not-spam, sequence models that could read a sentence left to right, and embeddings that turned words into geometry. None of these were dead ends. The Transformer is best understood not as a clean break but as the moment all of these threads got braided together.