From Mark V. Shaney to AI Village, and beyond

Last December, Rob Pike got big mad at AI Village. Simon Willison covered the associated events and discussions.

I sympathize with Rob Pike's reaction, though in fairness we need to note that he helped develop one of the very first systems to spread "AI slop" into the real world, more than four decades ago:

Mark V. Shaney is a synthetic Usenet user whose postings in the net.singles newsgroups were generated by Markov chain techniques, based on text from other postings. The username is a play on the words "Markov chain". Many readers were fooled into thinking that the quirky, sometimes uncannily topical posts were written by a real person.

The system was designed by Rob Pike with coding by Bruce Ellis. Don P. Mitchell wrote the Markov chain code, initially demonstrating it to Pike and Ellis using the Tao Te Ching as a basis. They chose to apply it to the net.singles netnews group.

The program is fairly simple. It ingests the sample text (the Tao Te Ching, or the posts of a Usenet group) and creates a massive list of every sequence of three successive words (triplet) which occurs in the text. It then chooses two words at random, and looks for a word which follows those two in one of the triplets in its massive list. If there is more than one, it picks at random (identical triplets count separately, so a sequence which occurs twice is twice as likely to be picked as one which only occurs once). It then adds that word to the generated text.

Then, in the same way, it picks a triplet that starts with the second and third words in the generated text, and that gives a fourth word. It adds the fourth word, then repeats with the third and fourth words, and so on. This algorithm is called a third-order Markov chain (because it uses sequences of three words).

I wonder whether Wikipedia's description of the algorithm is accurate — Markov language models in the mid-80s (and in fact from WWII onward) were using Good-Turing estimates of n-gram probability, not just picking at random from a prefix table.

For a more favorable reaction to real-world integration of AI-ish methods, see Alexy Khrabrov's latest idea, First Pair. It's not clear to me whether outputs like "AI is Just a Unix Pipe" and "The First Pair Bell Labs Manifesto" will make Rob more angry — but since there's a human in the loop, maybe those tributes will be less offensive to him.

See also: Paul Krugman, "When the chips are down", 7/29/2026.