I built a language model out of cellular automata

the first thing my language model wrote that looked like a sentence was:
the little red fox ran home.o..ittle red fox ran home.o..ittle red fox r i was happy about the first part. less happy about the rest.
there was no attention in this model. no transformer blocks. just a grid of cells, each holding a vector, repeatedly talking to its eight neighbours. i wanted to know if a next-character predictor could emerge from only that local conversation.