Everything is a UOp – reading the deep learning stack behind comma[dot]AI

i’ve used PyTorch for years. a @ b, .backward(), .cuda(). works every time. i never thought about what’s underneath.
then i came across tinygrad. it’s the deep learning stack behind comma.ai’s openpilot, the open-source self-driving system. 17K lines of Python. PyTorch is 3 million lines of C++. this thing fits the whole pipeline, including the compiler, in less code than some test suites.
i figured i’d spend an afternoon reading it.