Between the Idea and the Silicon: Learning Why MLIR Exists

I spent part of this morning reading MLIR: A Compiler Infrastructure for the End of Moore's Law, and I have been really enjoying it.

One thing that kept coming back to me while reading it is just how early we still are in the development of AI. We are still learning how to effectively communicate what we want to compute all the way down to the hardware actually doing the work.

Modern ML systems can move through graphs, domain-specific representations, compiler passes, runtimes, and eventually lower-level instructions intended for CPUs, GPUs, TPUs, and other accelerators. The paper describes how fragmented this became, with different systems building their own intermediate representations and compiler infrastructure, often solving similar problems independently. MLIR emerged partly from the realization that we needed a better way to preserve information and move between these different levels of abstraction.

Instead of immediately reducing a computation into the lowest representation possible, you preserve its higher-level meaning while that meaning is still useful. Then, little by little, you lower it toward the machine.

We can describe an operation at the level at which a human or ML framework understands it, preserve enough structure for meaningful optimization, and eventually translate that intention into something the hardware can physically execute.

Underneath so much of this is mature infrastructure like LLVM, which is a work of art in itself. The paper is not arguing that LLVM failed, quite the opposite. LLVM demonstrated how powerful shared compiler infrastructure could be. But LLVM IR lives at a particular level of abstraction, while modern heterogeneous computing increasingly asks us to reason across many levels at once.

Maybe that is part of why AI still feels so incredibly early to me.

We have these extraordinarily capable accelerators and increasingly capable models, but between the idea and the silicon exists this enormous problem of translation, representation, optimization, memory movement, parallelism, and coordination.

We are still figuring out how to express what we mean in a way that allows the machine beneath us to do its best work.

And I find that incredibly exciting.