MoE routing is just branch prediction

if you’ve spent time in both systems programming and ML infrastructure, you get this weird deja vu reading about transformer inference.
the words are new. KV cache, expert routing, paged attention. but the shape of the problems isn’t.
CPU architects ran into this exact wall thirty years ago: you have to act before you know what you need.
this post is about two places in a transformer’s inference path where that problem shows up almost unchanged.