A Visual Guide to Gemma 4 12B

A new Gemma 4 model has been released and it’s an interesting one. There was an empty spot between the E4B and 26B A4B that needed to be filled and a 12B model fits nicely there. However, it wouldn’t be Google DeepMind without trying to do something special with that model and sharing it with the public.

So what is different with Gemma 4 12B?

It is encoder-free!

I’m going to be honest with you. The first time I heard “encoder-free” I was confused. Aren’t generative LLMs these days decoder-only anyway? It took me a minute to realize that the encoders for audio and video understanding were actually removed in this model but still had those multimodal capabilities.

The removal of the encoders, which are typically in charge of making sense of the multimodal inputs, places the burden of ma…