Distilled DeepSeek into Gemma 4 26B-A4B vs 12B. Not very useful, but I learned a lot.

So I decided to learn how to fine-tune LLMs. Read a few guides from Unsloth, poked around, then stumbled on Unsloth Studio and wanted to test it out. The dataset I started from a set of relatively unrelated QA pairs — Natural Questions — and stripped the answers. Then I had DeepSeek v4 Pro (thinking disabled) repopulate them: - 1000 train + 200 val = 1200 requests total, cost $0.36 (~$0.0003/req). Honestly impressive on DeepSeek's side. Unsloth Studio It's a huge pain in the butt — infested with all kinds o