Memory is needed for more than just weights
Everyone quotes a model's file size, but that's only part of what it costs to run. This article breaks down where your VRAM really goes, weights, KV cache, activations, and overhead, then works out what fits on a modern GPU and what to do when a model doesn't.
评论
?
参与讨论