Memory is needed for more than just weights

Everyone quotes a model's file size, but that's only part of what it costs to run. This article breaks down where your VRAM really goes, weights, KV cache, activations, and overhead, then works out what fits on a modern GPU and what to do when a model doesn't.

添加评论
点赞收藏
点踩分享查看原文
评论
?
参与讨论