However if the GPU compute is not great (like in the M5 Max case) the full residency may not be faster the layer-major prefill (and will trash the experts cache in SSD streaming). So likely "3" is more interesting for the DGX Spark. We will see.

添加评论
点赞收藏
点踩分享查看原文
评论
?
参与讨论