The Economics of Speculative Decoding
Two underexplored axes: what MoE routing does to the decode roofline, and how
compressed attention takes away the slack that used to make speculated tokens free.
点赞
收藏
点踩
分享评论
?
参与讨论
Two underexplored axes: what MoE routing does to the decode roofline, and how
compressed attention takes away the slack that used to make speculated tokens free.