Inkling: A New Open-Weight 975B Moe with a Few Surprises
Short note on Thinking Machines Lab's 975B Inkling open-weight model, its benchmark profile, sparse MoE design, short convolutions, embedding RMSNorm, and relative-position bias.
点赞
收藏
点踩
分享评论
?
参与讨论
Short note on Thinking Machines Lab's 975B Inkling open-weight model, its benchmark profile, sparse MoE design, short convolutions, embedding RMSNorm, and relative-position bias.