SonicMoE: A Hardware-Efficient and Software-Extensible Blueprint for Fine-Grained MoEs

.post img{max-width:100%;height:auto}.post blockquote p{margin-top:.2em;margin-bottom:.2em;line-height:1.4}.post blockquote p:first-child{margin-top:0}.post blockquote p:nth-child(2),.post blockquote p:nth-child(3){margin-bottom:1em}.post blockquote strong{font-style:normal!important}.post blockquote{background-color:rgba(76,158,255,0.08);border:1px solid var(--global-theme-color,#4c9eff);border-left:4px solid var(--global-theme-color,#4c9eff);border-radius:4px;padding:1rem 1.5rem;font-size:inherit;color:in