Gigatoken: making tokenisation care about the hardware
A first-principles tour of the SIMD scanners, cache-aware layouts, huge pages and worker scheduling behind Gigatoken.
点赞
收藏
点踩
分享评论
?
参与讨论
A first-principles tour of the SIMD scanners, cache-aware layouts, huge pages and worker scheduling behind Gigatoken.