Logical word count: a cheap, tokenizer-free measure of text length that tracks LLM token counts across languages, code, and mixed content

logical-word-count.md Logical Word Count Logical word count is a cheap, deterministic, tokenizer-free measure of text length that stays roughly proportional to LLM token counts across natural languages, code, and mixed content — while remaining equal to ordinary word count for plain English prose. It is defined in two rules: Wide (East Asian) characters count as ½ logical word each. This covers Chinese characters, Japanese kana, Korean Hangul, and CJK punctuation. Everything else
评论
?
参与讨论