The bloom filter trick that turned 170 object-storage reads into one (2.6s → 89ms)
We tried to speed up random trace_id lookups with a bloom filter and found it sped some queries up 29× while making others slower, and which one you get depends entirely on how your IDs are generated. TL;DR: Looking up a random trace_id across 170 index files in object storage took 2,584ms. Tantivy prunes files by min/max term, but a random 16-byte ID is scattered across the whole 128-bit space, so every file's range is [0, 2¹²⁸], nothing prunes, and all 170 files get opened. On object storage every one of
评论
?
参与讨论