Index Compression, Query Execution Improvements

The Marginalia Search index has recently seen some design tweaks to make it perform better, primarily the introduction of postings list compression.

Last year, the index was partially re-implemented with SSDs in mind. This was largely a success, but left some lingering issues with tail latencies that sometimes weren’t what they needed to be.

To ensure predictable execution times, the query execution is provided a timeout value, after which it will wrap up and return the best results it’s found. Query execution was so flaky that the actual timeout used when terminating the execution used to be something like 50ms lower than the provided value. This is obviously not a fantastic state of affairs.