The sorry state of Java deserialization

I’ve been on a bit of a frustration-driven quest to solve a problem I frequently encounter working on the search engine, that is, reading data from disk.

You’d think this would be a pretty basic thing, but doing this in a way that is half-way performant is surprisingly hard and requires avoiding basically all the high level tools at your disposal.

There’s a common sentiment that modern hardware is fast, so this may not matter, but we aren’t speaking a 30% performance hit, the the question is how many orders of magnitude you’re willing to forego.