Distributed rate limiter with HRW in Elixir

This is a continuation of Elixir Cluster 101. So let's talk about putting what we learned into practice using the case of ratelimiting. Many implementations default to running in memory, meaning that they don't synchronize across multiple nodes. In most programming languages you would immediately reach for something like Redis to tackle this. It’s a great tool for sharing state across nodes, especially where your expectations on consistency and fault tolerance are lower, like the case of rate limits.

But we have the option of avoiding adding another service to our stack: we can take a rate limiter that runs in local memory and make it (mostly) consistent across a cluster of nodes. We do this by using an algorithm for assigning each key, whether IP, user ID, or organization ID, to a specific node, and then ensure that all rate limit lookups are routed to the correct node.

Traditionally this has been done using ExHashRing (the battle-tested consistent hashing implementation for Elixir), but for clusters with less than 10 nodes there’s an alternative that’s potentially even faster and has slightly better distribution: HRW (highest random weight, also known as rendezvous hashing). I wrote about this before on this blog. They both do the same thing, use magic math to associate any given key with a specific node, given a specific set of nodes. And both HRW and consistent hashing share the same incredibly important property: they cause minimal key re-assignment as the list of nodes changes. This means that if you auto-scale a node here and there, it won’t invalidate every key→node assignment, instead just a minimal subset.

Ok, that’s enough of that. Let’s take a look at the example code. I’m using Hammer here, but you can use any rate limiter.

Setting up the Hammer backend.

defmodule HammerBackend do
  use Hammer, backend: :ets
end

and then our rate limiter.

defmodule RateLimiter do
  use GenServer

  require Logger

  @scale :timer.minutes(60)
  @limit 10

  def…