Why I don’t like highest posterior density (HPD) intervals

This post is by Bob

There are several reasons I prefer central intervals.

First, you can’t tell how much of the probability is above or below a highest posterior density interval. For example, if I look at the HPD for an exponential distribution, it will run from 0 to the inverse cdf of the interval probability. If I look at the HPD for a normal, it matches the central interval.

Second, the range is going to be parameterization dependent. If I parameterize a probability in terms of log odds in (-infinity, infinity) vs. a log probability in (-infinity, 0) versus a probability (0, 1), I get different HPDs. Similarly, you’ll get different intervals for standard deviations vs. variance (even accounting for the squaring). In contrast, quantile-based intervals like central intervals are invariant to changes of variables.

Third, you don’t even get an interval if the highest density regions are disconnected, as in something like a Beta(0.5, 0.5) distribution. There, the highest density regions are from 0 up and from 1 down, so any highest density region is going to exclude the central interval of lowest density.

There is a lot of confusion about probabilities vs. densities in the wild. For instance, here’s a document from RStudio: Highest Posterior Density (HPD) Intervals. The document defines quantile-based (e.g., central) credible intervals in the usual way. Then it makes the common mistake of conflating density and probability, with the claim, “However, this definition does not guarantee the interval contains the highest probability regions.” Both the 90% HPD and 90% central interval contain exactly the same amount of probability mass, namely 90%. What this means is that the probability that a random draw falls in either is the same—90%. What the authors presumably meant is that central intervals don’t necessarily contain the highest density points. I think this confusion between probability and density is a lot of what’s motivating HPD. And also, the highest density points are going to depend on the parameterization (my second objection above).

On the other hand, Andrew’s the one who was pushing for replacing central intervals with HPD intervals in Stan, so I’m curious what his motivation for that change was given that he’s definitely not confused about probability versus density.