Survey Statistics: ANOVA follow-up

Rodney Sparapani commented on the ANOVA post about struggles with multilevel models, saying ANOVA is more straightforward. I also struggle with multilevel models, but I don’t find ANOVA more straightforward.

Alan Zaslavsky (the inspiration for this series) gave an example in his discussion of Andrew’s 2005 ANOVA paper: a survey of members of Medicare managed care health plans from Zaslavsky, Zaborski, and Cleary (2004). They found that for ratings of doctors, the variance explained by geography was greater than the variance explained by the health plans. To do this, they fit a multilevel model and used the estimated variance components to compute the percent of explained variance, e.g., sigma_state^2 / (sigma_plan^2 + sigma_state^2 + …) though they use Greek letter subscripts:

I think this would be more challenging to see in a classic multi-way ANOVA table with sums of squares, mean squares, F-statistics, and p-values. Let’s turn back to Andrew’s 2005 ANOVA paper for an example with a direct comparison.

In the latin square experiment example, say we want to see how much variance is explained by treatment (A, B, C, D, or E) versus the row and column effects. The paper proposes a display as in Figure 3, plotting estimated standard deviations for each component, analogous to the sigmas in Zaslavsky, Zaborski, and Cleary (2004):

The classical ANOVA table in Figure 1 does not give this information:

There is room for improvement in the standard analysis of variance table: it is read in order to assess the relative importance of different sources of variation, but the numbers in the table do not directly address this issue…
In summary, the standard ANOVA table gives all sorts of information, but nothing to directly compare the listed sources of variation.