Scheffe's Test

Is that Superman? A nova? Oh, no...just ANOVA. The boring one.

ANOVA is a statistical tool used to analyze variance. It helps statisticians see how much their independent variables have an effect on the dependent variable (as opposed to random effects).

For instance, say you notice your boss wears blue shirts often on Mondays and red shirts on Fridays. You start collecting data. What day of the week is it? What’s the weather? Does he have a meeting that day? The dependent variable is what you’re trying to predict, since it “depends” on another factor: his shirt color. Your independent variables are the fixed ones: the days of the week, meeting or no meeting, the weather.

An ANOVA test will help you determine how much the weather, meetings, and day of the week affect your boss’s shirt color...or if his shirt color is more due to other factors, like random chance.

Scheffe’s Test is another tool...a tool used after using the ANOVA tool. If the ANOVA test was statistically significant, it’s time to pull out Scheffe’s test. This only works for “unplanned” comparison of groups. It was unplanned, since you’re only doing it because the ANOVA test kinda suggested you should. Unplanned tests, like Scheffe’s Test, are less statistically powerful than planned tests; they're determined before even getting your statistician hands on the data.

Other tests that can be done on unplanned comparisons are the Tukey-Kramer method and the Bonferroni test. It can be good to try a few of these out, since they all have low statistical power.