On graphical procedures for multiple comparisons

In a graphical procedure for comparing k treatment means in a one-way ANOVA, one displays uncertainty intervals around the sample means and judges any pair to be significantly different if and only if their uncertainty intervals do not overlap. A graphical procedure is a Multiple Comparison Procedure (MCP) if and only if it controls the experimentwise error rate. In this paper we consider some new graphical MCP’s for the unbalanced one-way ANOVA design. These procedures are based on different approximations to the Tukey-Kramer (TK) procedure (e.g., Kramer 1956). As such, they constitute alternatives to Gabriel (1978) (and its modification by Andrews, Snee, and Sarner 1980), which is based on approximating a less efficient MCP (the GT2 of Hochberg 1974). Two of the four procedures considered here are based on best and simple upper bounds to all the confidence-interval lengths of the TK method and hence must be conservative. The other two procedures are based on approximations (here too we have the best vs. the simple procedure), but simulations were used to find that their true experiment-wise error rates are less than the nominal ones; that is, these procedures are still on the conservative side. The choice of a particular procedure will depend then on the relative importance of simplicity, efficiency, and the security of having a controlled experimentwise error rate.