Describe the logic underlying the use of multiple comparison

Describe the logic underlying the use of multiple comparison procedures

Solution

When there are g groups, there are g(g-1)/2 comparisons that can be made. There is some redundancy here because there are only g-1 independent pieces of information. Use the Studentized range statistic for g groups and the appropriate number of error degrees of freedom. To remove the penalty on the g-1 independent pieces of information, perform the Studentized range test at the 1-(1-alpha)^g-1 level of significance. In the case of 4 groups (3 independent pieces of information), this corresponds to performing the Studentized range test at the 0.143 level of significance.

When \'m\' independent tests of true null hypotheses are carried out at some level alpha, the probability that none are statistically significant is (1-alpha)^m and the Type I error is 1-(1-alpha)^m. Therefore, to insure that the Studentized range statistic does not penalize me, I use at the level that corresponds to having used for my individual tests. In the case of 4 groups, there are three independent pieces of information. Testing the three peices at the 0.05 level is like using the Studentized range statistic at the 1-(1-0.05)3 (=0.143) level. That is, if I use the Studentized range statistic with =0.143, it is just as though I performed my 3 independent tests at the 0.05 level.