A group of psychologists once measured 60 different variable

A group of psychologists once measured 60 different variables on a sample of schizophrenic people. Using theses 60 variables, they did 60 separate hypothesis tests of the mean (one mean for each of the 60 variables). Two of these tests were significant at the 5% level. Suppose that in REALITY all 60 null hypotheses are true.

a) What is the probability of rejecting any one specific test at the 5% significance level? (I\'m asking you the pick just one hypothesis test. Then, you need to tell me the chance that if the experiment is repeated for just this one test, the new result will lead him or her to decide to reject the null hypothesis.)

b) Now suppose this group of psychologists repeats all 60 experiments. About how many of the 60 hypothesis tests do we expect will have a p-value ? .05?

Solution

(a)

level of significance
= alpha
= P(Type I error)
= P(reject a true null hypothesis)
= .05 (given)

So, if we let alpha = .05, 5% of the time we will reject Ho when it is true.

So, if we randomly select 1 variable where Ho is true, there is a 5% of rejecting it.

Answer: .05

(b)

If all 60 null hypothesis are true, we expect to reject Ho 5% of the time. So, are expected to reject .05(60) = 3 null hypotheses. So, we expect 3 of them to have p-value\'s less than or equal to .05, because we reject Ho when the p-value is less than or equal to alpha.

Answer: 3

----------------

I hope this helped. If you have any questions, please ask them in the comment section. :)