A Yes this could be important B Maybe depends on the actua

A. Yes - this could be important.

B. Maybe - depends on the actual results.

C. No - too many tests were done.

Several years ago, some researchers at my university asked me to recheck the statistics on a paper they were writing about the impacts of a group of new science courses. They had performed 63 statistical tests on their data at alpha = 5%, and had found 4 \'\'significant\'\' results. They were excited. Should they have been? A. Yes - this could be important. B. Maybe - depends on the actual results. C. No - too many tests were done.

Solution

? = 5%

So, probability of a false significant result = 0.05

Probability of getting 4 false positives in 63 tests = (1