Why are effect sizes rather than test statistics used when c

Solution

For studies involving large sample sizes (e.g., > ~ 400), a different problem occurs with significance testing because even small effects are likely to become significant significant, although these effects may be trivial. In these situations, more attention should be paid to effect sizes than to statistical significance testing.

When there is no interest in generalising (e.g., we are only interested in the results for the sample), there is no need for significance testing. In these situations, effect sizes are sufficient and suitable.

When examining effects using small sample sizes, significance testing can be misleading because its subject to Type II errors. Contrary to popular opinion, statistical significance is not a direct indicator of size of effect, but rather it is a function of sample size, effect size and p level. In these situations, effect sizes and confidence intervals are more informative than significance testing.

When examining effects using large samples, significant testing can be misleading because even small or trivial effects are likely to produce statistically significant results. This can be dealt with by reporting and emphasising significance test results.

Some advantages of effect size reporting are that:

It tends to be easier for practitioners to intuitively relate to effect sizes (once it is explained) than significance testing

Effect sizes facilitate ready comparison with internal or external benchmarks

Confidence intervals can be placed around effect sizes (providing an equivalent to significance testing if desired)