Consider performing an Ftest in a single factor ANOVA study

Consider performing an F-test in a single factor ANOVA study. Suppose that ANOVA model is applicable. Explain what would happen to your estimate of the error variance MSE, test statistic F, and P-value p under the following scenarios:
(a) The differences between the treatment means get larger, but the variation within the
groups remains the same.
(b) The variation within the groups gets larger, but the differences between the treatment
means remain the same.
(c) The differences between the treatment means stay the same, the variation within the
groups remains the same, but the sample size for each group gets larger

Solution

ANOVA. Analysis of variance, which is usually shortened to ANOVA, is the most commonly used statistical method for testing hypotheses about 3 or more means. The ANOVA statistic is called the F-test, after its developer, Fisher. We use ANOVA when we want to test the null hypothesis (Ho) that 3 or more means are drawn from the same population. If we have 2 means, we use the t-test which turns out to be just a special case of ANOVA.

Like the t, F depends on degrees of freedom to determine probabilities and critical values. But there is a difference between t and F in terms of the degrees of freedom concept. F has two different degrees of freedom to calculate. In contrast, t has only one formula for calculating degrees of freedom.

We will start by describing some kinds of scientific research contexts where you would use ANOVA.

ANOVA stands for analysis of variance. There are many forms of ANOVA, just as there are many forms of t-tests. We are beginning with the simplest form of ANOVA, the \"one-way analysis of variance for independent groups.\"

The one-way analysis of variance for independent groups applies to an experimental situation where there might be more than just two groups. The t-test was limited to two groups, but the Analysis of Variance can analyze as many groups as you want.

INDEPENDENT GROUPS. Imagine you would like to do an experiment where there are several independent groups, two or more. The research participants are randomly assigned to the various groups so that we can assume that the groups are independent of each other.

TREAT EACH GROUP DIFFERENTLY. Once you have the groups, you treat each group differently. The ways in which the groups are treated differently depends on what your independent variable is.

EXAMPLES OF IV\'S. For example, if your IV is Diet, there may be several groups of participants and each group eats a different diet. Or, if your IV is Memory Strategy, the several groups of research participants might receive different memory instructions. Or, if your IV is Type of Advertisement, the several groups might see different political campaign commercials. In general, each of several groups is treated differently in some manner that corresponds to the IV.

This is a generalization of the t-test for independent means in which there are only two groups. In ANOVA there can be three or four groups, or as many groups as you want. So the experimental context for ANOVA is just an extension of one for which the t-independent is appropriate. Keep in mind that, like the t for independent means, we are assuming independent groups. This is an important distinction. In a later lecture we will describe an ANOVA for correlated groups.

Treatment Levels

DIET EXAMPLE. Suppose that you want to study the effect of dietary fat on weight when calories are held constant. Suppose your research participants are Congolese males in their late twenty\'s. You take your pool of participants and randomly divide them into three groups. All groups will eat 2000 calories per day. The Low Fat group will get only 10% of its 2000 calories from fat. The Normal group will get 30% of its 2000 calories from fat. The High Fat group will get 50% of its 2000 calories eating from fat. After four months, you measure the weight of all participants.

INDEPENDENT VARIABLE. The IV is type of diet. More precisely, the IV is the percent of a person\'s calories that come from fat.

DEPENDENT VARIABLE. The DV is weight in pounds.

One Population or Three Populations?

We are once again going to call on your experience with the Double Sample Tool and the Detect Difference Game to understand statistical theory. This theory is just an extension of the logic we laid out in the Independent-t lecture.

In the Detect Difference lecture (based upon the Double Sample Tool and Detect Difference Game) we learned that two groups of data might each be separately drawn from two different populations or they might both be drawn from a single population. We will now extend that logic to three groups of data which might be sampled from three different populations or which might just be three samples from one population.

Treatment Effects

The topic of \"treatment effects\" should be a review and an extension of what we covered in the Detect Difference and Independent-t lectures.

ASSUME SCIENTIFIC HYPOTHESIS IS RIGHT. To think about treatment effects we must switch from the skeptic\'s model to the scientist\'s model by assuming that percent of dietary fat does affect body weight, even with calories held constant. In this case, then a good probability model of the experiment is that the data in the three groups is sampled from three treatment populations (scientist\'s model).

TREATMENT EFFECTS. Assume that the normal weight of Congolese males eating a normal diet is 150 pounds (Green Distribution or Baseline Population). Then (assuming the scientific hypothesis is true) a low (10%) diet should affect weight by lowering it from that baseline. And a high (50%) fat diet should affect weight by raising it from that baseline. And a normal (30%) diet should have no effect on weight because it is the same as the baseline assumption.

Treatment Effect Sizes

The size of the treatment effects for each treatment population will be the difference between the mu of the treatment population and the mu of the untreated baseline population shown in green.

We have assumed in our model that the mu of the untreated (green) population is 150 pounds. We get the treatment effect size for each treatment population by subtracting the mu of the green population from the mu of each of the treatment populations.

The current graphic shows explicit examples of calculating treatment effect size. Make these calculations for yourself and write them down in your notes.

Now let\'s let all this theory settle for a bit and go on to a scientific example which we will later use to learn how to calculate the F statistic.