Describe 3 threats to internal validity using examples that

Describe 3 threats to internal validity using examples that you create. How does each threat impact the ability to determine cause-and-effect relationships? (15 points) MARKETING ANALYTICS

Solution

Threats to internal validity:

History: • Is not a threat for the two group (treatment/experimental and comparison/control) design because the comparison is between the treatment group and the comparison group. • If the history threat occurs for both groups, the difference between the two groups will not be due to the history event.

ex:  In a short experiment designed to investigate the effect of computer-based instruction, Ss missed some instruction because of a power failure at the school.

Maturation: Were changes in the dependent variable due to normal developmental processes operating within the subject as a function of time? • Is a threat to for the one group design. • Is not a threat to the two group design, assuming that participants in both groups change (“mature”)at same rate.

ex: The performance of first graders in a learning experiment begins decreasing after 45 minutes because of fatigue.

Statistical regression: An effect that is the result of a tendency for subjects selected on the bases of extreme scores to regress towards the mean on subsequent tests. When measurement of the dependent variable is not perfectly reliable, there is a tendency for extreme scores to regress or move toward the mean. The amount of statistical regression is inversely related to the reliability of the test.

ex: In an experiment involving reading instruction, subjects grouped because of poor pre-test reading scores show considerably greater gain than do the groups who scored average and high on the pre-test.

Selection: Refers to selecting participants for the various groups in the study. Are the groups equivalent at the beginning of the study? If subjects were selected by random sampling and random assignment, all had equal chance of being in treatment or comparison groups, and the groups are equivalent. Were subjects self-selected into experimental and comparison groups? This could affect the dependent variable. Selection is not a threat for the one group design but it is a threat for the two group design.

ex:The experimental group in an instructional experiment consisted of a high-ability class, while the comparison group was an averageability class.

Experimental Mortality: Differential loss of participants across groups. • Did some participants drop out? Did this affect the results? • Did about the same number of participants make it through the entire study in both experimental and comparison groups? • Is a threat for any design with more than one group.

ex: In a health experiment designed to determine the effect of various exercises, those subjects who find the exercise most difficult stop participating.

Testing: Did the pre-test affect the scores on the post-test? • A pre-test may sensitize participant in unanticipated ways and their performance on the post-test may be due to the pre-test, not to the treatment, or, more likely, and interaction of the pre-test and treatment. • Is a threat to the one group design. • Not a threat to the two group design. Both groups are exposed to the pre-test and so the difference between groups is not due to testing.

ex : In an experiment in which performance on a logical reasoning test is the dependent variable, a pre-test cues the subjects about the post-test