Explain why evaluation researchers find it difficult to empl

Explain why evaluation researchers find it difficult to employ the classical experimental design. Which designs are more typically used and why? Subject is Criminal Justice Research.

Solution

EXPERIMENTAL DESIGNS

True experimental designs include:

Pre-test/Post-test control group design

Solomon Four-Group design

Post-test only control group design

Pre-test/Post-test control group design
This is also called the classic controlled experimental design, and the randomized pre-test/post-test design because it:

1) Controls the assignment of subjects to experimental (treatment) and control groups through the use of a table of random numbers.

This procedure guarantees that all subjects have the same change of being in the experimental or control group. Because of strict random assignment of subjects, it is assumed that the two groups are equivalent on all important dimensions and that there are no systematic differences between the two groups. Researchers may substitute matching for random assignment. Subjects in the two groups are matched on a list of characteristics that might affect the outcome of the research (e.g., sex, race, income). This may be cheaper but matching on more than 3 or 4 characteristics is very difficult. And if the researcher does not know which characteristics to match on, this compromises internal validity.

2) Controls the timing of the independent variable (treatment) and which group is exposed to it.

Both group experience the same conditions, with the exception of the experimental group, which receives the influence of the independent variable (treatment) in addition to the shared conditions of the two groups.

3) Controls all other conditions under which the experiment takes place.

Nothing but the intervention of the independent (treatment) variable is assumed to produce the observed changes in the values of the dependent variable.

The steps in the classic controlled experiment are:

1) randomly assign subjects to treatment or control groups;

2) administer the pre-test to all subjects in both groups;

3) ensure that both groups experience the same conditions except that in addition the experimental group experiences the treatment;

4) administer the post-test to all subjects in both groups;

5) assess the amount of change on the value of the dependent variable from the pre-test to the post-test for each group separately.

experimental designs are often touted as the most \"rigorous\" of all research designs or, as the \"gold standard\" against which all other designs are judged. In one sense, they probably are. If you can implement an experimental design well (and that is a big \"if\" indeed), then the experiment is probably the strongest design with respect to internal validity. Why? Recall that internal validity is at the center of all causal or cause-effect inferences. When you want to determine whether some program or treatment causes some outcome or outcomes to occur, then you are interested in having strong internal validity. Essentially, you want to assess the proposition:

If X, then Y

or, in more colloquial terms:

If the program is given, then the outcome occurs

Unfortunately, it\'s not enough just to show that when the program or treatment occurs the expected outcome also happens. That\'s because there may be lots of reasons, other than the program, for why you observed the outcome. To really show that there is a causal relationship, you have to simultaneously address the two propositions:

If X, then Y

and

If not X, then not Y

Or, once again more colloquially:

If the program is given, then the outcome occurs

and

If the program is not given, then the outcome does not occur

If you are able to provide evidence for both of these propositions, then you\'ve in effect isolated the program from all of the other potential causes of the outcome. You\'ve shown that when the program is present the outcome occurs and when it\'s not present, the outcome doesn\'t occur. That points to the causal effectiveness of the program.