Direct Instruction Evidence: Project Follow Through
Note: Glossary links are in boldface.
Recall that Project Follow Through was conceived as a social program whose primary goal was to provide disadvantaged school-age children with compensatory education. Operating from the observation that children from poorer neighborhoods generally performed well below the national average, in 1967 President Lyndon Johnson directed the US Office of Education to improve the early educational experiences of children in need. The experimental design used a methodology referred to as planned variation. Under the planned variation method a variety of different educational models were implemented. Each model was allowed to follow its own theory, to implement its own curriculum and procedures, and was responsible for training and supervising its teachers. The result was a crude experiment comparing the composite models with one another - not a controlled analysis of the specific elements of the models. Coombs (1998) reported that over 700,000 children in 170 disadvantaged communities across the United States participated in this 1 billion-dollar study. Data on a variety of standardized achievement measures were collected by Stanford Research Institute and analyzed by Abt Associates. Outcome comparisons were also made with children whose schools did not participate in special programming funded by Project Follow Through. These children served as "control" subjects in that they received the educational experiences that were standard in their school. The three goals of the project were to increase basic knowledge and skills, to improve cognitive and problem-solving skills, and to promote positive self-concept among the participating children. The purpose of the experiment was to identify "best practices" so that the most effective methods could subsequently be applied to achieve the three core goals in the disadvantaged children across the United States. The project was a cornerstone of President Johnson's War on Poverty.
The various models that were selected to participate fell into three broad categories mirroring the three core goals. The Affective models focused upon promoting positive self-esteem as a means to improve the learning and development of disadvantaged learners. The assumption of the affective models was that experiences that raised the children's self-esteem would provide a motivational catalyst to learning and achievement at all levels. The Cognitive models emphasized the development of higher-order cognitive skills -- a top-down approach that targeted thinking and problem-solving skills. The cognitive models made the assumption that the lower-order basic skills and self-esteem would be acquired automatically in the process of mastery complex problems. The Basic Skills models (including Engelmann's Direct Instruction) focused upon the children's acquisition of basic knowledge and skills - a bottom-up approach that targeted basic knowledge and foundational language and numeracy skills. The basic skills models made the assumption that higher-order thinking and problem-solving composite skills, and self-esteem, would arise from mastery of component skills.
Coombs (1998) presented some of the comparative data. Figure 1 below shows the achievement scores for the various models in reading, mathematics, spelling and language. Shown are the average percentile scores on measures of each skill area for the nine models that were involved in the analysis. To appreciate these data one must be aware that the typical scores of disadvantaged children are around the 20th percentile, compared with the national average that would be, of course, at the 50th percentile. This 20th percentile level for the population of disadvantaged children is the baseline against which the treatments were compared. Engelmann's Direct Instruction model and the Behavior Analysis model developed at the University of Kansas were the only treatments to have across-the-board positive effects on these essential components of early education. Direct Instruction was the clear winner, raising participating children's average scores in these basic skill areas to near the national average. Direct Instruction was first in reading, math, spelling and language. Two models (Southwest Lab and Open Education) had the opposite effects, yielding overall negative effects in all four skills areas relative to controls. Most models had mixed effects.
Figure 1. Reading, math, spelling and language achievement for the nine
models. Scores above the horizontal line at the 20th
percentile indicate a positive effect of the program on achievement
in that skill area relative to "control" children who did
not participate in the project. Scores below the horizontal line
indicate a negative effect, relative to controls. Copyright 1998 News World Communications, Inc. Reprinted
with permission of The Washington Times. Visit their web site at
http://www.washtimes.com
Basic Skills
As might be expected, the models that emphasized the learning of basic skills and knowledge were generally superior in promoting the acquisition of those skills. Compared to Non-Follow Through controls, most of the cognitive models and all of the self-esteem models actually had a negative impact on core knowledge (See the dark bars in Figure 2 below [from Coombs, 1998]).
Figure 2. Basic skill, cognitive skill and self-esteem scores of children
who participated in the nine models. Scores are relative to a
baseline of zero that represents children who did not participate in
Project Follow Through programs. Scores above the zero line indicate
a positive change in performance in the target area, and scores below
the zero line indicate a negative change in performance in the target
area. (See figure for legend.) Copyright 1998 News World Communications, Inc. Reprinted
with permission of The Washington Times. Visit their web site at
http://www.washtimes.com
Cognitive Skills
Figure 2 (gray bars) also shows the relative performance in the cognitive area. Noteworthy was the finding that the Direct Instruction model was the only model to show positive impact on cognitive skill development. Models designed to explicitly teach cognitive skills actually had negative impact on those skills relative to Non-Follow Through controls. The focus on top-down learning seemed to be based upon a false assumption.
Affective Development
In the area of self-esteem the results were clear. Figure 2 (white bars) shows those models that focused on the teaching of basic skills, and one cognitive model, had positive effects on children's self-esteem. The remaining models, including those designed explicitly to improve self-esteem, appeared to have a negative impact on the children's affective development.
Overall, the only model that had a consistently positive effect on all three aspects of learning was Direct Instruction. The winner was clear. Given the dramatic learning and achievement outcomes in basic skills, cognitive skills and self-esteem of this large scale study, one would have expected that the United States Office of Education would have strongly endorsed and supported the use of Direct Instruction as the model of choice. But the support and adoption decisions seemed to be contrary to the learning outcomes (See Watkins (1988) for an extensive discussion.)
"The results make a mockery of current reforms, because Follow Through clearly showed that some approaches work well and some flop; however, the ones that flopped the most emphatically are still alive today and still promoted vehemently by teachers' groups like the International Reading Association, the National Council of Teachers of English, and the National Council of Teachers of Mathematics. The approaches that did well were roughly the opposite of the romantic notions and theories espoused by these groups. The better performing sponsors presented highly structured instruction that had tight teacher-performance requirements and practices that are 'behavioral'." Englemann, 1992, pp. 4-5)
Why? It would appear that Direct Instruction was not endorsed or supported because the theory and methods espoused by Engelmann were inconsistent with the dominant thinking of American educators. In describing the rejection of Direct Instruction by the educational establishment, the web site of the American Federation of Teachers (American Federation of Teachers, 1998) asserts that Engelmann's programs were criticized for being too rigid and for emphasizing basic skills. The outcome data were discredited and ignored. Schools of Education in universities, boards of education, the Ford Foundation and commercial publishers argued against the research and the data, and they won. Opinion triumphed over data (Engelmann, 1992; Adams & Engelmann, 1996)