It is no simple matter to look beyond preconceptions to see what happens when computers are used in education. Perceptions are shaped by values, and just as in paleontology, there are practical limitations to obtaining evidence. The process of adopting a computer-based innovation is often a gradual one, so that characterizing the use of the innovation requires long-term study. The relevant variables are often unknown in the beginning. And much of what children learn, especially with significant innovations, is not captured by standard assessment techniques, so that new methods of evaluation must be developed.
The result is that visionary statements about the effects of computer use in schools must be questioned. Publications and presentations on the use of computers in education are often characterized by visionary images of technology, but often, a close examination of how computers are really used in schools leads to the discovery that the reality of use does not match the ideal. Software developers, publishers, teachers, administrators, and researchers often describe what they want technology to be like in the classroom, not how it really works in the majority of classrooms.
At the same time, one must question broad critiques of classroom computer use that assert that no educational changes occur. It is unfortunately true that the educational system resists change, and computers alone are certainly no solution to problems of inequity, inadequate funding, cultural conflict, poverty, systemic resistance to change, and so on. Nevertheless, to see all classroom computer use as inconsequential tinkering is as unrealistic as to accept uncritically the most optimistic visions of computer use. As David Cohen says, the point is
... not that instructional change is impossible. It is that there are different kinds of change in instruction, and different organizational locales for it. There is a continual busy flutter at the heart of public education, as one infatuation after another holds sway. While everyone deplores these little romances many must find them enticing, for they persist. But when substantial pressure has been mobilized for change at the margins of education, the results often have been fairly impressive . . . One implication of my argument is that if we looked for significant instructional innovation in these subregions, rather than looking for it on average, everywhere, we would find more of it. [Cohen, 1988, p. 245]
For these reasons, one must be careful in analyzing the processes of social change, particularly with respect to the incorporation of innovations. The simplest approaches can easily go awry. One fallacy is to assume that an innovation per se causes changes; the converse fallacy is to assume that the operation of an innovation is always subsumed by existing cultural practices, and that its details are irrelevant to the issue of social change. One can miss other critical phenomena by failing to consider the distinctive functions and roles of different participants, such as users and developers, or by seeing an innovation as comprising merely fixed, physical artifacts, and not as cultural objects.
In order to study the processes of change involved with the introduction of QUILL into diverse classroom settings, we found it useful to adopt an idealization/realizations model for change. Briefly, the model says that what we see in a case of incorporation of an innovation into a social setting is a realization of that innovation. What the developers intend is only an idealization. Since realizations are likely to differ greatly from one another and from the idealization, we need to study the various realizations to understand the use of the innovation, its impact on the community, or its meaning for the users.
We see this diversity as a problem for traditional approaches to evaluation, which conceive of an innovation as fixed and well-defined, and evaluated by a set of uniform measures. These approaches are rightly concerned with precise assessments of performance before and after the use of the innovation and with selection of appropriate treatment and control groups. But methodological care in these areas is worth little if we cannot say what the innovation is or whether there is any consistent sense to the notion of the "treatment." Good evaluations of educational innovations need to include a "situated" component, which seeks to understand how that innovation looks in each of its different contexts of use.
The inevitable diversity of use for innovations also presents problems for traditional models of teacher education in which teachers are "trained" in the use of specific methods, approaches, or innovations. Our use of "training" in our early work with QUILL now appears misguided, particularly since it emphasized teachers learning procedures, rather than being supported in their own inquiries into teaching and learning. The training model misses the most salient fact about implementation: that it is a creative process involving critical analysis of the innovation's potential in the light of institutional and sociocultural context, physical resources, student needs, and pedagogical goals. The innovation process doesn't end, but begins, with the teacher.
Implications for curriculum development follow from this view of the teacher's role. Since the innovation doesn't even come into being until it is realized in an actual setting, the goal should not be to establish the endpoint for instruction, but rather, to supply the most useful tools possible for the re-creation process. That is, while an innovation may include technologies, activities, assessments, sequencing, and so on, it should be conceived as only a rough guide, to be actively shaped and re-defined to fit classroom realities and alternate goals.
When an innovation that calls for significant changes in teacher practices meets an established classroom system, "something has to give." Often, what gives is that the innovation is simply not used. Rarely is an innovation adopted in exactly the way the developers intended. Our study shows that the process of re-creation of the innovation is not only unavoidable, but a vital part of the process of educational change. Critical analysis of re-creations needs to be an important part of any evaluation. We believe that a deeper understanding of this process will highlight the fact that teachers need more support in attempting these re-creations. Their role in the innovation process is as innovators, not as recipients of completed products.