Evaluating New Technology: Formative Evaluation of Intelligent Computer Assisted Instruction
Evaluation processes are touted to be productive mechanisms for the improvement of educational systems and products. And there is hard evidence of the utility of evaluation in actually improving technology based products and efforts in instructional development. Evaluation is known as well as to contain a strong negative potential. Evaluation can identify weaknesses in such a way as to inhibit exploratory behavior and risk taking on the part of researchers and developers. Playing it safe may be seen to be the winning strategy. Evidence of evaluation utilization studies suggests that when the focus of the evaluation is classification or accountability (good vs. bad; useful vs. wasteful), the openness of R & D project personnel to evaluation processes is inhibited. Formative evaluation, on the other hand, is evaluation whose specific function is to identify strengths and weaknesses for the purpose of improving the product or system under development (Baker, 1974; Baker and Alkin, 1973; Markle, 1967; Baker and Soloutos, 1974). The trick, of course, is in determining what should be studied, in what context the evaluation should take place, when evaluation processes are most useful, and in skilled hypothesis generation about what improvement options logically and feasibly may be implemented. In addition, the identification of weaknesses (no matter how benign the intentions of the evaluation may be) creates a documentary trail that might be misused by project managers or funding agency monitors.
KeywordsFormative Evaluation Project Staff Instructional Development Winning Strategy Evaluation Team
Unable to display preview. Download preview PDF.
- Baker, E.L. & Alkin, M.C. (1973). Formative evaluation in instructional development. “AV Communication Review,” “21(4)”.Google Scholar
- Baker, E.L. (1974). Formative evaluation of instruction. In J. Popham (Ed.), “Evaluation in education,” McCutchan.Google Scholar
- Baker, E.L. & Soloutos, W.A. (1974). “Formative evaluation of instruction.” Los Angeles: UCLA Center for the Study of Evaluation.Google Scholar
- Baker, E.L. (1983). “Evaluating educational quality: A rational design.” Invited paper, Educational Policy and Management, University of Oregon.Google Scholar
- Byrk, A. (Ed.) (1983). Stakeholder-based evaluation. “New Directions for Program Evaluation.” Vol. 17. San Francisco: Jossey Bass.Google Scholar
- Cronbach, L.S., et. al. (1980). “Toward reform of program evaluation.” San Francisco: Jossey Base.Google Scholar
- Markle, D.G. (1967). “An exercise in the application of empirical methods to instructional systems design.” (Final report: The development of the Bell system first aid and personal safety course, American Institutes for Research, Palo Alto, CA.) New York: American Telephone and Telegraph Company.Google Scholar
- Markle, S.M. (1967). Empirical testing of programs. In P.C. Lange (Ed.), “Programmed instruction.” The Sixty-sixth Yearbook of the National Society for the Study of Education, Part I I. Chicago: LASSE.Google Scholar