Abstract
The elements of desirable research design for the evaluation of educational technology are discussed with reference to the context of existing research. Sources of internal invalidity, type of compared educational activity, and outcome measures are considered. Finally, recommendations regarding the direction of evaluation research are made. Research designs that take into account the characteristics of the learner, the software, and the teacher preferably within the framework of a model of the learning process should be adopted.
Article PDF
Similar content being viewed by others
Avoid common mistakes on your manuscript.
References
Baker, E. L., &O’Neil, H. F., Jr. (1987). Assessing instructional outcomes. In R. Gagne, (Ed.),Instructional technology: Foundations (pp. 343–377). Hillsdale, NJ: Erlbaum.
Borich, G. D., &Jemelka, R. P. (1982).Programs and systems: An evaluation perspective. New York: Academic Press.
Bradley, D. R. (1989). Computer simulation with DATASIM.Behavior Research Methods, Instruments, & Computers,21, 99–112.
Bradley, D. R., Hemstreet, R. L., &Ziegenhagen, S. T. (1992). A simulation laboratory for statistics.Behavior Research Methods, Instruments, & Computers,24, 190–204.
Chute, D. L. (1986). MacLaboratory for psychology: General experimental psychology with Apple’s Macintosh.Behavior Research Methods, Instruments, & Computers,18, 205–209.
Conn, G., Stafiniak, P., DiPasquale, M. C., &Harper, L. (1988). Effects of teaching an introductory psychology laboratory using a computerized research tool.Behavior Research Methods, Instruments, & Computers,20, 184–187.
Cooley, W. W., &Lohnes, P. R. (1976).Evaluation research in education. New York: Irvington.
Cordell, B. J. (1991). A study of learning styles and computer-assisted instruction.Computers & Education,16, 175–183.
Dougherty, T. J. (1990). Contour: A hypermedia environment for teaching about subjective contours and other visual illusions.Behavior Research Methods, Instruments, & Computers,22, 223–227.
Duncan, N. C. (1991). CAI-enhanced exam performance in a research design course.Behavior Research Methods, Instruments, & Computers,23, 324–327.
Goolkasian, P. (1989). Computerized laboratories for psychology instruction: How successful are they?Behavior Research Methods, Instruments, & Computers,21, 148–150.
Grabe, M., Petros, T., &Sawler, B. (1989). An evaluation of computer assisted study in controlled and free access settings.Journal of Computer Based Instruction,16, 110–116.
Hannafin, M. J., &Carney, B. W. (1991). Effects of elaboration strategies on learning and depth of processing during computer-based instruction.Journal of Computer-Based Instruction,18, 77–82.
Heinessen, R. K., Glass, C. R., &Knight, L. A. (1987). Assessing computer anxiety: Development and validation of the Computer Anxiety Rating Scale.Computers in Human Behavior,3, 49–59.
Hunt, E. (1985). Verbal ability. In R. Sternberg (Ed.),Human abilities: An information-processing approach (pp. 99–123). New York: W. H. Freeman.
Isaacs, M., Costenbader, V., Reading-Brown, M., &Goodman, G. (1992). Using a computer simulation in the research, training, and evaluation of school psychologists.Behavior Research Methods, Instruments, & Computers,24, 165–168.
Kulik, C. L. C., &Kulik, J. A. (1991). Effectiveness of computer-based instruction: An updated analysis.Computers in Human Behavior,7, 75–94.
Kulik, J. A., &Kulik, C. L. C. (1987). Review of recent research literature on computer-based instruction.Contemporary Educational Psychology,12, 222–230.
Lambert, M. E., &Lenthall, G. (1989). Effects of psychology courseware use on computer anxiety in students.Computers in Human Behavior,5, 207–214.
Leinhardt, G. (1980). Modeling and measuring educational treatment in evaluation.Review of Educational Research,50, 393–420.
Levidow, B. B., Hunt, E., &McKee, C. (1991). The DIAGNOSER: A HyperCard tool for building theoretically based tutorials.Behavior Research Methods, Instruments, & Computers,23, 249–252.
Liefeld, J. P., &Herrmann, T. F. (1990). Learning consequences for University students using computerized mastery testing.Educational Technology, Research & Development,38, 19–25.
McNeil, B. J., &Nelson, K. R. (1991). Meta-analysis of interactive video instruction: A 10 year review of achievement effects.Journal of Computer-Based Instruction,18, 1–6.
Meier, S. T. (1988). Predicting individual differences in performance on computer-administered tests and tasks: Development of the Computer Aversion Scale.Computers in Human Behavior,4, 175–187.
Neale, J. M., &Liebert, R. M. (1986).Science and behavior: An introduction to methods of research (3rd ed.). Englewood Cliffs, NJ: Prentice-Hall.
Neuman, D. (1991). Learning disabled students’ interactions with commercial courseware: A naturalistic study.Educational Technology, Research & Development,39, 31–49.
Niemiec, R., &Walberg, H. J. (1987). Comparative effects of computer-assisted instruction: A synthesis of reviews.Journal of Educational Computing Research,3, 19–37.
Perone, M. (1991). Computer-based methodology laboratories: 1. An undergraduate course in experimental psychology.Behavior Research Methods, Instruments, & Computers,23, 121–126.
Petty, L. C., &Rosen, E. F. (1990). Increase in mastery levels using a computer-based tutorial/simulation in experimental psychology.Behavior Research Methods, Instruments, & Computers,22, 216–218.
Ransdell, S. E. (1990). Using a real-time replay of students’ word processing to understand and promote better writing.Behavior Research Methods, Instruments, & Computers,22, 142–144.
Ransdell, S. [E.] (1992). Incorporating educational software into large introductory psychology lectures and labs.Behavior Research Methods, Instruments, & Computers,24, 172–173.
Ransdell, S. [E.], &Levy, C. M. (1989). A flexible and interactive software package for a laboratory in cognition and perception.Behavior Research Methods, Instruments, & Computers,21, 160–162.
Ray, R. D., &Mitchell, N. (1992). BehSys Anal: An artificially intelligent hypermedia resource library for descriptive behavioral research.Behavior Research Methods, Instruments, & Computers,24, 180–189.
Reeves, T. C. (1986). Research and evaluation models for the study of interactive video.Journal of Computer-Based Instruction,13, 102–106.
St. James, J. D. (1989). The MEL Library in the undergraduate research methods course.Behavior Research Methods, Instruments, & Computers,21, 245–247.
Sawyer, T. A. (1988). The effects of computerized and conventional study guides on achievement in college students.Journal of ComputerBased Instruction,15, 80–82.
Schloss, P. J., Sindelar, P. T., Cartwright, G. P., &Schloss, C. N. (1986). Efficacy of higher cognitive and factual questions in computer assisted instruction modules.Journal of Computer-Based Instruction,13, 75–80.
Schloss, P. J., Wisniewski, L. A., &Cartwright, G. P. (1988). The differential effect of learner control and feedback in college students’ performance on CAI modules.Journal of Educational Computing Research,4, 141–150.
Schneider, W. (1988). Micro Experimental Laboratory: An integrated system for IBM PC compatibles.Behavior Research Methods, Instruments, & Computers,20, 206–217.
Shlechter, T. M. (1988). An examination of the research evidence for computer-based instruction. In H. R. Hartson & D. Hix (Eds.),Advances in human-computer interaction (Vol. 2, pp. 316–367). Norwood, NJ: Ablex.
Simonson, M. R., Maurer, M., Montag-Torardi, M., &Whitaker, M. (1987). Development of a standardized test of computer literacy and a computer anxiety index.Journal of Educational Computing Research,3, 231–247.
Thomas, D. L., &Bostow, D. E. (1991). Evaluation of pre-therapy computer-interactive instruction.Journal of Computer-Based Instruction,18, 66–70.
Tobias, S. (1987). Learner characteristics. In R. Gagne (Ed.),Instructional technology: Foundations (pp. 207–231). Hillsdale, NJ: Erlbaum.
Trowbridge, D. (1987). An investigation of groups working at the computer. In D. E. Berger, K. Pezdek, & W. Banks (Eds.),Applications of cognitive psychology: Problem solving, education and computing (pp. 47–57). Hillsdale, NJ: Erlbaum.
Underwood, G., &Underwood, J. D. M. (1987). The computer in education: A force for change? In F. Blackler & D. Oborne, (Eds.),Information technology and people: Designing for the future (pp. 167–190). Letchworth, U.K.: British Psychological Society.
Varnhagen, C. K., &Zumbo, B. D. (1990). CAI as an adjunct to teaching introductory statistics: Affect mediates learning.Journal of Educational Computing Research,6, 29–40.
Welsh, J. A., &Null, C. H. (1991). The effects of computer-based instruction on college students’ comprehension of classic research.Behavior Research Methods, Instruments, & Computers,23, 301–305.
Author information
Authors and Affiliations
Rights and permissions
About this article
Cite this article
Duncan, N.C. Evaluation of instructional software: Design considerations and recommendations. Behavior Research Methods, Instruments, & Computers 25, 223–227 (1993). https://doi.org/10.3758/BF03204501
Issue Date:
DOI: https://doi.org/10.3758/BF03204501