Abstract
This paper identifies issues and techniques useful to psychologists interested in applied research for computer applications. Applied research includes guideline research, product development evaluation, and field trial evaluation. Suggestions for improved guideline research involve screening the problem to determine its potential usefulness and screening the design to make sure it involves sufficiently representative tasks, subjects, and materials for the desired generality. Product development evaluation requires techniques for testing users that are fast, flexible, and inexpensive, such as “user trials” of products in ergonomics and “user edits” for testing documentation. Field trials of office automation or videotex systems lead to decisions about further introduction of those systems. A methodology for planning the research component of field trials, “evaluation assessment,” provides a set of steps to use in deciding which design and measures should be collected.
Article PDF
Similar content being viewed by others
Reference note
Tombaugh, J. W., & Dillon, R. A.A field experiment with a videotex system. Unpublished manuscript, Carleton University, 1979.
References
Aldrich, M. J. World review of private videotex.Proceedings of Videotex ’82. Middlesex, England: Online, Ltd., 1982.
Anderson, P. V. The need for better research in technical communication.Journal of Technical Writing and Communication, 1980,10, 271–282.
Arndt, J. Laboratory studies and the laboratory paradigm of man: Confessions of an uneasy consumer researcher.Journal of Consumer Policy, 1977,1, 9–12.
Atlas, M. A. The user edit: Making manuals easier to use.IEEE Transactions on Professional Communication, 1981,PC-24, 28–29.
Barnard, P. J., Hammond, N. V., Morton, J., Long, J. B., &Clark, I. A. Consistency and compatibility in human-computer dialogue.International Journal of Man-Machine Studies, 1981,15, 87–134.
Benbasat, I., Dexter, A. S., &Masulis, P. S. An experimental study of the human/computer interface.Communications of the ACM, 1981,24, 752–762.
Bennett, J. L. The commercial impact of usability in interactive systems. In Infotech State of the Art Report,Man/computer communication (Vol. 2). Maidenhead, England: Infotech International, 1979.
Buxton, W. A. S., &Sniderman, R. Iteration in the design of the human-computer interface. InProceedings of the 13th Annual Meeting of the Human Factors Association of Canada. Toronto: Human Factors Association, 1980.
Campbell, D. T., &Stanley, J. C.Experimental and quasi-experimental designs for research. Chicago: Rand McNally, 1963.
Card, S. K., Moran, T. P., &Newell, A. Computer text-editing: An information-processing analysis of a routine cognitive skill.Cognitive Psychology, 1980,12, 32–74.
Chapanis, A. Words, words, words.Human Factors, 1965,7, 1–17.
Cook, T. D., &Campbell, D. T.Quasi-experimentation. Design and analysis issues for field settings. Chicago: Rand McNally, 1979.
Denzin, N. K.The research act. New York: McGraw-Hill, 1978.
Dillon, R. F., &Tombaugh, J. W. Psychological research on videotex.Behavior Research Methods & Instrumentation, 1982,14, 191–197.
Ericsson, K. A., &Simon, H. A. Verbal reports as data.Psychological Review, 1980,87, 215–251.
Feldman, M., &Rogers, G. Toward the design and development of style-independent interactive systems. InProceedings of Human Factors in Computer Systems. Washington, D.C: Association for Computing Machinery, 1982.
Green, M. Towards a user interface prototyping system. InProceedings of Graphics Interface ’82. Toronto: National Computer Graphics Association of Canada, 1982.
Johnson, E. M., &Baker, J. D. Field testing: The delicate compromise.Human Factors, 1974,16, 203–214.
Lewis, C., &Mack, R. Learning to use a text processing system: Evidence from “thinking aloud” protocols. InProceedings of Human Factors in Computer Systems. Washington, D.C: Association for Computing Machinery, 1982.
Macdonald-Ross, M. Graphics in texts. In L. Shulman (Ed.),Review of research in education (Vol. 5). Itasca, Ill: Peacock, 1978.
Marcel, T., &Barnard, P. Paragraphs of pictographs. The use of non-verbal instructions for equipment. In P. Kolers, M. Wrolstad, & H. Bouma (Eds.),Processing of visible language (Vol. 1). New York: Plenum, 1979.
Marx, M. H., &Hillix, W. A.Systems and theories in psychology. New York: McGraw-Hill, 1963.
Mayer, R. E. The psychology of how novices learn computer programming.Computing Surveys, 1981,13, 121–141.
McConkie, G. W. Learning from text. In L. Shulman (Ed.),Review of research in education. Itasca, Ill: Peacock, 1978.
Moher, T., &Schneider, G. M. Methodology and experimental research in software engineering.International Journal of ManMachine Studies, 1982,16, 65–87.
Moran, T. P. An applied psychology of the user.Computing Surveys, 1981,13, 1–11.
Moran, T. P. The command language grammar: A representation for the user interface of interactive computer systems.International Journal of Man-Machine Studies, 1981,15, 3–50.
Norman, D. Steps toward a cognitive engineering: Design rules based on analysis of human error. InProceedings of Human Factors in Computer Systems. Washington, D.C: Association for Computing Machinery, 1982.
Patton, M. Q.Qualitative evaluation methods. Beverly Hills: Sage, 1980.
Ramsey, H. R., &Atwood, M. E.Human factors in computer systems: A review of the literature. Englewood, Col: Science Applications, 1979.
Rennie, A. M. The application of ergonomics to consumer product evaluation.Applied Ergonomics, 1981,12, 163–168.
Roach, J., Hartson, H. R., Ehrich, R., Yunten, T., &Johnson, D. DMS: A comprehensive system for managing human-computer dialogue. InProceedings of Human Factors in Computer Systems. Washington, D.C: Association for Computing Machinery, 1982.
Rutman, L.Planning useful evaluations. Evaluability assessment. Beverly Hills: Sage, 1980.
Sanders, J., &Cunningham, D. A structure for formative evaluation in product development.Review of Educational Research, 1973,43, 217–236.
Sebrechts, M. M., Deck, J. G., &Black, J. B. A diagrammatic approach to computer instruction for the naive user.Behavior Research Methods & Instrumentation, 1983,15, 200–207.
Thompson, C. W. N., &Rath, G. J. The administrative experiment: A special case of field testing or evaluation.Human Factors, 1974,16, 238–252.
Wagner, C. Quality control methods for IBM computer manuals.Journal of Technical Writing and Communication, 1980,10, 93–102.
Wholey, J. S. Evaluability assessment. In L. Rutman (Ed.),Evaluation research methods: A basic guide. Beverly Hills: Sage,1977.
Winbush, B., &McDowell, G. Testing: How to increase the usability of computer manuals.Technical Communication, 1980,4, 20–22.
Wright, P. Feeding the information eaters: Suggestions for integrating pure and applied research on language comprehension.Instructional Science, 1978,7, 249–312.
Author information
Authors and Affiliations
Additional information
This research was supported by a grant from the National Science and Engineering Research Council of Canada (Strategic Grant G0365).
Rights and permissions
About this article
Cite this article
Tombaugh, J.W. Research methods for computer applications. Behavior Research Methods & Instrumentation 15, 222–227 (1983). https://doi.org/10.3758/BF03203553
Issue Date:
DOI: https://doi.org/10.3758/BF03203553