Abstract
A computer program was developed to collect solution protocols with written explanations, with a level of detail difficult to obtain with traditional pencil-and-paper tasks. The program automatically presents the instructions, problems, solution entered, changes to the solution, explanation prompts, solution feedback, and system help in a Windows environment. The time to make each move and type each explanation is recorded, making it possible to compare solution times across treatment groups. Protocols are stored in individual ASCII files, eliminating transcription, and a summary data set for all protocols is created. The program was tested in a study where individual protocols were collected in small groups. This program can readily be adapted to different experimental contexts and can be a useful educational tool.
Article PDF
Similar content being viewed by others
Avoid common mistakes on your manuscript.
References
Ahlum-Heath, M. E., &DiVesta, F. J. (1986). The effect of conscious controlled verbalization of a cognitive strategy on transfer in problem solving.Memory & Cognition,14, 281–285.
Allwood, C. M. (1990). On the relation between justification of solution method and correctness of solution in statistical problem solving.Scandinavian Journal of Psychology,31, 181–190.
Berardi-Coletta, B. (1990).The effects of verbalization on problem solving and problem solution transfer—A metacognitive perspective. Unpublished doctoral dissertation, University of Illinois at Chicago.
Berardi-Coletta, B., Buyer, L. S., Dominowski, R. L., &Rellinger, E. S. (1995). Metacognition and problem solving: A process-oriented approach.Journal of Experimental Psychology: Learning, Memory, & Cognition,21, 205–223.
Chi, M. T. H., &Bassok, M. (1989). Learning from examples via self-explanations. In L. B. Resnick (Ed.),Knowing, learning, and instruction: Essays in honor of Robert Glaser (pp. 251–282). Hillsdale, NJ: Erlbaum.
Chi, M. T. H., Bassok, M., Lewis, M. W., Reimann, P., &Glaser, R. (1989). How students study and use examples in learning to solve problems.Cognitive Science,13, 145–182.
Davis, J. M., Carey, M. H., Foxman, D. N., &Tarr, D. B. (1968). Verbalization, experimenter presence, and problem solving.Journal of Personality & Social Psychology,8, 299–302.
Dörner, D. (1978). Self-reflection and problem solving. In F. Klix (Ed.),Human and artifical intelligence (pp. 101–107). Berlin: Deutscher Verlag der Wissenschaften.
Ericsson, K. A., &Simon, H. A. (1980). Verbal reports as data.Psychological Review,87, 215–251.
Ericsson, K. A., &Simon, H. A. (1994).Protocol analysis: Verbal reports as data. Cambridge, MA: MIT Press.
Gagné, R. M., &Smith, E. C., Jr. (1962). A study of the effects of verbalizations on problem solving.Journal of Experimental Psychology,65, 12–18.
Greeno, J. G. (1978). Natures of problem solving abilities. In W. K. Estes (Ed.),Handbook of learning and cognitive processes: Vol. 5. Human information processing (pp. 240–269). Hillsdale, NJ: Erlbaum.
Johnson, M. M. S. (1993). Thinking about strategies during, before and after making a decision.Psychology & Aging,8, 231–241.
Katona, G. (1940).Organizing and memorizing. New York: Columbia University Press.
Kristovich, S. A. R. (1988).The effect of verbal report time and reason-giving on solution transfer. Paper presented at the annual meeting of the Midwestern Psychological Association, Chicago.
Luchins, A. S. (1942). Mechanization in problem solving the effect of Einstellung.Psychological Monographs,54 (3, Whole No. 248).
McGeorge, P., &Burton, A. M. (1989). The effects of concurrent verbalization on performance in a dynamic systems task.British Journal of Psychology,80, 455–465.
Polich, J. M., &Schwartz, S. H. (1974). The effect of problem size on representation in deductive problem solving.Memory & Cognition,2, 683–686.
Schwartz, S. H. (1971). Modes of representation and problem solving: Well evolved is half solved.Journal of Experimental Psychology,91, 347–350.
Stanley, W. B., Mathews, R. C., Buss, R. R., &Kotler-Cope, S. (1989). Insight without awareness: On the interaction of verbalisation, instruction and practice in a simulated process control task.Quarterly Journal of Experimental Psychology,41A, 553–577.
Wilder, L., &Harvey, D. J. (1971). Overt and covert verbalization in problem solving.Speech Monographs,38, 171–176.
Author information
Authors and Affiliations
Corresponding author
Additional information
This paper is based on a part of S.A.R.K.’s doctoral dissertation. The author greatly appreciates the comments of David Kristovich on earlier versions of this paper.
Rights and permissions
About this article
Cite this article
Kristovich, S.A.R. A real-time data collection system for problem-solving protocols. Behavior Research Methods, Instruments, & Computers 28, 241–247 (1996). https://doi.org/10.3758/BF03204774
Received:
Accepted:
Issue Date:
DOI: https://doi.org/10.3758/BF03204774