Abstract
This paper reports an analysis of response strategies employed by 300 first year undergraduate students on a chemistry test to answer questions posed in multiple choice and constructed response formats. Test answers were categorized and analyzed for format-related differences in solution strategies and error patterns. A subgroup of 21 students who had written the test were interviewed following the test with the intent of further exploring the response strategies which they had used. The verbal reports from the interviews were scored for the presence of typical problem-solving behaviours. Analysis of the test data indicated no significant differences in the types of solution strategies employed, nor in the types of errors committed, across test format. Most students used algorithmic production strategies and “worked out” the answers without regard to question format. However, the verbal data revealed evidence of subtler differences in the types and frequencies of problem-solving behaviours, particularly with respect to problem representation and answer evaluation.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Aiken, L.R. (1987). ‘Testing with multiple choice items’, Journal of Research and Development in Education 20, 44–58.
Arter, J.A. and Salmon, J.R. (1987). Assessing Higher Order Thinking Skills. Portland, Oregon: Northwest Regional Educational Laboratory.
Black, P.J. (1994). ‘Performance assessment and accountability: The experience in England and Wales’, Educational Evaluation and Policy Analysis 16, 191–203.
Bloom, B.S. and Broder, L.J. (1950). ‘Problem solving processes of college students’, Supplementary Educational Monograph 73. Chicago: University of Chicago Press.
Boulton-Lewis, G. (1994). ‘Tertiary students knowledge of their own learning and a SOLO taxonomy’, Higher Education (Amsterdam), 28, 387–402.
Bunce, D.M., Gabel, D.L. and Samuel, J.V. (1991). ‘Enhancing chemistry problem-solving achievement using problem categorization’, Journal of Research in Science Teaching 28(6), 505–521.
Chi, M.T.H., Feltovich, P.J. and Glaser, R. (1981). ‘Categorization and representation of physics problems by experts and novices’, Cognitive Science 5, 121–152.
Connolly, J.A. and Wantman, M.J. (1964). ‘An exploration of oral reasoning skills in responding to objective test items’, Journal of Educational Measurement 1, 59–64.
Costa, A.L. (1985) (ed.) Developing Minds: A Resource Book for Teaching Thinking. Washington: Association for Supervision and Curriculum Development.
Council of Ministers of Education, Canada (1993). School Achievement Indicators Project: Report on Mathematics Assessment, 1993. Toronto: Council of Ministers of Education.
Ebel, R. and Frisbie, D.A. (1986). Essentials of educational measurement (4th ed.). Englewood Cliffs, N.J.: Prentice Hall.
Farr, R., Pritchard, R. and Smitten, B. (1990). ‘A description of what happens when an examinee takes a multiple choice reading comprehension test’, Journal of Educational Measurement 27, 209–226.
Frederickson, N. (1984). ‘Implications of cognitive theory for instruction in problem solving’, Review of Educational Research 54, 363–407.
Gabel, D.L., Sherwood, R.D. and Enoch, L. (1984). ‘Problem solving skills of high school chemistry students’, Journal of Research in Science Teaching 21, 221–233.
Gick, M.L. (1986). ‘Problem solving strategies’, Educational Psychologist 21, 99–120.
Gifford, B. and O'Connor, M. (eds). (1992). Changing Assessments: Alternative Views of Aptitude, Achievement and Instruction. Boston: Kluwer Academic Publishers.
Green, B.F. (1981). ‘A Primer of Testing’, American Psychologist 36 (10), 1001–1011.
Hayes, J.R. (1981). The Complete Problem Solver. Philadelphia: Franklin Institute Press.
Hayes, J.R., Flower, L., Schriver, K., Stratman, J. and Carey, L. (1987). ‘Cognitive processes in revision’, in Rosenberg, S. (ed.), Advances in Applied Psycholinguistics. (Vol. 2, pp. 176–240). Cambridge: Press Syndicate of the University of Cambridge.
Herron, J.D. (1986). Understanding Chemistry (2nd edition) New York: Random House.
Hogan, T.P. (1981). Relationship Between Free-response and Choice-type Tests of Achievement: A Review of the Literature. Green Bay, Wisconsin: University of Wisconsin. (Eric Document Ed 224 811)
Kropp, R.P. (1956). ‘The relationship between process and correct item-response’, Journal of Educational Research 49, 385–388.
Larkin, J., McDermott, J., Simon, D.P. and Simon, H.A. (1980). ‘Expert and novice performance in solving physics problems’, Science 208, 1335–1342.
Loftus, G.R. (1971). ‘Comparison of recognition and recall in a continuous memory task’, Journal Of Experimental Psychology 91, 220–226.
McPeck, J. (1981). Critical Thinking and Education. New York: St. Martin's Press.
Morgenstern, C.F. and Renner, J.W. (1984). ‘Measuring thinking with standardized test’, Journal of Research in Science Teaching 21, 639–648.
Nakhleh, M.B. and Mitchell, R.C. (1993). ‘Concept learning versus problem-solving’, Journal of Chemical Education 70(3), 190–192.
Niaz, M. (1995). ‘Progressive transitions from algorithmic to conceptual understanding in student ability to solve chemistry problems: a lakatosian interpretation’, Science Education 79(1), 19–36.
Niaz, M. and Robinson, W.R. (1992). ‘From ‘algorithmic mode’ to ‘conceptual gestalt’ in understanding the behaviour of gases: an epistemological perspective’, Research in Science and Technological Education 10, 53–64.
Norris, S.P. (1992). ‘A demonstration of the use of verbal reports of thinking in multiple choice critical thinking test design’, The Alberta Journal of Educational Research 38, 155–176.
Nurrenbern, S.C. (1979). ‘Problem solving behaviors of concrete and formal operational high school chemistry students when solving chemistry problems requiring formal reasoning skills’, Dissertation Abstracts International 40(4986A), 167. (University Microfilm No. DEM 80–05926).
Nurrenbern, S.C. and Pickering, M. (1987). ‘Concept learning versus problem-solving - is there a difference?’, Journal of Chemical Education 64(6), 508–510
Petrie, H. (1986). ‘Testing for critical thinking’, in Nyberg, D. (ed.), Philosophy of Education 1985. Normal IL: Philosophy of Education Society, pp. 3–19.
Sawrey, B.A. (1990). ‘Concept learning versus problem-solving revisited’, Journal of Chemical Education 67(3), 253–254.
Snow, R.E. (1980). ‘Cognitive process analyses of aptitude’, Aptitude, Learning and Instruction 1, 27–64.
Thissen, D. and Steinberg, L. (1984). ‘A response model for multiple choice items’, Psychometrika 49, 501–519.
Towns, M.H. and Robinson, W.R. (1993). ‘Student use of test-wiseness strategies in solving multiple choice chemistry examinations’, Journal of Research in Science Teaching 30, 709–722.
Traub, R.E. (1992). ‘On the equivalence of traits assessed by multiple choice and constructed response tests’, in Bennett, R. and Ward, W. (eds.), Construction versus Choice in Cognitive Measurement. Hillsdale, N.J.: Lawrence Erlbaum Associates, pp. 29–44.
Traub, R.E. and MacRury, K. (1990). ‘Multiple choice versus free response in the testing of scholastic achievement’, in Ingenkamp, K. and Jäger, R.S. (eds.) Tests und Trends 8: Jahrbuch der Pädagogischen Diagnostik. Weinheim and Basel: Beltz Verlag, pp. 128–159.
Treversky, B. (1973). ‘Encoding processes in recognition and recall’, Cognitive Psychology 5, 275–287.
Whitely, S.E. and Barnes, G.M. (1979). ‘The implications of processing event sequences for the theories of analogical reasoning’, Memory and Cognition 7(4), 323–331.
Author information
Authors and Affiliations
Rights and permissions
About this article
Cite this article
Barnett-Foster, D., Nagy, P. Undergraduate student response strategies to test questions of varying format. High Educ 32, 177–198 (1996). https://doi.org/10.1007/BF00138395
Issue Date:
DOI: https://doi.org/10.1007/BF00138395