Do Students Know What They Know and What They Don’t Know? Using a Four-Tier Diagnostic Test to Assess the Nature of Students’ Alternative Conceptions
- 1.3k Downloads
This study reports on the development and application of a four-tier multiple-choice (4TMC) diagnostic instrument, which has not been reported in the literature. It is an enhanced version of the two-tier multiple-choice (2TMC) test. As in 2TMC tests, its answer and reason tiers measure students’ content knowledge and explanatory knowledge, respectively. The two additional tiers measure the level of confidence of students in the correctness of their chosen options for the answer and reason tiers respectively. The 4TMC diagnostic test focused on the properties and propagation of mechanical waves. It was administered to 598 upper secondary students after they were formally instructed on the foregoing topics. The vast majority of the respondents were found to have an inadequate grasp of the topics tested. Mean scores and mean confidence associated with the answer tier was higher than those associated with the reason tier. The students tended to be poorly discriminating between what they know and what they do not know. Familiarity with the topic tested was associated with greater percentage of students giving correct answers, higher confidence, and better discrimination quotient. Nine genuine alternative conceptions (which were expressed with moderate levels of confidence by students) were identified.
KeywordsAlternative conceptions Confidence ratings Diagnostic tests Four-tier test Misconceptions Multiple-choice test Two-tier test Waves
We thank the Nanyang Technological University for the award of a research scholarship to the first author and a research grant (RI 9/06 RS) to the second author.
- Al-Rubayea, A. A. M. (1996). An analysis of Saudi Arabia high school students’ misconceptions about physics concepts. Dissertation Abstracts International, 57(04), 1462, (UMI No. 9629018).Google Scholar
- Altman, D. G. (1991). Practical statistics for medical research. London: Chapman & Hall.Google Scholar
- diSessa, A. A. (1993). Toward an epistemology of physics. Cognition and Instruction, 10(2 & 3), 105–225.Google Scholar
- Echternacht, G. J. (1972). The use confidence testing in objective tests. Review of Educational Research, 42–2, 217–236.Google Scholar
- Franklin, B. J. (1992). The development and application of a two-tier diagnostic instrument to detect misconceptions in the area of force, heat light and electricity. Dissertation Abstracts International, 53(12), 4186, (UMI No. 9301049).Google Scholar
- Hill, G. D. (1997). Conceptual change through the use of student-generated analogies of photosynthesis and respiration by college non-science majors. Dissertation Abstracts International, 58(06), 2142, (UMI No. 9735480).Google Scholar
- McKelvie, S. J. (1978). Effects of some variations in rating scale on the means and reliabilities of ratings. British Journal of Psychology, 69, 185–202.Google Scholar
- McKelvie, S. (1992). Does memory contaminate test-retest reliability? The Journal of General Psychology, 119(1), 59–72.Google Scholar
- Menchen, K. V. P., & Thomson, J. R. (2004). Pre-service teachers’ understanding of propagation and resonance in sound phenomena. In J. Max, S. Franklin, & J. Cummings (Eds.), 2003 Physics education Research Conference (pp. 65–68). New York: American Institute of Physics.Google Scholar
- Ministry of Education [MOE] (2008). Independent schools, special assistance plan (SAP) schools, autonomous schools and niche programme schools. http://www.moe.gov.sg/education/admissions/secondary-one-posting/files/secondary-one-posting-english.pdf.
- Reynolds, C. R., Livingston, R. B., & Willson, V. (2006). Measurement and assessment in education. Boston: Pearson.Google Scholar
- Stankov, L., & Dolph, B. (2000). Metacognitive aspects of test-taking and intelligence. Psychologische Beiträge, 42(2), 213–227.Google Scholar
- Zakay, D., & Glicksohn, J. (1992). Overconfidence in a multiple-choice test and its relationship to achievement. Psychological Record, 42(4), 519–525.Google Scholar