Advertisement

Research in Science Education

, Volume 40, Issue 3, pp 313–337 | Cite as

Do Students Know What They Know and What They Don’t Know? Using a Four-Tier Diagnostic Test to Assess the Nature of Students’ Alternative Conceptions

  • Imelda S. Caleon
  • R. SubramaniamEmail author
Article

Abstract

This study reports on the development and application of a four-tier multiple-choice (4TMC) diagnostic instrument, which has not been reported in the literature. It is an enhanced version of the two-tier multiple-choice (2TMC) test. As in 2TMC tests, its answer and reason tiers measure students’ content knowledge and explanatory knowledge, respectively. The two additional tiers measure the level of confidence of students in the correctness of their chosen options for the answer and reason tiers respectively. The 4TMC diagnostic test focused on the properties and propagation of mechanical waves. It was administered to 598 upper secondary students after they were formally instructed on the foregoing topics. The vast majority of the respondents were found to have an inadequate grasp of the topics tested. Mean scores and mean confidence associated with the answer tier was higher than those associated with the reason tier. The students tended to be poorly discriminating between what they know and what they do not know. Familiarity with the topic tested was associated with greater percentage of students giving correct answers, higher confidence, and better discrimination quotient. Nine genuine alternative conceptions (which were expressed with moderate levels of confidence by students) were identified.

Keywords

Alternative conceptions Confidence ratings Diagnostic tests Four-tier test Misconceptions Multiple-choice test Two-tier test Waves 

Notes

Acknowledgment

We thank the Nanyang Technological University for the award of a research scholarship to the first author and a research grant (RI 9/06 RS) to the second author.

References

  1. Al-Rubayea, A. A. M. (1996). An analysis of Saudi Arabia high school students’ misconceptions about physics concepts. Dissertation Abstracts International, 57(04), 1462, (UMI No. 9629018).Google Scholar
  2. Altman, D. G. (1991). Practical statistics for medical research. London: Chapman & Hall.Google Scholar
  3. Barman, C. R., & Barman, S. N. (1996). Two teaching methods and students’ understanding of sound. School Science and Mathematics, 96(2), 63–68.CrossRefGoogle Scholar
  4. Clement, J., Brown, D. E., & Zietsman, A. (1989). Not all preconceptions are misconceptions: finding ‘anchoring conceptions’ for grounding instruction on students’ intuition. International Journal of Science Education, 11(5), 554–565. doi: 10.1080/0950069890110507.CrossRefGoogle Scholar
  5. diSessa, A. A. (1993). Toward an epistemology of physics. Cognition and Instruction, 10(2 & 3), 105–225.Google Scholar
  6. Echternacht, G. J. (1972). The use confidence testing in objective tests. Review of Educational Research, 42–2, 217–236.Google Scholar
  7. Eshach, H., & Schwarz, J. L. (2006). Sound stuff: naïve materialism in middle-school students’ conceptions of sound. International Journal of Science Education, 7(1), 733–764.CrossRefGoogle Scholar
  8. Franklin, B. J. (1992). The development and application of a two-tier diagnostic instrument to detect misconceptions in the area of force, heat light and electricity. Dissertation Abstracts International, 53(12), 4186, (UMI No. 9301049).Google Scholar
  9. Glenberg, A. M., & Epstein, W. (1985). Calibration of comprehension. Journal of Experimental Psychology: Learning Memory and Cognition, 11, 702–708.CrossRefGoogle Scholar
  10. Glenberg, A. M., Sanocki, T., Epstein, W., & Morris, C. (1987). Enhancing calibration of comprehension. Journal of Experimental Psychology: General, 116, 119–136.CrossRefGoogle Scholar
  11. Griffard, P. B., & Wandersee, J. H. (2001). The two-tier instrument on photosynthesis: what does it diagnose? International Journal of Science Education, 23(10), 1039–1052.CrossRefGoogle Scholar
  12. Hasan, S., Bagayoko, D., & Kelley, E. L. (1999). Misconceptions and the Certainty of Response Index (CRI). Physics Education, 34(5), 294–299.CrossRefGoogle Scholar
  13. Hill, G. D. (1997). Conceptual change through the use of student-generated analogies of photosynthesis and respiration by college non-science majors. Dissertation Abstracts International, 58(06), 2142, (UMI No. 9735480).Google Scholar
  14. Lang, H. G. (1982). Criterion-referenced tests in science: an investigation of reliability, validity and standards-setting. Journal of Research in Science Teaching, 19(8), 665–674.CrossRefGoogle Scholar
  15. Linder, C. J. (1993). University physics students’ conceptualizations of factors affecting the speed of sound propagation. International Journal of Science Education, 15(6), 655–662.CrossRefGoogle Scholar
  16. Lundeberg, M. A., Fox, P. W., & Punćhochaŕ, J. (1994). Highly confident but wrong: gender differences and similarities in confidence judgments. Journal of Educational Psychology, 86(1), 114–121.CrossRefGoogle Scholar
  17. Lundeberg, M. A., Fox, P. W., Brown, A. C., & Elbedour, S. (2000). Cultural influences on confidence: country and gender. Journal of Educational Psychology, 92(1), 152–159.CrossRefGoogle Scholar
  18. Maurines, L. (1992). Spontaneous reasoning on the propagation of visible mechanical signals. International Journal of Science Education, 14(3), 279–293.CrossRefGoogle Scholar
  19. McKelvie, S. J. (1978). Effects of some variations in rating scale on the means and reliabilities of ratings. British Journal of Psychology, 69, 185–202.Google Scholar
  20. McKelvie, S. (1992). Does memory contaminate test-retest reliability? The Journal of General Psychology, 119(1), 59–72.Google Scholar
  21. Menchen, K. V. P., & Thomson, J. R. (2004). Pre-service teachers’ understanding of propagation and resonance in sound phenomena. In J. Max, S. Franklin, & J. Cummings (Eds.), 2003 Physics education Research Conference (pp. 65–68). New York: American Institute of Physics.Google Scholar
  22. Ministry of Education [MOE] (2008). Independent schools, special assistance plan (SAP) schools, autonomous schools and niche programme schools. http://www.moe.gov.sg/education/admissions/secondary-one-posting/files/secondary-one-posting-english.pdf.
  23. Morris, C. (1990). Retrieval process underlying confidence in comprehension judgments. Journal of Experimental Psychology: Learning Memory and Cognition, 16, 223–232.CrossRefGoogle Scholar
  24. Palacios, F. J. P., Cazorla, F. N., & Cervantes, A. (1989). Misconceptions on geometric optics and their association with relevant educational variables. International Journal of Science Education, 11(3), 273–286.CrossRefGoogle Scholar
  25. Popham, W. J., & Husek, T. R. (1969). Implications of criterion-referenced measurement. Journal of Educational Measurement, 6(1), 1–9.CrossRefGoogle Scholar
  26. Reiner, M., Slotta, J. D., Chi, M. T. H., & Resnick, L. B. (2000). Naïve physics reasoning: a commitment to substance-based conceptions. Cognition and Instruction, 18(1), 1–34.CrossRefGoogle Scholar
  27. Reynolds, C. R., Livingston, R. B., & Willson, V. (2006). Measurement and assessment in education. Boston: Pearson.Google Scholar
  28. Stankov, L., & Crawford, J. D. (1997). Self-confidence and performance on test of cognitive abilities. Intelligence, 25(2), 93–109.CrossRefGoogle Scholar
  29. Stankov, L., & Dolph, B. (2000). Metacognitive aspects of test-taking and intelligence. Psychologische Beiträge, 42(2), 213–227.Google Scholar
  30. Tobin, K. G., & Capie, W. (1981). The development and validation of a group test of logical thinking. Educational and Psychological Measurement, 11, 413–423.CrossRefGoogle Scholar
  31. Treagust, D. F. (1986). Evaluating students’ misconceptions by means of diagnostic multiple-choice items. Research in Science Education, 16, 199–207.CrossRefGoogle Scholar
  32. Treagust, D. F. (1988). Development and use of diagnostic tests to evaluate students’ misconceptions in science. International Journal of Science Education, 10, 159–170.CrossRefGoogle Scholar
  33. Tsai, C. C., & Chou, C. (2002). Diagnosing students’ alternative conceptions in science. Journal of Computer Assisted Learning, 18, 157–165.CrossRefGoogle Scholar
  34. Voska, K. W., & Heikkinen, H. W. (2000). Identification and analysis of student conceptions used to solve chemical equilibrium problems. Journal of Research in Science Teaching, 37(2), 160–176.CrossRefGoogle Scholar
  35. Witmann, M. C. (2002). The object coordination class applied to wave pulses: analysing student reasoning in wave physics. International Journal of Science Education, 24(1), 97–118.CrossRefGoogle Scholar
  36. Wittmann, M. C., Steinberg, R. N., & Redish, E. F. (1999). Making sense of how students make sense of mechanical waves. The Physics Teacher, 37, 15–21.CrossRefGoogle Scholar
  37. Wittmann, M. C., Steinberg, R. N., & Redish, E. F. (2003). Understanding and affecting student reasoning about sound waves. International Journal of Science Education, 25(8), 991–1013.CrossRefGoogle Scholar
  38. Zakay, D., & Glicksohn, J. (1992). Overconfidence in a multiple-choice test and its relationship to achievement. Psychological Record, 42(4), 519–525.Google Scholar

Copyright information

© Springer Science+Business Media B.V. 2009

Authors and Affiliations

  1. 1.National Institute of EducationNanyang Technological UniversitySingaporeSingapore

Personalised recommendations