Journal of Science Education and Technology

, Volume 27, Issue 6, pp 492–507 | Cite as

Click-On-Diagram Questions: a New Tool to Study Conceptions Using Classroom Response Systems

  • Nicole D. LaDueEmail author
  • Thomas F. Shipley


Geoscience instructors depend upon photos, diagrams, and other visualizations to depict geologic structures and processes that occur over a wide range of temporal and spatial scales. This proof-of-concept study tests click-on-diagram (COD) questions, administered using a classroom response system (CRS), as a research tool for identifying spatial misconceptions. First, we propose a categorization of spatial conceptions associated with geoscience concepts. Second, we implemented the COD questions in an undergraduate introductory geology course. Each question was implemented three times: pre-instruction, post-instruction, and at the end of the course to evaluate the stability of students’ conceptual understanding. We classified each instance as (1) a false belief that was easily remediated, (2) a flawed mental model that was not fully transformed, or (3) a robust misconception that persisted despite targeted instruction. Geographic Information System (GIS) software facilitated spatial analysis of students’ answers. The COD data confirmed known misconceptions about Earth’s structure, geologic time, and base level and revealed a novel robust misconception about hot spot formation. Questions with complex spatial attributes were less likely to change following instruction and more likely to be classified as a robust misconception. COD questions provided efficient access to students’ conceptual understanding. CRS-administered COD questions present an opportunity to gather spatial conceptions with large groups of students, immediately, building the knowledge base about students’ misconceptions and providing feedback to guide instruction.


Clickers Diagrams Conceptions Mental models Spatial thinking Geology 



The authors appreciate assistance from Kerri Gefeke and Sheldon Turner for assisting with the ArcGIS analysis protocol, Mark Howland for re-drafting the graphics used for the CRS questions, Allison Jaeger for input on the COD questions, Mia Velazquez for generating the figures, and Doug Lombardi for thoughtful suggestions on this manuscript.


This study was funded in part by the National Science Foundation (Grant 1640800). Preparation of this manuscript was supported in part by NIU’s PI Academy and National Science Foundation grant 1640800 to TFS & NDL.

Compliance with Ethical Standards

Conflict of interest

The authors declare that they have no conflict of interest.

Ethical Approval

All procedures performed in studies involving human participants were in accordance with the ethical standards of the institutional and/or national research committee and with the 1964 Helsinki declaration and its later amendments or comparable ethical standards.

Informed Consent

Informed consent was obtained from all individual participants included in the study.


  1. Atit, K., Shipley, T. F., & Tikoff, B. (2013). Twisting space: are rigid and non-rigid mental transformations separate spatial skills? Cognitive Processing, 14(2), 163–173.CrossRefGoogle Scholar
  2. Ault, C. R. (1998). Criteria of excellence for geological inquiry: the necessity of ambiguity. Journal of Research in Science Teaching, 35(2), 189–212.CrossRefGoogle Scholar
  3. Beatty, I. D., & Gerace, W. J. (2009). Technology-enhanced formative assessment: a research-based pedagogy for teaching science with classroom response technology. Journal of Science Education and Technology, 18(2), 146–162.CrossRefGoogle Scholar
  4. Brewer, W. F. (2008). Naïve theories of observational astronomy: review, analysis, and theoretical implications. In S. Vosniadou (Ed.), International handbook of research on conceptual change (pp. 155–204). New York: Routledge.Google Scholar
  5. Carlson, L. A. (1999). Selecting a reference frame. Spatial Cognition and Computation, 1(4), 365–379.CrossRefGoogle Scholar
  6. Catley, K. M., & Novick, L. R. (2009). Digging deep: exploring college students’ knowledge of macroevolutionary time. Journal of Research in Science Teaching, 46(3), 311–332.CrossRefGoogle Scholar
  7. Catuneanu, O. (2006). Principles of sequence stratigraphy. Amsterdam: Elsevier.Google Scholar
  8. Cheek, K. A. (2010). Commentary: A summary and analysis of twenty-seven years of geoscience conceptions research. Journal of Geoscience Education, 58(3), 122–134.CrossRefGoogle Scholar
  9. Cheek, K. A. (2013). Exploring the relationship between students’ understanding of conventional time and deep (geologic) time. International Journal of Science Education, 35(11), 1925–1945.CrossRefGoogle Scholar
  10. Cheek, K. A., LaDue, N. D., & Shipley, T. F. (2017). Learning about spatial and temporal scale: current research, psychological processes, and classroom implications. Journal of Geoscience Education, 65(4), 455–472.CrossRefGoogle Scholar
  11. Chi, M. T. H. (1997). Creativity: shifting across ontological categories flexibly. In T. B. Ward, S. M. Smith, & J. Vaid (Eds.), Creative thought: an investigation of conceptual structures and processes (pp. 209–234). Washington, DC: American Psychological Association. Scholar
  12. Chi, M. T. H. (2008). Three types of conceptual change: belief revision, mental model transformation, and categorical shift. In S. Vosniadou (Eds.), International handbook of research on conceptual change (pp. 61–82). New York: Routledge.Google Scholar
  13. Clark, S. K., & Libarkin, J. C. (2011). Designing a mixed-methods research instrument and scoring rubric to investigate individuals’ conceptions of plate tectonics. Geological Society of America Special Papers, 474, 81–96.CrossRefGoogle Scholar
  14. Clark, S. K., Libarkin, J. C., Kortz, K. M., & Jordan, S. C. (2011). Alternative conceptions of plate tectonics held by nonscience undergraduates. Journal of Geoscience Education, 59(4), 251–262.CrossRefGoogle Scholar
  15. Cohen, J. (1988). Statistical power analysis for the behavioral sciences (2nd). New York: Routledge Academic.Google Scholar
  16. Dodick, J., & Orion, N. (2003). Cognitive factors affecting student understanding of geologic time. Journal of Research in Science Teaching, 40(4), 415–442.CrossRefGoogle Scholar
  17. Dolphin, G., & Benoit, W. (2016). Students’ mental model development during historically contextualized inquiry: how the ‘tectonic plate’ metaphor impeded the process. International Journal of Science Education, 38(2), 276–297.CrossRefGoogle Scholar
  18. Dove, J. E. (1998). Students’ alternative conceptions in Earth science: a review of research and implications for teaching and learning. Research Papers in Education, 13(2), 183–201.CrossRefGoogle Scholar
  19. Emenike, M. E., & Holme, T. A. (2012). Classroom response systems have not “crossed the chasm”: estimating numbers of chemistry faculty who use clickers. Journal of Chemical Education, 89(4), 465–469.CrossRefGoogle Scholar
  20. Forbus, K. D., Ferguson, R. W., Lovett, A., & Gentner, D. (2017). Extending SME to handle large-scale cognitive modeling. Cognitive Science, 41(5), 1152–1201.CrossRefGoogle Scholar
  21. Francek, M. (2013). A compilation and review of over 500 geoscience misconceptions. International Journal of Science Education, 35(1), 31–64.CrossRefGoogle Scholar
  22. Gagnier, K. M., Atit, K., Ormand, C. J., & Shipley, T. F. (2017). Comprehending 3D diagrams: sketching to support spatial reasoning. Topics in Cognitive Science, 9(4), 883–901.CrossRefGoogle Scholar
  23. Garvin-Doxas, K., & Klymkowsky, M. W. (2008). Understanding randomness and its impact on student learning: lessons learned from building the Biology Concept Inventory (BCI). CBE Life Sciences Education, 7(2), 227–233.CrossRefGoogle Scholar
  24. Gentner, D. (1989). Analogical learning. In S. Vosniadou & A. Ortony (Eds.), Similarity and analogical reasoning (p. 199). Cambridge: Cambridge University PressGoogle Scholar
  25. Gobert, J. D. (2000). A typology of causal models for plate tectonics: inferential power and barriers to understanding. International Journal of Science Education, 22(9), 937–977.CrossRefGoogle Scholar
  26. Gurel, D. K., Eryılmaz, A., & McDermott, L. C. (2015). A review and comparison of diagnostic instruments to identify students’ misconceptions in science. Eurasia Journal of Mathematics, Science and Technology Education, 11(5), 989–1008.Google Scholar
  27. Hegarty, M. (2011). The cognitive science of visual-spatial displays: implications for design. Topics in Cognitive Science, 3(3), 446–474.CrossRefGoogle Scholar
  28. Hegarty, M., & Waller, D. (2004). A dissociation between mental rotation and perspective-taking spatial abilities. Intelligence, 32(2), 175–191.CrossRefGoogle Scholar
  29. Hegarty, M., Crookes, R. D., Dara-Abrams, D., & Shipley, T. F. (2010). Do all science disciplines rely on spatial abilities? Preliminary evidence from self-report questionnaires. In International Conference on Spatial Cognition (pp. 85–94). Berlin, Heidelberg: Springer.CrossRefGoogle Scholar
  30. Herrera, J. S., & Riggs, E. M. (2013). Identifying students’ conceptions of basic principles in sequence stratigraphy. Journal of Geoscience Education, 61(1), 89–102.CrossRefGoogle Scholar
  31. Hestenes, D., Wells, M., & Swackhamer, G. (1992). Force concept inventory. Physics Teacher, 30(3), 141–158.CrossRefGoogle Scholar
  32. Jarrard, R. D., & Clague, D. A. (1977). Implications of Pacific island and seamount ages for the origin of volcanic chains. Reviews of Geophysics, 15(1), 57–76.CrossRefGoogle Scholar
  33. Jones, M. G., Tretter, T., Taylor, A., & Oppewal, T. (2008). Experienced and novice teachers’ concepts of spatial scale. International Journal of Science Education, 30(3), 409–429.CrossRefGoogle Scholar
  34. Jones, G., Taylor, A., & Broadwell, B. (2009). Estimating linear size and scale: Body rulers. International Journal of Science Education, 31(11), 1495–1509.CrossRefGoogle Scholar
  35. Karlstrom, K., Semken, S., Crossey, L., Perry, D., Gyllenhaal, E. D., Dodick, J., Williams, M., Hellmich-Bryan, J., Crow, R., Watts, N. B., & Ault, C. (2008). Informal geoscience education on a grand scale: the trail of time exhibition at Grand Canyon. Journal of Geoscience Education, 56(4), 354–361.CrossRefGoogle Scholar
  36. Kastens, K. A., & Rivet, A. (2010). Using analogical mapping to assess the affordances of scale models used in earth and environmental science education. In International Conference on Spatial Cognition (pp. 112–124). Berlin, Heidelberg: Springer.CrossRefGoogle Scholar
  37. Kastens, K. A., Pistolesi, L., & Passow, M. J. (2014). Analysis of spatial concepts, spatial skills and spatial representations in New York State regents Earth science examinations. Journal of Geoscience Education, 62(2), 278–289.CrossRefGoogle Scholar
  38. King, C. (2008). The earth science misconceptions of some science writers: how wrong can they be. Teaching Earth Sciences, 33(2), 9–11.Google Scholar
  39. King, C. J. H. (2010). An analysis of misconceptions in science textbooks: Earth science in England and Wales. International Journal of Science Education, 32(5), 565–601.CrossRefGoogle Scholar
  40. Konrad, K., Koppers, A. A., Steinberger, B., Finlayson, V. A., Konter, J. G., & Jackson, M. G. (2018). On the relative motions of long-lived Pacific mantle plumes. Nature Communications, 9(1), 854.CrossRefGoogle Scholar
  41. LaDue, N. D. (2018). Scaffolding temporal reasoning with geologic timelines. Retrieved from: Accessed 2 Apr 2018.
  42. LaDue, N. D., Libarkin, J. C., & Thomas, S. R. (2015). Visual representations on high school biology, chemistry, earth science, and physics assessments. Journal of Science Education and Technology, 24(6), 818–834.CrossRefGoogle Scholar
  43. Lakoff, G., & Johnson, M. (2008). Metaphors we live by. Chicago: University of Chicago press.Google Scholar
  44. Larkin, J. H., & Simon, H. A. (1987). Why a diagram is (sometimes) worth ten thousand words. Cognitive Science, 11(1), 65–100.CrossRefGoogle Scholar
  45. Lee, H., & Feldman, A. (2015). Photographs and classroom response systems in middle school astronomy classes. Journal of Science Education and Technology, 24(4), 496–508.CrossRefGoogle Scholar
  46. Lee, H., & Schneider, S. E. (2015). Using astronomical photographs to investigate misconceptions about galaxies and spectra: question development for clicker use. Physical Review Special Topics - Physics Education Research, 11(2), 020101.CrossRefGoogle Scholar
  47. Libarkin, J. C. (2006). College student conceptions of geological phenomena and their importance in classroom instruction. Planet, 17(1), 6–9.CrossRefGoogle Scholar
  48. Libarkin, J. C., & Anderson, S. W. (2005). Assessment of learning in entry-level geoscience courses: results from the Geoscience Concept Inventory. Journal of Geoscience Education, 53(4), 394–401.CrossRefGoogle Scholar
  49. Libarkin, J. C., & Kurdziel, J. P. (2006). Ontology and the teaching of Earth system science. Journal of Geoscience Education, 54(3), 408–413.CrossRefGoogle Scholar
  50. Libarkin, J. C., Anderson, S. W., Beilfuss, M., & Boone, W. (2005). Qualitative analysis of college students’ ideas about the Earth: interviews and open-ended questionnaires. Journal of Geoscience Education, 53(1), 17–26.CrossRefGoogle Scholar
  51. Libarkin, J. C., Kurdziel, J. P., & Anderson, S. W. (2007). College student conceptions of geological time and the disconnect between ordering and scale. Journal of Geoscience Education, 55(5), 413–422.CrossRefGoogle Scholar
  52. Lombardi, D., & Sinatra, G. M. (2012). College students’ perceptions about the plausibility of human-induced climate change. Research in Science Education, 42(2), 201–217.CrossRefGoogle Scholar
  53. Lombardi, D., Brandt, C. B., Bickel, E. S., & Burg, C. (2016). Students’ evaluations about climate change. International Journal of Science Education, 38(8), 1392–1414.CrossRefGoogle Scholar
  54. Manduca, C. A., & Kastens, K. A. (2012). Geoscience and geoscientists: uniquely equipped to study Earth. Geological Society of America Special Papers, 486, 1–12.CrossRefGoogle Scholar
  55. Marshak, S. (2016). Essentials of geology (5th ed.). New York: W.W. Norton & Company, Inc..Google Scholar
  56. Mills, R., Tomas, L., & Lewthwaite, B. (2016). Learning in Earth and space science: a review of conceptual change instructional approaches. International Journal of Science Education, 38(5), 767–790.CrossRefGoogle Scholar
  57. National Research Council. (2012). Discipline-based education research: understanding and improving learning in undergraduate science and engineering. S. R. Singer, N. R. Nielsen, & H. A. Schweingruber (Eds.), Washington, D.C.: National Academies Press.Google Scholar
  58. New York State Education Department [NYSED]. (2018). Science Regents Exams: Physical Setting/Earth Science. Retrieved from:
  59. Newcombe, N. S., & Shipley, T. F. (2015). Thinking about spatial thinking: new typology, new assessments. In Studying visual and spatial reasoning for design creativity (pp. 179–192). Dordrecht: Springer.Google Scholar
  60. Parham, T. L., Cervato, C., Gallus, W. A., Larsen, M., Hobbs, J., Stelling, P., Greenbowe, T., Gupta, T., Knox, J.A., Gill, T. E. (2018). The InVEST volcanic concept survey: Exploring student understanding about volcanoes. Journal of Geoscience Education, 58(3), 177–187.CrossRefGoogle Scholar
  61. Posner, G. J., Strike, K. A., Hewson, P. W., & Gertzog, W. A. (1982). Accommodation of a scientific conception: toward a theory of conceptual change. Science Education, 66(2), 211–227.CrossRefGoogle Scholar
  62. Resnick, I., & Shipley, T. F. (2013). Breaking new ground in the mind: an initial study of mental brittle transformation and mental rigid rotation in science experts. Cognitive Processing, 14(2), 143–152.CrossRefGoogle Scholar
  63. Resnick, I., Davatzes, A., Newcombe, N. S., & Shipley, T. F. (2017). Using relational reasoning to learn about scientific phenomena at unfamiliar scales. Educational Psychology Review, 29(1), 11–25.CrossRefGoogle Scholar
  64. Richardson, R. (2005). Teaching time in large enrollment intro classes: an active approach. Geological Society of America Abstracts with Programs, 37(7), 153.Google Scholar
  65. Schnotz, W., & Bannert, M. (2003). Construction and interference in learning from multiple representation. Learning and Instruction, 13(2), 141–156.CrossRefGoogle Scholar
  66. Shea, K. M. (2016). Beyond clickers, next generation classroom response systems for organic chemistry. Journal of Chemical Education, 93(5), 971–974.CrossRefGoogle Scholar
  67. Shipley, T. F., Tikoff, B., Ormand, C., & Manduca, C. (2013). Structural geology practice and learning, from the perspective of cognitive science. Journal of Structural Geology, 54, 72–84.CrossRefGoogle Scholar
  68. Sweller, J. (1994). Cognitive load theory, learning difficulty, and instructional design. Learning and Instruction, 4(4), 295–312.CrossRefGoogle Scholar
  69. Trend, R. (2000). Conceptions of geological time among primary teacher trainees, with reference to their engagement with geoscience, history, and science. International Journal of Science Education, 22(5), 539–555.CrossRefGoogle Scholar
  70. Trend, R. D. (2001). Deep time framework: a preliminary study of UK primary teachers’ conceptions of geological time and perceptions of geoscience. Journal of Research in Science Teaching, 38(2), 191–221.CrossRefGoogle Scholar
  71. Tretter, T. R., Jones, M. G., & Minogue, J. (2006). Accuracy of scale conceptions in science: mental maneuverings across many orders of spatial magnitude. Journal of Research in Science Teaching, 43(10), 1061–1085.CrossRefGoogle Scholar
  72. United States Geological Survey [USGS]. (2017). Geology and the National Parks. Retrieved from:
  73. Uttal, D. H., Meadow, N. G., Tipton, E., Hand, L. L., Alden, A. R., Warren, C., & Newcombe, N. S. (2013). The malleability of spatial skills: a meta-analysis of training studies. Psychological Bulletin, 139(2), 352–402.CrossRefGoogle Scholar
  74. Vosniadou, S. (2002). On the nature of naive physics. In Reconsidering conceptual change: issues in theory and practice (pp. 61–76). Dordrecht: Springer.CrossRefGoogle Scholar
  75. Vosniadou, S., & Brewer, W. F. (1987). Theories of knowledge restructuring in development. Review of Educational Research, 57(1), 51–67.CrossRefGoogle Scholar
  76. Vosniadou, S. & Brewer, W. F. (1992) Mental models of the earth: A study of conceptual change in childhood. Cognitive Psychology, 24(4), 535–585.CrossRefGoogle Scholar
  77. Vosniadou, S., Vamvakoussi, X., & Skopeliti, I. (2008). The framework theory approach to the problem of conceptual change. In S. Voisniadou (Ed.), International handbook of research on conceptual change (pp. 3–34). New York: Routledge.Google Scholar
  78. Wade, N. J. (1996). Frames of reference in vision. Minimally Invasive Therapy & Allied Technologies, 5(5), 435–439.CrossRefGoogle Scholar
  79. Wade, N. J., & Swanston, M. (2013). Visual perception: An introduction. London: Psychology Press.CrossRefGoogle Scholar

Copyright information

© Springer Science+Business Media, LLC, part of Springer Nature 2018

Authors and Affiliations

  1. 1.Geology and Environmental Geosciences DepartmentNorthern Illinois UniversityDeKalbUSA
  2. 2.Department of PsychologyTemple UniversityPhiladelphiaUSA

Personalised recommendations