Skip to main content
SpringerLink
Log in

Aligning Assessment Goals with the Current and Future Technologies Needed to Achieve Them

  • Chapter
  • First Online:
Technologies in Biomedical and Life Sciences Education

Part of the book series: Methods in Physiology ((METHPHYS))

  • 474 Accesses

Abstract

The issue of assessment involves two interdependent drivers: the purpose(s) of the assessment and how such assessments can be applied at scale, that is, in large and, in some cases, remote settings. The simplest assessment goal, to sort students by what content they know or can recognize as correct, often involves a variety of “forced-choice” or fill in the blank questions that are readily analyzed by computers. Higher-level assessments that evaluate the extent to which students can access and apply their knowledge to new situations (as opposed to remembering previously presented examples), and can be used to develop students’ working knowledge, demand more sophisticated Socratic approaches aimed at making student presumptions explicit, together with their relevance and implications. Progress along these lines involves the automated analysis and response to drawn responses (graphs and such), as in the beSocratic™ system. Future extensions will require an iterative feedback system that can analyze students’ textual responses “on the fly” and pose disciplinarily relevant and clarifying Socratic questions. We consider the current state of affairs in achieving this goal.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 189.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 249.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD 249.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Similar content being viewed by others

Notes

  1. 1.

    Reverse Dunning-Kruger effects and science education

References

  • Becker, N. M., Noyes, K., & Cooper, M. M. (2016). Characterizing students’ mechanistic reasoning about London dispersion forces. Journal of Chemical Education. https://doi.org/10.1021/acs.jchemed.6b00298

  • Briggs, D. C., Alonzo, A. C., Schwab, C., & Wilson, M. (2006). Diagnostic assessment with ordered multiple-choice items. Educational Assessment, 11(1), 33–63.

    Article  Google Scholar 

  • Bryfczynski, S. P. (2012). BeSocratic: An intelligent tutoring system for the recognition, evaluation, and analysis of free-form student input (UMI No. 3550201). Doctoral dissertation. Clemson University.

    Google Scholar 

  • Butler, A. C., Karpicke, J. D., & Roediger, H. L., III. (2007). The effect of type and timing of feedback on learning from multiple-choice tests. Journal of Experimental Psychology: Applied, 13(4), 273.

    PubMed  Google Scholar 

  • Butler, A. C., & Roediger, H. L. (2008). Feedback enhances the positive effects and reduces the negative effects of multiple-choice testing. Memory & Cognition, 36(3), 604–616.

    Article  Google Scholar 

  • Cooper, M. M., & Klymkowsky, M. W. (2013). Chemistry, life, the universe and everything: A new approach to general chemistry, and a model for curriculum reform. Journal of Chemical Education, 90, 1116–1122. https://doi.org/10.1020/ed300456y

    Article  CAS  Google Scholar 

  • Cooper, M. M., Kouyoumdjian, H., & Underwood, S. M. (2016). Investigating students’ reasoning about acid–base reactions. Journal of Chemical Education, 93(10), 1703–1712. https://doi.org/10.1021/acs.jchemed.6b00417

    Article  CAS  Google Scholar 

  • Cooper, M. M., Stowe, R. L., Crandell, O. M., & Klymkowsky, M. W. (2019). Organic chemistry, life, the universe and everything (OCLUE): A transformed organic chemistry curriculum. Journal of Chemical Education. https://doi.org/10.1021/acs.jchemed.9b00401

  • Cooper, M. M., Underwood, S. M., Bryfczynski, S. P., & Klymkowsky, M. W. (2014). A short history of the use of technology to model and analyze student data for teaching and research. In R. S. Cole & D. Bunce (Eds.), Tools of chemistry education research (pp. 219–239). American Chemical Society.

    Chapter  Google Scholar 

  • Cooper, M. M., Underwood, S. M., Hilley, C. Z., & Klymkowsky, M. W. (2012). Development and assessment of a molecular structure and properties learning progression. Journal of Chemical Education, 89(11), 1351–1357. https://doi.org/10.1021/ed300083a

    Article  CAS  Google Scholar 

  • Cooper, M. M., Williams, L. C., & Underwood, S. M. (2015). Student understanding of intermolecular forces: A multimodal study. Journal of Chemical Education, 92(8), 1288–1298. https://doi.org/10.1021/acs.jchemed.5b00169

    Article  CAS  Google Scholar 

  • Crandell, O. M., Kouyoumdjian, H., Underwood, S. M., & Cooper, M. M. (2019). Reasoning about reactions in organic chemistry: Starting it in general chemistry. Journal of Chemical Education, 96(2), 213–226. https://doi.org/10.1021/acs.jchemed.8b00784

    Article  CAS  Google Scholar 

  • Crandell, O. M., Lockhart, M. A., & Cooper, M. M. (2020). Arrows on the page are not a good gauge: Evidence for the importance of causal mechanistic explanations about nucleophilic substitution in organic chemistry. Journal of Chemical Education, 97(2), 313–327. https://doi.org/10.1021/acs.jchemed.9b00815

    Article  CAS  Google Scholar 

  • D’mello, S., & Graesser, A. (2013). AutoTutor and affective AutoTutor: Learning by talking with cognitively and emotionally intelligent computers that talk back. ACM Transactions on Interactive Intelligent Systems (TiiS), 2(4), 1–39.

    Google Scholar 

  • Embretson, S. E., & Reise, S. P. (2013). Item response theory. Psychology Press.

    Book  Google Scholar 

  • Foltz, P. W., Gilliam, S., & Kendall, S. (2000). Supporting content-based feedback in on-line writing evaluation with LSA. Interactive Learning Environments, 8(2), 111–127.

    Article  Google Scholar 

  • Freeman, S., Eddy, S. L., McDonough, M., Smith, M. K., Okoroafor, N., Jordt, H., & Wenderoth, M. P. (2014). Active learning increases student performance in science, engineering, and mathematics. Proceedings of the National Academy of Sciences of the United States of America, 111, 8410–8415. https://doi.org/10.1073/pnas.1319030111

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  • Gagne, R. M. (1965). The conditions of learning. Holt Rinehart & Winston.

    Google Scholar 

  • Gierl, M. J., Bulut, O., Guo, Q., & Zhang, X. (2017). Developing, analyzing, and using distractors for multiple-choice tests in education: A comprehensive review. Review of Educational Research, 87(6), 1082–1116. https://doi.org/10.3102/0034654317726529

    Article  Google Scholar 

  • Hake, R. R. (1998). Interactive-engagement versus traditional methods: A six-thousand-student survey of mechanics test data for introductory physics courses. American Journal of Physics, 66(1), 64–74.

    Article  Google Scholar 

  • Hammer, D. (2000). Student resources for learning introductory physics. American Journal of Physics, 68, S52–S59.

    Article  Google Scholar 

  • Hattie, J., & Timperley, H. (2007). The power of feedback. Review of Educational Research, 77, 81–112.

    Article  Google Scholar 

  • Hestenes, D., Wells, M., & Swackhamer, G. (1992). Force concept inventory. The Physics Teacher, 30(3), 141–158. https://doi.org/10.1119/1.2343497

    Article  Google Scholar 

  • Klymkowsky, M. W., & Garvin-Doxas, K. (2020). Concept inventories: Design, application, uses, limitations, and next steps. In Active learning in college science (pp. 775–790). Springer.

    Chapter  Google Scholar 

  • Klymkowsky, M. W., Rentsch, J. D., Begovic, E., & Cooper, M. M. (2016). The design and transformation of biofundamentals: A nonsurvey introductory evolutionary and molecular biology course. CBE Life Sciences Education, 15(4). https://doi.org/10.1187/cbe.16-03-0142

  • Lee, H.-S., Liu, O. L., & Linn, M. C. (2011). Validating measurement of knowledge integration in science using multiple-choice and explanation items. Applied Measurement in Education, 24(2), 115–136. https://doi.org/10.1080/08957347.2011.554604

    Article  Google Scholar 

  • Miller, C. M., & Parlett, M. (1974). Up to the mark: A study of the examination game. Society for Research into Higher Education.

    Google Scholar 

  • Mislevy, R. J., & Riconscente, M. M. (2011). Evidence-centered assessment design. In S. Downing & T. Haladyna (Eds.), Handbook of test development. Erlbaum.

    Google Scholar 

  • National Academies of Sciences, Engineering, and Medicine. (2018). How people learn II: Learners, contexts, and cultures. https://doi.org/10.17226/24783.

  • National Research Council. (1999). How people learn: Brain, mind, experience, and school. National Academies Press.

    Google Scholar 

  • National Research Council. (2001). In J. W. Pellegrino, N. Chudowsky, & R. Glaser (Eds.), Knowing what students know: The science and design of educational assessment. National Academies Press.

    Google Scholar 

  • National Research Council. (2012). In S. R. Singer, N. R. Nielson, & H. A. Schweingruber (Eds.), Discipline-based education research: Understanding and improving learning in undergraduate science and engineering. National Academies Press.

    Google Scholar 

  • Nicol, D. J., & Macfarlane-Dick, D. (2006). Formative assessment and self-regulated learning: A model and seven principles of good feedback practice. Studies in Higher Education, 31(2), 199–218. https://doi.org/10.1080/03075070600572090

    Article  Google Scholar 

  • Noyes, K., & Cooper, M. M. (2019). Investigating student understanding of london dispersion forces: A longitudinal study. Journal of Chemical Education. https://doi.org/10.1021/acs.jchemed.9b00455

  • Noyes, K., McKay, R. L., Neumann, M., Haudek, K. C., & Cooper, M. M. (2020). Developing computer resources to automate analysis of students’ explanations of London dispersion forces. Journal of Chemical Education, 97(11), 3923–3936. https://doi.org/10.1021/acs.jchemed.0c00445

    Article  CAS  Google Scholar 

  • Pane, J. F., Griffin, B. A., McCaffrey, D. F., & Karam, R. (2014). Effectiveness of cognitive tutor algebra I at scale. Educational Evaluation and Policy Analysis, 36(2), 127–144. https://doi.org/10.3102/0162373713507480

    Article  Google Scholar 

  • Posner, G. J., Strike, K. A., Hewson, P. W., & Gertzog, W. A. (1982). Accommodation of a scientific conception: Toward a theory of conceptual change. Science Education, 66(2), 211–227.

    Article  Google Scholar 

  • Powell, J. (2019). Trust me, I’m a chatbot: How artificial intelligence in health care fails the Turing test. Journal of Medical Internet Research, 21(10), e16222.

    Article  Google Scholar 

  • Romito, B. T., Krasne, S., Kellman, P. J., & Dhillon, A. (2016). The impact of a perceptual and adaptive learning module on transoesophageal echocardiography interpretation by anaesthesiology residents. BJA: British Journal of Anaesthesia, 117(4), 477–481.

    Article  CAS  Google Scholar 

  • Schmidt, H.-J., Kaufmann, B., & Treagust, D. F. (2009). Students’ understanding of boiling points and intermolecular forces. Chemistry Education Research and Practice, 10, 265–272. https://doi.org/10.1039/B920829C

    Article  CAS  Google Scholar 

  • Shepard, L. A. (2000). The role of assessment in a learning culture. Educational Researcher, 29(7), 4–14. https://doi.org/10.3102/0013189X029007004

    Article  Google Scholar 

  • Skinner, B. F. (1954). The science of learning and the art of teaching. Cambridge, Mass, USA, 99, 113.

    Google Scholar 

  • Smith, C. L., Wiser, M., Anderson, C. W., & Krajcik, J. (2006). Focus article: Implications of research on children’s learning for standards and assessment: A proposed learning progression for matter and the atomic-molecular theory. Measurement: Interdisciplinary Research & Perspective, 4(1–2), 1–98.

    Google Scholar 

  • Treagust, D. F. (1988). Development and use of diagnostic tests to evaluate students’ misconceptions in science. International Journal of Science Education, 10, 159–169. https://doi.org/10.1080/0950069880100204

    Article  Google Scholar 

  • Underwood, S. M., Reyes-Gastelum, D., & Cooper, M. M. (2015). Answering the questions of whether and when student learning occurs: Using discrete-time survival analysis to investigate how college chemistry students’ understanding of structure-property relationships evolves. Science Education, 99(6), 1055–1072. https://doi.org/10.1002/sce.21183

    Article  Google Scholar 

  • Urban-Lurain, M., Cooper, M. M., Haudek, K. C., Kaplan, J. J., Knight, J. K., Lemons, P. P., Lira, C. T., Merrill, J. E., Nehm, R. H., Prevost, L. B., Smith, M. K., & Sydlik, M. (2015). Expanding a national network for Automated Analysis of Constructed Response assessments to reveal student thinking in STEM. Computers in Education Journal, 6(1), 65–81.

    Google Scholar 

  • Williams, L. C., Underwood, S. M., Klymkowsky, M. W., & Cooper, M. M. (2015). Are noncovalent interactions an Achilles heel in chemistry education? A comparison of instructional approaches. Journal of Chemical Education, 92, 1979–1987. https://doi.org/10.1021/acs.jchemed.5b00619

    Article  CAS  Google Scholar 

  • Winstone, N. E., Nash, R. A., Rowntree, J., & Parker, M. (2017). ‘It’d be useful, but I wouldn’t use it’: Barriers to university students’ feedback seeking and recipience. Studies in Higher Education, 42(11), 2026–2041. https://doi.org/10.1080/03075079.2015.1130032

    Article  Google Scholar 

Download references

Author information

Author notes

  1. Melanie M. Cooper

    Present address: Department of Chemistry, Michigan State University, East Lansing, MI, USA

Authors and Affiliations

  1. Department of Molecular, Cellular Developmental Biology, University of Colorado Boulder, Boulder, CO, USA

    Michael W. Klymkowsky

Authors
  1. Melanie M. Cooper
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Michael W. Klymkowsky
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Michael W. Klymkowsky .

Editor information

Editors and Affiliations

  1. Department of Neuroscience, Brighton and Sussex Medical School, Brighton, UK

    Harry J. Witchel

  2. Department of Medical Education, Geisel School of Medicine, Dartmouth College, Hanover, NH, USA

    Michael W. Lee

Rights and permissions

Reprints and permissions

Copyright information

© 2022 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this chapter

Check for updates. Verify currency and authenticity via CrossMark

Cite this chapter

Cooper, M.M., Klymkowsky, M.W. (2022). Aligning Assessment Goals with the Current and Future Technologies Needed to Achieve Them. In: Witchel, H.J., Lee, M.W. (eds) Technologies in Biomedical and Life Sciences Education. Methods in Physiology. Springer, Cham. https://doi.org/10.1007/978-3-030-95633-2_8

Download citation

Publish with us

Policies and ethics

Search

Navigation