Integrating Real-Time Drawing and Writing Diagnostic Models: An Evidence-Centered Design Framework for Multimodal Science Assessment

  • Andy Smith
  • Osman Aksit
  • Wookhee Min
  • Eric Wiebe
  • Bradford W. Mott
  • James C. Lester
Conference paper

DOI: 10.1007/978-3-319-39583-8_16

Part of the Lecture Notes in Computer Science book series (LNCS, volume 9684)
Cite this paper as:
Smith A., Aksit O., Min W., Wiebe E., Mott B.W., Lester J.C. (2016) Integrating Real-Time Drawing and Writing Diagnostic Models: An Evidence-Centered Design Framework for Multimodal Science Assessment. In: Micarelli A., Stamper J., Panourgia K. (eds) Intelligent Tutoring Systems. ITS 2016. Lecture Notes in Computer Science, vol 9684. Springer, Cham

Abstract

Interactively modeling science phenomena enables students to develop rich conceptual understanding of science. While this understanding is often assessed through summative, multiple-choice instruments, science notebooks have been used extensively in elementary and secondary grades as a mechanism to promote and reveal reflection through both drawing and writing. Although each modality has been studied individually, obtaining a comprehensive view of a student’s conceptual understanding requires analyses of knowledge represented across both modalities. Evidence-centered design (ECD) provides a framework for diagnostic measurement of data collected from student interactions with complex learning environments. This work utilizes ECD to analyze a corpus of elementary student writings and drawings collected with a digital science notebook. First, a competency model representing the core concepts of each exercise, as well as the curricular unit as a whole, was constructed. Then, evidence models were created to map between student written and drawn artifacts and the shared competency model. Finally, the scores obtained using the evidence models were used to train a deep-learning based model for automated writing assessment, as well as to develop an automated drawing assessment model using topological abstraction. The findings reveal that ECD provides an expressive unified framework for multimodal assessment of science learning with accurate predictions of student learning.

Keywords

Assessment Multimodalilty Evidence-centered design 

Copyright information

© Springer International Publishing Switzerland 2016

Authors and Affiliations

  • Andy Smith
    • 1
  • Osman Aksit
    • 2
  • Wookhee Min
    • 1
  • Eric Wiebe
    • 2
  • Bradford W. Mott
    • 1
  • James C. Lester
    • 1
  1. 1.Department of Computer ScienceNorth Carolina State UniversityRaleighUSA
  2. 2.Department of STEM EducationNorth Carolina State UniversityRaleighUSA

Personalised recommendations