Skip to main content

Process Data Analysis in ILSAs

An Ecological Framework and Literature Review

  • 26 Accesses

Part of the Springer International Handbooks of Education book series (SIHE)

Abstract

Computational advancements in the last couple of decades have brought forth a new era of international large-scale assessments (ILSAs) in which the administration of computer-based tests is becoming the norm. Beyond collecting the correct/incorrect answers for each item, computer-based assessments are also able to collect a range of actions performed by the respondents in the computer testing application during the course of the test administration. Both respondents’ actions – starting a unit, clicking a button, spending time until inputting or submitting an answer, and so forth – and their overall behavior – what they do with their keyboards, mice, and even their own eyes – are recorded as a new set of data called “process data,” which are normally time-stamped and can be stored in the so-called log files. There may be plenty of useful insight into the respondent’s cognitive process, and process data can potentially become a relevant element in the scoring process of an assessment, validate test score interpretations, to name a few possibilities for the analysis of such data. This chapter aims to contribute to the body of knowledge in this area by offering (1) an introduction to process data, what kind of data it contains, its relation to the cognitive process, and how it can be organized into a proposed six-layered ecological framework that facilitates its analysis; (2) a literature review of 37 seminal and state-of-the-art publications produced in the last decade on the topic, which are then analyzed both in their chronological perspective as well as in how they fit into the ecological framework; and (3) a discussion of the potential and limitations of using process data in the assessment framework, including a view of what could be the next steps in the analysis of process data from ILSAs.

Keywords

  • Computer-based assessment
  • Process data
  • Log files
  • Ecological framework
  • ILSA

This is a preview of subscription content, access via your institution.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7

References

Download references

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Denise Reis Costa .

Editor information

Editors and Affiliations

Section Editor information

Rights and permissions

Reprints and Permissions

Copyright information

© 2022 Springer Nature Switzerland AG

About this entry

Verify currency and authenticity via CrossMark

Cite this entry

Reis Costa, D., Leoncio Netto, W. (2022). Process Data Analysis in ILSAs. In: Nilsen, T., Stancel-Piątak, A., Gustafsson, JE. (eds) International Handbook of Comparative Large-Scale Studies in Education. Springer International Handbooks of Education. Springer, Cham. https://doi.org/10.1007/978-3-030-38298-8_60-1

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-38298-8_60-1

  • Received:

  • Accepted:

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-38298-8

  • Online ISBN: 978-3-030-38298-8

  • eBook Packages: Springer Reference EducationReference Module Humanities and Social Sciences