Learning Analytics and eAssessment—Towards Computational Psychometrics by Combining Psychometrics with Learning Analytics
- 456 Downloads
From a psychometric point of view, assessment means to infer what a learner knows and can do in the real world from limited evidence observed in a standardized testing situation. From a learning analytics perspective assessment means to observe real behavior in digital learning environments to conclude the learner status with the intent to positively influence the learning process. Although psychometrics and learning analytics share similar goals, for instance, formative assessment, while applying different methods and theories, the two disciplines are so far highly separated. This chapter aims at paving the way for an advanced understanding of assessment by comparing and integrating the learning analytics and the psychometric approach of assessment. We will discuss means to show this new way of assessment of educational concepts such as (meta-) cognition, motivation, and reading comprehension skills that can be addressed either from data-driven approach (learning analytics) or from a theory-driven approach (psychometrics). Finally, we show that radically new ways of assessment are located in the middle space where both disciplines are combined into a new research discipline called ‘Computational Psychometrics’.
KeywordsPsychometrics Learning analytics Formative assessment Multimodal data Process data (Meta-) cognition Motivation Reading comprehension skills
- Arieli-Attali, M., Ward, S., Thomas, J., Deonovic, B., & von Davier, A. A. (2019). The expanded evidence-centered design (e-ECD) for learning and assessment systems: A framework for incorporating learning goals and processes within assessment design. Frontiers in psychology, 10(853). https://doi.org/10.3389/fpsyg.2019.00853.
- Arnold, K., & Pistilli, M. D. (2012). Course signals at Purdue: using learning analytics to increase student success. In S. Buckingham Shum, D. Gasevic & R. Ferguson (Eds.), Proceedings of the 2nd International Conference on Learning Analytics and Knowledge (LAK 2012) (pp. 267–270). New York, NY, USA: ACM. http://dx.doi.org/10.1145/2330601.2330666.
- Biedermann, D., Schneider, J. & Drachsler, H. (2018). The learning analytics indicator repository. In 13th European Conference on Technology Enhanced Learning, EC-TEL 2018, Leeds, UK, September 3–5, 2018, Proceedings (Lecture Notes in Computer Science, Vol. 11082, pp. 579–582). Cham: Springer.Google Scholar
- Blikstein, P. (2013). Multimodal learning analytics. In Proceedings of the Third International Conference on Learning Analytics and Knowledge—LAK 2013 (pp. 102–106). New York, USA: ACM. https://doi.org/10.1145/2460296.2460316.
- Cukurova, M., Kent, C., & Luckin, R. (2019). The value of multimodal data in classification of social and emotional aspects of tutoring. AIED, 2(2019), 46–51.Google Scholar
- DiCerbo, K. E., Shute, V., & Kim, Y. J. (2017). The future of assessment in technology rich environments: Psychometric considerations of ongoing assessment. In J. M. Spector, B. Lockee, & M. Childress (Eds.), Learning, design, and technology: An international compendium of theory, research, practice, and policy (pp. 1–21). New York, NY: Springer.Google Scholar
- Di Mitri, D., Schneider, J., Specht, M., & Drachsler, H. (2019). Detecting mistakes in CPR training with multimodal data and neural networks. Sensors, 17, 3099.Google Scholar
- Drachsler, H., & Schneider, J. (2018). Special Issue on multimodal learning analytics. Journal of Computer Assisted Learning, 34.Google Scholar
- Echeverria, V., Martinez-Maldonado, R., Granda, R., Chiluiza, K., Conati, C., & Shum, S. B. (2018). Driving data storytelling from learning design. In Proceedings of the 8th International Conference on Learning Analytics and Knowledge–LAK ’18 2018 (pp. 131–140). https://doi.org/10.1145/3170358.3170380.
- Goldhammer, F., & Zehner, F. (2017). What to make of and how to interpret process data. Measurement: Interdisciplinary Research and Perspectives, 15, 128–132.Google Scholar
- Goldhammer, F., Hahnel, C., & Kroehne, U. (2020). Analyzing log file data from PIAAC. In D. B. Maehler & B. Rammstedt (Eds.), Large-scale cognitive assessment: Analyzing PIAAC data. Cham: Springer.Google Scholar
- Greller, W., & Drachsler, H. (2012). Translating learning into numbers. Journal of Educational Technology & Society, 15, 42–57.Google Scholar
- Jivet, I., Scheffel, M., Drachsler, H., & Specht, M. (2017). Awareness is not enough. Pitfalls of learning analytics dashboards in the educational practice. In 12th European Conference on Technology-Enhanced Learning. Tallinn, Estonia, 12–15 September 2017.Google Scholar
- Mislevy, R. (2019). On integrating psychometrics and learning analytics in complex assessments. In H. Jiao, R. W. Lissitz, & A. van Wie (Eds.), Data analytics and psychometrics (pp. 1–52). Charlotte, NC, USA: Information Age Publishing.Google Scholar
- Mislevy, R., Behrens, J., Dicerbo, K., & Levy, R. (2012). Design and Discovery in Educational Assessment: Evidence-Centered Design, Psychometrics, and Educational Data Mining. Journal of Educational Data Mining, 4(1), 11–48.Google Scholar
- Mislevy, R. J., Almond, R. G., & Lukas, J. F. (2003). A brief introduction to evidence-centered design. ETS Research Report Series, 2003(1), i–29. https://doi.org/10.1002/j.2333-8504.2003.tb01908.x.CrossRefGoogle Scholar
- Nguyen, Q., Huptych, M., & Rienties, B. (2018). Linking students’ timing of engagement to learning design and academic performance. Paper presented at the Proceedings of the 8th International Conference on Learning Analytics and Knowledge, Sydney, New South Wales, Australia.Google Scholar
- Scheffel, M., van Limbeek, E., Joppe, D., van Hooijdonk, J., Kockelkoren, C., Schmitz, M., Ebus, P., Sloep, P., & Drachsler, H. (2019). The means to a blend: A practical model for the redesign of face-to-face education to blended learning. In 14th European Conference on Technology Enhanced Learning, EC-TEL 2019, Delft, 16–19 September 2019, Proceedings (Lecture Notes in Computer Science). Cham: Springer.Google Scholar
- Schön, D. (1983). The reflective practitioner. New York: Basic.Google Scholar
- Schneider, J., Di Mitri, D., Limbu, B., & Drachsler, H. (2018). Multimodal learning hub: A tool for capturing customizable multimodal learning experiences. In 13th European Conference on Technology Enhanced Learning, EC-TEL 2018, Leeds, UK, 3–5, September 2018, Proceedings (Lecture Notes in Computer Science, Vol. 11082, pp. 45–58). Cham: Springer.Google Scholar
- Shute, V. J. (2011). Stealth assessment in computer-based games to support learning. Computer Games Instruction, 55, 503–524.Google Scholar
- Ternier S., Scheffel M., Drachsler H. (2018). Towards a cloud-based big data infrastructure for higher education institutions. In: Spector J. et al. (Eds), Frontiers of cyberlearning. Lecture Notes in Educational Technology. Springer, Singapore.Google Scholar
- von Davier, A. A. (2017). Computational psychometrics in support of collaborative educational assessments. Journal of Educational Measurement, 54(1), 3–11.Google Scholar
- Weidenbach, M., Drachsler, H., Wild, F., Kreutter, S., Razek, V., Grunst, G., Ender, J., Berlage, T., & Janousek J. (2007). EchoComTEE—A simulator for transoesophageal echocardiography. Anaesthesia, 62(4), 347–357.Google Scholar
- Winne, P. H. (2017). Leveraging big data to help each learner upgrade learning and accelerate learning science. Teachers College Record, 119(3).Google Scholar