Making Use of Data for Assessments: Harnessing Analytics and Data Science

Living reference work entry
Part of the Springer International Handbooks of Education book series (SIHE)

Abstract

The increased availability of vast and highly varied amounts of data from learners, teachers, learning environments, and administrative systems within educational settings is overwhelming. The focus of this chapter is on how data with a large number of records, of widely differing datatypes, and arriving rapidly from multiple sources can be harnessed for meaningful assessments and supporting learners in a wide variety of learning situations. Distinct features of analytics-driven assessments may include self-assessments, peer assessments, and semantic rich and personalized feedback as well as adaptive prompts for reflection. The chapter concludes with future directions in the broad area of analytics-driven assessments for teachers and educational researchers.

Keywords

Assessment analytics Learning analytics Formative assessment Large-Scale assessment Data analytics 

References

  1. Almond, R. G., Steinberg, L. S., & Mislevy, R. J. (2002). Enhancing the design and delivery of assessment systems: A four process architecture. Journal of Technology, Learning, and Assessment, 1(5), 3–63.Google Scholar
  2. Baker, R. S. J. d., & Siemens, G. (2015). Educational data mining and learning analytics. In R. K. Sawyer (Ed.), The Cambridge handbook of the learning sciences (2nd ed., pp. 253–272).Google Scholar
  3. Berland, M., Baker, R. S. J. d., & Bilkstein, P. (2014). Educational data mining and learning analytics: Applications to constructionist research. Technology, Knowledge and Learning, 19(1–2), 205–220.  https://doi.org/10.1007/s10758-014-9223-7.CrossRefGoogle Scholar
  4. Black, P. J. (1998). Testing: Friend or foe? The theory and practice of assessment and testing. London: Falmer Press.Google Scholar
  5. Bloom, B. S., Hastings, J. T., & Madaus, G. F. (1971). Handbook of formative and summative evaluation of student learning. New York: McGraw-Hill.Google Scholar
  6. Boud, D. (2000). Sustainable assessment: Rethinking assessment for the learning society. Studies in Continuing Education, 22(2), 151–167.  https://doi.org/10.1080/713695728.CrossRefGoogle Scholar
  7. Buckingham Shum, S., & Ferguson, R. (2012). Social learning analytics. Educational Technology & Society, 15(3), 3–26.Google Scholar
  8. Carless, D. (2007). Learning-oriented assessment: Conceptual bases and practical implications. Innovations in Education and Teaching International, 44(1), 57–66.CrossRefGoogle Scholar
  9. Cleophas, T. J., & Zwinderman, A. H. (2013). Machine learning in medicine. Amsterdam: Springer.CrossRefGoogle Scholar
  10. Coronges, K. A., Stacy, A. W., & Valente, T. W. (2007). Structural comparison of cognitive associative networks in two populations. Journal of Applied Social Psychology, 37(9), 2097–2129.  https://doi.org/10.1111/j.1559-1816.2007.00253.x.
  11. d’Aquin, M., Dietze, S., Herder, E., Drachsler, H., & Taibi, D. (2014). Using linked data in learning analytics. eLearning Papers, 36, 1–9.Google Scholar
  12. Drachsler, H., Hummel, H., & Koper, R. (2008). Personal recommender systems for learners in lifelong learning: Requirements, techniques, and model. International Journal of Learning Technologies, 3(4), 404–423.CrossRefGoogle Scholar
  13. Ellis, C. (2013). Broadening the scope and increasing usefulness of learning analytics: The case for assessment analytics. British Journal of Educational Technology, 44(4), 662–664.  https://doi.org/10.1111/bjet.12028.CrossRefGoogle Scholar
  14. Ge, X., & Ifenthaler, D. (2017). Designing engaging educational games and assessing engagement in game-based learning. In R. Zheng & M. K. Gardner (Eds.), Handbook of research on serious games for educational applications (pp. 255–272). Hershey: IGI Global.Google Scholar
  15. Gibson, D. C., & Ifenthaler, D. (2017). Preparing the next generation of education researchers for big data in higher education. In B. Kei Daniel (Ed.), Big data and learning analytics: Current theory and practice in higher education (pp. 29–42). New York: Springer.CrossRefGoogle Scholar
  16. Gibson, D. C., & Webb, M. (2015). Data science in educational assessment. Education and Information Technologies, 20(4), 697–713.  https://doi.org/10.1007/s10639-015-9411-7.CrossRefGoogle Scholar
  17. Gibson, D. C., Ifenthaler, D., & Orlic, D. (2016). Open assessment resources for deeper learning. In P. Blessinger & T. J. Bliss (Eds.), Open education: International perspectives in higher education (pp. 257–279). Cambridge, UK: Open Book Publishers.Google Scholar
  18. Gierl, M. J. (2007). Making diagnostic inferences about cognitive attributes using the rule-space model and attribute hierarchy method. Journal of Educational Measurement, 44(4), 325–340.  https://doi.org/10.1111/j.1745-3984.2007.00042.x.CrossRefGoogle Scholar
  19. Goldhammer, F., Naumann, J., Stelter, A., Toth, K., Rölke, H., & Klieme, E. (2014). The time on task effect in reading and problem solving is moderated by task difficulty and skill. Insights from a computer-based large-scale assessment. Journal of Educational Psychology, 106, 608–626.CrossRefGoogle Scholar
  20. Greiff, S., Niepel, C., Scherer, R., & Martin, R. (2016). Understanding students’ performance in a computer-based assessment of complex problem solving: An analysis of behavioral data from computer-generated log files. Computers in Human Behavior, 61, 36–46.  https://doi.org/10.1016/j.chb.2016.02.095.CrossRefGoogle Scholar
  21. Greiff, S., Molnar, G., Martin, R., Zimmermann, J., & Csapo, B. (submitted). Students’ exploration strategies in complex problem environments. A latent class approach. Journal of Educational Psychology.Google Scholar
  22. Greller, W., & Drachsler, H. (2012). Translating learning into numbers: A generic framework for learning analytics. Educational Technology & Society, 15(3), 42–57.Google Scholar
  23. Ifenthaler, D. (2009). Model-based feedback for improving expertise and expert performance. Technology, Instruction, Cognition and Learning, 7(2), 83–101.Google Scholar
  24. Ifenthaler, D. (2012). Determining the effectiveness of prompts for self-regulated learning in problem-solving scenarios. Journal of Educational Technology & Society, 15(1), 38–52.Google Scholar
  25. Ifenthaler, D. (2015). Learning analytics. In J. M. Spector (Ed.), The SAGE encyclopedia of educational technology (Vol. 2, pp. 447–451). Thousand Oaks: Sage.Google Scholar
  26. Ifenthaler, D. (2016). Automated grading. In S. Danver (Ed.), The SAGE encyclopedia of online education (p. 130). Thousand Oaks: Sage.Google Scholar
  27. Ifenthaler, D. (2017). Are higher education institutions prepared for learning analytics? TechTrends, 61(4), 366–371.  https://doi.org/10.1007/s11528-016-0154-0.CrossRefGoogle Scholar
  28. Ifenthaler, D., & Dikli, S. (2015). Automated scoring of essays. In J. M. Spector (Ed.), The SAGE encyclopedia of educational technology (Vol. 1, pp. 64–68). Thousand Oaks: Sage.Google Scholar
  29. Ifenthaler, D., & Seel, N. M. (2013). Model-based reasoning. Computers & Education, 64, 131–142.  https://doi.org/10.1016/j.compedu.2012.11.014.
  30. Ifenthaler, D., & Pirnay-Dummer, P. (2014). Model-based tools for knowledge assessment. In J. M. Spector, M. D. Merrill, J. Elen, & M. J. Bishop (Eds.), Handbook of research on educational communications and technology (4th ed., pp. 289–301). New York: Springer.CrossRefGoogle Scholar
  31. Ifenthaler, D., & Seel, N. M. (2005). The measurement of change: Learning-dependent progression of mental models. Technology, Instruction, Cognition and Learning, 2(4), 317–336.Google Scholar
  32. Ifenthaler, D., & Widanapathirana, C. (2014). Development and validation of a learning analytics framework: Two case studies using support vector machines. Technology, Knowledge and Learning, 19(1–2), 221–240.  https://doi.org/10.1007/s10758-014-9226-4.CrossRefGoogle Scholar
  33. Ifenthaler, D., Pirnay-Dummer, P., & Seel, N. M. (Eds.). (2010). Computer-based diagnostics and systematic analysis of knowledge. New York: Springer.Google Scholar
  34. Ifenthaler, D., Eseryel, D., & Ge, X. (2012). Assessment for game-based learning. In D. Ifenthaler, D. Eseryel, & X. Ge (Eds.), Assessment in game-based learning. Foundations, innovations, and perspectives (pp. 3–10). New York: Springer.CrossRefGoogle Scholar
  35. Klosgen, W., & Zytkow, J. (2002). Handbook of data mining and knowledge discovery. New York: Oxford University Press.Google Scholar
  36. Lehmann, T., Haehnlein, I., & Ifenthaler, D. (2014). Cognitive, metacognitive and motivational perspectives on preflection in self-regulated online learning. Computers in Human Behavior, 32, 313–323.  https://doi.org/10.1016/j.chb.2013.07.051
  37. Loh, C. S., Sheng, Y., & Ifenthaler, D. (2015). Serious games analytics: Theoretical framework. In C. S. Loh, Y. Sheng, & D. Ifenthaler (Eds.), Serious games analytics. Methodologies for performance measurement, assessment, and improvement (pp. 3–29). New York: Springer.Google Scholar
  38. Long, P. D., & Siemens, G. (2011). Penetrating the fog: Analytics in learning and education. Educause Review, 46(5), 31–40.Google Scholar
  39. Mertler, C. A. (2009). Teachers’ assessment knowledge and their perceptions of the impact of classroom assessment professional development. Improving Schools, 12(2), 101–113.  https://doi.org/10.1177/1365480209105575.CrossRefGoogle Scholar
  40. Narciss, S. (2008). Feedback strategies for interactive learning tasks. In J. M. Spector, M. D. Merrill, J. J. G. van Merriënboer, & M. P. Driscoll (Eds.), Handbook of research on educational communications and technology (pp. 125–143). New York: Taylor & Francis Group.Google Scholar
  41. Newton, P. E. (2007). Clarifying the purposes of educational assessment. Assessment in Education: Principles, Policy & Practice, 14(2), 149–170.  https://doi.org/10.1080/09695940701478321.CrossRefGoogle Scholar
  42. OECD. (2013a). OECD skills outlook 2013: First results from the survey of adult skills. Paris: OECD Publishing.Google Scholar
  43. OECD. (2013b). PISA 2012 assessment and analytical framework. Paris: OECD Publishing.CrossRefGoogle Scholar
  44. OECD. (2014). PISA 2012 results: Creative problem solving. Paris: OECD Publishing.CrossRefGoogle Scholar
  45. Pellegrino, J. W., Chudowsky, N., & Glaser, R. (Eds.). (2001). Knowing what students know: The science and design of educational assessment. Washington, DC: National Academy Press.Google Scholar
  46. Piaget, J. (1950). La construction du réel chez l’enfant. Neuchatel: Delachaux et Niestlé S.A.Google Scholar
  47. Pirnay-Dummer, P., & Ifenthaler, D. (2011). Reading guided by automated graphical representations: How model-based text visualizations facilitate learning in reading comprehension tasks. Instructional Science, 39(6), 901–919.  https://doi.org/10.1007/s11251-010-9153-2.
  48. Plake, B. S. (1993). Teacher assessment literacy: Teachers’ competencies in the educational assessment of students. Mid-Western Educational Researcher, 6(2), 21–27.Google Scholar
  49. Romero, C., & Ventura, S. (2015). Learning analytics: From research to practice. In J. A. Larusson & B. White (Eds.), Technology, knowledge and learning.  https://doi.org/10.1007/s10758-015-9244-x.Google Scholar
  50. Sadler, D. R. (1989). Formative assessment and the design of instructional systems. Instructional Science, 18, 119–144.CrossRefGoogle Scholar
  51. Sadler, D. R. (2010). Beyond feedback: Developing student capability in complex appraisal. Assessment & Evaluation in Higher Education, 35(5), 535–550.CrossRefGoogle Scholar
  52. Scriven, M. (1967). The methodology of evaluation. Washington, DC: American Educational Research Association.Google Scholar
  53. Shermis, M. D., & Hamner, B. (2013). Contrasting state-of-the-art automated scoring of essays. In M. D. Shermis & J. Burstein (Eds.), Handbook of automated essay evaluation (pp. 213–246). New York: Routledge.Google Scholar
  54. Shute, V. J. (2008). Focus on formative feedback. Review of Educational Research, 78(1), 153–189.CrossRefGoogle Scholar
  55. Shute, V. J. (2011). Stealth assessment in computer-based games to support learning. In S. Tobias & J. D. Fletcher (Eds.), Computer games and instruction (pp. 503–524). Charlotte: Information Age Publishers.Google Scholar
  56. Shute, V. J., Wang, L., Greiff, S., Zhao, W., & Moore, G. (2016). Measuring problem solving skills via stealth assessment in an engaging video game. Computers in Human Behavior, 63, 106–117.  https://doi.org/10.1016/j.chb.2016.05.047.CrossRefGoogle Scholar
  57. Spector, J. M. (2009). Adventures and advances in instructional design theory and practice. In L. Moller, J. B. Huett, & D. M. Harvey (Eds.), Learning and instructional technologies for the 21st century (pp. 1–14). New York: Springer.Google Scholar
  58. Spector, J. M., Ifenthaler, D., Sampson, D. G., Yang, L., Mukama, E., Warusavitarana, A.,⋯ Gibson, D. C. (2016). Technology enhanced formative assessment for 21st century learning. Educational Technology & Society, 19(3), 58–71.Google Scholar
  59. Stiggins, R. J. (1995). Assessment literacy for the 21st century. Phi Delta Kappan, 77(3), 238–245.Google Scholar
  60. Tatsuoka, K. (2009). Cognitive assessment: An introduction to the rule-space method. New York: Routledge.Google Scholar
  61. Wagner, W., & Wagner, S. U. (1985). Presenting questions, processing responses, and providing feedback in CAI. Journal of Instructional Development, 8(4), 2–8.CrossRefGoogle Scholar
  62. Wiliam, D. (2011). What is assessment for learning? Studies in Educational Evaluation, 37(1), 3–14.CrossRefGoogle Scholar
  63. Wüstenberg, S., Greiff, S., Molnar, G., & Funke, J. (2014). Cross-national gender differences in complex problem solving and their determinants. Learning and Individual Differences, 29, 18–29.  https://doi.org/10.1016/j.lindif.2013.10.006.CrossRefGoogle Scholar

Copyright information

© Springer International Publishing AG 2018

Authors and Affiliations

  1. 1.Learning, Design and TechnologyUniversity of MannheimMannheimGermany
  2. 2.University of LuxembourgLuxembourg CityLuxembourg
  3. 3.Curtin UniversityPerthAustralia

Section editors and affiliations

  • Mary Webb
    • 1
  • Dirk Ifenthaler
    • 2
  1. 1.King's College LondonLondonUK
  2. 2.University of MannheimMannheimGermany

Personalised recommendations