Skip to main content
SpringerLink
Log in

Awareness Is Not Enough: Pitfalls of Learning Analytics Dashboards in the Educational Practice

  • Conference paper
  • First Online:
Data Driven Approaches in Digital Education (EC-TEL 2017)

Part of the book series: Lecture Notes in Computer Science ((LNISA,volume 10474))

Included in the following conference series:

Abstract

It has been long argued that learning analytics has the potential to act as a “middle space” between the learning sciences and data analytics, creating technical possibilities for exploring the vast amount of data generated in online learning environments. One common learning analytics intervention is the learning dashboard, a support tool for teachers and learners alike that allows them to gain insight into the learning process. Although several related works have scrutinised the state-of-the-art in the field of learning dashboards, none have addressed the theoretical foundation that should inform the design of such interventions. In this systematic literature review, we analyse the extent to which theories and models from learning sciences have been integrated into the development of learning dashboards aimed at learners. Our critical examination reveals the most common educational concepts and the context in which they have been applied. We find evidence that current designs foster competition between learners rather than knowledge mastery, offering misguided frames of reference for comparison.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 39.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Anderson, T.: The Theory and Practice of Online Learning. Athabasca University Press, Edmonton (2008)

    Google Scholar 

  2. Beheshitha, S.S., Hatala, M., Gašević, D., Joksimović, S.: The role of achievement goal orientations when studying effect of learning analytics visualizations. In: Proceedings of LAK 2016, pp. 54–63. ACM (2016)

    Google Scholar 

  3. Bloom, B., Krathwohl, D., Masia, B.: Bloom Taxonomy of Educational Objectives. Allyn and Bacon, Pearson Education, Boston (1984)

    Google Scholar 

  4. Bodily, R., Verbert, K.: Trends and issues in student-facing learning analytics reporting systems research. In: Proceedings of LAK 2017, pp. 309–318. ACM (2017)

    Google Scholar 

  5. Charleer, S., Klerkx, J., Duval, E., Verbert, K., De Laet, T.: Creating effective learning analytics dashboards: lessons learnt. In: Verbert, K., Sharples, M., Klobučar, T. (eds.) EC-TEL 2016. LNCS, vol. 9891, pp. 42–56. Springer, Cham (2016). doi:10.1007/978-3-319-45153-4_4

    Chapter  Google Scholar 

  6. Corey, M.L., Leinenbach, M.T.: Universal design for learning: theory and practice. In: Proceedings of 2004 Society for Information Technology and Teacher Education International Conference, pp. 4919–4926 (2004)

    Google Scholar 

  7. Davis, D., Jivet, I., Kizilcec, R.F., Chen, G., Hauff, C., Houben, G.J.: Follow the successful crowd: raising MOOC completion rates through social comparison at scale. In: Proceedings of LAK 2017, pp. 454–463. ACM (2017)

    Google Scholar 

  8. DeCarvalho, R.J.: The humanistic paradigm in education. Hum. Psychol. 19(1), 88 (1991)

    Google Scholar 

  9. Durall, E., Gros, B.: Learning analytics as a metacognitive tool. In: CSEDU, vol. 1, pp. 380–384 (2014)

    Google Scholar 

  10. Ekman, P., Friesen, W.V.: Facial action coding system (1977)

    Google Scholar 

  11. Elder, L., Paul, R.: Critical thinking: why we must transform our teaching. J. Dev. Educ. 18(1), 34 (1994)

    Google Scholar 

  12. Engeström, Y.: Expansive visibilization of work: an activity-theoretical perspective. Comput. Support. Coop. Work (CSCW) 8(1), 63–93 (1999)

    Article  Google Scholar 

  13. Ferguson, R.: Learning analytics: drivers, developments and challenges. Int. J. Technol. Enhanc. Learn. 4(5–6), 304–317 (2012)

    Article  Google Scholar 

  14. Festinger, L.: A theory of social comparison processes. Hum. Relat. 7(2), 117–140 (1954)

    Article  Google Scholar 

  15. Few, S.: Information Dashboard Design: Displaying Data for At-a-Glance Monitoring. Analytics Press, Berkeley (2013)

    Google Scholar 

  16. Fredricks, J.A., Blumenfeld, P.C., Paris, A.H.: School engagement: potential of the concept, state of the evidence. Rev. Educ. Res. 74(1), 59–109 (2004)

    Article  Google Scholar 

  17. Gašević, D., Dawson, S., Siemens, G.: Lets not forget: learning analytics are about learning. TechTrends 59(1), 64–71 (2015)

    Article  Google Scholar 

  18. Gelfand, M.J., Raver, J.L., Nishii, L., Leslie, L.M., Lun, J., Lim, B.C., Duan, L., Almaliach, A., Ang, S., Arnadottir, J., et al.: Differences between tight and loose cultures: a 33-nation study. Science 332(6033), 1100–1104 (2011)

    Article  Google Scholar 

  19. Greller, W., Drachsler, H.: Translating learning into numbers: a generic framework for learning analytics. Educ. Technol. Soc. 15(3), 42–57 (2012)

    Google Scholar 

  20. Guerra, J., Hosseini, R., Somyurek, S., Brusilovsky, P.: An intelligent interface for learning content: combining an open learner model and social comparison to support self-regulated learning and engagement. In: Proceedings of IUI 2016, pp. 152–163. ACM (2016)

    Google Scholar 

  21. Haggis, T.: Constructing images of ourselves? A critical investigation into ‘approaches to learning’ research in higher education. Br. Educ. Res. J. 29(1), 89–104 (2003)

    Article  Google Scholar 

  22. Hofstede, G.: Cultures and Organizations: Software of the Mind. McGraw-Hill, London (1991)

    Google Scholar 

  23. Huguet, P., Galvaing, M.P., Monteil, J.M., Dumas, F.: Social presence effects in the Stroop task: further evidence for an attentional view of social facilitation. J. Pers. Soc. Psychol. 77(5), 1011 (1999)

    Article  Google Scholar 

  24. Kim, B.: Social constructivism. Emerg. Perspect. Learn. Teach. Technol. 1(1), 16 (2001)

    Google Scholar 

  25. Kirkpatrick, D.L.: Evaluating Training Programs. Tata McGraw-Hill Education, San Francisco (1975)

    Google Scholar 

  26. Klerkx, J., Verbert, K., Duval, E.: Enhancing learning with visualization techniques. In: Spector, J., Merrill, M., Elen, J., Bishop, M. (eds.) Handbook of Research on Educational Communications and Technology, pp. 791–807. Springer, New York (2014)

    Chapter  Google Scholar 

  27. Loboda, T.D., Guerra, J., Hosseini, R., Brusilovsky, P.: Mastery grids: an open source social educational progress visualization. In: Rensing, C., de Freitas, S., Ley, T., Muñoz-Merino, P.J. (eds.) EC-TEL 2014. LNCS, vol. 8719, pp. 235–248. Springer, Cham (2014). doi:10.1007/978-3-319-11200-8_18

    Google Scholar 

  28. Lonn, S., Aguilar, S.J., Teasley, S.D.: Investigating student motivation in the context of a learning analytics intervention during a summer bridge program. Comput. Hum. Behav. 47, 90–97 (2015)

    Article  Google Scholar 

  29. Marzouk, Z., Rakovic, M., Liaqat, A., Vytasek, J., Samadi, D., Stewart-Alonso, J., Ram, I., Woloshen, S., Winne, P.H., Nesbit, J.C.: What if learning analytics were based on learning science? Australas. J. Educ. Technol. 32(6), 1–18 (2016)

    Article  Google Scholar 

  30. McAlpine, L., Weston, C.: Reflection: issues related to improving professors’ teaching and students’ learning. Instr. Sci. 28(5), 363–385 (2000)

    Article  Google Scholar 

  31. Moher, D., Liberati, A., Tetzlaff, J., Altman, D.G., PRISMA Group, et al.: Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement. PLoS Med. 6(7), 1000097 (2009)

    Google Scholar 

  32. Pintrich, P.R.: Multiple goals, multiple pathways: the role of goal orientation in learning and achievement. J. Educ. Psychol. 92(3), 544 (2000)

    Article  Google Scholar 

  33. Pintrich, P.R., De Groot, E.V.: Motivational and self-regulated learning components of classroom academic performance. J. Educ. Psychol. 82(1), 33 (1990)

    Article  Google Scholar 

  34. Sadler, D.R.: Formative assessment and the design of instructional systems. Instr. Sci. 18(2), 119–144 (1989)

    Article  Google Scholar 

  35. Schwendimann, B., Rodriguez-Triana, M., Vozniuk, A., Prieto, L., Boroujeni, M., Holzer, A., Gillet, D., Dillenbourg, P.: Perceiving learning at a glance: a systematic literature review of learning dashboard research. IEEE Trans. Learn. Technol. 10(1), 30–41 (2016)

    Article  Google Scholar 

  36. Shum, S.B., Crick, R.D.: Learning dispositions and transferable competencies: pedagogy, modelling and learning analytics. In: Proceedings of LAK 2012, pp. 92–101. ACM (2012)

    Google Scholar 

  37. Siemens, G., Gašević, D.: Guest editorial-learning and knowledge analytics. Educ. Technol. Soc. 15(3), 1–2 (2012)

    Google Scholar 

  38. Suthers, D., Verbert, K.: Learning analytics as a middle space. In: Proceedings of LAK 2013, pp. 1–4. ACM (2013)

    Google Scholar 

  39. Tan, J.P.L., Yang, S., Koh, E., Jonathan, C.: Fostering 21st century literacies through a collaborative critical reading and learning analytics environment: user-perceived benefits and problematics. In: Proceedings of LAK 2016, pp. 430–434. ACM (2016)

    Google Scholar 

  40. Trilling, B., Fadel, C.: 21st Century Skills: Learning for Life in Our Times. Wiley, Hoboken (2009)

    Google Scholar 

  41. Venant, R., Vidal, P., Broisin, J.: Evaluation of learner performance during practical activities: an experimentation in computer education. In: Proceedings of ICALT 2016, pp. 237–241. IEEE (2016)

    Google Scholar 

  42. Verbert, K., Duval, E., Klerkx, J., Govaerts, S., Santos, J.L.: Learning analytics dashboard applications. Am. Behav. Sci. 57(10), 1500–1509 (2013)

    Article  Google Scholar 

  43. Verbert, K., Govaerts, S., Duval, E., Santos, J.L., Van Assche, F., Parra, G., Klerkx, J.: Learning dashboards: an overview and future research opportunities. Pers. Ubiquit. Comput. 18(6), 1499–1514 (2014)

    Google Scholar 

  44. Winne, P.H., Zimmerman, B.J.: Self-regulated learning viewed from models of information processing. Self Regul. Learn. Acad. Achiev. Theor. Perspect. 2, 153–189 (2001)

    Google Scholar 

  45. Wise, A.F.: Designing pedagogical interventions to support student use of learning analytics. In: Proceedings of LAK 2014, pp. 203–211. ACM (2014)

    Google Scholar 

  46. Wu, W.H., Hsiao, H.C., Wu, P.L., Lin, C.H., Huang, S.H.: Investigating the learning-theory foundations of game-based learning: a meta-analysis. J. Comput. Assist. Learn. 28(3), 265–279 (2012)

    Article  Google Scholar 

  47. Yoo, Y., Lee, H., Jo, I.-H., Park, Y.: Educational dashboards for smart learning: review of case studies. In: Chen, G., Kumar, V., Kinshuk, Huang, R., Kong, S.C., et al. (eds.) Emerging Issues in Smart Learning. LNET, pp. 145–155. Springer, Heidelberg (2015). doi:10.1007/978-3-662-44188-6_21

  48. Zimmerman, B.J.: Self-regulated learning and academic achievement: an overview. Educ. psychol. 25(1), 3–17 (1990)

    Article  MathSciNet  Google Scholar 

  49. Zimmerman, B.J., Boekarts, M., Pintrich, P., Zeidner, M.: A social cognitive perspective. In: Handbook of Self-Regulation, vol. 13, No. 1, pp. 695–716 (2000)

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Open Universiteit, Valkenburgerweg 177, 6419 AT, Heerlen, Netherlands

    Ioana Jivet, Maren Scheffel, Hendrik Drachsler & Marcus Specht

  2. Goethe University Frankfurt, Frankfurt, Germany

    Hendrik Drachsler

  3. German Institute for International Educational Research (DIPF), Frankfurt, Germany

    Hendrik Drachsler

Authors
  1. Ioana Jivet
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Maren Scheffel
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Hendrik Drachsler
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Marcus Specht
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Ioana Jivet .

Editor information

Editors and Affiliations

  1. Jean Moulin Lyon 3 University, Lyon, France

    Élise Lavoué

  2. German Institute for International Educational Research, Goethe University, Frankfurt, Germany

    Hendrik Drachsler

  3. KU Leuven, Leuven, Belgium

    Katrien Verbert

  4. University of Toulouse, Toulouse, France

    Julien Broisin

  5. Pontificia Universidad, Santiago de Chile, Chile

    Mar Pérez-Sanagustín

Rights and permissions

Reprints and permissions

Copyright information

© 2017 Springer International Publishing AG

About this paper

Cite this paper

Jivet, I., Scheffel, M., Drachsler, H., Specht, M. (2017). Awareness Is Not Enough: Pitfalls of Learning Analytics Dashboards in the Educational Practice. In: Lavoué, É., Drachsler, H., Verbert, K., Broisin, J., Pérez-Sanagustín, M. (eds) Data Driven Approaches in Digital Education. EC-TEL 2017. Lecture Notes in Computer Science(), vol 10474. Springer, Cham. https://doi.org/10.1007/978-3-319-66610-5_7

Download citation

  • DOI: https://doi.org/10.1007/978-3-319-66610-5_7

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-319-66609-9

  • Online ISBN: 978-3-319-66610-5

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation