Advertisement

Creating Effective Learning Analytics Dashboards: Lessons Learnt

  • Sven Charleer
  • Joris Klerkx
  • Erik Duval
  • Tinne De Laet
  • Katrien Verbert
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 9891)

Abstract

Learning Analytics (LA) dashboards help raise student and teacher awareness regarding learner activities. In blog-supported and inquiry-based learning courses, LA data is not limited to student activities, but also contains an abundance of digital learner artefacts, such as blog posts, hypotheses, and mind-maps. Exploring peer activities and artefacts can help students gain new insights and perspectives on learning efforts and outcomes, but requires effort. To help facilitate and promote this exploration, we present the lessons learnt during and guidelines derived from the design, deployment and evaluation of five dashboards.

Keywords

Learning analytics Learning dashboards Information visualisation Guidelines Collaboration 

Notes

Acknowledgements

The research leading to these results has received funding from the European Community’s 7th Framework Programme (FP7/2007–2013) under grant agreement No. 318499 - weSPOT project and the Erasmus+ programme, Key Action 2 Strategic Partnerships, of the European Union under grant agreement 2015-1-UK01-KA203-013767 ABLE project.

References

  1. 1.
    Arnold, K.E., Pistilli, M.D.: Course signals at purdue: using LA to increase student success. In: Proceedings of LAK 2012, pp. 267–270. ACM, New York (2012)Google Scholar
  2. 2.
    Black, P., Wiliam, D.: Assessment and classroom learning. Assess. Educ. Principle Policy Pract. 5(1), 7–74 (1998)CrossRefGoogle Scholar
  3. 3.
    Brennan, K., Resnick, M.: New frameworks for studying and assessing the development of computational thinking. In: Proceedings of the Annual Meeting of the American Educational Research Association (2012)Google Scholar
  4. 4.
    Brusilovsky, P.: Adaptive hypermedia: from intelligent tutoring systems to web-based education. In: Gauthier, G., VanLehn, K., Frasson, C. (eds.) ITS 2000. LNCS, vol. 1839, pp. 1–7. Springer, Heidelberg (2000)CrossRefGoogle Scholar
  5. 5.
    Bull, S., Johnson, M.D., Alotaibi, M., Byrne, W., Cierniak, G.: Visualising multiple data sources in an independent open learner model. In: Lane, H.C., Yacef, K., Mostow, J., Pavlik, P. (eds.) AIED 2013. LNCS, vol. 7926, pp. 199–208. Springer, Heidelberg (2013)CrossRefGoogle Scholar
  6. 6.
    Charleer, S., Klerkx, J., Duval, E.: Exploring inquiry-based learning analytics through interactive surfaces. In: Proceedings of 1st International Workshop on Visual Aspects of Learning Analytics. CEUR-WS (2015)Google Scholar
  7. 7.
    Charleer, S., Klerkx, J., Santos, J.L., Duval, E.: Improving awareness and reflection through collaborative, interactive visualizations of badges. In: CEUR Workshop Proceeding of ARTEL@EC-TEL, vol. 1103, pp. 69–81. CEUR-WS.org (2013)Google Scholar
  8. 8.
    Charleer, S., Odriozola, S., Luis, J., Klerkx, J., Duval, E.: Larae: learning analytics reflection and awareness environment. In: CEUR Workshop Proceedings, vol. 1238, pp. 85–87. CEUR-WS (2014)Google Scholar
  9. 9.
    Charleer, S., Santos, J.L., Klerkx, J., Duval, E.: Improving teacher awareness through activity, badge and content visualizations. In: Cao, Y., Väljataga, T., Tang, J.K.T., Leung, H., Laanpere, M. (eds.) ICWL 2014 Workshops. LNCS, vol. 8699, pp. 143–152. Springer, Heidelberg (2014)Google Scholar
  10. 10.
    Clow, D., Makriyannis, E.: iSpot analysed: participatory learning and reputation. In: Proceedings of LAK 2011, pp. 34–43. ACM, New York (2011)Google Scholar
  11. 11.
    Conde, M.A., Garcia-Penalvo, F.J., Gomez-Aguilar, D.A., Theron, R.: Visual learning analytics techniques applied in software engineering subjects. In: Frontiers in Education Conference (FIE 2014), pp. 1–9. IEEE (2014)Google Scholar
  12. 12.
    Dillenbourg, P., Zufferey, G., Alavi, H.S., Jermann, P., Do, L.H.S., Bonnard, Q., Cuendet, S., Kaplan, F.: Classroom orchestration: the third circle of usability. In: CSCL 2011 Conference Proceedings, vol. 1, pp. 510–517. International Society of the Learning Sciences (2011)Google Scholar
  13. 13.
    Fields, D.A., Quirke, L., Amely, J., Maughan, J.: Combining big data and thickdata analyses for understanding youth learning trajectories in a summer coding camp. In: Proceedings of SIGCSE 2016, pp. 150–155. ACM, New York (2016)Google Scholar
  14. 14.
    Fulantelli, G., Taibi, D., Arrigo, M.: A framework to support educational decision making in mobile learning. Comput. Hum. Behav. 47, 50–59 (2015)CrossRefGoogle Scholar
  15. 15.
    Govaerts, S., Verbert, K., Duval, E., Pardo, A.: The student activity meter for awareness and self-reflection. In: Proceedings of ACM Annual Conference Extended Abstracts on Human Factors in Computing Systems, pp. 869–884. ACM (2012)Google Scholar
  16. 16.
    Hu, Y.-H., Lo, C.-L., Shih, S.-P.: Developing early warning systems to predict students online learning performance. Comput. Hum. Behav. 36, 469–478 (2014)CrossRefGoogle Scholar
  17. 17.
    Huang, D., Tory, M., Aseniero, B.A., Bartram, L., Bateman, S., Carpendale, S., Tang, A., Woodbury, R.: Personal visualization and personal visual analytics. IEEE Trans. Visual. Comput. Graph. 21(3), 420–433 (2015)CrossRefGoogle Scholar
  18. 18.
    Isenberg, P., Elmqvist, N., Scholtz, J., Cernea, D., Ma, K.-L., Hagen, H.: Collaborative visualization: definition, challenges, and research agenda. Inf. Vis. 10(4), 310–326 (2011)CrossRefGoogle Scholar
  19. 19.
    Ji, M., Michel, C., Lavoué, É., George, S.: DDART, a dynamic dashboard for collection, analysis and visualization of activity and reporting traces. In: Rensing, C., de Freitas, S., Ley, T., Muñoz-Merino, P.J. (eds.) EC-TEL 2014. LNCS, vol. 8719, pp. 440–445. Springer, Heidelberg (2014)Google Scholar
  20. 20.
    Kapros, E., Peirce, N.: Empowering L&D managers through customisation of inline learning analytics. In: Zaphiris, P., Ioannou, A. (eds.) LCT 2014. LNCS, vol. 8523, pp. 282–291. Springer, Heidelberg (2012)Google Scholar
  21. 21.
    Klamma, R.: Emerging research topics in social learning. In: Proceedings of the 7th International Conference on Networked Learning, vol. 2010, pp. 224–231 (2010)Google Scholar
  22. 22.
    Klerkx, J., Verbert, K., Duval, E.: Learning analytics dashboards. In: Lang, C., Siemens, G. (eds.) Handbook of Learning Analytics & Educational Data Mining (Accepted 2016)Google Scholar
  23. 23.
    Lin, W.-J., Liu, Y.-L., Kakusho, K., Yueh, H.-P., Murakami, M., Minoh, M.: Blog as a tool to develop e-learning experience in an international distance course. In: 6th International Conference on Advanced Learning Technologies, pp. 290–292. IEEE (2006)Google Scholar
  24. 24.
    Marques, A.M., Krejci, R., Siqueira, S.W., Pimentel, M., Braz, M.H.L.: Structuring the discourse on social networks for learning: case studies on blogs and microblogs. Comput. Hum. Behav. 29(2), 395–400 (2013)CrossRefGoogle Scholar
  25. 25.
    Martinez-Maldonado, R., Pardo, A., Mirriahi, N., Yacef, K., Kay, J., Clayphan, A.: Latux: an iterative workflow for designing, validating and deploying learning analytics visualisations. J. Learn. Anal. 2(3), 9–39 (2016)CrossRefGoogle Scholar
  26. 26.
    Martinez-Maldonado, R., Yacef, K., Dimitriadis, Y., Edbauer, M., Kay, J.: MT-Classroom and MTDashboard: supporting analysis of teacher attention in an or-chestrated multi-tabletop classroom. In: International Conference on CSCL 2013, pp. 119–128 (2013)Google Scholar
  27. 27.
    Mikroyannidis, A., Okada, A., Scott, P., Rusman, E., Specht, M., Stefanov, K., Boytchev, P., Protopsaltis, A., Held, P., Hetzner, S., Kikis-Papadakis, K., Chaimala, F.: weSPOT: a personal and social approach to inquiry-based learning. J. Univ. Comput. Sci. 19(14), 2093–2111 (2013)Google Scholar
  28. 28.
    Monroy, C., Rangel, V.S., Whitaker, R.: Stemscopes: contextualizing learning analytics in a k-12 science curriculum. In: Proceedings of LAK 2013, pp. 210–219. ACM (2013)Google Scholar
  29. 29.
    Nakahara, J., Hisamatsu, S., Yaegashi, K., Yamauchi, Y.: iTree: does the mobile phone encourage learners to be more involved in collaborative learning? In: Proceedings of CSCL 2005, pp. 470–478. International Society of the Learning Sciences (2005)Google Scholar
  30. 30.
    Ogata, H., Mouri, K.: Connecting dots for ubiquitous learning analytics. In: Cheung, S.K.S., Kwok, L.-F., Yang, H., Fong, J., Kwan, R. (eds.) ICHL 2015. LNCS, vol. 9167, pp. 46–56. Springer, Heidelberg (2015)CrossRefGoogle Scholar
  31. 31.
    Pham, M.C., Derntl, M., Cao, Y., Klamma, R.: Learning analytics for learning blogospheres. In: Popescu, E., Li, Q., Klamma, R., Leung, H., Specht, M. (eds.) ICWL 2012. LNCS, vol. 7558, pp. 258–267. Springer, Heidelberg (2012)CrossRefGoogle Scholar
  32. 32.
    Poulos, A., Mahony, M.J.: Effectiveness of feedback: the students perspective. Assess. Eval. High. Educ. 33(2), 143–154 (2008)CrossRefGoogle Scholar
  33. 33.
    Protopsaltis, A., Seitlinger, P., Chaimala, F., Firssova, O., Hetzner, S., Kikis-Papadakis, K., Boytchev, P.: Working environment with social and personal open tools for inquiry based learning: pedagogic and diagnostic frameworks. Int. J. Sci. Math. Technol. Learn. 20(4), 51–63 (2014)CrossRefGoogle Scholar
  34. 34.
    Rivera-Pelayo, V., Munk, J., Zacharias, V., Braun, S.: Live interest meter: learning from quantified feedback in mass lectures. In: Proceedings of LAK 2013, pp. 23–27. ACM, New York (2013)Google Scholar
  35. 35.
    Santos, J.L., Charleer, S., Parra, G., Klerkx, J., Duval, E., Verbert, K.: Evaluating the use of open badges in an open learning environment. In: Hernández-Leo, D., Ley, T., Klamma, R., Harrer, A. (eds.) EC-TEL 2013. LNCS, vol. 8095, pp. 314–327. Springer, Heidelberg (2013)CrossRefGoogle Scholar
  36. 36.
    Santos, J.L., Govaerts, S., Verbert, K., Duval, E.: Goal-oriented visualizations of activity tracking: a case study with engineering students. In: Proceedings of LAK 2012, pp. 143–152. ACM, New York (2012)Google Scholar
  37. 37.
    Santos, J.L., Verbert, K., Govaerts, S., Duval, E.: Addressing learner issues with StepUp!: an evaluation. In: Proceedings of LAK 2013, pp. 14–22. ACM (2013)Google Scholar
  38. 38.
    Santos, J.L., Verbert, K., Klerkx, J., Ternier, S., Charleer, S., Duval, E.: Tracking data in open learning environments. J. Univ. Comput. Sci. 21(7), 976–996 (2015)Google Scholar
  39. 39.
    Shneiderman, B.: The eyes have it: a task by data type taxonomy for information visualizations. In: IEEE Symposium on Visual Languages, pp. 336–343. IEEE (1996)Google Scholar
  40. 40.
    Siemens, G., Long, P.: Penetrating the Fog: Analytics in Learning and Education, vol. 46, pp. 30–32. EDUCAUSE, Boulder (2011)Google Scholar
  41. 41.
    Silius, K., Tervakari, A.-M., Kailanto, M.: Visualizations of user data in a social media enhanced web-based environment in higher education. In: Global Engineering Education Conference, pp. 893–899. IEEE (2013)Google Scholar
  42. 42.
    Buckingham Shum, S., Ferguson, R.: Social learning analytics. J. Educ. Technol. Soc. 15(3), 3–26 (2012)Google Scholar
  43. 43.
    Suthers, D.D.: Computer-supported collaborative learning. In: Seel, N.M. (ed.) Encyclopedia of the Sciences of Learning, pp. 719–722. Springer, Heidelberg (2012)Google Scholar
  44. 44.
    Verbert, K., Duval, E., Klerkx, J., Govaerts, S., Santos, J.L.: Learning analytics dashboard applications. Am. Behav. Sci. 57(10), 1500–1509 (2013)CrossRefGoogle Scholar
  45. 45.
    Verbert, K., Govaerts, S., Duval, E., Santos, J., Van Assche, F., Parra, G., Klerkx, J.: Learning dashboards: an overview and future research opportunities. Pers. Ubiquit. Comput. 18(6), 1499–1514 (2014)Google Scholar
  46. 46.
    Waters, S.: The Current State of Educational Blogging (2016). https://www.theedublogger.com/2016/01/15/educational-blogging-2015/. Accessed 29 Mar 2016
  47. 47.
    Wise, A.F.: Designing pedagogical interventions to support student use of learning analytics. In: Proceedings of LAK 2014, pp. 203–211. ACM (2014)Google Scholar

Copyright information

© Springer International Publishing Switzerland 2016

Authors and Affiliations

  • Sven Charleer
    • 1
  • Joris Klerkx
    • 1
  • Erik Duval
    • 1
  • Tinne De Laet
    • 2
  • Katrien Verbert
    • 1
  1. 1.Department of Computer ScienceKU LeuvenLeuvenBelgium
  2. 2.Tutorial Services of Engineering ScienceKU LeuvenLeuvenBelgium

Personalised recommendations