Skip to main content
SpringerLink
Log in

The design, development, and implementation of student-facing learning analytics dashboards

  • Published:
Journal of Computing in Higher Education Aims and scope Submit manuscript

Abstract

We have designed, developed, and implemented a student-facing learning analytics dashboard in order to support students as they learn in online environments. There are two separate dashboards in our system: a content recommender dashboard and a skills recommender dashboard. The content recommender helps students identify gaps in their content knowledge; the skills recommender helps students improve their metacognitive strategies. We discuss the technical requirements needed to develop a real-time student dashboard as well as report our inquiry into the functionality students want in a dashboard. The dashboards were evaluated with focus groups and a perceptions survey. Students were positive in their perceptions of the dashboards and 79% of the students that used the dashboards found them user-friendly, engaging, useful, and informative. One challenge encountered was low student use of the dashboard. Only 25% of students used the dashboard multiple times, despite favorable student perceptions of the dashboard. Additional research should examine how to motivate and support students to engage with dashboard feedback in online environments.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13

Similar content being viewed by others

References

  • Allen, I. E., & Seaman, J. (2014). Tracking online education in the United States, 1–45. Retrieved from http://www.onlinelearningsurvey.com/reports/gradechange.pdf. Accessed 18 Nov 2016.

  • Anaya, A. R., Luque, M., & Peinado, M. (2016). A visual recommender tool in a collaborative learning experience. Expert Systems with Applications, 45, 248–259.

    Article  Google Scholar 

  • Arnold, K. E. (2010). Signals: Applying academic analytics. Educause Quarterly, 33(1), n1.

    Google Scholar 

  • Arnold, K. E., & Pistilli, M. D. (2012, April). Course signals at Purdue: Using learning analytics to increase student success. In Proceedings of the 2nd international conference on learning analytics and knowledge (pp. 267–270). ACM.

  • Baker, R. S., Corbett, A. T., Koedinger, K. R., & Wagner, A. Z. (2004, April). Off-task behavior in the cognitive tutor classroom: When students game the system. In Proceedings of the SIGCHI conference on human factors in computing systems (pp. 383–390). ACM.

  • Bangert-Drowns, R. L., Kulik, C. L. C., Kulik, J. A., & Morgan, M. (1991). The instructional effect of feedback in test-like events. Review of Educational Research, 61(2), 213–238.

    Article  Google Scholar 

  • Bekele, T. A. (2010). Motivation and satisfaction in internet-supported learning environments: A review. Educational Technology and Society, 13(2), 116–127.

    Google Scholar 

  • Bodily, R., & Verbert, K. (2017, March). Trends and issues in student-facing learning analytics reporting systems research. In Proceedings of the seventh international learning analytics & knowledge conference (pp. 309–318). ACM.

  • Charleer, S., Klerkx, J., Odriozola, S., Luis, J., & Duval, E. (2013, December). Improving awareness and reflection through collaborative, interactive visualizations of badges. In ARTEL13: Proceedings of the 3rd workshop on awareness and reflection in technology-enhanced learning (vol. 1103, pp. 69–81). CEUR-WS.

  • Corrin, L., & de Barba, P. (2015, March). How do students interpret feedback delivered via dashboards? In Proceedings of the fifth international conference on learning analytics and knowledge (pp. 430–431). ACM.

  • Danado, J., Davies, M., Ricca, P., & Fensel, A. (2010, September). An authoring tool for user generated mobile services. In Future internet symposium (pp. 118–127). Berlin: Springer.

    Chapter  Google Scholar 

  • Doorn, D. J., Janssen, S., & O’Brien, M. (2010). Student attitudes and approaches to online homework. International Journal for the Scholarship of Teaching and Learning, 4(1), 5.

    Article  Google Scholar 

  • Elias, T. (2011). Learning analytics. Learning Analytics: Definitions, Processes and Potential. Retrieved June 2015, from http://learninganalytics.net/LearningAnalyticsDefinitionsProcessesPotential.pdf.

  • Feild, J. (2015, June). Improving student performance using nudge analytics. In Proceedings of the 8th international conference on educational data mining (pp. 464–467).

  • Ferguson, R., & Shum, S. B. (2012, April). Social learning analytics: Five approaches. In Proceedings of the 2nd international conference on learning analytics and knowledge (pp. 23–33). ACM.

  • Few, S. (2006). Information dashboard design: The effective visual communication of data. Sebastopol, CA: O’Reilly Media.

    Google Scholar 

  • Few, S. (2013). Information dashboard design: Displaying data for at-a-glance monitoring. Burlingame, CA: Analytics Press.

    Google Scholar 

  • Garrison, D. R. (2003). Cognitive presence for effective asynchronous online learning: The role of reflective inquiry, self-direction and metacognition. Elements of Quality Online Education: Practice and Direction, 4(1), 47–58.

    Google Scholar 

  • Govaerts, S., Verbert, K., Duval, E., & Pardo, A. (2012, May). The student activity meter for awareness and self-reflection. In CHI’12 extended abstracts on human factors in computing systems (pp. 869–884). ACM.

  • Govaerts, S., Verbert, K., Klerkx, J., & Duval, E. (2010, December). Visualizing activities for self-reflection and awareness. In International conference on web-based learning (pp. 91–100). Berlin: Springer.

    Chapter  Google Scholar 

  • Grann, J., & Bushway, D. (2014, March). Competency map: Visualizing student learning to promote student success. In Proceedings of the fourth international conference on learning analytics and knowledge (pp. 168–172). ACM.

  • Greller, W., & Drachsler, H. (2012). Translating learning into numbers: A generic framework for learning analytics. Educational Technology and Society, 15(3), 42–57.

    Google Scholar 

  • Hacker, D. J., Dunlosky, J., & Graesser, A. C. (Eds.). (1998). Metacognition in educational theory and practice. New York: Routledge.

    Google Scholar 

  • Harrer, A. (2015, July). A design proposal for learner-centered visualisations of learning analytics in collaborative scenarios. In 2015 IEEE 15th international conference on advanced learning technologies (ICALT) (pp. 208–210). IEEE.

  • Harrer, A., & Göhnert, T. (2015, March). Integrated representations and small data: Towards contextualized and embedded analytics tools for learners. In Proceedings of the fifth international conference on learning analytics and knowledge (pp. 406–407). ACM.

  • Hatziapostolou, T., & Paraskakis, I. (2010). Enhancing the impact of formative feedback on student learning through an online feedback system. Electronic Journal of e-Learning, 8(2), 111–122.

    Google Scholar 

  • Holman, C., Aguilar, S., & Fishman, B. (2013, April). GradeCraft: What can we learn from a game-inspired learning management system? In Proceedings of the third international conference on learning analytics and knowledge (pp. 260–264). ACM.

  • Huizing, A., & Cavanagh, M. (2011). Planting contemporary practice theory in the garden of information science. Information Research Journal, 16(4), 497.

    Google Scholar 

  • Iandoli, L., Quinto, I., De Liddo, A., & Shum, S. B. (2014). Socially augmented argumentation tools: Rationale, design and evaluation of a debate dashboard. International Journal of Human-Computer Studies, 72(3), 298–319.

    Article  Google Scholar 

  • Jeon, J. H., Yeon, J., Lee, S. G., & Seo, J. (2014, January). Exploratory visualization of smartphone-based life-logging data using Smart Reality Testbed. In 2014 International conference on big data and smart computing (BIGCOMP) (pp. 29–33). IEEE.

  • Jones, A., & Issroff, K. (2007). Learning technologies: Affective and social issues. In G. Conole & M. Oliver (Eds.), Contemporary perspectives in e-learning research: Themes, methods and impact on practice (pp. 190–202). London: Routledge.

    Google Scholar 

  • Jugo, I., Kovačić, B., & Slavuj, V. (2014, May). Using data mining for learning path recommendation and visualization in an intelligent tutoring system. In 2014 37th international convention on information and communication technology, electronics and microelectronics (MIPRO) (pp. 924–928). IEEE.

  • Kerly, A., Ellis, R., & Bull, S. (2008). CALMsystem: A conversational agent for learner modelling. Knowledge-Based Systems, 21(3), 238–246. https://doi.org/10.1016/j.knosys.2007.11.015.

    Article  Google Scholar 

  • Kim, J., Jo, I. H., & Park, Y. (2016). Effects of learning analytics dashboard: Analyzing the relations among dashboard utilization, satisfaction, and learning achievement. Asia Pacific Education Review, 17(1), 13–24.

    Article  Google Scholar 

  • Kortemeyer, G. (2015). An empirical study of the effect of granting multiple tries for online homework. American Journal of Physics, 83(7), 646–653.

    Article  Google Scholar 

  • Kuosa, K., Distante, D., Tervakari, A., Cerulo, L., Fernández, A., Koro, J., et al. (2016). Interactive visualization tools to improve learning and teaching in online learning environments. International Journal of Distance Education Technologies (IJDET), 14(1), 1–21.

    Article  Google Scholar 

  • Laffey, J. M., Amelung, C., & Goggins, S. (2014). Using analytics for activity awareness in learning systems. International Journal of Designs for Learning, 5(2), 101.

    Article  Google Scholar 

  • Lent, R. W., Brown, S. D., & Larkin, K. C. (1984). Relation of self-efficacy expectations to academic achievement and persistence. Journal of Counseling Psychology, 31(3), 356.

    Article  Google Scholar 

  • Manso-Vázquez, M., & Llamas-Nistal, M. (2015). Proposal of a learning organization tool with support for metacognition. IEEE Revista Iberoamericana de Tecnologias del Aprendizaje, 10(2), 35–42.

    Article  Google Scholar 

  • McAuley, J., O’Connor, A., & Lewis, D. (2012, April). Exploring reflection in online communities. In Proceedings of the 2nd international conference on learning analytics and knowledge (pp. 102–110). ACM.

  • Melero, J., Hernández-Leo, D., Sun, J., Santos, P., & Blat, J. (2015). How was the activity? A visualization support for a case of location-based learning design. British Journal of Educational Technology, 46(2), 317–329.

    Article  Google Scholar 

  • Millman, J. (1989). If at first you don’t succeed setting passing scores when more than one attempt is permitted. Educational Researcher, 18(6), 5–9.

    Google Scholar 

  • Muldner, K., Wixon, M., Rai, D., Burleson, W., Woolf, B., & Arroyo, I. (2015). Exploring the impact of a learning dashboard on student affect. In  C. Conati, N. Heffernan, A. Mitrovic, M. F. Verdejo (Eds.), International conference on artificial intelligence in education (pp. 307–317). Berlin: Springer.

    Chapter  Google Scholar 

  • Odriozola, S., Luis, J., Verbert, K., Govaerts, S., & Duval, E. (2011, August). Visualizing PLE usage. In Proceedings of EFEPLE11 1st workshop on exploring the fitness and evolvability of personal learning environments (Vol. 773, pp. 34–38). CEUR WS.

  • Olmos, M. M., & Corrin, L. (2012). Learning analytics: A case study of the process of design of visualizations. Journal of Asynchronous Learning Networks, 16(3), 39–49.

    Google Scholar 

  • Ott, C., Robins, A., Haden, P., & Shephard, K. (2015). Illustrating performance indicators and course characteristics to support students’ self-regulated learning in CS1. Computer Science Education, 25(2), 174–198.

    Article  Google Scholar 

  • Park, Y., & Jo, I. H. (2015). Development of the learning analytics dashboard to support students’ learning performance. Journal of Universal Computer Science, 21(1), 110–133.

    Google Scholar 

  • Resnick, P., & Varian, H. R. (1997). Recommender systems. Communications of the ACM, 40(3), 56–58.

    Article  Google Scholar 

  • Ruipérez-Valiente, J. A., Muñoz-Merino, P. J., & Kloos, C. D. (2013, November). An architecture for extending the learning analytics support in the Khan Academy framework. In Proceedings of the first international conference on technological ecosystem for enhancing multiculturality (pp. 277–284). ACM.

  • Ruipérez-Valiente, J. A., Muñoz-Merino, P. J., Leony, D., & Kloos, C. D. (2015). ALAS-KA: A learning analytics extension for better understanding the learning process in the Khan Academy platform. Computers in Human Behavior, 47, 139–148.

    Article  Google Scholar 

  • Rydberg, E. (2011). Visualizing the software development process by analyzing software engineering data. Stockholm: Royal Institute of Technology.

    Google Scholar 

  • Santos, J. L., Govaerts, S., Verbert, K., & Duval, E. (2012, April). Goal-oriented visualizations of activity tracking: A case study with engineering students. In Proceedings of the 2nd international conference on learning analytics and knowledge (pp. 143–152). ACM.

  • Santos, J. L., Verbert, K., & Duval, E. (2012b). Empowering students to reflect on their activity with StepUp!: Two case studies with engineering students (p. 73). Editorial: Awareness and Reflection in Technology Enhanced Learning.

    Google Scholar 

  • Santos, J. L., Verbert, K., Govaerts, S., & Duval, E. (2013, April). Addressing learner issues with StepUp!: An evaluation. In Proceedings of the third international conference on learning analytics and knowledge (pp. 14–22). ACM.

  • Santos, J., Verbert, K., Klerkx, J., Duval, E., Charleer, S., & Ternier, S. (2015). Tracking data in open learning environments. Journal of Universal Computer Science, 21(7), 976–996.

    Google Scholar 

  • Schmitz, H. C., Scheffel, M., Friedrich, M., Jahn, M., Niemann, K., & Wolpers, M. (2009, September). CAMera for PLE. In European conference on technology enhanced learning (pp. 507–520). Berlin: Springer.

    Google Scholar 

  • Schwendimann, B. A., Rodriguez-Triana, M. J., Vozniuk, A., Prieto, L. P., Boroujeni, M. S., Holzer, A., et al. (2017). Perceiving learning at a glance: A systematic literature review of learning dashboard research. IEEE Transactions on Learning Technologies, 10(1), 30–41.

    Article  Google Scholar 

  • Shute, V. J. (2008). Focus on formative feedback. Review of Educational Research, 78(1), 153–189.

    Article  Google Scholar 

  • Siemens, G. (2010, July 22). 1st International conference on learning analytics and knowledge 2011. Retrieved March 30, 2016, from https://tekri.athabascau.ca/analytics/.

  • Silius, K., Miilumäki, T., Huhtamäki, J., Tebest, T., Meriläinen, J., & Pohjolainen, S. (2010). Students’ motivations for social media enhanced studying and learning. Knowledge Management and E-Learning: An International Journal, 2(1), 51–67.

    Google Scholar 

  • Silius, K., Tervakari, A. M., & Kailanto, M. (2013, March). Visualizations of user data in a social media enhanced web-based environment in higher education. In 2013 IEEE global engineering education conference (EDUCON) (pp. 893–899). IEEE.

  • Stallings, J. (1980). Allocated academic learning time revisited, or beyond time on task. Educational researcher, 9(11), 11–16.

    Article  Google Scholar 

  • Steel, P. (2007). The nature of procrastination: A meta-analytic and theoretical review of quintessential self-regulatory failure. Psychological Bulletin, 133(1), 65–94. https://doi.org/10.1037/0033-2909.133.1.65.

    Article  Google Scholar 

  • Tervakari, A. M., Silius, K., Koro, J., Paukkeri, J., & Pirttila, O. (2014, April). Usefulness of information visualizations based on educational data. In 2014 IEEE global engineering education conference (EDUCON) (pp. 142–151). IEEE.

  • Verbert, K., Duval, E., Klerkx, J., Govaerts, S., & Santos, J. L. (2013). Learning analytics dashboard applications. American Behavioral Scientist, 57(10), 1–10. https://doi.org/10.1177/0002764213479363.

    Article  Google Scholar 

  • Verbert, K., Govaerts, S., Duval, E., Santos, J. L., Van Assche, F., Parra, G., et al. (2014). Learning dashboards: An overview and future research opportunities. Personal and Ubiquitous Computing, 18(6), 1499–1514. https://doi.org/10.1007/s00779-013-0751-2.

    Article  Google Scholar 

  • Wilson, B. G. (2013). A practice-centered approach to instructional design. In Learning, problem solving, and mind tools: Essays in honor of David H. Jonassen, (pp. 35–54).

  • Yoo, Y., Lee, H., Jo, I., & Park, Y. (2015). Educational dashboards for smart learning: Review of case studies. Emerging Issues in Smart Learning. https://doi.org/10.1007/978-3-662-44188-6.

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Brigham Young University, 150 MCKB, Provo, UT, 84602, USA

    Robert Bodily, Tarah K. Ikahihifo, Benjamin Mackley & Charles R. Graham

Authors
  1. Robert Bodily
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Tarah K. Ikahihifo
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Benjamin Mackley
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Charles R. Graham
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Robert Bodily.

Ethics declarations

Conflict of interest

The authors declare that they have no conflict of interest.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Bodily, R., Ikahihifo, T.K., Mackley, B. et al. The design, development, and implementation of student-facing learning analytics dashboards. J Comput High Educ 30, 572–598 (2018). https://doi.org/10.1007/s12528-018-9186-0

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s12528-018-9186-0

Keywords

Search

Navigation