Skip to main content
SpringerLink
Log in

Effects of learning analytics dashboard: analyzing the relations among dashboard utilization, satisfaction, and learning achievement

  • Published:
Asia Pacific Education Review Aims and scope Submit manuscript

Abstract

The learning analytics dashboard (LAD) is a newly developed learning support tool for virtual classrooms that is believed to allow students to review their online learning behavior patterns intuitively through the provision of visual information. The purpose of this study was to empirically validate the effects of LAD. An experimental study was conducted with a dashboard treatment group and a control group. The researchers developed a LAD and evaluated its effectiveness on the sample of 151 college students at a private university located in Korea, who were taking the online course titled “Management Statistics” in the first semester of 2014. The following results were obtained. First, the students who received dashboard treatment presented higher final score than those who did not. Second, the dashboard usage frequency, as measured by the number of times the dashboard was opened, did not have a significant impact on learning achievement. However, a slightly positive correlation between satisfaction with LAD and learning achievement was observed. Further analysis indicated that learners who used the dashboard only a few times showed relatively high satisfaction with LAD. On the other hand, high academic achievers who opened LAD relatively frequently showed lower satisfaction with dashboard. The results guide that LAD should be revised in a way to motivate learners consistently and support learners who have different academic achievement levels. The study discusses the further research tasks in terms of LAD development as an effective and personalized feedback tool to improve learners’ academic achievement.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8

Similar content being viewed by others

References

  • Ali, L., Hatala, M., Gašević, D., & Jovanović, J. (2012). A qualitative evaluation of evolution of a learning analytics tool. Computers & Education, 58(1), 470–489.

    Article  Google Scholar 

  • Arnold, K. E., & Pistilli, M. D. (2012, April). Course signals at Purdue: Using learning analytics to increase student success. In Proceedings of the 2nd international conference on learning analytics and knowledge (pp. 267–270). ACM.

  • Brown, M. (2011). Learning analytics: The coming third wave. EDUCAUSE Learning initiative.

  • Butler, D. L., & Winne, P. H. (1995). Feedback and self-regulated learning: A theoretical synthesis. Review of Educational Research, 65(3), 245–281.

    Article  Google Scholar 

  • Card, S. K., Mackinlay, J. D., & Shneiderman, B. (Eds.). (1999). Readings in information visualization: Using vision to think. Burlington: Morgan Kaufmann.

    Google Scholar 

  • Chatti, M. A., Dyckhoff, A. L., Schroeder, U., & Thüs, H. (2012). A reference model for learning analytics. International Journal of Technology Enhanced Learning, 4(5), 318–331.

    Article  Google Scholar 

  • Chen, G. D., Chang, C. K., & Wang, C. Y. (2008). Ubiquitous learning website: Scaffold learners by mobile devices with information-aware techniques. Computers & Education, 50(1), 77–90.

    Article  Google Scholar 

  • Corrin, L., & de Barba, P. (2014). Exploring students’ interpretation of feedback delivered through learning analytics dashboards. In Proceedings of the ascilite 2014 conference.

  • Dashboard. n.d. In Wikipedia, Retrieved February 22, 2015, from http://en.wikipedia.org/w/index.php?title=Dashboard&oldid=642517668.

  • Dollár, A., & Steif, P. S. (2012). Web-based statics course with learning dashboard for instructors. In Proceedings of computers and advanced technology in education (CATE 2012), Napoli, Italy.

  • Duval, E. (2011). Attention please!: Learning analytics for visualization and recommendation. Paper presented at the 1st International Conference on Learning Analytics and Knowledge, Alberta, Canada.

  • Elias, T. (2011). Learning analytics: Definitions, processes and potential. Retrieved February 10, 2012, from http://learninganalytics.net/LearningAnalyticsDefinitionsProcessesPotential.pdf.

  • Ferguson, R. (2012). Learning analytics: Drivers, developments and challenges. International Journal of Technology Enhanced Learning, 4(5), 304–317.

    Article  Google Scholar 

  • Few, S. (2013). Information dashboard design: Displaying data for at-a-glance monitoring. Hillsdale: Analytics Press.

    Google Scholar 

  • Govaerts, S., Verbert, K., Duval, E., & Pardo, A. (2012). The student activity meter for awareness and self-reflection. In Proceedings of the 2012 ACM annual conference on human factors in computing systems extended abstracts (pp. 869–884). New York, NY: ACM.

  • Grann, J., & Bushway, D. (2014). Competency map: Visualizing student learning to promote student success. Paper presented at the Fourth International Conference on Learning Analytics and Knowledge, Indianapolis, IN.

  • Ha, K., Jo, I., & Lim, S. (2013). Usability study of visual dashboard as learning analytics interventions. Paper presented at the International Conference of Educational Technology, Sejong University.

  • Jo, I., Heo, H., Lim, K., Choi, J., & Noh, J. (2013a). Usability study of middle school english digital textbook: A stimulated recall approach. Educational Technology International, 14(1), 109–136.

    Google Scholar 

  • Jo, I., & Kim, Y. (2013a). Impact of learner’s time management strategies on achievement in an e-learning environment: A learning analytics approach. Journal of Educational Information and Media Research, 19(1), 83–107.

    Google Scholar 

  • Jo, I., & Kim, Y. (2013b). Impact of learner’s time management strategies on achievement in an e-learning environment: A learning analytics approach. The Journal of Educational Information and Media, 19(1), 83–107.

    Google Scholar 

  • Jo, I., Kim, D., & Yoon, M. (2014). Analyzing the log patterns of adult learners in LMS using learning analytics. Paper presented at the 4th International Conference on Learning Analytics Knowledge, Indianapolis, IN.

  • Jo, I., Lim, S., Park, Y., & Ha, K. (2014). Tracking students’ eye-movements on visual dashboard presenting their online learning behavior patterns. Paper presented at the International Conference on Smart Environments (ICSLE 2014), Hong Kong, China.

  • Jo, I., Yoon, M., Kang, M., & Ha, K. (2013). Examining relationships among time management strategy, regularity of learning, interval and learning performance in corporate e-learning environment: An empirical study. Paper presented at the Journal of The Korean Society for Educational Technology, 144-161.

  • Jo, I., & Yu, T. (2014). Educational technology approach toward learning analytics: Relationship between student online behavior and learning performance in higher education. Paper presented at the 4th International Conference on Learning Analytics Knowledge, Indianapolis, Indiana, USA.

  • Kang, S., Park, Y., & Jo, I. (2013). Student’s perception on learning analytics dashboard (LAD) presenting online activities in LMS. Paper presented at the International Conference of Educational Technology, Sejong University.

  • Kobsa, E., Dimitrova, V., & Boyle, R. (2005). Using student and group models to support teachers in web-based distance education. In Proceedings of the 10th international conference on user modeling (pp. 124–133). Berlin: Springer

  • Mazzola, L., & Mazza, R. (2011). Visualizing learner models through data aggregation: A test case. Paper presented at The Red-Conference, Rethinking Education in the Knowledge Society, Switzerland.

  • Park, Y., & Jo, I. (2014). Design and application of visual dashboard based on learning analytics. The Journal of Educational Information and Media, 20(2), 191–216.

    Google Scholar 

  • Park, Y., & Jo, I. (2015). Development of learning analytics dashboard supporting learning performance. Journal of Universal Computing Science, 21(1), 110–133.

    Google Scholar 

  • Pistilli, M. D., & Arnold, K. E. (2010). In practice: Purdue Signals: Mining real-time academic data to enhance student success. About Campus, 15(3), 22–24.

    Article  Google Scholar 

  • Podgorelec, V., & Kuhar, S. (2011a). Taking advantage of education data: Advanced data analysis and reporting in virtual learning environments. Elektronika ir Elektrotechnika, 114(8), 111–116.

    Article  Google Scholar 

  • Podgorelec, V., & Kuhar, S. (2011b). Taking advantage of education data: Advanced data analysis and reporting in virtual learning environments. Electronics and Electrical Engineering, 114, 111–116.

    Google Scholar 

  • Roscoe, R., Segedy, J., Sulcer, B., Jeong, H., & Biswas, G. (2013). Shallow strategy development in a teachable agent environment designed to support self-regulated learning. Computers & Education, 62, 286–297.

    Article  Google Scholar 

  • Santos, J. L., Govaerts, S., Verbert, K., & Duval, E. (2012). Goal-oriented visualizations of activity tracking: A case study with engineering students. In Proceedings of the 2nd international conference on learning analytics and knowledge (pp. 143–152). ACM.

  • Scheuer, O., & Zinn, C. (2007). How did the e-learning session go? The Student Inspector. Frontiers in Artificial Intelligence and Applications, 158, 487.

    Google Scholar 

  • Siemens, G. (2013). Learning analytics: The emergence of a discipline. American Behavioral Scientist, 57(10), 1380–1400.

    Article  Google Scholar 

  • Siemens, G., & d Baker, R. S. (2012). Learning analytics and educational data mining: Towards communication and collaboration. In Proceedings of the 2nd international conference on learning analytics and knowledge (pp. 252–254). ACM.

  • Siemens, G., Gasevic, D., Haythornthwaite, C., Dawson, S., Shum, S. B., Ferguson, R., et al. (2011). Open Learning Analytics: An integrated & modularized platform. Doctoral dissertation, Open University Press.

  • Smith, S. W., Daunic, A. P., & Taylor, G. G. (2007). Treatment fidelity in applied educational research: Expanding the adoption and application of measures to ensure evidence-based practice. Education and Treatment of Children, 30(4), 121–134.

    Article  Google Scholar 

  • Soller, A., Martínez, A., Jermann, P., & Muehlenbrock, M. (2005). From mirroring to guiding: A review of state of the art technology for supporting collaborative learning. International Journal of Artificial Intelligence in Education, 15(4), 261–290.

    Google Scholar 

  • Verbert, K., Duval, E., Klerkx, J., Govaerts, S., & Santos, J. L. (2013). Learning analytics dashboard applications. American Behavioral Scientist., 57(10), 1500–1509.

    Article  Google Scholar 

  • Wise, A. F. (2014, March). Designing pedagogical interventions to support student use of learning analytics. In Proceedings of the Fourth International Conference on Learning Analytics and Knowledge (pp. 203–211). ACM.

  • Yoo, Y., Lee, H., & Park, Y. (2014). Educational dashboards for smart learning: Review of case studies. Paper presented at the International Conference on Smart Environments (ICSLE 2014), Hong Kong, China.

  • Yu, T., & Jo, I. (2014). Educational technology approach toward learning analytics: Relationship between student online behavior and learning performance in higher education. Poster presented at the 4th International Conference on Learning Analytics Knowledge, Indianapolis, IN.

  • Zimmerman, B. J., & Schunk, D. H. (Eds.). (2001). Self-regulated learning and academic achievement: Theoretical perspectives. London: Routledge.

    Google Scholar 

Download references

Acknowledgments

We appreciate the experts who participated in the review of our initial instrument. This work was supported by the Ministry of Education of the Republic of Korea and the National Research Foundation of Korea (NRF-2015S1A5B6036244).

Author information

Authors and Affiliations

  1. Ewha Womans University, Seoul, South Korea

    Jeonghyun Kim, Il-Hyun Jo & Yeonjeong Park

Authors
  1. Jeonghyun Kim
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Il-Hyun Jo
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Yeonjeong Park
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Il-Hyun Jo.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Kim, J., Jo, IH. & Park, Y. Effects of learning analytics dashboard: analyzing the relations among dashboard utilization, satisfaction, and learning achievement. Asia Pacific Educ. Rev. 17, 13–24 (2016). https://doi.org/10.1007/s12564-015-9403-8

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s12564-015-9403-8

Keywords

Search

Navigation