Advertisement

Asia Pacific Education Review

, Volume 17, Issue 1, pp 13–24 | Cite as

Effects of learning analytics dashboard: analyzing the relations among dashboard utilization, satisfaction, and learning achievement

  • Jeonghyun Kim
  • Il-Hyun JoEmail author
  • Yeonjeong Park
Article

Abstract

The learning analytics dashboard (LAD) is a newly developed learning support tool for virtual classrooms that is believed to allow students to review their online learning behavior patterns intuitively through the provision of visual information. The purpose of this study was to empirically validate the effects of LAD. An experimental study was conducted with a dashboard treatment group and a control group. The researchers developed a LAD and evaluated its effectiveness on the sample of 151 college students at a private university located in Korea, who were taking the online course titled “Management Statistics” in the first semester of 2014. The following results were obtained. First, the students who received dashboard treatment presented higher final score than those who did not. Second, the dashboard usage frequency, as measured by the number of times the dashboard was opened, did not have a significant impact on learning achievement. However, a slightly positive correlation between satisfaction with LAD and learning achievement was observed. Further analysis indicated that learners who used the dashboard only a few times showed relatively high satisfaction with LAD. On the other hand, high academic achievers who opened LAD relatively frequently showed lower satisfaction with dashboard. The results guide that LAD should be revised in a way to motivate learners consistently and support learners who have different academic achievement levels. The study discusses the further research tasks in terms of LAD development as an effective and personalized feedback tool to improve learners’ academic achievement.

Keywords

Learning analytics Dashboard Satisfaction Learning achievement 

Notes

Acknowledgments

We appreciate the experts who participated in the review of our initial instrument. This work was supported by the Ministry of Education of the Republic of Korea and the National Research Foundation of Korea (NRF-2015S1A5B6036244).

References

  1. Ali, L., Hatala, M., Gašević, D., & Jovanović, J. (2012). A qualitative evaluation of evolution of a learning analytics tool. Computers & Education, 58(1), 470–489.CrossRefGoogle Scholar
  2. Arnold, K. E., & Pistilli, M. D. (2012, April). Course signals at Purdue: Using learning analytics to increase student success. In Proceedings of the 2nd international conference on learning analytics and knowledge (pp. 267–270). ACM.Google Scholar
  3. Brown, M. (2011). Learning analytics: The coming third wave. EDUCAUSE Learning initiative.Google Scholar
  4. Butler, D. L., & Winne, P. H. (1995). Feedback and self-regulated learning: A theoretical synthesis. Review of Educational Research, 65(3), 245–281.CrossRefGoogle Scholar
  5. Card, S. K., Mackinlay, J. D., & Shneiderman, B. (Eds.). (1999). Readings in information visualization: Using vision to think. Burlington: Morgan Kaufmann.Google Scholar
  6. Chatti, M. A., Dyckhoff, A. L., Schroeder, U., & Thüs, H. (2012). A reference model for learning analytics. International Journal of Technology Enhanced Learning, 4(5), 318–331.CrossRefGoogle Scholar
  7. Chen, G. D., Chang, C. K., & Wang, C. Y. (2008). Ubiquitous learning website: Scaffold learners by mobile devices with information-aware techniques. Computers & Education, 50(1), 77–90.CrossRefGoogle Scholar
  8. Corrin, L., & de Barba, P. (2014). Exploring students’ interpretation of feedback delivered through learning analytics dashboards. In Proceedings of the ascilite 2014 conference.Google Scholar
  9. Dashboard. n.d. In Wikipedia, Retrieved February 22, 2015, from http://en.wikipedia.org/w/index.php?title=Dashboard&oldid=642517668.
  10. Dollár, A., & Steif, P. S. (2012). Web-based statics course with learning dashboard for instructors. In Proceedings of computers and advanced technology in education (CATE 2012), Napoli, Italy.Google Scholar
  11. Duval, E. (2011). Attention please!: Learning analytics for visualization and recommendation. Paper presented at the 1st International Conference on Learning Analytics and Knowledge, Alberta, Canada.Google Scholar
  12. Elias, T. (2011). Learning analytics: Definitions, processes and potential. Retrieved February 10, 2012, from http://learninganalytics.net/LearningAnalyticsDefinitionsProcessesPotential.pdf.
  13. Ferguson, R. (2012). Learning analytics: Drivers, developments and challenges. International Journal of Technology Enhanced Learning, 4(5), 304–317.CrossRefGoogle Scholar
  14. Few, S. (2013). Information dashboard design: Displaying data for at-a-glance monitoring. Hillsdale: Analytics Press.Google Scholar
  15. Govaerts, S., Verbert, K., Duval, E., & Pardo, A. (2012). The student activity meter for awareness and self-reflection. In Proceedings of the 2012 ACM annual conference on human factors in computing systems extended abstracts (pp. 869–884). New York, NY: ACM.Google Scholar
  16. Grann, J., & Bushway, D. (2014). Competency map: Visualizing student learning to promote student success. Paper presented at the Fourth International Conference on Learning Analytics and Knowledge, Indianapolis, IN.Google Scholar
  17. Ha, K., Jo, I., & Lim, S. (2013). Usability study of visual dashboard as learning analytics interventions. Paper presented at the International Conference of Educational Technology, Sejong University.Google Scholar
  18. Jo, I., Heo, H., Lim, K., Choi, J., & Noh, J. (2013a). Usability study of middle school english digital textbook: A stimulated recall approach. Educational Technology International, 14(1), 109–136.Google Scholar
  19. Jo, I., & Kim, Y. (2013a). Impact of learner’s time management strategies on achievement in an e-learning environment: A learning analytics approach. Journal of Educational Information and Media Research, 19(1), 83–107.Google Scholar
  20. Jo, I., & Kim, Y. (2013b). Impact of learner’s time management strategies on achievement in an e-learning environment: A learning analytics approach. The Journal of Educational Information and Media, 19(1), 83–107.Google Scholar
  21. Jo, I., Kim, D., & Yoon, M. (2014). Analyzing the log patterns of adult learners in LMS using learning analytics. Paper presented at the 4th International Conference on Learning Analytics Knowledge, Indianapolis, IN.Google Scholar
  22. Jo, I., Lim, S., Park, Y., & Ha, K. (2014). Tracking students’ eye-movements on visual dashboard presenting their online learning behavior patterns. Paper presented at the International Conference on Smart Environments (ICSLE 2014), Hong Kong, China.Google Scholar
  23. Jo, I., Yoon, M., Kang, M., & Ha, K. (2013). Examining relationships among time management strategy, regularity of learning, interval and learning performance in corporate e-learning environment: An empirical study. Paper presented at the Journal of The Korean Society for Educational Technology, 144-161.Google Scholar
  24. Jo, I., & Yu, T. (2014). Educational technology approach toward learning analytics: Relationship between student online behavior and learning performance in higher education. Paper presented at the 4th International Conference on Learning Analytics Knowledge, Indianapolis, Indiana, USA.Google Scholar
  25. Kang, S., Park, Y., & Jo, I. (2013). Student’s perception on learning analytics dashboard (LAD) presenting online activities in LMS. Paper presented at the International Conference of Educational Technology, Sejong University.Google Scholar
  26. Kobsa, E., Dimitrova, V., & Boyle, R. (2005). Using student and group models to support teachers in web-based distance education. In Proceedings of the 10th international conference on user modeling (pp. 124–133). Berlin: SpringerGoogle Scholar
  27. Mazzola, L., & Mazza, R. (2011). Visualizing learner models through data aggregation: A test case. Paper presented at The Red-Conference, Rethinking Education in the Knowledge Society, Switzerland.Google Scholar
  28. Park, Y., & Jo, I. (2014). Design and application of visual dashboard based on learning analytics. The Journal of Educational Information and Media, 20(2), 191–216.Google Scholar
  29. Park, Y., & Jo, I. (2015). Development of learning analytics dashboard supporting learning performance. Journal of Universal Computing Science, 21(1), 110–133.Google Scholar
  30. Pistilli, M. D., & Arnold, K. E. (2010). In practice: Purdue Signals: Mining real-time academic data to enhance student success. About Campus, 15(3), 22–24.CrossRefGoogle Scholar
  31. Podgorelec, V., & Kuhar, S. (2011a). Taking advantage of education data: Advanced data analysis and reporting in virtual learning environments. Elektronika ir Elektrotechnika, 114(8), 111–116.CrossRefGoogle Scholar
  32. Podgorelec, V., & Kuhar, S. (2011b). Taking advantage of education data: Advanced data analysis and reporting in virtual learning environments. Electronics and Electrical Engineering, 114, 111–116.Google Scholar
  33. Roscoe, R., Segedy, J., Sulcer, B., Jeong, H., & Biswas, G. (2013). Shallow strategy development in a teachable agent environment designed to support self-regulated learning. Computers & Education, 62, 286–297.CrossRefGoogle Scholar
  34. Santos, J. L., Govaerts, S., Verbert, K., & Duval, E. (2012). Goal-oriented visualizations of activity tracking: A case study with engineering students. In Proceedings of the 2nd international conference on learning analytics and knowledge (pp. 143–152). ACM.Google Scholar
  35. Scheuer, O., & Zinn, C. (2007). How did the e-learning session go? The Student Inspector. Frontiers in Artificial Intelligence and Applications, 158, 487.Google Scholar
  36. Siemens, G. (2013). Learning analytics: The emergence of a discipline. American Behavioral Scientist, 57(10), 1380–1400.CrossRefGoogle Scholar
  37. Siemens, G., & d Baker, R. S. (2012). Learning analytics and educational data mining: Towards communication and collaboration. In Proceedings of the 2nd international conference on learning analytics and knowledge (pp. 252–254). ACM.Google Scholar
  38. Siemens, G., Gasevic, D., Haythornthwaite, C., Dawson, S., Shum, S. B., Ferguson, R., et al. (2011). Open Learning Analytics: An integrated & modularized platform. Doctoral dissertation, Open University Press.Google Scholar
  39. Smith, S. W., Daunic, A. P., & Taylor, G. G. (2007). Treatment fidelity in applied educational research: Expanding the adoption and application of measures to ensure evidence-based practice. Education and Treatment of Children, 30(4), 121–134.CrossRefGoogle Scholar
  40. Soller, A., Martínez, A., Jermann, P., & Muehlenbrock, M. (2005). From mirroring to guiding: A review of state of the art technology for supporting collaborative learning. International Journal of Artificial Intelligence in Education, 15(4), 261–290.Google Scholar
  41. Verbert, K., Duval, E., Klerkx, J., Govaerts, S., & Santos, J. L. (2013). Learning analytics dashboard applications. American Behavioral Scientist., 57(10), 1500–1509.CrossRefGoogle Scholar
  42. Wise, A. F. (2014, March). Designing pedagogical interventions to support student use of learning analytics. In Proceedings of the Fourth International Conference on Learning Analytics and Knowledge (pp. 203–211). ACM.Google Scholar
  43. Yoo, Y., Lee, H., & Park, Y. (2014). Educational dashboards for smart learning: Review of case studies. Paper presented at the International Conference on Smart Environments (ICSLE 2014), Hong Kong, China.Google Scholar
  44. Yu, T., & Jo, I. (2014). Educational technology approach toward learning analytics: Relationship between student online behavior and learning performance in higher education. Poster presented at the 4th International Conference on Learning Analytics Knowledge, Indianapolis, IN.Google Scholar
  45. Zimmerman, B. J., & Schunk, D. H. (Eds.). (2001). Self-regulated learning and academic achievement: Theoretical perspectives. London: Routledge.Google Scholar

Copyright information

© Education Research Institute, Seoul National University, Seoul, Korea 2015

Authors and Affiliations

  1. 1.Ewha Womans UniversitySeoulSouth Korea

Personalised recommendations