Skip to main content

Advertisement

SpringerLink
Log in

The effect of learning analytics assisted recommendations and guidance feedback on students’ metacognitive awareness and academic achievements

  • Published:
Journal of Computing in Higher Education Aims and scope Submit manuscript

Abstract

This research examined the effect of learning analytics (LA) on students’ metacognitive awareness and academic achievement in an online learning environment. In this study, a mixed methods approach was used and applied as a quasi-experimental design. The results of LA were sent to students weekly in LA group (experimental group) via learning system. The LA was not sent to the non-LA group (control group). The quantitative data of the research were obtained through the metacognitive awareness inventory and academic achievement test used as pretest and posttest. A semi-structured student opinion form was used to find out the students' opinions about the learning environment and LA. The research was conducted within the scope of the Computing course and lasted 12 weeks. The results showed that LA assisted recommendations and guidance feedback had a significant effect on students' metacognitive awareness and academic achievement. The qualitative findings revealed that LA are useful for increasing participants’ metacognitive awareness and academic achievement. In line with the findings, various suggestions were made for instructors, instructional designers and researchers in the design and management of online learning environments.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2

Similar content being viewed by others

References

  • AECT (Association for Educational Communications and Technology). (2008). Definition. In A. Januszewski & M. Molenda (Eds.), Educational Technology: A Definition with Commentary. pp. 1–14. Lawrence Erlbaum Associates.

  • Akin, A., Abaci, R., & Cetin, B. (2007). The validity and reliability of the Turkish version of the metacognitive awareness inventory. Educational Sciences: Theory and Practice, 7(2), 671–678.

    Google Scholar 

  • Allen, I. E., & Seaman, J. (2010). Class differences: Online education in the United States, 2010. Retrieved August 10, 2019, from http://sloanconsortium.org/sites/default/files/class_differences.pdf

  • Arnold, K. E., & Pistilli, M. D. (2012). Course signals at Purdue: Using learning analytics to increase student success. In Proceedings of the 2nd International Conference on Learning Analytics and Knowledge. pp. 267–270. ACM.

  • Baker, L., & Brown, A. L. (1984). Metacognitive skills and reading. In P. D. Pearson, R. Barr, & M. L. Kamil (Vol. Eds.), Handbook of Reading Research: Vol. 1, pp. 353– 394. Psychology Press.

  • Cho, V., Cheng, T. E., & Lai, W. J. (2009). The role of perceived user-interface design in continued usage intention of self-paced e-learning tools. Computers and Education, 53(2), 216–227. https://doi.org/10.1016/j.compedu.2009.01.014

    Article  Google Scholar 

  • Clow, D. (2013). An overview of learning analytics. Teaching in Higher Education, 18(6), 683–695. https://doi.org/10.1080/13562517.2013.827653

    Article  Google Scholar 

  • Cohen, J. (1988). Statistical power analysis for the behavioral sciences (2nd ed.). Erlbaum.

    Google Scholar 

  • Derouin, R. E., Fritzsche, B. A., & Salas, E. (2005). E-learning in organizations. Journal of Management, 31(6), 920–940. https://doi.org/10.1177/0149206305279815

    Article  Google Scholar 

  • Duval, E. (2011). Attention please! Learning analytics for visualization and recommendation. In Proceedings of the 1st International Conference on Learning Analytics and Knowledge, LAK ’11. pp. 9–17. ACM.

  • Dyckhoff, A. L., Zielke, D., Bültmann, M., Chatti, M. A., & Schroeder, U. (2012). Design and implementation of a learning analytics toolkit for teachers. Journal of Educational Technology and Society, 15(3), 58–76.

    Google Scholar 

  • Dyckhoff, A. L., Lukarov, V., Muslim, A., Chatti, M. A., & Schroeder, U. (2013, April). Supporting action research with learning analytics. In Proceedings of the Third International Conference on Learning Analytics and Knowledge. pp. 220–229. ACM

  • Elias, T. (2011). Learning analytics: definitions, process and potential. Retrieved August 10, 2019, from http://learninganalytics.net/LearningAnalyticsDefinitionsProcessesPotential.pdf

  • Ferguson, R. (2012). Learning analytics: Drivers, developments and challenges. International Journal of Technology Enhanced Learning, 4(5/6), 304–317. https://doi.org/10.1504/IJTEL.2012.051816

    Article  Google Scholar 

  • Fernández-Gallego, B., Lama, M., Vidal, J. C., & Mucientes, M. (2013). Learning analytics framework for educational virtual worlds. Procedia Computer Science, 25, 443–447. https://doi.org/10.1016/j.procs.2013.11.056

    Article  Google Scholar 

  • Giannakos, M. N., Sampson, D. G., & Kidziński, Ł. (2016). Introduction to smart learning analytics: Foundations and developments in video-based learning. Smart Learning Environments, 3(12), 1–9. https://doi.org/10.1186/s40561-016-0034-2

    Article  Google Scholar 

  • Jivet, I., Scheffel, M., Drachsler, H., & Specht, M. (2017, September). Awareness is not enough: pitfalls of learning analytics dashboards in the educational practice. In European Conference on Technology Enhanced Learning. pp. 82–96. Springer.

  • Johnson, M. W., & Barnes, T. (2010). EDM visualization tool: Watching students learn. In Proceedings of the 3rd International Conference on Educational Data Mining. pp. 297–298.

  • Karaoglan Yilmaz, F. G., Olpak, Y. Z., & Yilmaz, R. (2018). The effect of the metacognitive support via pedagogical agent on self-regulation skills. Journal of Educational Computing Research, 56(2), 159–180. https://doi.org/10.1177/0735633117707696

    Article  Google Scholar 

  • Karaoglan Yilmaz, F. G. (2020). Modeling different variables in flipped classrooms supported with learning analytics feedback. Journal of Information and Communication Technologies, 2(1), 1–16.

    Google Scholar 

  • Karaoglan Yilmaz, F. G., & Yilmaz, R. (2020a). Learning analytics as a metacognitive tool to influence learner transactional distance and motivation in online learning environments. Innovations in Education and Teaching International. https://doi.org/10.1080/14703297.2020.1794928

    Article  Google Scholar 

  • Karaoglan Yilmaz, F. G., & Yilmaz, R. (2020b). Student opinions about personalized recommendation and feedback based on learning analytics. Technology, Knowledge and Learning, 25(4), 753–768. https://doi.org/10.1007/s10758-020-09460-8

    Article  Google Scholar 

  • Karaoglan Yilmaz, F. G. (2021). Utilizing learning analytics to support students’ academic self-efficacy and problem-solving skills. The Asia-Pacific Education Researcher. https://doi.org/10.1007/s40299-020-00548-4

    Article  Google Scholar 

  • Kerly, A., Ellis, R., & Bull, S. (2008). CALMsystem: A conversational agent for learner modelling. In R. Ellis, T. Allen, & M. Petridis (Eds.), Applications and innovations in intelligent systems XV (pp. 89–102). Springer.

    Chapter  Google Scholar 

  • Kloos, C.D., Pardo, A., Muñoz-Merino, P.J., Gutiérrez, I., & Leony, D. (2013). Learning analytics @ UC3M. 2013 IEEE Global Engineering Education Conference (EDUCON). Berlin, Germany.

  • Ma, J., Han, X., Yang, J., & Cheng, J. (2015). Examining the necessary condition for engagement in an online learning environment based on learning analytics approach: The role of the instructor. Internet and Higher Education, 24(2015), 26–34. https://doi.org/10.1016/j.iheduc.2014.09.005

    Article  Google Scholar 

  • Miles, M. B., & Huberman, A. M. (1994). Qualitative data analysis: An expanded sourcebook. Sage.

    Google Scholar 

  • Moore, M. G., & Kearsley, G. (2011). Distance education: A systems view of online learning. Cengage Learning.

  • Nunnally, J. C. (1967). Psychometric theory. McGraw-Hill.

    Google Scholar 

  • Park, Y., & Jo, I. H. (2015). Development of the learning analytics dashboard to support students’ learning performance. Journal of Universal Computer Science, 21(1), 110–133. https://doi.org/10.3217/jucs-021-01-0110

    Article  Google Scholar 

  • Peña-Ayala, A. (2014). Educational data mining: A survey and a data mining-based analysis of recent works. Expert Systems with Applications, 41(4, Part 1), 1432–1462. https://doi.org/10.1016/j.eswa.2013.08.042

    Article  Google Scholar 

  • Ribbe, E., & Bezenilla, M. J. (2013). Scaffolding learner autonomy in online university courses. Digital Education Review, 24, 98–113. https://doi.org/10.1344/der.2013.24.98-112

    Article  Google Scholar 

  • Ruipérez-Valiente, J. A., Muñoz-Merino, P. J., Leony, D., & Kloos, C. D. (2015). ALAS-KA: A learning analytics extension for better understanding the learning process in the Khan Academy platform. Computers in Human Behavior, 47(2015), 139–148. https://doi.org/10.1016/j.chb.2014.07.002

    Article  Google Scholar 

  • Sahin, M., & Yurdugul, H. (2019). An intervention engine design and development based on learning analytics: The intelligent intervention system (In 2 S). Smart Learning Environments, 6(1), 1–18. https://doi.org/10.1186/s40561-019-0100-7

    Article  Google Scholar 

  • Sahin, M., & Yurdugul, H. (2020). The framework of learning analytics for prevention, intervention, and postvention in e-learning environments. In D. Ifenthaler & D. Gibson (Eds.), Adoption of data analytics in higher education learning and teaching (pp. 53–69). Springer International Publishing.

    Chapter  Google Scholar 

  • Scheffel, M., Drachsler, H., Stoyanov, S., & Specht, M. (2014). Quality indicators for learning analytics. Journal of Educational Technology & Society, 17(4), 117–132.

    Google Scholar 

  • Schraw, G., & Dennison, R. S. (1994). Assessing metacognitive awareness. Contemporary Educational Psychology, 19(4), 460–475. https://doi.org/10.1006/ceps.1994.1033

    Article  Google Scholar 

  • Siemens, G., & Long, P. (2011). Penetrating the fog: Analytics in learning and education. EDUCAUSE Review, 46(5), 30–32.

    Google Scholar 

  • Siemens, G., & Gasevic, D. (2012). Guest editorial - learning and knowledge analytics. Educational Technology and Society, 15(3), 1–2.

    Google Scholar 

  • Simpson, O. (2004). The impact on retention of interventions to support distance learning students. Open Learning: THe Journal of Open, Distance and e-Learning, 19(1), 79–95. https://doi.org/10.1080/0268051042000177863

    Article  Google Scholar 

  • Ustun, A. B. (2019). Effects of mobile learning in blended learning environments. Bilgi Ve İletişim Teknolojileri Dergisi/journal of Information and Communication Technologies, 1(1), 1–14.

    Google Scholar 

  • Wright, M. C., McKay, T., Hershock, C., Miller, K., & Tritz, J. (2014). Better than expected: Using learning analytics to promote student success in gateway science. Change: the Magazine of Higher Learning, 46(1), 28–34. https://doi.org/10.1080/00091383.2014.867209

    Article  Google Scholar 

  • Van Laer, S., & Elen, J. (2019). The effect of cues for calibration on learners’ self-regulated learning through changes in learners’ learning behaviour and outcomes. Computers and Education, 135(2019), 30–48. https://doi.org/10.1016/j.compedu.2019.02.016

    Article  Google Scholar 

  • Verbert, K., Govaerts, S., Duval, E., Santos, J., Van Assche, F., Parra, G., & Klerkx, J. (2014). Learning dashboards: An overview and future research opportunities. Personal and Ubiquitous Computing, 18(6), 1499–1514. https://doi.org/10.1007/s00779-013-0751-2

    Article  Google Scholar 

  • Yilmaz, R. (2020). Enhancing community of inquiry and reflective thinking skills of undergraduates through using learning analytics-based process feedback. Journal of Computer Assisted Learning, 36(6), 909–921. https://doi.org/10.1111/jcal.12449

    Article  Google Scholar 

  • Yilmaz, F. G. K., Ustun, A. B., & Yilmaz, R. (2020). Investigation of pre-service teachers’ opinions on advantages and disadvantages of online formative assessment: An example of online multiple-choice exam. Journal of Teacher Education and Lifelong Learning, 2(1), 1–8.

    Article  Google Scholar 

Download references

Funding

The author declare that they have no funding of this study.

Author information

Authors and Affiliations

  1. Faculty of Science, Department of Computer Technology and Information Systems, Bartin University, Bartin, Turkey

    Fatma Gizem Karaoglan Yilmaz

Authors
  1. Fatma Gizem Karaoglan Yilmaz
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Fatma Gizem Karaoglan Yilmaz.

Ethics declarations

Conflict of interest

The author declare that they have no conflict of interest.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Karaoglan Yilmaz, F. The effect of learning analytics assisted recommendations and guidance feedback on students’ metacognitive awareness and academic achievements. J Comput High Educ 34, 396–415 (2022). https://doi.org/10.1007/s12528-021-09304-z

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s12528-021-09304-z

Keywords

Search

Navigation