Technological barriers and incentives to learning analytics adoption in higher education: insights from users
Learning analytics (LA) tools promise to improve student learning and retention. However, adoption and use of LA tools in higher education is often uneven. In this case study, part of a larger exploratory research project, we interviewed and observed 32 faculty and advisors at a public research university to understand the technological incentives and barriers related to LA tool adoption and use. Findings indicate that lack of a trustworthy technological infrastructure, misalignment between LA tool capabilities and user needs, and the existence of ethical concerns about the data, visualizations, and algorithms that underlie LA tools created barriers to adoption. Improving tool integration, clarity, and accuracy, soliciting the technological needs and perspectives of LA tool users, and providing data context may encourage inclusion of these tools into teaching and advising practice.
KeywordsLearning analytics Predictive analytics Technology adoption Technological barriers Technological incentives Higher education
This research was supported in part by a grant from the National Science Foundation under Grant IIS-1447489.
Compliance with ethical standards
Conflict of interest
The authors declare that they have no conflict of interest.
The authors have complied with all ethical standards.
- Aguilar, S., Lonn, S., & Teasley, S. D. (2014, March). Perceptions and use of an early warning system during a higher education transition program. In Proceedings of the fourth international conference on learning analytics and knowledge, ACM (pp. 113–117).Google Scholar
- Appleby, D. C. (2008). Advising as teaching and learning. Academic advising: A comprehensive handbook, 2, 85–102.Google Scholar
- Arnold, K. E., & Pistilli, M. D. (2012, April). Course signals at Purdue: using learning analytics to increase student success. In Proceedings of the 2nd International Conference on Learning Analytics and Knowledge, Association for Computing Machinery (pp. 267–270).Google Scholar
- Austin, A. E. (2011). Promoting evidence-based change in undergraduate science education. National Academies National Research Council. Retrieved from: tidemarkinstitute.org.Google Scholar
- Creswell, J. W. (2002). Educational research: Planning, conducting, and evaluating quantitative (pp. 146–166). Upper Saddle River, NJ: Prentice Hall.Google Scholar
- Dahlstrom, E., Brooks, D. C., & Bichsel, J. (2014). The Current Ecosystem of Learning Management Systems in Higher Education: Student, Faculty, and IT Perspectives. Research report. Louisville, CO: ECAR, Sept 2014. http://www.educause.edu/ecar.
- Dawson, S., Jovanovic, J., Gašević, D., & Pardo, A. (2017, March). From prediction to impact: Evaluation of a learning analytics retention program. In Proceedings of the Seventh International Learning Analytics & Knowledge Conference, ACM (pp. 474–478).Google Scholar
- Dawson, S., McWilliam, E., & Tan, J.P.L. (2008). Teaching smarter: How mining ICTdata can inform and improve learning and teaching practice. In Hello! Where are you in the landscape of educational technology? Proceedings ascilite Melbourne 2008. http://www.ascilite.org.au/conferences/melbourne08/procs/dawson.pdf.
- Hagen, P. L., & Jordan, P. (2008). Theoretical foundations of academic advising. Academic Advising: A Comprehensive Handbook, 2, 17–35.Google Scholar
- Hora. M. T, Bouwma-Gearhart, J., & Park, H. J. (2014). Using Practice-based Research to Frame and Focus Pedagogical Reform: Exploring the Use of Data and Other Information to Guide Instructional Decision-making (WCER Working Paper No. 2014–3). Retrieved from University of Wisconsin–Madison, Wisconsin Center for Education Research website: http://www.wcer.wisc.edu/publications/workingPapers/papers.php.
- Kezar, A. J., & Lester, J. (2009). Organizing higher education for collaboration: a guide for campus leaders. San Francisco: Jossey-Bass.Google Scholar
- Klein, C., Lester, J., Rangwala, H., & Johri, A. (2019). Learning analytics tools in higher education: Adoption at the intersection of institutional commitment and individual action. The Review of Higher Education, 42(2), 565–593.Google Scholar
- Klein, C., Lester, J., Rangwala, H., & Johri, A. (in press). Learning analytics for learning assessment: Complexities in efficacy, implementation, and broad use. In K. Webber, & H. Zheng (Eds.), Analytics and data-informed decision making in higher education: Concepts and real-world applications. Baltimore: Johns Hopkins University Press.Google Scholar
- Knight, D. B., Brozina, C., Kinoshita, T., Novoselich, B., Young, G., & Grohs, J. R. (2018). Discipline-focused learning analytics approaches with instead of for users. In J. Lester, C. Klein, A. Johri, & H. Rangwala (Eds.), Learning analytics in higher education: Current innovations, future potential, and practical applications. New York: Routledge.Google Scholar
- Lincoln, Y. S., & Guba, E. G. (2000). The only generalization is: There is no generalization. Case study method: Key issues, key texts, 17, 27–44.Google Scholar
- Mertens, D. M. (2005). Research and evaluation in education and psychology: Integrating diversity with quantitative, qualitative, and mixed methods. Thousand Oaks: SAGE Publications.Google Scholar
- Norris, D. M., & Baer, L. L. (2013). Building organizational capacity for analytics. Educause Learning Initiative, EDUCAUSE. Retrieved from: https://net.educause.edu/ir/library/pdf/PUB9012.pdf.
- Oster, M., Lonn, S., Pistilli, M. D., & Brown, M. G. (2016, April). The learning analytics readiness instrument. In Proceedings of the Sixth International Conference on Learning Analytics & Knowledge,. ACM (pp. 173–182).Google Scholar
- Papamitsiou, Z., & Economides, A. A. (2014). Learning analytics and educational data mining in practice: A systematic literature review of empirical evidence. Journal of Educational Technology and Society, 17(4), 49.Google Scholar
- Privateer, P. M. (1999). Academic technology and the future of higher education: Strategic paths taken and not taken. Journal of Higher Education, 70(1), 60–79.Google Scholar
- Rogers, E. (1995). Diffusion of innovations (4th ed.). New York: Free Press.Google Scholar
- Saldaña, J. (2015). The coding manual for qualitative researchers. Thousand Oaks: Sage.Google Scholar
- Siemens, G. (2011). Learning and academic analytics [website blog post]. Learning and Knowledge Analytics. http://www.learninganalytics.net/?p=131.
- Stake, R. (2003). Case studies. In N. K. Denzin & Y. S. Lincoln (Eds.), Strategies of qualitative inquiry (2nd ed., pp. 134–164). Thousand Oaks: Sage.Google Scholar
- Stake, R. E. (2005). Qualitative case studies. In N. K. Denzin & Y. S. Lincoln (Eds.), The SAGE handbook of qualitative research (3rd ed., pp. 433–466). Thousand Oaks: Sage.Google Scholar
- Strauss, A., & Corbin, J. (1990). Basics of qualitative research: Grounded theory procedures and techniques. Newbury Park: SAGE Publications.Google Scholar
- Zellweger Moser, F. (2007a). Faculty adoption of educational technology. EDUCAUSE quarterly, 30(1), 66.Google Scholar
- Zellweger Moser, F. (2007b). The strategic management of E-learning support. New York: Waxmann Münster.Google Scholar