Advertisement

Journal of Computing in Higher Education

, Volume 31, Issue 3, pp 604–625 | Cite as

Technological barriers and incentives to learning analytics adoption in higher education: insights from users

  • Carrie KleinEmail author
  • Jaime Lester
  • Huzefa Rangwala
  • Aditya Johri
Article

Abstract

Learning analytics (LA) tools promise to improve student learning and retention. However, adoption and use of LA tools in higher education is often uneven. In this case study, part of a larger exploratory research project, we interviewed and observed 32 faculty and advisors at a public research university to understand the technological incentives and barriers related to LA tool adoption and use. Findings indicate that lack of a trustworthy technological infrastructure, misalignment between LA tool capabilities and user needs, and the existence of ethical concerns about the data, visualizations, and algorithms that underlie LA tools created barriers to adoption. Improving tool integration, clarity, and accuracy, soliciting the technological needs and perspectives of LA tool users, and providing data context may encourage inclusion of these tools into teaching and advising practice.

Keywords

Learning analytics Predictive analytics Technology adoption Technological barriers Technological incentives Higher education 

Notes

Acknowledgements

This research was supported in part by a grant from the National Science Foundation under Grant IIS-1447489.

Compliance with ethical standards

Conflict of interest

The authors declare that they have no conflict of interest.

Ethical standards

The authors have complied with all ethical standards.

References

  1. Aguilar, S., Lonn, S., & Teasley, S. D. (2014, March). Perceptions and use of an early warning system during a higher education transition program. In Proceedings of the fourth international conference on learning analytics and knowledge, ACM (pp. 113–117).Google Scholar
  2. Ali, L., Hatala, M., Gašević, D., & Jovanović, J. (2012). A qualitative evaluation of evolution of a learning analytics tool. Computers and Education, 58(1), 470–489.CrossRefGoogle Scholar
  3. Appleby, D. C. (2008). Advising as teaching and learning. Academic advising: A comprehensive handbook, 2, 85–102.Google Scholar
  4. Arnold, K. E., & Pistilli, M. D. (2012, April). Course signals at Purdue: using learning analytics to increase student success. In Proceedings of the 2nd International Conference on Learning Analytics and Knowledge, Association for Computing Machinery (pp. 267–270).Google Scholar
  5. Austin, A. E. (2011). Promoting evidence-based change in undergraduate science education. National Academies National Research Council. Retrieved from: tidemarkinstitute.org.Google Scholar
  6. Balcer, Y., & Lippman, S. A. (1984). Technological expectations and adoption of improved technology. Journal of Economic Theory, 34(2), 292–318.CrossRefGoogle Scholar
  7. Brown, M. G. (2016). Blended instructional practice: A review of the empirical literature on instructors’ adoption and use of online tools in face-to-face teaching. The Internet and Higher Education, 31, 1–10.CrossRefGoogle Scholar
  8. Creswell, J. W. (2002). Educational research: Planning, conducting, and evaluating quantitative (pp. 146–166). Upper Saddle River, NJ: Prentice Hall.Google Scholar
  9. Crookston, B. B. (1994). A developmental view of academic advising as teaching. NACADA Journal, 14(2), 5–9.CrossRefGoogle Scholar
  10. Dahlstrom, E., Brooks, D. C., & Bichsel, J. (2014). The Current Ecosystem of Learning Management Systems in Higher Education: Student, Faculty, and IT Perspectives. Research report. Louisville, CO: ECAR, Sept 2014. http://www.educause.edu/ecar.
  11. Daniel, B. (2015). Big data and analytics in higher education: Opportunities and challenges. British Journal of Educational Technology, 46(5), 904–920.CrossRefGoogle Scholar
  12. Davis, F. (1989). Perceived usefulness, perceived ease of use, and user acceptance of information technology. MIS Quarterly, 13, 319–340.CrossRefGoogle Scholar
  13. Dawson, S., Jovanovic, J., Gašević, D., & Pardo, A. (2017, March). From prediction to impact: Evaluation of a learning analytics retention program. In Proceedings of the Seventh International Learning Analytics & Knowledge Conference, ACM (pp. 474–478).Google Scholar
  14. Dawson, S., McWilliam, E., & Tan, J.P.L. (2008). Teaching smarter: How mining ICTdata can inform and improve learning and teaching practice. In Hello! Where are you in the landscape of educational technology? Proceedings ascilite Melbourne 2008. http://www.ascilite.org.au/conferences/melbourne08/procs/dawson.pdf.
  15. Hagen, P. L., & Jordan, P. (2008). Theoretical foundations of academic advising. Academic Advising: A Comprehensive Handbook, 2, 17–35.Google Scholar
  16. Hora. M. T, Bouwma-Gearhart, J., & Park, H. J. (2014). Using Practice-based Research to Frame and Focus Pedagogical Reform: Exploring the Use of Data and Other Information to Guide Instructional Decision-making (WCER Working Paper No. 2014–3). Retrieved from University of Wisconsin–Madison, Wisconsin Center for Education Research website: http://www.wcer.wisc.edu/publications/workingPapers/papers.php.
  17. Kezar, A. J., & Lester, J. (2009). Organizing higher education for collaboration: a guide for campus leaders. San Francisco: Jossey-Bass.Google Scholar
  18. Klein, C., Lester, J., Rangwala, H., & Johri, A. (2019). Learning analytics tools in higher education: Adoption at the intersection of institutional commitment and individual action. The Review of Higher Education, 42(2), 565–593.Google Scholar
  19. Klein, C., Lester, J., Rangwala, H., & Johri, A. (in press). Learning analytics for learning assessment: Complexities in efficacy, implementation, and broad use. In K. Webber, & H. Zheng (Eds.), Analytics and data-informed decision making in higher education: Concepts and real-world applications. Baltimore: Johns Hopkins University Press.Google Scholar
  20. Knight, D. B., Brozina, C., Kinoshita, T., Novoselich, B., Young, G., & Grohs, J. R. (2018). Discipline-focused learning analytics approaches with instead of for users. In J. Lester, C. Klein, A. Johri, & H. Rangwala (Eds.), Learning analytics in higher education: Current innovations, future potential, and practical applications. New York: Routledge.Google Scholar
  21. Knight, D. B., Brozina, C., & Novoselich, B. (2016). An investigation of first-year engineering student and instructor perspectives of learning analytics approaches. Journal of Learning Analytics, 3(3), 215–238.CrossRefGoogle Scholar
  22. Lester, J., Klein, C., Rangwala, H., & Johri, A. (2017). Learning analytics in higher education. ASHE Higher Education Report, 43(5), 9–135.CrossRefGoogle Scholar
  23. Lincoln, Y. S., & Guba, E. G. (2000). The only generalization is: There is no generalization. Case study method: Key issues, key texts, 17, 27–44.Google Scholar
  24. Lockyer, L., Heathcote, E., & Dawson, S. (2013). Informing pedagogical action: Aligning learning analytics with learning design. American Behavioral Scientist, 57(10), 1439–1459.CrossRefGoogle Scholar
  25. Mertens, D. M. (2005). Research and evaluation in education and psychology: Integrating diversity with quantitative, qualitative, and mixed methods. Thousand Oaks: SAGE Publications.Google Scholar
  26. Mishra, P., & Koehler, M. J. (2006). Technological pedagogical content knowledge: A framework for integrating technology in teachers’ knowledge. Teachers College Record, 108(6), 1017–1054.CrossRefGoogle Scholar
  27. Norris, D. M., & Baer, L. L. (2013). Building organizational capacity for analytics. Educause Learning Initiative, EDUCAUSE. Retrieved from: https://net.educause.edu/ir/library/pdf/PUB9012.pdf.
  28. Oster, M., Lonn, S., Pistilli, M. D., & Brown, M. G. (2016, April). The learning analytics readiness instrument. In Proceedings of the Sixth International Conference on Learning Analytics & Knowledge,. ACM (pp. 173–182).Google Scholar
  29. Papamitsiou, Z., & Economides, A. A. (2014). Learning analytics and educational data mining in practice: A systematic literature review of empirical evidence. Journal of Educational Technology and Society, 17(4), 49.Google Scholar
  30. Peña-Ayala, A. (2014). Educational data mining: A survey and a data mining-based analysis of recent works. Expert Systems with Applications, 41(4), 1432–1462.CrossRefGoogle Scholar
  31. Picciano, A. G. (2012). The evolution of big data and learning analytics in American higher education. Online Learning, 16(3), 9–20.CrossRefGoogle Scholar
  32. Privateer, P. M. (1999). Academic technology and the future of higher education: Strategic paths taken and not taken. Journal of Higher Education, 70(1), 60–79.Google Scholar
  33. Rhode, J., Richter, S., Gowen, P., Miller, T., & Wills, C. (2017). Understanding faculty use of the learning management system. Online Learning, 21(3), 68–86.CrossRefGoogle Scholar
  34. Rogers, E. (1995). Diffusion of innovations (4th ed.). New York: Free Press.Google Scholar
  35. Saldaña, J. (2015). The coding manual for qualitative researchers. Thousand Oaks: Sage.Google Scholar
  36. Siemens, G. (2011). Learning and academic analytics [website blog post]. Learning and Knowledge Analytics. http://www.learninganalytics.net/?p=131.
  37. Stake, R. (2003). Case studies. In N. K. Denzin & Y. S. Lincoln (Eds.), Strategies of qualitative inquiry (2nd ed., pp. 134–164). Thousand Oaks: Sage.Google Scholar
  38. Stake, R. E. (2005). Qualitative case studies. In N. K. Denzin & Y. S. Lincoln (Eds.), The SAGE handbook of qualitative research (3rd ed., pp. 433–466). Thousand Oaks: Sage.Google Scholar
  39. Straub, E. T. (2009). Understanding technology adoption: Theory and future directions for informal learning. Review of educational research, 79(2), 625–649.CrossRefGoogle Scholar
  40. Strauss, A., & Corbin, J. (1990). Basics of qualitative research: Grounded theory procedures and techniques. Newbury Park: SAGE Publications.Google Scholar
  41. Svinicki, M. D., Williams, K., Rackley, K., Sanders, A. J., Pine, L., & Stewart, J. (2016). Factors associated with faculty use of student data for instructional improvement. International Journal for the Scholarship of Teaching and Learning, 10(2), n2.CrossRefGoogle Scholar
  42. Venkatesh, V., Morris, M. G., Davis, G. B., & Davis, F. (2003). User acceptance of information technology: Toward a unified view. MIS Quarterly, 27, 425–478.CrossRefGoogle Scholar
  43. Viberg, O., Hatakka, M., Bälter, O., & Mavroudi, A. (2018). The current landscape of learning analytics in higher education. Computers in Human Behavior, 89, 98–110.CrossRefGoogle Scholar
  44. Zellweger Moser, F. (2007a). Faculty adoption of educational technology. EDUCAUSE quarterly, 30(1), 66.Google Scholar
  45. Zellweger Moser, F. (2007b). The strategic management of E-learning support. New York: Waxmann Münster.Google Scholar

Copyright information

© Springer Science+Business Media, LLC, part of Springer Nature 2019

Authors and Affiliations

  1. 1.Higher Education ProgramGeorge Mason UniversityFairfaxUSA
  2. 2.Computer ScienceGeorge Mason UniversityFairfaxUSA
  3. 3.Informational Sciences and TechnologyGeorge Mason UniversityFairfaxUSA

Personalised recommendations