Advertisement

Technology, Knowledge and Learning

, Volume 22, Issue 3, pp 271–298 | Cite as

Using Data to Understand How to Better Design Adaptive Learning

  • Min Liu
  • Jina Kang
  • Wenting Zou
  • Hyeyeon Lee
  • Zilong Pan
  • Stephanie Corliss
Original research
First Online:

Abstract

There is much enthusiasm in higher education about the benefits of adaptive learning and using big data to investigate learning processes to make data-informed educational decisions. The benefits of adaptive learning to achieve personalized learning are obvious. Yet, there lacks evidence-based research to understand how data such as user behavior patterns can be used to design effective adaptive learning systems. The purpose of this study, therefore, is to investigate what behavior patterns learners with different characteristics demonstrate when they interact with an adaptive learning environment. Incoming 1st-year students in a pharmacy professional degree program engaged in an adaptive learning intervention that aimed to provide remedial instruction to better prepare these professional students before they began their formal degree program. We analyzed the participants’ behavior patterns through the usage data to understand how they used the adaptive system based upon their needs and interests. Using both statistical analyses and data visualization techniques, this study found: (1) apart from learners’ cognitive ability, it is important to consider affective factors such as motivation in adaptive learning, (2) lack of alignment among various components in an adaptive system can impact how learners accessed the system and, more importantly, their performance, and (3) visualizations can reveal interesting findings that can be missed otherwise. Such research should provide much needed empirical evidences and useful insights about how the analytics can inform the effective designs of adaptive learning.

Keywords

Adaptive learning Personalized learning Learning analytics Data visualization 
This is a preview of subscription content, log in to check access.

References

  1. Abdous, M. H., He, W., & Yen, C. J. (2012). Using data mining for predicting relationships between online question theme and final grade. Educational Technology & Society, 15(3), 77–88.Google Scholar
  2. Arroyo, I., Woolf, B. (2005) Inferring learning and attitudes from a Bayesian Network of log file data. In Proceedings of the 12th international conference on artificial intelligence in education, pp. 33–40.Google Scholar
  3. Baghaei, N., Mitrovic, A., & Irwin, W. (2007). Supporting collaborative learning and problem-solving in a constraint-based CSCL environment for UML class diagrams. International Journal of Computer-Supported Collaborative Learning, 2(2), 159–190. doi: 10.1007/s11412-007-9018-0.CrossRefGoogle Scholar
  4. Bienkowski, M., Feng, M., & Means, B. (2012). Enhancing teaching and learning through educational data mining and learning analytics: An issue brief (pp. 1–57). Office of Educational Technology: US Department of Education.Google Scholar
  5. Brown, J. (2015). Personalizing post-secondary education: an overview of adaptive learning solutions for higher education. Ithaka S + R. doi: 10.18665/sr.221030.Google Scholar
  6. Bruckman, A. (2006). Learning in online communities. In R. K. Sawyer (Ed.), The Cambridge handbook of the learning sciences (pp. 461–472). New York: Cambridge University Press.Google Scholar
  7. Brusilovsky, P. (2001). Adaptive hypermedia. User Modeling and User-Adapted Interaction, 11, 87–110. doi: 10.1007/978-94-017-0617-9_1.CrossRefGoogle Scholar
  8. Brusilovsky, P., & Millán, E. (2007). User models for adaptive hypermedia and adaptive educational systems. The adaptive web: Methods and strategies of web personalization (pp. 3–53). Berlin: Springer.Google Scholar
  9. Brusilovsky, P., & Vassileva, J. (2003). Course sequencing techniques for large-scale web-based education. International Journal of Continuing Engineering Education and Lifelong learning, 13(1–2), 75–94. doi: 10.1504/IJCEELL.2003.002154.CrossRefGoogle Scholar
  10. Dernoncourt, F., Do, C., Halawa, S., O’Reilly, U.-M., Taylor, C., Veeramachaneni, K., & Wu, S. (2013, December). MoocViz: A large scale, open access, collaborative, data analytics platform for MOOCs. Paper presented at the NIPS Workshop on Data-Driven Education, Lake Tahoe, Nevada, USA.Google Scholar
  11. Dunlosky, J., Rawson, K. A., Marsh, E. J., Nathan, M. J., & Willingham, D. T. (2013). Improving students’ learning with effective learning techniques promising directions from cognitive and educational psychology. Psychological Science in the Public Interest, 14(1), 4–58.CrossRefGoogle Scholar
  12. Dziuban, C. D., Moskal, P. D., Cassisi, J., & Fawcett, A. (2016). Adaptive learning in psychology: Wayfinding in the digital age. Online Learning, 20(3), 74–96.CrossRefGoogle Scholar
  13. Evans, J. D. (1996). Straightforward statistics for the behavior sciences. Pacific Grove: Brooks/Cole Publishing.Google Scholar
  14. Graf, S., & Liu, T. C. (2010). Analysis of learners’ navigational behavior and their learning styles in an online course. Journal of Computer Assisted learning, 26(2), 116–131. doi: 10.1111/j.1365-2729.2009.00336.x.CrossRefGoogle Scholar
  15. Hsieh, T. C., Wang, T. I., Su, C. Y., & Lee, M. C. (2012). A fuzzy logic-based personalized learning system for supporting adaptive english learning. Educational Technology & Society, 15(1), 273–288. Retrieved from http://www.jstor.org/stable/jeductechsoci.15.1.273.
  16. Hsu, P. (2012). Learner characteristic based learning effort curve mode: The core mechanism on developing personalized adaptive e-learning platform. TOJET: The Turkish Online Journal of Educational Technology, 11(4), 210–219.Google Scholar
  17. Huang, S. L., & Shiu, J. H. (2012). A user-centric adaptive learning system for e-learning 2.0. Educational Technology & Society, 15(3), 214–225.Google Scholar
  18. Hurley, T., & Weibelzahl, S. (2007, September). Using MotSaRT to support on-line teachers in student motivation. In Proceedings of 2nd European conference on technology enhanced learning, pp. 101–111. doi:  10.1007/978-3-540-75195-3_8.
  19. Johnson, L., Adams Becker, S., Cummins, M., Estrada, V., Freeman, A., & Hall, C. (2016). NMC horizon report: 2016 higher education (ed.). Austin: The New Media Consortium.Google Scholar
  20. Kelly, D. (2008). Adaptive Versus learner control in a multiple intelligence learning environment. Journal of educational multimedia and hypermedia, 17(3), 307–336. Retrieved from https://www.learntechlib.org/p/24252.
  21. Kerr, P. (2016). Adaptive learning. ELT Journal, 70(1), 88–93. doi: 10.1093/elt/ccv055.CrossRefGoogle Scholar
  22. Klasnja-Milicevic, A., Vesin, B., Ivanovic, M., & Budimac, Z. (2011). E-learning personalization base on hybrid recommendation strategy and learning style identification. Computers & Education, 56(3), 885–899. doi: 10.1016/j.compedu.2010.11.001.CrossRefGoogle Scholar
  23. Koedinger, K., Cunningham, K., Skogsholm, A., & Leber, B. (2008). An open repository and analysis tools for fine-grained, longitudinal learner data. In International conference on educational data mining (pp. 157–166).Google Scholar
  24. Liu, M., Lee, J., Kang, J., & Liu, S. (2016). What we can learn from the data: A multiple- case study examining behavior patterns by students with different characteristics in using a serious game. The Technology, Knowledge and Learning Journal, 21(1), 33–57. doi: 10.1007/s10758-015-9263-7.CrossRefGoogle Scholar
  25. Liu, M., McKelroy, E., Corliss, S. B., & Carrigan, J. (2017, under review). Investigating the effect of an adaptive learning intervention on students’ learning. Submitted to Educational technology research and development. Google Scholar
  26. Melero, J., Hernández-Leo, D., Sun, J., Santos, P., & Blat, J. (2015). How was the activity? A visualization support for a case of location-based learning design. British Journal of Educational Technology, 46(2), 317–329. doi: 10.1111/bjet.12238.CrossRefGoogle Scholar
  27. Minović, M., Milovanović, M., Šošević, U., & González, M. Á. C. (2015). Visualisation of student learning model in serious games. Computers in Human Behavior, 47, 98–107. doi: 10.1016/j.chb.2014.09.005.CrossRefGoogle Scholar
  28. Nakic, J., Granic, A., & Glavinic, V. (2015). Anatomy of student models in adaptive learning systems: A systematic literature review of individual differences from 2001 to 2013. Journal of Educational Computing Research, 51(4), 459–489. doi: 10.2190/EC.51.4.e.Google Scholar
  29. Papamitsiou, Z. K., & Economides, A. A. (2014). Learning analytics and educational data mining in practice: A systematic literature review of empirical evidence. Educational Technology & Society, 17(4), 49–64.Google Scholar
  30. Premlatha, K. R., Dharani, B., & Geetha, T. V. (2014). Dynamic learner profiling and automatic learner classification for adaptive e-learning environment. Interactive Learning Environments, 24(6), 1–22. doi: 10.1080/10494820.2014.948459.Google Scholar
  31. Qu, H., & Chen, Q. (2015). Visual analytics for MOOC data. IEEE Computer Graphics and Applications, 35(6), 69–75. doi: 10.1109/MCG.2015.137.CrossRefGoogle Scholar
  32. Roediger, H. L., & Butler, A. C. (2011). The critical role of retrieval practice in long-term retention. Trends in cognitive sciences, 15(1), 20–27.CrossRefGoogle Scholar
  33. Roediger, H. L., III, & Karpicke, J. D. (2006). Test-enhanced learning: Taking memory tests improves long-term retention. Psychological Science, 17, 249–255.CrossRefGoogle Scholar
  34. Romero, C., & Ventura, S. (2010). Educational data mining: a review of the state of the art. IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews), 40(6), 601–618. doi: 10.1109/TSMCC.2010.2053532.CrossRefGoogle Scholar
  35. Sangineto, E., Capuano, N., Gaeta, M., & Micarelli, A. (2008). Adaptive course generation through learning styles representation. Universal Access in the Information Society, 7(1), 1–23. doi: 10.1007/s10209-007-0101-0.CrossRefGoogle Scholar
  36. Shi, C., Fu, S., Chen, Q., & Qu, H. (2015, April). VisMOOC: Visualizing video clickstream data from massive open online courses. In 2015 IEEE Pacific visualization symposium (PacificVis) (pp. 159–166). IEEE.Google Scholar
  37. Shih, B., Koedinger, K. R., & Scheines, R. (2011). A response time model for bottom-out hints as worked examples. Handbook of Educational Data Mining, 201–212.Google Scholar
  38. Siemens, G. (2013). Learning analytics: The emergence of a discipline. American Behavioral Scientist, 57(10), 1380–1400. doi: 10.1177/0002764213498851.CrossRefGoogle Scholar
  39. Siemens, G., & Baker, R. S. J. D. (2012, April). Learning analytics and educational data mining: towards communication and collaboration. In Proceedings of the 2nd international conference on learning analytics and knowledge (pp. 252–254). ACM.Google Scholar
  40. Snow, E. L., Allen, L. K., Jacovina, M. E., & McNamara, D. S. (2015). Does agency matter? Exploring the impact of controlled behaviors within a game-based environment. Computers & Education, 82, 378–392. doi: 10.1016/j.compedu.2014.12.011.CrossRefGoogle Scholar
  41. Van Seters, J. R., Ossevoort, M. A., Tramper, J., & Goedhart, M. J. (2012). The influence of student characteristic on the use of adaptive e-learning material. Computers & Education, 58(3), 942–952. doi: 10.1016/j.compedu.2011.11.002.CrossRefGoogle Scholar
  42. Van Wijk, J. J. (2013). Evaluation: a challenge for visual analytics. Computer, 46(7), 56–60. doi: 10.1109/MC.2013.151.CrossRefGoogle Scholar
  43. Wallner, G., & Kriglstein, S. (2013). Visualization-based analysis of gameplay data—A review of literature. Entertainment Computing, 4(3), 143–155. doi: 10.1016/j.entcom.2013.02.002.CrossRefGoogle Scholar
  44. Yang, T., Hwang, G., & Yang, S. J. (2013). Development of an adaptive learning system with multiple perspectives based on students’ learning styles and cognitive styles. Educational Technology & Society, 16(4), 185–200.Google Scholar

Copyright information

© Springer Science+Business Media B.V. 2017

Authors and Affiliations

  1. 1.Learning Technologies Program, Department of Curriculum and InstructionThe University of Texas at AustinAustinUSA
  2. 2.Department of Medical Education in Dell Medical SchoolThe University of Texas at AustinAustinUSA

Personalised recommendations

Cite article

Buy options