Skip to main content

Advertisement

SpringerLink
Log in

Using Data to Understand How to Better Design Adaptive Learning

  • Original research
  • Published:
Technology, Knowledge and Learning Aims and scope Submit manuscript

Abstract

There is much enthusiasm in higher education about the benefits of adaptive learning and using big data to investigate learning processes to make data-informed educational decisions. The benefits of adaptive learning to achieve personalized learning are obvious. Yet, there lacks evidence-based research to understand how data such as user behavior patterns can be used to design effective adaptive learning systems. The purpose of this study, therefore, is to investigate what behavior patterns learners with different characteristics demonstrate when they interact with an adaptive learning environment. Incoming 1st-year students in a pharmacy professional degree program engaged in an adaptive learning intervention that aimed to provide remedial instruction to better prepare these professional students before they began their formal degree program. We analyzed the participants’ behavior patterns through the usage data to understand how they used the adaptive system based upon their needs and interests. Using both statistical analyses and data visualization techniques, this study found: (1) apart from learners’ cognitive ability, it is important to consider affective factors such as motivation in adaptive learning, (2) lack of alignment among various components in an adaptive system can impact how learners accessed the system and, more importantly, their performance, and (3) visualizations can reveal interesting findings that can be missed otherwise. Such research should provide much needed empirical evidences and useful insights about how the analytics can inform the effective designs of adaptive learning.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11

Similar content being viewed by others

References

  • Abdous, M. H., He, W., & Yen, C. J. (2012). Using data mining for predicting relationships between online question theme and final grade. Educational Technology & Society, 15(3), 77–88.

    Google Scholar 

  • Arroyo, I., Woolf, B. (2005) Inferring learning and attitudes from a Bayesian Network of log file data. In Proceedings of the 12th international conference on artificial intelligence in education, pp. 33–40.

  • Baghaei, N., Mitrovic, A., & Irwin, W. (2007). Supporting collaborative learning and problem-solving in a constraint-based CSCL environment for UML class diagrams. International Journal of Computer-Supported Collaborative Learning, 2(2), 159–190. doi:10.1007/s11412-007-9018-0.

    Article  Google Scholar 

  • Bienkowski, M., Feng, M., & Means, B. (2012). Enhancing teaching and learning through educational data mining and learning analytics: An issue brief (pp. 1–57). Office of Educational Technology: US Department of Education.

    Google Scholar 

  • Brown, J. (2015). Personalizing post-secondary education: an overview of adaptive learning solutions for higher education. Ithaka S + R. doi:10.18665/sr.221030.

    Google Scholar 

  • Bruckman, A. (2006). Learning in online communities. In R. K. Sawyer (Ed.), The Cambridge handbook of the learning sciences (pp. 461–472). New York: Cambridge University Press.

    Google Scholar 

  • Brusilovsky, P. (2001). Adaptive hypermedia. User Modeling and User-Adapted Interaction, 11, 87–110. doi:10.1007/978-94-017-0617-9_1.

    Article  Google Scholar 

  • Brusilovsky, P., & Millán, E. (2007). User models for adaptive hypermedia and adaptive educational systems. The adaptive web: Methods and strategies of web personalization (pp. 3–53). Berlin: Springer.

    Google Scholar 

  • Brusilovsky, P., & Vassileva, J. (2003). Course sequencing techniques for large-scale web-based education. International Journal of Continuing Engineering Education and Lifelong learning, 13(1–2), 75–94. doi:10.1504/IJCEELL.2003.002154.

    Article  Google Scholar 

  • Dernoncourt, F., Do, C., Halawa, S., O’Reilly, U.-M., Taylor, C., Veeramachaneni, K., & Wu, S. (2013, December). MoocViz: A large scale, open access, collaborative, data analytics platform for MOOCs. Paper presented at the NIPS Workshop on Data-Driven Education, Lake Tahoe, Nevada, USA.

  • Dunlosky, J., Rawson, K. A., Marsh, E. J., Nathan, M. J., & Willingham, D. T. (2013). Improving students’ learning with effective learning techniques promising directions from cognitive and educational psychology. Psychological Science in the Public Interest, 14(1), 4–58.

    Article  Google Scholar 

  • Dziuban, C. D., Moskal, P. D., Cassisi, J., & Fawcett, A. (2016). Adaptive learning in psychology: Wayfinding in the digital age. Online Learning, 20(3), 74–96.

    Article  Google Scholar 

  • Evans, J. D. (1996). Straightforward statistics for the behavior sciences. Pacific Grove: Brooks/Cole Publishing.

    Google Scholar 

  • Graf, S., & Liu, T. C. (2010). Analysis of learners’ navigational behavior and their learning styles in an online course. Journal of Computer Assisted learning, 26(2), 116–131. doi:10.1111/j.1365-2729.2009.00336.x.

    Article  Google Scholar 

  • Hsieh, T. C., Wang, T. I., Su, C. Y., & Lee, M. C. (2012). A fuzzy logic-based personalized learning system for supporting adaptive english learning. Educational Technology & Society, 15(1), 273–288. Retrieved from http://www.jstor.org/stable/jeductechsoci.15.1.273.

  • Hsu, P. (2012). Learner characteristic based learning effort curve mode: The core mechanism on developing personalized adaptive e-learning platform. TOJET: The Turkish Online Journal of Educational Technology, 11(4), 210–219.

    Google Scholar 

  • Huang, S. L., & Shiu, J. H. (2012). A user-centric adaptive learning system for e-learning 2.0. Educational Technology & Society, 15(3), 214–225.

    Google Scholar 

  • Hurley, T., & Weibelzahl, S. (2007, September). Using MotSaRT to support on-line teachers in student motivation. In Proceedings of 2nd European conference on technology enhanced learning, pp. 101–111. doi: 10.1007/978-3-540-75195-3_8.

  • Johnson, L., Adams Becker, S., Cummins, M., Estrada, V., Freeman, A., & Hall, C. (2016). NMC horizon report: 2016 higher education (ed.). Austin: The New Media Consortium.

    Google Scholar 

  • Kelly, D. (2008). Adaptive Versus learner control in a multiple intelligence learning environment. Journal of educational multimedia and hypermedia, 17(3), 307–336. Retrieved from https://www.learntechlib.org/p/24252.

  • Kerr, P. (2016). Adaptive learning. ELT Journal, 70(1), 88–93. doi:10.1093/elt/ccv055.

    Article  Google Scholar 

  • Klasnja-Milicevic, A., Vesin, B., Ivanovic, M., & Budimac, Z. (2011). E-learning personalization base on hybrid recommendation strategy and learning style identification. Computers & Education, 56(3), 885–899. doi:10.1016/j.compedu.2010.11.001.

    Article  Google Scholar 

  • Koedinger, K., Cunningham, K., Skogsholm, A., & Leber, B. (2008). An open repository and analysis tools for fine-grained, longitudinal learner data. In International conference on educational data mining (pp. 157–166).

  • Liu, M., Lee, J., Kang, J., & Liu, S. (2016). What we can learn from the data: A multiple- case study examining behavior patterns by students with different characteristics in using a serious game. The Technology, Knowledge and Learning Journal, 21(1), 33–57. doi:10.1007/s10758-015-9263-7.

    Article  Google Scholar 

  • Liu, M., McKelroy, E., Corliss, S. B., & Carrigan, J. (2017, under review). Investigating the effect of an adaptive learning intervention on students’ learning. Submitted to Educational technology research and development.

  • Melero, J., Hernández-Leo, D., Sun, J., Santos, P., & Blat, J. (2015). How was the activity? A visualization support for a case of location-based learning design. British Journal of Educational Technology, 46(2), 317–329. doi:10.1111/bjet.12238.

    Article  Google Scholar 

  • Minović, M., Milovanović, M., Šošević, U., & González, M. Á. C. (2015). Visualisation of student learning model in serious games. Computers in Human Behavior, 47, 98–107. doi:10.1016/j.chb.2014.09.005.

    Article  Google Scholar 

  • Nakic, J., Granic, A., & Glavinic, V. (2015). Anatomy of student models in adaptive learning systems: A systematic literature review of individual differences from 2001 to 2013. Journal of Educational Computing Research, 51(4), 459–489. doi:10.2190/EC.51.4.e.

    Google Scholar 

  • Papamitsiou, Z. K., & Economides, A. A. (2014). Learning analytics and educational data mining in practice: A systematic literature review of empirical evidence. Educational Technology & Society, 17(4), 49–64.

    Google Scholar 

  • Premlatha, K. R., Dharani, B., & Geetha, T. V. (2014). Dynamic learner profiling and automatic learner classification for adaptive e-learning environment. Interactive Learning Environments, 24(6), 1–22. doi:10.1080/10494820.2014.948459.

    Google Scholar 

  • Qu, H., & Chen, Q. (2015). Visual analytics for MOOC data. IEEE Computer Graphics and Applications, 35(6), 69–75. doi:10.1109/MCG.2015.137.

    Article  Google Scholar 

  • Roediger, H. L., & Butler, A. C. (2011). The critical role of retrieval practice in long-term retention. Trends in cognitive sciences, 15(1), 20–27.

    Article  Google Scholar 

  • Roediger, H. L., III, & Karpicke, J. D. (2006). Test-enhanced learning: Taking memory tests improves long-term retention. Psychological Science, 17, 249–255.

    Article  Google Scholar 

  • Romero, C., & Ventura, S. (2010). Educational data mining: a review of the state of the art. IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews), 40(6), 601–618. doi:10.1109/TSMCC.2010.2053532.

    Article  Google Scholar 

  • Sangineto, E., Capuano, N., Gaeta, M., & Micarelli, A. (2008). Adaptive course generation through learning styles representation. Universal Access in the Information Society, 7(1), 1–23. doi:10.1007/s10209-007-0101-0.

    Article  Google Scholar 

  • Shi, C., Fu, S., Chen, Q., & Qu, H. (2015, April). VisMOOC: Visualizing video clickstream data from massive open online courses. In 2015 IEEE Pacific visualization symposium (PacificVis) (pp. 159–166). IEEE.

  • Shih, B., Koedinger, K. R., & Scheines, R. (2011). A response time model for bottom-out hints as worked examples. Handbook of Educational Data Mining, 201–212.

  • Siemens, G. (2013). Learning analytics: The emergence of a discipline. American Behavioral Scientist, 57(10), 1380–1400. doi:10.1177/0002764213498851.

    Article  Google Scholar 

  • Siemens, G., & Baker, R. S. J. D. (2012, April). Learning analytics and educational data mining: towards communication and collaboration. In Proceedings of the 2nd international conference on learning analytics and knowledge (pp. 252–254). ACM.

  • Snow, E. L., Allen, L. K., Jacovina, M. E., & McNamara, D. S. (2015). Does agency matter? Exploring the impact of controlled behaviors within a game-based environment. Computers & Education, 82, 378–392. doi:10.1016/j.compedu.2014.12.011.

    Article  Google Scholar 

  • Van Seters, J. R., Ossevoort, M. A., Tramper, J., & Goedhart, M. J. (2012). The influence of student characteristic on the use of adaptive e-learning material. Computers & Education, 58(3), 942–952. doi:10.1016/j.compedu.2011.11.002.

    Article  Google Scholar 

  • Van Wijk, J. J. (2013). Evaluation: a challenge for visual analytics. Computer, 46(7), 56–60. doi:10.1109/MC.2013.151.

    Article  Google Scholar 

  • Wallner, G., & Kriglstein, S. (2013). Visualization-based analysis of gameplay data—A review of literature. Entertainment Computing, 4(3), 143–155. doi:10.1016/j.entcom.2013.02.002.

    Article  Google Scholar 

  • Yang, T., Hwang, G., & Yang, S. J. (2013). Development of an adaptive learning system with multiple perspectives based on students’ learning styles and cognitive styles. Educational Technology & Society, 16(4), 185–200.

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Learning Technologies Program, Department of Curriculum and Instruction, The University of Texas at Austin, 1912 Speedway Stop D5700, Austin, TX, 78712-1293, USA

    Min Liu, Jina Kang, Wenting Zou, Hyeyeon Lee & Zilong Pan

  2. Department of Medical Education in Dell Medical School, The University of Texas at Austin, Austin, TX, USA

    Stephanie Corliss

Authors
  1. Min Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Jina Kang
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Wenting Zou
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Hyeyeon Lee
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Zilong Pan
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Stephanie Corliss
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Min Liu.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Liu, M., Kang, J., Zou, W. et al. Using Data to Understand How to Better Design Adaptive Learning. Tech Know Learn 22, 271–298 (2017). https://doi.org/10.1007/s10758-017-9326-z

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10758-017-9326-z

Keywords

Search

Navigation