Towards a Personalized Learning Experience Using Reinforcement Learning

  • Doaa ShawkyEmail author
  • Ashraf Badawi
Part of the Studies in Computational Intelligence book series (SCI, volume 801)


Cognitive computing has become one of the most promising fields, especially in education, where building adaptive learning systems that provide different learning paths and material based on learners’ states and needs are developed. One of the most challenging issues in designing such systems is to correctly identify the factors that will influence the learning experience, especially when these factors highly differ from one learner to another. In addition, for one particular learner, the values of these factors change with time. In this paper, we present an approach that adapts to the most influential factors in learning in a way that varies from one learner to another and in different learning settings, including individual and collaborative learning. The approach utilizes reinforcement learning (RL) for building an intelligent environment that, not only does it provide a method for suggesting suitable learning materials, but it also provides a methodology for accounting for the continuously-changing students’ states and acceptance of the technology. We evaluate our system through simulations. The obtained results are promising and show the feasibility of the proposed approach. In addition, we propose a “rich” personalized learning system that relies on RL as its backend, while utilizing big data tools and learning analytics to continuously feed the system with new generated states.


Adaptive learning Personalized learning Reinforcement learning 


  1. 1.
    McCarthy, B., Schauer, K.: Journey to Personalized Learning: Bright Future—A Race to the Top-District Initiative in Galt Joint Union Elementary School District. WestEd, San Francisco, CA (2017)Google Scholar
  2. 2.
    Twyman, J.S.: Competency-Based Education: Supporting Personalized Learning. Connect: Making Learning Personal. Center on Innovations in Learning, Temple University (2014)Google Scholar
  3. 3.
    Shawky, D., Badawi, A., Said, T., Hozayin, R.: Affordances of computer-supported collaborative learning platforms: a systematic review. In: 2014 International Conference on Interactive Collaborative Learning (ICL), pp. 633–651. IEEE, December 2014Google Scholar
  4. 4.
    Fahmy, A., Said, Y., Shawky, D., Badawi, A.: Collaborate-it: a tool for promoting knowledge building in face-to-face collaborative learning. In: 2016 15th International Conference on Information Technology Based Higher Education and Training (ITHET), pp. 1–6. IEEE, September 2016Google Scholar
  5. 5.
    Ashraf, B., Doaa, S.: The need for a paradigm shift in CSCL. In: The Computing Conference 2017. IEEE, London (2017)Google Scholar
  6. 6.
    Said, T., Shawky, D., Badawi, A.: Identifying knowledge-building phases in computer supported collaborative learning: a review. In: 2015 International Conference on Interactive Collaborative Learning (ICL), pp. 608–614. IEEE (2015)Google Scholar
  7. 7.
    Taraman, S., et al.: Employing game theory and multilevel analysis to predict the factors that affect collaborative learning outcomes: an empirical study. arXiv:1610.05075 (2017)
  8. 8.
    Bingham, A.J., et al.: Ahead of the curve: implementation challenges in personalized learning school models. Educ. Policy (2016)Google Scholar
  9. 9.
    Chrysafiadi, K., Virvou, M.: Student modeling for personalized education: a review of the literature. In: Advances in Personalized Web-Based Education, pp. 1–24. Springer (2015)Google Scholar
  10. 10.
    Lu, J.: A personalized e-learning material recommender system. In: International Conference on Information Technology and Application. Macquarie Scientific Publishing (2004)Google Scholar
  11. 11.
    Ruggieri, S.: Efficient C4. 5 classification algorithm. IEEE Trans. Knowl. Data Eng. 14(2), 438–444 (2002)CrossRefGoogle Scholar
  12. 12.
    Ouf, S., et al.: A proposed paradigm for smart learning environment based on semantic web. Comput. Hum. Behav. 72, 796–818 (2017)CrossRefGoogle Scholar
  13. 13.
    Christudas, B.C.L., Kirubakaran, E., Thangaiah, P.R.J.: An evolutionary approach for personalization of content delivery in e-learning systems based on learner behavior forcing compatibility of learning materials. Telemat. Inform. (2015)Google Scholar
  14. 14.
    Garrido, A., Morales, L., Serina, I.: On the use of case-based planning for e-learning personalization. Expert Syst. Appl. 60, 1–15 (2016)CrossRefGoogle Scholar
  15. 15.
    Gavriushenko, M., Khriyenko, O., Tuhkala, A.: An intelligent learning support system. In: CSEDU 2017—Proceedings of the 9th International Conference on Computer Supported Education (2017)Google Scholar
  16. 16.
    Idris, N., et al.: Intelligent learning model based on significant weight of domain knowledge concept for adaptive e-learning. Int. J. Adv. Sci. Eng. Inf. Technol. 7(4–2 Special Issue), 1486–1491 (2017)Google Scholar
  17. 17.
    Pardos, Z.A., et al.: Enabling real-time adaptivity in MOOCs with a personalized next-step recommendation framework. In: Proceedings of the 4th (2017) ACM Conference on Learning at Scale (2017)Google Scholar
  18. 18.
    Liu, M., et al.: Using data to understand how to better design adaptive learning. Technol. Knowl. Learn. 22(3), 271–298 (2017)CrossRefGoogle Scholar
  19. 19.
    Zhou, Y., et al.: Monitoring cognitive workload in online videos learning through an EEG-based brain-computer interface. In: Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), pp. 64–73 (2017)Google Scholar
  20. 20.
    Tam, V., Gupta, M.: Facilitating the open learning and education through facial analytics and video streaming. In: 2017 IEEE 17th International Conference on Advanced Learning Technologies (ICALT), pp. 31–33. IEEE (2017)Google Scholar
  21. 21.
    Mavratzakis, A., Herbert, C., Walla, P.: Emotional facial expressions evoke faster orienting responses, but weaker emotional responses at neural and behavioural levels compared to scenes: a simultaneous EEG and facial EMG study. NeuroImage 124, 931–946 (2016)CrossRefGoogle Scholar
  22. 22.
    Dmochowski, J.P., et al.: Audience preferences are predicted by temporal reliability of neural processing. Nat. Commun. 5, 4567 (2014)CrossRefGoogle Scholar
  23. 23.
    Poulsen, A.T., et al.: EEG in the classroom: synchronised neural recordings during video presentation. Sci. Rep. 7, 43916 (2017)CrossRefGoogle Scholar
  24. 24.
    Hettich, D.T., et al.: EEG responses to auditory stimuli for automatic affect recognition. Front. Neurosci. 10, 244 (2016)CrossRefGoogle Scholar
  25. 25.
    Soleymani, M., et al.: Analysis of EEG signals and facial expressions for continuous emotion detection. IEEE Trans. Affect. Comput. 7(1), 17–28 (2016)CrossRefGoogle Scholar
  26. 26.
    DeMink-Carthew, J., et al.: An analysis of approaches to goal setting in middle grades personalized learning environments. RMLE Online 40(10), 1–11 (2017)CrossRefGoogle Scholar
  27. 27.
    Sahin, M., Kisla, T.: An analysis of university students’ attitudes towards personalized learning environments. Tur. Online J. Educ. Technol. TOJET 15(1), 1–10 (2016)Google Scholar
  28. 28.
    Anand, V.: Design considerations for personalized learning. In: International Conference on Research into Design, pp. 969–985. Springer, SingaporeGoogle Scholar
  29. 29.
    Sutton, R.S., Barto, A.G.: Reinforcement Learning: An Introduction, vol. 1. MIT Press, Cambridge (1998)Google Scholar
  30. 30.
    Tsitsiklis, J.N., Van Roy, B.: Analysis of temporal-difference learning with function approximation. In: Advances in Neural Information Processing Systems, pp. 1075–1081 (1997)Google Scholar
  31. 31.
    Watkins, C.J.C.H., Dayan, P.: Q-learning. Mach. Learn. 8(3–4), 279–292 (1992)zbMATHGoogle Scholar
  32. 32.
    Wiersema, M.F., Bantel, K.A.: Top management team demography and corporate strategic change. Acad. Manag. J. 35(1), 91–121 (1992)Google Scholar
  33. 33.
    Ausubel, D.P., Novak, J.D., Hanesian, H.: Educational psychology: a cognitive view (1978)Google Scholar
  34. 34.
    Carr, T.H., Curran, T.: Cognitive factors in learning about structured sequences: applications to syntax. Stud. Second Lang. Acquis. 16(2), 205–230 (1994)CrossRefGoogle Scholar
  35. 35.
    Wu, J.-H., Tennyson, R.D., Hsia, T.-L.: A study of student satisfaction in a blended e-learning system environment. Comput. Educ. 55(1), 155–164 (2010)CrossRefGoogle Scholar
  36. 36.
    Vasileva-Stojanovska, T., et al.: Impact of satisfaction, personality and learning style on educational outcomes in a blended learning environment. Learn. Individ. Differ. 38, 127–135 (2015)CrossRefGoogle Scholar
  37. 37.
    Chamorro-Premuzic, T., Furnham, A.: Personality, intelligence and approaches to learning as predictors of academic performance. Pers. Individ. Differ. 44(7), 1596–1603 (2008)CrossRefGoogle Scholar
  38. 38.
    Eom, S.B., Wen, H.J., Ashill, N.: The determinants of students’ perceived learning outcomes and satisfaction in university online education: an empirical investigation. Decis. Sci. J. Innov. Educ. 4(2), 215–235 (2006)CrossRefGoogle Scholar
  39. 39.
    Gagne, R.M.: Learning outcomes and their effects: useful categories of human performance. Am. Psychol. 39(4), 377 (1984)CrossRefGoogle Scholar
  40. 40.
    Dika, S.L., Singh, K.: Applications of social capital in educational literature: a critical synthesis. Rev. Educ. Res. 72(1), 31–60 (2002)CrossRefGoogle Scholar
  41. 41.
    Van den Bossche, P., et al.: Social and cognitive factors driving teamwork in collaborative learning environments: team learning beliefs and behaviors. Small Group Res. 37(5), 490–521 (2006)CrossRefGoogle Scholar
  42. 42.
    Song, Y., Looi, C.-K.: Linking teacher beliefs, practices and student inquiry-based learning in a CSCL environment: a tale of two teachers. Int. J. Comput. Support. Collab. Learn. 7(1), 129–159 (2012)CrossRefGoogle Scholar
  43. 43.
    Clark, H.: Building Education: The Role of the Physical Environment in Enhancing Teaching and Research. Issues in Practice. ERIC (2002)Google Scholar
  44. 44.
    Shawky, D., Badawi, A.: A reinforcement learning-based adaptive learning system. In: International Conference on Advanced Machine Learning Technologies and Applications, pp. 221–231. Springer, Cham (2018)Google Scholar
  45. 45.
    Dishon, G.: New data, old tensions: big data, personalized learning, and the challenges of progressive education. Theory Res. Educ. 15(3), 272–289 (2017)CrossRefGoogle Scholar
  46. 46.
    Aguilar, S.J.: Learning Analytics: At the Nexus of Big Data, Digital Innovation, and Social Justice in Education, pp. 1–9. TechTrends (2017)Google Scholar
  47. 47.
    Ardchir, S., Talhaoui, M.A., Azzouazi, M.: Towards an adaptive learning framework for MOOCs. In: Lecture Notes in Business Information Processing, pp. 236–251 (2017)Google Scholar
  48. 48.
    Lynch, C.F.: Who prophets from big data in education? New insights and new challenges. Theory Res. Educ. 15(3), 249–271 (2017)CrossRefGoogle Scholar
  49. 49.
    Siemens, G.: What are learning analytics? (2016). Accessed 1 Oct 2015

Copyright information

© Springer Nature Switzerland AG 2019

Authors and Affiliations

  1. 1.Faculty of Engineering, Engineering Mathematics DepartmentCairo UniversityGizaEgypt
  2. 2.Center for Learning Technologies, Zewail City of Science and TechnologyGizaEgypt

Personalised recommendations