Skip to main content

Advertisement

SpringerLink
Log in

Learning teaching strategies in an Adaptive and Intelligent Educational System through Reinforcement Learning

  • Published:
Applied Intelligence Aims and scope Submit manuscript

Abstract

One of the most important issues in Adaptive and Intelligent Educational Systems (AIES) is to define effective pedagogical policies for tutoring students according to their needs. This paper proposes to use Reinforcement Learning (RL) in the pedagogical module of an educational system so that the system learns automatically which is the best pedagogical policy for teaching students. One of the main characteristics of this approach is its ability to improve the pedagogical policy based only on acquired experience with other students with similar learning characteristics. In this paper we study the learning performance of the educational system through three important issues. Firstly, the learning convergence towards accurate pedagogical policies. Secondly, the role of exploration/exploitation strategies in the application of RL to AIES. Finally, a method for reducing the training phase of the AIES.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. Brusilovsky P (1999) Adaptive and intelligent technologies for Web-based education. Kunstl Intell 4:19–25. Special Issue on Intelligent Tutoring Systems and Teleteaching

    Google Scholar 

  2. Lebowitz M (1987) Experiments with incremental concept formation: Unimen. Mach Learn 2:103–138

    Google Scholar 

  3. Murray RC, VanLehn K, Mostow J (2001) A decision-theoretic approach for selecting tutorial discourse actions. In: Proc of the NAACL workshop on adaptation in dialogue systems, pp 41–48

  4. Iglesias A, Martínez P, Aler R, Fernando F (2002) Applying reinforcement learning in intelligent tutoring systems. In: Proc of international conference on new educational environments (ICNEE), Lugano (Switzerland), pp 11–14

  5. Sutton RS, Barto AG (1998) Reinforcement learning: an introduction. MIT Press, Cambridge

    Google Scholar 

  6. Wenger E (1987) Artificial intelligence and tutoring systems. Morgan Kaufmann, San Mateo

    Google Scholar 

  7. Murray T (1999) Authoring intelligent tutoring systems: an analysis of the state of the art. J Artif Intell Educ 10:98–129

    Google Scholar 

  8. Andre E, Finkler W, Graf W, Schauder A, Wahister W (1993) Intelligent multimedia presentations. In: Maybury MT (ed) The automatic synthesis of multimodal presentations. MIT Press, Cambridge

    Google Scholar 

  9. Khan T, Yip YJ (1996) Pedagogical principles of case-based cal. J Comput Assist Learn 1(12):172–192

    Article  Google Scholar 

  10. Sleeman A (1977) A system which allows student to explore algorithms. In: Proceedings of international joint conference on artificial intelligence, pp 780–786

  11. Woolf BP (1987) Representing complex knowledge in an intelligent machine tutor. Comput Intell 3:45–55

    Article  Google Scholar 

  12. Beck J (2001) ADVISOR: A machine learning architecture for intelligent tutor construction. PhD thesis, University of Massachusetts Amherst

  13. Burns H, Capps C (1998) Foundations of intelligent tutoring systems: an introduction. Foundations of intelligent tutoring systems. Lawrence Erlbaum Associates, Hillsdale, pp 1–19

    Google Scholar 

  14. Linton F, Schaefer HP (2000) Recommender systems for learning: building user and expert models through long-term observation of application use. User Model User-Adapt Interact 10:181–208

    Article  Google Scholar 

  15. Kaelbling LP, Littman ML, Moore AW (1996) Reinforcement learning: a survey. Int J Artif Intell Res 4:237–285

    Google Scholar 

  16. Puterman ML (1994) Markov decision processes: discrete stochastic dynamic programming. Wiley series in probability and mathematical statistics: Applied probability and statistics. Wiley, New York

    MATH  Google Scholar 

  17. Watkins CJCH (1989) Learning from delayed rewards. Phd Thesis, King’s College, Cambridge

  18. Thrun SB (1992) The role of exploration in learning control. In: White DA, Sofge DA (eds) Handbook of intelligent control: neural, fuzzy and adaptive approaches. Van Nostrand Reinhold, New York

    Google Scholar 

  19. Elmasri R, Navathe SB (1994) Fundamentals of database systems, 2nd edn. Benjamin/Cummings, Redwood City

    MATH  Google Scholar 

  20. Rich E (1979) User modelling via stereotypes. Cogn Sci 4:329–354

    Article  Google Scholar 

  21. Carr B, Goldstein I (1977) Overlays: A theory of modelling for computer aided instruction. Technical Report ai memo 406, AI Laboratory, Massachusetts Institute of Technology, Cambridge

  22. VanLehn K, Zhendong N (2001) Bayesian student modelling, user interfaces and feedback: a sensitivity analysis. Int J Artif Intell Educ 2:155–184

    Google Scholar 

  23. Iglesias A, Martínez P, Aler R, Fernando F (2002) Learning to teach database design by trial and error. In: Proc of 4th international conference on enterprise information systems (ICEIS), Ciudad Real (Spain), pp 500–505

  24. Sison R, Shimura M (1998) Student modeling and machine learning. Int J Artif Intell Educ 9:128–158

    Google Scholar 

  25. Quinlan JR (1993) C4.5 Programs for machine learning. Morgan Kaufmann, San Mateo

    Google Scholar 

  26. Ur S, VanLehn K (1995) STEPS: A simulated, tutorable physics student. J Artif Intell Educ 6(4):405–437

    Google Scholar 

  27. VanLehn K, Ohlsson S, Nason R (1994) Applications of simulated students: An exploration. J Artif Intell Educ 5(2):135–175

    Google Scholar 

  28. Montgomery DC (2005) Design and analysis of experiments, 5th edn. Wiley, New York

    MATH  Google Scholar 

  29. Brusilovsky P (1996) Methods and techniques of adaptive hipermedia. User Model User Adapt Interact 6(2–3):87–129

    Article  Google Scholar 

  30. Iglesias A, Martínez P, Aler R, Fernando F (2003) An experience applying reinforcement learning in a web-based adaptive and intelligent education system. Inform Educ Int J 2:1–18

    Google Scholar 

  31. Doak ED, Keith M (1986) Simulation in teacher education: The knowledge base and the process. Tenn Educ 16(2):14–17

    Google Scholar 

  32. Carroll JM, Peterson ST (2002) Fixed vs dynamic sub-transfer in reinforcement learning. In: Proc ICMLA 2002, M Arif Wani, Las Vegas, Nevada

  33. Stankov S, Božičević J (1998) The computer tutor in the new model of learning and teching control principles. In: Proc of 1st international workshop: mechatronics and industrial engineering, Bratislava, Slovak Republic, pp 78–88

  34. VanLehn K, Zhendong N (2001) Bayesian student modelling, user interfaces and feedback: a sensitivity analysis. Int J Artif Intell Educ 2:155–184

    Google Scholar 

  35. Aleven V, Koedinger KR (2000) The need for tutorial dialog to support self-explanation. In: Building dialogue systems for tutorial applications, papers of the 2000 AAAI Fall Symposium, pp 65–73

  36. Kumar R, Rosé CP, Aleven V, Iglesias A, Robinson A (2006) Evaluating the effectiveness of tutorial dialogue instruction in an exploratory learning context. Intell Tutor Syst 2066:666–674

    Article  Google Scholar 

  37. Brusilovsky P, Schwarz E, Weber G (1996) Elm-art: An intelligent tutoring system on world wide web. In: Frasson C, Gauthier G, Lesgold A (eds) Intelligent tutoring systems, vol 1086. Springer, Berlin, pp 261–269

    Google Scholar 

  38. André E, Müller J, Rist T (1996) WIP/PPP: Automatic generation of personalized multimedia presentations. In: Proc of Multimedia 96, 4th ACM International Multimedia Conference, Boston, pp 407–408

  39. Anderson J, Reiser B (1985) The lisp tutor. Byte 10(4):159–175

    Google Scholar 

  40. Prentzas J, Hatzilygeroudis I, Garofalakis J (2002) A web-based intelligent tutoring system using hybrid rules as its representational basis. In: Cerri SA, Gouarderes G, Paraguacu̧ F (eds) Proceedings of the 6th international conference, ITS 2002. Lecture notes in computer science, vol 1. Springer, Berlin, pp 119–128

    Google Scholar 

  41. Rich E (1979) User modelling via stereotypes. Cogn Sci 3(4):329–354

    Article  Google Scholar 

  42. Carr B, Goldstein I (1977) Overlays: A theory of modelling for computer aided instruction. Technical Report ai memo 406, AI Laboratory, Massachusetts Institute of Technology, Cambridge

  43. Langley P, Ohlsson S (1984) Automated cognitive modelling. In: Proceedings of the second national conference on artificial intelligence

  44. Baffes P, Mooney R (1996) Refinement-based student modelling and automated bug library construction. J Artif Intell Educ 7(1):75–116

    Google Scholar 

  45. Sison R, Nũmao M, Shimura M (2000) Multistrategy discovery and detection of novice programmer errors. Mach Learn 38:157–180

    Article  Google Scholar 

  46. Sleeman D, Brown S (1982) Intelligent tutoring systems. Computers and people series. Academic Press, London

    Google Scholar 

  47. Hoppe U (1994) Deductive error diagnosis and inductive error generalization for intelligent tutoring systems. J Artif Intell Educ 5:27–49

    Google Scholar 

  48. Beck J (2001) ADVISOR: A machine learning architecture for intelligent tutor construction. PhD thesis, University of Massachusetts Amherst

Download references

Author information

Authors and Affiliations

  1. Computer Science Department, Universidad Carlos III de Madrid, Avda. de la Universidad, 30, 28911, Leganés, Madrid, Spain

    Ana Iglesias, Paloma Martínez, Ricardo Aler & Fernando Fernández

Authors
  1. Ana Iglesias
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Paloma Martínez
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Ricardo Aler
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Fernando Fernández
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Ana Iglesias.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Iglesias, A., Martínez, P., Aler, R. et al. Learning teaching strategies in an Adaptive and Intelligent Educational System through Reinforcement Learning. Appl Intell 31, 89–106 (2009). https://doi.org/10.1007/s10489-008-0115-1

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10489-008-0115-1

Keywords

Search

Navigation