Combining the Best of the Two Worlds: Inheritance Versus Experience

Evolutionary Knowledge-Based Control and Q-Learning to Solve Autonomous Robots Motion Control Problems
  • Darío Maravall
  • Javier de Lope
  • José Antonio Martín H.
Part of the Lecture Notes in Computer Science book series (LNCS, volume 4528)


In this paper a hybrid approach to the autonomous navigation of robots in cluttered environment with unknown obstacles is introduced. It is shown the efficiency of the hybrid solution by combining the optimization power of evolutionary algorithms and at the same time the efficiency of the Reinforcement Learning in real-time and on-line situations. Experimental results concerning the navigation of a L-shaped robot in a cluttered environment with unknown obstacles in which appear real-time and on-line constraints well-suited to RL algorithms and extremely high dimension of the state space usually unpractical for RL algorithms but at the same time well-suited to evolutionary algorithms, are also presented. The experimental results confirm the validity of the hybrid approach to solve hard real-time, on-line and high dimensional robot motion control problems.


Evolutionary Algorithm Reinforcement Learn Markov Decision Process Control Rule Goal Position 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Sutton, R.S., Barto, A.G.: Reinforcement Learning: An Introduction. MIT Press, Cambridge (1998)Google Scholar
  2. 2.
    Wang, P.P.: Computing with Words. Wiley Interscience, New York (2001)Google Scholar
  3. 3.
    Watkins, C.J.: Models of Delayed Reinforcement Learning. PhD Thesis Dissertation, Psychology Department, Cambridge University, Cambridge, UK (1989)Google Scholar
  4. 4.
    Watkins, C.J., Dayan, P.: Technical note Q-learning. Machine Learning 8, 279 (1992)zbMATHGoogle Scholar
  5. 5.
    Mendel, J., Wang, L.: Generating Fuzzy Rules by Learning Through Examples. IEEE Trans. on Systems, Man and Cybernetics 22, 1414–1427 (1992)CrossRefMathSciNetGoogle Scholar
  6. 6.
    Pedrycz, W.: Fuzzy Control and Fuzzy Systems, 2nd edn. Addison Wesley Longman, Menlo Park (1993)zbMATHGoogle Scholar
  7. 7.
    Holland, J.H.: Escaping brittleness: The possibilities of general purpose learning algorithms applied in parallel rule-based systems. In: Michaiski, R.S., Carbonell, J.G., Mitchell, T.M. (eds.) Machine Learning II, pp. 593–623 (1986)Google Scholar

Copyright information

© Springer Berlin Heidelberg 2007

Authors and Affiliations

  • Darío Maravall
    • 1
  • Javier de Lope
    • 2
  • José Antonio Martín H.
    • 3
  1. 1.Department of Artificial Intelligence, Universidad Politécnica de Madrid 
  2. 2.Department of Applied Intelligent Systems, Universidad Politécnica de Madrid 
  3. 3.Departamento de Sistemas Informáticos y Computación, Universidad Complutense de Madrid 

Personalised recommendations