Grey Reinforcement Learning for Incomplete Information Processing

  • Chunlin Chen
  • Daoyi Dong
  • Zonghai Chen
Part of the Lecture Notes in Computer Science book series (LNCS, volume 3959)


New representation and computation mechanisms are key approaches for learning problems with incomplete information or in large probabilistic environments. In this paper, traditional reinforcement learning (RL) methods are combined with grey theory and a novel grey reinforcement learning (GRL) framework is proposed to solve complex problems with incomplete information. Typical example of mobile robot navigation is given out to evaluate the performance and practicability of GRL. Related issues are also briefly discussed.


Mobile Robot Reinforcement Learning Markov Decision Process Grey Model Partially Observable Markov Decision Process 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Sutton, R., Barto, A.G.: Reinforcement Learning: An Introduction. MIT Press, Cambridge (1998)Google Scholar
  2. 2.
    Kaelbling, L.P., Littman, M.L., Moore, A.W.: Reinforcement Learning: A Survey. Journal of Artificial Intelligence Research 4, 237–287 (1996)Google Scholar
  3. 3.
    Sutton, R.: Learning to Predict by the Methods of Temporal Difference. Machine Learning 3, 9–44 (1988)Google Scholar
  4. 4.
    Bertsekas, D. P., Tsitsiklis, J. N.: Neuro-Dynamic Programming. Athena Scientific, Belmont, MA(1996) MATHGoogle Scholar
  5. 5.
    Watkins, C., Dayan, P.: Q-learning. Machine Learning 8, 279–292 (1992)MATHGoogle Scholar
  6. 6.
    Santamaría, J., Sutton, R., Ram, A.: Experiments with Reinforcement Learning in Problems with Continuous State and Action Spaces. Adaptive Behavior 6, 163–217 (1997)CrossRefGoogle Scholar
  7. 7.
    Smart, W.D., Kaelbling, L.P.: Effective reinforcement learning for mobile robots. In: Proceedings of the IEEE International Conference on Robotics and Automation, pp. 3404–3410. IEEE Press, Los Alamitos (2002)Google Scholar
  8. 8.
    Kondo, T., Ito, K.: A reinforcement learning with evolutionary state recruitment strategy for autonomous mobile robots control. Robotics and Autonomous Systems 46, 111–124 (2004)CrossRefGoogle Scholar
  9. 9.
    Beom, H.R., Cho, H.S.: A sensor-based navigation for a mobile robot using fuzzy logic and reinforcement learning. IEEE Transactions on Systems Man and Cybernetics 25, 464–477 (1995)CrossRefGoogle Scholar
  10. 10.
    Wiering, M., Schmidhuber, J.: HQ-Learning. Adaptive Behavior 6, 219–246 (1997)CrossRefGoogle Scholar
  11. 11.
    Barto, A.G., Mahanevan, S.: RecentAdvances in Hierarchical Reinforcement Learning. Discrete Event Dynamic Systems: Theory and Applications 13, 41–77 (2003)MATHCrossRefMathSciNetGoogle Scholar
  12. 12.
    Sutton, R., Precup, D., Singh, S.: Between Mdps and Semi-mdps: A Framework for Temporal Abstraction in Reinforcement Learning. Artificial Intelligence 112, 181–211 (1999)MATHCrossRefMathSciNetGoogle Scholar
  13. 13.
    Parr, R., Russell, S.: Reinforcement learning with hierarchies of machines. In: Proceedings of Advances in Neural Information Processing Systems (NIPS), pp. 1043–1049. MIT Press, Cambridge (1998)Google Scholar
  14. 14.
    Dietterich, T.G.: Hierarchical Reinforcement Learning with the Maxq Value Function Decomposition. Journal of Artificial Intelligence Research 13, 227–303 (2000)MATHMathSciNetGoogle Scholar
  15. 15.
    Theocharous, G.: Hierarchical Learning and Planning in Partially Observable Markov Decision Processes. Department of Computer Science and Engineering, Michigan State University, USA (2002)Google Scholar
  16. 16.
    Dong, D.Y., Chen, C.L., Chen, Z.H.: Quantum reinforcement learning. In: Natural Computation-ICNC2005. LNCS, vol. 3611, pp. 686–689. Springer, Heidelberg (2005)CrossRefGoogle Scholar
  17. 17.
    Smith, A.J.: Applications of the self-organising map to reinforcement learning. Neural Networks 15, 1107–1124 (2002)CrossRefGoogle Scholar
  18. 18.
    Glorennec, P.Y., Jouffe, L.: Fuzzy Q-learning. In: Proceedings of the Sixth IEEE International Conference on Fuzzy Systems, pp. 659–662. IEEE Press, Los Alamitos (1997)CrossRefGoogle Scholar
  19. 19.
    Dong, D.Y., Chen, C.L., Zhang, C.B., Chen, Z.H.: An Autonomous Mobile Robot Based on Quantum Algorithm. In: CIS 2005. LNCS (LNAI), vol. 3801, pp. 394–399. Springer, Heidelberg (2005)Google Scholar
  20. 20.
    Likas, A.: Reinforcement learning using the stochastic fuzzy min-max neural network. Neural Processing Letters 13, 213–220 (2001)MATHCrossRefGoogle Scholar
  21. 21.
    Deng, J. L.: Elements on Grey Theory. HUST Press, Wuhan, China (2002) (in Chinese) Google Scholar
  22. 22.
    Kuipers, B.: Qualitative simulation. Artificial Intelligence, 289–338 (1986)Google Scholar
  23. 23.
    Berleant, D., Kuipers, B.: Qualitative and quantitative simulation: bridging the gap. Artificial Intelligence 95, 215–255 (1997)MATHCrossRefGoogle Scholar
  24. 24.
    Huang, Y.L.: Research on Grey Qualitative Simulation Base. University of Science and Technology of China (2004) (in Chinese)Google Scholar
  25. 25.
    Huang, Y.L., Chen, Z.H., Gui, W.S.: Grey Qualitative Simulation. The Journal of Grey System 1, 5–20 (2004)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2006

Authors and Affiliations

  • Chunlin Chen
    • 1
  • Daoyi Dong
    • 1
  • Zonghai Chen
    • 1
  1. 1.Department of AutomationUniversity of Science and Technology of ChinaHefei, AnhuiP.R. China

Personalised recommendations