Fuzzy-Q Knowledge Sharing Techniques with Expertness Measures: Comparison and Analysis

  • Panrasee Ritthipravat
  • Thavida Maneewarn
  • Jeremy Wyatt
  • Djitt Laowattana
Part of the Lecture Notes in Computer Science book series (LNCS, volume 3967)


Four knowledge sharing techniques based on fuzzy-Q learning are investigated in this paper. These knowledge sharing techniques are ‘Shared Memory’, ‘Adaptive Weighted Strategy Sharing’, ‘Exploration Guided Method’, and ‘Greatest Mass Method’. Different robot expertness measures are applied to these knowledge sharing techniques in order to improve learning performance. We proposed a new robot expertness measure based on regret evaluation. The regret takes uncertainty bounds of two best actions, i.e. greedy action and the second best action, into account. Simulations were performed to compare the effectiveness of the three expertness measures i.e. expertness based on accumulated rewards, on average move and on regret measure, when applied to different sharing techniques. Our proposed measure resulted in better performance than the other expertness measures. Analysis and comparison of different knowledge sharing techniques are also provided herein.


Knowledge Sharing Knowledge Source Probability Mass Function Reinforcement Learn Agent External Knowledge Source 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Tan, M.: Multi-agent reinforcement learning: Independent vs cooperative agents. In: Proc. 10th Int. Conf. Machine Learning (1993)Google Scholar
  2. 2.
    Ahmadabadi, M.N., Asadpour, M.: Expertness Based Cooperative Q-Learning. IEEE Trans. SMC–Part B 32(1), 66–76 (2002)Google Scholar
  3. 3.
    Ahmadabadi, M.N., Asadpour, M., Khodanbakhsh, S.H., Nakano, E.: Expertness Measuring in Cooperative Learning. In: IEEE Int. Conf. on IROS, pp. 2261–2267 (2000)Google Scholar
  4. 4.
    Ahmadabadi, M.N., Asadpour, M., Nakano, E.: Cooperative Q-learning: The Knowledge Sharing Issue. J. Adv. Robotics. 15(8), 815–832 (2002)CrossRefGoogle Scholar
  5. 5.
    Bitaghsir, A.A., Moghimi, A., Lesani, M., Keramati, M.M., Ahmadabadi, M.N., Arabi, B.N.: Successful Cooperation between Heterogeneous Fuzzy Q-learning Agents. In: IEEE Int. Conf. SMC (2004)Google Scholar
  6. 6.
    Dixon, K.R., Malak, R.J., Khosla, P.K.: Incorporating Prior Knowledge and Previously Learned Information into Reinforcement Learning Agents. Tech. report. Institute for Complex Engineered Systems, Carnegie Mellon University (2000)Google Scholar
  7. 7.
    Malak, R.J., Khosla, P.K.: A framework for the Adaptive Transfer of Robot Skill Knowledge Using Reinforcement Learning Agents. In: Proc. IEEE Int. Conf. ICRA (2001)Google Scholar
  8. 8.
    Moreno, D.L., Regueiro, C.B., Iglesias, R., Barro, S.: Using Prior Knowledge to Improve Reinforcement Learning in Mobile Robotics. In: Proc. Towards Autonomous Robotics Systems. Univ. of Essex, UK (2004)Google Scholar
  9. 9.
    Ritthipravat, P., Maneewarn, T., Laowattana, D., Wyatt, J.: A Modified Approach to Fuzzy Q-Learning for Mobile Robots. In: Proc. IEEE Int. Conf. SMC (2004)Google Scholar
  10. 10.
    Ritthipravat, P., Maneewarn, T., Wyatt, J.C., Laowattana, D.: Comparison and analysis of expertness measure in knowledge sharing among robots. In: Ali, M., Dapoigny, R. (eds.) IEA/AIE 2006. LNCS, vol. 4031, pp. 60–69. Springer, Heidelberg (2006)CrossRefGoogle Scholar
  11. 11.
    Peterson, T.S., Owens, N.E., Carroll, J.L.: Towards Automatic Shaping in Robot Navigation. In: Proc. IEEE Int. Conf. ICRA (2001)Google Scholar
  12. 12.
    Carroll, J.L., Peterson, T.S., Owens, N.E.: Memory-guided Exploration in Reinforcement Learning. In: Proc. IEEE Int. Conf. IJCNN (2001)Google Scholar
  13. 13.
    Whitehead, S., Karlsson, J., Karlsson, J.: Learning Multiple Goal Behavior via Task Decomposition and Dynamic Policy Merging. In: Connell, J.H., Mahadevan, S. (eds.) Robot Learning, Kluwer Academic Publishers, Norwell, MA (1993)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2006

Authors and Affiliations

  • Panrasee Ritthipravat
    • 1
  • Thavida Maneewarn
    • 1
  • Jeremy Wyatt
    • 2
  • Djitt Laowattana
    • 1
  1. 1.Institute of FIeld roBOtics (FIBO)King Mongkut’s University of TechnologyBangkokThailand
  2. 2.School of Computer ScienceUniversity of BirminghamBirminghamUnited Kingdom

Personalised recommendations