Abstract
A new self-learning visual servoing system for the robot manipulators is proposed. This system includes two main properties: on-line self training and lifelong learning that are implemented by the Q-Learning algorithm and Explanation-based Fuzzy Neural Networks (EBFNN) respectively. We demonstrate that the number of training samples and the training time for a specific robot positioning accuracy can be reduced using explanation-based fuzzy neural networks and the Q-Learning algorithm. The system uses Q-learning to find the optimal policy in conjunction with the reinforcement learning. This policy is used by a robot to reach an object that has been randomly placed in a static workspace. Background knowledge about the robot and its environment is transferred to the robot agent during the learning process using a set of previously trained neural networks. This system learns the optimal policy in order to select the best action that maximizes the cumulative reward received at each time step. This learning approach does not use either a robot or camera model, or require calibration. Simulation results prove the effectiveness of this methodology to improve the learning process and the performance of the self-learning visual servoing system.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Mitchell, T., Thrun, S.: Explanation-based learning: A comparison of symbolic and neural net approaches. In: Proceedings of the 10th International Conference on Machine Learning, Morgan Kaufmann, pp. 197–204 (1993)
Mitchell, T.M.: Machine Learning. McGraw-Hill, New York (1997)
Christopher, J.C.H.: Q-Learning. Mach. Learn. 8, 279–292 (1992)
Hutchinson, S., Hager, G.D.: A tutorial on visual servo control. IEEE Trans. Robot. Autom. 12(5), 651–670 (1996)
Yang, Y.-X., Liu, D., Liu, H.: Robot end-effector 2D visual positioning using neural networks. In: Proceedings of the 2nd International Conferene on Machine Learning and Cybernetics, vol. 5, pp. 2940–2943 (2003)
Krose Ben, J.A., van der Krose, M.J., Groen Frans, C.A.: Learning strategies for a vision based neural controller for a robot arm. In: Proceedings of International IEEE Workshop on Intelligence Motion Control, pp. 1199–203 (1990)
Zhao, Q., Sun, Z. , Sun, F., Zhu, J.: Appearance-based robot visual servo via a wavelet neural network. Int. J. Control. Autom. Syst. 6(2), 607–612 (2008)
Nayar, S.K., Murase, H., Nene, S.A. : Learning, positioning, and tracking visual appearance. In: Proceedings of the IEEE International Conference on Robotics and Automation, vol. 4, pp. 3237–3244 (1994)
Dinh, B.H., Dunnigan, M.W. , Reay, D.S.: Position control of a robotic manipulator using a radial basis function network and simple vision system. In: IEEE Interantional Symposium on Industry Electronics, pp. 1371–1376. IEEE, Cambridge (2008). doi:10.1109/ISIE.2008.4677070
Zhang, J., Schmidt, R. , Knoll, A.: Appearance-based visual learning in neuro-fuzzy model for fine-positioning of manipulator. In: Proceedings of the IEEE International Conference on Roboics and Automation, vol. 2, pp. 1164–1169 (1999)
Hao, M. , Sun, Z., Fujii, M.: Image based visual servoing using Takagi-Sugeno fuzzy neural network controller. In: 22nd IEEE International Symposium on Intelligence. Control, pp. 53–58. IEEE, Singapore (2007). doi:10.1109/ISIC.2007.4450860
Ganapathy, V., Yun, S.C., Joe, H.K.: Neural Q-learning controller for mobile robot. IEEE/ASME International Conference on Advance Intelligence Mechatronics, pp. 863–868. 14–17 July 2009, IEEE, Singapore (2009). doi:10.1109/AIM.2009.5229901
Qiao, J., Hou, Z. , Ruan, X.: Q-learning based on neural network in learning action selection of mobile robot. In: Proceedings of the IEEE International Conference on Automation and Logis. 18–21 August 2007, pp. 263–267 (2007)
Wang, S.-C., Song, Z.-X., Ding, H. , Shi, H.-B.: An improved reinforcement Q-learning method with BP neural networks in robot soccer. Fourth International. Sympossium on Computer Intelligence and Design, vol. 1, 177–180. IEEE, Hangzhou (2011). doi:10.1109/ISCID.2011.53
Thrun, S., Mitchell, T.M.: Lifelong robot learning. The biology and technology of intelligent autonomous agents. NATO ASI Series 144, 165–196 (1995)
Huang, B., Cao, G., Guo, M.: Reinforcement learning neural network to the problem of autonomous mobile robot obstacle avoidance. Proc. of the 4th Int. Conf. on Machine Learn. and Cybern. Guangzhou (2005)
Cervera, E., del. Pobil, A.P. : Sensor-based learning for practical planning of fine motions in robotics. Inf. Sci. 145, 147–168 (2002)
Jyh-Shing, Jang, R.: ANFIS: Adaptive-network-based fuzzy inference system. IEEE Trans. on Syst., Man and Cybern. 23 (1993)
Craig, J.J.: Introduction to Robotics Mechanics and Control. Pearson Education Inc. 3rd Edition (2005)
Harris, C., Stephen, M.: A Combined Corner and Edge Detector. In: 4th Alvey Cision Conference., pp. 147–151. Plessey Research Roke Manor, United Kingdom (1988)
Madsen, K., Nielsen, H.B., Tingleff, O.: Methods for Non-linear Least Squares Problems, Second Edition, Technical University of Denmark (2004)
Geisser, S., Johnson, W.M.: Modes of Parametric Statistical Inference. John Wiley & Sons, New York (2006)
Dickinson, G.J., Subhabrata, C.: Nonparametric Statistical Inference, 4th Edition. CRC, FL (2003)
Rice, J.A. : Mathematical Statistical and Data Analysis, Third Editionm, Duxbury Advances (2006)
Hollander, M., Wolfe, D.A.: Nonparametric Statistical Methods. Second Edition, New York (1999)
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Sadeghzadeh, M., Calvert, D. & Abdullah, H.A. Self-Learning Visual Servoing of Robot Manipulator Using Explanation-Based Fuzzy Neural Networks and Q-Learning. J Intell Robot Syst 78, 83–104 (2015). https://doi.org/10.1007/s10846-014-0151-5
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10846-014-0151-5