Skip to main content
SpringerLink
Log in

Adaptive neural controller for visual servoing of robot manipulators with camera-in-hand configuration

  • Published:
Journal of Mechanical Science and Technology Aims and scope Submit manuscript

Abstract

In this paper, an adaptive neural controller is proposed for visual servoing of robot manipulators with camera-in-hand configuration. The controller is designed as a combination of a PI kinematic controller and feedforward neural network controller that computes the required torque signals to achieve the tracking. The visual information is provided using the camera mounted on the end-effector and the defined error between the actual image and desired image positions is fed to the PI controller that computes the joint velocity inputs needed to drive errors in the image plane to zero. Then the feedforward neural network controller is designed such that the robot’s joint velocities converges to the given velocity inputs. The stability of combined PI kinematic and feedforward neural network computed torque is proved by Lyapunov theory. It is shown that the neural network can cope with the unknown nonlinearities through the adaptive learning process and requires no preliminary off learning. Simulation results are carried out for a three degrees of freedom microbot robot manipulator to evaluate the controller performance.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. P. K. Allen, B. Yoshimi and A. Timcenko, Real-time visual servoing, Proc. IEEE Inter. Conf. Robot. Autom (1991) 851–856.

  2. A. J. Koivo and N. Houshangi, Real-time vision feedback for servoing of a robotic manipulator with self-tuning controller, IEEE Trans. Sys. Man Cyber, 21(1) (1991) 134–142.

    Article  Google Scholar 

  3. N. P. Papanikolopoulos and P. K. Khosla, Featurebasedrobotic visual tracking of 3-D translational motion, Proc. 30th Conf. Decision Control, Brighton, England (1991) 1877–1882.

  4. N. P. Papanikolopoulos, B. Nelson and P. K. Khosla, Monocular 3-D visual tracking of a moving target by an eye-inhand robotic system, Proc. 31st Conf. Decision Control, Tucson-Arizona (1992) 3805–3810.

  5. K. Hashimoto, T. Ebine and H. Kimura, Visual servoing with hand-eye manipulator: optimal control approach, IEEE Trans. Robot. Autom., 12(5) (1996) 766–774.

    Article  Google Scholar 

  6. S. Hutchinson, G. D. Hager and P. I. Corke, A tutorial on visual servo control, IEEE Trans. Robot. Automa., 12(5) (1996) 651–670.

    Article  Google Scholar 

  7. L. Weiss, A. C. Sanderson and C. P. Neuman, Dynamic sensor based control of robots with visual feedback, IEEE Journ. Robot. Autom. RA-3(5) (1987) 404–417.

    Article  Google Scholar 

  8. J. T. Feddema and O. R. Mitchell, Vision-guided servoing with feature-based trajectory generation, IEEE Trans. Robot. Autom., 5(5) (1989) 691–700.

    Article  Google Scholar 

  9. N. P. Papanikolopoulos, P. K. Khosla and T. Kanade, Visual tracking of a moving target by a camera mounted on a robot: A combination of control and vision, IEEE Trans. Robotic. Autom., 9(1) (1993) 14–35.

    Article  Google Scholar 

  10. N. P. Papanikolopoulos and P. K. Khosla, Adaptive robotic visual tracking: Theory and experiments, IEEE Trans. Automatic Control, 38(3) (1993) 429–445.

    Article  MathSciNet  MATH  Google Scholar 

  11. M. Asada, T. Tanaka and K. Hosoda, Adaptive binocular visual servoing for independently moving target tracking, Proc. IEEE Inter. Conf. Robotic Autom., San Francisco, CA (2000) 2076–2081.

  12. L. Hsu, R. Costa and P. Aquino, Stable adaptive visual servoing for moving targets, Proc. American Contr. Conf., Chicago, Illinois (2000) 2008–2012.

  13. A. Astoli, L. Hsu, M. Netto and R. Ortega, A solution to the adaptive visual servoing problem, Proc. IEEE Inter. Conf. Robot. Autom., Seoul, Korea (2001) 743.748.

  14. O. Nasisi and R. Carelli, Adaptive servo visual robot control, Robot. Autonomous Sys., 43 (2003) 51–78.

    Article  Google Scholar 

  15. E. Zergeroglu, D. M. Dawson, M. Queiroz and A. Behal, Vision-based nonlinear tracking controller s with uncertain robot-camera parameters, IEEE/ASME Trans. Mechatronics, 6(3) (2001) 322–337.

    Article  Google Scholar 

  16. Y. Liu, H. Wang, C. Wang and K. Lam, Uncalibrated visual servoing of robots using a depth-independent interaction matrix. IEEE Trans. Robotics, 22(4) (2006) 804–817.

    Article  Google Scholar 

  17. C. Kim, E. Mo, S. Han, M. Jie and K. Lee, Image-based robust control of robot manipulators with Ima ge Jacobian and dynamics uncertainties, Proc. 4th IE EE Conference on Automation Science and Engineering, Washington DC, USA (2008) 732–737.

  18. G. J. Garcia, C. A. Jara, J. Pomares and F. Torres, Direct visal servo control of a robot to track trajectories in supervision tasks, Proc. 11 Int. Conf. Control, Automation, Robotics and Vision, Singapore (2010) 1434–1439.

  19. A. Ishiguro, T. Furuhashi, S. Okuma and Y Uchikawa, A neural network compensator for uncertainties of robot manipulators, IEEE Trans. Indust. Electron, 39(6) (1992) 565–570.

    Article  Google Scholar 

  20. F. L. Lewis, S. Jagannathan and A. Yesildirek, Neural network control of robot manipulators and nonlinear systems, Taylor & Francis (1999).

  21. F. L. Lewis, K. Liu and A. Yesildirek, Neural net robot controller with guaranteed tracking performance, IEEE Trans. Neural Networks, 6(3) (1995) 703–715.

    Article  Google Scholar 

  22. X. Ren, F. L. Lewis and J. Zhang, Neural network compensation control for mechanical systems with disturbances. Automatica, 45 (2009) 1221–1226.

    Article  MathSciNet  MATH  Google Scholar 

  23. H. Hashimoto, T. Kubota, M. Sato and F. Harashima, Visual control of robotic manipulators based on neural networks, IEEE Trans. Indust. Electron, 39(6) (1992) 490–496.

    Article  Google Scholar 

  24. R. Garcia-Rodriguez, E. Dean-Leon, V. Parra-Vega, F. Ruiz-Sanchez, An adaptive neural network controller for visual tracking of constrained robot manipulators, Proc. American Control Conference, Portland, OR, USA (2005) 3694–3700.

  25. W. Yu and M. Moreno-Armendariz, Robust visual servoing of robot manipulators with neuro compensation, Journal of the Franklin Institute, 342 (2005) 824–838.

    Article  MathSciNet  MATH  Google Scholar 

  26. Y. Zhao and C. C. Cheah, Vision based neural network control for constrained robots with constraint uncertainty, IET Control Theory and Applications, 2(10) (2008) 906–916.

    Article  MathSciNet  Google Scholar 

  27. J. Feddema, C. Lee and O. R. Mitchell, Weighted selection of image features for resolved rate visual feedback control, IEEE Trans. Robot. Autom, 7 (1991) 31–47.

    Article  Google Scholar 

  28. J. J. Craig, Introduction to robotics mechanics and control, Addison-Wesley (1986).

  29. K. Hashimoto, T. Kimoto, T. Ebine and H. Kimura, Manipulator Control with Image Based Visual Servo, Proc. IEEE Inter. Conf. Robot. Autom, Sacramento, California (1991) 2267–2272.

  30. M. H. Hassoun, Fundamentals of artificial neural networks. Prentice-Hall, India (1998).

    Google Scholar 

  31. L. Sciavicco and B. Siciliano, Modeling and control of robot manipulators, McGraw-Hill (1996).

  32. W. A. Wolovich, Robotics: Basic analysis and design, Holt Reinehart and Winston (1987).

Download references

Author information

Authors and Affiliations

  1. Division of Mechanical Engineering, Ajou University, Suwon, 443-749, Korea

    Jungmin Kim, Naveen Kumar, Jin-Hwan Borm & Jangbom Chai

  2. Department of Applied Mathematics, Defense Institute of Advanced Technology, Pune, 411025, Maharashtra, India

    Vikas Panwar

Authors
  1. Jungmin Kim
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Naveen Kumar
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Vikas Panwar
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Jin-Hwan Borm
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Jangbom Chai
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Naveen Kumar.

Additional information

Recommended by Associate Editor Cong Wang

Jungmin Kim received his Master’s degree in Mechanical Engineering from the Ajou University in 2004. He is currently a Ph.D student in Mechanical Engineering of Ajou University in Korea. His area of research includes robotics and virtual manufacturing system.

Naveen Kumar received his Master’s Degree in Mathematics from CCS University Meerut India, in 2001. He completed Ph.D from Indian Institute of Technology Roorkee, India in 2009. Presently he is postdoctoral fellow at the Department of Mechanical Engineering, Ajou University, Korea. His research interests include robot dynamics and control, soft computing techniques and reliability analysis.

Vikas Panwar received B.Sc. degree from CCS University Meerut in 1998 and M.Sc. degree in Applied Mathematics form IIT Roorkee in 2000. He completed his Ph.D. from Indian Institute of Technology Roorkee, in 2006. From July 2004 to March 2010, he was Lecturer in the Department of Mathematics, CDL University Sirsa. He worked in the Department of Applied Mathematics, Defense Institute of Advanced Technology Pune from April 2010 to August 2010. Currently he is Assistant Professor in School of Applied Sciences, Gautam Buddha University Greater Noida. His research interests include neural network control, fuzzy control for robotic systems and nonlinear systems.

Jin-Hwan Borm received his M.S. and Ph.D in Mechanical Engineering from the Ohio State University in 1985 and 1988 respectively. He is currently a professor in the Department of Mechanical Engineering of Ajou University in Korea. His area of research includes robotics and virtual manufacturing system.

Jangbom Chai received his Ph.D in Mechanical Engineering from Massachusetts Institute of Technology in 1993. He is currently a professor in the Department of Mechanical Engineering of Ajou University in Korea. His area of research includes dynamics and diagnostics.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Kim, J., Kumar, N., Panwar, V. et al. Adaptive neural controller for visual servoing of robot manipulators with camera-in-hand configuration. J Mech Sci Technol 26, 2313–2323 (2012). https://doi.org/10.1007/s12206-012-0610-5

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s12206-012-0610-5

Keywords

Search

Navigation