Skip to main content
SpringerLink
Log in

Visual servoing using an optimized trajectory planning technique for a 4 DOFs robotic manipulator

  • Regular Papers
  • Robot and Applications
  • Published:
International Journal of Control, Automation and Systems Aims and scope Submit manuscript

Abstract

Although visual servoing has been considered as a solution to increase dexterity and intelligence of the robotic systems specially in unstructured environments, some prominent deficiencies are preventing it from practical employment. Trajectory planning is a solution to overcome the shortcomings of visual servoing and makes it practical for industrial applications. In this paper, a new trajectory planning technique is developed to perform image-based visual servoing (IBVS) tasks for a 4 DOFs robotic manipulator system. In this method, the camera’s velocity screw is parameterized using time-based profiles. The parameters of the velocity profile are then determined such that the velocity profile takes the robot to its desired position. This is done by minimizing the errors between the initial and desired features. A depth estimation technique is proposed to provide the trajectory planning algorithm with an accurate initial depth. This algorithm is tested and validated via the experiment on a 4 DOFs Denso robot in an eye-in-hand configuration. Experimental results demonstrate that the proposed method provides with a reliable visual servoing algorithm by overcoming the IBVS drawbacks such as surpassing the system limits and causing instability of the system in fulfilling the tasks which require a 180° rotation of the camera about its center.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. F. Chaumette and S. Hutchinson, “Visual servo control. part i. basic approaches,” IEEE Transactions on Robotics and Automation, vol. 13, no. 4, pp. 82–90, Dec. 2006.

    Article  Google Scholar 

  2. L. Gracia and C. Perez-Vidal, “A new control scheme for visual servoing,” International Journal of Control, Automation and Systems, vol. 7, no. 5, pp. 764–776, 2009. [click]

    Article  Google Scholar 

  3. K. Hashimoto, “A review on vision-based control of robot manipulators,” Advanced Robotics, vol. 17, no. 10, pp. 969–991, 2003. [click]

    Article  Google Scholar 

  4. C.-S. Kim, E.-J. Mo, S.-M. Han, M.-S. Jie, and K.-W. Lee, “Robust visual servo control of robot manipulators with uncertain dynamics and camera parameters,” International Journal of Control, Automation and Systems, vol. 8, no. 2, pp. 308–313, 2010.

    Article  Google Scholar 

  5. H. Wang, Y.-H. Liu, and D. Zhou, “Adaptive visual servoing using point and line features with an uncalibrated eyein-hand camera,” IEEE Transactions on Robotics, vol. 24, no. 4, pp. 843–857, Aug. 2008.

    Article  Google Scholar 

  6. F. Chaumette, “Potential problems of stability and convergence in image-based and position-based visual servoing,” in The Confluence of Vision and Control, LNCIS Series, No 237, Springer-Verlag, 1998, pp. 66–78.

    Chapter  Google Scholar 

  7. F. Chaumete, “Image moments: a general and useful set of features for visual servoing,” IEEE Transactions on Robotics, vol. 20, no. 4, pp. 713–723, 2004.

    Article  Google Scholar 

  8. P. Corke and S. Hutchinson, “A new partitioned approach to image-based visual servo control,” IEEE Transactions on Robotics and Automation, vol. 17, no. 4, pp. 507–515, 2001.

    Article  Google Scholar 

  9. Y. Zhao, W.-F. Xie, and S. Liu, “Image-based visual servoing using improved image moments in 6-bof robot systems,” International Journal of Control, Automation and Systems, vol. 11, no. 3, pp. 586–596, 2013. [click]

    Article  Google Scholar 

  10. G. Allibert, E. Courtial, and F. Chaumette, “Predictive control for constrained image-based visual servoing,” IEEE Transactions on Robotics, vol. 26, no. 5, pp. 933–939, Oct. 2010.

    Article  Google Scholar 

  11. M. Keshmiri, W.-F. Xie, and A. Mohebbi, “Augmented image based visual servoing of a manipulator using acceleration command,” IEEE Transactions on Industrial Electronics, vol. 61, no. 9, Sept. 2014.

    Google Scholar 

  12. G. Chesi, “Visual servoing path planning via homogeneous forms and lmi optimizations,” IEEE Transactions on Robotics, vol. 25, no. 2, pp. 281–291, 2009.

    Article  Google Scholar 

  13. G. Chesi and Y. Hung, “Global path-planning for constrained and optimal visual servoing,” IEEE Transactions on Robotics, vol. 23, no. 5, pp. 1050–1060, 2007.

    Article  Google Scholar 

  14. M. Keshmiri, M. Keshmiri, and A. Mohebbi, “Augmented online point to point trajectory planning, a new approach in catching a moving object by a manipulator,” Proc. of 8th IEEE International Conference on Control and Automation (ICCA), pp. 1349–1354, 2010.

    Google Scholar 

  15. M. Keshmiri and M. Keshmiri, “Performance comparison of various navigation guidance methods in interception of a moving object by a serial manipulator considering its kinematic and dynamic limits,” Proc. of 15th International Conference on Methods and Models in Automation and Robotics (MMAR), pp. 212–217, 2010.

    Google Scholar 

  16. M. Keshmiri and W. F. Xie, “Catching moving objects using a navigation guidance technique in a robotic visual servoing system,” Proc. of American Control Conference (ACC), pp. 6302–6307, 2013.

    Google Scholar 

  17. A. Masoud and M. Bayoumi, “Intercepting a maneuvering target in a multidimensional stationary environment using a wave equation potential field strategy,” Proc. of IEEE International Symposium on Intelligent Control, pp. 243–248, 1994.

    Google Scholar 

  18. S. Masoud and A. Masoud, “Motion planning in the presence of directional and regional avoidance constraints using nonlinear, anisotropic, harmonic potential fields: a physical metaphor,” IEEE Transactions on Systems, Man and Cybernetics, Part A: Systems and Humans, vol. 32, no. 6, pp. 705–723, 2002.

    Article  Google Scholar 

  19. M. Kazemi, K. Gupta, and M. Mehrandezh, “Randomized kinodynamic planning for robust visual servoing,” IEEE Transactions on Robotics, vol. 29, no. 5, pp. 1197–1211, Oct. 2013.

    Article  Google Scholar 

  20. L. Deng, F. Janabi-Sharifi, and W. J. Wilson, “Hybrid motion control and planning strategies for visual servoing,” IEEE Transactions on Industrial Electronics, vol. 52, no. 4, pp. 1024–1040, 2005.

    Article  Google Scholar 

  21. Y. Mezouar and F. Chaumette, “Path planning for robust image-based control,” IEEE Transactions on Robotics and Automation, vol. 18, no. 4, pp. 534–549, 2002.

    Article  Google Scholar 

  22. Z. Zhang, “Camera parameters (intrinsic, extrinsic),” Computer Vision, K. Ikeuchi, Ed., Springer US, pp. 81–85, 2014.

    Chapter  Google Scholar 

  23. M. Keshmiri and W. F. Xie, “Augmented imaged based visual servoing controller for a 6 dof manipulator using acceleration command,” Proc. of IEEE 51st Annu. Conf. on Decision and Control (CDC), pp. 556–561, Dec. 2012.

    Google Scholar 

  24. B. Barrois and C. Wohler, “3d pose estimation of vehicles using stereo camera,” in Transportation Technologies for Sustainability, M. Ehsani, F.-Y. Wang, and G. Brosch, Eds. Springer New York, pp. 1–24, 2013.

    Chapter  Google Scholar 

  25. C. Strecha and L. Gool, “Motion stereo integration for depth estimation,” in Computer Vision, ECCV 2002, ser. Lecture Notes in Computer Science, A. Heyden, G. Sparr, M. Nielsen, and P. Johansen, Eds. Springer Berlin Heidelberg, vol. 2351, pp. 170–185, 2002.

    Article  MATH  Google Scholar 

  26. S. Zhang, C. Wang, and S. Chan, “A new high resolution depth map estimation system using stereo vision and kinect depth sensing,” Journal of Signal Processing Systems, vol. 79, no. 1, pp. 19–31, Apr. 2015.

    Article  Google Scholar 

  27. J. Arora, Introduction to Optimum Design, Elsevier Science, 2004.

    Google Scholar 

  28. J. W. Chinneck, “Discovering the characteristics of mathematical programs via sampling,” Optimization Methods and Software, vol. 17, no. 2, pp. 319–352, 2002. [click]

    Article  MathSciNet  MATH  Google Scholar 

  29. H. Scolnik and J. Gambini, “A new method to compute second derivatives,” Journal of Computer Science & Technology, vol. 2, no. 6, pp. 1–9, July 2001.

    Google Scholar 

  30. G. Allibert, E. Courtial, and F. Chaumette, “Visual servoing via nonlinear predictive control,” in Visual Servoing via Advanced Numerical Methods, ser. Lecture Notes in Control and Information Sciences, G. Chesi and K. Hashimoto, Eds. Springer London, vol. 401, pp. 375–393, 2010.

    Article  MathSciNet  Google Scholar 

  31. C. Harris and M. Stephens, “A combined corner and edge detector,” in Proc. of Fourth Alvey Vision Conference, pp. 147–151, 1988.

    Google Scholar 

  32. R. Byrd, M. Hribar, and J. Nocedal, “An interior point algorithm for large-scale nonlinear programming,” SIAM Journal on Optimization, vol. 9, no. 4, pp. 877–900, 1999. [click]

    Article  MathSciNet  MATH  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Mechanical and Industrial Engineering Department, Concordia University, 1455 De Maisonneuve Blvd. W., Montreal, Quebec, H3G 1M8, Canada

    Mohammad Keshmiri, Wen-Fang Xie & Ahmad Ghasemi

Authors
  1. Mohammad Keshmiri
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Wen-Fang Xie
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Ahmad Ghasemi
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Wen-Fang Xie.

Additional information

Recommended by Associate Editor Huaping Liu under the direction of Editor-in-Chief Young Hoon Joo. This project was funded by NSERC Discovery.

Mohammad Keshmiri received his B.Sc. and M.Sc. degrees in mechanical engineering from Isfahan University of Technology (IUT), Isfahan, Iran, in 2006 and 2009, respectively, and his Ph.D. degree in mechatronics from Concordia University, Montreal, QC, Canada, in 2014. He was an active member of the Dynamic and Robotic Research Group, IUT, and was involved in several projects of the group. He was a Postdoctoral Fellow with the Department of Computer Science at McGill University, Montreal, QC, Canada. Currently, he is a robotic researcher at Hypertherm Robotic Software Inc. His research interests include robotics and control, machine vision, artificial intelligence and nonlinear systems.

Wen-Fang Xie received her Ph.D. from The Hong Kong Polytechnic University in 1999 and her Master’s degree in from Beihang University in 1991. She is a professor with the Department of Mechanical and Industrial Engineering at Concordia University, Montreal, Canada. She joined Concordia University as an assistant professor in 2003 and was promoted to associate professor, professor, in 2008 and 2014, respectively. Her research interests include nonlinear control and identification in mechatronics, visual servoing, model predictive control, neural network, and advanced process control and system identification.

Ahmad Ghasemi received his B.Sc. and M.Sc. degrees in mechanical engineering from Isfahan University of Technology (IUT), Isfahan, Iran, in 2005 and 2008, respectively. He was a member of the Dynamic and Robotic Research Group, IUT, and was involved in some projects of the group. He is currently doing research on vision-based control of robots as Ph.D. student at Concordia University, Montreal, QC, Canada. His research interests include robotics and control, machine vision, machine learning and nonlinear systems.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Keshmiri, M., Xie, WF. & Ghasemi, A. Visual servoing using an optimized trajectory planning technique for a 4 DOFs robotic manipulator. Int. J. Control Autom. Syst. 15, 1362–1373 (2017). https://doi.org/10.1007/s12555-015-0187-8

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s12555-015-0187-8

Keywords

Search

Navigation