Homography-Based Visual Servo Tracking Control of Wheeled Mobile Robots with Simultaneous Depth Identification

  • Yu Qiu
  • Baoquan Li
  • Wuxi Shi
  • Yimei Chen
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 10639)


In this paper, a visual servo tracking control method is developed for the wheeled mobile robot subject to nonholonomic motion constraints, while the unknown feature depth information is simultaneously identified in the visual servoing process. Firstly, a video feature points are prerecorded as the desire trajectory for the mobile robot. Secondly, Euclidean homographies are constructed by utilizing projective geometric relationships of feature points. Subsequently, trajectory tracking errors are obtained after Euclidean homographies decomposition. Then, the kinematic controller is designed for the mobile robot to achieve the visual servo trajectory tracking task. Moreover, by utilizing the concurrent learning framework, the historical and current system data is used to construct an adaptive updating mechanism for recovering the unknown feature depth. Simulation results are collected to prove the efficiency and utility of the proposed strategy.


Wheeled mobile robot Visual servo control Adaptive control Depth identification 



This work is supported in part by National Natural Science Foundation of China under Grant 61603271, and in part by the Natural Science Foundation of Tianjin under Grant 15JCYBJC47800 and Grant 16JCQNJC03800.


  1. 1.
    Sun, N., Wu, Y., Fang, Y., Chen, H.: Nonlinear antiswing control for crane systems with double-pendulum swing effects and uncertain parameters: design and experiments. J. IEEE Trans. Autom. Sci. Eng. PP, 1–10 (2017)Google Scholar
  2. 2.
    Kriegman, D.J., Triendl, E., Binford, T.O.: Stereo vision navigation in buildings for mobile robots. J. IEEE Trans. Robot. Autom. 5(6), 792–803 (1989)CrossRefGoogle Scholar
  3. 3.
    Thorpe, C.E., Hebert, M., Kanade, T., Shafer, S.: Vision and navigation for the Carnegie-Mellon Navlab. J. IEEE Trans. Pattern Anal. Mach. Intell. 10(3), 362–373 (1988)CrossRefGoogle Scholar
  4. 4.
    Sun, N., Fang, Y., Chen, H., Lu, B.: Amplitude-saturated nonlinear output feedback antiswing control for underactuated cranes with double-pendulum cargo dynamics. J. IEEE Trans. Ind. Electron. 64(3), 2135–2146 (2017)CrossRefGoogle Scholar
  5. 5.
    Hebert, M., Kanade, T.: 3-D vision for outdoor navigation by an autonomous vehicle. In: Proceedings of Image Understanding Workshop, pp. 593–601. MIT Press, San Mateo (1988)Google Scholar
  6. 6.
    Sun, N., Fang, Y., Chen, H., Fu, Y., Lu, B.: Nonlinear stabilizing control for ship-mounted cranes with ship roll and heave movements: design, analysis, and experiments. J. IEEE Trans. Syst. Man Cybern. Syst. PP, 1–13 (2017)Google Scholar
  7. 7.
    Assa, A., Janabi-Sharifi, F.: Virtual visual servoing for multicamera pose estimation. J. IEEE/ASME Trans. Mechatron. 20(2), 789–798 (2015)CrossRefGoogle Scholar
  8. 8.
    Li, B., Fang, Y., Hu, G., Zhang, X.: Model-free unified tracking and regulation visual servoing of wheeled mobile robots. J. IEEE Trans. Control Syst. Technol. 24(4), 328–1339 (2016)Google Scholar
  9. 9.
    MacKunis, W., Gans, N., Kaiser, K., Dixon, W.E.: Unified tracking and regulation visual servo control for wheeled mobile. In: IEEE International Conference on Control Applications, pp. 88–93, Singapore (2007)Google Scholar
  10. 10.
    Hao, M., Sun, Z.: A universal state-space approach to uncalibrated model-free visual servoing. J. IEEE/ASME Trans. Mechatron. 17(5), 833–846 (2012)CrossRefGoogle Scholar
  11. 11.
    Li, B., Fang, Y., Zhang, X.: Visual servo regulation of wheeled mobile robots with an uncalibrated onboard camera. J. IEEE/ASME Trans. Mechatron. 21(4), 2230–2342 (2016)Google Scholar
  12. 12.
    Wang, H.Y., Itani, S., Fukao, T., Adachi, N.: Image-based visual adaptive tracking control of nonholonomic mobile robots. In: Proceedings IEEE/RSJ International Conference on Intelligent Robots and Systems. IEEE Press, Maui (2001)Google Scholar
  13. 13.
    Mahony, R., Hamel, T., Visual servoing using linear features for under-actuated rigid body dynamics. In: Proceedings IEEE/RSJ International Conference on Intelligent Robots and Systems, pp. 1153–1158. IEEE Press, Maui, Hawaii (2001)Google Scholar
  14. 14.
    Tan, F.X., Guan, X.P., Chen, Y., Sun, D.D.: Tracking control of nonholonomic mobile robot based on unfalsified adaptive PID theory. In: 8th World Congress on Intelligent Control and Automation. IEEE Press, Jinan (2010)Google Scholar
  15. 15.
    Chen, J., Dixon, W., Dawson, M., McIntyre, M.: Homography-based visual servo tracking control of a wheeled mobile robot. J. IEEE Trans. Robot. 22(2), 406–415 (2006)CrossRefGoogle Scholar
  16. 16.
    Liang, X., Wang, H., Liu, Y.H., Chen, W., Hu, G., Zhao, J.: Adaptive task-space cooperative tracking control of networked robotic manipulators without task-space velocity measurements. J. IEEE Trans. Cybern. 46(10), 2386–2398 (2016)CrossRefGoogle Scholar
  17. 17.
    Wang, H., Liu, Y.H., Chen, W., Wang, Z.: A new approach to dynamic eye-in-hand visual tracking using nonlinear observers. J. IEEE/ASME Trans. Mechatron. 16(2), 387–394 (2011)CrossRefGoogle Scholar
  18. 18.
    Wang, H., Liu, Y.H., Chen, W.: Uncalibrated visual tracking control without visual velocity. J. IEEE Trans. Control Syst. Technol. 18(6), 1359–1370 (2010)CrossRefGoogle Scholar
  19. 19.
    Wang, K., Liu, Y., Li, L.: Visual servoing trajectory tracking of nonholonomic mobile robots without direct position measurement. J. IEEE Trans. Robot. 30(4), 1026–1035 (2014)CrossRefGoogle Scholar
  20. 20.
    Parikh, A., Kamalapurkar, R., Chen, H.Y., Dixon, W.E.: Homography based visual servo control with scene reconstruction. In: 54th IEEE Conference on Decision and Control, pp. 6972–6977. IEEE Press, Osaka, Japan (2015)Google Scholar
  21. 21.
    Li, B., Fang, Y., Zhang, X., Shi, W.: Visual servo regulation of wheeled mobile robots with simultaneous depth identification. J. IEEE Trans. Ind. Electron. PP, 1 (2017)Google Scholar
  22. 22.
    Faugeras, O.: Three-Dimensional Computer Vision. MIT Press, Cambridge (2001)Google Scholar
  23. 23.
    Zhang, Z., Hanson, A.R.: Scaled Euclidean 3D reconstruction based on externally uncalibrated cameras. In: Proceedings of International Symposium on Computer Vision, pp. 37–42. IEEE Press, Coral Gables (1995)Google Scholar
  24. 24.
    Zhang, X., Fang, Y., Li, B., Wang, J.: Visual servoing of nonholonomic mobile robots with uncalibrated camera-to-robot parameters. J. IEEE Trans. Ind. Electron. 64(1), 390–400 (2017)CrossRefGoogle Scholar

Copyright information

© Springer International Publishing AG 2017

Authors and Affiliations

  1. 1.Tianjin Key Laboratory of Advanced Technology of Electrical Engineering and Energy, School of Electrical Engineering and AutomationTianjin Polytechnic UniversityTianjinChina

Personalised recommendations