Adaptive homography-based visual servo for micro unmanned surface vehicles

  • Ning WangEmail author
  • Hongkun He


In this paper, a novel adaptive homography-based visual servo (AHBVS) scheme is proposed to regulate a micro unmanned surface vehicle (MUSV) to the desired pose in the presence of both unknown image depth and unmatched dynamics. By virtue of homography decomposition technique, the scaled pose errors are directly retrieved from the live and desired images which are captured by a monocular camera. On the basis of the MUSV characteristics, a completely new visual servo system is firstly derived from visual measurements including both kinematics and dynamics. Different from kinematics solutions, the dynamics-level AHBVS controllers adapting to unknown image depth and compensating unmatched dynamics are developed by incorporating backstepping technique and Lyapunov synthesis, and thereby facilitating practical implementations. Lyapunov analysis proves that the proposed AHBVS scheme renders the closed-loop visual servo system globally uniformly asymptotically stable (GUAS). Simulation results demonstrate remarkable performance on a prototype MUSV.


Adaptive homography-based visual servo Dynamics-level controller Micro unmanned surface vehicle 


Funding information

This work is supported by the National Natural Science Foundation of P. R. China (under Grants 51009017 and 51379002), the Fund for Dalian Distinguished Young Scholars (under Grant 2016RJ10), the Liaoning Revitalization Talents Program (under Grant XLYC1807013), the Stable Supporting Fund of Science and Technology on Underwater Vehicle Laboratory (SXJQR2018WDKT03), and the Fundamental Research Funds for the Central Universities (under Grants 3132016314 and 3132018126).


  1. 1.
    Karimi HR (2018) Variable structure control via coupled surfaces for control effort reduction in remotely operated vehicles. In: Offshore Mechatron Syst Eng, pp 171–191Google Scholar
  2. 2.
    Wang N, Karimi HR, Li HY, Su SF (2019) Accurate trajectory tracking of disturbed surface vehicles: a finite-time control approach. IEEE/ASME Trans Mechatron. CrossRefGoogle Scholar
  3. 3.
    Wu Y, Karimi HR, Lu R (2018) Sampled-data control of network systems in industrial manufacturing. Trans Ind Electron 65(11):9016–9024CrossRefGoogle Scholar
  4. 4.
    Xie M, Shakoor A, Shen Y, Mills JK, Sun D (2019) Out-of-plane rotation control of biological cells with a robot-tweezers manipulation system for orientation-based cell surgery. Trans Biomed Eng 66(1):199–207. CrossRefGoogle Scholar
  5. 5.
    Wu Z, Jiang B, Kao Y (2019) Finite-time \(\mathcal {H}_{\infty }\) filtering for Itô stochastic Markovian jump systems with distributed time-varying delays based on optimisation algorithm. IET Control Theory Appl 13(5):702–710. MathSciNetCrossRefGoogle Scholar
  6. 6.
    Wang N, Xie G, Pan X, Su SF (2019) Full-state regulation control of asymmetric underactuated surface vehicles. Trans Ind Electron. CrossRefGoogle Scholar
  7. 7.
    Fang Y, Dixon WE, Dawson DM, Prakash C (2005) Homography-based visual servo regulation of mobile robots. Trans Syst Man Cybern Cybern 35(5):1041–1050CrossRefGoogle Scholar
  8. 8.
    Chaumette F, Hutchinson S (2006) Visual servo control. I. Basic approaches. Robot Autom Mag 13(4):82–90CrossRefGoogle Scholar
  9. 9.
    Chaumette F, Hutchinson S (2007) Visual servo control. II. Advanced approaches. Robot Autom Mag 14 (1):109–118CrossRefGoogle Scholar
  10. 10.
    Deng C, Yang GH (2019) Distributed adaptive fault-tolerant control approach to cooperative output regulation for linear multi-agent systems. Automatica 103:62–68MathSciNetzbMATHCrossRefGoogle Scholar
  11. 11.
    Malis E, Chaumette F (2000) 2 1/2 D Visual servoing with respect to unknown objects through a new estimation scheme of camera displacement. Int J Comput Vision 37(1):79–97zbMATHCrossRefGoogle Scholar
  12. 12.
    Benhimane S, Malis E (2007) Homography-based 2D visual tracking and servoing. Int J Robot Res 26(7):661–676CrossRefGoogle Scholar
  13. 13.
    Chen J, Dixon WE, Dawson DM, Mcintire M (2006) Homography-based visual servo tracking control of a wheeled mobile robot. Trans Robot 22(2):406–415CrossRefGoogle Scholar
  14. 14.
    Li B, Zhang X, Fang Y, Shi W (2018) Visual servo regulation of wheeled mobile robots with simultaneous depth identification. Trans Ind Electron 65(1):460–469CrossRefGoogle Scholar
  15. 15.
    Zhang X, Wang R, Fang Y, Li B, Ma B (2017) Acceleration-level pseudo-dynamic visual servoing of mobile robots with backstepping and dynamic surface control. Trans Syst Man Cybern Syst (99):1–11. CrossRefGoogle Scholar
  16. 16.
    Zhang K, Chen J, Li Y, Gao Y (2018) Unified visual servoing tracking and regulation of wheeled mobile robots with an uncalibrated camera. Trans Mechatron 23(4):1728–1739CrossRefGoogle Scholar
  17. 17.
    Hu G, Dixon WE, Gupta S, Fitz-Coy N (2006) A quaternion formulation for homography-based visual servo control. In: Int Conf Robot Automat, pp 2391–2396Google Scholar
  18. 18.
    Chitrakaran VK, Dawson DM, Kannan H, Feemster M (2006) Vision assisted autonomous path following for unmanned aerial vehicles. In: Conf Decision Control, pp 63–68Google Scholar
  19. 19.
    De Plinval H, Morin P, Mouyon P, Hamel T (2014) Visual servoing for underactuated VTOL UAVs: a linear, homography-based framework. Int J Robust Nonlinear Control 24(16):2285–2308zbMATHCrossRefGoogle Scholar
  20. 20.
    Hua MD, Allibert G, Krupínski S, Hamel T (2014) Homography-based visual servoing for autonomous underwater vehicles. IFAC Proc 47(3):5726–5733CrossRefGoogle Scholar
  21. 21.
    Krupínski S, Allibert G, Hua MD, Hamel T (2017) An inertial-aided homography-based visual servo control approach for (almost) fully actuated autonomous underwater vehicles. Trans Robot 33(5):1041–1060CrossRefGoogle Scholar
  22. 22.
    Wang N, Deng Q, Xie G, Pan X (2019) Hybrid finite-time trajectory tracking control of a quadrotor. ISA transactions. CrossRefGoogle Scholar
  23. 23.
    Qin H, Wu Z, Sun Y, Chen H (2019) Disturbance-observer-based prescribed performance fault-tolerant trajectory tracking control for ocean bottom flying node. IEEE Access 7:49004–49013CrossRefGoogle Scholar
  24. 24.
    Yu C, Xiang X, Lapierre L, Zhang Q (2018) Robust magnetic tracking of subsea cable by AUV in the presence of sensor noise and ocean currents. IEEE J Oceanic Eng 43(2):311–322CrossRefGoogle Scholar
  25. 25.
    Wang N, Sun Z, Yin J, Zou Z, Su SF (2019) Fuzzy unknown observer-based robust adaptive path following control of underactuated surface vehicles subject to multiple unknowns. Ocean Eng 176:57–64. CrossRefGoogle Scholar
  26. 26.
    Martins A, Almeida JM, Ferreira H, Silva H, Dias N, Dias A, Almeida C, Silva E (2007) Autonomous surface vehicle docking manoeuvre with visual information. In: IEEE Int Conf Robot Automat. IEEE, pp 4994–4999Google Scholar
  27. 27.
    Dunbabin M, Lang B, Wood B (2008) Vision-based docking using an autonomous surface vehicle. In: IEEE Int Conf Robot Automat. IEEE, pp 26–32Google Scholar
  28. 28.
    Kim YH, Lee SW, Yang HS, Shell DA (2012) Toward autonomous robotic containment booms: visual servoing for robust inter-vehicle docking of surface vehicles, vol 5. CrossRefGoogle Scholar
  29. 29.
    Wang J, Liu JY, Yi H (2017) Formation control of unmanned surface vehicles with sensing constraints using exponential remapping method. Mathemat Problems Eng 2017:1–14MathSciNetGoogle Scholar
  30. 30.
    Wang K, Liu Y, Li L (2015) Vision-based tracking control of underactuated water surface robots without direct position measurement. IEEE Trans Control Syst Technol 23(6):2391–2399CrossRefGoogle Scholar
  31. 31.
    Hu G, MacKunis W, Gans N, Dixon WE, Chen J, Behal A, Dawson D (2009) Homography-based visual servo control with imperfect camera calibration. Trans Autom Control 54(6):1318–1324MathSciNetzbMATHCrossRefGoogle Scholar
  32. 32.
    Lowe DG (1999) Object recognition from local scale-invariant features. In: ICCV, vol 99, pp 1150–1157Google Scholar
  33. 33.
    Leutenegger S, Chli M, Siegwart R (2011) Brisk: binary robust invariant scalable keypoints. In: ICCV, pp 2548–2555Google Scholar
  34. 34.
    Bay H, Ess A, Tuytelaars T, Van Gool L (2008) Speeded-up robust features (SURF). Comput Vis Image Underst 110(3):346–359CrossRefGoogle Scholar
  35. 35.
    Rosten E, Porter R, Drummond T (2010) Faster and better: a machine learning approach to corner detection. Trans Pattern Anal Mach Intell 32(1):105–119CrossRefGoogle Scholar
  36. 36.
    Wang N, Su SF, Han M, Chen WH (2018) Backpropagating constraints-based trajectory tracking control of a quadrotor with constrained actuator dynamics and complex unknowns. IEEE Trans Sys Man Cybern Syst (99):1–16.
  37. 37.
    Zhang Z (1999) Flexible camera calibration by viewing a plane from unknown orientations. Int conf computer vision, vol 1, pp 666–673Google Scholar
  38. 38.
    Zhang H, Ostrowski JP (1999) Visual servoing with dynamics: control of an unmanned blimp. In: Int conf robot automat, vol 1, pp 618–623Google Scholar
  39. 39.
    Zhang Z, Hanson AR (1995) Scaled Euclidean 3D reconstruction based on externally uncalibrated cameras. In: Int Symp Computer Vision, pp 37–42Google Scholar
  40. 40.
    Faugeras OD, Lustman F (1988) Motion and structure from motion in a piecewise planar environment. Int J Pattern Recogn Artific Intell 2(03):485–508CrossRefGoogle Scholar
  41. 41.
    Wang N, Su SF, Pan X, Yu X, Xie G (2018) Yaw-guided trajectory tracking control of an asymmetric underactuated surface vehicle. Trans Ind Informat. CrossRefGoogle Scholar
  42. 42.
    Ghommam J, Mnif F, Benali A, Poisson G (2007) Observer design for Euler Lagrange systems: application to path following control of an underactuated surface vessel. In: Int Conf Intell Robots Syst, pp 2883–2888Google Scholar
  43. 43.
    Khalil H (1996) Nonlinear systems, 2nd edn. Prentice-Hall, Upper Saddle RiverGoogle Scholar

Copyright information

© Springer-Verlag London Ltd., part of Springer Nature 2019

Authors and Affiliations

  1. 1.School of Marine Electrical EngineeringDalian Maritime, UniversityDalianChina

Personalised recommendations