Abstract
The use of underwater vehicle manipulator systems (UVMS) equipped with cameras has gained significant attention due to their capacity to perform underwater tasks autonomously. However, controlling both the manipulator and the remotely operated vehicle (ROV) based on the vision system information is not an easy task, especially in situations where the vehicle cannot be parked/held stationary. Most of the existing approaches work based on complex matrix calculations for the inverse kinematics (IK), which can lead to high computational costs and the need to deal with singularity problems. A problem arises when the amount of time needed to calculate the UVMS configuration can result in reduced frequency of target pose estimation, beyond the point where the target has moved out of the camera field of view. Therefore, this paper proposes an autonomous visual servoing approach for UVMS, including an extension of a heuristic technique named M-FABRIK (Mobile - Forward and Backward Reaching IK) to calculate the UVMS inverse kinematics in a simple and fast way. This approach aims to control both the configuration of the manipulator and ROV position in order to allow underwater intervention in situations where the ROV cannot be parked/held stationary. This solution allows the vehicle to be positioned according to additional criteria, besides avoiding matrix inversion and being robust to singularities. Trials have been performed with a manipulator mounted on a work-class ROV for an autonomous underwater monitoring task and results demonstrate a simple and fast approach, which is able to set the configuration of the manipulator as well as the ROV for visual servoing applications in real-time, such as for monitoring, tracking and intervention tasks underwater.
Article PDF
Similar content being viewed by others
Avoid common mistakes on your manuscript.
Code or Data Availability
Data sharing not applicable to this article.
References
Jun, B.H., Lee, P.M., Kim, S.: Manipulability analysis of underwater robotic arms on ROV and application to task-oriented joint configuration. J. Mech. Sci. Technol. 22(5), 887–894 (2008)
Antonelli, G., Antonelli, G.: Underwater robots, vol. 3. Springer International Publishing, Cham (2014)
Marani, G., Choi, S.K., Yuh, J.: Underwater autonomous manipulation for intervention missions AUVs. Ocean Eng. 36(1), 15–23 (2009)
Hamzaoui, N., Zaoui, C., Hachicha, S., Dallagi, H., Nejim, S., Maalej, A.: Development and modeling of an underwater ship’s hull inspection station equipped with two arms manipulator. 2019 International Conference on Advanced Systems and Emergent Technologies (IC_ASET). IEEE (2019)
Christ, R.D., Wernli, R.L. Sr: The ROV manual: a user guide for remotely operated vehicles. Butterworth-Heinemann (2013)
Zhang, Y., Wang, X.: An underwater vehicle for cleaning dirty bottom of ships. IOP Conference Series: Earth and Environmental Science. vol. 567. No. 1 IOP Publishing (2020)
Sivčev, S., Rossi, M., Coleman, J., Dooly, G., Omerdić, E., Toal, D.: Fully automatic visual servoing control for work-class marine intervention ROVs. Control. Eng. Pract. 74, 153–167 (2018)
Asokan, T., Seet, G., Iastrebov, V., Senanayake, R.: Kinematic design and analysis of a 7 degree-of-freedom dual-stage inspection manipulator for dexterous subsea applications. J. Intell. Robot. Syst. 38(3), 277–95 (2003)
Bruno, F., Lagudi, A., Barbieri, L., Rizzo, D., Muzzupappa, M., De Napoli, L.: Augmented reality visualization of scene depth for aiding ROV pilots in underwater manipulation. Ocean Eng. 168, 140–154 (2018)
Coleman, D.F., Ballard, R.D., Gregory, T.: Marine archaeological exploration of the Black Sea. Oceans 2003. Celebrating the Past... Teaming Toward the Future (IEEE Cat. No. 03CH37492). vol. 3. IEEE (2003)
Sivčev, S., Coleman, J., Omerdić, E., Dooly, G., Toal, D.: Underwater manipulators: A review. Ocean Eng. 163, 431–450 (2018)
Soylu, S., Buckham, B.J., Podhorodeski, R.P.: Redundancy resolution for underwater mobile manipulators. Ocean Eng. 37(2-3), 325–343 (2010)
Cooke, N.J.: Human factors of remotely operated vehicles. Proceedings of the Human Factors and Ergonomics Society Annual Meeting. vol. 50. No. 1. Sage CA: Los Angeles, CA: SAGE Publications (2006)
Santhakumar, M., Kim, J.: Indirect adaptive control of an autonomous underwater vehicle-manipulator system for underwater manipulation tasks. Ocean Eng. 54, 233–243 (2012)
Jonghui, H., Park, J., Chung, W.K.: Robust coordinated motion control of an underwater vehicle-manipulator system with minimizing restoring moments. Ocean Eng. 38(10), 1197–1206 (2011)
Santhakumar, M., Kim, J.: Coordinated motion control in task space of an autonomous underwater vehicle–manipulator system. Ocean Eng. 104, 155–167 (2015)
Gianluca, A., Chiaverini, S.: A fuzzy approach to redundancy resolution for underwater vehicle-manipulator systems. Control. Eng. Pract. 11(4), 445–452 (2003)
Simetti, E., Casalino, G.: A novel practical technique to integrate inequality control objectives and task transitions in priority based control. J. Intell. Robot. Syst. 84(1), 877–902 (2016)
Santos, P.C., Freire, R.C.S., Carvalho, E.A.N., Molina, L., Freire, E.O.: M-FABRIK: A new inverse kinematics approach to mobile manipulator robots based on FABRIK. IEEE Access 8, 208836–208849 (2020)
Aristidou, A., Lasenby, J.: FABRIK: A fast, iterative solver for the Inverse Kinematics problem. Graph. Model. 73(5), 243–260 (2011)
Gianluca, A., Chiaverini, S.: Fuzzy redundancy resolution and motion coordination for underwater vehicle-manipulator systems. IEEE Trans. Fuzzy Syst. 11(1), 109–120 (2003)
Gianluca, A., Caccavale, F., Chiaverini, S.: Adaptive tracking control of underwater vehicle-manipulator systems based on the virtual decomposition approach. IEEE Trans. Robot. Autom. 20(3), 594–602 (2004)
Zool, I.H., Dunnigan, M.W.: Redundancy resolution for underwater vehicle-manipulator systems with congruent gravity and buoyancy loading optimization. 2009 IEEE International Conference on Robotics and Biomimetics (ROBIO) (2009)
Salloom, T., Yu, X., He, W., Kaynak, O.: Adaptive neural network control of underwater robotic manipulators tuned by a genetic algorithm. J. Intell. Robot. Syst. 97(3), 657–72 (2020)
Li, J., Huang, H., Xu, Y., Wu, H., Wan, L.: Uncalibrated visual servoing for underwater vehicle manipulator systems with an eye in hand configuration camera. Sensors 19(24), 5469 (2019)
Gao, J., Liang, X., Chen, Y., Zhang, L., Jia, S.: Hierarchical image-based visual serving of underwater vehicle manipulator systems based on model predictive control and active disturbance rejection control. Ocean Eng. 229, 108814 (2021)
Simetti, E.: Autonomous underwater intervention. Curr. Robot. Rep. 1(3), 117–122 (2020)
Youakim, D., Ridao, P., Palomeras, N., Spadafora, F., Ribas, D., Muzzupappa, M.: MoveIt!: Autonomous underwater free-floating manipulation. IEEE Robot. Autom. Mag. 24(3), 41–51 (2017)
Moe, S., Antonelli, G., Teel, A.R., Pettersen, K.Y., Schrimpf, J.: Set-based tasks within the singularity-robust multiple task-priority inverse kinematics framework: General formulation, stability analysis, and experimental results. Front. Robot. AI 3, 16 (2016)
Wang, L.C., Chen, C.C.: A combined optimization method for solving the inverse kinematics problems of mechanical manipulators. IEEE Trans. Robot. Autom. 7(4), 489–499 (1991)
Muller-Cajar, R., Mukundan, R.: Triangualation-a new algorithm for inverse kinematics. University of Canterbury (2007)
Santos, M.C., Molina, L., Carvalho, E.A., Freire, E.O., Carvalho, J.G., Santos, P.C.: FABRIK-R: an extension developed based on FABRIK for robotics manipulators. IEEE Access 9, 53423–53435 (2021)
Aristidou, A., Chrysanthou, Y., Lasenby, J.: Extending FABRIK with model constraints. Comput. Animat. Virtual Worlds 27(1) (2016)
Agarwal, A., Jawahar, C.V., Narayanan, P.J.: A survey of planar homography estimation techniques. Centre for Visual Information Technology, Tech. Rep IIIT/TR/2005/12 (2005)
Vincent, L., Fua, P.: Monocular model-based 3D tracking of rigid objects. Now Publishers Inc (2005)
CRIS-UL. “[Video file] Inverse Kinematics for Underwater Vehicle Manipulator Systems Based on M-FABRIK”. https://youtu.be/Svm_tf2KM4w. Accessed 22 March 2022
Funding
Open Access funding provided by the IReL Consortium. This work was partially supported by Coordenação de Aperfeiçoamento de Pessoal de Nível Superior - Brasil (CAPES) under Grant Finance Code 001 and SEAI Research, Development and Demonstration Funding Programme 2021 (grant No.; 21/RDD/747). It has also emanated from research supported by the Science Foundation Ireland under the MaREI Centre research programme (grant No.; SFI/12/RC/2302P2, and SFI/14/SP/2740), LERO Science Foundation Ireland (grant No.; 13/RC/2094) and CONFIRM (grant No.; 16/RC/3918). Additionally, it is co-funded under the European Regional Development Fund through the Southern and Eastern Regional Operational Programme to MaREI, Lero and CONFIRM centre 001.
Author information
Authors and Affiliations
Contributions
All authors contributed to the proposed approach. The development of the PBVS system and the extension of the M-FABRIK algorithm were performed by Phillipe C. Santos, Raimundo C. S. Freire, Elyson A. N. Carvalho, Lucas Molina, Eduardo O. Freire and Matheus C. Santos. The implementation of the algorithms and control systems on the robot were performed by Phillipe C. Santos, Matheus C. Santos, Petar Trslic, Edin Omerdic, Gerard Dooly and Daniel Toal. The experimental trials were performed by by Phillipe C. Santos, Matheus C. Santos, Anthony Weir, Petar Trslic, Edin Omerdic, Gerard Dooly and Daniel Toal. The first draft of the manuscript was written by Phillipe C. Santos and all authors commented on previous versions of the manuscript. All authors read and approved the final manuscript.
Corresponding author
Ethics declarations
Ethics approval
This study did not require ethics approval.
Consent to Participate
Not applicable. This study did not involve human subjects.
Consent for Publication
The authors affirm that the individuals present in Fig. 7 are all on the list of authors of this study and they informed consent for publication of the image.
Competing interests
The authors have no relevant financial or non-financial interests to disclose.
Additional information
Publisher’s Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/.
About this article
Cite this article
Santos, P.C., Freire, R.C.S., Carvalho, E.A.N. et al. Fully Automatic Visual Servoing Control for Underwater Vehicle Manipulator Systems Based on a Heuristic Inverse Kinematics. J Intell Robot Syst 107, 42 (2023). https://doi.org/10.1007/s10846-023-01827-0
Received:
Accepted:
Published:
DOI: https://doi.org/10.1007/s10846-023-01827-0