Skip to main content
SpringerLink
Log in

Study of 3D measurement of ships using dense stereo vision: towards application in automatic berthing systems

  • Original article
  • Published:
Journal of Marine Science and Technology Aims and scope Submit manuscript

Abstract

Autonomous navigation of ships is actively discussed nowadays. It is essential to recognize the surrounding conditions using radars and acquire position information using GPS. However, each of these existing sensors has its own problems. In order to achieve safer autonomous navigation in scenes such as automatic berthing, passing in a narrow line and closing to other ships, an imaging system that integrates various information is essential. Therefore, the authors propose a 3D position measurement method for ships using a stereo camera. In the case of the automatic berthing system, detailed information such as the position of various parts of the hull, heading, roll angle, etc. is necessary. This paper describes a system that measures 3D locations of many parts of ship’s hull with a dense stereo algorithm applied to stereo-pair images captured by cameras set up on land. An experiment to measure an actual ship as a target was conducted. The result indicated that the more disparity data, which meant depth of the ship from the cameras, were obtained by semi-global matching (SGM) even from the featureless images like ship’s hull and the measurement error was within 1 m.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9

Similar content being viewed by others

References

  1. Rolls-Royce plc, Rolls-Royce and Finferries demonstrate world’s first Fully Autonomous Ferry. https://www.rolls-royce.com/media/press-releases/2018/03-12-2018-rr-and-finferries-demonstrate-worlds-first-fully-autonomous-ferry.aspx. Accessed 30 Dec 2019

  2. Yara International ASA, Yara Birkeland press kit. https://www.yara.com/news-and-media/press-kits/yara-birkeland-press-kit/. Accessed 30 Dec 2019

  3. Shen H, Hashimoto H, Matsuda A, Taniguchi Y, Terada D, Guo C (2019) Automatic collision avoidance of multiple ships based on deep Q-learning. Appl Ocean Res 86:268–288

    Article  Google Scholar 

  4. Xue Y, Clelland D, Lee BS, Han D (2011) Automatic simulation of ship navigation. Ocean Eng 38:2290–2305

    Article  Google Scholar 

  5. Perera L, Carvaho J, Soares C (2009) Autonomous guidance and navigation based on the COLREGs rules and regulations of collision avoidance. In: Proceedings of the international workship “advanced ship design for pollution prevention”, pp 205–216

  6. Perera L, Ferrari V, Santos F, Hinostroza M, Soares C (2015) Experimental evaluations on ship autonomous navigation and collision avoidance by intelligent guidance. IEEE J Oceanic Eng 40(2):374–387

    Article  Google Scholar 

  7. The Ship’s Electric Installation Contractrors’ Association of Japan (2003) The lecture of marine electrical equipment technology (Lader, AIS, VDR, GPS). The Nippon Foundation Libraly

  8. Kocak G, Yamamoto S, Hashimoto T (2013) Detection and tracking of ships using a stereo vision system. Sci Res Essays 8(7):288–303

    Google Scholar 

  9. Ueno M (2000) A GPS-based system for precise shipping guidance and control. J Mar Sci Technol 5(1):9–15

    Article  Google Scholar 

  10. TOKYO KEIKI INC. (2016) Docking Support System DL-3000. https://www.tokyokeiki.jp/Portals/0/images/e/products/pdf/measurement/dl3000.pdf. Accessed 30 Dec 2019

  11. Koden Electronics Co., Ltd. (no date) SRD-303i Ship's docking aid system—laser docking ranger. https://www.koden-electronics.co.jp/en/enindustrial/srd-303i-e. Accessed 30 Dec 2019

  12. Hoshino K, Hara S, Ikemoto Y, Kuroda T (2007) Development and evaluation test of the speedometer for berthing operation support system. Conf Jpn Soc Naval Archit Ocean Eng 5E:151–154 (in Japanese)

    Google Scholar 

  13. Hirata N, Nishijima T, Kakino F (2013) Acquisition of the numerical information for support berthing maneuver by using image processing technology. J Jpn Inst Navig 129:99–104 (in Japanese)

    Article  Google Scholar 

  14. Mizuchi Y, Ogura T, Kim Y, Hagiwara Y, Choi Y (2017) Camera-based measurement for close-distance relative vessel positioning. J Mech Sci Technol 31(4):1899–1907

    Article  Google Scholar 

  15. Lee S, Tzeng C, Kehr Y, Kang C, Huang C (2010) Design and application of an image-processing-based fuzzy autopilot for small-boat approaching maneuvers. J Mar Sci Technol 18(4):558–567

    Google Scholar 

  16. Kiriya N, Imasato M (2006) Marine monitoring technique using omnidirectional camera on berthing. J Jpn Inst Navig 165:88–93 (in Japanese)

    Google Scholar 

  17. Ohyama Y, Ezaki N, Seta H, Hamaji Y (2010) Technology to display image around ship by using Fisheye. J Jpn Inst Navig 122:45–52 (in Japanese)

    Article  Google Scholar 

  18. Nagashima Y, Furuya T, Inaishi M, Kayano J (2012) The study of surveillance around the ship. J Jpn Inst Navig 127:141–148 (in Japanese)

    Article  Google Scholar 

  19. Tall M, Rynne P, Lorio J, Ellenrieder K (2009) Visual-Based Navigation of an Autonomous Tugboat. In: IEEE, OCEANS Oct 26–29

  20. Bui V, Kawai H, Kim Y, Lee K (2011) A Ship berthing system design with four tug boats. J Mech Sci Technol 25(5):1257–1264

    Article  Google Scholar 

  21. Bui V, Kim Y (2011) Development of constrained control allocation for ship berthing by using autonomous tugboats. Int J Control Autom Syst 9(6):1203–1208

    Article  Google Scholar 

  22. Hirschmüller H (2005) Accurate and efficient stereo processing by semi-global matching and mutual information. In: IEEE, CVPR June 20–26

  23. opencv dev team (no date) Camera Calibration and 3D Reconstruction. https://docs.opencv.org/2.4/modules/calib3d/doc/camera_calibration_and_3d_reconstruction.html. Accessed 30 Dec 2019

  24. Zhang Z (2000) A flexible new technique for camera calibration. IEEE Trans Pattern Anal Mach Intell 22(11):1330–1334

    Article  Google Scholar 

  25. Graduate School of Maritime Sciences, Kobe University (no date) Fukaemaru. https://www.edu.kobe-u.ac.jp/gmsc-fukaemaru/ship/index.html#GA. Accessed 30 Dec 2019

  26. Zha D, Jin X, Xiang T (2016) A real-time global stereo-matching on FPGA. Microprocess Microsyst 47:419–428

    Article  Google Scholar 

  27. Cambuim L, Oliveira L, Barros E, Ferreira A (2019) An FPGA-based real-time occlusion robust stereo vision system using semi-global matching. J Real Time Image Proc 30:1–22

    Google Scholar 

  28. Hofmann J, Korinth J, Koch A (2016) A scalable high-performance hardware architecture for real-time stereo vision by semi-global matching. In: IEEE conference of CVPRW, pp 27–35

  29. Jain A, Fell A, Anand S (2017) A parallel architecture for high frame rate stereo using semi-global matching. In: Published in BMVC

  30. Ueno M, Santerre R (2000) A combined method for GPS ambiguity resolution with single-frequency receivers: application for ship berthing. J Inst Navig 41(2):100–111

    Article  Google Scholar 

  31. Nakamura H, Sonobe T et al (2019) Robust positioning method for berthing support using fusion of tightly coupled GNSS carrier phase with IMU and visual odometry. J IPNTJ 10(1):1–12 (in Japanese)

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Graduate School of Maritime Sciences, Kobe University, 5-1-1 Fukaeminami-machi, Higashinada-ku, Kobe, 658-0022, Japan

    Yasuhiro Nomura & Shigehiro Yamamoto

  2. Faculty of Engineering, Shizuoka University, 3-5-1 Johoku, Naka-ku, Hamamatsu, 432-8011, Japan

    Takeshi Hashimoto

Authors
  1. Yasuhiro Nomura
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Shigehiro Yamamoto
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Takeshi Hashimoto
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Yasuhiro Nomura.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Nomura, Y., Yamamoto, S. & Hashimoto, T. Study of 3D measurement of ships using dense stereo vision: towards application in automatic berthing systems. J Mar Sci Technol 26, 573–581 (2021). https://doi.org/10.1007/s00773-020-00761-2

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00773-020-00761-2

Keywords

Search

Navigation