We’re sorry, something doesn't seem to be working properly.

Please try refreshing the page. If that doesn't work, please contact support so we can address the problem.

Learning from Synthetic Data: Enhancing Refraction Correction Accuracy for Airborne Image-Based Bathymetric Mapping of Shallow Coastal Waters | SpringerLink

Learning from Synthetic Data: Enhancing Refraction Correction Accuracy for Airborne Image-Based Bathymetric Mapping of Shallow Coastal Waters

We’re sorry, something doesn't seem to be working properly.

Please try refreshing the page. If that doesn't work, please contact support so we can address the problem.

Abstract

The increasing need for accurate bathymetric mapping is essential for a plethora of offshore activities. Even though aerial image datasets through Structure from Motion (SfM) and Multi-View Stereo (MVS) techniques can provide a low-cost alternative compared to LiDAR and SONAR, offering additionally, important visual information, water refraction poses significant obstacles in delivering accurate bathymetry. In this article, the generation of manned and unmanned airborne synthetic datasets of dry and water covered areas is presented. These data are used to train models for correcting the geometric effects of refraction on real-world image-based point clouds and aerial images. Based on a thorough evaluation, important improvements are presented, indicating the increased accuracy and the reduced noise in the point clouds of the derived bathymetric products, meeting also the International Hydrographic Organization’s (IHO) standards.

Zusammenfassung

Zusammenfassung Lernen aus synthetischen Daten: Verbesserung der Genauigkeit der Refraktionskorrektur für flugzeuggestützte bathymetrische Kartierung von flachen Küstengewässern. Für eine Vielzahl von Offshore-Aktivitäten besteht ein zunehmender Bedarf an genauen bathymetrischen Kartierungen. Obwohl Luftbilddatensätze durch Structure-from-Motion (SfM) und Multi-View-Stereo (MVS)-Techniken eine kostengünstige Alternative zu LiDAR und SONAR darstellen können und zusätzlich wichtige visuelle Informationen bieten, stellt die Refraktion an der Wasseroberfläche ein erhebliches Hindernis für die Bereitstellung präziser Bathymetrie dar. In diesem Artikel wird die Erzeugung von synthetischen Datensätzen von trockenen und wasserbedeckten Gebieten aus bemannter und unbemannter Erfassung vorgestellt. Diese Daten werden verwendet, um Modelle zur Korrektur der Strahlbrechung bei photogrammetrisch bestimmten Punktwolken aus realen Luftbildern zu trainieren. Basierend auf einer gründlichen Auswertung werden maßgebliche Verbesserungen vorgestellt, die eine erhöhte Genauigkeit und ein reduziertes Rauschen der abgeleiteten bathymetrischen Punktwolken belegen, die auch die Genauigkeitsstandards der International Hydrographic Organization (IHO) erfüllen.

This is a preview of subscription content, access via your institution.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9

References

  1. Agrafiotis P (2020) Image-based bathymetry mapping for shallow waters. PhD thesis, National Technical University of Athens

  2. Agrafiotis P, Georgopoulos A (2015) Camera constant in the case of two media photogrammetry. ISPRS Int Arch Photogramm Remote Sens Spatial Inf Sci XL–5/W5:1–6. https://doi.org/10.5194/isprsarchives-XL-5-W5-1-2015

    Article  Google Scholar 

  3. Agrafiotis P, Skarlatos D, Forbes T, Poullis C, Skamantzari M, Georgopoulos A (2018) Underwater photogrammetry in very shallow waters: main challenges and caustics effect removal. ISPRS Int Arch Photogramm Remote Sens Spatial Inf Sci XLII–2:15–22. https://doi.org/10.5194/isprs-archives-XLII-2-15-2018

    Article  Google Scholar 

  4. Agrafiotis P, Skarlatos D, Georgopoulos A, Karantzalos K (2019a) Depthlearn: Learning to correct the refraction on point clouds derived from aerial imagery for accurate dense shallow water bathymetry based on SVMs-fusion with LiDAR point clouds. Remote Sens 11(19):2225

    Article  Google Scholar 

  5. Agrafiotis P, Skarlatos D, Georgopoulos A, Karantzalos K (2019b) Shallow water bathymetry mapping from UAV imagery based on machine learning. ISPRS Int Arch Photogramm Remote Sens Spatial Inf Sci XLII–2/W10:9–16. https://doi.org/10.5194/isprs-archives-XLII-2-W10-9-2019

    Article  Google Scholar 

  6. Agrafiotis P, Karantzalos K, Georgopoulos A, Skarlatos D (2020) Correcting image refraction: Towards accurate aerial image-based bathymetry mapping in shallow waters. Remote Sens 12(2):322

    Article  Google Scholar 

  7. Awad M, Khanna R (2015) Support vector regression. Efficient learning machines. Springer, New York, pp 67–80

    Chapter  Google Scholar 

  8. Barbosa IB, Cristani M, Caputo B, Rognhaugen A, Theoharis T (2018) Looking beyond appearances: synthetic training data for deep CNNs in re-identification. Comput Vis Image Underst 167:50–62

    Article  Google Scholar 

  9. Bianco S, Ciocca G, Marelli D (2018) Evaluating the performance of Structure from Motion pipelines. J Imaging 4(8):98

    Article  Google Scholar 

  10. Bishop CM (2006) Pattern recognition and machine learning. Springer, New York

    Google Scholar 

  11. Butler J, Lane S, Chandler J, Porfiri E (2002) Through-water close range digital photogrammetry in flume and field environments. Photogramm Rec 17(99):419–439

    Article  Google Scholar 

  12. Cao B, Fang Y, Jiang Z, Gao L, Hu H (2019) Shallow water bathymetry from Worldview-2 stereo imagery using two-media photogrammetry. Eur J Remote Sens 52(1):506–521

    Article  Google Scholar 

  13. Cao B, Deng R, Zhu S (2020) Universal algorithm for water depth refraction correction in through-water stereo remote sensing. Int J Appl Earth Obs Geoinf 91:102108

    Article  Google Scholar 

  14. Chirayath V, Li A (2019) Next-Generation Optical Sensing Technologies for Exploring Ocean Worlds—NASA FluidCam, MiDAR, and NeMO-Net. Front Mar Sci 6:521

    Article  Google Scholar 

  15. Dietrich JT (2017) Bathymetric structure-from-motion: extracting shallow stream bathymetry from multi-view stereo photogrammetry. Earth Surf Proc Land 42(2):355–364

    Article  Google Scholar 

  16. Fryer JG (1983) Photogrammetry through shallow water. Aust J Geodesy Photogramm Surv 38:25–38

    Google Scholar 

  17. Georgopoulos A, Agrafiotis P (2012) Documentation of a submerged monument using improved two media techniques. In: 2012 18th international conference on virtual systems and multimedia, IEEE, pp 173–180

  18. Hinterstoisser S, Lepetit V, Wohlhart P, Konolige K (2018) On pre-trained image features and synthetic images for deep learning. In: Proceedings of the European conference on computer vision (ECCV)

  19. IHO (2019) Zones of confidence. IHO

    Google Scholar 

  20. IHO (2020) Iho s-44 edition 6.0.0. IHO

    Google Scholar 

  21. Kahmen O, Rofallski R, Conen N, Luhmann T (2019) On scale definition within calibration of multi-camera systems in multimedia photogrammetry. Int Arch Photogramm Remote Sens Spatial Inf Sci XLII-2/W10, 93–100

  22. Kahmen O, Rofallski R, Luhmann T (2020) Impact of stereo camera calibration to object accuracy in multimedia photogrammetry. Remote Sens 12(12):2057

    Article  Google Scholar 

  23. Li C, Zhang J, Guo B (2011) New method of landslide monitoring based on close-range photogrammetry. Jisuanji Gongcheng yu Yingyong (Comput Eng Appl) 47(3):6–8

    Google Scholar 

  24. Luhmann T (2016) Learning photogrammetry with interactive software tool PhoX. Int Arch Photogramm Remote Sens Spatial Inf Sci 41:39

    Article  Google Scholar 

  25. Maas HG (2015) On the accuracy potential in underwater/multimedia photogrammetry. Sensors 15(8):18140–18152

    Article  Google Scholar 

  26. Mandlburger G (2018) A case study on through-water dense image matching. ISPRS Int Arch Photogramm Remote Sens Spatial Inf Sci XLII–2:659–666. https://doi.org/10.5194/isprs-archives-XLII-2-659-2018

    Article  Google Scholar 

  27. Mandlburger G (2019) Through-water dense image matching for shallow water bathymetry. Photogramm Eng Remote Sens 85(6):445–455

    Article  Google Scholar 

  28. Mandlburger G, Pfennigbauer M, Schwarz R, Flöry S, Nussbaumer L (2020) Concept and performance evaluation of a novel UAV-borne topo-bathymetric LiDAR sensor. Remote Sens 12(6):986

    Article  Google Scholar 

  29. Menna F, Nocerino E, Remondino F (2018) Photogrammetric modelling of submerged structures: influence of underwater environment and lens ports on three-dimensional (3d) measurements. Latest developments in reality-based 3D surveying and modelling MDPI Basel, Switzerland pp 279–303

  30. Mulsow C (2010) A flexible multi-media bundle approach. Int Arch Photogramm Remote Sens Spat Inf Sci 38:472–477

    Google Scholar 

  31. Mulsow C, Mandlburger G, Maas HG (2020) Comparison of subaquatic digital elevation models from airborne laser scanning and imagery. ISPRS Ann Photogramm Remote Sens Spatial Inf Sci V–2–2020:671–677. https://doi.org/10.5194/isprs-annals-V-2-2020-671-2020

    Article  Google Scholar 

  32. Nesbit PR, Hugenholtz CH (2019) Enhancing UAV-SfM 3D model accuracy in high-relief landscapes by incorporating oblique images. Remote Sens 11(3):239

    Article  Google Scholar 

  33. Pedregosa F, Varoquaux G, Gramfort A, Michel V, Thirion B, Grisel O, Blondel M, Prettenhofer P, Weiss R, Dubourg V, Vanderplas J, Passos A, Cournapeau D, Brucher M, Perrot M, Duchesnay E (2011) Scikit-learn: machine learning in Python. J Mach Learn Res 12:2825–2830

    Google Scholar 

  34. Peng X, Sun B, Ali K, Saenko K (2015) Learning deep object detectors from 3d models. In: Proceedings of the IEEE international conference on computer vision, pp 1278–1286

  35. Richardson E, Sela M, Kimmel R (2016) 3d face reconstruction by learning from synthetic data. In: 2016 fourth international conference on 3D vision (3DV), IEEE, pp 460–469

  36. Ros G, Sellart L, Materzynska J, Vazquez D, Lopez AM (2016) The synthia dataset: a large collection of synthetic images for semantic segmentation of urban scenes. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 3234–3243

  37. Saleh FS, Aliakbarian MS, Salzmann M, Petersson L, Alvarez JM (2018) Effective use of synthetic data for urban scene semantic segmentation. European conference on computer vision. Springer, New York, pp 86–103

    Google Scholar 

  38. Sankaranarayanan S, Balaji Y, Jain A, Nam Lim S, Chellappa R (2018) Learning from synthetic data: addressing domain shift for semantic segmentation. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 3752–3761

  39. Shan J (1994) Relative orientation for two-media photogrammetry. Photogramm Rec 14(84):993–999

    Article  Google Scholar 

  40. Shrivastava A, Pfister T, Tuzel O, Susskind J, Wang W, Webb R (2017) Learning from simulated and unsupervised images through adversarial training. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 2107–2116

  41. Skarlatos D, Agrafiotis P (2018) A novel iterative water refraction correction algorithm for use in structure from motion photogrammetric pipeline. J Mar Sci Eng 6(3):77

    Article  Google Scholar 

  42. Skarlatos D, Georgopoulos A (2006) The method of two overlapping orthoimages for checking the produced DTM. Technical report

  43. Smith MW, Vericat D (2015) From experimental plots to experimental landscapes: topography, erosion and deposition in sub-humid badlands from structure-from-motion photogrammetry. Earth Surf Proc Land 40(12):1656–1671

    Article  Google Scholar 

  44. Sun Y, Sun H, Yan L, Fan S, Chen R (2016) Rba: Reduced bundle adjustment for oblique aerial photogrammetry. ISPRS J Photogramm Remote Sens 121:128–142

    Article  Google Scholar 

  45. Vapnik VN, Chervonenkis AY (2015) On the uniform convergence of relative frequencies of events to their probabilities. Measures of complexity. Springer, New York, pp 11–30

    Chapter  Google Scholar 

  46. Wang Z (1990) Principles of photogrammetry: (with remote sensing). Press of Wuhan Technical University of Surveying and Mapping, Wuhan

    Google Scholar 

  47. Wimmer M (2016) Comparison of active and passive optical methods for mapping river bathymetry. TU Wien, Vienna

    Google Scholar 

  48. Woodget A, Carbonneau P, Visser F, Maddock IP (2015) Quantifying submerged fluvial topography using hyperspatial resolution uas imagery and structure from motion photogrammetry. Earth Surf Proc Land 40(1):47–64

    Article  Google Scholar 

Download references

Acknowledgements

We are grateful to the reviewers for their valuable comments and suggestions. Also, we would like to acknowledge the Dep. of Land and Surveys of Cyprus for providing the LiDAR reference data, the Cyprus Dep. of Antiquities for permitting the flight over the Amathouda site and commissioning the flight over Agia Napa, and HFF for partially funding the acquisition of Cyprus’ datasets.

Author information

Affiliations

Authors

Corresponding author

Correspondence to Panagiotis Agrafiotis.

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Agrafiotis, P., Karantzalos, K., Georgopoulos, A. et al. Learning from Synthetic Data: Enhancing Refraction Correction Accuracy for Airborne Image-Based Bathymetric Mapping of Shallow Coastal Waters. PFG 89, 91–109 (2021). https://doi.org/10.1007/s41064-021-00144-1

Download citation

Keywords

  • Bathymetry
  • Refraction correction
  • UAV
  • Airborne
  • Synthetic data
  • Support vector regression
  • Machine learning
  • Shallow waters
  • Coastal mapping