Abstract
Hydrographic offices hold large valuable historic bathymetric data sets, many of which were collected using older generation survey systems that contain little or no metadata and/or uncertainty estimates. These bathymetric data sets generally contain large outlier (errant) data points to clean, yet standard practice does not include rigorous automated procedures for systematic cleaning of these historical data sets and their subsequent conversion into reusable data formats. In this paper, we propose an automated method for this task. We utilize statistically diverse threshold tests, including a robust least trimmed squared method, to clean the data. We use LOESS weighted regression residuals together with a Student-t distribution to attribute uncertainty for each retained sounding; the resulting uncertainty values compare favorably with native estimates of uncertainty from co-located data sets which we use to estimate a point-wise goodness-of-fit measure. Storing a cleansed validated data set augmented with uncertainty in a re-usable format provides the details of this analysis for subsequent users. Our test results indicate that the method significantly improves the quality of the data set while concurrently providing confidence interval estimates and point-wise goodness-of-fit estimates as referenced to current hydrographic practices.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
Notes
See Tharp and Frankel (1986) for a historical perspective on intuitive methodologies.
Not all current surveys provide uncertainty either.
These standards apply to current, but not historic, data sets.
We assume independent white noise in valid data points.
Our Type A uncertainty follows the NIST guidelines (Taylor and Kuyatt 1994).
50% is used, as it is the threshold for breakdown.
For computational efficiency, our LOESS regression uses 1/3 of actual samples provided in the ping window. Alternatively, we could scale to use a portion of the beams within a swath. For the sample datasets, the USNS Henson has a swath consisting of 121 samples. Thus 10 pings provides a sample set of 1210 samples of which approximately 400 were used in the weighted LOESS regression with those closest receiving much higher weights. Dividing the ping in half would have resulted in 605 samples with approximately 200 used in the LOESS regression.
The thresholding method used is sensitive to bathymetric aliasing that changes with the resolution and/or terrain. Here, the resolution of our reference surface and the footprint size of the survey system are comparable, so this issue does not affect the results shown here.
For computational efficiency, our LOESS regression uses 1/3 of actual samples provided in the ping window. Alternatively, we could scale to use a portion of the beams within a swath. For the sample datasets, the USNS Henson has a swath consisting of 121 samples. Thus 10 pings provide a sample set of 1210 samples of which approximately 400 were used in the weighted LOESS regression with those closest receiving much higher weights. Dividing the ping into half would have resulted in 605 samples with approximately 200 used in the LOESS regression.
References
Becker JJ, Sandwell DT, WHF Smith, Braud J, Binder B, Jl Depner, Fabre D et al (2009) “Global bathymetry and elevation data at 30 Arc seconds resolution: SRTM30_PLUS”. Mar Geodesy 32(4):355–371. doi:10.1080/01490410903297766
Bourgeois, B, Elmore P, Avera W, Zambo S (2016) “Achieving comparable uncertainty estimates with kalman filters or linear smoothers for bathymetric data”. Geochem Geophys Geosyst 17:2576–2590. doi:10.1002/2015GC006239
Calder BR (2006) “On the uncertainty of archive hydrographic datasets”. IEEE J Oceanic Eng 312:249–265. doi:10.11109/JOE.2006.872215
Calder BR, Garder JV (2008) “U.S. Law of the sea cruise to map the foot of the slope of the Northeast U.S. Atlantic Continental Margin: Leg 6”, University of New Hampshire (UNH), Center for Coastal and Ocean Mapping (CCOM)/Joint Hydrographic Center (JHC). http://ccom.unh.edu/sites/default/files/publications/Calder_08_cruise_report_Atlantic_KNOX17RR_0.pdf. Accessed 10 Mar 2017
Calder B, Mayer LA (2003) “Automatic processing of high-rate, high-density multibeam echosounder data”. Geochem Geophys Geosyst (American Geophysical Union) 4(6):1048. doi:10.1029/2002GC000486.
Chandler MT, Wessel P (2008) “Improving the quality of marine geophysical track line data: along-track analysis”. J Geophys Res Solid Earth 113(B2):B02102. doi:10.1029/2007JB005051
Cleveland WS (1979) “Robust locally weighted regression and smoothing scatterplots”. J Am Stat Assoc 74:829–836
Cleveland WS, Devlin SJ (1988) “Locally-weighted regression: an approach to regression analysis by local fitting”. J Am Stat Assoc 83:596–610. doi:10.2307/2289282
Debese N, Moitie R, Seube N (2012) “Multibeam echosounder data cleaning through a hierarchic adaptive and robust local surfacing”. Comput Geosci 46:330–339. doi:10.1016/j.cageo.2012.01.012
Doornik, JA (2011) Robust estimation using least trimmed squares. economic department, Oxford University, Oxford, United Kingdom: Economics Department, University of Oxford. http://econ.au.dk/fileadmin/site_files/filer_oekonomi/subsites/creates/Seminar_Papers/2011/ELTS.pdf
Draper N, Smith H (1981) Applied regression analysis, 2nd edn. Wiley, New York
Elmore P, Fabre D, Sawyer R, Ladner R (2012) “Uncertainty estimation of historical bathymetric data from bayesian networks”. Geochem Geophys Geosyst 13 (11). doi:10.1029/2012GC004144
Free Software Foundation (2015) GSL—GNU scientific library, Version 2.1. http://www.gnu.org/software/gsl/. Accessed May 15 2016
Galassi M, Davies J, Theiler J, Gough R, Jungman G, Alken P, Booth M, Rossi F (2009) GNU scientific library reference manual, 3rd edn. Network Theory Ltd, London
Golub GH, Reinsch C (1970) “Singular value decomposition and least squares solutions”. Numer Math 14:403–420. doi:10.1007/BF02163027
Hare R (1995) “Depth and position error budgets for multibeam echosounding”. Int Hydrogr Rev 72:37–69. https://journals.lib.unb.ca/index.php/ihr/article/viewFile/23178/26953
Hare R, Eakins B, Amante C (2011) “Modelling bathymetric uncertainty”. Int Hydrogr Rev 6:31–42. https://journals.lib.unb.ca/index.php/ihr/article/view/20888. Accessed 31 May 2016
IHO (2008) IHO standards for hydrographic surveys: special publication No. 44 Fifth edition. International Hydrographic Bureau, Monaco, p 28
Jakobsson M, Mayer LA, Coakley B, Dowdeswell JA, Forbes S, Fridman B, Hodnesdal H et al. (2012) “The international bathymetric chart of THE arctic ocean (IBCAO) version 3.0.” Geophys Res Lett. doi:10.1029/2012GL052219
JCGM (2008) “Evaluation of measurement data—Guide to the expression of uncertainty in measurement.” http://www.bipm.org/utils/common/documents/jcgm/JCGM_100_2008_E.pdf. September. Accessed May 31 2016. http://www.bipm.org/utils/common/documents/jcgm/JCGM_100_2008_E.pdf
Ladner R, Moseley J (2002) “Managing heterogeneous databases to support diverse product requirements”. OCEANS ‘02 MTS/IEEE. IEEE. 895–899. doi:10.1109/OCEANS.2002.1192086
Lawson CL, Hanson RJ (1995) Solving least squares problems. S.I.A.M
Mount DM, Netanyahu NS, Piatko CD, Silverman R, Wu AY (2014) “On the least trimmed squares estimator”. Algorithmica 69:148–183. doi:10.1007/s00453-012-9721-8
Press WH, Teukolsky SA, Vetterling WT, Flannery BP (2007) Numerical recipes—the art of scientific computing. 3rd. Cambridge University Press, New York
Rousseeuw PJ, Leroy AM (1987) Robust regression and outlier detection. Wiley, New York
Ryan WBF, Carbotte SM, Coplan JO, O’Hara S, Melkonian A, Arko R, Weissel RA et al (2009) “Global multi-resolution topography synthesis.” Geochem Geophys Geosyst. doi:10.1029/2008gc002332
Smith WH, Sandwell DT (1997) “Global sea floor topography from satellite altimetry and ship depth soundings.” Science 277. doi:10.1126/science.277.5334.1956
Taylor BN, Kuyatt CE (1994) “NIST technical note 1297 guidelines for evaluating and expressing the uncertainty of NIST measurement results”. http://nist.gov/pml/pubs/tn1297/. September. Accessed May 31, 2016
Tharp M, Frankel H (1986) “Mappers of the deep”. Nat Hist 95:49–62. http://www.faculty.umb.edu/anamarija.frankic/files/Oceanography%20Spring%2008/MidAtlantic%20ridge%20discovery.pdf.gz
Weatherall P, Marks KM, Jakobsson M, Schmitt T, Tani S, Arndt JE, Rovere M, Chayes D, Ferrini V, Wigley R (2015) “A new digital bathymetric model of the world’s oceans”. Earth Space Sci. doi:10.1002/2015EA000107
Weber JR (1983) “Maps of the arctic basin sea floor: a history of bathymetry and its interpretation”. Arctic 36 (2):121–142. http://www.jstor.org/stable/40509566
Wessel P, Chandler MT (2011) “The spatial and temporal distribution of marine geophysical surveys”. Acta Geophys 59(1):55–71. doi:10.2478/s11600-010-0038-1
Acknowldegements
We would like to thank the Office of Naval Research for sponsoring this research under the Worldwide High-Resolution Bathymetry project of the Naval Research Laboratory’s Base Program.
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Ladner, R.W., Elmore, P., Perkins, A.L. et al. Automated cleaning and uncertainty attribution of archival bathymetry based on a priori knowledge. Mar Geophys Res 38, 291–301 (2017). https://doi.org/10.1007/s11001-017-9304-9
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11001-017-9304-9