Introduction

Visual mapping of the seafloor is a versatile tool for underwater research with applications including benthic habitat mapping [1,2,3], survey of mineral resources [4], and mapping of archaeological sites [5]. In particular, the 3D imaging techniques developed in recent years can generate high resolution digital reconstructions of the seafloor [3, 4, 6, 7]. The addition of colour information enhances the representation of the seafloor compared to conventional bathymetric mapping techniques, and potentially allows for a greater diversity of classification and statistical analysis of features of interest. As such it also is a versatile tool for assessing and monitoring the human impact on the sea bottom environment.

The application of ecological survey and surveying slow changes in highly heterogeneous environments such as deep-sea hydrothermal vent fields can be broken down into the following three, independent points. The mapping system must:

  • Have sufficiently high resolution and colour fidelity to quantitatively measure physically small (centimetre order) features and correctly identify animals or deposits from their morphology and colour.

  • Achieve sufficient area coverage (in the order of hectares) to collect data that is representative of the targeted environment.

  • Be robust to changes in measurement conditions caused by the imperfect altitude control of the robotic platform due to rugged underwater terrains and underwater currents and the variable turbidity of seawater which affect colour attenuation.

Collecting meaningful ecological and geological data in hydrothermal environments over statistically relevant spatial scales requires all three of the above points to be fulfilled, which is not possible with existing underwater imaging systems and algorithms used to process their data. This study overcomes these technical challenges and realises such a system by extension of the long-baseline structured light and colour mapping algorithm described in [4] to use imagery from a separate, high-sensitivity colour camera synchronised with a high-power strobe. This extension allows us to achieve centimetre order resolution 3D bathymetric measurements and retain colour information through the use of the strobed images taken from significantly higher altitudes than was previously possible (10 m instead of 2 m). The ability to map and image from high altitudes at the necessary resolution and colour fidelity makes it possible to cover significantly larger areas than was previously possible (1000–10,000 m2/h) making it feasible now to map an entire hydrothermal site in a single autonomous underwater vehicle (AUV) or remotely operated vehicle (ROV) deployment.

The system was deployed in the hydrothermally active Iheya North field to map megabenthos [8] and hydrothermal deposits [9] around and on top of artificial hydrothermal vents. The Iheya North field is a kilometre scale active deep-sea hydrothermal field off the west coast of Okinawa, at a depth of approximately 1000 m [10]. It was drilled during the Integrated Ocean Drilling Program (IODP) 331 “Deep Hot Biosphere” expedition during September and October 2010 [11, 12]. Surveys that followed the IODP 331 drilling campaign reported noticeable changes in the seafloor landscape and growth of mineral deposits near the drill site [13]. In addition, significant increases in the amount of mega benthos were reported near the C0014 drill site which was previously not inhabited [14]. While it is clear that the changes in the environment observed following a human drilling effort have direct relevance when considering the future deep-sea mining, the observations made were either qualitative in nature or highly localised and it is clear that there is a need to document these effects quantitatively over spatial scales that are sufficient to draw meaningful conclusions regarding the effect on the entire hydrothermal site in order to inform future decisions. In order to illustrate how this can be practically achieved, we introduce a method for achieving wide-area 3D visual reconstruction and demonstrate its application by quantitatively mapping the distribution of Shinkaia crosnieri squat lobsters, which is a representative species of megabenthos in the Iheya North Field and was reported in large numbers at the drill affected area [14], and also demonstrate how the growth of deposits in the drilled areas can be quantitatively determined from the high resolution bathymetry obtained by this technology.

During the NT12-27 cruise of the Japan Agency for Marine Science and Technology (JAMSTEC), the long range “SeaXerocks 2” mapping system developed in this work was mounted on the ROV Hyper-Dolphin to obtain 3D imagery of the seafloor between three artificial hydrothermal vents and the North Big Chimney (NBC) mound. 3D colour reconstructions of the seafloor have been generated using the mapping algorithm described in [4], that has been adapted for long range mapping using multiple synchronised flashes and a high power sheet laser for seafloor illumination and bathymetric measurements, respectively.

To date, surveys of this type were performed in a series of hierarchical stages that lose a significant amount of information at each stage. Typically, broad scale site surveys relied on acoustic methods, such as side-scan sonar and multibeam sonar surveys that can cover entire, multi-hectare hydrothermal sites. However the resolution of the data collected by these systems is in the order of 0.5–2 m and does not contain colour information. It is not possible to measure growth that takes place in these environments due to the slow rates (<1 m/year) and biota (<0.5 m) cannot be recognised. The information is augmented typically by small footprint sampling or imaging surveys of regions that are smaller than 1000 m2 in various, characteristic areas identified by the acoustic surveys. This type of data however, can only give statistical information about the relative abundance of animals and cannot be used for quantitative studies or for temporal comparisons on the scale of a entire site. The technique described is transformative from the viewpoint of its application to mapping hydrothermal sites because it gives biologists and geologists the ability to perform quantitative whole-site surveys from a single AUV or ROV deployment.

Mapping device and generation of 3D maps

Fig. 1
figure 1

Illustration of the 3D visual mapping method. While the underwater vehicle moves forward, the sheet laser scans the seafloor and a camera takes images of the laser line projection, from which a high resolution bathymetry map can be generated. Flashed images are taken at a lower frequency, but frequent enough to cover the same area with some overlap between the photos, from which the colour can be mapped to the bathymetry

Full size image

The mapping method, illustrated in Fig. 1, is based on laser profiling extended to use colour. In this study, we extend the method described in our previous work [4] in order to achieve mapping from significantly higher altitudes than was previously possible. In our previous work, the algorithm assumed a single camera to capture both the laser line projection and the illuminated area of the seafloor. While this is efficient in terms of hardware requirements, it also means that the choice of camera and method for illumination become limited. In order to achieve mapping from higher altitudes, the requirements for collecting images of the laser projection and those required for imagery diverge. First, images of the laser profile must be collected at a fast frame rate to maintain a high resolution in the forward direction. Since the mapping altitude is high, it is desirable to have a large baseline between the laser and the camera observing the laser projection to achieve a high resolution in the vertical direction, which typically requires the camera and the laser to be at opposite ends of the vehicle. Since the ability to map from high altitudes also makes mapping possible over more rugged terrains, it is also desirable to tilt the camera towards the laser, in order to maintain a large range of altitudes over which the projection is inside the cameras’ field of view. The requirements for collecting high quality imagery are fundamentally different. First, the footprint of an illuminated image is large, and scales linearly with altitude. Therefore, for the large footprints made possible at high altitude, it is sensible to record images at a relatively slow repetition rate and use strobes delivering a more powerful impulse of light than continuous lights are capable of. Second, the rugged shape of the terrain means that the camera becomes susceptible to occlusions in areas with large profiles, and that shadows formed by the terrain can be detrimental. In order to minimise the effect of shadowing, it is desirable to have multiple strobes in an array with the camera central and looking vertically downwards. Therefore, unlike the camera used for laser detection, the best position to have the camera for imaging is in the centre of the vehicle, with the strobes located at the corners of the vehicle. In order to make this possible, the algorithm was extended to process imagery data from a dedicated camera and model the light path of any number of strobes located at arbitrary locations around the camera, to allow high colour fidelity reconstructions to be generated.

Fig. 2
figure 2

The SeaXerocks 2 mapping device mounted on the ROV Hyper-Dolphin payload skid

Full size image

The mapping system that was used is shown in Fig. 2. It consists of high sensitivity cameras to collect seafloor images illuminated by four synchronised Xenon flashes, with a total illumination of \(\sim\)3 klms emitted per pulse, distributed around the camera, and a separate camera to capture the sheet laser projection at a high frame rate. The setup allowed us to make full use in exploiting the advantages offered by the modified mapping algorithm. The laser profiling system employs a 532 nm, 120 mW sheet laser, offset 1.027 m from the laser line camera (tilted) that captures images of the laser line projection at a rate of 15 frames per second. This enables the setup to acquire shape information from high altitudes. It is robust in moderately turbid deep-sea environments because there is no backscatter from continuous lights, which causes particles to appear blurred and larger than their actual dimensions, and the laser line provides a clear visual feature that is easy to automatically extract. The camera mounted vertically is synchronised with the flashes and captures images from which colour is applied to the 3D reconstruction.

Navigation data recorded from a depth sensor and a Doppler velocity log (DVL) with a compass and a tilt sensor are used in the algorithm to determine the relative position from which the images were taken. The surface vessel’s super short baseline (SSBL) acoustic positioning system is used to georeference the 3D map. The mapping device is rated to 3000 m depth and can be deployed by an AUV or ROV.

During mapping, the sheet laser projects a line perpendicular to the direction of motion on the seafloor. This is captured by a camera with a translational offset to the laser and is used to calculate the vertical profile of the illuminated section. As the vehicle moves, the seafloor is scanned and the set of images of the laser line projection allow the bathymetry of the scanned area to be generated. With the setup used during the cruise to the Iheya North field, the resolution along the track is 20 mm, across the track 7 and 56 mm in vertical direction when mapping from an altitude of 8 m at a forward speed of 30 cm/s. From 10 m altitude the resolution is 20 mm along track, 9 mm across track and 87 mm in vertical direction. While the resolution of the data obtained in the forwards direction does not change with altitude, the localisation accuracy of the seafloor measurements reduces with altitude due to the inherent uncertainty of the pitch and roll sensors on the underwater vehicle and inherent uncertainties in the calibration of the lens, both of which increase the position uncertainty of the point measured on the seafloor in Cartesian coordinates. While the cumulative effects of these are not negligible, both effects scale linearly with altitude and are in the order of centimetres at an altitude of 8 m.

Fig. 3
figure 3

Flowchart showing the basic steps of the data processing algorithm

Full size image

Flashed photos are taken every 6 s, leading to an overlap of 75% at an altitude of 8 m, from which the colour is applied to each bathymetry point in order to obtain a coloured 3D digital representation of the seafloor. Data collected during each dive is processed off-line by the algorithm whose basic steps are illustrated in Fig. 3. Since the flash imagery from which colour information is retrieved and the images of the laser line projection are asynchronous, an algorithm was developed to associate colour information from each point of bathymetry based on the vehicle’s localisation estimate. The location of the camera with flashed imagery is determined from the vehicle’s position and the geometry of the camera setup. Based on this location and the camera calibrations, it is possible to find the projection of each pixel in the illuminated images on the seafloor, and so determine the colour information that corresponds to each point of bathymetry measured. The colour values are corrected for the effects of wavelength dependent attenuation and the beam pattern of the flashes. Since it is likely that the bathymetry point appears in more than one flashed image, a weighted average is computed, where values from images where the backprojection of the bathymetry point is closer to the centre of the image have a larger weight than corresponding pixels close to the border of the image in order to minimise the effects of uncertainty in the calibration and light attenuation model. The so generated coloured point cloud is then converted into a mesh using Delaunay triangulation and output as a ply file, which can be easily viewed and lends itself well to further processing of the data to directly extract the numerical information needed for scientific analysis. Based on measurements of known features in the 3D reconstruction, it can be determined that the dimensional accuracy is \(\sim\)12.5%.

Results from deployment at sea

Fig. 4
figure 4

3D map generated from data collected by the SeaXerocks 2 mapping system in 1 h, overlaid on the depth map of dive site. While all the data was recorded in high resolution, three areas of interest, which are the artificial hydrothermal chimneys, are shown enlarged

Full size image

On the NT12-27 cruise, mapping operations were performed using the ROV Hyper-Dolphin from the research vessel Natsushima for 1 h. Compressed images were sent up over a RS-232 data link for checking the image quality in real time. A total of 13,532 images were obtained along a 920 m trajectory from an average altitude of 7.7 m, resulting in a seafloor data acquisition rate of 8144 m\(^2\)/h at an average pixel resolution of 6.4 mm. This represents an order-of-magnitude increase in the area data acquisition rate compared to conventional colour imaging of the seafloor where images are obtained from an altitude of 2–3 m [1, 4]. The top-view of the 3D map generated from this data overlaid on an acoustic bathymetry map is shown in Fig. 4, with enlargements of the three artificial hydrothermal chimneys C0013E, C0014G and C0016B.

In the mapped area accumulations of hydrothermal deposits, as well as numerous S. crosnieri squat lobsters [15] are visible. Both the deposits and S. crosnieri, which feed on methane and sulfide reducing bacteria are indicators of hydrothermal activity. The mapping data was used to quantify both the volume of deposits as well as the number and distribution of S. crosnieri in the mapped area.

Mapping megabenthos distribution

Fig. 5
figure 5

3D reconstruction of the area around the C0014G artificial hydrothermal vent mapped in a single swath. Detail view A shows a colony of S. crosnieri. Detail view B shows the open drill hole C0014D/E, which is also populated by a large number of S. crosnieri

Full size image

Figure 5 shows the C0014G artificial hydrothermal vent and its surroundings. The reconstruction was generated from a single pass over the artificial chimney from an altitude of about 10 m and it covers an area of 37.5 m by 15.5 m. This was mapped in 2 min 40 s. The white hydrothermally altered region extends west of the vent for a distance of 24 m, and is densely populated by S. crosnieri, visible in enlargement A. Their appearance in the map is relatively small, but the species was determined from sampled individuals and they were also identified in ROV footage taken from closer range. It is also known that there are no other animals of similar size or colour living in this specific area. The depression seen 4.7 m west of the artificial vent, enlarged in detail view B, is drill hole C0014D/E made during the IODP 331 expedition and large numbers of S. crosnieri are also seen inside of it.

Fig. 6
figure 6

Population density map of S. crosnieri at chimney C0014G. The density map of the S. crosnieri squat lobsters is overlaid on a top view of the 3D reconstruction of the area around the C0014G artificial hydrothermal vent

Full size image
Fig. 7
figure 7

Histogram showing the relationship between the number of S. crosnieri and the radial distance from the chimney mouth of chimney C0014G

Full size image

The S. crosnieri were identified in a top view of the 3D reconstruction and manually labelled. From this data the population density maps for the area around the natural and the three artificial hydrothermal vents were generated using a 2D Gaussian window with a standard deviation of 15 cm. Figure 6 shows the 3D map of chimney C0014G with the S. crosnieri population density overlaid. The total number of S. crosnieri found in the area was 3140 and the highest concentration was 349 individuals/m2. The map shows that the highest population densities are not necessarily found nearest the artificial vent but rather in the surrounding areas. The histogram of the number of S. crosnieri in relation to the distance from the mouth of the artificial chimney shown in Fig. 7 shows that the majority of S. crosnieri are found at a distance of 6–14 m from the chimney. There was a particularly high population density of S. crosnieri in and around the drill hole C0014D/E (feature B in Fig. 5) west of the artificial chimney. It is possible that there may be hydrothermal diffuse flows seeping up through the ground in these areas, which are more suitable for supporting sea life than the strong flow gushing out of the artificial vent [14, 16].

Fig. 8
figure 8

Top view of a 3D colour reconstruction of the C0016B artificial hydrothermal vent and its surroundings

Full size image
Fig. 9
figure 9

Top view of the C0016B artificial hydrothermal vent with the density map of S. crosnieri overlaid

Full size image

Figure 8 shows the top view of the artificial chimney C0016B with its surroundings and Fig. 9 shows the population density maps of S. crosnieri. There are significantly less S. crosnieri than around chimney C0014G (261 individuals at C0016B vs. 3140 at C0014G) and they are mainly found close to the base of the artificial chimney. The histogram shown in Fig. 10 also shows that the peak density is closer to the artificial chimney than in the case of C0014G.

Fig. 10
figure 10

Histogram showing the relationship between the number of S. crosnieri and the radial distance from the chimney mouth of chimney C0016B

Full size image
Fig. 11
figure 11

Top view of 3D colour reconstruction of the NBC mound vent

Full size image

Figure 11 shows the top view of a transect going over the NBC mound and Fig. 12 shows the population density map of S. crosnieri in this area. A total number of 5025 S. crosnieri were identified in the 3D reconstruction with a maximum density of 210 individuals/m2. The actual total population is likely to be larger, because some parts of the 3D reconstruction are not clearly visible due to saturation in some of the photos, in particular around the peak and the reconstruction only covers part of the chimney.

Fig. 12
figure 12

Top view of the 3D reconstruction of the area around the NBC natural hydrothermal chimney with the density map of S. crosnieri overlaid

Full size image
Table 1 Number and density of S. crosnieri (S.c.) identified in the Iheya North field drilling sites
Full size table

Around the artificial chimney C0013E, shown in the enlargement at the bottom left of Fig. 4, no S. crosnieri were found. Unlike C0014G and C0016B the vent is not active [13] and so it cannot support chemosynthetic primary production. Table 1 summarises the numerical data of the S. crosnieri population around the three artificial and the one natural hydrothermal vent (NBC).

Deposit volume and mass estimation

Among the three artificial hydrothermal vents, chimney C0016B has the largest amount of deposit growing on top of the artificial structure. Before the area was mapped during the NT12-27 cruise deposits at this location were sampled by the ROV, meaning that little deposit was left at the time of mapping. There are also some deposits on the seafloor directly next to the artificial chimney. This was not there when the chimney was installed in September 2010 and so it can be concluded that this has grown in the two year period between when the site was drilled and mapped [11].

Fig. 13
figure 13

Photo taken by ROV and depth map of chimney C0016B’s top plate with the remains of the hydrothermal deposits

Full size image

An algorithm was developed that analyses the 3D outputs of the mapping system and calculates the volume of protrusion of a feature. It was used to compute the amount of deposit on the top plate of chimney C0016B. Figure 13 shows the elevation map of the top plate based on the mapping data. A photo of the top plate taken using the ROV forward looking camera is also shown. The deposit on the top plate has a volume of 0.032 ± 0.007 m3.

Based on hydrothermal deposits sampled by the ROV, the dry density was measured to be 2.6 g/cm3 with a standard deviation of 0.16 g/cm3. Similar values were reported in [17]. Using the above-mentioned density measurements, the dry mass of deposits on the chimney top is estimated to be 83 ± 18 kg.

Conclusions

A long range 3D imaging system has been demonstrated that can generate 3D colour reconstructions of the seafloor from altitudes of more than 10 m. The data presented shows that the method allows the seafloor to be mapped both rapidly and with high resolution. The system is as a powerful tool to study the marine environment on hectare order scales. Entire vent colonies can potentially be mapped in a short time [18], and the resolution it achieves is sufficient to map deposits protruding by as little as a few centimetres from their surroundings. The applications to quantify megabenthos and deposit volumes illustrate the opportunities high resolution visual and bathymetric data offers scientific studies. It is also hoped that data from repeated surveys will help document the changes of the environment over a longer time span and so make it possible to better assess the impact of human drilling activities in deep-sea environments.

Future directions for the technology relate mainly to the interpretation of data and verification of measurement precision and repeatability. In particular, the data collection for this survey took 1 h and the processing computational time is in the order of a few hours. However, the manual labelling of biota took five person days to achieve and is the obvious bottleneck in the analysis. Methods to assist our interpretation of the data through partial automation can allow for more efficient interpretation and so push forward our understanding of deep-sea processes. With regard to the precision and repeatability of the method, it is necessary to perform repeat surveys and analyse the uncertainties introduced at the various stages of data collection, processing and interpretation. This type of data will be crucial for future, temporal studies in order to identify trends and changes in environments as the minimum unit of change will be strongly coupled to the repeatability of the method described.