1 Introduction

Computer vision has received a boost in the last 30 years mainly because the recent inexpensive and technically improved digital cameras are attractive for low-cost measuring systems (Jain et al. 1995; Shapiro and Stockman 2001). Image-based measuring systems are suitable for laboratories of hydraulics and have been used in several experimental practices (Erikson and Hanson 2005; Yao and Wu 2005; Iglesias et al. 2009; Wang et al. 2012; Gabl et al. 2018). In order to study impulsive water wave generated by landslides, a specific application of image-based measurements has been set up in the Fluvial Morphdodynamics I laboratory of the Sediment Transport Research Group (GITS) in the Technical University of Catalonia (Bregoli et al. 2017). When a landslide falls into a water body, a huge impulsive wave can be generated (Kamphuis and Bowering 1970; Ward 2001; Mergili et al. 2018). This wave is considered a tsunami for its great amplitude and length, its capacity of traveling long distances as well as its potential destructivity. The mentioned experimental facility has been set up to define empirical relationships between landslide parameters and generated wave characteristics as tools useful for risk assessment. The system consists of a landslide generator releasing a granular mass of white gravel at high speed (up to 7 m/s) in a wave basin of 4.10 × 2.44 m with a water depth hw ≥ 0.20 m (Fig. 1). The landslide generator is formed by a sliding box of 1.00 m of length, 0.60 m of height and 0.34 m of width. The box is filled with the gravel and runs down a high-slope flume. When the box suddenly stops at the end of the flume, the flaps instantly open and the granular material is launched into the wave basin (Figs. 2 and 3).

Fig. 1
figure 1

3D rendering of the landslide tsunamis laboratory setup consisting in a landslide generator and a wave basin. The landslide generator is made of a metallic box of 1 m length, 0.60 m height and 0.34 m width. The box is sliding along a chute of 6.20 m length and variable slope. The chute and the wave basin bottom are connected by a wedge which have the same chute slope

Full size image
Fig. 2
figure 2

Sketch of the measuring system including the array of cameras recording landslide at impact and water surface displacement along the green laser sheet

Full size image
Fig. 3
figure 3

Granular material released in water basin after flaps open. Frame extracted from a demonstrative video clip of the HS camera recording at 640 f/s. The arrow identifies the mass movement direction

Full size image

Particle image velocimetry techniques have been previously employed in similar experiments to evaluate the velocity field at the impact zone (Fritz et al. 2003; Sælevik et al. 2009; Sue et al. 2011; Mohammed and Fritz 2012; Viroulet et al. 2013; Whittaker et al. 2015; Mulligan and Take 2017; McFall et al. 2018). In the previous applications, camera sensor and plane of measurement are parallel as a requirement. When the landslide and water wave propagate along a glass-wall straight channel (2D wave experiments), the above-mentioned parallel condition could be easily achieved. But in 3D experiments, where landslide and wave propagate in a rectangular water tank, the parallel condition between camera and plane of measurement is not easily achievable. McFall et al. (2018) overcome this issue by perpendicularly filming the granular landslide with aerial and underwater cameras to measure landslide thickness. However, the problem of measuring the water surface displacement (wave) at the tank center still exists.

In the present application, a system based upon computer vision techniques and an extensive use of images post-processing was designed ad-hoc. The system is able to measure the velocity and geometry of the sliding mass at impact as well as the characteristics of the generated wave in the impact and propagation zone with an adequate spatial and temporal resolution. The measurement technique is improved to seek accuracy in case of pronounced lens distortions and non-parallelism condition between camera sensor and plane of measurement. An overview of the measuring technique and its quality assessment through error analysis is presented.

2 Measuring system

The images of the experiments are captured through an array of different video cameras. The fast landslide is recorded by a BASLER A504k high-speed (HS) industrial camera, able to shoot up to 1000 frames per second (FPS) and focused upon the exit of the landslide generator (Figs. 2, 3, 4). The HS camera has a fast monochrome digital CMOS sensor having resolution of 1280 × 1024 pixels and pixel size of 12 × 12 μm. The 8 GB dedicated RAM ensures the quick storage of up to 8 s of video clip at full resolution. This time is widely sufficient to record the entire landslide evolution. The HS camera mounts the Nikon AF Nikkor 50 mm f/1.4D lens. Besides its narrow focal length, the lens was selected thanks to its wide aperture: The high-speed camera needs a large amount of light in order to capture readable images at high frequency and short exposure time. The camera is managed with XCAP™, which is the ready-to-run image analysis software for PIXCI® frame grabbers. The exposure time is crucial when recording the granular material in our application: It should be long enough to capture sufficient light and short enough to obtain crisp, blur-free images. Increasing the lens aperture could help capture more light and thus decrease the frame period. However, an excessively large aperture produces an extremely shallow depth-of-field, resulting in a strong background blur, which limits clear observation only to the image area where the lens is focused. The full image frame can be cropped in the area of interest (AOI). AOI is carefully chosen to ensure the area being recorded is sufficiently large to include the entire process and, at the same time, small enough to limit the image dimensions so as to minimize memory space requirements. Similarly, the frame rate must be chosen so as properly to describe the granular flow while limiting the amount of memory space required. Based on the analysis of the issues mentioned above, the camera settings have been chosen as follows: AOI of 800 × 800 pixels; lens aperture of f/1.4; exposition time of 2 ms; frame rate of 500 FPS. The camera records the evolution of the granular material on a centerline drawn by a collimated 1 W green (wavelength 532 nm) laser sheet of 1 cm width and 4 m length. The choice of white gravel is due to its high reflectivity of light.

Fig. 4
figure 4

Wave tank aerial view with laser sheets position and point of view of the three HD cameras focusing on wave tank, as well as of HS camera recording the granular material. Hatched areas represent the angles of view of HD1 camera and HS camera. HD2 and HD3 cameras are not considered in this document

Full size image

The green laser sheet previous described and six collimated red laser sheets of 0.25 cm width and 3 m length (wavelength 650 nm, power 5 mW) project lines on the water surface of the wave basin. The green laser marks the water surface along the landslide axial direction, while the red lasers mark transversal sections of the wave tank, orthogonally to the green laser (Fig. 4). The water has been previously loaded with a small amount of kaolin (5‰ in volume) that colors the fluid in white, reflecting the lasers at water surface. The low kaolin content does not alter the viscosity of the water. Three high definition (HD) video cameras (named HD1, HD2 and HD3) focus on the water tank, recording from different points of view the water displacement along the laser sheets (Figs. 2 and 4). The angles of view of the HS and HD cameras overlap at the landslide entrance: This is necessary to connect the observations of HS and HD cameras with the instant of impact (Fig. 4). The HD cameras are consumer Canon EOS 550D reflex cameras able to record progressive videos at full HD. In general, optics for reflex cameras have less distortions than those for consumer video cameras. The manual settings of the reflex cameras also are an asset comparing against video cameras. The Canon EOS 550D has a RGB CMOS sensor with a pixel size of 4.3 × 4.3 μm. The lenses mounted are Canon AF 18–55 mm zoom. The following camera settings have been selected: resolution of 1280 × 720 pixels; 400 ISO; lens aperture of f/3.5; exposure time of 1/160 s; frame rate of 50 FPS; zoom at 18 mm (wide angle). The red lasers have been used to observe wave lateral expansion and wave reflection by lateral tank walls. For simplification purposes, because red lasers are handled similarly to the green laser, this work describes and analyses only the measurements that have been made along the green laser. Accordingly to that, only the HS camera and the HD1 camera have been taken into account.

3 Calibration techniques

The recorded video frames need a proper calibration to be translated into the spatial references of the measurements. Camera calibration in the context of computer vision is a technique extensively used in robotics and automation. The measurement framework of this study employs the camera calibration technique introduced by the masterpiece work of Tsai (1987), already applied in fluvial-morphodynamics experiments by Bateman et al. (2006) and Bregoli (2008). This technique allows defining the parameters of transformation between the spatial references of the camera (images) and the spatial references of the measurement (global), through the optimization of a number of parameters. The parameters are divided in two subsets: intrinsic parameters (known camera internal geometric and optical characteristics such as the focal and the CMOS sensor characteristics); extrinsic parameters (3D position and orientation of the camera frame relative to a certain global system of coordinates). The optimization can be linear or nonlinear. Linear optimizations require less effort to be solved and are more stable than nonlinear ones. But, on the contrary of nonlinear optimization, linear optimization ignores lens distortions. In our experimental application, lens distortion is crucial and unknown because of the use of an inherently distorted commercial lens. Thus, the nonlinear pinhole method of Tsai (1987), including the optimization of lens distortion, is used. The measurements are performed on the laser line drawing the intersection between the laser plane and the surface to be measured. Thus the calibration of images is exactly applied to the plane of the laser sheet where a calibration panel must be aligned. The calibration panel is formed by a grid of equally spaced black dots (coplanar calibration points) over a white board (Fig. 5a and Supplementary Material).

As a consequence of the setup of the present experimental application, the shooting angle of the cameras with respect to the plane of measurement is necessarily high (Fig. 5a and Supplementary Material). This introduces a massive perspective distortion resulting in non-parallel rows and columns of points in the calibration image. The distortion is so high that it is extremely complicated to automatically detect the main coordinate directions and therefore identify and organize the calibration points of the panel. Thus an algorithm to label and organize the point in the N–E and E–W directions has been implemented as follow:

  1. 1.

    Recognize the 4 corners of the calibration panel

  2. 2.

    Define the mean N–S, E–W axes of the calibration panel

  3. 3.

    Define the local N–S, E–W axes for each point of the calibration panel, starting from the center point (where the axes have a direction similar to that of point 2) and spreading through the adjacent points, with a “flooding” algorithm.

This calibration technique has been implemented in an algorithm that allows optimizing the parameters of transformation between camera spatial reference and global spatial reference. An example of points detection and labeling of the calibration panel used for HS camera is given in Fig. 5a, while an example of validation of the same calibration panel is given in Fig. 5b. More results on points detection and calibration validation for HS camera and HD1 camera are provided in Figs. 1 to 8 of the Supplementary Material. The calibration must be repeated if any change on camera setting or camera displacement is detected. Systematic errors of measurement associated to accidental movement of the cameras after calibration are analyzed in Sect. 5.

Fig. 5
figure 5

Calibration of HS camera at landslide entrance: a image of calibration panel with recognized and labeled points after running the calibration algorithm; b application of calibration results to the same calibration points of panel in a, where spacing between points is 20 mm both in x and y directions. Axes reference is arbitrary

Full size image

4 Measurements

The video frames are systematically pre-processed to be suitable for our measurement method. Each frame is extracted from the video and converted from RGB (24 bit) to grayscale (8 bit) by selecting the most relevant channel or the better combination of the 3 channels. The choice of green and red laser is made in order to be easily identified within the same RGB images. To limit the analysis on the projected laser lines on gravel and water surface, a threshold on pixel intensity is introduced so as to obtain binary images (1 bit). The threshold is chosen depending on the intensity of environmental and laser light. To clean and define the zones of interest, basic morphological operations are applied on the binary images, e.g.: removal of isolated pixels, dilation and majority. The resulting binary images define exactly the masks for the zones of measurement. The masks must be applied on the grayscale images that are the input for the measurement methodology. It is recommended to take de-interlaced videos to avoid artifacts on fast moving objects recording (Erikson and Hanson 2005). An algorithm implemented ad-hoc is applied to perform the measurements. The algorithm uses the intrinsic parameters of the cameras and the output of camera calibration to measure the illuminated zones in the masked images. The center of the laser line is meant to be where the intensity of light is the highest. Thus the pixels intensity is used to determine the centroid of the line to be measured.

4.1 Granular flow measurements

Once the landslide is released and impacts the water surface, the sliding granular material is filmed with the HS camera at 500 FPS. The frame’s high frequency is important in capturing the mass moving at such high velocity. Being the initial length of the sliding mass 1.00 m, its velocity in the order of 5 m/s and the HS camera frame rate Δt = 1/500 = 0.002 s, the camera captures around 100 frames of the sliding mass at impact. A hundred frames is adequate to measure the temporal evolution of pertinent landslide characteristics. For this study, the main landslide parameters to be measured are: velocity, thickness and length. The landslide consists of a granular mass composed by white gravel having mean grain size of 16.9 mm, bulk density of 1692 kg/m3 and basal friction angle between gravel and slope of 30°. The landslide deforms during its release along the connecting wedge and on the wave basin. Inside an observation Eulerian control volume, the evolution of the landslide velocity vs(t) and thickness hs(t) in time are measured on the external mass surface which is illuminated by the laser sheet. Within the same control volume, an averaged velocity vs and thickness hs related to the centroid of the mass at impact are measured. The length of the landslide ls, similarly to vs and hs, is measured as an average value.

In computer vision, blob detection methods are aimed at detecting regions in a digital image that differ in properties, i.e., brightness, compared to surrounding regions (Rosenfeld 1981; Wang et al. 2008). The measurement algorithm recognizes and enumerates each illuminated grain as blobs within each frame (Fig. 6a). Using particle image tracking technique, each grain blob is followed in sub-sequential frames (Fig. 6b). The grain velocity is calculated taking into account the blob centroid displacement in the time step Δt corresponding to the given HS camera frame rate. To obtain a unique average value of velocity vs(t) for each time step, the mean velocity of the different grains traveling inside the control volume placed at intersection between wedge and initial water level is calculated. An example of measured vs(t) and hs(t) is presented in Fig. 7. The temporal evolution of landslide in Fig. 7 shows the landslide—bottom—water body interaction. The height hs initially increases due to the spread at the front (just after the box opening), and subsequently it rapidly decreases because of the shear stress due to the basal friction between landslide and wedge bottom. Similar behavior is reflected in the decreasing velocity. This results in mass stretching and therefore increasing of landslide length. Similar behavior was observed experimentally by, e.g., Fritz et al. (2003) and numerically by Pudasaini (2014) and Kafle et al. (2019). In researching hs, vs and ls, a Lagrangian average on the entire granular mass passing through the control volume is calculated, considering total mass M and momentum p per width unit and density unit as follow:

$$M = \sum {\left( {\underline{h}_{s,i} \cdot \underline{v}_{s,i} } \right)\Delta t}$$
(1)
$$p = M \cdot \underline{v}_{s,i} = \sum {\left( {\underline{h}_{s,i} \cdot \underline{v}_{s,i} } \right)\Delta t} \cdot \underline{v}_{s,i}$$
(2)
$$h_{s} = \frac{M}{{\sum {\underline{v}_{s,i} \Delta t} }} = \frac{{\sum {\left( {\underline{h}_{s,i} \cdot \underline{v}_{s,i} } \right)\Delta t} }}{{\sum {\underline{v}_{s,i} \Delta t} }}$$
(3)
$$v_{s} = \frac{p}{M} = \frac{{\sum {\left( {\underline{h}_{s,i} \cdot \underline{v}_{s,i} } \right)\Delta t} \cdot \underline{v}_{s,i} }}{{\sum {\left( {\underline{h}_{s,i} \cdot \underline{v}_{s,i} } \right)\Delta t} }}$$
(4)
$$l_{s} \approx \Delta t\sum\limits_{i} {\underline{v}_{s,i} \left( t \right)}$$
(5)
Fig. 6
figure 6

Identification of blobs in the landslide front at 0.012 s prior to impact with water (a) and magnified image of combination of two sequential frames separated by a time step of 0.002 s (b) where red dots are the centroid of grains and yellow arrows are the displacement vectors

Full size image
Fig. 7
figure 7

Time evolution of landslide thickness (solid line) and velocity (dashed line) for a selected experimental run

Full size image

As an example, the average values for the run in Fig. 7 are hs = 0.25 m, vs = 5.18 m/s and ls = 1.50 m.

4.2 Wave surface measurements

The water surface elevation is measured along the direction r for each time step fixed by the HD1 camera frame rate as Δt = 1/50 = 0.02 s. The first frame when the contact of landslide with water is detected corresponds to the frame of initial time t = 0. In Fig. 9, it is possible to observe the vertical water surface displacement η measured from the frame reported in Fig. 8. The laser line thickness is 1 cm. As explained in Chapter 4, the measuring algorithm assigns the center line of the laser thanks to a weighted average along the laser line thickness. Because the center of the laser line has a higher light intensity, the weight average is based on pixel intensity. In Fig. 9, one observes that the measurement starts at a distance from the impact point r = 1 m: The area of measurement was masked close to impact because of splash interferences. Indeed, although masking and morphological operations are applied on pictures, the signal of the water surface profile is still noisy due to splash close to impact, splash drops falling along the laser lines and landslide grains surfing on the water surface. Each of these effects can create peaks or discontinuities along the wave profile. Undesired peaks and discontinuities have been mitigated appropriately filtering by a low-pass digital Butterworth filter having filter order O = 2 and normalized cutoff frequency wc = 0.01. Close to wave formation is where the filter is mostly engaged to eliminate the splash and drops inducing a maximum mismatch between measured and filtered series of about 0.9%. After t = 0.5 s, the filtered series deviates less than 0.25%. In any case, the average relative difference between measured and filtered series is always less than 1% (see Fig. 9 in Supplementary Material), and it is considered an acceptable maximum mismatch in designing the filter. Series of wave can be conveniently drawn in space and time as in Fig. 10. This type of graphic is extremely useful to evaluate crests and troughs celerity as well as the presence of wave reflections, being the latter a common feature in confined water bodies impacted by landslides (Kafle et al. 2019).

Fig. 8
figure 8

A frame extracted from a video recorded by the HD1 camera. The green laser highlights the vertical water surface displacement at t = 0.9 s after the landslide impact. r is the distance from landslide impact. The picture has been horizontally flipped in order to match to the representation in Fig. 9

Full size image
Fig. 9
figure 9

Result of measurement of the vertical water surface displacement η after image treatment and signal filtering for the video frame in Fig. 8. r is the distance from landslide impact. The dash-dot line represents the still water level

Full size image
Fig. 10
figure 10

Water surface profiles series in space and time along the green laser line: interpolated representation of 250 time steps. The dashed line represents the water surface profile at the same time step of Figs. 8 and 9

Full size image

5 Errors and uncertainty

Two sources of errors exist: (1) errors derived from the calibration method; (2) systematic errors arisen from the measurement technique. The first source of errors gives the global degree of accuracy and precision of the measurement tools, while the second source gives the degree of uncertainty. The measurement methodology is tested to analyze the possible errors and uncertainty. The maximum measurement resolution is set by the calibration and measurement framework at 10−5 m. Accuracy and precision can vary depending on image calibration and camera model. The found lower accuracy and precision for this application are specified in the calibration algorithm output and are, respectively, 0.90 and 0.53 mm. The tool was originally designed to work with a higher image resolution than the one required by the present application. Thus the resolution is downscaled to 1 mm.

The high quality of the calibration method contrasts with the higher possible uncertainty due to systematic errors. Therefore, an analysis of the sensitivity of the measuring methodology in response to possible systematic errors is made. The systematic errors of measurement that can occur in our laboratory concern the possible rotations and translations of the measuring system, meaning: (1) the accidental movement of the cameras after calibration; (2) the misaligned position of the calibration panel along the laser plane. To avoid the first error source, the calibration is performed for each camera prior to each experiment and thus the associated errors are neglected. Instead, the second error source is investigated by forcing rotations and translations of the calibration panel. The analysis is performed for the case of the HS camera, which is meant to be the most sensitive camera in our pool, and successively extrapolated to the HD1 camera. Systematic errors due to translations along z and rotations in x and y of the calibrating panel for the HS camera (Fig. 11) are independently investigated by measuring a known object (Fig. 12). Positive rotations are in clockwise direction. Different positions of the calibration panel are calibrated. Then, the calibration outputs are used to measure the known object. The results of this series of tests produce curves of sensitivity for each family of induced errors. The sensitivity curves are provided in Figs. 10 to 15 of the Supplementary Material.

Fig. 11
figure 11

Evaluation of uncertainty associated to systematic error of calibration of the HS camera: calibration panel aligned with the laser plane, and reference of the induced systematic movements

Full size image
Fig. 12
figure 12

Picture of the test object and its dimensions along the laser line. The camera angle of view is the same of Fig. 11

Full size image

The green laser sheet has a width (or projected line thickness on the measuring surface) of 1 cm. Thus, the maximum estimated error of panel positioning in z is of the order of \(z_{{{\text{tra,max}}}} = \pm 5{\text{mm}}\). On this basis, the maximum estimated error of panel x and y rotation for the HS camera calibration panel having dimensions of 150 × 225 mm are given by \(x_{{{\text{rot,max}}}} = \pm \tan^{ - 1} (5/150) = \pm 1.91\deg\) and \(y_{{{\text{rot,max}}}} = \pm \tan^{ - 1} (5/225) = \pm 1.27\deg\). Using the same concept, the maximum estimated errors of panel positioning and rotation for the HD1 camera calibration panel (ISO A0 sheet dimensions) can be deduced. The errors are collated in Table 1 where the relative maximum uncertainties per axis are calculated using the linear fits of the graphics reported in Figs. 10 to 15 of the Supplementary Material. The calculated uncertainties include the accuracy and precision of the measurement tool. The systematic error propagates through the empirical formulas defined after the experimental analyses. In Bregoli et al. (2017), a sensitivity analysis on the proposed formulas has been performed including artificially the worst maximum systematic errors along z.

Table 1 Maximum systematic errors and associated uncertainty in measurements
Full size table

6 Discussion and conclusions

A system able to measure the main parameters of experiments of tsunamis generated by landslides have been presented. The measurement tool is based on computer vision and employs a versatile camera calibration technique based on the Tsai’s pinhole method (Tsai 1987) accounting for lens distortion. The method is improved to account for perspective high distortions. This last feature has been requested by the pronounced angle of view of the cameras been constrained by the experimental setup features. The system permitted to measure the rapid granular landslide form and velocity as well as the water surface displacement. Particularly, the methodology allowed measuring the wave displacement at high resolution in space and high frequency in time, avoiding the classical point measurements given by probes and depicting wave series measurement which is not space dependent. For the specific experiments of tsunamis generated by landslides, the space-independent measurement is particularly beneficial because the point of maximum wave amplitude condition, corresponding with the beginning of wave propagation, forms at a certain distance from impact and it is strongly dependent on the landslide characteristics, thus varying for each run initial conditions (Bregoli et al. 2017). This measurement technique addresses the observation of multi-phase mass-flow complex phenomena at high details and versatility. Therefore, it can be employed in experiments to test multi-phase numerical models that are recently receiving a remarkable scientific focus (Pitman and Le 2005; Abadie et al. 2010; Cecioni and Bellotti 2010; Montagna et al. 2011; Pudasaini 2012, 2014; Pudasaini and Mergili 2019; Kim et al. 2019).

The measurement technique has been tested and in order to correctly demonstrate its reliability, the associated errors and uncertainties have been evaluated. The maximum resolution of the measurement tool is 0.01 mm, and the accuracy and precision are estimated, respectively, in 0.90 mm and 0.53 mm. The maximum uncertainty due to systematic error is estimated in 15% for the worst-case scenario. The resolution is downscaled for this application to 1 mm. The calibration framework defined for the specified experiments is recommended to be repeated periodically to avoid further systematic errors due to cameras settings changes or cameras accidental displacement. The particular experimental setup features can somehow complicate the calibration repetition. As an example, the presence of water hinders the calibrating operations inside the wave tank: The calibration needs to be done when the wave tank is empty. However, in adequate condition of control it has been demonstrated that the presented measuring technique is certainly suitable in measuring rapid mass movements and water surface displacement in space and time. Due to its versatility and relatively low cost, the measurement methodology can be easily used in several experimental applications, e.g., in hydraulics, fluid mechanics and granular flows. The technique is particularly suitable in those experiments which require high resolution, precision and accuracy and no disturbance of the physical processes by the measuring system itself.