Keywords

1 Introduction

Slope safety monitoring typically involves various methods such as manual on-site monitoring, GNSS (Global Navigation Satellite System), inclinometers, total station robots, static leveling instruments, 3D laser scanners, synthetic aperture radar, etc. These monitoring methods have their own disadvantages. For example, manual on-site monitoring is labor-intensive and infrequent, GNSS, inclinometers, total station robots, and static leveling instruments provide point measurements and are challenging to maintain, while 3D laser scanners and synthetic aperture radar require substantial financial investment [1,2,3,4].

Since the 20th century, the rapid development of video image processing technology has greatly transformed measurement techniques and is now widely applied in areas such as national defense, aerospace, robot vision, biomedical engineering, industrial product inspection [5]. Image processing monitoring technology uses the information reflected by targets through light, combined with image processing and recognition techniques, to obtain parameters of the target. This process does not require physical contact with the slope and allows for long-distance measurements.

Researchers like Billie F. Spencer Jr. at the University of Illinois have applied video technology to infrastructure inspection and monitoring. Tung Khuc at the University of Central Florida has researched video-based non-point infrastructure displacement monitoring. Maria Q. Feng at Columbia University has studied video-based multi-point displacement measurement for structural health monitoring. In China, researchers have early applications of close-range measurement technology for displacement fields, existing tunnel lining deformations, and new tunnel contour under-excavation measurements [6]. Sun has studied the application of close-range photography technology in slope deformation monitoring in open-pit mining areas [7]. Zhou at Huazhong University of Science and Technology has conducted dynamic displacement monitoring using video images for high-rise buildings and bridges [8]. Gao et al. has used an image monitoring system for engineering monitoring of bridge structures [9]. Researchers have applied photogrammetry technology to terrain measurement, building measurement, foundation pit measurement, and landslide monitoring, achieving an accuracy of 1:10,000 [10]. It is evident that measurement technology based on video images is widely used in various industries.

Video-based slope safety monitoring calculates the displacement information of the object under test by analyzing images before and after changes in the object. Through research on equipment selection for image acquisition, supplementary lighting under low-visibility conditions, sub-pixel edge detection technology, video interference removal, monitoring software development, and integration of field power supply systems, a high-precision slope displacement monitoring system based on real-time video images has been developed. This system enables 24-h real-time online non-contact monitoring of slopes at a long distance.

2 Video Image Monitoring Technology Principle

2.1 Sub-pixel Edge Detection and Positioning Principle

The method of calculating the most suitable position for the target feature from the image is called image target sub-pixel positioning technology. For example, in an ideal situation, a square digital image with a size of 4x4 after digitization has a center coordinate of (1.5, 1.5), as shown in Fig. 1. If the integer pixel value is taken as the center coordinate of the digital image, it will result in a positioning error of 0.5 pixel value. If the average value of the pixel coordinates of the target image is calculated as the center coordinate of the target image, the positioning will be more accurate.

Fig. 1.
figure 1

4 × 4 square target

Full size image

2.2 Principle of Slope Image Displacement Monitoring

To achieve slope image displacement monitoring, the first step is to complete object image measurement, establish an imaging geometric model, and determine the model parameters. This is a crucial step in realizing image detection and positioning, which involves camera modeling and camera calibration. The purpose is to infer three-dimensional information from the image by obtaining the cameras intrinsic and extrinsic parameters, or to calculate the coordinates of the target in the image from the three-dimensional information.

Camera Imaging Model.

To establish the geometric relationship between spatial points and their corresponding image points, we use (u, v) to represent the coordinates in the pixel-based image coordinate system, and (x, y) to represent the coordinates in the physical unit-based imaging plane coordinate system. If the coordinates of O1 in the Ouv coordinate system are (u0, v0), and the physical dimensions of each pixel in the x and y directions are dx and dy respectively, then the two coordinate systems have the following relationship.

$$ \left[ {\begin{array}{*{20}c} \mu \\ \nu \\ 1 \\ \end{array} } \right] = \left[ {\begin{array}{*{20}c} {1/dx} & 0 & {\mu_{0} } \\ 0 & {1/dy} & {\nu_{0} } \\ 0 & 0 & 1 \\ \end{array} } \right]\left[ {\begin{array}{*{20}c} x \\ y \\ 1 \\ \end{array} } \right] $$
(1)

Camera Calibration.

Camera calibration essentially involves calculating the correspondence between image coordinates and reference coordinates. In the image coordinate system, the imaging plane coordinate system, the camera coordinate system, and the world coordinate system, the first two coordinate systems are two-dimensional, and the image coordinate system is known. The last two coordinate systems are three-dimensional. If the size of the object is known and the world coordinate system is selected with the object as the reference, then the world coordinate system is also known. In most cases, determining the relationship between the world coordinate system and the image coordinate system is referred to as camera calibration.

3 Image Displacement Monitoring of Slopes

3.1 Image Displacement Monitoring of Slopes

In order to accurately measure the displacement of a specific point on a slope, it is necessary to select measuring points within the monitored slope area and stable points outside the area. The measurement camera should be installed in a stable area. To achieve good monitoring results and minimize interference caused by external disturbances, fixed monitoring markers (such as highly reflective infrared reflective markers) are selected and installed at critical positions with high risks on the slope. Then, measurement stations are installed in the stable section of the slope, away from the monitored displacement area.

3.2 Measurement and Calculation of Displacement Images at Slope Monitoring Points

Assuming that the position of the monitoring camera is located in the stable area and remains stationary with the reference point. It is assumed that the position of the monitoring camera and the reference point are located in the stable area and remain stationary.

Based on the principle of light propagation in a straight line, if it is possible to image both the displacement target point and the fixed reference point in the same camera, the actual variation of the displacement point can be measured by observing the relative change (δL = L1 − L2) of the displacement target point with respect to the reference point in the image. The schematic diagram of the imaging effect is shown in Fig. 2.

Fig. 2.
figure 2

Image representation of displacement changes at monitoring points

Full size image

4 High-Precision Real-Time Image Measurement System for Slope Displacement Research and Development

4.1 Overall Design of High-Precision Real-Time Image Monitoring System for Slope Displacement

The overall design of the high-precision real-time image monitoring system for slope displacement is shown in Fig. 3. The entire system consists of an equipment terminal, a data management platform, and accessories.

Fig. 3.
figure 3

System Overall Design Proposal

Full size image

4.2 Research and Development of High-Precision Real-Time Image Monitoring Equipment for Slope Displacement

To meet the requirements of complex monitoring environmental conditions, the project fully considers all-weather real-time monitoring, outdoor power supply, and communication conditions in the system design. The system consists of an image pickup device, low-light compensation device for dark environments, dust and rain protection device, outdoor power supply system, edge computing terminal, and communication transmission device.

The displacement image perception device adopts an integrated design of image acquisition, supplementary lighting, dust-proof, rain-proof, and moisture-proof functions. The displacement image monitoring software is developed using QT software programming to build the system’s user interface.

5 Experimental Testing of Slope Displacement Image Monitoring

To verify that the algorithm can achieve micro-displacement monitoring, the project conducted micro-displacement monitoring tests in an underground garage and an outdoor environment. These tests were conducted to evaluate the monitoring performance in both dark and bright conditions during the night and day, respectively.

5.1 Night Testing

Night testing was conducted in an underground parking garage at a distance of 70 m with a field of view of 10 m. Laser infrared illumination was used to provide supplementary lighting to the reference points. The purpose of the testing was to evaluate whether the hardware solution and software algorithms could accurately identify and monitor small displacements of the targets in low-light conditions.

The X-axis movement of the reference points is 1 mm, 2 mm, 3 mm, 4 mm, and 5mm respectively. A monitoring is conducted for every movement. The ellipse fitting is performed on the above image to calculate the coordinates of the center of the reflective markers at different positions during the movement. The experimental results for a monitoring distance of 70 m are shown in Fig. 4. In the figure, the horizontal axis represents the number of experiments, and the vertical axis represents the pixel values.

At a distance of 70 m, the experimental results are shown in Fig. 4.

Fig. 4.
figure 4

Monitoring Results of a 70 m Distance from The Center of The Circle Displacement

Full size image

As the retroreflective marker moves, the corresponding circle center coordinates also shift, and the captured images show evenly distributed spots from the retroreflective marker. By using sub-pixel algorithm detection and recognition, the tiny horizontal movements of the spot’s center point can be accurately monitored, with a measured static accuracy of less than 1 mm.

5.2 Outdoor Glare Testing

Under strong sunlight conditions, the field of vision on the highway does not require supplementary lighting. Reflective markers were placed at a distance of 70m, and the camera focal length was set to achieve a field of view size of 10 m. The reflective markers were horizontally moved on the X-axis by 1 mm, 2 mm, 3 mm, 4 mm, and 5 mm, and monitoring was conducted 5 times for each movement. The experimental results of the 70 m monitoring distance are shown in Fig. 5.

Fig. 5.
figure 5

Monitoring results of a 70m distance from the center of the circle displacement

Full size image

From the above image, it can be seen that at a monitoring distance of 70m, significant horizontal displacement changes can be detected, while the vertical displacement remains relatively unchanged. The measured static accuracy is better than 1mm. Therefore, the hardware design and software compatibility of this system can effectively achieve real-time monitoring of slope displacement.

5.3 Field Experiment

In April 2022, the research team conducted an on-site industrial experiment of a high-precision real-time image monitoring system for slope displacement at the Panguilu Station slope of the west extension section of Chongqing Rail Transit Line 4. The experiment involved one set of high-precision real-time image monitoring system for slope displacement, one set of total station, and several reflective markers.

After one month of video image monitoring and manual monitoring with a total station, the data comparison analysis is as follows.

  1. (1)

    The lateral displacement range of monitoring point in the video monitoring is between 1.7 mm and 1.1 mm, while the lateral displacement range in the total station monitoring is between 1.4 mm and 1.0 mm. The monitoring comparison curve is shown in Fig. 6(a).

  2. (2)

    The longitudinal displacement range of monitoring point in the video monitoring is between 9.4 mm and 8.6 mm, while the longitudinal displacement range in the total station monitoring is between 9.0 mm and 8.6 mm. The monitoring comparison curve is shown in Fig. 6(b).

Fig. 6.
figure 6

Comparison curve of displacement at monitoring point

Full size image

The on-site experimental results show that the high-precision real-time image monitoring system for slope displacement can achieve 24-h real-time online monitoring of slope displacement in engineering slopes. The comparison with high-precision Leica total station monitoring data shows consistent deformation values and trends. The monitoring accuracy is 1 mm within a measurement distance of 100 m, demonstrating its excellent value for promotion.

6 Summary

Through research on equipment selection for image acquisition, fill light under low vision conditions, subpixel edge detection technology, video interference removal, monitoring software development, integration of field power supply systems, etc., a set of high-precision real-time image monitoring technology equipment based on subpixel edge detection for slope displacement monitoring is formed.

The high-precision real-time image monitoring system for slope displacement can effectively achieve long-distance non-contact 24-h real-time synchronous online monitoring of slopes, with a monitoring accuracy of millimeters. Through the “one machine, multiple uses” function, it can effectively capture and store on-site images triggered by abnormal slope displacement. The system has the characteristics of simple installation layout, high cost-effectiveness, strong applicability, and functional suitability.

The video image displacement deformation monitoring technology is novel, but there are still many aspects worth further research in practical applications. For example, achieving monitoring distances of hundreds or even thousands of meters, expanding the monitoring field of view, improving the accuracy of non-target point monitoring, and identifying radial changes are all key research directions for future development.