Abstract
The combination of a geodetic total station with a digital camera opens up the possibilities of digital image analysis of the captured images together with angle measurement. In general, such a combination is called imageassisted total station (IATS). The prototype of an IATS called MoDiTa (Modular Digital Imaging Total Station) developed at i3mainz is designed in such a way that an existing total station or a tachymeter can be extended by an industrial camera in a few simple steps. The ad hoc conversion of the measuring system opens up further areas of application for existing commercial measuring systems, such as highfrequency aiming, autocollimation tasks or tracking of moving targets. MoDiTa is calibrated directly on site using imageprocessing and adjustment methods. The crosshair plane is captured for each image and provides identical points in the camera image as well as in the reference image. However, since the camera is not precisely coaxially mounted and movement of the camera cannot be ruled out, the camera is continuously observed during the entire measurement. Various imageprocessing algorithms determine the crosshairs in the image and compare the results to detect movement. In the following, we explain the selfcalibration and the methods of crosshair detection as well as the necessary matching. We use exemplary results to show to what extent the parameters of selfcalibration remain valid even if the distance and thus the focus between instrument and target object changes. Through this, one calibration is applicable for different distances and eliminates the need for repeated, timeconsuming calibrations during typical applications.
Zusammenfassung
Selbstkalibrierung und Strichkreuzverfolgung mittels einer modularen digitalen bildgebenden Totalstation. Die Kombination einer geodätischen Totalstation mit einer digitalen Kamera eröffnet die Möglichkeit der digitalen Bildanalyse der aufgenommenen Bilder zusammen mit der Winkelmessung. Im Allgemeinen wird eine solche Kombination als Image Assisted Total Station (IATS) bezeichnet. Der am i3mainz entwickelte Prototyp einer IATS namens MoDiTa (Modular Digital Imaging Total Station) ist so konzipiert, dass eine bestehende Totalstation oder ein Tachymeter mit nur wenigen Handgriffen um eine Industriekamera erweitert werden kann. Die AdhocErweiterung des Messsystems eröffnet weitere Anwendungsbereiche für bestehende kommerzielle Messsysteme wie hochfrequente Zielerfassung, Autokollimationsaufgaben oder die Verfolgung bewegter Ziele. MoDiTa wird direkt vor Ort mittels Bildverarbeitungs und Ausgleichungsmethoden kalibriert. Die Fadenkreuzebene wird für jedes Bild erfasst und liefert identische Punkte sowohl im Kamerabild als auch im Referenzbild. Da die Kamera jedoch nicht exakt koaxial montiert ist und eine Bewegung der Kamera nicht auszuschließen ist, wird die Kamera während der gesamten Messung kontinuierlich beobachtet. Verschiedene Bildverarbeitungsalgorithmen bestimmen während der Messung das Fadenkreuz im Bild und vergleichen diese Ergebnisse, um Bewegungen zu erkennen. Im Folgenden werden die Selbstkalibrierung und die Methoden der Fadenkreuzerkennung sowie der notwendige Abgleich erläutert. Anhand exemplarischer Ergebnisse wird gezeigt, inwieweit die Parameter der Selbstkalibrierung auch dann gültig bleiben, wenn sich der Abstand und damit der Fokus zwischen Instrument und Zielobjekt ändert. Damit ist eine Kalibrierung für unterschiedliche Entfernungen anwendbar und erspart bei typischen Anwendungen die Wiederholung der zeitraubenden Kalibrierungen.
Similar content being viewed by others
1 Introduction
Modular Digital Imaging Total Stations show a wide range of experimental applications in the fields of engineering surveying and metrology (Atorf et al. 2019; Guillaume et al. 2016; Wagner et al. 2014). Recent reviews have been presented by Paar et al. (2021) and Zschiesche (2022). In short, the development of surveying instruments into more powerful and userfriendly tools is taking place through automation and the addition of new sensor technology. This can be seen, for example, in automatic target recognition (ATR) or the autofocus of socalled total stations. By extending a total station with one or more cameras, the possibilities of image processing also become available. In addition, a further advantage is the users’ independence due to the subjective sense of the observer’s eye. However, the cameras used so far by instrument manufacturers primarily serve to improve interactive user workflows, but they currently do not provide a realtime interface for userspecific image analysis or deep learning applications in the measurement process. Furthermore, the highest possible frame rate is significantly lower than the frame rate achievable by industrial cameras. For example, a multistation MS50 from the manufacturer Leica/Hexagon achieves 20 frames per second from a 5 MP coaxial camera and thus allows smooth user interaction. Looking into detail, the frame rate of 20 Hz is only achieved with respect to VGA resolution of the display, 640 × 480 pixels (Grimm and Zogg 2013). Saving a full 2560 × 1920 pixel image to an SD card usually takes more than 2 s with JPEG compression and even more than 6 s in raw format. To be able to target applications that we believe require frame rates of around 1 Hz–1000 Hz, we have decided to continue the concept of external cameras to achieve these frame rates (Hauth et al. 2013), while integrating the motorised focus support of the multistations. We consider it an advantage that the external camera does not disturb the thermal design of the multistation even at high pixel clock rates.
During the process of prototype development, different types of construction emerged. External implementations make it possible to mount the camera on the ocular or replace it. These are used in combination with commercial total stations or tacheometers and can be converted and adapted to the particular conditions and requirements. One example of such a modular system is DAEDALUS of ETH Zurich (Bürki et al. 2010; Charalampous et al. 2014; Guillaume et al. 2012, 2016). In this concept, a CCD chip replaces the eyepiece. The camera does not capture the crosshairs in the image. No additional optical component is added in between. This makes it necessary to attach a meniscus lens to the front of the telescope for distances of 13 m or more. This front lens shifts the focal plane to the image sensor for focused images. Similar to Huang and Harley (1989), the calibration is carried out by virtual control points, where the central projection is expressed by an affine approach. Instrument errors are not taken into account.
The developed application of the University of Zagreb attached a GoPro5 directly to the ocular (Paar et al. 2017, 2021). Another setup where the camera is directly attached to the eyepiece can also be found in Schlüter et al. (2009). For the measurement with the IATS at the University of Zagreb, videos are recorded which are later split into images. Here, photo targets with predefined circles of known diameter and distance between the circle centres are used. This enables the evaluation of the image data. For the frequency analysis, raw image coordinates are used and no camera calibration is required. However, the photo target must be attached to the object to be observed.
The second design offers the advantage of a fixed camera with the instrument like commercial IATS. Commercial IATS often have too low speed of image acquisition for kinematic measurements (e.g. for frequency analysis in structural health monitoring). The fixed camera provides constant calibration parameters as opposed to the modular version which requires calibration after reconfiguration. An early prototype is mentioned in Walser (2004), and the prototype series IATS2 from the manufacturer Leica in Reiterer and Wagner (2012), Wagner et al. (2013, 2016), Wasmeier (2009b). Walser (2004) describes the camera with an affine chip model and uses a combined approach to take camera and instrument errors into account. Wasmeier (2009a) shows a comparison of different methods.
The measuring system MoDiTa developed at i3mainz extends an existing instrument modularly by an external industrial camera. The selfcalibration based on the photogrammetric camera model fully integrates the external camera into the measurement process. By permanently tracking the crosshair, the accuracy characteristics of the total station are maintained. In the following, we explain the measurement system and the calibration. The necessary imagebased acquisition of the crosshair for the calibration and the further measurement process will be discussed in the following in more detail. The approach used here shows how a calibration can be calculated flexibly on site using software and various cameras and total stations (compatible to TCA, TPS, TS and MS series from the manufacturer Leica) without any additional equipment.
2 Measurement System
The Modular Digital Imaging Total Station (MoDiTa) combines a highend industrial camera with a digital total station in a modular and flexible way and is currently on prototype level (Fig. 1). As described in Hauth et al. (2013), the standard eyepiece of the total station is replaced by an industrial camera via a bayonet ring. To balance the weight of the camera, we attached a counterweight to the telescope. The cameras can be mounted in any rotation around the target axis by means of a simple clamping screw. By means of a corresponding adapter, the eyepiece camera used takes images directly from the crosshair plane. The crosshair is thus captured in every image. Among other things, this enables automatic, imagebased targeting, which is within the accuracies of the total station (standard deviation according to ISO 171233 2001). With the help of template matching, nonsignalled distinctive features are captured without contact. The use of the total station’s motorised autofocus is advantageous because, among other things, it enables simple selfcalibration. After selfcalibration, we calculate the corresponding horizontal or vertical angle for each point of interest in the image.
Due to the modular design of the measuring system, a camera can be selected depending on the respective project requirements. Project requirements might include:

a monochrome, NIR or RGB (Bayer pattern) sensor,

low light suitability (usually by large pixel pitch) or high resolution,

a global or rolling shutter,

availability of a hardware trigger,

availability of line scan modes,

high frame rate (frames per second).
The industry standard Cmount used makes it easy to replace components. Depending on the industrial camera used, images can be captured in different modes. By selecting an area of interest (AoI), the range of captured lines and columns can be defined. In linewise mode, only one line is captured over the width of the image. The data to be transmitted can thus be reduced, enabling a higher image capture frequency. A more detailed description can be found in Hauth et al. (2013).
3 SelfCalibration
To obtain measurement results within the measurement accuracy of the total station, calibration of the entire system is required. This is done by selfcalibration directly on site. Given the speed of the selfcalibration process, we do not intend to achieve repeatability of the calibration parameters of the camera in different setups. The aim is rather to be able to use the measuring system quickly and in an applicationoriented manner. The determination of interpolable parameters for a particular combination of camera and total station was never attempted.
The user installs or replaces the camera on site and the measurement can be continued after calibration. Due to the simple mounting of the camera and the modular design, it is near impossible to recreate an identical setup. As a result, there are minimal differences in the optical path for each setup. Differences of several pixels in the image are possible.
Calibration is mainly carried out automatically and only needs to be operated manually by the user at the beginning. Before calibration, it is necessary to detect the crosshair to provide a reference image of the crosshair. The crosshair reference image ensures consistency of visual aiming through the eyepiece to camerabased aiming. Furthermore, the reference image of the crosshairs is used to correct any camera movements computationally, cf. Sect. 4. The telescope is moved relative to a fixed target point in such a way that the target point is imaged at favourably distributed locations on the image plane (Schlüter et al. 2009). This allows for the collection of data for an overdetermined linear system of equations. The software provides for different patterns with different distributions of the observation points in the image. The selection of patterns makes it possible to open up new fields of application in an applied, scientific environment by means of an inexpensive measuring system. For example, a comprehensive highprecision calibration with up to 36 measurements can be carried out for an investigation into atmospheric refraction. The implemented maximum number of observation points in the image is set to 9 points per quadrant (4 quadrants × 9 observation points = 36). It is possible to define fewer observation points, thus reducing the over determination. For a simpler example, see Fig. 11a. After the measurement, we have 12 images, each with the target at different positions in the image. In this case, we use a black and white laserscanning target with a checkerboard pattern. For the automatic, rough approach of these target directions, the knowledge of a rough start transformation is sufficient, which only includes the camera constant and the rotation of the camera coordinate system around the target axis of the total station. We merely tilt the telescope to the side by a small, fixed amount to determine the start transformation. During the measurement, the software continuously observes the crosshair, the socalled matching. Due to the simple mounting of the camera, the crosshair is not in the centre of the image. It is also possible to rotate it around the optical axis.
In the context of selfcalibration, we calculate the parameters via parameter estimation based on the least squares method. The functional model is based on the mapping relations between sensor space and object space (Walser 2004).
Optical distortion and a possible tilt of the camera are compensated by the distortion approach according to Luhmann et al. (2020). With c as the camera constant for the entire optics, the unknown angles \(\widetilde{H}\) and \(\widetilde{V}\) for the target. \(\kappa\), c and the photogrammetric radial, tangential and asymmetric distortions (A_{1}, A_{2}, A_{3}, B_{1}, B_{2}, C_{1}, C_{2}) are obtained. We describe the illustration of the camera chip by a 2D transformation using a photogrammetric distortion model. \(\Delta x^{\prime}\) and \(\Delta y^{\prime}\) represent the parameters of the distortion, \(x^{\prime}_{0}\) and \(y^{\prime}_{0}\) represent the principal point, respectively, and the detected crosshair. The angle readings to the fixed target are not measured directly, but result from the pixel coordinates of the reference crosshair. The index P represents the measured values to the target point.
The unit vector to the searched target point is formed from the total station readings and the pixel position of the image point of one measurement:
The corresponding point in object space is
with
The total vector in image space is normalised to unity, which is indicated by the division by \(\left {\left[ \ldots \right]} \right\). The spatial distance D is actually not required for the calibration. D prolongs the unit vector to the object point.
The matrix R_{K} describes the rotation of the camera sensor around the optical axis.
\(R_{{H_{P} }}\) and \(R_{{V_{P} }}\) follow from the graduated circle reading. The required values are supplied by the total station. The matrices describe the necessary rotations to transform the direction vector from the system of the total station into a coordinate system of its ancestries. Thus, when the instrument is previously stationed, the coordinates are converted into the used system directly.
\(R_{{IP_{P} }}\) and \(R_{{IA_{P} }}\) describe the rotation of noncompensator corrected total station readings into compensator corrected ones (I_{P} and I_{A} are calculated from the inclination in the direction of the target axis and transversely to the direction of the target axis). \(\overline{X}_{P}\), \(\overline{Y}_{P}\) and \(\overline{Z}_{P}\) are calculated and denote the normalised direction vector to the target point.
By using (7) and (8), the angles \(\overline{H}_{P}\) and \(\overline{V}_{P}\) can be calculated.
else
The different telescope positions H_{P,} V_{P} result in the corresponding image point \(x^{\prime}_{P}\),\(y^{\prime}_{P}\). From Eq. (2) follows \(\overline{H}_{P}\), \(\overline{V}_{P}\) to the target point. We use this concept for selfcalibration. This means that the target point does not have to be aimed directly, but introduced as an unknown \(\tilde{H}\) and \(\tilde{V}\).
We calculate residuals through a summary modelling of all stochastic influences. Due to practical reasons, the stochastic portions of image coordinates, circle readings and compensator readings are not modelled separately from each other. Atmospheric flicker can be reduced by grouped multiple exposures suitably.
Here, the corrected tachymeter readings to the target point correspond to the direct measurement to the target. We did not differentiate between total station and camerarelated corrections.
\(x^{\prime}_{{{\text{Ch}}}}\) and \(y^{\prime}_{{{\text{Ch}}}}\) represent the pixel position of the crosshair. Distortion and \(\kappa\) have no effect on the principal point.
We consider measurements independently and equally accurately. The weight matrix P = I is defined with the ones on the main diagonal or zeros if the measurement is not to be included in the equation as an error.
The Cholesky factorisation according to Förstner and Wrobel (2016) is used to solve the system of normal equations. The normal equation matrix is split into an upper and lower triangular matrix C and CT. We solve the system of normal equations by subsequent forward and backward substitution. This saves computing time because instead of the entire normal equation matrix N, only one triangular matrix needs to be inverted (Förstner and Wrobel 2016; Luhmann et al. 2020).
From the linear dependent residuals of the unknowns follows the estimation of the unknowns as
The unknowns and the termination criterion are calculated per iteration. Termination occurs after the limit has been reached.
As a result, the compensated direction angles to the target are provided. A transformation into Cartesian coordinates can be done afterwards by a distance measurement, if required. For this purpose, the determined target point is directly entered by the total station and a reflectorless measurement is carried out. The measured distance is used to extend the unit vector to the target point.
4 Crosshair Tracking
Based on the modular adapter for mounting a camera, it is possible to capture the crosshair. The crosshair is a geodetic crosshair that is not located in the exact centre of the image. We distinguish between detecting and matching. Detection of the reference crosshair should take place as soon as possible after the camera is mounted. As with manual eyepiece adjustment, a monotone image background is preferred for this step, e.g. a sky or grossly outoffocus image, so as to get an even background. The position and orientation of the crosshair in the pixel coordinate system of the camera is determined with the help of further crosses, which we refer to as (virtual) réseau crosses in the following. The software continuously observes the position of the crosshair during further measurement. Any image coordinate is transformed to the reference crosshair using 2D transformation including two translations and one rotation. Smaller deviations are recognised and taken into account by matching. If the change is too large, a new detection is necessary. Figure 9 provides a simplified overview of the single steps. In the following, we distinguish between the inner and the outer crosshair. The two inner lines mark the centre of the crosshair. The outer crosshair is composed of the six outer lines of the geodetic crosshair (Fig. 2). The elaborate modelling of the reference crosshair makes it possible to ensure a largely continuous tracking later on, even if line elements are only recognisable in parts.
4.1 Crosshair Detection
During crosshair detection, first the outer crosshair with its six lines will be roughly determined. According to Steger (1996a, b, 1998), a Gaussian smoothing filter in combination with its partial directional derivatives is applied. According to Canny (1983), the smoothing and the threshold values are determined. If the value of the second partial derivative of an image point exceeds the upper threshold in a pixel, this is detected as a line point with subpixel accuracy. If the second partial derivative is smaller than the lower threshold, we dismiss the pixel. If the value is between both thresholds, the pixel is only used if it can be connected by detected line points. As a result, we obtain several line segments. These are then examined for false detections according to Haralick and Shapiro (1992) or Suesse and Voss (1993). By calculating a regression line through the image points of the lines, the mean distance of the individual points to the line can be calculated. We reject points with greater distance than the mean. The regression line is then calculated again. By defining a limit value for the direction difference and the distance of the end points of neighbouring lines, these are merged if necessary (Fig. 3). The regression line is then calculated again. This is repeated until the maximum values for the direction difference and the distance of the line end points are no longer undercut. The longest six lines of the calculations correspond to the outer geodetic crosshair. These are still in the form of polylines. They are calculated individually by adjustment according to Haralick and Shapiro (1992), Suesse and Voss (1993) as a straight line equation. In addition, the line width is determined for later use in the precise determination of the crosshair. Simplified, we calculate the width of the longest line by edge detection. Starting from a point on the line, we search the perpendicular distance on both sides up to the edge. The length of the perpendicular is determined for each pixel on the vector line. The mean value then gives the line width. The contour width can differ depending on the camera and the total stations crosshairs, but it must be at least one pixel wide in order to be detectable.
The results are the start and end points, the straight line equation of the six crosshair lines and the line width.
We use the six compensated lines of the outer crosshair to determine the rough crosshair centre. All intersections of the straight lines are formed. We eliminate negative coordinates. By forming the median of the nine intersections, the rough centre is calculated. The maximum distance from the rough centre to the intersection points and the approach of an isosceles triangle are used to calculate the distance between two parallel crosshair lines.
For the definition of the inner crosshair, we define a circle with the centre equal to the roughly determined centre and the radius equal to the distance between two parallel crosshair lines. According to Luhmann et al. (2020), onedimensional grey value profiles are formed vertically to the circumference.
These are obtained by averaging all existing grey values of a line that lies vertically to the circular ring. Using a Gaussian smoothing filter and a Laplace operator, we are able to calculate the edge positions with subpixel accuracy. We compare the edge amplitudes with a previously defined threshold value. If the amplitude is greater than the threshold value, an edge is present at the corresponding image position. A total of eight points are detected on the inner crosshair, two points per line (Fig. 4a). If more than four lines intersect with the circular ring, only the best four edges are used. The selection is made via the calculated edge amplitude. The greater the amplitude, the higher is the contrast of the contour at the image position. The start and end points of the edges per line are calculated as a mean so that there is one point for each inner line of the crosshair (Fig. 4b). For the orientation of the cross, the direction angle from the rough centre to the point with the largest edge amplitude is used.
For the precise determination of the crosshair, we consider the lines individually and points are determined at equal distances on each line. With the help of the direction angles, we form sectors of circles around the rough crosshairs. Within two defined circles with different radii, the edges per pixel of the lines are detected again according to Luhmann et al. (2020) (Fig. 5a). As a result, the edge beginnings and ends of the inner crosshair are available for each line as coordinates between the two circles. These are averaged again so that the centre points of the line contours are available. Points with the same direction on the inner crosshair are combined. This results in two lines: one horizontal and one vertical. These are equalised according to the same principle of Haralick and Shapiro (1992) and Suesse and Voss (1993) (Fig. 5c). The exact image coordinates of the crosshair centre are now available with subpixel accuracy via the point of intersection.
We calculate réseau crosses for the alignment of the precise crosshair. This is done by calculating two points per line. By defining two circles with different radii (small circle: 3 times the distance of the parallel crosshair lines, large circle: shortest distance from the centre to the edge of the image reduced by 10%). The resulting intersections with the outer cross lines are all within the image area.
The smaller circle is intersected with the precise inner crosshair lines (Fig. 6a). Similar to the procedure already described for the rough determination of the inner crosshair, the edge contours are determined perpendicular to the line from the crosshair centre according to Luhmann et al. (2020) (Fig. 6b). The detection of the contours is again carried out via grey value profiles and the start and end points are then averaged. We repeat the steps for the outer radius so that two points are available for each outer crosshair line. Finally, we sort the detected points separately for the inner and outer circle in the correct quadrant. The 12 points on the outer cross lines are determined precisely and are available in the correct position as a pair of points per line. One hundred further points are then determined between two points of a line according to the procedure shown in Fig. 5. We defined the number for which a large number of points is available for the definition of the straight line and the following adjustment. A different definition is also possible. The one hundred points to be detected are distributed at equal intervals along the length between the start and end point of a line. As in the previous step, the detection for each point of the line contour perpendicular to the line is performed. The points on the opposite crosshair lines are combined so that the final result is a horizontal and a vertical line (Fig. 7). The adjustment calculation is carried out according to the least squares method. We detect and eliminate outliers before the adjustment.
The calculation of the straight lines is based on the procedure according to Kampmann and Renner (2004), method 3. We transferred the model of the adjustment calculation mentioned to the model of the crosshair with two straight lines and adapted to its special features. The double lines of the crosshair result in an additional unknown so that the functional model in coordinate form is as follows:
The variable δ corresponds to the half parallel distance of a double line to the adjusted straight line. These are chosen with different signs for the double lines and should be defined the same for both straight lines. The parameter δ is omitted for the single line. The equation is set up independently for the two lines, so that the parameters must be determined separately for each equation. The eight parameters of the two lines to be determined are listed in the unknown vector \(X\).
The adjustment of both straight lines is done in one calculation. The approximate values for the variables d_{1/2} and δ_{1,2} are determined empirically. b_{1} and a_{1} are given the value zero and are thus defined as parallel straight lines to the image coordinate system. Since the underlying equation of the functional model is linear, the partial derivatives according to the parameters are equal to the observations used. Simplified, the condition equation can be regarded as an observation equation for the software solution, but with a significantly higher weight than the observations. The weight for all observations equals 1, and the conditional equation is given the weight 10^{6} (Kampmann and Renner 2004). As previously described in the section on calibration, the solution of the system of normal equations is carried out by means of Cholesky factorisation according to Luhmann et al. (2020) or Förstner and Wrobel (2016). The adjustment is iterative until the termination criterion is reached. For the second iteration the parameter estimates are updated to the extent that the unknown \(X\) are defined as parameter estimates \({X}_{0}\) for the following iterations. For the determination of the two straight lines of the outer crosshair in the subpixel area with one decimal place, a few iterations are already sufficient. We choose the termination criterion in such a way that in normal cases only a few iterations are necessary. The three opposite lines with approximately the same orientation are balanced to form a straight line (Fig. 7). The outer line cross has thus been determined precisely so that the réseau crosses can then be determined as described. These are located on the straight lines determined at this point and, together with the crosshair centre, define the precise position and orientation of the crosshair in the image coordinate system.
4.2 Crosshair Matching
Due to the possibility that the crosshair position changes after a longer period of time and when the telescope position changes, this must be determined continuously (Atorf et al. 2019). In the case of smaller movements of the crosshair, this can be matched by calculating a normalised correlation coefficient (NCC). The current centre of the crosshair in the image is compared with the last detected crosshair. The current position of the crosshair centre must be within a generated model in order to be matched. If the difference in position is too large, the crosshairs must be detected again. Matching is again carried out using the NCC procedure according to Luhmann et al. (2020). This procedure corresponds to a simplified detection of the precise crosshair centre since no homogeneous background is required throughout. The current crosshair centre should be able to be continuously tracked in the image during a measurement.
For the model image, we define a circle with 0.75 times the distance between the parallel lines around the precise centre of the crosshair. Within this image area, the similarity comparison is carried out by means of normalised crosscorrelation. The model image, in this case the pixels of the circle area, is generated on several image pyramids in different planes and rotations. We generate the image pyramids until the top level still provides enough information about the image. The processing effort is higher than with other correlation methods due to the large number of images generated. However, for the selected image area and due to today’s technology standards, this is not a disadvantage (Luhmann et al. 2020). Finally, the origin of the model image is set to the precise crosshair centre.
For the crosshair matching, the NCC model of the last detected crosshair centre together with the current crosshair is required. We define the search area for the inner crosshair by a circle around the last detected crosshair. We define the radius with 1.5 times of the distance between the parallel lines. This saves unnecessary computing time since the centre of the crosshair cannot be located at the edge of the image. Then, according to Luhmann et al. (2020), all defined instances of the crosshair are detected from within the image section of the current crosshair. We only use the best instance for the crosshair since it is unique in the image. The best instance is characterised as the highest value of the correlation coefficient, whereby this can take values between zero and one. The calculation of the precise crosshairs together with the resulting réseau crosses is then carried out according to the procedure already described. The software saves the coordinates again in a local file. However, the inner crosshair can be determined mathematically by detecting the outer crosshair (Fig. 8). By determining the crosshair lines and the geometric reference to the target axis in subpixel accuracy, the intersection of the crosshair lines does not have to be determined repeatedly in every image, but, for example, may also be overlapped. In the user interface of the control software, coordinate differences between the last detected and the matched crosshair centre are displayed to the user (see Fig. 9).
5 Practical Applications
In the following, we cover exemplary studies on different applications of MoDiTa in the structural health monitoring (SHM) of existing structures.
5.1 Studies on Distance Independence
A practical application for IATS is the SHM of structures, such as factory chimneys, dams or bridges (Paar et al. 2021; Zschiesche 2022). What all these structures have in common is that they usually have elongated dimensions. Often it is impossible to stand directly perpendicular to the structure or to measure the entire structure from the same distance due to environmental conditions, such as rivers or railway lines (Fig. 10). Changes in the distance to the measured object lead to the refocusing of the optics and thus also to changes in the distortions. To discuss this aspect in more detail, we have carried out measurements from different distances to the instrument. For the measurement, we used a TS30 (Leica Geosystems AG 2009) and an industrial camera UI3250 MLM (IDS Imaging Development Systems GmbH 2015) with 1.92 MPixel.
The measurement took place on 3 February 2022 in the courtyard of Mainz University of Applied Sciences between 10 am and 1 pm (CET).
We limited the distance between 3.5 and 100 m. Over this distance calibration measurements were carried out with MoDiTa with a sample of 12 measurements for one distance. These measurements were taken over the entire image area. For comparison, we applied the calibration of the measurement with 20 m distance also to the measurements with shorter or longer distance. We compared the results in the form of residuals or deviations in Fig. 11. The different positions that IATS moves to are visible.
Figure 11b shows residuals of the 20 m measurement calculated with the 20 m calibration in the range of − 0.3 to 0.2 mgon for the horizontal and zenith angle, which is within the expected angular accuracy of the measurement system. We assume that the adjustment modulates the system successfully. In comparison, even at a longer distance to the object, deviations appear only slightly. However, the use of the same calibration shows that at a greater distance significantly larger deviations occur in the marginal area of the measurement image (Fig. 11c). Close to the crosshair, in this case also close to the centre of the image, the deviations have values of max. 0.3 mgon; in the outer range of − 0.6 to 0.9 mgon horizontally and − 0.1 to 0.5 mgon zenith distance. Therefore, the middle part is still modelled within the accuracy of measurement, but systematics can already be identified in the outer area. From a distance of approximately 7 m, the deviations increase significantly. For clarification, Fig. 11e shows the measurement with 4.5 m distance. The range of the residuals is from − 0.7 to 1.1 mgon for the horizontal and zenith angle. Here again, the values are smaller towards the centre, but on average well above 0.3 mgon. The systematic deviations can be traced back to an unsuitable model of the adjustment. Walser (2004) and Wasmeier (2009a) achieved similar results with an integrated coaxial camera and thus correspond to our expectations. We suspect the influence of a scale factor, possibly the optical system’s focal length. Figure 11d, f shows the residuals of the selfcalibrations with the corresponding measurements. In both cases, the residuals remain within the measurement accuracy of the total station and do not exceed 0.3 mgon.
Figure 12 shows the estimated distortion parameters over the entire distance. The estimated nine parameters are presented with their respective accuracies as described in chapter 3. The calculation is made for the entire optics, i.e. for the telescope of the total station and the adapter used (Fig. 1). In the parameter estimation, we calculated a separate calibration for each distance and autofocus setting.
In addition to the individual distances, we gave the respective stepper motor positions since these reflect the respective required mechanical movements of the focusing optics, which are responsible for the changes in the distortion parameters (Wasmeier 2009a).
For the template matching with the crosscorrelation method, a circular section with a radius of less than 40 pixels was used in each case. The maximum mean error of unit of weight from the adjustment is 0.32 mgon. The smallest value is reached during measurement at a distance of 20 m, where it is 0.13 mgon. All measured values of the 12 measurements were used, and no measured values were eliminated during the adjustment.
Table 1 shows the maximum and minimum estimated parameter values and empirical standard deviations corresponding to Fig. 12a–h.
All symmetric radial distortions show their minima and maxima at close range (Fig. 12a–c). In total, the calculated empirical standard deviation of the tangential distortions B_{1} and B_{2} (Fig. 12d, e) show similar orders of magnitude and are more consistent than those of the symmetric radial distortions. Both B_{1} and B_{2} show their greatest value at a long distance, but fluctuate more in the near range.
The results show slightly strong changes of the parameters in the near range. This is in line with our expectations: with larger movements of the focusing optics, we anticipate correspondingly larger changes in the calibration parameters.
Overall, our studies show that micromovements of 0.5 mm/100 m can be resolved. The achievable accuracy depends on the atmospheric conditions and the illumination situation and decreases with increasing distance.
The investigations indicate that in the case of typical distances to structures for SHM of 10 m or greater, it is sufficient to use a single selfcalibration for a complete project.
5.2 Deformation Measurement on a Steel Bridge
The identification of dynamic structural characteristics is an important aspect of SHM. It enables the analysis of monitored vibrations and the calculation of natural frequencies. Changes in natural frequency indicate possible structural damages. Conventionally, acceleration sensors are attached to the structure with a high labour input. MoDiTa enables highfrequency recordings of the movement behaviour without having to enter the structure. For the measurement of frequencies, the maximum frames per second (fps) are essential. Only at an adequately frequent sampling rate can the natural frequency be determined from the measured values and aliasing ruled out. BruschetiniAmbro et al. (2017), Lachinger et al. (2022) do not recommend determining the damping from the excitation of a train crossing, because the selection of the ambient window has too great an influence on the result and therefore the results scatter too much.
To capture the deformation behaviour of a bridge during a crossing, we observed a steel bridge using MoDiTa (Fig. 13a, b). We used a Leica TS60 (Leica Geosystems AG 2020) in combination with an industrial camera UI3080 CPM with 5.04 MPixel (IDS Imaging Development Systems GmbH 2016). The distance to the bridge was approximately 10 m. The recording frequency was 500 Hz with an exposure time of 0.002 s. To achieve this high frequency, we only observed an area of interest. The observed point is located in the upper centre of the western bridge. We attached no targets to the structure. Figure 13c shows the recorded deformation over time. For further analysis, we used the ambient window (shown in Fig. 13c by a green box). The ambient window captures the oscillation behaviour of the bridge without additional mass from the train.
Figure 13d shows the results of the Fast Fourier Transform (FFT) of the ambient window to determine the natural frequencies. Only the results of the low frequencies are shown. A clear peak is visible at 3.9 Hz.
To calculate the damped oscillation using the least squares fit, we evaluated the measured values from second 19.0 to 20.0 (Fig. 13e). The result is a calculated natural frequency of 3.79 Hz. The mean error of unit of weight is estimated at 0.046 mm/10 m. This corresponds to the expected resolution and accuracy, cf. 5.1.
Both evaluations come to almost the same result and confirm each other.
6 Conclusion and Outlook
In this article we explain how the general setup of the measuring system in combination with the developed software detects the crosshair and, thus, performs selfcalibration. The selfcalibration achieves accuracies within the measuring accuracy of the total station. We made assumptions such as the lines of the crosshairs are parallel and six precise lines represent the outer crosshair. We have not investigated either assumption further.
The algorithms explained track the crosshair position and orientation correctly and with subpixel accuracy, although the inner crosshair cannot be found by means of crosscorrelation. Due to the determination of the crosshair lines and the subpixel accurate geometric reference to the target axis, the intersection point of the crosshair lines does not have to be determined again in every image but may also, for example, be covered. This offers the measuring system more flexibility.
In the explained practical application we show results of several calibration calculations. We have shown that it is not necessary to calculate a separate photogrammetric calibration for each distance to the object. For the combination of camera and total station used, areas can be defined which, for example, meet the requirements of the SHM and can be carried out with one calibration of the optical system. As the distance to the calibration distance increases, so do the angular deviations. Generally, the deviations towards the edge of the measurement image are larger. This shows that the distortion approach varies significantly, especially in a closeup range. We assume the influence of the changed camera constant c is responsible for the systematic deviations. The use of one calibration can save a lot of time during the measurement, as it is no longer necessary to measure the calibration pattern again and calculate it.
Furthermore, we carried out an exemplary deformation measurement on a steel bridge and demonstrated the successful use for the determination of natural frequencies. Due to the high recording frequency, it is possible to record the vibration behaviour. Further research is needed in this area, such as the comparison to other sensor systems.
Availability of Data and Material
The data presented in this study are available on request from the corresponding author.
Change history
10 October 2022
A Correction to this paper has been published: https://doi.org/10.1007/s4106402200222y
References
Atorf P, Heidelberg A, Schlüter M, Zschiesche K (2019) Berührungslose Positionsbestimmung von spiegelnden Kugeln mit Methoden des maschinellen Sehens. zfv – Zeitschrift für Geodäsie, Geoinformation und Landmanagement 144: 317–322. https://doi.org/10.12902/zfv02682019
BruschetiniAmbro SZ, Fink J, Lachinger S, Reiterer M (2017) Ermittlung der dynamischen Kennwerte von Eisenbahnbrücken unter Anwendung von unterschiedlichen Schwingungsanregungsmethoden. Bauingenieur 92:2–13
Bürki B, Guillaume S, Sorber P, Oesch H (2010) DAEDALUS: a versatile usable digital clipon measuring system for Total Stations. In: 2010 International Conference on indoor positioning and indoor navigation (IPIN), pp 1–10. https://www.researchcollection.ethz.ch/handle/20.500.11850/159968
Canny J (1983) Finding edges and lines in images. Master's thesis, Massachusetts Institute of Technology, Department of Electrial Engineering and Computer Science
Charalampous E, Psimoulis P, Guillaume S, Spiridonakos M, Klis R, Bürki B, Rothacher M, Chatzi E, Luchsinger R, Feltrin G (2014) Measuring submm structural displacements using QDaedalus: a digital clipon measuring system developed for total stations. Appl Geomat. https://doi.org/10.1007/s125180140150z
Förstner W, Wrobel BP (2016) Photogrammetric computer vision: statistics, geometry, orientation and reconstruction, vol 11. Springer International Publishing, Cham
Grimm D, Zogg HM (2013) Leica Nova MS50 White paper. Leica Geosystems AG, Heerbrugg
Guillaume S, Bürki B, Griffet S, Mainaud Durand H (2012) QDaedalus : Augmentation of total stations by CCD sensor for automated contactless highprecision metrology. In: FIG Working Week 2012
Guillaume S, Clerc J, Leyder C, Ray J, Kistler M (2016) Contribution of the ImageAssisted Theodolite System QDaedalus to Geodetic Static and Dynamic Deformation Monitoring. In: Conference and Seminar Proceedings: 3rd Joint International Symposium on Deformation Monitoring (JISDM). International Federation of Surveyors, FIG, Copenhagen, p 66. https://www.researchcollection.ethz.ch/handle/20.500.11850/126892
Haralick RM, Shapiro LG (1992) Computer and robot vision: Vol. 2. Chapter 14.9. AddisonWesley Longman Publishing Co. Inc
Hauth S, Schlüter M, Thiery F (2013) Schneller und ausdauernder als das menschliche Auge: Modulare Okularkameras am Motortachymeter. In: avn. Allgemeine Vermessungsnachrichten, vol 120. VDE VERLAG GmbH, Berlin  Offenbach, pp 210–216
Huang YD, Harley I. (1989) Calibration of closerange photogrammetric stations using a free network bundle adjustment. In: Gruen, Kahmen (ed) Proceedings of first Conference on Optical 3D measurement techniques. Wichmann Verlag, Vienna/Austria, pp 49–56
IDS Imaging Development Systems GmbH (2015) UI3250MLMGL: Data sheet. https://en.idsimaging.com/store/ui3250ml.html. Accessed 4 Feb 2015
IDS Imaging Development Systems GmbH (2016) UI3080CPMGL: Data sheet. https://en.idsimaging.com/IDS/datasheet_pdf.php?sku=AB00848. Accessed 26 June 2016
ISO 171233 (2001) Optics and optical instruments—field procedures for testing geodetic and surveying instruments—part 3: theodolites. International Organization for Standardization
Kampmann G, Renner B (2004) Vergleich verschiedener Methoden zur Bestimmung ausgleichender Ebenen und Geraden. avn  Allgemeine Vermessungsnachrichten, pp 56–67
Lachinger S, Vorwagner A, Reiterer M, Fink J, Ambro SZ (2022) Entwicklung eines neuen Regelwerkes für dynamische Messungen von Eisenbahnbrücken der ÖBB. In: VDI Wissensforum GmbH (ed) 7. VDIFachtagung Baudynamik, pp 53–65
Leica Geosystems AG (2009) TS30 Technical Data. https://leicageosystems.com/sftp/files/archivedfiles/TS30_Technical_Data_en.pdf. Accessed 4 Feb 2022
Leica Geosystems AG (2020) Leica Nova TS60: Data sheet. https://leicageosystems.com/engb/products/totalstations/robotictotalstations/leicanovats60. Accessed 4 Mar 2021
Luhmann T, Robson S, Kyle S, Boehm J (2020) Closerange photogrammetry and 3D imaging, 3rd edn. De Gruyter, Berlin
Paar R, Marendić A, Wagner A, Wiedemann W, Wunderlich T, Roić M, Damjanović D (2017) Using IATS and digital levelling staffs for the determination of dynamic displacements and natural oscillation frequencies of civil engineering structures. In: Kopačik A, Kyrinovič P, Maria JH (eds) Proceedings of the 7th international conference on engineering surveying  INGEO 2017. Lisbon, Portugal, pp 49–58. https://www.bib.irb.hr/900625
Paar R, Roić M, Marendić A, Miletić S (2021) Technological development and application of photo and video theodolites. Appl Sci 11:3893. https://doi.org/10.3390/app11093893
Reiterer A, Wagner A (2012) System Considerations of an Image Assisted Total Station – Evaluation and Assessment. avn  Allgemeine VermessungsNachrichten, pp 83–94
Schlüter M, Hauth S, Heß H (2009) Selbstkalibrierung motorisierter Digitalkameratheodolite für technische Präzisionsmessungen. In: DVW e. V. – Gesellschaft für Geodäsie, Geoinformation und Landmanagement (ed) zfv – Zeitschrift für Geodäsie, Geoinformation und Landmanagement. WißnerVerlag, Augsburg, pp 22–28
Steger C (1996a) Extracting curvilinear structures: a differential geometric approach. In: Buxton B, Cipolla R (eds) Computer vision—ECCV ’96. Springer Berlin Heidelberg, Berlin, pp 630–641
Steger C (1996b) Extraction of curved lines from images. In: IEEE (ed) Proceedings of 13th International Conference on pattern recognition, 2: 251–255
Steger C (1998) An unbiased detector of curvilinear structures. IEEE Trans Pattern Anal Mach Intell 20:113–125. https://doi.org/10.1109/34.659930
Suesse H, Voss K (1993) Adaptive Ausgleichsrechnung und Ausreißerproblematik für die digitale Bildverarbeitung. In: Pöppl SJ, Handels H (ed) Mustererkennung. Springer, Berlin Heidelberg, pp 600–607
Wagner A, Wasmeier P, Reith, Wunderlich T (2013) Bridge monitoring by means of videotacheometer—a case study. avn  Allgemeine VermessungsNachrichten, pp 283–292
Wagner A, Huber B, Wiedemann W, Paar G (2014) Longrange geomonitoring using image assisted total stations. J Appl Geodesy. https://doi.org/10.1515/jag20140014
Wagner A, Wiedemann W, Wasmeier P, Wunderlich T (2016) Monitoring concepts using image assisted total stations. In: Paar R, Marendić A, Zrinjski M (eds) SIG 2016. Croatian Geodetic Society
Walser BH (2004) Development and calibration of an image assisted total station. Dissertation, ETH Zurich
Wasmeier P (2009a) Grundlagen der Deformationsbestimmung mit Messdaten bildgebender Tachymeter. Zugl.: München, Techn. Univ., PhD thesis, 2009a, Technische Universität München
Wasmeier P (2009b) Videotachymetrie – Sensorfusion mit Potenzial. In: avn. Allgemeine Vermessungsnachrichten, vol 7. VDE VERLAG GmbH, Berlin  Offenbach, pp 261–267
Zschiesche K (2022) Image assisted total stations for structural health monitoring—a review. Geomatics 2:1–16. https://doi.org/10.3390/geomatics2010001
Acknowledgements
The authors would like to thank Alexander Milch, Cedric Jager, Lukas Haas and Michael Biele for their work on the project.
Funding
Open Access funding enabled and organized by Projekt DEAL. The project was funded by the CarlZeissFoundation, BAM—BigDataAnalytics in Environmental and Structural Monitoring, P201702003.
Author information
Authors and Affiliations
Corresponding author
Ethics declarations
Conflicts of Interest/Competing Interests
The authors declare no conflict of interest. The funders had no role in the design of the study; in the collection, analyses, or interpretation of data; in the writing of the manuscript, or in the decision to publish the results.
Additional information
The original online version of this article was revised: there were errors in some equations.
Rights and permissions
Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/.
About this article
Cite this article
Zschiesche, K., Fitzke, M. & Schlüter, M. SelfCalibration and Crosshair Tracking with Modular Digital Imaging Total Station. PFG 90, 543–557 (2022). https://doi.org/10.1007/s41064022002200
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s41064022002200