A calibration method for non-overlapping cameras based on mirrored absolute phase target

Non-overlapping cameras are widely applied in visual measurement system and mobile robotics. The inspection accuracy and performance of the systems highly depend on the calibration accuracy. A novel method for the calibration of non-overlapping cameras is proposed in this paper. A LCD screen is used as a phase target to display two groups of orthogonal phase-shifted sinusoidal patterns during the calibration process. Through a mirror reflection, the phase target is captured by the cameras respectively. The relations between each camera and the phase target can be obtained according the proposed algorithm. Then, the relation between the cameras can be calculated by treating the phase target as an intermediate value. The proposed method is more flexible than a conventional mirror-based approach, because it does not require the common identification points and is robust to out-of-focus images. Both simulation work and experimental results show the proposed calibration method has a good result in calibrating a non-overlapping cameras system. The calibration accuracy can reach 0.2 mm and 0.007° in translations and rotation respectively. A stereo deflectometry system is calculated based on the proposed calibration method. The global measurement accuracy of a standard concave mirror measured by the system can reach 96.8 nm.


Introduction
Multiple cameras are applied in visual measurement [1][2][3], scene surveillance [4,5], and mobile robotics [6][7][8] in case single camera could not satisfy the functional requirement.Due to the design restrictions and cost savings, the field of view (FOV) of the cameras is unable to be guaranteed overlapped.In these cases, the camera parameter and the relative positions of the cameras should be calibrated for a system consisted of non-overlapping cameras.The calibration accu-racy is highly related to the inspection accuracy and performance of the systems.
Recently, several types of calibration approaches are researched to calibrate the non-overlapping cameras.One type of the approaches introduces extra calibration tools to assist the calibration process.Pagel et al. [6] developed an approach for calibrating non-overlapping cameras with hand-eye calibration (HEC) technique [9].Guan et al. [10] researched an approach to obtain the internal and external parameters of non-overlapping camera rig based on HEC.Lamprecht et al. [11] calibrated a non-overlapping cameras system on a vehicle online.However, these approaches are sensitive to the localization accuracy of the calibration tools and are failure when the cameras are unable to move.Other approaches were researched based on a target which moves in the cameras' FOV.Ali et al. [12] proposed an approach for simultaneously recovering the trajectory of a target and the external calibration parameters of non-overlapping cameras.However, this type of approaches cannot meet the accuracy demand of visual measurement.Researchers studied to extend the target size to cover the FOV of cameras simultaneously.Liu et al. [13] proposed an approach to calibrate the extrinsic parameters of multiple vision sensors based on 1D target.Dong et al. [14] combined a multiple cameras system using arbitrarily distributed encoded targets on a wall.However, this type of approaches is limited by the target size and cannot calibrate cameras with 180°a ngle.The mirror-based calibration approaches [15,16] applied a planar mirror to generate an overlapping view between cameras and calibrate cameras based on a 2D calibration target.This type of approaches improves the flexibility in the design of calibration target size, avoids the error introduced by positioning tools, and does not require the cameras' movement during calibration process.However, it is less convenient to implement because cameras have to find the same identification points though the mirror reflection.Moreover, the common points cannot be extracted successfully when the target is not in the depth of field (DOF) of the cameras.The constantly increasing demand to improve surface quality and credibly determinate its functional properties of machine parts has resulted in increase of research in manufacturing and surface metrology area.Krolczyk et al. [17] analyzed the influence of clean manufacturing methods on the surface morphology of duplex stainless steel.Nieslony et al. [18] present an analysis of surface topography of explosively clad material such as titanium-plated steel in drilling process.Huang et al. [19] researched a distance calibration method for direct phase me as u rin g d ef le ct ome t r y to me as ur e t he thr eedimensional shape of specular objects with discontinuous surfaces.Stereo deflectometry is an essential technique to obtain the form information of freeform specular surfaces [20].The measurement accuracy of stereo deflectometry is very sensitive to systematic errors; therefore, the calibration of the geometrical relation between the components plays an important role.Traditional calibration approaches apply extra special equipment [21][22][23] to calibrate stereo deflectometry.However, the equipment not only make the calibration process complex but also bring new error source.Therefore, researchers [24,25] tried to put a flat mirror at the measured position and used the mirrored screen as calibration target to calibrate the system.Since the cameras of stereo deflectometry are focused and have overlapping fields on the measured position, the capture fields on the mirrored screen are defocused and non-Fig. 1 The illustration of the proposed method Fig. 2 Workflow of the proposed method Fig. 3 The simulation setup overlapping.Therefore, the cameras can be treated as non-overlapping cameras system when conducting the calibration with the mirrored screen.However, previous non-overlapping camera calibration approaches [6,[9][10][11][12][13][14][15][16] cannot meet the calibration requirement of stereo deflectometry.
This paper presents a novel calibration method based on mirrored phase target for non-overlapping cameras.Through the mirror reflection, the relation between each camera and the phase target can be obtained.Then, the relation between the cameras can be calculated by treating the phase target as an intermediate value.Since the phase target is consisted of sinusoidal fringe patterns, the proposed method is robust to out-of-focus.

Principle
The illustration of the proposed method is illustrated in Fig. 1.A LCD screen is used to display two groups of orthogonal phase-shifting sinusoidal patterns.Through the reflection of flat mirrors, the patterns are captured by the cameras.After phase-shifting and phase-unwrapping algorithm [26] applied, two orthogonal absolute phase maps can be obtained.The calculated phase maps are used as targets during the calibration process.
For a pixel m of camera 1, the corresponding physical position M ' (x w , y w ) in the mirrored LCD coordinate system can be uniquely located based on its absolute phase value (φ x , φ y ) according to Eq. ( 1).
where p is the size of LCD pixel pitch and n p is the number of LCD pixels per fringe period.Camera parameter A 1 and the relation [R ' t ' ] between mirrored screen and the camera coordinate system can be obtained based on the pinhole model by moving mirror 1 to at least three arbitrary positions: The relation R 1 t 1 ½ between the camera and the real screen can be calculated with the least-squares solution by at least three mirror reflections according to Eq. (3).where n is the mirror normal vector expressed in camera frame, d the distance between the mirror and the camera center.n can be obtained based Eq. ( 4).
where i, j, k represent three arbitrary mirror positions.m is the unit vector that is perpendicular to both two mirror normal vectors, and can be calculated according to Eq. ( 5).
The remaining unknown parameter d in Eq. ( 3) can be solved by a linear equations as: where I is a 3 × 3 identity matrix.Until now, the initial value of the camera parameter and relation between camera 1 and the LCD are obtained, and then they are optimized by minimizing the following function with Levenberg-Marquardt algorithm: where M is the corresponding point of m in terms of the LCD, g is the number of mirror movement, and k is the number of camera pixels.Based on the same algorithm, the parameter A 2 of camera 2 and the relation R 2 t 2 ½ between camera 2 and the LCD can be calculated.Then, the relation R c t c ½ between cameras can be determined according to Eq. ( 8) by treating the LCD as an intermediate value.
The whole calibration workflow is shown in Fig. 2.
3 Experiment and results

Simulation study
In order to test the practicability and accuracy of the proposed calibration method, a system with two non-overlapping cameras was simulated and calibrated with the proposed calibration method as shown in Fig. 3. Fig. 5 The tested stereo deflectometry system The size of the simulated camera image is 1280 × 960 with the principal point at (640, 480) pixel.The Euler gles between the two camera systems are 0°, 0°, and 180°in terms of α, β, and During the calibration process, a screen is used as the phase target to calibrate the system parameters.The phase target is captured by the two cameras through the reflection of two flat mirrors.In order to simulate the real experimental environment, random noise with a maximum value of 0.005 mm is added to the obtained physical positions.Arbitrary moving the flat mirror to more than three positions, the mirrored screens can be located at different calibration positions in terms of the cameras.The relation of the cameras can be calculated from the camera pixels and the corresponding physical points on the mirrored screen based on the proposed calibration method.In order to evaluate the accuracy of the proposed method, the calculated result from the proposed method is compared with the simulated true value as shown in Table 1.

Experimental study and discussion
An experiment has been conducted on a non-overlapping cameras system.The setup hardware system is illustrated in Fig. 4. The LCD monitor is Dell E151Fpp with a resolution of 1280 × 1024.The pixel pitch of the LCD is 0.297 mm.The camera is Lumenera CCD sensor (model Lw235M) with a resolution of 1616 × 1216.The camera lens is a Navitar lens with 35-mm-fixed focal length.The cameras are mounted with an approximate angle of 180°.In fact, it is difficult to obtain the real relation between the two camera coordinate systems.In order to compare with the calibration result from the proposed method, the relation between the camera coordinate systems is estimated by ignoring the deviation between the camera coordinate center and the CCD geometric center in terms of the x-y plane.The translations between the two coordinate systems along with u and v direction are estimated to be both 0. The Euler angles between the two camera systems are estimated to be 0°, 0°, and 180°in terms of α, β, and γ respectively.Because the translation along z direction is impossible to be estimated, this value is not be compared.The calibration result is shown in Table 2.The residual between the calibration results with the true value is within 0.007°in rotation and 0.2 mm in translation as demonstrated in Table 1.
The results demonstrate the reliable accuracy of the proposed method can reach three decimal places degree and one decimal places millimeter in rotation and translation respectively.Therefore, the calibration results are written to four decimal places degree and two decimal places millimeter in rotation and translation respectively in Tables 1 and 2.
In order to demonstrate the superiority of the proposed calibration method, a stereo deflectometry system was tested.The stereo deflectometry system which consists of a screen and two cameras is shown in Fig. 5. Since the cameras of stereo deflectometry are focused and have overlapping fields on the measured position, the capture fields on the mirrored are defocused and non-overlapping, as shown in Fig. 6.Therefore, the cameras can be as a non-overlapping cameras system when conducting the calibration with the mirrored screen.The initial value of the geometrical relation of the cameras was calculated based on the proposed calibration method and then optimized with other geometrical parameters based on the calibration method proposed in [27].A concave mirror (stock no.40-913) as shown in Fig. 7a from Edmund (https://www.edmundoptics.com/) with λ/8 surface accuracy was measured based on the calibration result.The effective focal length of the mirror provided by the manufacturer is 635 mm.The focal length tolerance of the mirror is ±2%.The measurement result of the mirror is shown in Fig. 7b.A standard sphere is fitted based on the measurement result to evaluate the result.The effective focal length of the fitted sphere is 633.1 mm which is within the focal length tolerance.The PV and the RMS of the deviation between the measurement result and the fitted sphere are 96.8 and 17.1 nm respectively, as shown in Fig. 7c.

Conclusion
A calibration method based on mirrored phase target is proposed to calibrate the system with non-overlapping cameras.Comparing with the previous calibration approaches, the proposed method does not require the common identification points and is robust to out-of-focus images.Experiments demonstrate the proposed method is flexible and accurate.The translations accuracy and rotation accuracy can reach 0.2 mm and 0.007°respectively.The proposed calibration method has been used to calibrate a stereo deflectometry system.The global measurement accuracy of a standard concave mirror measured by the system can reach 96.8 nm.Further studies will be made on increasing the calibration speed.

Fig. 6
Fig.6The illustration of treating stereo deflectometry as a nonoverlapping cameras systems