A new calibration method for line-structured light vision sensors based on concentric circle feature
Abstract
Background
Determining the relative ubiety between the camera and the laser projector in a line-structured light vision sensor is a classical yet important task. Typical calibration methods often confront problems, such as difficulty of producing the target precisely and introduction of perspective projection errors.
Methods
In this work, a new calibration method based on a concentric circle feature is introduced. The proposed method is based on geometrical properties and can reduce the perspective projection error. In our method, the vanishing line of the light plane is firstly deduced from the imaged concentric circles. Then the normal vector of the light plane is determined. Consequently, the complete expression can be confirmed from the principle of the intersecting planes.
Results and conclusion
The proposed method is simple and robustness as the basic theory is geometrical properties. Accuracy evaluation experiment shows that the accuracy of the calibration method can reach 0.07 mm within the view field of about 200 × 200 mm. This accuracy is comparable to the commonly used calibration method with a checkerboard planar target, whereas our target is simple to produce.
Keywords
Calibration Concentric circles Light plane Vanishing lineBackground
The Line-Structured Light Vision Sensor (LSLVS) plays an important role in the field of industry measurement owing to its wide measurement range, high precision, real-time ability, simple information extracting, and so forth [1, 2]. The typical structure consists of one camera and one laser projector. Estimation of the relationship between the camera and the laser projector, which is called as calibration of LSLVS, is one of the most important tasks.
Heretofore, there are many calibration methods. According to ways of obtaining feature points on the structured light plane, these methods can be classified into three categories: three-dimensional (3-D) target method [3, 4], planar target (2-D) method [5, 6, 7], and one-dimensional (1-D) target method [8, 9, 10, 11, 12, 13].
The 3D target method is based on the invariance of double cross-ratio. With a special calibration 3D target, enough non-collinear feature points on the structured light plane can be determined based on the theory of the invariance of double cross-ratio. Afterwards, relative ubiety between the camera and the structured light plane projector is confirmed. Different planar targets with different features are used according to different theories. In Ref. [10], a planar target that contains a pattern of squares is utilized to finish the calibration of LSLVS. The intersection points of the light stripe and calibration squares with exactly known size can be obtained under the image coordinate system (ICS). Accordingly, feature points on the structured light plane can be gained based on the invariance of cross-ratio. A 1-D target method is proposed owing to its convenient operation. The feature point, namely the intersection point of the light stripe and a 1D target, can be determined based on the invariance of cross-ratio. Therefore, enough feature points can be obtained from the random movements of the 1D target to different positions.
Unfortunately, the 3D target based method is not well suited for on-site calibration as there will be some inevitable problems, e.g., mutual occlusion between different planes of the target, the difficulty of producing the target precisely, the cumbersomeness of the target, etc. In the 1D target based method, few feature points are obtained and the calibration result is not precise enough. In comparison, the planar target-based method is with easy operation and satisfactory results but perspective projection error is inevitable.
In this paper, a planar target with a pattern of two (or more) concentric circles is utilized to finish the calibration of LSLVS. In our calibraiton algorithm, the vanishing line of the light plane is deduced from imaged circles, then normal vector of the light plane is confirmed. The complete expression can be confirmed from the intersecting planes under a camera coordinate system. In the application, we evaluate our algorithms and accurate results are achieved. The contribution of our paper is that we initiate a new method based on a concentric circles feature to calibrate LSLVS. The algorithm can reduce the perspective deviation with a satisfying result. Moreover, the planar target used in our method is easy to make precisely, and meanwhile, the proposed method is efficient and convenient, especially for onsite calibration.
Measurement model
The location relationship between the camera in LSLVS and the structured-light plane projector remains unchangeable in the process of calibration and measurement. So the structured-light plane can be expressed as a fixed function, which is defined as Eq.(1) under camera coordinate system.
where r^{2} = x^{2} + y^{2}, (x, y)^{T} is the distorted image coordinate, and \( {\left(\overline{x,}\overline{y}\right)}^T \) is the idealized one, k_{1}, k_{2} are the radial distortion coefficients of the lens.
Properties of concentric circles
The TCS is not unique when the concentric circles are described as Eq.(4) and Eq.(5). Under any one coordinate system whose origin coincides with the center of these concentric circles, expressions of these concentric circles are the same.
Where p is the homography expression of the point located on the circle.
where \( V=\left[\begin{array}{ccc}1& 0& {x}_0\\ {}0& 1& {y}_0\\ {}0& 0& 1\end{array}\right] \) and \( D=\left[\begin{array}{ccc}1& 0& 0\\ {}0& 1& 0\\ {}0& 0& {r}_1^2/{r}_2^2\end{array}\right] \). From matrix D, we find that \( {C}_2^{\hbox{'}-1}{C}_1^{\hbox{'}} \) has three eigenvalues, of which two are identical and one is different. From matrix V, we find that the corresponding eigenvectors of the identical eigenvalues are [1 0 0]^{T}, [0 1 0]^{T}, which are points on the infinity line. We also find the corresponding eigenvector of the different eigenvalue is [x_{0} y_{0} 1]^{T}, which is the circle center. Based on the above analysis, we can conclude that the circle center and the line at infinity can be recovered by the eigenvectors of the matrix \( {C}_2^{-1}{C}_1 \) [14].
As the world coordinate system can be defined freely, the concentric circles can be expressed as Eq.(4) and Eq.(5). Eq.(13) is the form of AX = XB, which is widely used in robot hand-eye calibration [15]. The general solutions of Eq.(8) can be calculated easily. One solution is corresponding to one coordinate system (TCS). So any one of the solutions can be chosen, then the matrix H transformed from the image plane to the target plane (under TCS) can be confirmed.
Calibration
As described in Eq.(11), matrix \( {C}_2^{-1}{C}_1 \) is similar with the matrix \( {C}_2^{\hbox{'}-1}{C}_1^{\hbox{'}} \), defining the eigenpair of \( {C}_2^{-1}{C}_1 \) as (λ, x). According to the property of similarity transformation, the eigenpair of \( {C}_2^{\hbox{'}-1}{C}_1^{\hbox{'}} \) is (λ, Hx). As is known, the vanishing point is the image of the infinity point and the vanishing point must be located on the vanishing line, which is the image of the infinity line. When projections of the two concentric circles (\( {C}_1^{\hbox{'}} \) and \( {C}_2^{\hbox{'}} \)) are determined, the image circle center and the vanishing line of the target plane can be deduced from their eigenvectors.
Eq.(19), k is a scale factor. As the normal vector of the light plane crossing the plane stripe is known, the parameter D_{L} can be easily be confrimed from Eq.(19) and Eq.(1).
Briefly, the calibration procedure is given as follow:
Step 1: Extract concentric circles images \( {C}_1^{\hbox{'}} \) and \( {C}_2^{\hbox{'}} \).
Step 2: Calculate the eigenvectors of \( {C}_2^{\hbox{'}-1}{C}_2^{\hbox{'}} \), then the vanishing line and the imaged circle center can be confirmed.
Step 3: Confirm the normal vector of the light plane from its vanishing line.
Step 4: Confirm the homography matrix from the target plane to the image plane. Then the rotation matrix and translation matrix from the world coordinate system to the camera coordinate system is deduced.
Step 5: The back projection plane can be confirmed based on the light strip on the target plane.
Step 6: Parameter D_{L} which is defined in Eq.(1) can be confirmed from Eq.(19).
Simulation
The pattern on the plane target contains two concentric circles with a radius of 25 mm and 20 mm respectively.
Influence of image noise on calibration accuracy
In our simulations, the target is moved to 5 different positions. 200 feature points on each circle image are chosen to fit the ellipse and the least squares ellipse fitting algorithm is utilized. The light strip is fitted by 30 image points. Gaussian noise with standard deviations varying from 0 to 1.0 pixels is added to both coordinates of the image points to generate the perturbed image points.
Each point in Fig. 5 represents result averaged 200 uniformly distributed rotations. From Fig. 5, we can see that errors increase over the noise level, including root mean square error and mean absoulte error.
Influence of circle number on calibration accuracy
As illustrated in Fig. 6 and Fig. 7, the calibration result is better as the number of concentric circles increases. When more concentric circles are used, the vanishing line and the homography matrix have a precise result. In this case, calibration result is more accuracy.
Results and discussion
Camera calibration
Intrinsic parameters of the camera
f _{x} | f _{y} | u _{0} | v _{0} | k _{1} | k _{2} |
---|---|---|---|---|---|
5124.211 | 5125.933 | 1271.232 | 1047.570 | −0.224 | −0.473 |
In Table 1, f_{x} is the scale factor in the x-coordinate direction, f_{y} is the scale factor in the y-coordinate direction, (u_{0}, v_{0})^{T} is the coordinates of the principal point. k_{1} and k_{2} are the distortion coefficients of lens.
Sensor calibration
Accuracy evaluation
A planar checkerboard-pattern target is used to evaluate the accuracy of the proposed calibration method. As the side length of each checkerboard is known exactly, the coordinate of the feature points under TCS can be solved based on the invariance of cross-ratio. The theory is descirbed as follows:
The real length of AD can be solved, so can A_{1}D_{1}. Then the distance between point D and point D_{1} can be worked out, i.e. the distance between each of the adjacent feature points (dTru) can be obtained, which can be treated as the true value owing to its high accuracy.
Evaluation of our calibration results
NO. | dTru (mm) | dMea (mm) | Error (mm) |
---|---|---|---|
1 | 20.934 | 20.999 | 0.065 |
2 | 17.214 | 17.151 | −0.063 |
3 | 22.095 | 22.097 | 0.002 |
4 | 22.604 | 22.655 | 0.051 |
5 | 21.072 | 21.173 | 0.101 |
6 | 21.546 | 21.664 | 0.118 |
7 | 21.459 | 21.471 | 0.012 |
8 | 19.353 | 19.260 | −0.093 |
9 | 20.933 | 20.843 | −0.090 |
10 | 18.027 | 17.965 | −0.062 |
RMS | 0.072 |
In Table 2, dTru denotes the real distance between two feature points, while dMea is the measurement result based on the calibration result using the proposed method. As listed in Table 2, the root mean square error of the calibration result (RMS) obtained by our proposed method is 0.072 mm. The calibration results are precise enough as two concentric circles are used and the target is just placed to four different positions.
Comparisons & Discussions
- A.
The method based on the invariance of double cross-ratio (named 3D method) [7]. The utilized 3D target consists of two rigid planes that are perpendicular to each other, and squares and/or rectangles exist on each plane. Enough features points can be obtained based on the invariance of double cross-ratio. The root mean square error of the measurement is less than 0.151 mm as mentioned in Ref. [7] within the field of about 300×300 mm.
- B.
The planar target based method (named 2D method) [10] approach uses a planar target with a pattern of 3×3 squares to calibrate the LSLVS, the size of each square is 35×35 mm. The intersection points of the light stripe and calibration squares with exactly known size can be obtained under the image coordinate system. Accordingly, feature points on the structured light plane can be gained based on the invariance of cross-ratio. The root mean square error of 30 distances as mentioned in the experiment of [10] is 0.085 mm within the field of about 200×200 mm.
- C.The 1D target based method (named 1D method) [8] in the calibration approach, has a target length of about 400 mm. And six small holes, the distance of each adjacent pair is 40 mm, are located on the target. One feature point can be determined based on the invariance of cross-ratio each time. Enough feature points can be obtained to calibrate the LSLVS from the random movements of the 1D target. The root mean square error of the measurement can reach 0.065 mm in [8] within the field of about 300×200 mm. These corresponding target are illustrated in Fig. 13.
As metioned above, the 3D target based method is not well suited for on-site calibration as some inevitable problems, e.g., mutual occlusion between different planes of the target, the difficulty of producing precisely, the cumbersomeness of the target, etc. Comparatively, the 2D method and 1D method is more suitable for on-site calibration. But as the feature points of the 1D target is less than the 2D target, the target should be moved to more positions to finish the calibration (or get an accurate result). Therefore, the 2D method is more convenient and is popularly utilized. In this case, different planar targets appeared.
Evaluation of Zhou’s calibration results
No. | 1 | 2 | 3 | 4 | 5 | 6 | 7 | 8 | 9 | 10 |
---|---|---|---|---|---|---|---|---|---|---|
dTru (mm) | 20.934 | 17.214 | 22.095 | 22.604 | 21.072 | 21.546 | 21.459 | 19.353 | 20.933 | 18.027 |
dMea (mm) | 20.997 | 17.295 | 22.02 | 22.687 | 21.098 | 21.466 | 21.415 | 19.362 | 21.025 | 18.12 |
Error (mm) | 0.063 | 0.081 | −0.075 | 0.083 | 0.026 | −0.08 | −0.044 | 0.009 | 0.092 | 0.093 |
RMS | 0.070 |
In Fig. 15, the method with two circles is the proposed calibration method using a planar target with two concentric circles, while the method with three circles is the proposed calibration method using a planar target with three concentric circles. Compared with the calibration results listed in Table 2, we can get the conclusion that the accuracy of our proposed calibration method is comparable to the method which uses a planar target.
Comparison of calibration results
NO. | 3D method | 2D method | 1D method | Our method one |
---|---|---|---|---|
RMS | 0.151 | 0.070 | 0.065 | 0.072 |
Comparison of calibration features
Items | 3D method | 2D method | 1D method | Our method one |
---|---|---|---|---|
computational complexity | Normal | Normal | Complex | Normal |
operation complexity | Complex | Normal | Easy | Normal |
Target machining | Hard | Normal | Easy | Easy |
Time costs | Max | Min | Median | Min |
Suitable for on-site calibration | Unsuitable | Suitable | Suitable | Suitable |
Conclusions
In this paper, a calibration method based on properties of concentric circles is described. A planar target with a pattern of several concentric circles is utilized to finish the calibration of LSLVS. The normal vector of the light plane is deduced from its vanishing line. Then the parameter D is confirmed by the back projection plane deduced based on the light strip on the target plane.
The contribution of our paper is that we initiate a new method based on concentric circles to calibrate LSLVS. The introduced method can reduce the perspective deviation and obtain a precise result. Moreover, the planar target used in our calibration method is easy to make precisely, meanwhile, the proposed method is efficient and convenient, especially for onsite calibration.
Notes
Acknowledgements
This work was supported by Postdoctoral Sustentation Fund of Qingdao (861805033068), the National Natural Science Fundations of China (U1706218 and 41576011). The authors express their gratitude to vision laboratory of Ocean University of China for supply of experiment conditions. We would also like to thank one anonymous reviewer for helpful suggestions that improved this manuscript.
Funding
Declared at acknowledgements.
Availability of data and materials
Data sharing is not applicable to this article as no datasets were generated.
Authors’ contributions
Conceived and designed the experiments: MS JD. Performed the experiments: MS. Analyzed the data: MS. Contributed reagents/materials/analysis tools: MS JD. Wrote the paper: MS AM. All authors read and approved the final manuscript.
Competing interests
The authors declare that they have no competing interests.
Publisher’s Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
References
- 1.BM. Chung, “Improved least-squares method for phase-to-height relationship in fringe projection profilometry”, https://doi.org/10.1186/s41476-016-0014-8, 2016.CrossRefGoogle Scholar
- 2.Dewar, R.: Self-generated targets for spatial calibration of structured light optical sectioning sensors with respect to an external coordinate system. In: Proceedings of the Robots and Vision Conference, pp. 5–13. Detroit, Mich, USA (1988)Google Scholar
- 3.James, K.W.: “Noncontact machine vision metrology with a CAD coordinate system,” Autofact’88 Conference Proceedings, pp. 9–17 (1988)Google Scholar
- 4.Duan, F., Liu, F., Ye, S.: A new accurate method for the calibration of line structured light sensor. Chinese Journal of Scientific Instrument. 211, 108–110 (2000)Google Scholar
- 5.Xu, G., Liu, L., Zeng, J.: A new method of calibration in 3D vision system based on structure-light. Chinese Journal of Computers. 18(6), 450–456 (1995)Google Scholar
- 6.Huynh, D.Q., Owens, R.A., Hartmann, P.E.: Calibrating a structured light stripe system: a novel approach. Int. J. Comput. Vis. 33(1), 73–86 (1999)CrossRefGoogle Scholar
- 7.Wei, Z., Zhang, G., Xu, Y.: Calibration approach for structured-light-stripe vision sensor based on the invariance of double cross-ratio. Opt. Eng. 42(10), 2956–2966 (2003)ADSCrossRefGoogle Scholar
- 8.Wei, Z., Cao, L., Zhang, G.: A novel 1D target-based calibration method with unknown orientation for structured light vision sensor. Opt. Laser Technol. 42(4), 570–574 (2010)ADSCrossRefGoogle Scholar
- 9.Zhou, F., Cai, F.: Calibrating structured-light vision sensor with one-dimensional target. Journal of Mechanical Engineering. 46(18), 7–12 (2010)CrossRefGoogle Scholar
- 10.Zhou, F., Zhang, G.: Complete calibration of a structured light stripe vision sensor through planar target of unknown orientations. Image Vis. Comput. 23(1), 59–67 (2005)CrossRefGoogle Scholar
- 11.Wei, Z., Xie, M., Zhang, G.: “Calibration method for line structured light vision sensor based on vanish points and lines”, ICPR, pp. 794–797 (2010)Google Scholar
- 12.Xu, K.: Monolithically integrated Si Gate-controlled light-emitting device: science and properties. Journal of Optics. 024014: doi.org/10.1088/2040-8986/aaa2b7
- 13.Harun, S.W., Lim, K.S., Damanhuri, S.S.A., Ahmad, H.: Microfiber loop resonator based temperature sensor, vol. 6, (2011). https://doi.org/10.2971/jeos.2011.11026 CrossRefGoogle Scholar
- 14.Huang, H., Zhang, H., Cheung, Y.: “The common self-polar triangle of concentric circles and its application to camera calibration”, IEEE Conference on Computer Vision and Pattern Recognition, pp. 4065–4072 (2015)Google Scholar
- 15.Mao, J., Huang, X., Jiang, L.: “A flexible solution to AX=XB for Robot Hand-Eye calibration”, 10th WSEAS Int. In: Conference on ROBOTICS, CONTROL and MANUFACTURING TECHNOLOGY, pp. 118–122Google Scholar
- 16.Shiu, Y., Ahmad, S.: 3D location of circular spherical features by monocular model-based vision. In: Proceedings of the IEEE Conference on System, Man and Cybernetics, pp. 576–581. Cambridge, Mass, USA (1989)CrossRefGoogle Scholar
- 17.Safaee-Rad, R., Tchoukanov, I., Smith, K.C., Benhabib, B.: Three-dimensional location estimation of circular features for machine vision. IEEE Trans. Robot. Autom. 8(5), 624–640 (1992)CrossRefGoogle Scholar
- 18.R. Hartley and A. Zisserman, “Multiple View Geometry in Computer Vision”, Cambridge University Press, 2003, ch.2–8Google Scholar
- 19.Zhang, Z.: A flexible new technique for camera calibration. IEEE Trans. Pattern Anal. Mach. Intell. 22(11), 1330–1334 (2000)CrossRefGoogle Scholar
- 20.J. Bouguet, “Camera calibration toolbox for Matlab”, [Online] Available from: http://www.vision.caltech.edu/bouguetj/calib_doc/
- 21.Steger, C.: Unbiased extraction of curvilinear structures from 2D and 3D image [Ph.D. Dissertation], Technische Universitaet Muenchen (1998)Google Scholar
Copyright information
Open AccessThis article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made.