Determination of Focal Length for Targets Positioning with Binocular Stereo Vision

Abstract

Targets positioning with binocular stereo vision has the potential of usage for surveillance on airdrome surface. This paper analyses the impact of focal length of a camera on the accuracy of positioning targets on the ground and proposes a way of determining the value of the focal length, with which measuring errors on the edges of the surveillance area could be mitigated. With profound analysis of the impact of many system parameters on the coordinates of targets, it is found that the focal length predominates. In order to establish the basis for calibration, a set of non-linear equations are solved for relevant system parameters, thus yielding their explicit expressions. An on-site experiment is implemented for calibration for parameters, as well as measuring the positions of targets. A test environment is established and data are obtained for calculating curves of coordinates versus the focal length, which gives us a clearer indication of determining the focal length of the camera. In a similar way, the curve gives also an implication of selecting the appropriate points where to place calibration objects, thus reducing errors of targets positioning.

This work is supported by the joint funded project of National Natural Science Fund Committee and the Civil Aviation Administration of China (No. 61179043).

1 Introduction

With an ever increasing traffic volume of surface movement on airdromes, the situation is complex and maneuvering is becoming more difficult for air traffic controllers. Although there have been already several means of surveillance quipped on the airdromes, most of them are just for cooperative targets, which is not sufficient in the next generation of air traffic control management. Therefore, finding a new method of surveillance for non cooperative targets on a surface is of great importance. A method of positioning these sorts of targets based on binocular stereo vision has the potential for this purpose.

Having finished modeling binocular stereo vision for remote coordinate measurement in our previous work, reasonable calibration is critical for on-site positioning targets with improved accuracy. In three-dimensional machine vision systems, the position, size and other information of objects are calculated from the information of acquired images from cameras [1]. In order to obtain the correspondence of spatial points to camera image pixels accurately, the cameras are supposed to calibrated [2]. There are several ways of obtaining parameters from camera calibration, but most of them just calibrate parameters of the camera lens based on pinhole imaging [3, 4].

Analysis of the impact of system and camera parameters on the measurement accuracy of targets shows that the focal length contributes the most to the accuracy. Therefore, finding a reasonable focal length is meaningful, although its nominal value is usually provided. This paper proposes a method of determining the focal length for targets positioning with binocular stereo vision. An explicit formulation of expressing the focal length is derived from a well established calibration equation, a non linear one in which iteration is needed for solution before. Simulation is made showing the relationship between the coordinates and the focal length and other parameters. It is necessary to establish an appropriate mathematical model of the camera to solve for the relation between the two-dimensional image coordinates and the three-dimensional world coordinates [5]. Papers [4, 6] have made several experiments to calculate system parameters, positions of objects, and analyzed the accuracy concerned under the prerequisite of β equaling 0. In this paper, β is calibrated and the focal length is probed as an internal parameter mainly for the final goal of finding a reasonable value of the focal length.

2 Derivation of Explicit Expressions for Calculation

Considering a positioning system applied in a wide area based on binocular stereo vision, it is preferable to begin with the pinhole imaging model to get an expression of solving for coordinates of targets. In the same way, the model of calibrating system and camera parameters is also from the same principle based on perspective projection matrix, with which a mapping equation is established, expressing the relation among a world coordinate, a camera coordinate and an image coordinate system. When the origin of world coordinate is a point where two optical axes of binocular cameras meet, the equation could be well simplified and written as follows [5, 6]:

$$ (X_{w} + S_{x} )(X\sin \alpha - Y\cos \alpha \sin \beta - f\cos \alpha \cos \beta ) + (Z_{w} + S_{z} )(X\cos \alpha + Y\sin \alpha \sin \beta + f\sin \alpha \cos \beta ) = 0 $$

(1)

where, (X,Y) is a point expressed in pixel of a target on an image, (α,β) and f are the rotation angles of and the focal length of the camera, (S _x ,S _z ) is the coordinate of a camera in the world plane coordinate system. Suppose system and camera parameters can be made known, and the pixel value can be determined with image processing, there remain only two unknowns X _w and Z _w , representing the surface position of a target in the world coordinate system.

In the typical configuration of using a binocular stereo vision, two cameras are separated apart, focusing the same area for obtaining an image, denoted as pixel numbers (X ₁ ,Y ₁ ) and (X ₂ ,Y ₂ ) for the same target. When two cameras are installed, two equations, with the form of Eq. (1) but a pair of different parameters denoted as (α ₁ ,β ₁ ), f ₁ , (S _x1 ,S _z1 ) and (α ₂ ,β ₂ ), f ₂ , (S _x2 ,S _z2 ), are combined to solve for X _w and Z _w , yielding the coordinate of the target.

For calibration, parameters (α,β) and f are supposed to be unknown. In this case, solving for these three parameters, three equations as (1) with different (X _w ,Z _w ) and (X,Y) are needed and combined. For this purpose, targets should be placed in the field, with known positions in the world and image coordinate systems. Suppose the coordinates are denoted as P(X _w1 ,Z _w1 ), P(X _w2 ,Z _w2 ) in the world coordinate system, (X ₁ ,Y ₁ ) and (X ₂ ,Y ₂ ) in the image coordinate system, equations can be obtained by taking them into the Eq. (1). With α being less than 90°_, it can be solved from each equation for one camera as follows:

$$ \tan \alpha = \frac{{(X_{w1} + S_{x} )(Y_{1} \sin \beta + f\cos \beta ) - (Z_{w1} + S_{z} )X_{1} }}{{(X_{w1} + S_{x} )X_{1} + (Z_{w1} + S_{z} )(Y_{1} \sin \beta + f\cos \beta )}} $$

(2)

$$ \tan \alpha = \frac{{(X_{w2} + S_{x} )(Y_{2} \sin \beta + f\cos \beta ) - (Z_{w2} + S_{z} )X_{2} }}{{(X_{w2} + S_{x} )X_{2} + (Z_{w2} + S_{z} )(Y_{2} \sin \beta + f\cos \beta )}}. $$

(3)

When combining expressions (2) and (3), rotation angle β can be found to be one of four roots from a quartic equation of β by eliminating the parameter f.

After parameters (α,β) have been solved for, the focal length can be written as an expression as follows:

$$ f = \frac{{((X_{\text{w1}} + S_{x} ) {\text{sin}}\alpha\,{ \cos }\beta \, - (Y_{\text{w1}} + S_{y} ){ \sin }\beta + (Z_{w1} + S_{Z} )\cos \alpha \cos \beta )X_{1} }}{{(X_{\text{w1}} + S_{x} )\cos \alpha - (Z_{w1} + S_{Z} )\sin \alpha }} $$

(4)

Calibration of both α and β requires that at least two targets should be used. The focal length can be calculated under the premise that some values remain unchanged.

To show the impact of focal length on the coordinates (X _w ,Z _w ) of a target, expressions for both X _w and Z _w can be solved for from equations similar to Eq. (1) and calculation will yield the following relation, fixing all other parameters, except the focal length on the left and right camera. From this figure, it is obvious that focal length has great impact on X _w and Z _w (Fig. 1).

Fig. 1.

Relation between world coordinate and $f$

3 Determination of Focal Length

Although the nominal value of the focal length of a camera is usually provided, finding a reasonable value of the focal length is meaningful for targets positioning with binocular stereo vision.

3.1 Setup of Experiment

An experimental schematic diagram is shown in Fig. 2, with an overall arrangement for positioning and calibration system parameters are as follows, the distance from the coordinate origin to the left station and to the right station are 345.2046 m and 328.58 m, respectively. Angles between a line from the origin to the station and the north direction from the station are 160.3395° and 129.7745°. These parameters are obtained by using differential GPS with accuracy of 20 ml.

Fig. 2.

Experimental schematic diagram

Components of one station for this experiment are shown in Fig. 3. The two stations transmit images with microwave, and the left and right computers transmit images to a host computer, which processes the data from the experiment and output results on site.

Fig. 3.

Components of a station

3.2 Experimental Data and Results

With the data ($f_{l} = 24.7859{\kern 1pt} {\text{mm}}$, $f_{r} = 25.2338{\kern 1pt} {\text{mm}}$) above and the data in the Table 1 below.

Table 1. Data from experiment

We can get the parameters $\alpha$ and β, the results are shown below in the Table 2.

Table 2. Calibration of α and β (Unit: rad)

In the experiment, and the image planes keep vertical (β = 0) or β can be a fixed small value for the sake of convenience. So here $\alpha$ and $f$ are recalculated again and $\alpha_{l} = - 14.6514$, $\alpha_{r} = 16.0415$ and $f_{l} = 24.8{\kern 1pt} {\text{mm}}$, $f_{r} = 25.3{\kern 1pt} {\text{mm}}$. The subscripts l and r denote the left and right station, respectively.

3.3 Reasonable Value of the Focal Length

Though values of $\alpha$ and β have been known, it’s essential to find suitable ($X_{w}$, $Z_{w}$) to find a more reasonable value of the focal length in the experimental scene. When a target or a checkpoint is in one place, it constitutes a straight line with a station on the plane of the world coordinate system. In order to obtain a more accurate value of $f$ to calculate object coordinates, it’s indispensable to find some proper points to calculate it in the line. This requires us to find corresponding relation between $X_{w}$ and the focal length according to the expression (4) above once an equation is determined. Additionally, left station and target 1, checkpoint 1 and checkpoint 2 constitute three lines as $Z_{w} = - 7X_{w} + 278$, $Z_{w} = - 5X_{w} + 98$ and $Z_{w} = - 2X_{w} - 137$ respectively according to their coordinates in the world coordinate system. The relation between $X_{w }$ and the focal length is in Fig. 4 below.

Fig. 4.

The relation between $X_{w}$ and $f$ (left)

There are many values between 0.0247 m and 0.0249 m if the range of these values and step length can be smaller than above. It’s instructive to select suitable locations to calculate $f$ of cameras based on the relation in Fig. 4 if target 1, checkpoint 1 or checkpoint 2 and the left station constitute a straight line on the plane of the world coordinate system. From the Fig. 4, it can be seen that the relation between left station and the check point 2 changes obviously while the other two lines are steady relatively. It is more easier to calibrate the cameras if the target 1 is selected compared with the checkpoint 1 and 2. In the experimental scene it is hard to select the proper points directly though the lines are determined here. So quick selecting the points according to the grid generated by the linear equation $Z_{w} = - 7X_{w} + 278$ above seems be advisable in Fig. 5 below.

Fig. 5.

The relation between $X_{w}$ and the focal length

Analogously, when the right station and target 1, checkpoint 1 and checkpoint 2 constitute three lines as $Z_{w} = - 2.25X_{w} - 112$, $Z_{w} = - 2.639X_{w} - 71.1732$ and $Z_{w} = 3.9325X_{w} + 41.2750$ respectively according to their coordinates in the world coordinate system, the relation between $X_{w }$ and the focal length is in Fig. 6 below.

Fig. 6.

The relation between $X_{w}$ and $f$ (right)

Similarly, from the Fig. 6, it can be seen that the relation between right station and the target 1 changes obviously while the other two lines are steady relatively. It seems that calibration of the two cameras should choose the target 1 as the proper point to calculate the focal length. In the experimental scene it is also hard to select the proper points directly though the lines are determined for right station. So quick selecting the points according to the grid generated by the linear equation $Z_{w} = 2.25X_{w} - 112$ above seems be advisable in Fig. 7 below.

Fig. 7.

The relation between $X_{w}$ and the focal length

So, it can be included that the focal length and $X_{w}$ constitute a non-linear relationship in a defined area or the focal length remains steady decline or rise as $X_{w}$ changes according to the Figs. 4 and 6. Additionally, the points about the calibration of $f$ should be selected in a line once the first point has been placed somewhere for the accuracy of the focal length, and some proper points are supposed to be found under the premise that some values, for example pixels and elevations of targets, remain unchanged.

4 Conclusion

The experiment shows the correctness and reliability of the calibrated model with the calibration of $\alpha$ and β showing its correctness. The relation between $f$ and ($X_{w}$, $Z_{w}$) has been investigated, and it’s important to calculate ($X_{w}$, $Z_{w}$) if we can narrow the difference between the reference value and $f$ calibrated. It can be concluded that selecting suitable locations can improve the accuracy of objects on the plane of the world coordinate system. Therefore, the method related can be instructive for calibration based on binocular vision measurement in the paper, and the cameras will have a fast calibration about its parameters.