Suboptimal Solutions to the Algebraic-Error Line Triangulation

Abstract

Line triangulation, a foundational problem in computer vision, is to estimate the 3D line position from a set of measured image lines with known camera projection matrices. Aiming to improve the triangulation’s efficiency, in this work, two algorithms are proposed to find suboptimal solutions under the algebraic-error optimality criterion of the Plücker line coordinates. In these proposed algorithms, the algebraic-error optimality criterion is reformulated by the transformation of the Klein constraint. By relaxing the quadratic unit norm constraint to six linear constraints, six new single-quadric-constraint optimality criteria are constructed in the new formulation, whose optimal solutions can be obtained by solving polynomial equations. Moreover, we prove that the minimum algebraic error of either the first three or the last three of the six new criteria is not more than $\sqrt{3}$ times of that of the original algebraic-error optimality criterion. Thus, with three new criteria and all the six criteria, suboptimal solutions under the algebraic error minimization and the geometric error minimization are obtained. Experimental results show the effectiveness of our proposed algorithms.

Appendices

Appendix A

For

$$\mathrm{M} ( \lambda ) = \left( \begin{array}{c@{\quad}c} \varSigma '_{1} + \lambda\mathrm{I}_{2} & \mathrm{B}_{2} \\ \mathrm{B}_{2}^{T} & \varSigma_{2} - \lambda\mathrm{I}_{3} \end{array} \right), $$

where λ is unknown, its determinant can be expanded as:

$$\begin{aligned} \det(\mathrm{M}) =& - \det\bigl(\left( \begin{array}{c@{\quad}c} \varSigma'_{1} & - \mathrm{B}_{2} \\ \mathrm{B}_{2}^{T} & - \varSigma_{2} \end{array} \right) + \lambda \mathrm{I}_{n}\bigr) \\ =& - \lambda^{n} - \sum_{i = 1}^{n} \left[ \left( \begin{array}{c@{\quad}c} \varSigma'_{1} & - \mathrm{B}_{2} \\ \mathrm{B}_{2}^{T} & - \varSigma_{2} \end{array} \right) \right]_{i} \lambda^{n - i} \end{aligned}$$

(19)

where [⋅] _i is the sum of all the i×i principal minors, I _n is the n×n identity matrix and n is the order of M. Then, from (17) we know that just the last three columns of the adjoint matrix of M are needed. So, here we just discuss the cofactors of the last three rows in M and there are three situations:

(1) Elements on the main diagonal: their cofactors can be obtained according to (19).

(2) Nonzero elements at the ith row and the jth column (i≠j): their cofactors are the determinants of matrices like the following form, where the matrices are obtained by row or column exchanges of the submatrices formed by deleting the ith row and the jth column:

$$\begin{aligned} &{\left\vert \begin{array}{c@{\quad}c@{\quad}c@{\quad}c} 0 & h_{12} & h_{13} & h_{14} \\ p + \lambda& h_{22} & h_{23} & h_{24} \\ h_{31} & 0 & q_{1} - \lambda& 0 \\ h_{41} & 0 & 0 & q_{2} - \lambda \end{array} \right\vert} \\ &{\quad = - h_{12}\lambda^{3} - h_{12}(p - q_{1} - q_{2})\lambda^{2}- (h_{12}h_{23}h_{31}} \\ &{\qquad {} + h_{12}h_{24}h_{41} + h_{12}q_{1}q_{2} - h_{13}h_{22}h_{31} - h_{14}h_{22}h_{41}} \\ &{\qquad {}- h_{12}pq_{1} - h_{12}pq_{2})\lambda+ h_{12}h_{24}h_{41}q_{1} + h_{12}h_{23}h_{31}q_{2}} \\ &{\qquad {}- h_{14}h_{22}h_{41}q_{1} - h_{13}h_{22}h_{31}q_{2} - h_{12}pq_{1}q_{2}} \end{aligned}$$

(3) Zero elements at the ith row and the jth column (i≠j): their cofactors are the negative determinants of matrices like the following form, where the matrices are obtained by row or column exchanges of the submatices formed by deleting the ith row and the jth column:

$$\begin{aligned} &{\left\vert \begin{array}{c@{\quad}c@{\quad}c@{\quad}c} p_{1} + \lambda& 0 & h_{13} & h_{14} \\ 0 & p_{2} + \lambda& h_{23} & h_{24} \\ h_{31} & h_{32} & 0 & 0 \\ h_{41} & h_{42} & 0 & q - \lambda \end{array} \right\vert} \\ &{\quad = ( h_{13}h_{31} + h_{23}h_{32} )\lambda^{2}} \\ &{\qquad {}+ ( h_{23} h_{32}p_{1} + h_{13}h_{31}p_{2} - h_{13}h_{31}q - h_{23} h_{32}q )\lambda} \\ &{\qquad {}+ h_{14}h_{23}h_{32}h_{41} + h_{13}h_{24}h_{31}h_{42} - h_{13}h_{24}h_{32}h_{41}} \\ &{\qquad {}- h_{14}h_{23}h_{31}h_{42} - h_{23} h_{32}p_{1}q - h_{13}h_{31}p_{2}q} \end{aligned}$$

Appendix B

If the first m _i elements in are zero, we discuss how to compute the candidates of $( \hat{\mathbf{L}}_{1,2}^{T},\hat{\mathbf{L}}_{1,3}^{T} )^{T}$ from the following two situations.

2.1 B.1 Situation m _i =1

$$ \left( \begin{array}{c} \hat{\mathbf{L}}_{1,2} \\ \hat{\mathbf{L}}_{1,3} \end{array} \right) = \mathrm{U}_{i}\left( \begin{array}{c} \mu_{i} \\ \mathbf{v}_{i} \end{array} \right) $$

(20)

where μ _i is an unknown, is a constant vector. Substituting (20) into the second equation in (15), we obtain a quadratic equation. The candidates of $( \hat{\mathbf{L}}_{1,2}^{T},\hat{\mathbf{L}}_{1,3}^{T} )^{T}$ can be obtained by real roots of the quadratic equation.

2.2 B.2 Situation m _i >1

$$ \left( \begin{array}{c} \hat{\mathbf{L}}_{1,2} \\ \hat{\mathbf{L}}_{1,3} \end{array} \right) = \mathrm{U}_{i}\left( \begin{array}{c} \boldsymbol{\mu}_{i} \\ \mathbf{v}_{i} \end{array} \right) $$

(21)

where μ _i is an m _i -dimensional unknown vector, is a constant vector when m _i ≤3 or a constant when m _i =4. Substituting (21) into the second equation in (15), a quadratic equation of m _i variants is obtained, which has infinitely many solutions. To reduce the number of the candidates of $( \hat{\mathbf{L}}_{1,2}^{T},\hat{\mathbf{L}}_{1,3}^{T} )^{T}$ from the quadratic equation, we select the candidate that gives the smallest value of the cost function in (13), which is equivalent to the optimal solution to the minimization problem of m _i -vector μ _i constructed by substituting (21) into the criterion (13)

$$\begin{array}{l@{\quad}l} \displaystyle \min_{\boldsymbol{\mu}_{i}} & ( \boldsymbol{\mu}_{i}^{T}, \mathbf{v}_{i}^{T},1 )\mathrm{D}\left( \begin{array}{c} \boldsymbol{\mu}_{i} \\ \mathbf{v}_{i} \\ 1 \end{array} \right) \\ \displaystyle \mbox{s.t. } & \bigl( \boldsymbol{\mu}_{i}^{T}, \mathbf{v}_{i}^{T},1 \bigr)\mathrm{S}\left( \begin{array}{c} \boldsymbol{\mu}_{i} \\ \mathbf{v}_{i} \\ 1 \end{array} \right) = 0 \end{array} $$

or equivalently

$$ \begin{array}{l@{\quad}l} \min & \boldsymbol{\mu}_{i}^{T}\mathrm{D}_{11}\boldsymbol{\mu}_{i} + 2\mathbf{d}_{1}\boldsymbol{\mu}_{i} + d_{2} \\ \mbox{s.t. }& \boldsymbol{\mu}_{i}^{T}\mathrm{S}_{11}\boldsymbol{\mu}_{i} + 2\mathbf{s}_{1}\boldsymbol{\mu}_{i} + s_{2} = 0 \end{array} $$

(22)

where

$$\begin{aligned} &{\mathrm{D} = \left( \begin{array}{c@{\quad}c} \mathrm{U}_{i}^{T} & 0 \\ 0 & 1 \end{array} \right)\left( \begin{array}{c@{\quad}c@{\quad}c} \varSigma'_{1} & \mathrm{B}_{2} & 0 \\ \mathrm{B}_{2}^{T} & \varSigma_{2} & \mathbf{B}_{1}^{T} \\ 0 & \mathbf{B}_{1} & \sigma_{1} \end{array} \right)\left( \begin{array}{c@{\quad}c} \mathrm{U}_{i} & 0 \\ 0 & 1 \end{array} \right),} \\ &{\mathrm{S} = \left( \begin{array}{c@{\quad}c} \mathrm{U}_{i}^{T} & 0 \\ 0 & 1 \end{array} \right)\left( \begin{array}{c@{\quad}c@{\quad}c} \mathrm{I}_{2} & 0 & 0 \\ 0 & - \mathrm{I}_{3} & 0 \\ 0 & 0 & 1 \end{array} \right)\left( \begin{array}{c@{\quad}c} \mathrm{U}_{i} & 0 \\ 0 & 1 \end{array} \right),} \end{aligned}$$

$\mathbf{d}_{1} = ( \mathbf{v}_{i}^{T},1 )\mathrm{D}_{12}^{T}$, , $\mathbf{s}_{1} = ( \mathbf{v}_{i}^{T},1 )\mathrm{S}_{12}^{T}$, and S₁₁ are the upper-left m _i ×m _i submatrix of D and S,D₁₂ and S₁₂ are the upper-right m _i ×(6−m _i ) submatrices of D and S,D₂₂ and S₂₂ are the lower-right (6−m _i )×(6−m _i ) submatrices of D and S, respectively.

By Lagrange’s multiplier method, we obtain the following Lagrange’s equations according to (22):

$$ \left\{ \begin{array}{l} ( \mathrm{D}_{11} + \gamma\mathrm{S}_{11} )\boldsymbol{\mu}_{i} + \mathbf{d}_{1} + \gamma\mathbf{s}_{1} = 0 \\ \boldsymbol{\mu}_{i}^{T}\mathrm{S}_{11}\boldsymbol{\mu}_{i} + 2\mathbf{s}_{1}\boldsymbol{\mu}_{i} + s_{2} = 0 \end{array} \right. $$

(23)

where γ is a multiplier. Here, three cases need to be considered respectively and the optimal solution is selected from the candidates of μ _i in these three cases.

Case A::

det(D₁₁+γS₁₁)≠0

According to the first equation in (23), we obtain

$$ \boldsymbol{\mu}_{i} = - \frac{ ( \mathrm{D}_{11} + \gamma \mathrm{S}_{11} )^{ *} ( \mathbf{d}_{1} + \gamma\mathbf{s}_{1} )}{\det ( \mathrm{D}_{11} + \gamma\mathrm{S}_{11} )} $$

(24)

where (D₁₁+γS₁₁)^∗ is the adjoint matrix of D₁₁+γS₁₁. Substituting (24) into the second equation of (23), an equation of degree 2m _i (2≤m _i ≤4) is obtained

$$ \begin{aligned}[b] &( \mathbf{d}_{1} + \gamma\mathbf{s}_{1} )^{T} ( \mathrm{D}_{11} + \gamma\mathrm{S}_{11} )^{ * T}\mathrm{S}_{11} ( \mathrm{D}_{11} + \gamma\mathrm{S}_{11} )^{ *} ( \mathbf{d}_{1} + \gamma\mathbf{s}_{1} ) \\ &\quad {}- 2\det ( \mathrm{D}_{11} + \gamma\mathrm{S}_{11} )\mathbf{s}_{1} ( \mathrm{D}_{11} + \gamma\mathrm{S}_{11} )^{ *} ( \mathbf{d}_{1} + \gamma\mathbf{s}_{1} ) \\ &\quad {}+ s_{2}\det ( \mathrm{D}_{11} + \gamma\mathrm{S}_{11} )^{2} = 0 \end{aligned} $$

(25)

Because the order of the square matrix D₁₁+γS₁₁ is not more than 4, its determinant and adjoint matrix can be computed easily. By choosing real roots of (25) which do not make det(D₁₁+γS₁₁)=0 and substituting them into (24), we can obtain the candidates of μ _i .

Case B::

A root γ′ of det(D₁₁+γS₁₁)=0 makes the first equation in (23) solvable and only one eigenvalue of matrix D₁₁+γ′S₁₁ is zero.

Referring to Situation m _i =1, by the eigen-decomposition of D₁₁+γ′S₁₁, an equation like (20) is obtained. Substituting the equation into the second equation of (23), if there exists real roots, like Situation m _i =1 the candidates of μ _i can be obtained with the real roots.

Case C::

A root γ″ of det(D₁₁+γS₁₁)=0 makes the first equation in (23) solvable and more than one eigenvalue of the matrix D₁₁+γ″S₁₁ are zero.

The problem returns to Situation m _i >1. By the eigen-decomposition of D₁₁+γ″S₁₁, an equation like (21) is obtained, then a minimization problem like (22) is constructed, which can be solved just like Case A, Case B and Case C. Because the Case C is similar to Situation m _i >1, the details to obtain the candidates of μ _i will be skipped here.