# On Some Properties of Calibrated Trifocal Tensors

## Abstract

In two-view geometry, the essential matrix describes the relative position and orientation of two calibrated images. In three views, a similar role is assigned to the calibrated trifocal tensor. It is a particular case of the (uncalibrated) trifocal tensor, and thus it inherits all its properties but, due to the fewer degrees of freedom, satisfies a number of additional algebraic constraints. Some of them are described in this paper. More specifically, we define a new notion—the trifocal essential matrix. On the one hand, it is a generalization of the ordinary (bifocal) essential matrix, while, on the other hand, it is closely related to the calibrated trifocal tensor. We prove the two necessary and sufficient conditions that characterize the set of trifocal essential matrices. Based on these characterizations, we propose three necessary conditions on a calibrated trifocal tensor. They have the form of 15 quartic and 99 quintic polynomial equations. We show that in the practically significant real case the 15 quartic constraints are also sufficient.

### Keywords

Multiple view geometry Calibrated trifocal tensor Trifocal essential matrix Quartic and quintic constraints on calibrated trifocal tensor## 1 Introduction

In multiview geometry, the fundamental matrix and the trifocal tensor describe the relative orientation of two and three (uncalibrated) images respectively. If the cameras are calibrated, i.e., we are given the calibration matrices for each view, the fundamental matrix is transformed to the so-called essential matrix. It was first introduced by Longuet-Higgins in [14]. The essential matrix has fewer degrees of freedom and additional algebraic properties, compared to the fundamental matrix. A detailed investigation of these properties is given by Demazure, Faugeras, Maybank and other researchers in [6, 7, 12, 13, 15]. We shortly recall the most important of them in the next section.

The object of study of the present paper is the trifocal tensor for calibrated cameras. We call this entity the calibrated trifocal tensor (CTFT for short), although some authors use the term Euclidean trifocal tensor. It first appeared as a tool of scene reconstruction from line correspondences in the papers by Spetsakis and Aloimonos [23] and Weng et al. [27]. Later, Hartley [10] generalized the trifocal tensor for uncalibrated cameras and showed that it can be applied for projective scene reconstruction from different combinations of point and line correspondences. Afterward, the properties and applications of (uncalibrated) trifocal tensors have been intensively investigated by Hartley, Shashua, Triggs, and many other researchers in [3, 11, 21, 22, 25, 26].

One of the most important problems regarding multifocal tensors is their algebraic characterization, i.e., a set of necessary and sufficient conditions under which an arbitrary tensor of suitable order becomes the multifocal tensor. The conditions are usually given in a form of polynomial equations. While the fundamental matrix is completely characterized by the single equation—its determinant must be zero—the set of essential matrices is characterized by the nine cubic constraints, cf. [6, 15] or Eq. (7) below. The significance of this result lies in its practical applications. A few efficient algorithms for some metric structure-from-motion problems have been developed based on the characterization of essential matrices; see, e.g., [16, 24].

The investigation of constraints characterizing *uncalibrated* trifocal tensors has received considerable attention in the last two decades: see [4, 17, 20] and recent works [1, 2, 18] for details. We would like to mention that the structure of the set of trifocal tensors is much more complicated compared to the bifocal case. In particular, the characterizations from [2] and [1] consist of 36 and 2071 (!) polynomials, respectively.

On the other hand, constraints on trifocal tensors specific to the calibrated case have not previously been studied in the published literature. The present paper is a first step in this direction. Its main contribution is the characterization of the set of *real* CTFTs. Namely, we show that the necessary and sufficient conditions for a real trifocal tensor to be calibrated consist of 15 quartic polynomial equations. Most of them arise from the study of certain complex \(3\times 3\) matrix associated with a CTFT. Alternatively, we propose the characterization consisting of 99 quintic polynomial equations.

The rest of the paper is organized as follows. In Sect. 2, we recall some definitions and results from multiview geometry. In Sect. 3, we introduce a new notion—the trifocal essential matrix. On the one hand, it is a generalization of the ordinary (bifocal) essential matrix, and, on the other hand, it is closely related to the calibrated trifocal tensor. We prove the two necessary and sufficient conditions that characterize the set of trifocal essential matrices. In Sect. 4, we define the trifocal essential matrix associated with a CTFT and, based on the characterizations from Sect. 3, we propose our three necessary conditions. They have the form of 15 quartic and 99 quintic homogeneous polynomial equations in the entries of a CTFT. In Sect. 5, we show that in the practically significant real case the 15 quartic constraints are also sufficient. Combining this result with the existing characterization of trifocal tensors, we give a complete set of constraints characterizing a real CTFT. In Sect. 6, we discuss the paper’s results and propose further directions of research. In Appendix, we have included some auxiliary lemmas used in the paper.

## 2 Preliminaries

### 2.1 Notation

We preferably use \(\alpha , \beta , \ldots \) for scalars, \(a, b, \ldots \) for column 3-vectors or polynomials, and \(A, B, \ldots \) both for matrices and column 4-vectors. For a matrix *A*, the entries are \((A)_{ij}\), the transpose is \(A^{{\mathrm {T}}}\), the determinant is \(\det A\), and the trace is \({{\mathrm{tr}}}A\). For two 3-vectors *a* and *b* the cross product is \(a\times b\). For a vector *a* the entries are \((a)_i\), the notation \([a]_\times \) stands for the skew-symmetric matrix such that \([a]_\times b = a \times b\) for any vector *b*. We use *I* for the identity matrix.

The group of \(n\times n\) matrices subject to \(R^{{\mathrm {T}}}R = I\) and \(\det R = 1\) is denoted by \({\mathrm {SO}}(n)\) in case *R* is real and \({{\mathrm {SO}}}(n, {\mathbb {C}})\) if *R* is allowed to have complex entries.

### 2.2 Pinhole Cameras

We briefly recall some definitions and results from multiview geometry; see [7, 8, 11, 15] for details.

A *pinhole camera* is a triple \((O, \Pi , P)\), where \(\Pi \) is the image plane, *P* is a central projection of points in three-dimensional Euclidean space onto \(\Pi \), and \(O \not \in \Pi \) is the camera center (center of projection *P*).

*Q*be a point in 3-space represented in homogeneous coordinates as a 4-vector, and

*q*be its image in \(\Pi \) represented as a 3-vector. Projection

*P*is then given by a \(3\times 4\) homogeneous matrix, which is called the

*camera matrix*and is also denoted by

*P*. We have

*P*.

The *focal length* is the distance between *O* and \(\Pi \), the orthogonal projection of *O* onto \(\Pi \) is called the *principal point*. All intrinsic parameters of a camera (such as the focal length and the principal point offsets) are combined into a single upper-triangular matrix, which is called the *calibration matrix*. A camera is called *calibrated* if its calibration matrix is known.

*rotation matrix*and

*translation vector*, respectively.

### 2.3 Two-View Case

*A*is a \(3\times 3\) matrix and

*a*is a 3-vector. Let

*Q*be a point in 3-space, and \(q_k\) be its

*k*th image. Then,

*incidence relation*for a pair \((q_1, q_2)\) says

*fundamental matrix*. It is important that relation (1) is linear in the entries of

*F*, so that given a number of point correspondences (eight or more), one can estimate the entries of

*F*by solving a linear system.

It follows from the definition of matrix *F* that \(\det F = 0\). One easily verifies that this condition is also sufficient. Thus we have

### Theorem 1

*F*is a fundamental matrix if and only if

*essential matrix*

*E*is the fundamental matrix for calibrated cameras \({\hat{P}}_1 = \begin{bmatrix} I&\quad 0 \end{bmatrix}\) and \({\hat{P}}_2 = \begin{bmatrix} R&\quad t\end{bmatrix}\), where \(R \in {\mathrm {SO}}(3)\),

*t*is a 3-vector, that is

*F*and

*E*are related by

*k*th camera. It follows that the incidence relation (1) for the essential matrix becomes

*normalized coordinates*.

Equality (3) can be thought of as the definition of the essential matrix, i.e., it is a \(3\times 3\) nonzero skew-symmetric matrix post-multiplied by a special orthogonal matrix. Moreover, we can even consider complex essential matrices assuming that in (3) vector \(t \in {\mathbb {C}}^3\) and matrix \(R \in {\mathrm {SO}}(3, {\mathbb {C}})\).

The real fundamental matrix has seven degrees of freedom, whereas the real essential matrix has only five degrees of freedom. It is translated into the following property [7, 11, 13]: Two of the singular values of matrix *E* are equal, and the third is zero. The condition is also sufficient. An equivalent form of this result is given by

### Theorem 2

It should be mentioned here that constraints (5) and (6) characterize only *real* essential matrices. There exist “non-essential” complex \(3\times 3\) matrices which nevertheless satisfy both conditions (5) and (6). The most general form of such matrices will be given in the next section.

The following theorem gives another characterization constraint on the entries of essential matrix *E*. It is also valid in case of complex *E*.

### Theorem 3

We note that there are “non-essential” rank one matrices which satisfy (7). However, it can be shown that all of them are limits of sequences of essential matrices [15]. Thus the closure of the set of essential matrices constitutes an algebraic variety generated by (7).

It is interesting to note that Theorem 3 is a key for developing efficient algorithms of the essential matrix estimation from five-point correspondences in two views [16].

### 2.4 Three-View Case

A (2, 1) tensor is a valency-3 tensor with two contravariant and one covariant indices. For a (2, 1) tensor *T*, we write \(T = \begin{bmatrix}T_1&\quad T_2&\quad T_3\end{bmatrix}\), where \(T_k\) are \(3\times 3\) matrices corresponding to the covariant index.

*A*and

*B*are \(3\times 3\) matrices,

*a*and

*b*are 3-vectors. The

*trifocal tensor*\(T = \begin{bmatrix}T_1&\quad T_2&\quad T_3\end{bmatrix}\) is a (2, 1) tensor defined by

*T*matrices \(T_k\) are called the

*correlation slices*.

*epipolar constraints*hold [11, 17]:

*extended rank constraint*. It is equivalent to ten (cubic in the entries of \(T_1, T_2, T_3\)) equations each of which is the coefficient in \(\alpha ^i \beta ^j \gamma ^k\) with \(i + j + k = 3\).

### Theorem 4

([9, 17]) Let \(T = \begin{bmatrix}T_1&\quad T_2&\quad T_3\end{bmatrix}\) be a real (2, 1) tensor such that \({{\mathrm{rank}}}T_k = 2\), \(k = 1, 2, 3\). Let *T* satisfy the two epipolar (9) and ten extended rank (10) constraints. Let the ranks of matrices \(\begin{bmatrix}l_1&\quad l_2&\quad l_3\end{bmatrix}\) and \(\begin{bmatrix}r_1&\quad r_2&\quad r_3\end{bmatrix}\) equal two. Then *T* is a trifocal tensor.

### Remark 1

Also, it should be mentioned that the characterization from Theorem 4 is not unique. There exist other sets of constraints characterizing trifocal tensors, cf. [1, 2, 4, 18, 20].

*k*th image of a point

*Q*in 3-space. The

*trifocal incidence relation*for a triple \((q_1, q_2, q_3)\) says [11]

*T*. It consists of nine scalar equations, but only four of them are linearly independent.

*calibrated trifocal tensor*(CTFT) \({\hat{T}}\) is the trifocal tensor for calibrated cameras \(P_1 = \begin{bmatrix} I&\quad 0 \end{bmatrix}\), \(P_2 = \begin{bmatrix} R_2&\quad t_2 \end{bmatrix}\) and \(P_3 = \begin{bmatrix} R_3&\quad t_3 \end{bmatrix}\), where \(R_2, R_3 \in {\mathrm {SO}}(3)\), \(t_2, t_3 \in {\mathbb {R}}^3\), i.e.,

*T*and \({\hat{T}}\) are related by

*k*th camera.

*M*and 3-vector

*t*, the following identity holds:

The tensors *T* and \({\hat{T}}\) have 18 and 11 degrees of freedom, respectively. It follows that matrices \({\hat{T}}_k\) must satisfy a number of additional algebraic constraints. Some of them are described in Sect. 4 below.

## 3 The Trifocal Essential Matrix and Its Characterization

*trifocal essential matrix*is, by definition, a \(3\times 3\) matrix

*S*which can be represented in form

*S*, \(S^{{\mathrm {T}}}\) and

*RSQ*, where \(R, Q \in {\mathrm {SO}}(3, {\mathbb {C}})\), simultaneously are (or are not) the trifocal essential matrices.

### Theorem 5

Let a \(3\times 3\) matrix *S* be a trifocal essential matrix. Then \(SS^{{\mathrm {T}}}\) has one zero and two other equal eigenvalues.

### Proof

*S*be a trifocal essential matrix, i.e., it can be represented in form (14). The matrix \(SS^{{\mathrm {T}}}\) has zero eigenvalue, as \(\det S = 0\). Taking into account that \(s_2^{{\mathrm {T}}}s_2 = 0\), we get

### Theorem 6

*S*is a trifocal essential matrix if and only if

### Proof

The“only if” part is due to Theorem 5 and Lemma 8 (see the Appendix). To prove the “if” part, let *S* be a \(3\times 3\) matrix satisfying (16), (17). We denote \(c_k\) the *k*th column of matrix *S*. Because *S* is degenerate, there exists a nonzero vector *a* such that \(Sa = 0\). There are two possibilities.

**Case 1:**\(a^{{\mathrm {T}}}a \ne 0\). Scaling *a* and post-multiplying *S* by an appropriate matrix from \({\mathrm {SO}}(3, {\mathbb {C}})\), we assume without loss of generality that \(a = \begin{bmatrix}0&\quad 0&\quad 1\end{bmatrix}^{{\mathrm {T}}}\). Therefore \(c_3 = 0\).

*S*by an appropriate rotation, we obtain

*S*into (17) gives

*S*by an appropriate rotation to get

*S*into (17) yields

*S*has rank one and

*s*is an arbitrary 3-vector satisfying \(s^{{\mathrm {T}}} s = 0\). Thus in either case

*S*is a trifocal essential matrix, as required.

**Case 2:**\(a^{{\mathrm {T}}}a = 0\). Due to Lemma 9, we can post-multiply *S* by an appropriate matrix from \({\mathrm {SO}}(3, {\mathbb {C}})\) and suppose without loss of generality that \(a = \begin{bmatrix}0&\quad 1&\quad i\end{bmatrix}^{{\mathrm {T}}}\). Therefore \(c_3 = ic_2\).

*S*is a trifocal essential matrix. Theorem 6 is proved. \(\square \)

We notice that constraints (16), (17) coincide with constraints (5), (6) from Theorem 2. Hence, if a trifocal essential matrix is real, then it is an essential matrix.

### Theorem 7

*S*is a trifocal essential matrix if and only if

### Proof

*M*, the following identity holds:

*M*.

Let matrix *S* be a trifocal essential matrix. By Theorem 6, \(\det S = \varphi (S) = 0\). Then it follows from (19) that \(\Phi (S) = 0_{3\times 3}\), i.e., (18) holds. The “only if” part is proved.

*S*satisfy (18), i.e., \(\Phi (S) = 0_{3\times 3}\). It suffices to show that \(\det S = 0\). Suppose, by hypothesis, that \(\det S \ne 0\). Then, post-multiplying (18) by \(S^{-1}\), we get

*S*is a trifocal essential matrix. Theorem 7 is proved. \(\square \)

The pair of scalar constraints (16), (17) of degrees 3 and 4 respectively is equivalent to the single matrix constraint (18) of degree 5.

The most general form of a \(3\times 3\) matrix satisfying Eqs. (16) and (17) is the trifocal essential matrix given by (14).

If a trifocal essential matrix is real, then it is an essential matrix.

Every essential matrix is a trifocal essential matrix, but the converse is not true in general.

## 4 Three Necessary Conditions on a Calibrated Trifocal Tensor

A new notion of trifocal essential matrix, introduced in the previous section, turns out to be closely related to calibrated trifocal tensors. The connection is established by the following lemma.

### Lemma 1

Let \({\hat{T}} = \begin{bmatrix}{\hat{T}}_1&{\hat{T}}_2&{\hat{T}}_3\end{bmatrix}\) be a CTFT. Then a \(3\times 3\) matrix \(S_{{\hat{T}}} = \alpha {\hat{T}}_1 + \beta {\hat{T}}_2 + \gamma {\hat{T}}_3\), where numbers \(\alpha , \beta , \gamma \) are such that \(\alpha ^2 + \beta ^2 + \gamma ^2 = 0\), is a trifocal essential matrix, i.e., it can be represented in form (14).

### Proof

*trifocal essential matrix associated with a CTFT*\({\hat{T}} = \begin{bmatrix}{\hat{T}}_1&{\hat{T}}_2&{\hat{T}}_3\end{bmatrix}\) is defined by

We are going to apply the two characterizations from Sect. 3 to derive the necessary conditions on the entries of CTFT \({\hat{T}}\).

In the rest of this section, calibrated trifocal tensors are allowed to have complex entries, that is in (12) matrices \(R_2, R_3\) belong to \({\mathrm {SO}}(3, {\mathbb {C}})\), and vectors \(t_2, t_3\) are in \({\mathbb {C}}^3\).

### Theorem 8

### Proof

Finally, the resulting nine polynomials can not be linearly dependent, since each of them contains monomials that are not contained in all the other polynomials. Examples of such monomials for (22), (23) and (24) are \(({\hat{T}}_3)^2_{11}({\hat{T}}_1)^2_{11}\), \(({\hat{T}}_3)^2_{11}({\hat{T}}_1)_{11}({\hat{T}}_2)_{11}\) and \(({\hat{T}}_1)^3_{11}({\hat{T}}_2)_{11}\) respectively. Theorem 8 is proved.

From now on, the nine equalities from Theorem 8 will be referred to as the *eigenvalue constraints*.

### Lemma 2

Let \(T = \begin{bmatrix}T_1&T_2&T_3\end{bmatrix}\) be a (2, 1) tensor satisfying the ten extended rank and nine eigenvalue constrains. Then matrix \(S_T = \alpha T_1 + \beta T_2 + \gamma T_3\) with \(\alpha ^2 + \beta ^2 + \gamma ^2 = 0\) is a trifocal essential matrix.

### Proof

### Theorem 9

(Second necessary condition) Let \({\hat{T}} = \begin{bmatrix}{\hat{T}}_1&{\hat{T}}_2&{\hat{T}}_3\end{bmatrix}\) be a CTFT. Then the entries of \({\hat{T}}\) are constrained by the 99 linearly independent quintic (of degree 5) polynomial equations.

### Proof

### Remark 2

*k*th version of the L.H.S. of (32). Then we have

### Theorem 10

A (2, 1) tensor \(T = \begin{bmatrix}T_1&T_2&T_3\end{bmatrix}\) satisfies the ten extended rank and nine eigenvalue constrains if and only if it satisfies the 99 quintic constraints from Theorem 9.

### Proof

The result immediately follows from Lemma 2 and the two characterizations of trifocal essential matrices (Theorems 6 and 7). \(\square \)

Theorem 10 implies the equivalence of the two sets of constraints. From this point of view the quintic polynomials contribute nothing new and seem to be redundant. Nevertheless, we expect them to be useful in deriving more (quintic) polynomial constraints on CTFTs. We return to this question in Sect. 6.

Finally, we propose the third necessary condition on a CTFT. It seems not to be directly related to the matrix \(S_{{\hat{T}}}\). However, this condition could be useful in applications, since it consists of another set of quartic polynomial equations that are satisfied by a calibrated trifocal tensor.

### Theorem 11

### Proof

Let tensor \({\hat{T}}\) be represented in form (12). First we replace \({\hat{T}}\) with \({\hat{T}}' = \begin{bmatrix}R_2^{{\mathrm {T}}}{\hat{T}}_1R_3&R_2^{{\mathrm {T}}}{\hat{T}}_2R_3&R_2^{{\mathrm {T}}}{\hat{T}}_3R_3\end{bmatrix}\). Then the correlation slices of \({\hat{T}}'\) are simplified to \({\hat{T}}'_k = e_kt_3^{{\mathrm {T}}} - t_2e_k^{{\mathrm {T}}}\). A straightforward computation proves that \({\hat{T}}'\) satisfies eqs. (34) – (35) and the four their consequences. Then so does \({\hat{T}}\), since \(U_k = R_2 U'_k R_2^{{\mathrm {T}}}\) and \(V_k = R_2 V'_k R_2^{{\mathrm {T}}}\).

The resulting six polynomials can not be linearly dependent, as each of them contains monomials that are not contained in all the other polynomials. Examples of such monomials for (34) and (35) are \(({\hat{T}}_3)^2_{11}({\hat{T}}_1)^2_{11}\) and \(({\hat{T}}_2)_{11}({\hat{T}}_3)^3_{11}\) respectively. Theorem 11 is proved. \(\square \)

The 15 equalities from Theorems 8 and 11 will be further referred to as the *quartic constraints*.

### Remark 3

Let us briefly describe the way by which the six constraints from Theorem 11 were initially derived. The nine eigenvalue polynomial equations consist in total of 2160 monomials. We constructed the dense matrix *A* of size \(2160\times 2160\) such that each row of *A* is the monomial vector taken on a randomly generated CTFT. Using the singular value decomposition of *A*, we found that the null space of *A* has dimension 15. It is meant that in addition to the eigenvalue constraints, there must exist six more quartic equations linearly independent with them. The (small integer) coefficients of these equations were derived by the Gauss–Jordan elimination on the basis vectors of the null space of *A*.

### Remark 4

*T*satisfies the eigenvalue constraints, but not the six ones from Theorem 11. Hence

*T*is not calibrated.

### Remark 5

*T*to be calibrated. Here is a counterexample. Consider a (2, 1) tensor

*T*satisfies all the 15 quartic constraints. Suppose that

*T*is calibrated. Then there must exist 3-vectors \(u_k, v_k, t_2, t_3\) such that \(u_k^{{\mathrm {T}}} u_k = v_k^{{\mathrm {T}}} v_k = 1\) and

*J*that \(1 \in J\). It follows that the affine variety defined by

*J*is empty, and thus

*T*cannot be calibrated.

## 5 A Characterization of Real Calibrated Trifocal Tensors

In this section, we are going to obtain a three-view analog of Theorem 2. First, it will be shown that a real trifocal tensor is calibrated if and only if it satisfies the 15 quartic constraints (Theorem 12). After that, it is straightforward to combine this result with the characterization of trifocal tensors, e.g., from [2] to get a complete set of constraints characterizing a real CTFT (Theorem 13).

First we prove several lemmas.

### Lemma 3

*T*and \(T'\) simultaneously

- 1.
are (or are not) calibrated trifocal tensors;

- 2.
satisfy (or do not satisfy) the 10 extended rank and 15 quartic constraints.

### Proof

*T*be a CTFT, so that its correlation slices can be represented in form (12). Then,

*T*is not a CTFT, then so is not \(T'\), since

*T*be a (2, 1) tensor satisfying the 10 extended rank and 15 quartic constraints. Let us construct the matrix \(S_T(s) = \sum \limits _{k = 1}^3(s)_k T_k\), where

*s*is an arbitrary 3-vector. The ten extended rank constraints are then equivalent to \(\det S_T(s) = 0\). We get

Further, if the vector *s* is such that \(s^{{\mathrm {T}}}s = 0\), then Lemma 2 implies that \(S_T(s)\) is a trifocal essential matrix. It follows from (39) that so is \(S_{T'}(s)\). After that, using the same arguments as in the proof of Theorem 8, one shows that tensor \(T'\) satisfies the nine eigenvalue constraints.

*k*th quartic equation on tensor

*T*so that \(p_{10}, \ldots , p_{15}\) are the six polynomials from Theorem 11. Then, by a straightforward computation, we get

*k*, then also \(p_j(T') = 0\) for all

*j*.

Finally, if *T* does not satisfy the extended rank and quartic equations, then due to (38) so does not \(T'\). This completes the proof of Lemma 3. \(\square \)

### Lemma 4

*T*can be transformed by (37) to the trifocal tensor

### Proof

*T*is transformed to \(T'\) by (37). Since

*T*is a trifocal tensor, we have

*T*is transformed to (41) by the rotations \(Q_1\), \(Q_2\) and \(Q_3\) defined in (42). Lemma 4 is proved. \(\square \)

*J*. We are going to derive several polynomials that belong to \(\sqrt{J}\). For convenience we divide them into two parts presented in Lemmas 5 and 6.

### Lemma 5

*J*is defined in (43).

### Proof

Let *p* be any polynomial from (44) to (50). We construct an ideal \(\tilde{J} = J + \langle 1 - \tau p \rangle \subset {\mathbb {C}}[\lambda _1, \ldots , \sigma _3, \tau ]\), where \(\tau \) is a new variable. By direct computation of the Gröbner basis of \(\tilde{J}\), we get \(1 \in \tilde{J}\). Hence, by Lemma 10 (see Appendix), \(p \in \sqrt{J}\). Lemma 5 is proved. \(\square \)

### Remark 6

*graded reverse lexicographic order*[5]:

### Lemma 6

*J*is defined in (43).

### Proof

See the proof of Lemma 5. \(\square \)

### Theorem 12

A real trifocal tensor \(T = \begin{bmatrix}T_1&T_2&T_3\end{bmatrix}\) is calibrated if and only if it satisfies the 15 quartic constraints from Theorems 8 and 11.

### Proof

The “only if” part is due to Theorems 8 and 11. We now prove the “if” part.

Let \(T = \begin{bmatrix}T_1&T_2&T_3\end{bmatrix}\) be a real trifocal tensor satisfying the 15 quartic equations \(p_1 = \ldots = p_{15} = 0\). By Lemmas 3 and 4, there exist matrices \(Q_1, Q_2, Q_3 \in {\mathrm {SO}}(3)\) such that the trifocal tensor \(T'\) defined by (37) has form (41) and satisfies the 15 quartic equations as well.

**Case 1:**\(\rho _3 \ne 0\). Then, \(\rho _3^2 + \sigma _3^2 \ne 0\) and, by Lemma 5, the entries of tensor \(T'\) are constrained by

*J*and denote the resulting ideal by \(J_1\). By the computation of the Gröbner basis of \(J_1\), we get \((\rho _3\mu _2\sigma _3)^3 \in J_1\). Since \(\rho _3 \ne 0\), it follows that either \(\mu _2 = 0\) or \(\sigma _3 = 0\). In both cases, equalities (56) and (57) imply \(A_3A_3^{{\mathrm {T}}} = \theta ^2 I\) with \(\theta ^2 = \rho _1^2 + \rho _2^2 + \rho _3^2 \ne 0\), and hence tensor \(T'\) is calibrated.

**Case 2:**\(\rho _3 = 0\). By Lemma 6, the entries of tensor \(T'\) are constrained by (we take into account that \(\lambda _1^2 = \mu _2^2\)).

Thus we have shown that the trifocal tensor \(T'\) is calibrated in either case. By Lemma 3, the tensor *T* is calibrated too. Theorem 12 is proved. \(\square \)

Finally, combining Theorem 12 with the characterization of trifocal tensors from [2], we get the following three-view analog of Theorem 2.

### Theorem 13

10 extended rank constraints;

20 equations of degree 9 and 6 of degree 12; see [2, Theorem 2.3];

15 quartic constraints.

## 6 Discussion

- 1.
We have defined a new notion—the trifocal essential matrix. Algebraically, it is a complex \(3\times 3\) matrix associated with a given CTFT \({\hat{T}}\) by the contraction of \({\hat{T}}\) and an arbitrary 3-vector whose squared components sum to zero. In this paper, the trifocal essential matrix plays a technical role. However its deeper investigation should help to explain why its properties are so close to the properties of an ordinary (bifocal) essential matrix. It is also interesting to what extent the trifocal essential matrix can be generalized in case of quadrifocal geometry.

- 2.
Based on the characterization of trifocal essential matrices, we have derived the three necessary conditions on a CTFT (Theorems 8, 9 and 11). They have form of 15 quartic and 99 quintic polynomial equations. It is worth emphasizing again that these constraints relate to the calibrated case only and do not hold for arbitrary trifocal tensors. Moreover, we have shown that the 15 quartic constraints are also sufficient for a real trifocal tensor to be calibrated (Theorem 12). Combining this result with the 36 polynomial equations from [2], we get the characterization of a real CTFT (Theorem 13). On the other hand, some of new 36 constraints may be redundant in the real calibrated case. This question requires more research.

- 3.
As we have seen at the end of Sect. 4, the quartic constraints do not characterize CTFTs over the field of complex numbers. Hence, there must exist other polynomial equations not related to the trifocal essential matrix. In degree 5, we can expect to derive some of them from the 99 quintic constraints using similar arguments as in Remark 3.

- 4.
The characterization of real calibrated trifocal tensors provides a new tool in investigating three-view metric structure-from-motion problems such as relative pose estimation and self-calibration. This could be a promising direction of future research, since we know that similar results for essential matrices have led to the efficient algorithms for analogous two-view problems [16, 24].

The approach could be as follows. The linear incidence relations are first used to parameterize the trifocal tensor. Then the system of 51 polynomial equations is constructed and solved using either Gröbner bases or numerical methods. Theorem 13 guarantees that the true solution (which is always real) is a root of that system. However, a number of spurious complex solutions may occur. For example, the problem of relative pose estimation from four points and one line correspondences in three views is known to have a unique solution in general [19]. We constructed the Gröbner basis for that problem and found that it has 192 (!) spurious complex roots.

To improve the practical applicability of the paper’s results we need a characterization of CTFTs over the field of complex numbers. The present paper provides a basis for that future research.

### References

- 1.Aholt, C., Oeding, L.: The ideal of the trifocal variety. Math. Comp.
**83**, 2553–2574 (2014)MathSciNetCrossRefMATHGoogle Scholar - 2.Alzati, A., Tortora, A.: A geometric approach to the trifocal tensor. J. Math. Imaging Vis.
**38**, 159–170 (2010)MathSciNetCrossRefMATHGoogle Scholar - 3.Armstiong, M., Zisserman, A., Hartley, R.: Self-calibiation from image triplets. In: European Conference on Computer Vision, 3–16. Springer, Berlin (1996)Google Scholar
- 4.Canterakis, N.: A minimal set of constraints for the trifocal tensor. In: European Conference on Computer Vision, 84–99. Springer, Berlin (2000)Google Scholar
- 5.Cox, D., Little, J., O’Shea, D.: Ideals, Varieties, and Algorithms, 2nd edn. Springer, New-York (1997)MATHGoogle Scholar
- 6.Demazure, M.: Sur Deux Problèmes de reconstruction. Technical Report No 882, INRIA (1988)Google Scholar
- 7.Faugeras, O., Maybank, S.: Motion from point matches: multiplicity of solutions. Int. J. Comput. Vision
**4**, 225–246 (1990)CrossRefMATHGoogle Scholar - 8.Faugeras, O.: Three-Dimensional Computer Vision: A Geometric Viewpoint. MIT Press, Cambridge (1993)Google Scholar
- 9.Faugeras, O., Luong, Q.-T.: The Geometry of Multiple Images. MIT Press, Cambridge (2001)MATHGoogle Scholar
- 10.Hartley, R.: Lines and points in three views and the trifocal tensor. Int. J. Comput. Vision
**22**, 125–140 (1997)CrossRefGoogle Scholar - 11.Hartley, R., Zisserman, A.: Multiple View Geometry in Computer Vision, 2nd edn. Cambridge University Press, Cambridge (2004)CrossRefMATHGoogle Scholar
- 12.Horn, B.: Relative orientation. Int. J. Comput. Vision
**4**, 59–78 (1990)CrossRefGoogle Scholar - 13.Huang, T., Faugeras, O.: Some properties of the \(E\)-matrix in two-view motion estimation. IEEE Trans. Pattern Anal.
**11**, 1310–1312 (1989)CrossRefGoogle Scholar - 14.Longuet-Higgins, H.C.: A computer algorithm for reconstructing a scene from two projections. Nature
**293**, 133–135 (1981)CrossRefGoogle Scholar - 15.Maybank, S.: Theory of Reconstruction from Image Motion. Springer, Berlin (1993)CrossRefMATHGoogle Scholar
- 16.Nistér, D.: An efficient solution to the five-point relative pose problem. IEEE T. Pattern Anal.
**26**, 756–777 (2004)CrossRefGoogle Scholar - 17.Papadopoulo, T., Faugeras, O.: A new characterization of the trifocal tensor. In: European Conference on Computer Vision, 109–123. Springer, Berlin (1998)Google Scholar
- 18.Ponce, J., Hebert, M.: Trinocular geometry revisited. In: Proceedings of the IEEE Conference on Computer Vision Pattern Recognition, 17–24 (2014)Google Scholar
- 19.Quan, L., Triggs, B., Mourrain, B.: Some results on minimal euclidean reconstruction from four points. J. Math. Imaging Vis.
**24**, 341–348 (2006)MathSciNetCrossRefGoogle Scholar - 20.Ressl, C.: A minimal set of constraints and a minimal parameterization for the trifocal tensor. Int. Arch. Photogramm. Remote Sens.
**34**, 277–282 (2002)Google Scholar - 21.Shashua, A.: Trilinearity in visual recognition by alignment. In: European Conference on Computer Vision, 479–484. Springer, Berlin (1994)Google Scholar
- 22.Shashua, A., Werman, M.: On the trilinear tensor of three perspective views and its underlying geometry. In: Proceedings of the IEEE International Conference on Computer Vision, 920–925 (1995)Google Scholar
- 23.Spetsakis, M., Aloimonos, J.: Structure from motion using line correspondences. Int. J. Comput. Vision
**4**, 171–183 (1990)CrossRefGoogle Scholar - 24.Stewénius, H., Nistér, D., Kahl, F., Schaffalitzky, F.: A minimal solution for relative pose with unknown focal length. Image Vis. Comput.
**26**, 871–877 (2008)CrossRefGoogle Scholar - 25.Triggs, B.: Matching constraints and the joint image. In: Proceedings of the IEEE International Conference on Computer Vision, 338–343 (1995)Google Scholar
- 26.Viéville, T., Luong, Q.: Motion of points and lines in the uncalibrated case. Report RR-2054, INRIA (1993)Google Scholar
- 27.Weng, J., Huang, T., Ahuja, N.: Motion and Structure from line correspondences; closed-form solution, uniqueness, and optimization. IEEE Trans. Pattern Anal.
**14**, 318–336 (1992)CrossRefGoogle Scholar