Augmented Lagrangian-based Algorithm for Projective Reconstruction from Multiple Views with Minimization of 2D Reprojection Error

Abstract

In this paper, we propose a new factorization-based algorithm for projective reconstruction from multiple views by minimizing the 2D reprojection error in the images. In our algorithm, the projective reconstruction problem is formulated as a constrained minimization problem, which minimizes the 2D reprojection error in multiple images. To solve this constrained minimization problem, we use the augmented Lagrangian approach to generate a sequence of unconstrained minimization problems, which can be readily solved by standard least-squares technique. Thus we can estimate the projective depths, the projection matrices and the positions of 3D points simultaneously by iteratively solving a sequence of unconstrained minimization problems. The proposed algorithm does not require the projective depths as prior knowledge, unlike bundle adjustment techniques. It converges more robustly and rapidly than the penalty based method. Furthermore, it readily handles the case of partial occlusion, where some points cannot be observed in some images.

Appendix: Brief Review of the Augmented Lagrangian

Without loss of generality, we consider the following constrained minimization problem

$$ \min_xf\left(x\right) \label{eq:A1} $$

(13)

subject to

$$ c_i\left(x\right)=0 \quad\mbox{where}\quad i=1,2,\cdots,m $$

By eliminating the constraints through a penalty function (for example, a quadratic penalty function), the penalty method [2–4, 16, 19] for problem (13) consists of a sequential unconstrained minimizations of the form

$$ \min_xf\left(x\right)+\left(\gamma^{(k)}\right)^2\sum_{i=1}^mc_i^2\left(x\right) \label{eq:A2} $$

(14)

where the superscript (k) represents the k-th minimization problem. $\left\{\gamma^{(k)}\right\}$ is a positive scalar sequence with γ ^(k + 1) > γ ^(k) for all k. It has been proved that when γ ^(k)→ ∞, the solutions of the sequence of minimization problems (14) converge to the original constrained minimization problem (13). Penalty methods were widely accepted in practice for the simplicity of the approach, its ability to handle nonlinear constraints, as well as the availability of very powerful unconstrained minimization methods for solving problem (14). However, on the negative side, penalty methods are hampered by slow convergence and numerical instabilities associated with ill-conditioning in problem (14) induced by large values of the penalty parameter γ ^(k) [2–4, 16, 19].

To moderate the disadvantages of the penalty method, Hestenes [16] proposes the augmented Lagrangian method: a penalty term as well as a Lagrangian term are added to the objective function $f\left(x\right)$ in problem (13), and a sequence of minimization problems are formed

$$ \begin{array}{lll} \min_xL_A\left(x;\gamma^{(k)},\eta^{(k)}\right)&=&f\left(x\right)+\sum\limits_{i=1}^m\eta_i^{(k)}c_i\left(x\right)\\ &&+\left(\gamma^{(k)}\right)^2\sum\limits_{i=1}^mc_i^2\left(x\right) \label{eq:A4} \end{array} $$

(15)

The Lagrangian parameter sequence $\left\{\eta_i^{(k)}\right\}$ and penalty parameter sequence $\left\{\gamma^{(k)}\right\}$ are generated according to the following updating rule:

$$ \eta_i^{(k+1)}=\eta_i^{(k)}+2\left(\gamma^{(k)}\right)^2c_i\left(x^{(k)}\right) \label{eq:A5} $$

(16a)

$$ \gamma^{(k+1)}=\alpha\gamma^{(k)},\quad \alpha>1 \label{eq:A5b} $$

(16b)

where x ^(k) is the solution of minimization problem in Eq. 15

The typical step of the original version of the augmented Lagrangian method is given in [16] as follows: Given a Lagrangian parameter $\eta_i^{(k)}$ and a penalty parameter γ ^(k), we solve $\min_xL_A\left(x;\gamma^{(k)},\eta^{(k)}\right)$ (given in Eq. 15) and obtain the solution x ^(k). We then update $\eta_i^{(k)}$ and γ ^(k) according to Eq. 16, and repeat the process until convergence.

The proof of its convergence is given in [2–4], showing that by increasing γ ^(k) (without the need to increase to infinity) and by updating $\eta_i^{(k)}$ according to Eq. 16, the solutions of the minimization problem sequence (15) converge to the solution of the original minimization problem (13).

The important aspect of the augmented Lagrangian is that convergence may occur without the need to increase γ ^(k) to infinity. Thus the ill-conditioning associated with the penalty methods can be avoided and the convergence speed can be improved.