Planar Structures from Line Correspondences in a Manhattan World

Abstract

Traditional structure from motion is hard in indoor environments with only a few detectable point features. These environments, however, have other useful characteristics: they often contain severable visible lines, and their layout typically conforms to a Manhattan world geometry. We introduce a new algorithm to cluster visible lines in a Manhattan world, seen from two different viewpoints, into coplanar bundles. This algorithm is based on the notion of “characteristic line”, which is an invariant of a set of parallel coplanar lines. Finding coplanar sets of lines becomes a problem of clustering characteristic lines, which can be accomplished using a modified mean shift procedure. The algorithm is computationally light and produces good results in real world situations.

Appendix: Characteristic Planes Revisited

We present here a different derivation of the characteristic planes concept, obtained through algebraic manipulations. For simplicity’s sake, we will restrict our attention to one canonical plane $\varPi _i$, assuming that both cameras are located on it. A 2-D reference system is centered at the first camera. In this 2-D world, each camera only sees an image line, and the cameras’ relative pose is specified by the (unknown) 2-D vector $\mathbf{t}$ and the (known) 2-D rotation matrix $\mathbf R$. We’ll assume that both cameras have identity calibration matrices. Consider a plane $\varPi _j$ with (known) normal $\mathbf{n}_j$, orthogonal to $\varPi _i$. A line $\mathcal{L}$ in $\varPi _j$ intersects $\varPi _i$ at one point, $\mathbf{X}$. Note that, from the image of this point in the first camera and knowledge of the plane normal $\mathbf{n}_j$, one can recover $\mathbf{X}/d$, where $d$ is the (unknown) distance of $\varPi _j$ from the first camera. Let $\hat{\mathbf{x}}_2$ be the location of the projection of $\mathbf X$ in the second camera’s (line) image, expressed in homogeneous coordinates. From Fig. 1 one easily sees that $\lambda \hat{\mathbf{x}}_2 = \mathbf{R} \mathbf{X} + \mathbf{t}$ for some $\lambda$, and thus

$$\begin{aligned} \mathbf{t}/d = \lambda \hat{\mathbf{x}}_2/d - \mathbf{R} \mathbf{X}/d= \lambda _2 \hat{\mathbf{x}}_2 - {(\mathbf R\mathbf X)}_{\bot }/d \end{aligned}$$

(6)

for some $\lambda _2$, where ${(\mathbf R\mathbf X)}_{\bot }=\mathbf{R \mathbf X} - (\hat{\mathbf{x}}_2^T\mathbf{R X}) \hat{\mathbf{x}}_2 /(\hat{\mathbf{x}}_2^T \hat{\mathbf{x}}_2)$ is the component of $\mathbf{R \mathbf X}$ orthogonal to $\hat{\mathbf{x}}_2$. This imposes a linear constraint on $\mathbf{t}/d$. It is not difficult to see that $\hat{\mathbf{x}}_2$ is orthogonal to the lever vector $\vec u_2$ in Fig. 1, and that $\Vert {(\mathbf R\mathbf X)}_{\bot }/d\Vert$ is equal to the modulus of the sin ratio for the line $\mathcal{L}$ seen by the two cameras. Hence, the linear constraint in (6) is simply an expression of the intersection of the characteristic plane $\varPi (\mathcal{L},{\vec n}_j)$ with $\varPi _i$.