Recursive drivable road detection with shadows based on two-camera systems

Abstract

This paper concerns the road reconstruction problem of on-road vehicles with shadows. To deal with the effects of shadows, images are transformed to the proposed illuminant invariant color space and are fused with raw images to obtain more details in both dark and bright regions. Then, the road region is reconstructed from a geometry point of view. Based on the two-view geometric model, the scene can be described by the projective parallax information with respect to a reference plane, which is the grounding plane of the vehicle in this paper. The road reconstruction is performed row-by-row recursively. For each row, a row-wise image registration method is used to estimate the parallax information, based on which height information with respect to the reference plane can be calculated. Based on the distributions of height information and image intensity, the road region is detected and is used to direct the image registration of the next row. This process stops when the road region is small enough and the entire road region is obtained. The proposed approach is general in the sense that it works with shadows for different road types and camera configurations. Experimental results are provided to show the effectiveness and robustness of the proposed method.

Appendix: two-view geometry development

As shown in Fig. 3, the ray from the origin of ${\mathcal {C}}$ to $O_i$ intersects with the plane $\pi$ at the point $O_{\pi i}$. Denote the coordinates of $O_{\pi i}$ with respect to ${\mathcal {C}}$ and ${\mathcal {C}}'$ as $\bar{m}_i \triangleq \begin{bmatrix} x_i&y_i&z_i \end{bmatrix}^\mathrm{T} \in {\mathbb {R}}^3$ and $\bar{m}'_i \triangleq \begin{bmatrix} x'_i&y'_i&z'_i \end{bmatrix}^\mathrm{T} \in {\mathbb {R}}^3$, respectively, which can be related by Chen et al. [7]:

$$\begin{aligned} \bar{m}_{\pi i}'=H \bar{m}_{\pi i}, \end{aligned}$$

(25)

where H is the Euclidean homography defined as:

$$\begin{aligned} H \triangleq R+x_f \frac{n^T}{d} . \end{aligned}$$

(26)

Construct a plane $\pi '$ through point $O_i$ parallel to $\pi$, and its distance to the plane $\pi$ is $D_i$. Similar to (25), the Euclidean homography relationship of the coordinates of point $O_i$ with respect to ${\mathcal {C}}$ and ${\mathcal {C}}'$ induced by plane $\pi '$ is given by

$$\begin{aligned} \bar{m}'_i=H' \bar{m}_i = \left( H+x_f \frac{D_i n^T}{d(d-D_i)} \right) \bar{m}_{i}. \end{aligned}$$

(27)

Since $n^T \bar{m}_i=d-D_i$, then (27) can be rewritten as:

$$\begin{aligned} \bar{m}'_i=H \bar{m}_i + \frac{D_i}{d} x_f . \end{aligned}$$

(28)

The normalized coordinates of $\bar{m}_i,\bar{m}'_i$ are denoted as $m_i\triangleq \frac{\bar{m}_i}{z_i},m'_i\triangleq \frac{\bar{m}'_i}{z'_i}$, respectively, and the projected image points $p,p'$ are derived using the pinhole model [16]:

$$\begin{aligned} p_i=Am_i, p'_i=A'm'_i . \end{aligned}$$

(29)

Denoting $G \triangleq A' H A^{-1}$ as the projective homography, the geometric model (8) is obtained using (28) and (29).