Detecting parametric objects in large scenes by Monte Carlo sampling

Abstract

Point processes constitute a natural extension of Markov random fields (MRF), designed to handle parametric objects. They have shown efficiency and competitiveness for tackling object extraction problems in vision. Simulating these stochastic models is however a difficult task. The performances of the existing samplers are limited in terms of computation time and convergence stability, especially on large scenes. We propose a new sampling procedure based on a Monte Carlo formalism. Our algorithm exploits the Markovian property of point processes to perform the sampling in parallel. This procedure is embedded into a data-driven mechanism so that the points are distributed in the scene in function of spatial information extracted from the input data. The performances of the sampler are analyzed through a set of experiments on various object detection problems from large scenes, including comparisons to the existing algorithms. The sampler is also tested as optimization algorithm for MRF-based labeling problems.

Appendices

Appendix 1: Population counting model

Let $x$ denote a configuration of ellipses for which the center of mass of an ellipse is contained in the compact set $K$ supporting the input image (see Fig. 19). The energy follows the form specified by Eq. 5. The unitary data term $D(x_i)$ and the potential $V(x_i,x_j)$ are given by:

$$\begin{aligned} D(x_i)&= \left\{ \begin{array}{ll} \quad 1 - \frac{d(x_i)}{d_0} &{}\text { if }d(x_i)<d_0\\ \quad exp(\frac{d_0-d(x_i)}{d_0}) -1 &{}\text { otherwise}\\ \end{array} \right. \end{aligned}$$

(20)

$$\begin{aligned} V(x_i,x_j)&= \beta \frac{A(x_i \cap x_j)}{\min (A(x_i),A(x_j))} \end{aligned}$$

(21)

where

• $d(x_i)$ represents the Bhattacharyya distance between the radiometry inside and outside the object $x_i$:

  $$\begin{aligned} d(x_i) = \frac{(m_{in}-m_{out})^2}{4(\sigma _{in}^2+\sigma _{out}^2)} - \frac{1}{2} ln\left( \frac{2\sigma _{in}\sigma _{out}}{\sigma _{in}^2+\sigma _{out}^2}\right) \end{aligned}$$

  (22)

  where $m_{in}$ and $\sigma _{in}$ (respectively $m_{out}$ and $\sigma _{out}$) are the intensity mean and standard deviation in $S_{in}$ (respectively in $S_{out}$).

• $d_0$ is a coefficient fixing the sensitivity of the object fitting. The higher the value of $d_0$, the more selective the object fitting. In particular, $d_0$ has to be high when the input images are corrupted by a significant amount of noise.

• $A(x_i)$ is the area of object $x_i$.

• $\beta$ is a coefficient weighting the non-overlapping constraint with respect to the data term.

Note that a basic mathematical dilatation is used in practice to roughly extract the class of interest from the image of birds for creating a space-partitioning tree.

Fig. 19

Objects and their parameters for the various presented models. (left) Ellipses and (middle) line-segments are defined by a 2D-point $p \in K$ (center of mass of the object) and some marks. These additional parameters are the semi-major axis $b$, the semi-minor axis $a$, and the angle $\theta$ for an ellipse, and the semi-length $b$, the semi-width $a$, the orientation $\theta$, and the anchor length $c$ for a line-segment. The inside (respectively bordering) volume of the object is denoted by $S_{in}$ (respectively $S_{out}$). The anchors are denoted by $A_1$ and $A_2$. (right) 3D-trees are defined by a 3D-point $p \in K$ (center of mass of the object), a type $t \in \{$conoidal, ellipsoidal, semi-ellipsoidal$\}$ illustrated on Fig. 20, and $3$ additional parameters which are the canopy height $a$, the trunk height $b$ and the canopy diameter $c$. The cylindrical volume $\mathcal C x_i$ represents the attraction space of object $x_i$ in which the input points are used to measure the quality of this object

Appendix 2: Line-network extraction model

A line-segment is defined by five parameters, including the 2D point corresponding to the center of mass of the object (Fig. 19). Similarly to the population counting model detailed in Appendix 1, the fitting quality with respect to the data is based on the Bhattacharyya distance: the unitary data term $D(x_i)$ of the energy is given by Eq. 20. The potential $V(x_i,x_j)$ penalizes strong object overlaps (see Eq. 21), but also takes into account a connection interaction in order to favor the linking of the line-segments. The potential term is thus given by:

$$\begin{aligned} V(x_i,x_j) \!=\! \beta _1 \frac{A(x_i \cap x_j)}{\min (A(x_i),A(x_j))} \!+\! \mathbf 1 _{x_i \sim _{nc} x_j} \times \beta _2 f(x_i,x_j) \nonumber \\ \end{aligned}$$

(23)

where

• $\beta _1$ and $\beta _2$ are two coefficients weighing respectively the non-overlapping and connection constraints with respect to the data term.

• $\sim _{nc}$ is the non-connection relationship between two objects. $x_i \sim _{nc} x_j$ if the anchor areas of $x_i$ and $x_j$ (see Fig. 19) do not overlap.

• $\mathbf 1 _{condition}$ is the indicative function returning one when condition is valid, and zero otherwise.

• $f(x_i,x_j)$ is a symmetric function weighting the penalization of two non-connected objects $x_i$ and $x_j$ with respect to their average fitting quality. The function $f$ is introduced to slightly relax the connection constraint when the two objects are of very good quality.

As for the bird counting problem, a basic mathematical dilatation has been used to roughly extract the class of interest from the aerial image shown on Fig. 14. Indeed the pixels corresponding to the class road in this image are relatively bright compared to the background. The segmented result is obviously not optimal, but sufficient to create an efficient space-partitioning tree.

Appendix 3: Tree recognition model formulation

Let $x$ represent a configuration of 3D-models of trees from a template library described in Fig. 20. The center of mass $p$ of a tree is contained in the compact set $K$ supporting the 3D bounding box of the input point cloud (Fig. 19). We denote by $\partial x_i$ the surface of the object $x_i$, and by $\mathcal C x_i$ the cylindrical volume having a vertical axis passing through the center of mass of $x_i$, in which the input points are considered to measure the quality of $x_i$. The unitary data term $D(x_i)$ and the pairwise potential $V(x_i,x_j)$ are given by:

$$\begin{aligned} D(x_i)&= \frac{1}{|\mathcal C x_i|} \underset{p_c \in \mathcal C x_i}{\prod } \gamma ( d(p_c, \partial x_i)) \end{aligned}$$

(24)

$$\begin{aligned} V(x_i,x_j)&= \beta _1 V_{overlap}(x_i,x_j) + \beta _2 V_{competition}(x_i,x_j) \nonumber \\ \end{aligned}$$

(25)

where

• $|\mathcal C x_i|$ is a coefficient normalizing the unitary data term with respect to the number of input points contained in $\mathcal C x_i$.

• $d(p_c, \partial x_i)$ is a distance measuring the coherence of the point $p_c$ with respect to the object surface $\partial x_i$. $d$ is not the traditional orthogonal distance from point to surface because, as real trees do not describe ellipsoidal/conoidal shapes, input points are not homogeneously distributed on the object surface. Here, $d$ is defined as the combination of the planimetric distance, i.e. the projection in the plane of equation $z=0$ of the Euclidean distance, and the altimetric variation such that points outside the object are more penalized than inside points. Note that $d$ is invariant by rotation around the Z-axis.

• $\gamma (.) \in [-1,1]$ is a quality function which is strictly increasing.

• $V_{overlap}$ is the pairwise potential penalizing strong overlapping between two objects, and given by:

  $$\begin{aligned} V_{overlap}(x_i,x_j) = \frac{A(x_i \cap x_j)}{\min (A(x_i),A(x_j))} \end{aligned}$$

  (26)

  where $A(x_i)$ is the area of the object $x_i$ projected onto the plane of equation $z=0$.

• $V_{competition}$ is the pairwise potential favoring a similar tree type $t$ in a local neighborhood:

  $$\begin{aligned} V_{competition}(x_i,x_j) = \mathbf 1 _{t_i \ne t_j} \end{aligned}$$

  (27)

  where $\mathbf 1 _{.}$ is the indicative function.

• $\beta _1$ and $\beta _2$ are two coefficients weighting respectively the non-overlapping constraint and the competition term with respect to the data term.

In order to roughly extract the class of interest from the point clouds, the scatter descriptor proposed by Lafarge and Mallet (2012) is used to identify the points which potentially correspond to trees.

Fig. 20

Library of tree models—the objects are specified by a 3D point (center of mass illustrated by a red dot) and additional parameters (blue arrows) including the canopy type whose shape can be conoidal (e.g. pine or fir), ellipsoidal (e.g. poplar or tilia) or semi-ellipsoidal (e.g. oak or maple)