Optimal Ellipse Based Algorithm as an Approximate and Robust Solution of Minimum Volume Covering Ellipse Problem

Abstract

We propose a algorithm to give a approximate solution of a minimal covering circle or ellipse of a set of points. The iterative algorithm is based on the optimal ellipse which best describe a given set of points.

1 Introduction

In order to more realistically simulate the behavior of solid in games, we have to check especially if they collide with each other every time they move, and if they do, we have to do something about it. This lead us to game physics engine that gives the game’s world life. Understanding collision physics in 2D and 3D games [1–3] is particularly important for game developers. The collision detection algorithms depends on the type of shapes that can collide. We can consider e.g. rectangle to rectangle or circle to circle collisions. In general case we will have a simple generic shape that covers the entity known as a “hitbox” so even though collision may not be pixel perfect, it will look good enough and be performance across multiple entities.

This problem can be also reformulated as finding the unique minimum volume covering ellipse (MVCE) problem of finite set of points in N-dimensional real space (namely in $\text {I}\!\text {R}{^N}$) if we consider just elliptical generic shapes. Such an ellipse has appealing mathematical properties which makes it theoretically interesting and practically useful. This convex constrained problem arises in a variety of applied computational settings, particularly in data mining and robust statistics. For example in computational statistics, the minimum-volume ellipsoid covering k of m ($0\le k\le m$, $k, m\in \text {I}\!\text {N}$) given points in $\text {I}\!\text {R}^N$ is well-known for its affine equivariance and positive breakdown properties as a multivariate location and scattering estimator [4]. In the area of robust statistics and data mining, efficiently finding outliers is a challenge that has attracted much research interest [5]. Indeed, one can identify data outliers quickly if one can compute the minimum-volume ellipsoid quickly, because outliers are essentially points on the boundary of the minimum-volume covering ellipsoid.

In this paper we focus on the image data, namely the points from $\text {I}\!\text {R}^N$. The extended version for arbitrary dimension with streaming approach is prepared as a full paper and will be published in the future.

A new algorithm for finding the MVCE is based on the Mahalanobis distance [6] and uses the statistical and analytical properties of circular and elliptical objects, is proposed herein. The main advantage of the proposed method is that no complicated mathematical computation is involved in the implementation. Thus its evaluation is very fast. However the outcome ellipse is not optimal one – this will be disused in further part of this article.

We compare our algorithm with the incremental algorithm of Welzl, with move-to-front heuristic [7] – the whole implementation is described in [8].

The remainder of the paper is organized as follows: in the next section we present basics of the covariance matrix and Mahalanobis distance. In the third section we provide the algorithm for finding approximate minimal covering ellipse. The next section gives same experiments o illustrate the performance of our method. Finally, the last section contains some concluding remarks and possible directions of future work.

2 Mathematical Basis

2.1 Covariance Matrix and Optimal Ellipse

Multivariate and multidimensional analysis has been one of the most important areas in the modern statistics. Thus, the researchers have developed many useful techniques to efficiently describe the behavior of complicated data.

The definition of the covariance matrix states as follows, if set C is discrete, namely $C=\{x_i\}_{i=1}^N\subset \text {I}\!\text {R}^N$, then the covariance matrix equals

$$ {\varvec{\Sigma }}_C=\frac{1}{N}\sum _{i=1}^N(x_i-\mathbf {m}_C)(x_i-\mathbf {m}_C)^T, $$

where $\mathbf {m}_C=\frac{1}{N}\sum _{i=1}^Nx_i$ is a mean of set C. In the general case by the covariance matrix of a set we understand the covariance of the uniform normalized density on S.

Theoretically, covariance is the object which represents the true statistical independent structure of the underlying population units. On the other hand, if based solely on empirical data, only a small piece of the independent structure is obtained. It is therefore important to select the efficient representations of the data for sampling the covariance matrix.

Remark 1

In practice the mean and the covariance matrix for a given set $C \subset \text {I}\!\text {R}^N$ is unknown. Thus they must be estimated from random variables (observed samples) by well-known estimators

$$ \hat{\mathbf {m}}_C =\sum _{i=1}^n w_ix_i, $$

$$ \hat{{\varvec{\Sigma }}}_C = \frac{\sum _{i=1}^n w_i}{(\sum _{i=1}^n w_i)^2-\sum _{i=1}^n w_i^2} \sum _{i=1}^n (x_i-\hat{\mathbf {m}}_C)(x_i-\hat{\mathbf {m}}_C)^T, $$

where $x_i\in C$ for $i=1,\ldots ,n$ and $w_i$ – the weight of the point $x_i$. If all weights are the same, the weighted mean and covariance reduce to the sample mean and covariance. The weight assigned with the point of space will be used to store (and use) the information about color while the work is done with grayscale images. For a binary image a natural simplification is employed: 1 for the foreground color (usually black) and 0 for the background color (usually white).

Remark 2

The above consideration discuses discrete sets. In general case the basic definition of covariance matrix can be stated as follows. For a random variable with a probability density function $f : \text {I}\!\text {R}^N \rightarrow \text {I}\!\text {R}$ the covariance matrix of set $S \subset \text {I}\!\text {R}^N$ can be computed¹ as

$$ \Sigma _S = \int _S (x-\mu ) (x-\mu )^Tf(x)dx, $$

where $\mu$ is an expected value (mean) of set S.

2.2 Pixels on Image as a Atomic Piece of Information in Covariance Matrix

When we apply the above investigations to image data, we shall threat a single pixel (point with coordinates , where $w\times h$ is the size of the image) like a square. It allows to convert the discrete data of an image to continuous, which is more natural and consistent with the human perception of images. To calculate the measure of such transformation we use the following remark.

Remark 3

Consider a set which represents the set $S=P+[-\frac{\delta }{2},\frac{\delta }{2}]^N$ for $\delta >0$. Then, obviously, covariance matrix is

$$ {\varvec{\Sigma }}_S={\varvec{\Sigma }}_P+{\varvec{\Sigma }}_{[-\frac{\delta }{2},\frac{\delta }{2}]^N}={\varvec{\Sigma }}_P+\frac{1}{12}\delta ^{N+2}\mathrm {I}, $$

where $\mathrm {I}$ denotes the identity matrix. This can be used to compute the measure and covariance of S efficiently.

In the case of the image data we set $\delta =1$, which is because the coordinates of each pixel are integers.

In general case we have to determine the appropriate value of $\delta$, for which investigated dataset can be threated as a subset of .

This section is concluded with a simple, but important observation for singletons in image data, namely for $X=\{x_0\}$ we get

$$ \mathbf {m}_X=x_0, $$

$$ {\varvec{\Sigma }}_X=\frac{1}{12}\mathrm {I}, $$

$$ \mathrm {card}(X)=1, $$

where I denotes identity matrix and .

2.3 An In-Place Covariance Modification

Following theorem highlights how mean and covariance matrix will change if we add same points to the set, which will be used for the algorithm in the further part of this work. While new algorithm in each step modify mean and covariance matrix respectively to added point, this theorem allows as not to recalculate the whole covariance matrix each time – we just calculate the change of covariance matrix.

Theorem 1

([9]). Let U, V be given finite subsets of $\text {I}\!\text {R}^N$. Assume additionally that $U\cap V=\emptyset$. Then

$$\begin{aligned} \mathbf {m}_{U\cup V}= & {} p_U \mathbf {m}_{U} + p_V \mathbf {m}_{V}\end{aligned}$$

(1)

$$\begin{aligned} {\varvec{\Sigma }}_{U\cup V}= & {} p_U{\varvec{\Sigma }}_{U}+p_V {\varvec{\Sigma }}_{V}u+p_Up_V(\mathbf {m}_{U}-\mathbf {m}_{V})(\mathbf {m}_{U}-\mathbf {m}_{V})^T \end{aligned}$$

(2)

where

$$p_U:=\frac{\mathrm {card}(U)}{\mathrm {card}(U)+\mathrm {card}(V)},\qquad p_V:=\frac{\mathrm {card}(V)}{\mathrm {card}(U)+\mathrm {card}(V)}.$$

2.4 Mahalanobis Distance

It is well-known that the Euclidean distance between two points x and y in real space is given by

$$ \Vert x-y\Vert =\sqrt{(x-y)^T(x-y)}=\sqrt{(x-y)^T\mathrm {I}(x-y)}, \text { for } x,y\in \text {I}\!\text {R}^N, $$

where T denotes the transpose operation and $\mathrm {I}$ is the identity matrix. It follows immediately that all points with the same distance from the origin $\Vert x-0\Vert = c$ satisfy $x_1^2+\ldots +x_n^2 = c^2$, which means that all components of the observation x contribute equally to the Euclidean distance of x from the center.

However, in statistics variability (how the set is spread out or closely clustered) is also considered. If the correlation between variables is to be taken into account when computing statistical distance, then we use Mahalanobis distance, which can be defined as follows.

Definition 1

(Mahalanobis Distance). Let C denote the given subset of $\text {I}\!\text {R}^N$. By $m_C$ the mean value of C is denoted, and the covariance matrix of C is denoted by ${\varvec{\Sigma }}_C$. The statistical distance or Mahalanobis distance between two points $x, y \in \text {I}\!\text {R}^N$ is defined as

$$\begin{aligned} \Vert x-y\Vert _{{\varvec{\Sigma }}_C}=\sqrt{(x-y)^T{\varvec{\Sigma }}_C^{-1}(x-y)}. \end{aligned}$$

(3)

If $y=0$ then the Eq. (3) is the general equation of an ellipsoid centered at the origin. In practice the center of the observations will differ from the origin, thus the distance from the mass center of observed samples chosen $y = \mathbf {m}_C$ will be of interest.

2.5 Ellipse in Mahalanobis Transformed Space

A brief summary of the Mahalanobis distance and transform properties was given in previous section. Some important observations crucial in the construction of the following algorithm with the use of Mahalanobis distance will be hereby presented.

Recall the value of the Mahalanobis distance of the unit N-dimension ball receptively to covariance matrix ${\varvec{\Sigma }}$ can be calculated, namely

$$\mathrm {B}_{\varvec{\Sigma }}(x_0,R) :=\{x\in \text {I}\!\text {R}^N:\Vert x-x_0\Vert _{\varvec{\Sigma }}\le R\}.$$

The following knowledge is crucial for the algorithm construction.

Theorem 2

([10]). Consider uniform probability density on the ellipse $E \subset \text {I}\!\text {R}^N$ with mean $\mathbf {m}_E$ and covariance ${\varvec{\Sigma }}_E$. Then

$$\begin{aligned} E=\mathrm {B}_{{\varvec{\Sigma }}_E}\bigg (\mathbf {m}_E,\sqrt{N+2}\bigg ). \end{aligned}$$

(4)

Theorem 2 gives the equation for optimal ellipse for given set and has the following consequences in understanding of the elliptical pattern recognition. Moreover, it describe how to reconstruct the set if we just have mean and covariance matrix.

Remark 4

Theorem 2 gives the necessary and sufficient condition for a given shape to be elliptical (generalization of plane ellipse) – general ellipse in arbitrary dimension. Namely, it can be stated that an object $E\subset \text {I}\!\text {R}^N$ is an ellipse if and only if it can be approximated by $\mathrm {B}_{{\varvec{\Sigma }}_E}\bigg (\mathbf {m}_E,\sqrt{N+2}\bigg )$ or, that is,

$$ E\approx \mathrm {B}_{{\varvec{\Sigma }}_E}\bigg (\mathbf {m}_E,\sqrt{N+2}\bigg ). $$

The above considerations gives us the specified condition to decide whether or not the current point x from the cluster $C \subset \text {I}\!\text {R}^N$ belong to the “elliptical” part of the cluster. Namely, the condition for point x is given by

$$ \sqrt{(x-\mathbf {m}_C)^T{\varvec{\Sigma }}_C^{-1}(x-\mathbf {m}_C)} \le \sqrt{N+2}, $$

where $\mathbf {m}_C$ and ${\varvec{\Sigma }}_C$ denote the mean and covariance of C respectively. This gives as full necessarily information for construction the proper algorithm for finding approximate minimum volume covering ellipse.

Fig. 1.

New algorithm for finding minimal covering ellipse: (a) original image and (b)-(f) steps of the algorithm.

3 The Algorithm

In this section we introduce the algorithm for finding approximate minimum volume covering ellipse. We will describe this algorithm using Fig. 1 which presents the example of investigated set (original image) and steps of the algorithm.

Let consider the finite set C of which we want to calculate the smallest covering ellipse. Since C is finite we can state $C=\{x_i\}_{i=1,\ldots ,M}$.

At the beginning of the algorithm we calculate the optimal ellipse for C, by calculating the mean and covariance matrix denoted $m_0$ and $\Sigma _0$ respectively. The optimal ellipse is current approximation of minimal covering ellipse.

Then in each i-step we proceed as follows:

• we check which of points in C are not covered by current approximation of minimal covering ellipse – we denote the set consist of such points by $l_i$. If the set $l_i$ is empty we end the algorithm;

• we calculate the mean and covariance matrix for set $l_i$ and denoted them $m_i$ and $\Sigma _i$ respectively;

• we update the current approximation of minimal covering ellipse according Theorem 1;

• then we proceed to next step.

The pseudo-code of the algorithm can be stated as follows:

The outcome of this algorithm are the mean and covariance matrix of approximate covering ellipse.

The following comment are needed to clarify the algorithm statements and used notations:

• the $\mathbf {m}_i$ and $\mathbf {m}_{l_{i}}$ denotes the mean of current approximation of convex hull and the mean value of set $l_{i}$ respectively;

• the ${\varvec{\Sigma }}_i$ and ${\varvec{\Sigma }}_{l_{i}}$ denotes the covariance matrix of current approximation of convex hull and the covariance matrix value of set $l_{i}$ respectively;

• the condition $x\not \in \mathrm {B}_{\Sigma _{i}}(m_{i},\sqrt{N+2})$ means that for tested x we get

  $$\sqrt{(x-\mathbf {m}_{i})^T{\varvec{\Sigma }}_{i}^{-1}(x-\mathbf {m}_{i})}>\sqrt{N+2};$$

• calculation of $\mathbf {m}_{i+1}$ and ${\varvec{\Sigma }}_{i+1}$ in each step are done according to Eqs. (1) and (2) from Theorem 1.

Table 1. Execution time for authors algorithm vs. CGAL implementation of the incremental algorithm of Welzl.

Fig. 2.

Comparison of results of two algorithm: CGAL implementation of the incremental algorithm of Welzl (second column) and authors’ algorithm applied for certain shapes (first column).

4 The New Algorithm in Simple Illustrations

Emo Welzl [7] proposed a simple randomized algorithm for the minimum covering circle problem that runs in expected O(N) time, based on a linear programming algorithm of Raimund Seidel [11]. The algorithm is recursive, and takes as arguments two sets of points S and Q. The algorithm computes the smallest enclosing circle of the union of S and Q, as long as every point of Q is one of the boundary points of the eventual smallest enclosing circle. Thus, the original smallest enclosing circle problem can be solved by calling the algorithm with S equal to the set of points to be enclosed and Q equal to the empty set; as the algorithm calls itself recursively, it will enlarge the set Q passed into the recursive calls until it includes all the boundary points of the circle.

In this work we compare result of our new algorithm with the incremental algorithm of Welzl implemented in CGAL library [12]. The Fig. 2 compares the results of finding smallest enclosing ellipse for same set of image data. As we can observe the results sames to be identical. However, the time needed to calculate those results differ for both algorithm – compare with Table 1. Thus our algorithm seem to be better in time consuming tests.

To confirm this we choose the benchmark image – Fig. 3 – which has originally size 300$\,\times \,$300 pixels and contains 8000 black pixels (points of interest). Then we resize the original image according to sizes given in Table 2. For each sizes we calculate the execution time of both algorithms.

This experiments shows that indeed the authors’ algorithm is faster. However it seam that the complexity of new algorithm is also O(N), but this complexity in this case is realized with smaller constance then in case of Welzl algorithm.

Fig. 3.

Input, output from CGAL

Table 2. Comparison of execution time for image presented on Fig. 3 for different image size (first column) calculated with CGAL implementation of the incremental algorithm of Welzl and authors’ method

5 Conclusions

Int this paper the new approach for finding minimal covering ellipse was presented. This new method based on calculating the covariance matrix and interpret this according to behavior of Mahalanobis distance. The main advantage of this approach is that the results can be obtained very quickly.

We are planning to test this approach for streaming data, to detect the changes in stream data trends.