Keywords

1 Introduction

Extracting ellipses from images is an important problem in computer vision and has a diverse area of applications from object detection to pose estimation [3, 10, 11, 15–17, 26]. Since the projections of circular objects appear as ellipse on the camera image plane, ellipse detection is employed in many real life applications. However, ellipse detection is much more difficult than circle detection because an ellipse has 5 degrees of freedom (the center coordinates x & y, semi-major and semi-minor axes a & b, and rotation angle \(\alpha \)) whereas a circle has 3. Due to the same reason, many different shapes (i.e., a rectangular or a line) can be represented by an ellipse with a reasonable amount of accuracy.

There are many studies on ellipse detection found in the literature and they are categorized in two groups, i.e. model-based and feature-based methods. Although both approaches have pros and cons, many state-of-the-art algorithms are feature-based methods and gives better results in terms of accuracy and speed. Model-based methods fits a mathematical model to plain pixel information. McLaughlin uses the famous Hough Transform (HT) for accurate ellipse detection [17]. A model-based search is a very slow operation for ellipse shape since it needs to be performed in 5-dimensional parameter space. Zhang and Liu utilize HT with convexity property of edges and obtain good results for detecting faces in images with ellipse detection [29]. Lei and Wong incorporate the symmetric property to HT to improve computation efficiency [12]. Feature-based approaches first extracts higher level geometric features, i.e. lines or arcs. In recent years, preprocessing of arc and line segments with geometrical properties are mostly investigated. Libuda et al. concatenates line and arc primitives to construct ellipses [13]. Nguyen et al. calculates ellipse parameters after getting arc pieces by using ellipse properties [18]. Nguyen and Kerautret describe an ellipse detection algorithm that works by obtaining arcs and lines from an edge map and combines these features [19]. Prasad et al. propose a heuristic ellipse detector based on convexity and edge curvature properties [21]. Fornaciari et al. choose arcs strategically and compute parameters of ellipses with HT [6]. Arcs are classified according to their convexity and then grouped to test if they compose an ellipse or not. A popular model fitting technique Random Sample Consensus (RANSAC) is also employed in both model-based and feature-based methods for ellipse detection. Watcharin and Kaewapichai makes use of the RANSAC and propose an efficient and robust ellipse detection algorithm [9]. Song and Wang use RANSAC with edge information and tangent directions to fit ellipses while eliminating useless feature points [24]. In RANSAC, fitting is applied to randomly selected points, so it is not a steady algorithm and there is no guarantee to get the equivalent results despite same inputs are given.

The reason behind the crowd literature of ellipse detection is the difficulty of the problem. In addition to the fact that different geometric shapes such as rectangle can be represented by an ellipse with a reasonable accuracy; occlusions in real life images significantly aggravates the problem. Many methods applies image specific thresholds to find the best possible result, however, this situation hinders applicability of the algorithms.

In this study, we propose a feature-based ellipse detection algorithm which gives promising results even in difficult occlusive cases. Proposed method does not perform any voting mechanism, therefore works in a parameter-free manner without applying image specific score threshold values. Unlike many other methods, the algorithm aims to evaluate all combinations of extracted elliptical arcs instead of grouping fixed number of arcs. In this way, detection of ellipses can be achieved even though arcs forming them are spread around the image. To obtain accurate results, we use analytic methods to compute ellipse fitting error instead of employing approximations. For the experimental validation, we assess our algorithm with recently published state-of-the-art methods and present quantitative results on different real image datasets.

2 Proposed Method

The proposed ellipse detection algorithm follows the steps of the flowchart shown in Fig. 1. Initially, the edge segments of an image are extracted as contiguous array of pixels to ease the further processing. [2]. In the second step we extract the elliptical arcs which may be part of an occluded ellipse. Next, we seek the arc combinations which form valid ellipses. For this purpose, arc pairing operation is done by utilizing the locations and orientations of detected arcs to determine whether two arcs are coelliptic or non-coelliptic in each pair. In this way, arc combinations which may not constitute a valid ellipse are eliminated without performing computationally expensive ellipse fitting and error computation operations. Finally, each of the remaining arc combinations are then tested by ellipse fitting and the final ellipses are found according to the fitting error and spanned angle ratio parameters. Steps of the proposed algorithm are explained in the following subsections in detail.

Fig. 1.
figure 1

Flowchart for ellipse detection

Full size image
Fig. 2.
figure 2

Test image and the extracted edge segments. Note that each edge segment is shown in different colors. (Color figure online)

Full size image

2.1 Detection of Edge Segments

The very first step of the algorithm is extracting the edge segments from the input image. For this task, we employ the EDPF method [2] which is based on the Edge Drawing algorithm [27]. It works by first determining certain anchor points in an image and joining these anchors by a smart routing procedure. By setting all ED’s parameters at extremes, EDPF detects all possible edge segments in an image. The false detections are then eliminated by a validation mechanism, and thus only valid edge segments are returned. In Fig. 2 a real test image and detected edge segments are shown.

2.2 Extraction of Elliptical Arcs

In this step, elliptical arcs are extracted from the edge segments in a two-stage algorithm. First, the corner locations are computed along each segment with a curvature based corner detection algorithm [4, 28]. Then ellipse fit is applied to the pixels in between two consecutive corners and the pixel sets resulting low fitting error are selected as elliptical arcs.

There are two popular ellipse fitting methods in the literature [5, 25]. Fitzgibbon’s method guarantees that the resulted conic is an ellipse, however, it usually ends up with more eccentric ellipses and higher fitting errors. In Taubin’s method, obtained ellipse contour better fits the points and gives lower error without the guarantee for an ellipse. Since we apply ellipse fitting to consecutive edge pixels rather than scattered points, we can obtain elliptical arcs with Taubin’s method.

Fig. 3.
figure 3

Extraction of the corners and the elliptical arcs. (a) corners, (b) elliptical arcs.

Full size image

There is no straightforward way to compute ellipse fitting error. In many studies various numerical estimation methods are employed [11, 16, 19]. In our study we derived an iterative method based on a modified version of [20] for precise ellipse fitting error computation. Although this method constitutes a major part of our total running time, we can precisely compute Euclidean distances between pixels and ellipse contour. In Fig. 3 detected corners and arcs obtained at the end of this step are shown for the same test image shown in Fig. 2.

2.3 Detection of Non-Coelliptic Arc Pairs

For n elliptical arcs extracted in the previous step, there are obviously \(2^n-1\) potential combinations to join and detect ellipses. In the example shown in Fig. 3(b) we have 16 arcs, hence 65535 combinations, each may yield a valid ellipse. Briefly, there is an exponential relationship between the detected elliptical arcs and ellipse hypotheses. Therefore, it becomes a troublesome procedure to test all arc combinations as the number of arcs grows.

To reduce the number of arc combinations to test, we follow a proactive way by analyzing all arcs in pairwise to determine whether any pair can be part of the same ellipse or not. In other words, we try to find out whether any two arcs are coelliptic or non-coelliptic. Once we determine any non-coelliptic arc pair, we remove all arc subsets which includes that pair from entire \(2^n-1\) combinations to reduce the computation time. Determining whether an arc pair is non-coelliptic can be done in different ways. In this study, we derive a heuristic with a simple observation to find non-coelliptic arc pairs by utilizing conic properties of detected arcs. If two arcs’ concavities are towards opposite directions, i.e. two arcs are located back-to-back, they can be considered as non-coelliptic since they cannot belong to the same ellipse. To find out whether two arcs are located back-to-back, we compute the equation of the line connecting centers of two arcs and determine that line intersects one of these arcs or not. For the arc center location, we use the center of ellipse that we obtain during the arc extraction process, i.e. center of the ellipse which is fit to pixels between two consecutive corners. If that line intersects one of the arcs, it means that those arcs are non-coelliptic and cannot be in the same ellipse. In Fig. 4 a couple of scenarios for coelliptic and non-coelliptic arcs are shown in various arc locations and orientations.

Fig. 4.
figure 4

Various scenarios on arc pairing. Successful (a, b) and unsuccessful (c, d) scenarious on arc pairing heuristic.

Full size image

For example, in Fig. 5(a) arcs 0 and 8 are non-coelliptic, since the line connecting the arc centers intersect the arc.

Fig. 5.
figure 5

Decision making on arc pairing. (a) Line connecting the centers of arcs 0 & 8 intersects the arc 0. Thus they cannot belong to the same ellipse. (b) Line connecting the centers of arcs 0 & 7 does not intersect any of the arcs, hence they can belong to the same ellipse.

Full size image

Therefore, both arcs 0 and 8 cannot be parts of the same ellipse and all arc combinations including this arc pair can be eliminated without applying computationally expensive ellipse fit and error computation operations. In the same image, arcs 0 and 7 are coelliptic (Fig. 5(b)) and the line connecting the arc centers does not intersect any of the arcs. In this way, we execute arc pairing operation for all arc pairs which produces the pairing information of all detected arcs. Notice that the asymptotic complexity of arc pairing operation is polynomial \((O(n^2))\) whereas testing the all combinations has exponential complexity \((O(2^n))\). In this way, each eliminated arc pair exponentially reduces the number of arc subsets in the \(2^n-1\) elements solution space.

After the arc combinations have been reduced, we finally obtain a pairing table as shown in Table 1 for the arcs shown in Fig. 3(b). This table shows whether an arc can be combined with any other arc. At the last step, we use this information to eliminate the arc combinations including at least one non-coelliptic arc pair without applying ellipse fit and error computation operations.

Table 1. Arc pairing table for test image in Fig. 5. “C” and “NC” stands for coelliptic and non-coelliptic, and indicates that the arc pair can be joined or not, respectively.
Full size table

Arc Pairing Test: In this step, we use a mathematical algorithm to find out whether two arcs are compatible to join in the same ellipse or not. To do this, we need to compute the intersection points of the arcs and the line connecting two arc centers to apply the proposed arc pairing test. We obtain ellipse center from parametric equation of ellipse by converting Eq. 1. Details of this operation can be found in [1]. Therefore, we first need to find the mutual solution set of the following equations:

$$\begin{aligned} A_{i}x^2 + B_{i}xy + C_{i}y^2 + D_{i}x + E_{i}y + F_{i} = 0 \end{aligned}$$
(1)
$$\begin{aligned} mx + n = y \end{aligned}$$
(2)

where Eq. 1 represents the conic equations for two arcs in the pair and Eq. 2 stands for the line equation which connects centers of two arcs. Using the center coordinates of the two arcs, \(\mathbf {c_1}={(x_1, y_1)}\) and \(\mathbf {c_2}={(x_2, y_2)}\), we simply find the m and n values in the line equation:

$$\begin{aligned} m= & {} \frac{y_2-y_1}{x_2-x_1}\end{aligned}$$
(3)
$$\begin{aligned} n= & {} y_1-mx_1 \end{aligned}$$
(4)

Then we substitute \(mx+n\) in place of y in the conic equation:

$$\begin{aligned} x^2(A_{i}+mB_{i}+m^2C_{i})&+ \nonumber \\ x(nB_{i}+2mnC_{i}+D_{i}+mE_{i})&+ \\ (C_{i}n^2+nE_{i}+F_{i})&= 0\nonumber \end{aligned}$$
(5)

This quadratic equation can be solved with discriminant analysis; \(\varDelta = b^2 - 4ac\) where

$$\begin{aligned}&a_{i} = A_{i}+mB_{i}+m^2C_{i} \end{aligned}$$
(6)
$$\begin{aligned}&b_{i} = nB_{i}+2mnC_{i}+D_{i}+mE_{i} \end{aligned}$$
(7)
$$\begin{aligned}&c_{i} = C_{i}n^2+nE_{i}+F_{i} \end{aligned}$$
(8)

They intersect always at two separate points on ellipse and we consider the point which is the closer one to the center of second arc as shown in Fig. 6. Having the \(\varDelta \) is computed, intersection point(s) \(\mathbf {i_1}=(x_{i_1},y_{i_1})\) and \(\mathbf {i_2}=(x_{i_2},y_{i_2})\) are computed as the following:

$$\begin{aligned} x_{i_1}= & {} \frac{-b_{i}-\sqrt{\varDelta _{i}}}{2a_{i}}\end{aligned}$$
(9)
$$\begin{aligned} x_{i_2}= & {} \frac{-b_{i}+\sqrt{\varDelta _{i}}}{2a_{i}}\end{aligned}$$
(10)
$$\begin{aligned} y_{i_1}= & {} mx_{i_1}+n \end{aligned}$$
(11)
$$\begin{aligned} y_{i_2}= & {} mx_{i_2}+n \end{aligned}$$
(12)
Fig. 6.
figure 6

Chosen intersection point is the closer point to the second arc.

Full size image

If we determine that the line and one of the arc segments intersect, we need to find whether the intersection point actually lies on the arc. Until now, we represent arcs with the conic equation which is obtained with the ellipse fitting operation (see Eq. 1). Note that, obtained equation by ellipse fitting stands for a complete ellipse, however extracted arcs are not complete. Therefore the intersection points that we compute may be on the ellipse contour but may not be lying on the arc actually. To solve this problem, we use delimiting (start and end) angles to validate whether the point is on the arc as shown in Fig. 7(a) and 7(b).

Fig. 7.
figure 7

Representation of elliptical arcs with delimiting angles. (a) Intersection is on the arc. (b) Intersection is not on the arc.

Full size image

In this way, the intersection point which is already on the ellipse contour can be used to determine whether it actually lies on the arc or not. First, we obtain the parametric equation of the ellipse \((\frac{x^2}{a}+\frac{y^2}{b}=1)\) from the conic form Eq. 1 with the steps in [1]. This operation moves the ellipse to the origin by proper translation and rotation transforms. Obviously, the same translation and rotation are applied to the computed intersection point to find the corresponding location in angular form. Once we evaluate the angular positions of starting-ending points of the arc and the intersection point (see Fig. 8), we can check whether the intersection point is in between the delimiting angles and determine the intersection is on the elliptical arc indeed.

Fig. 8.
figure 8

Determining whether the intersection point \({\varvec{i}}=(x',y')\) is in between the delimiting angles.

Full size image

Note that the method employed to find non-coelliptic arc pairs is a heuristic and may fail for some cases. In Fig. 9 two counter examples are presented. Although both pairs satisfy the heuristic, arcs in each pair are incompatible to lie along the same elliptic trajectory.

Fig. 9.
figure 9

Two counter examples of arc pairs that validate the proposed heuristic although they are incompatible to join.

Full size image

Beside the one proposed here, there can also be different ways to determine coelliptic (or non-coelliptic) arcs that proposed algorithm can benefit in reducing number of the arc combinations.

2.4 Detection of Final Ellipses

In this step we reduce the number of ellipse hypotheses composed of all possible arc combinations by using non-coelliptic arc pairs. Those combinations are simply skipped and not tested with computationally expensive ellipse fitting. For combinations consisting of multiple arcs, Taubin method may result in a general conic rather that an ellipse if the arcs are not form an elliptical shape. In such cases, obtained conic equation (Eq. 1) is checked to see if \(B^2-4AC < 0\), otherwise, combination is skipped without fitting error computation.

Eliminating arc subsets including non-coelliptic arc pairs significantly reduces the computation time. In the case of test image shown in Fig. 3(b), we have 16 arcs hence 65535 possible combinations which may yield an ellipse. If we try to check all these arc groups whether they compose an ellipse or not, we would have to execute 65535 computationally expensive ellipse fit and error computation routines. However, with the help of the Table 1 constructed by the proposed heuristic, the algorithm eliminates \(99.42\,\%\) of the arc subsets. Then the detection process ends up by only 382 fitting and error computation operations which 5 of them yield as a valid ellipse (see Fig. 10(b)). Eventually, we become able to check all reasonable arc subsets in the image without sacrificing computational constraints.

After elimination of arc subsets, remaining arc combinations have to pass two tests to be extracted as a valid ellipse. First, ellipse fit is applied to all arc pixels in the current arc group and fitting error is computed. If the resulted error is small, i.e. \(\le 2.0\) pixels, we test the current arc group for the spanned angle ratio (SAR) which is the ratio of the total length of arcs in the group over the perimeter of the yielded ellipse. With the computation of SAR, we can control both the allowed amount of occlusion and the number of false positive detections. For lower values of SAR, ellipses can be extracted with higher amount of occlusions, however, it also enables single and short arcs to be selected as ellipse which tends to an increase on the number of false positives. In the experiments, we set the SAR around 0.5 to make a compromise between the number of false positives and the amount of occlusion. Thus, we also prevent arc combinations consisting of only one arc to be extracted as ellipses with a small part of detected ellipse contour. In SAR computation we also need to calculate ellipse perimeter which does not have an exact formulation. Therefore we employ Ramanujan’s second approximation for this task [23].

Fig. 10.
figure 10

Detected arcs (a) and detected ellipses (b) by arc joining.

Full size image

During the detection process, if an arc combination ends up in a valid ellipse, other combinations which include any arc in the current successful arc combination are eliminated from the solution space. In this manner, we both ensure that each arc can be part of only one ellipse and accelerate the algorithm as the arc combinations to test deplete more quickly. To even boost the procedure, we sort the arc combinations according to the number of arcs in descending order and start processing from crowded combinations. In this way, successful arc combinations which yield a valid ellipse cause more combinations to be removed from the list. The final detected ellipses for the input image Fig. 2(a) are shown in Fig. 10(b).

3 Experimental Results

There are three publicly available datasets for ellipse detection in the literature, i.e. Prasad Dataset, Random Dataset, and Smartphone Dataset [7]. They contain \(198+400+629=1227\) real images including tough ones with severe occlusions. We compared our algorithm with three state-of-the-art ellipse detection algorithms with the datasets mentioned above. The first algorithm is proposed by Prasad et al. and they are the owners of the Prasad Dataset [21]. The second algorithm is proposed by Fornaciari et al. who own remaining two datasets used in the experiments [6]. And the last algorithm is proposed by Libuda et al. [13]. Source codes of these algorithms are also available online [8, 14, 22]. All experiments are performed on a laptop computer with an Intel i7 2.40 GHz processor and 8 GB of RAM.

On the contrary to our algorithm, these three methods have a validation step which obtains the final results with an image specific score threshold value. In this procedure score value is computed using the edge pixels lying under the ellipse contour and it is assigned to each detected ellipse. As a result, this validation scheme effects the algorithm performance as the threshold value is tuned, however our proposed algorithm is purely feature based and does not employ a score thresholding or voting scheme. For a fair comparison, we run each algorithm with the same threshold value for all three datasets and pick the best results. All images in three datasets are gathered together and the best score, which gives the best average F-Measure for all images, is calculated for each algorithm. These threshold values are determined as 0.83, 0.01, and 0.78 for Prasad, Libuda, and Fornaciari, respectively.

The comparison metric is the overlap ratio calculation proposed by [21] and it is also used in [6] as:

$$\begin{aligned} \textit{Overlap Ratio} = 1 - \frac{count(\textsc {XOR}(\epsilon _1,\epsilon _2))}{count(\textsc {OR}(\epsilon _1,\epsilon _2))} \end{aligned}$$
(13)

where \(\epsilon _1\) and \(\epsilon _2\) are respectively the ground-truth ellipse and a valid detection. If a valid detection has at least 0.8 overlap ratio compare to any other ground-truth ellipse, it is counted as a true positive elliptic hypothesis (true positive, TP) and if a ground-truth ellipse does not have a match with any of the detected ellipses, then a miss is found (false negative, FN). The number of actual ellipses is the sum of TP and FN. When the detected ellipse does not have a match with any of the ground-truth ellipses, it corresponds to a false positive (FP). Result of TP + FP corresponds to total number of elliptic hypothesis. As a result, we get Precision and Recall values:

$$\begin{aligned} Precision= & {} \frac{count(TP)}{count(TP+FP)}\end{aligned}$$
(14)
$$\begin{aligned} Recall= & {} \frac{count(TP)}{count(TP+FN)} \end{aligned}$$
(15)

Then, F-Measure is calculated with the following formula:

$$\begin{aligned} F\text {-}Measure ={\frac{2 \times Precision \times Recall}{Precision + Recall}} \end{aligned}$$
(16)

Average F-Measure values for each dataset is shown in Fig. 11.

Fig. 11.
figure 11

Performances of algorithms as F-measure for each dataset.

Full size image

According to results, Prasad performs best in its own dataset and the proposed method is in second place. In Random and Smartphone Datasets, proposed method outperforms all algorithms.

Eventually, proposed method performs best and it has an advantage of detecting all possible ellipses including occlusions in an image and localizing these ellipses perfectly as seen in Fig. 12.Footnote 1 For all 1227 images in 3 datasets, we found that the heuristic avoids 99.58 % of potentially unnecessary fitting operations in average. Note that there are rare situations where heuristic may fail.

Fig. 12.
figure 12

Some results from three datasets for all algorithms. From left to right: (1) Original Image, (2) Ground Truth, (3) Prasad, (4) Libuda, (5) Fornaciari, (6) Proposed.

Full size image

In Table 2 average timing results are presented for the proposed algorithm in detail per image for each dataset.

Table 2. Average execution times of the proposed method with respect to the algorithm steps in milliseconds.
Full size table

The table clearly shows that the total execution time of the proposed algorithm is strictly depends on the last step which runs with respect to the number of elliptical arcs. Timing performance of algorithms were measured in milliseconds and average timing values per image for each dataset is shown in Table 3.

Table 3. Average execution times of algorithms per image for each dataset.
Full size table

The implementations that we use are in C++ for Fornaciari, Libuda and the proposed method, and in MATLAB for Prasad. Timing results show that Fornaciari is the fastest one and its performance is very close to Libuda. In terms of computation speed, the proposed method falls behind the Fornaciari and Libuda, however, it still able to run in real-time. As a model-based method, Prasad is the slowest algorithm by far even though its timing values are divided by 50 due to the computation discrepancies between a typical application written in MATLAB and C++ with respect to the experiments held in [6].

We can list a couple of reasons that slows down our algorithm. First, our algorithm checks all possible combinations of the extracted elliptical arcs and requires exponential runtime with respect to the number of detected arcs. Second, opposite to the other methods, we employ a very precise yet computation intensive method to compute the fitting error.

4 Conclusions

Ellipse detection is an active research problem and there are two main strategies, i.e. model-based and feature-based. State-of-the-art methods in the literature are feature-based ones that extract primitive geometric features and aims to catch ellipses with pipelines that consist of various mathematical and geometric heuristics. This is the mainstream strategy of how recent research studies attempts to solve ellipse detection problem.

In this study an ellipse detection method is proposed, which works by extracting elliptical arcs in an image and then combines these arcs by a join strategy. For n elliptical arcs, there are \(2^n-1\) different arcs combinations such that each may constitute an ellipse. In this way, any elliptical structure can be detected no matter how many arcs it is divided into due to the tough occlusions. On the contrary to the traditional model based methods, our execution time does not depend on the image size. Instead, execution time is exponentially grows as the number of arcs increases. To boost the detection process, an arc pairing operation which eliminates incompatible arc combinations including non-coelliptic arcs is executed. The remaining arc combinations are finally tested with ellipse fitting and error computation.

Experimental results show that the proposed algorithm gives the best results compare to state-of-the-art methods within a reasonable computation time. In addition to the best accuracy results, proposed method works parameter-free as a substantial contribution. In future work, we will further examine the additional ways to eliminate incompatible arc groups by using properties of conics to even accelerate the detection procedure. Moreover, we will attempt to optimize the error calculation method in terms of computation to speed up the algorithm. We also plan to decrease false detection rate by applying a validation step based on Gestalt Theory to improve the accuracy of the method.