FitEllipsoid: a fast supervised ellipsoid segmentation plugin
Abstract
Background
The segmentation of a 3D image is a task that can hardly be automatized in certain situations, notably when the contrast is low and/or the distance between elements is small. The existing supervised methods require a high amount of user input, e.g. delineating the domain in all planar sections.
Results
We present FitEllipsoid, a supervised segmentation code that allows fitting ellipsoids to 3D images with a minimal amount of interactions: the user clicks on a few points on the boundary of the object on 3 orthogonal views. The quantitative geometric results of the segmentation of ellipsoids can be exported as a csv file or as a binary image. The core of the code is based on an original computational approach to fit ellipsoids to point clouds in an affine invariant manner. The plugin is validated by segmenting a large number of 3D nuclei in tumor spheroids, allowing to analyze the distribution of their shapes. User experiments show that large collections of nuclei can be segmented with a high accuracy much faster than with more traditional 2D slice by slice delineation approaches.
Conclusions
We designed a userfriendly software FitEllipsoid allowing to segment hundreds of ellipsoidal shapes in a supervised manner. It may be used directly to analyze biological samples, or to generate segmentation databases necessary to train learning algorithms. The algorithm is distributed as an opensource plugin to be used within the image analysis software Icy. We also provide a Matlab toolbox available with GitHub.
Keywords
Supervised segmentation Ellipsoid Icy pluginAbbreviations
 ADMM
Alternating direction method of multipliers
 CSV
Coma separated values
 LLS
Linear least squares
 SPIM
Single plane imaging microscope
 VTK
Visualization ToolKit
Background
Starting observation
Segmenting ellipsoidal structures in 2D or 3D imagescan be used to characterize the shape of organs, tissues, cells, nuclei or other cell organels [1, 2, 3], or serve as an initialization for more advanced algorithms such as active contours [4, 5, 6, 7].
While fully automatic detection algorithms [8, 9, 10, 11] are probably the ideal tool to limit subjectiveness and time of analysis, existing strategies are not sufficient to provide convincing segmentation results when images suffer from strong degradations (e.g. blur, noise, low resolution) or contain densely packed objects. In addition, automatic methods usually require tuning a few parameters, which may be more time consuming than using a simple supervised segmentation algorithm. Finally, the generation of learning databases or gold standards to test and compare existing segmentation algorithms still requires efficient supervised algorithms. Unfortunately, to the best of our knowledge, there currently exists no such freely available tool, which would benefit many different communities.
Contributions
These few considerations motivated us developing two simple plugins for the Icy image analysis software [12] that are based on a novel computational approach. They are dedicated to fitting ellipses in 2D images or ellipsoids in 3D images. The objectives of this paper are to present the methodology and describe the plugin for 3D ellipsoids.
The core of the algorithm consists in solving the well studied problem of ellipsoid fitting from point clouds. This is a notoriously difficult problem that attracted the attention of researchers from different fields such as computer vision, statistics or numerical analysis, to name a few [13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26]. We propose an original and robust computational algorithm that shares the same spirit as a recent work [26], but significantly outperforms it when the ellipsoids are not centered or anisotropic. An important feature of the proposed algorithm is affine invariance: the point cloud is registered prior to computation, ensuring a robust behavior whatever the shape of the point cloud.
The proposed algorithms are shared not only within the Icy plugin, but also through a set of Matlab codes delivered on a Github repository [27]. To the best of our knowledge, FitEllipsoid is the first opensource toolbox that allows fitting ellipsoids and not more general quadrics (e.g. hyperbolas).
In order to showcase the usefulness of the plugin, we propose to examine the morphology and the distribution of shapes of nuclei in a 3D tumor spheroid. Are they rather elongated, spherical or none of these? Using FitEllipsoid, we obtained the shape of hundreds of nuclei from 3D SPIM images of opticallycleared spheroids.
Implementation
Specifications

permit 3D visualization to allow for visual inspection of the segmentation,

minimize the time required for user interaction. This is particularly important in biology where hundreds or thousands of objects have to be analyzed routinely,

export the results as files that other programs can use for further processing,

deliver a free and opensource software.
Description
The need for a free software dedicated to biomedical imaging oriented us to the recently developed imaging tool Icy [12]. It is based on VTK (Visualization ToolKit) [28], allowing for nice 3D visualization.

A center (3 parameters), three angles of rotations and the length of each axis (3 parameters).

A center (3 parameters), the three axes (9 parameters linked through orthogonality relationships) and the length of each axis (3 parameters).

A center (3 parameters) and a positive symmetric definite matrix (6 coefficients).
Unfortunately, none of these representations can be easily used by a human. For instance, finding the center of the ellipsoid precisely by just looking at the image would result in inaccurate results.
In theory, it is possible to reconstruct an ellipsoid perfectly when knowing as little as 9 points in generic position lying on its surface (see the Additional file 1 for a detailed discussion). The estimation with just 9 points may be unstable to noise, which cannot be avoided due to imperfect pixel selection by the user. We therefore let the user select as many points on the boundary as desired.
Apart from the 3D visualization, the ellipsoids parameters (center, axes orientations and length of axes) are saved in a CSV file that can be read using standard spreadsheets or scientific computing softwares. In addition, it is possible to save a 3D binary image indicating the interior of each ellipsoid.
Tutorial
A video tutorial is provided here http://youtu.be/MjotgTZi6RQ. It describes the main features of the plugin.
Mathematical description
Given a set of n points X=(x_{i})_{1≤i≤n} in R^{d}, where d=2 or 3, the objective of this section is to describe a fast and robust algorithm to fit an ellipsoid to those points. This is a longstanding problem studied in more than 40 journal papers. We refer to the book [25] for a more comprehensive overview. Two main approaches have been proposed to solve it.
where dist(x_{i},E)= infx∈E∥x−x_{i}∥ is the Euclidean distance from the point x_{i} to the ellipsoid E. While this formulation has a clear geometrical meaning, it suffers from being highly nonconvex. Designing global minimization methods is therefore heavy from a computational point of view.
where A∈R^{d×d} is a symmetric positive definite matrix, b∈R^{d} is a vector and c∈R is a scalar.
where \(\mathcal {M}=\left \{(A,b,c)\in \mathbf {R}^{d\times d}\times \mathbf {R}^{d} \times \mathbf {R}, A\succeq 0, \text {Tr}(A)=1\right \}\).

There exists at least one minimizer. Moreover if the number of points n satisfies n≥d(d+1)/2+d and the points are in generic position, then the minimizer is unique, see Fig. 3 in 2D for an illustration and the Additional file 1 for a proof.

The minimizer is covariant to translation and rotation of the input point locations X. More precisely, let \(\hat E\) denote the ellipsoid solution of (4) and \(\hat E'\) denote the ellipsoid obtained by solving (4) with input coordinates X^{′}=(xi′)_{1≤i≤n}, where xi′=Rx_{i}+t,R∈R^{d×d} is a rotation matrix and t∈R^{d} is a translation vector. Then \(\hat E'=R\hat E + t\). The proof of this property is detailed in the Additional file 1.
A numerical algorithm
In [17], Calafiore suggested reformulating (4) as a semidefinite program and using interior point type methods to solve it. This type of algorithm is known to be robust and reliable but is rather hard to implement. Moreover, common primaldual interior point methods [23] have a complexity that does not scale well with the number of input data points n. Based on this observation, Lin and Huang [26] designed a method based on the alternating direction method of multipliers (ADMM) to solve problem (4). While the periteration complexity of this approach is lower than that of interior point methods, the number of iterations is hard to control from a theoretical point of view, and we will show through numerical experiments that it can be very large to yield satisfactory solutions. We propose a more robust approach in what follows.
In 3D, a similar decomposition can be performed, see details in the Additional file 1.
We solve (7) using the DouglasRachford algorithm, which was first proposed by Lions and Mercier [30]. The details are presented in the Additional file 1.
Invariance to affine transformations
Non invariance of the Algorithm
As discussed above, the minimizers of (7) are covariant to isometries. However the algorithm is not, this is illustrated in the Additional file 1. Moreover, the solutions of (4) are not invariant to affine transforms, which would be a desirable property. We propose to address both issues below. Similar ideas were proposed in [15] for the specific case of spheres.
Ensuring invariance using the SVD
In order to ensure invariance of the algorithm we change the coordinate system and work with a point cloud that is centered with covariance matrix equal to the identity. We obtain an ellipsoid in the modified system and finally map it back to the original one. This can be achieved using a singular value decomposition, as explained in the Additional file 1.
Results
Performance of the optimization algorithm
We report in the Additional file 1 experiments and comparisons on 2D data, as well as a robustness to noise study in 3D. We show that our numerical approach never requires more than 200 cheap iterations to reach machine precision, while the unnormalized method can require arbitrarily large computing times depending on the points set location. In addition, we provide comparisons with the simpler LLS algorithm [14] and show an improved robustness to noise.
Segmentation experiments on synthetic data
Segmentation time and accuracy for 3 different users
Time (s)  Center (px)  Angle (deg)  Length (px)  

User 1  25.7 ± 3.1  1.08 ± 0.32  4.6 ± 1.5  1.07 ± 0.41 
User 2  26.1 ± 5.0  0.90 ± 0.35  5.9 ± 2.5  0.54 ± 0.35 
User 3  20.9 ± 3.3  1.01 ± 0.36  4.9 ± 1.7  0.68 ± 0.36 
Notice that the accuracy on the center location and on the axis length is below the resolution of 1.5 pixel. It is then possible to claim that our plugin allows to obtain subresolution results for perfectly ellipsoidal objects in a few seconds. Note that this time is the time required for the user to select the points, the computation time is in fractions of a second. In addition, the angular accuracy is also satisfactory, suggesting that the plugin can be used to analyze the geometry of large collections of objects.
Segmentation experiments on real 3D tumor spheroids
The biological question we addressed was to estimate the distribution of shapes of nuclei, through the estimation of their semiaxes lengths ℓ_{1}≥ℓ_{2}≥ℓ_{3}. Two different experimental conditions have been explored: untreated freelygrown spheroids and spheroids treated for 8h with Latrunculin, a drug targetting actin cytoskeleton.
On the histograms, we can observe that the distribution is denser along the diagonal, and that there is no clear trend towards a prolate or oblate shape. It is however clear that the nuclei are not spherical. The 1D histograms on the right of Fig. 7 shows that the aspect ratio (defined as ℓ_{3}/ℓ_{1}) of a nuclei is shifted towards 1 when going from the treated to the control spheroids. The average aspect ratio is 0.58 for the control spheroid and 0.63 for the Latrunculin treated spheroid.
Overall we see that the plugin allows to distinguish subtle but statistically significant changes of shapes.
Conclusions
FitEllipsoid is a powerful tool for supervised ellipsoids segmentation, with a userfriendly interface. The computational part of the software is based on a novel algorithm that is invariant under affine transforms. It allowed to segment hundreds of cell nuclei in order to analyze statistically their shape.
Footnotes
 1.
We used this geometry since we thought that it would correspond to what is observed in real tumor spheroids. We will see later that this is actually not the case.
Notes
Acknowledgements
The authors wish to thank Guillaume de Brito for helping to implement the 2D version of the plugin, and Bernard Ducommun and Valérie Lobjois for helpful comments on the user interface.
Funding
This work was supported in part by Plan Cancer MIMMOSA and by the OPTIMUS project, funded by fondation Innabiosanté.
Availability of data and materials
The Icy plugin is available at: http://icy.bioimageanalysis.org/plugin/FitEllipsoid and the source code is available at: https://github.com/pierreweiss/FitEllipsoid.
Authors’ contributions
JF and PW led the project, developed the mathematical theory and the principle of the plugin, wrote the paper and performed numerical experiments. BK and PW implemented the plugin. PW made the video tutorial. LG obtained the real 3D images, used the plugin to segment the spheroid, and performed shapes statistics. All authors read and approved the final manuscript.
Ethics approval and consent to participate
Not applicable.
Consent for publication
All authors consent this publication.
Competing interests
The authors declare that they have no competing interests.
Publisher’s Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Supplementary material
References
 1.Lockett SJ, Sudar D, Thompson CT, Pinkel D, Gray JW. Efficient, interactive, and threedimensional segmentation of cell nuclei in thick tissue sections. Cytometry. 1998; 31(4):275–86.CrossRefGoogle Scholar
 2.Okada K, Comaniciu D, Krishnan A. Robust anisotropic gaussian fitting for volumetric characterization of pulmonary nodules in multislice ct. IEEE Trans Med Imaging. 2005; 24(3):409–23.CrossRefGoogle Scholar
 3.Mahdavi S, Salcudean SE. 3d prostate segmentation based on ellipsoid fitting, image tapering and warping. In: Engineering in Medicine and Biology Society, 2008. EMBS 2008. 30th Annual International Conference of the IEEE. Vancouver: IEEE: 2008. p. 2988–2991.Google Scholar
 4.Dufour A, Shinin V, Tajbakhsh S, GuillénAghion N, OlivoMarin JC, Zimmer C. Segmenting and tracking fluorescent cells in dynamic 3d microscopy with coupled active surfaces. IEEE Trans Image Process. 2005; 14(9):1396–410.CrossRefGoogle Scholar
 5.Thevenaz P, DelgadoGonzalo R, Unser M. The ovuscule. IEEE Trans Pattern Anal Mach Intell. 2011; 33(2):382–93.CrossRefGoogle Scholar
 6.Cuingnet R, Prevost R, Lesage D, Cohen LD, Mory B, Ardon R. Automatic detection and segmentation of kidneys in 3d ct images using random forests. In: International Conference on Medical Image Computing and ComputerAssisted Intervention. Berlin: Springer: 2012. p. 66–74.Google Scholar
 7.DelgadoGonzalo R, Chenouard N, Unser M. Splinebased deforming ellipsoids for interactive 3d bioimage segmentation. IEEE Trans Image Process. 2013; 22(10):3926–40.CrossRefGoogle Scholar
 8.OlivoMarin JC. Extraction of spots in biological images using multiscale products. Pattern Recog. 2002; 35(9):1989–96.CrossRefGoogle Scholar
 9.Jaqaman K, Loerke D, Mettlen M, Kuwata H, Grinstein S, Schmid SL, Danuser G. Robust singleparticle tracking in livecell timelapse sequences. Nat Methods. 2008; 5(8):695–702.CrossRefGoogle Scholar
 10.Soubies E, Weiss P, Descombes X. A 3d segmentation algorithm for ellipsoidal shapes. application to nuclei extraction. In: ICPRAMInternational Conference on Pattern Recognition Applications and Methods. Barcelona: ICPRAM: 2013. p. 97.Google Scholar
 11.Zhang W, Fehrenbach J, Desmaison A, Lobjois V, Ducommun B, Weiss P. Structure tensor based analysis of cells and nuclei organization in tissues. IEEE Trans Med Imaging. 2016; 35(1):294–306.CrossRefGoogle Scholar
 12.De Chaumont F, Dallongeville S, Chenouard N, Hervé N, Pop S, Provoost T, MeasYedid V, Pankajakshan P, Lecomte T, Le Montagner Y, et al.Icy: an open bioimage informatics platform for extended reproducible research. Nat Methods. 2012; 9(7):690–6.CrossRefGoogle Scholar
 13.Gander W, Golub GH, Strebel R. Leastsquares fitting of circles and ellipses. BIT Numer Math. 1994; 34(4):558–78.CrossRefGoogle Scholar
 14.Fitzgibbon A, Pilu M, Fisher RB. Direct least square fitting of ellipses. IEEE Trans Pattern Anal Mach Intell. 1999; 21(5):476–80.CrossRefGoogle Scholar
 15.Nievergelt Y. Hyperspheres and hyperplanes fitted seamlessly by algebraic constrained total leastsquares. Linear Algebra Appl. 2001; 331(1):43–59.CrossRefGoogle Scholar
 16.Ahn SJ, Rauh W, Warnecke HJ. Leastsquares orthogonal distances fitting of circle, sphere, ellipse, hyperbola, and parabola. Pattern Recogn. 2001; 34(12):2283–303.CrossRefGoogle Scholar
 17.Calafiore G. Approximation of ndimensional data using spherical and ellipsoidal primitives. IEEE Trans Syst Man Cybern Syst Hum. 2002; 32(2):269–78.CrossRefGoogle Scholar
 18.Ahn SJ, Rauh W, Cho HS, Warnecke HJ. Orthogonal distance fitting of implicit curves and surfaces. IEEE Trans Pattern Anal Mach Intell. 2002; 24(5):620–38.CrossRefGoogle Scholar
 19.Markovsky I, Kukush A, Van Huffel S. Consistent least squares fitting of ellipsoids. Numer Math. 2004; 98(1):177–94.CrossRefGoogle Scholar
 20.Li Q, Griffiths JG. Least squares ellipsoid specific fitting. In: Geometric Modeling and Processing, 2004. Proceedings. Beijing: IEEE: 2004. p. 335–340.Google Scholar
 21.Chernov N, Lesort C. Least squares fitting of circles. J Math Imaging Vis. 2005; 23(3):239–52.CrossRefGoogle Scholar
 22.Kleinsteuber M, Hüper K. Approximate geometric ellipsoid fitting: A cgapproach. In: Recent Advances in Optimization and Its Applications in Engineering. Berlin: Springer: 2010. p. 73–82.Google Scholar
 23.Ying X, Yang L, Zha H. A fast algorithm for multidimensional ellipsoidspecific fitting by minimizing a new defined vector norm of residuals using semidefinite programming. IEEE Trans Pattern Anal Mach Intell. 2012; 34(9):1856–63.CrossRefGoogle Scholar
 24.Saunderson J, Chandrasekaran V, Parrilo PA, Willsky AS. Diagonal and lowrank matrix decompositions, correlation matrices, and ellipsoid fitting. SIAM J Matrix Anal Appl. 2012; 33(4):1395–416.CrossRefGoogle Scholar
 25.Kanatani K, Sugaya Y, Kanazawa Y. Ellipse fitting for computer vision: implementation and applications. Synth Lect Comput Vis. 2016; 6(1):1–141.CrossRefGoogle Scholar
 26.Lin Z, Huang Y. Fast multidimensional ellipsoidspecific fitting by alternating direction method of multipliers. IEEE Trans Pattern Anal Mach Intell. 2016; 38(5):1021–6.CrossRefGoogle Scholar
 27.Matlab code repository. https://github.com/pierreweiss/FitEllipsoid. Accessed 20 Feb 2019.
 28.Schroeder WJ, Lorensen B, Martin KM. The visualization toolkit: an objectoriented approach to 3D graphics. Kitware publisher. 2004. http://cds.cern.ch/record/798217/files/1930934122_TOC.pdf.
 29.Heller GV, Cerqueira MD, Weissman NJ, Dilsizian V, Jacobs AK, Kaul S, Laskey WK, Pennell DJ, Rumberger JA, Ryan T, et al.Standardized myocardial segmentation and nomenclature for tomographic imaging of the heart: a statement for healthcare professionals from the cardiac imaging committee of the council on clinical cardiology of the american heart association. J Nucl Cardiol. 2002; 9(2):240–5.CrossRefGoogle Scholar
 30.Lions PL, Mercier B. Splitting algorithms for the sum of two nonlinear operators. SIAM J Numer Anal. 1979; 16(6):964–979.CrossRefGoogle Scholar
Copyright information
Open Access This article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated.