Local edge-enhanced active contour for accurate skin lesion border detection
Abstract
Background
Dermoscopy is one of the common and effective imaging techniques in diagnosis of skin cancer, especially for pigmented lesions. Accurate skin lesion border detection is the key to extract important dermoscopic features of the skin lesion. In current clinical settings, border delineation is performed manually by dermatologists. Operator based assessments lead to intra- and inter-observer variations due to its subjective nature. Moreover it is a tedious process. Because of aforementioned hurdles, the automation of lesion boundary detection in dermoscopic images is necessary. In this study, we address this problem by developing a novel skin lesion border detection method with a robust edge indicator function, which is based on a meshless method.
Result
Our results are compared with the other image segmentation methods. Our skin lesion border detection algorithm outperforms other state-of-the-art methods. Based on dermatologist drawn ground truth skin lesion borders, the results indicate that our method generates reasonable boundaries than other prominent methods having Dice score of 0.886 ±0.094 and Jaccard score of 0.807 ±0.133.
Conclusion
We prove that smoothed particle hydrodynamic (SPH) kernels can be used as edge features in active contours segmentation and probability map can be employed to avoid the evolving contour from leaking into the object of interest.
Keywords
Dermoscopy Skin lesion segmentation Skin lesion border detectionAbbreviations
- AC
Active contour
- ESF
Edge stop function
- FN
False negative
- FP
False positive
- KNN
K-nearest neighbors algorithm
- LEEAC
Local Edge-Enhanced Active Contour
- LSE
Level set evolution
- LSF
Level set function
Probability density function
- RD
Reaction Diffusion
- ROI
Region of interest
- SPH
Smoothed particle hydrodynamics
- SVM
Support Vector Machines
- TN
True negative
- TP
True positive
Background
Image segmentation is a process of finding meaningful regions in an image. Many of the image processing and analysis methods rely on the accuracy of a proper image segmentation method. In dermoscopic image processing and analysis, image segmentation corresponds to detection of lesion border precisely. Accuracy of skin lesion border detection in dermoscopic images is critical [1] to extract important structural features, such as irregularity, symmetry, and abrupt border cutoff; and dermoscopic features, such as globules, blue-white areas, and atypical pigment network. However, automated border detection is a challenging task especially among the lesions with a) fuzzy borders, b) low contrast between lesion boundary and surrounding skin, c) low color and texture variations, and d) existence of artifacts such as sweat, hair, and blood vessels.
In the USA approximately 3.5 million people are diagnosed with skin cancers in a year. Skin cancer is rarely fatal except for melanoma, which is malignancy of melanocytes [2]. In its January 2017 report, American Cancer Society estimates that in the U.S. 87,110 adults will be diagnosed with melanoma, and approximately 9730 cases are expected to be fatal [2]. Since melanoma develops in melanocytes, which are special cells on epidermis, it can be detected by visual inspection of skin. Early diagnosis and treatment of melanoma are key to increase chances of survival [3]. However, high rate of false-negative diagnosis in melanoma cases poses challenge for early treatments [3].
Dermoscopy is an effective and noninvasive imaging modality in diagnosis of skin cancers, especially for pigmented lesions. It enables clinicians to closely examine predefined diagnostic features that are not seen otherwise. For this very reason, accurate skin lesion border detection is key to extract important dermoscopic features of the lesion. These features are evaluated to detect melanoma and other skin diseases [4, 5, 6, 7]. It is shown that dermoscopy increases accuracy of naked eye examination of clinicians [8]. There are various methods used to segment skin lesions [9]. One of these methods is using the algorithm of active contour.
Active contour based methods (a.k.a. snakes) are widely used in image segmentation. These methods are also used in lesion segmentation [10, 11, 12, 13]. Active contours can be categorized into two main groups: edge-based methods [14] and region-based methods [15]. The former employs edge information [14] while the latter selects a region feature to adjust the movement of active contour toward the boundary of object(s) to be segmented [16, 17]. Active contour methods start with a curve around the region of interest (ROI) to be detected, the curve moves toward its interior normals and has to stop on the boundary of the ROI. While some parameters control the smoothness of the contour, others attract the contour toward the center of the ROI. The most optimum state of the contour is selected using an iterative process, in which internal and external energy functions reach equilibrium and stop the further iterations. Edge based active contours use level sets and have the advantage of handling complicated shapes. However, their parameters are not naturally connected to visual features; therefore, very difficult to use for naive users. The edge based active contours are found more suitable for lesion boundary detection [10, 11]. On the other hand, for border detection of skin lesions, active contours were reported [13] to have slower computation time since they require to solve the underlying optimization problem. In general, for an active contour method to achieve high accuracy for skin lesion detection, the lesion is expected to have strong edges to stop at the border.
Edge-based active contour methods suffer from poorly defined edges, whereas region-based methods are sensitive to inhomogeneity of image intensities. For the images with weakly formed object boundaries (e.g., skin lesions with fuzzy borders), the edge-stop function (ESF) fails to cease the curve move and as a result contour leaks through the object border [18]. Thus, they suffer in skin lesion segmentation when morphological and color variations exist. Specifically, for the cases where skin lesion doesn’t have a strong border (e.g., fuzzy borders, or insufficient contrast between lesion boundary and surrounding skin), active contour methods fail to find lesion borders accurately. One of the main contributions of this study is to overcome this failing point. The proposed method of segmentation starts with a novel local edge extraction algorithm using smoothed particle hydrodynamics (SPH). Using the edge information coming from SPH, object border is strengthened using geodesic distances that involves probability of pixels (whether they are foreground, background, or border pixels). Later we give the object edge information into active contours to accurately detect skin lesion borders. Due to the additional edge information given to active contours, they become robust to leaks.
Methods
This section reviews the developed computational core for lesion segmentation. Figure 1 shows its processing steps. Each of these steps are detailed in the following subsections.
Filtering
where g∇(I) represents the diffusion coefficient, ∇(I) gradient map of the image I. As can be inferred from the Eq. 1, ∇(I) and g are inversely proportional to maintain the notion of Perona-Malik method. γ is a constant to control the sensitivity against gradients on the image domain. Diffusion process will be declined at the regions where ∣∇I∣≫γ. Without smoothing, initial contour is trapped by noise(s) (weak edges) and cannot delineate the lesion border. After denoising completed, SPH kernel is used to overcome active contour leaking problems, especially for the fuzzy borders of skin lesions.
A new local edge extraction method: SPH
In Eq. 2, the contribution of a particle to a physical property are weighted by their proximity to the particle of interest and its density. Commonly used kernel functions deploy cubic spline and Gaussian functions. Cubic spline is exactly zero for particles located at a distance equals two times of the smoothing length, 2h. This decreases the computational cost by discarding the particles’ minor contributions to the interpolation.
Kernel approximation
where I is the image, G is denoising function, and ∣∇(G_{α}∗I)∣^{p} is the edge map produced by image gradients. In this paper, we used SPH formulations that are to calculate surface normals, instead of image gradients. Edge indicator functions are commonly represented by g as shown in Eq. 12. Next subsection reviews the mathematical pipeline that robustly minimizes the obtained g function.
Probability map for stronger edges
Probability map
To address the drawback seen at traditional ESFs in edge-based AC, this study introduces a computational pipeline which is based on constructing a robust ESF that utilizes probability scores (between 0−1) rather than predicted class labels provided by a classifier as given in [18]. Probability scores indicate whether a pixel is foreground or background pixel. These scores are computed in O(n) where n is the number of pixels. Whereas Pratondo et al. [18] uses fuzzy KNN or SVM classifiers to predict whether a pixel is a foreground or background pixel in O(n^{2}). Classifier scores (between 0−1) at boundary pixels tend to be close to zero. So far, considerable amount of work has been done to have ESF collaborate with the likelihood of pixels (whether a pixel belongs to background, foreground, or edge) to avoid contour leakages through the border. Pratondo et al. [18] extended methods of [25, 26] that rely on only class probability using Bayes rule, by utilizing the probability scores from fuzzy KNN and SVM classifiers.
where w^{i} represents the weights which are to impose the channel (i∈N_{c}) capacity in terms of abstracting the foreground from background, and N_{c} represents the channel number.
However, image segmentation merely relies on PDF since probability map is potent to fail. As seen in Fig. 5, pixel X, which locates inside of the object of interest, has similar intensity features with the background. In order to address this problem, [27] combined the PDF distribution with geodesic distances of each pixels to these boxes. Following subsection reviews the geodesic distances concept offered by [27].
Geodesic distances
Geodesic distances are weighted distances. To expand, assume that going to city of B from city of A takes two hours, and distance between A and B is 100 km. Whereas, going to city of C from city of A takes four hours while the distance between A and C is only 50 km. Since C is a city of another country, traveling from A to C requires more effort. Hence, the weight of passing a country border increased the travel time from city A to city C.
Here, pixel assignment to background or foreground is performed by comparing the minimum distance to the background and the minimum distance to the foreground. Let us select a pixel X; if this pixel’s minimum distance to the background is less than the minimum distance to the foreground, then this pixel is assigned to background, or vice versa.
For instance, if ∇P_{F,B}(x) applies more weight, which means more probability change along the path, in Eq. 19, that yields increase in distance and decrease the possibility of being foreground. Consequently, pixel labeling is conducted by comparing minimum of the distances of D_{F}(x) (foreground) and D_{B}(x) (background) which are represented in Eq. 20. All of these computations toward generating the probability map are performed in linear time. Interested readers are referred to [27], for more details.
Note that resulting probability maps are used to further strengthen edge indicator function using the formulations given in [18]. Therefore, the edge indicator function will be more robust for the active contour guided image segmentation. Next subsection reviews how we minimize the obtained pixel probability matrix.
Minimizing probability matrix
The equations given in Eqs. 21 and 22 help us minimize the edge indicator function even at the poorly defined object boundaries for the active contours guided segmentation. Therefore, contour evolution will terminate at the desired boundaries.
Level set configuration
where F represents evolution speed and ∇ is gradient operator.
where d_{p} is obtained from a potential function derived as d_{p}(s)=p^{′}(s)/s, δ_{ε} is the Dirac delta function, and μ, λ and β are constants to weight data terms in Eq. 26. The first term on the right side of Eq. 26 is the distance regularization term, the second term represents the length term, and the third term is the area term.
where, ϕ represents the level set function, ϕ^{n} equals to \(\phi ^{n+\frac {1}{2}}\) (for the second raw), τ_{1} and τ_{2} are time steps of the gradient descent which is to solve the Eq. 27, κ is curvature of the level set function, |∇ϕ^{n}| is magnitude of the gradient of the level set function, Δϕ^{n} is Laplacian of the level set function, g_{new} is the new edge indicator function, and v is a constant to adjust propagation velocity of the level set function. Next section presents the results of our segmentation method including the comparisons with state-of-the art methods.
Results and discussion
We tested our novel skin lesion border detection method (LEEAC) on the data set that has 100 dermoscopy images provided by [32]. Note that we kept the images in their original sizes to avoid data losses due to down-sampling. We utilized the level set implementation given in [31]. To segment an image, our method requires two patches (boxes); one for background and one for foreground. Then probability map is generated based on the pixels bounded by these rectangular areas. Other algorithms [18, 31, 33] used Gaussian filtering to denoise the data set in their applications, which requires adjusting the standard deviation of Gaussian filter in order to avoid leakages (small values of standard deviation) and long delays in border detection (large values of standard deviation). In this context, delay refers that the initial contour trapped by artifacts in dermoscopy images such as hairs and/or sweat/bubble, ruler markings etc., which cast strong edges on the images.
In SPH map, we used 6^{th} polynomial kernel and selected the smoothing range, h as 1. The default parameter values for level set scheme are set as τ_{1}=0.3, which is the time-step for level set evolution equation; τ_{2}=0.01, which is the time-step of the equation for diffusion regularization; and v=0.7, which is to adjust the speed of curve evolution toward the skin lesion boundary. Iteration loop is stopped when polygon area of the closed contour does not show change more than 10 units compared to area of the contour in previous iteration. In order to conduct a fair comparison, we changed the segmentation configuration given in [18] from [28], to [14], otherwise nested iterations in the implementation of [28] increases computational time drastically. The algorithm of [28] generates inaccurate segmentation results. Note that, in region based segmentation [31, 33] output may contain regions which are not part of the lesion; however, these regions are represented by similar intensity features to the skin lesion. This ultimately decreases their segmentation accuracy. While evaluating our method, we did not take any post-segmentation action such as removing irrelevant connected components (dilation & eroding) far from the lesion to abstract the lesion alone.
Evaluation of Segmentation Methods with respect to the Ground Truth
Method | Dice | Jaccard | P-value |
---|---|---|---|
LEEAC | 0.8866 ±0.0944 | 0.8074 ±0.1339 | 8.16e-14 |
Zhang et al. [31] | 0.8640 ±0.1453 | 0.7838 ±0.1856 | 7.33e-11 |
Li et al. [33] | 0.8565 ±0.1681 | 0.7779 ±0.2021 | 2.01e-11 |
Mete et al. [13] | 0.8692 ±0.0652 | 0.7743 ±0.0985 | 1.08e-14 |
Pratondo et al. [18] with SVM | 0.8395 ±0.1609 | 0.7511 ±0.1985 | 3.99e-08 |
Pratondo et al. [18] with KNN | 0.5914 ±0.3564 | 0.5038 ±0.3392 | 8.36e-04 |
We have observed that selection of noise filtering technique has a tremendous impact on the duration of segmentation. If we consider images with their actual sizes, methods of [18, 31, 33] used merely Gaussian filtering for denoising purpose and average time for segmenting an image with the size of 484x737 takes more than 10 min. Average time for training KNN and SVM in the approach of [18] is almost an hour. Segmentation method of Mete et al. [13] involves density based clustering, hence it is very sensitive to parameters and computationally expensive.
Conclusions
This study introduces an accurate skin lesion border detection method based on active contours. One of the main problems of active contours is leaking problem. This problem becomes especially visible in dermoscopy images when there are fuzzy lesion borders and/or dermoscopic artifacts, such as hair and water. When such features exist, active contour is not able to properly find skin lesions or region of interest. We overcome these problems by introducing SPH kernels and probability maps into active contours (called LEEAC). This in turn removed leaking problems and increased accuracy of segmentation. We tested our approach on 100 dermoscopy images and compared our results with the state of the art methods. LEEAC outperforms other prominent methods as reported in the “Results and discussion” section.
Notes
Acknowledgements
We would like to thank Dr. Sreekanth Arikatla from Kitware, Inc. for providing insights on smoothed particle hydrodynamics. We also would like to thank Dr. Deepak Chittajallu from Kitware, Inc. for validating accuracy of our results and checking mathematical notations in this study.
Funding
This study is supported by the Arkansas INBRE program, with an award# P20 GM103429 from the National Institutes of Health/the National Institute of General Medical Sciences (NIGMS).
Availability of data and material
Not applicable.
About this supplement
This article has been published as part of BMC Bioinformatics Volume 20 Supplement 2, 2019: Proceedings of the 15th Annual MCBIOS Conference. The full contents of the supplement are available online at https://bmcbioinformatics.biomedcentral.com/articles/supplements/volume-20-supplement-2.
Authors’ contributions
SK, TH, and MM conceived the study. MB did all experiments and implementations. HKW guided SK in skin lesion understanding. KI helped MB in development of active contour algorithms. All authors read and approve the manuscript.
Ethics approval and consent to participate
Not applicable.
Consent for publication
Not applicable.
Competing interests
The authors declare that they have no competing interests.
Publisher’s Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
References
- 1.Celebi ME, Iyatomi H, Schaefer G, Stoecker WV. Lesion border detection in dermoscopy images. CoRR. 2010; abs/1011.0640. http://arxiv.org/abs/1011.0640.Google Scholar
- 2.American Cancer Society’s. Cancer Facts. Am Cancer Soc. 2018:1–76. Accessed: 2017-11-21. https://www.cancer.org/content/dam/cancer-org/research/cancer-facts-and-statistics/annual-cancer-facts-and-figures/2018/cancer-facts-and-figures-2018.pdf.
- 3.David B T. Trends in pathology malpractice claims. Am J Surg Pathol. 2012; 36(1):1–5. https://doi.org/10.1097/PAS.0b013e31823836bb.CrossRefGoogle Scholar
- 4.di Meo N, Stinco G, Bonin S, Gatti A, Trevisini S, Damiani G, Vichi S, Trevisan G. Cash algorithm versus 3-point checklist and its modified version in evaluation of melanocytic pigmented skin lesions: The 4-point checklist. J Dermatol. 2016; 43(6):682–5. https://doi.org/10.1111/1346-8138.13201.CrossRefGoogle Scholar
- 5.Walter F, Provest A, Vasconcelos J, Burrows PHN, Morris H, Kinmonth A, Emery J. Using the 7-point checklist as a diagnostic aid for pigmented skin lesions in general practice: a diagnostic validation study. Br J Gen Pract. 2013; 63(610):345–53. https://doi.org/10.3399/bjgp13X667213.CrossRefGoogle Scholar
- 6.Nachbar F, Stolz W, Merkle T. The abcd rule of dermatoscopy. J Am Acad Dermatol. 1994; 30:551–9.CrossRefGoogle Scholar
- 7.Stolz W, Riemann A, Cognetta A, Pillet L, Abmayr W. Abcd rule of dermatoscopy: a new practical method for early recognition of malignant melanoma. Eur J Dermatol. 1994; 4:521–7.Google Scholar
- 8.Vestergaard M, Macaskill P, Holt P, Menzies S. Dermoscopy compared with naked eye examination for the diagnosis of primary melanoma: a meta-analysis of studies performed in a clinical settings. Br J Dermatology. 2008; 159(3):669–76. https://doi.org/10.1111/j1365-2133.2008.08713.x.Google Scholar
- 9.Celebi EM, Wen Q, Iyatomi H, Shimizu K, Zhou H, Schaefer G. A state of art survey on lesion border detection in dermoscopy images In: Celebi E, Mendoca T, Marquez JS, editors. Dermoscopy Image Analysis. Chap. 4. Baco Raton, FL: CRC Press: 2015. p. 97–130.CrossRefGoogle Scholar
- 10.Erkol B, Moss H, Stanley J, William S, Hvatum E. Automatic lesion boundary detection in dermoscopy images using gradient vector flow snakes. Skin Res Technology. 2005; 1(1):17–26. https://doi.org/10.1111/j.1600-0846.2005.00092.x.CrossRefGoogle Scholar
- 11.Abbas Q, Fondon I, Sarmiento A, Emre MC. An improved segmentation method for non-melanoma skin lesions using active contour model; 2014, pp. 193–200. https://doi.org/10.1007/978-3-319-1175-3_22.Google Scholar
- 12.Kass M, Witkin A TD. Snakes: Active contour models. Int J Comput Vis. 1998; 1(4):321–31.CrossRefGoogle Scholar
- 13.Mete M, Sirakov NM. Lesion detection in demoscopy images with novel density-based and active contour approaches. BMC Bioinforma. 2010; 11(6):23. https://doi.org/10.1186/1471-2105-11-S6-S23.CrossRefGoogle Scholar
- 14.Caselles V, Kimmel R, Sapiro G. Geodesic active contours. Int J Comput Vis. 1997; 22(1):61–79. https://doi.org/doi:10.1023/A:1007979827043.CrossRefGoogle Scholar
- 15.Vese LA, Chan TF. A multiphase level set framework for image segmentation using the mumford and shah model. Int J Comput Vis. 2002; 50(3):271–93. https://doi.org/doi:10.1023/A:1020874308076.CrossRefGoogle Scholar
- 16.Chan T, Vese L. In: Nielsen M, Johansen P, Olsen OF, Weickert J, (eds).An Active Contour Model without Edges. Berlin, Heidelberg: Springer; 1999, pp. 141–51. https://doi.org/10.1007/3-540-48236-9_13. https://doi.org/10.1007/3-540-48236-9_13.Google Scholar
- 17.Mumford D, Shah J. Optimal approximations by piecewise smooth functions and associated variational problems. Commun Pur Appl Math. 1989; 42(5):577–685. https://doi.org/10.1002/cpa.3160420503.CrossRefGoogle Scholar
- 18.Pratondo A, Chui CK, Ong SH. Robust edge-stop functions for edge-based active contour models in medical image segmentation. IEEE Signal Proc Lett. 2016; 23(2):222–6. https://doi.org/10.1109/LSP.2015.2508039.CrossRefGoogle Scholar
- 19.RIDLER T. Picture thresholding using an iterative selection method. IEEE Trans Syst Man Cybern. 1978; 8(8):630–2. https://doi.org/10.1109/TSMC.1978.4310039.CrossRefGoogle Scholar
- 20.Perona P, Malik J. Scale-space and edge detection using anisotropic diffusion. IEEE Trans Pattern Anal Mach Intell. 1990; 12(7):629–39. https://doi.org/10.1109/34.56205.CrossRefGoogle Scholar
- 21.Monaghan JJ. Smoothed particle hydrodynamics. Rep Prog Phys. 2015; 68(8):1703–59. https://doi.org/doi:10.1088/0034-4885/68/8/R01.CrossRefGoogle Scholar
- 22.Kelager M. Lagrangian fluid dynamics using smoothed particle hydrodynamics. University of Copenhagen: Department of Computer Science; 2006. http://image.diku.dk/projects/media/kelager.06.pdf.Google Scholar
- 23.Liu MB, Liu GR. Smoothed particle hydrodynamics (sph): an overview and recent developments. Arch Comput Methods Eng. 2010; 17(1):25–76. https://doi.org/10.1007/s11831-010-9040-7.CrossRefGoogle Scholar
- 24.Müller M, Charypar D, Gross M. Particle-based fluid simulation for interactive applications; 2003, pp. 154–9.Google Scholar
- 25.Smeets D, Loeckx D, Stijnen B, Dobbelaer BD, Vandermeulen D, Suetens P. Semi-automatic level set segmentation of liver tumors combining a spiral-scanning technique with supervised fuzzy pixel classification. Med Image Anal. 2010; 14(1):13–20. https://doi.org/10.1016/j.media.2009.09.002.CrossRefGoogle Scholar
- 26.Wu J, Yin Z, Xiong Y. The fast multilevel fuzzy edge detection of blurry images. IEEE Signal Proc Lett. 2007; 14(5):344–7. https://doi.org/10.1109/LSP.2006.888087.CrossRefGoogle Scholar
- 27.Protiere A, Sapiro G. Interactive image segmentation via adaptive weighted distances. IEEE Trans Image Process. 2007; 16(4):1046–57. https://doi.org/10.1109/TIP.2007.891796.CrossRefGoogle Scholar
- 28.Li C, Xu C, Gui C, Fox MD. Distance regularized level set evolution and its application to image segmentation. IEEE Trans Image Process. 2010; 19(12):3243–54. https://doi.org/10.1109/TIP.2010.2069690.CrossRefGoogle Scholar
- 29.Chunming L, Chenyang X, Changfeng G, Fox MD. Level set evolution without reinitialization: a new variational formulation. In: Proc. IEEE Computer Soc. Conf. Computer Vision and Pattern Recognition CVPR’05: 2005. p. 430–436.Google Scholar
- 30.Malladi R, Sethian JA, Vemuri BC. Shape modeling with front propagation: a level set approach. IEEE Trans Pattern Anal Mach Intell. 1995; 17(2):158–75. https://doi.org/10.1109/34.368173.CrossRefGoogle Scholar
- 31.Zhang K, Zhang L, Song H, Zhang D. Reinitialization-free level set evolution via reaction diffusion. IEEE Trans Image Process. 2013; 22(1):258–71. https://doi.org/10.1109/TIP.2012.2214046.CrossRefGoogle Scholar
- 32.Argenziano G, Soyer HP, De Giorgio V, Piccolo D, Carli P, Delfino M, Ferrari A, Hofmann-Wellenhof R, Massi D, Mazzocchetti G, Scalvenzi M, Wolf IH. Interactive atlas of dermoscopy.Edra Medical Publishing & New Media; 1999. p. 208. ISBN: 8886457308, 1999, 208 pages. (Book & CD/Web Resource) Interactive multimedia. 5.25 in. disc. https://espace.library.uq.edu.au/view/UQ:229410.
- 33.Li C, Kao C, Gore JC, Ding Z. Implicit active contours driven by local binary fitting energy; 2007. https://doi.org/10.1109/CVPR.2007.383014.
Copyright information
Open Access This article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The Creative Commons Public Domain Dedication waiver(http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated.