Skip to main content
SpringerLink
Log in

Image segmentation evaluation: a survey of methods

  • Published:
Artificial Intelligence Review Aims and scope Submit manuscript

Abstract

Image segmentation is a prerequisite for image processing. There are many methods for image segmentation, and as a result, a great number of methods for evaluating segmentation results have also been proposed. How to effectively evaluate the quality of image segmentation is very important. In this paper, the existing image segmentation quality evaluation methods are summarized, mainly including unsupervised methods and supervised methods. Based on hot issues, the application of metrics in natural, medical and remote sensing image evaluation is further outlined. In addition, an experimental comparison for some methods were carried out and the effectiveness of these methods was ranked. At the same time, the effectiveness of classical metrics for remote sensing and medical image evaluation is also verified.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11

Similar content being viewed by others

References

  • Angulo J, Velasco-Forero S, Chanussot J (2009) Multiscale stochastic watershed for unsupervised hyperspectral image segmentation. In: 2009 IEEE international geoscience and remote sensing symposium, vol 3, pp III-93–III-96

  • Arhid K, Bouksim M, Zakani FR, Aboulfatah M, Gadi T (2016) New evaluation method using sampling theory to evaluate 3D segmentation algorithms. In: ElMohajir M, Chahhou M, AlAchhab M, ElMohajir BE (eds) 2016 4th IEEE international colloquium on information science and technology (CIST), pp 410–415

  • Aspert N, Santa-Cruz D, Ebrahimi T (2002) Mesh: Measuring errors between surfaces using the Hausdorff distance. In: Proceedings of the IEEE international conference on multimedia and expo, vol I and II, pp 705–708. https://doi.org/10.1109/ICME.2002.1035879

  • Benes M, Zitova B (2015) Performance evaluation of image segmentation algorithms on microscopic image data. J Microsc 257(1):65–85. https://doi.org/10.1111/jmi.12186

    Article  Google Scholar 

  • Berezsky O, Melnyk G, Batko Y, Pitsun O (2016) Regions matching algorithms analysis to quantify the image segmentation results. In: 2016 XITH international scientific and technical conference computer sciences and information technologies (CSIT), pp 33–36

  • Bernard O, Bosch JG, Heyde B (2016) Standardized evaluation system for left ventricular segmentation algorithms in 3D echocardiography. IEEE Trans Med Imaging 35(4):967–977. https://doi.org/10.1109/TMI.2015.2503890

    Article  Google Scholar 

  • Boeck S, Immitzer M, Atzberger C (2017) On the objectivity of the objective function-problems with unsupervised segmentation evaluation based on global score and a possible remedy. Remote Sens 9(8):2017. https://doi.org/10.3390/rs9080769

    Article  Google Scholar 

  • Borsotti M, Campadelli P, Schettini R (1998) Quantitative evaluation of color image segmentation results. Pattern Recognit Lett 19(8):741–747. https://doi.org/10.1016/S0167-8655(98)00052-X

    Article  MATH  Google Scholar 

  • Cai Z, Liang Y, Huang H (2017) Unsupervised segmentation evaluation: an edge-based method. Multimed Tools Appl 76(8):11097–11110. https://doi.org/10.1007/s11042-016-3542-8

    Article  Google Scholar 

  • Cappabianco FAM, de Miranda PAV, Udupa JK (2017) A critical analysis of the methods of evaluating MRI brain segmentation algorithms. In: 2017 IEEE international conference on image processing (ICIP), pp 3894–3898

  • Cappabianco FAM, Ribeiro PFO, de Miranda PAV, Udupa JK (2019) A general and balanced region-based metric for evaluating medical image segmentation algorithms. In: 2019 IEEE international conference on image processing (ICIP), pp 1525–1529

  • Cardoso J, Corte-Real L (2005) Toward a generic evaluation of image segmentation. IEEE Trans Image Process 14(11):1773–1782. https://doi.org/10.1109/TIP.2005.854491

    Article  Google Scholar 

  • Chabrier S, Emile B, Laurent H, Rosenberger C, Marche P (2004) Unsupervised evaluation of image segmentation application to multi-spectral images. In: Proceedings of the 17th international conference on pattern recognition, vol 1, pp 576–579. https://doi.org/10.1109/ICPR.2004.1334206

  • Chang H-H, Zhuang AH, Valentino DJ, Chu W-C (2009) Performance measure characterization for evaluating neuroimage segmentation algorithms. Neuroimage 47(1):122–135. https://doi.org/10.1016/j.neuroimage.2009.03.068

    Article  Google Scholar 

  • Chen Z, Zhu H (2019) Visual quality evaluation for semantic segmentation: subjective assessment database and objective assessment measure. IEEE Trans Image Process 28(12):5785–5796

    Article  MathSciNet  Google Scholar 

  • Chen Y, Ming D, Zhao L, Lv B, Zhou K, Qing Y (2018) Review on high spatial resolution remote sensing image segmentation evaluation. Photogramm Eng Remote Sens 84(10):629–646. https://doi.org/10.14358/PERS.84.10.629

    Article  Google Scholar 

  • Chen H, Wang S (2004) The use of visible color difference in the quantitative evaluation of color image segmentation. In: 2004 IEEE international conference on acoustics, speech, and signal processing, vol III, pp 593–596

  • Chouhan SS, Kaul A, Singh UP (2018) Soft computing approaches for image segmentation: a survey. Multimed Tools Appl 77(21):28483–28537. https://doi.org/10.1007/s11042-018-6005-6

    Article  Google Scholar 

  • Correia P, Pereira F (2003) Objective evaluation of video segmentation quality. IEEE Trans Image Process 12(2):186–200. https://doi.org/10.1109/TIP.2002.807355

    Article  Google Scholar 

  • Cruz H, Eckert M, Meneses JM, Martinez JF (2017) Fast evaluation of segmentation quality with parallel computing. Sci Program. https://doi.org/10.1155/2017/5767521

    Article  Google Scholar 

  • Dey N, Rajinikanth V, Ashour AS (2018) Tavares JMRS social group optimization supported segmentation and evaluation of skin melanoma images. Symmetry-Basel. https://doi.org/10.3390/sym10020051

    Article  MATH  Google Scholar 

  • Dogra DP, Majumdar AK, Sural S (2012) Evaluation of segmentation techniques using region area and boundary matching information. J Vis Commun Image Represent 23(1):150–160. https://doi.org/10.1016/j.jvcir.2011.09.005

    Article  Google Scholar 

  • Domingo J, Dura E, Goceri E (2016) Iteratively learning a liver segmentation using probabilistic atlases: preliminary results. In: 2016 15th IEEE international conference on machine learning and applications (ICMLA 2016), pp 593–598. https://doi.org/10.1109/ICMLA.2016.194

  • Eftekhari-Moghadam A-M, Abdechiri M (2010) An unsupervised evaluation method based on probability density function. In: IEEE international symposium on industrial electronics (ISIE 2010), pp 1573–1578

  • Erdem C, Sankur B, Tekalp A (2004) Performance measures for video object segmentation and tracking. IEEE Trans Image Process 13(7):937–951. https://doi.org/10.1109/TIP.2004.828427

    Article  Google Scholar 

  • Feng Y, Shen X, Chen H, Zhang X (2016) A weighted-ROC graph based metric for image segmentation evaluation. Signal Process 119:43–55. https://doi.org/10.1016/j.sigpro.2015.07.010

    Article  Google Scholar 

  • Fernandez MA, Lopes RM, Hirata NST (2015) Image segmentation assessment from the perspective of a higher level task. In: 2015 28th SIBGRAPI conference on graphics, patterns and images, pp 111–118. https://doi.org/10.1109/SIBGRAPI.2015.46

  • Flores FC, Lotufo RdA (2008) Benchmark for quantitative evaluation of assisted object segmentation methods to image sequences. In: SIBGRAPI 2008: XXI Brazilian symposium on computer graphics and image processing, pp 95–102. https://doi.org/10.1109/SIBGRAPI.2008.22

  • Gao H, Tang Y, Jing L, Li H, Ding H (2017) A novel unsupervised segmentation quality evaluation method for remote sensing images. Sensors. https://doi.org/10.3390/s17102427

    Article  Google Scholar 

  • Garcia-Lamont F, Cervantes J, Lopez A, Rodriguez L (2018) Segmentation of images by color features: a survey. Neurocomputing 292:1–27. https://doi.org/10.1016/j.neucom.2018.01.091

    Article  Google Scholar 

  • Gautam AK, Bhutiyani MR (2016) Performance evaluation of hyperspectral image segmentation implemented by recombination of pct and bilateral filter based fused images. In: 2016 3rd international conference on signal processing and integrated networks (SPIN), pp 152–156

  • Ge Feng, Wang Song, Liu Tiecheng (2006) Image-segmentation evaluation from the perspective of salient object extraction. In: 2006 IEEE computer society conference on computer vision and pattern recognition (CVPR’06), vol 1, pp 1146–1153

  • Getto R, Kuijper A, von Landesberger T (2015) Extended surface distance for local evaluation of 3D medical image segmentations. Vis Comput 31(6–8):989–999. https://doi.org/10.1007/s00371-015-1113-z

    Article  Google Scholar 

  • Göçeri E (2013) A comparative evaluation for liver segmentation from spir images and a novel level set method using signed pressure force function. Thesis (Doctoral)–Izmir Institute of Technology, Electronics and Communication Engineering

  • Goceri E (2016) Automatic labeling of portal and hepatic veins from MR images prior to liver transplantation. Int J Comput Assist Radiol Surg 11(12):2153–2161. https://doi.org/10.1007/s11548-016-1446-8

    Article  Google Scholar 

  • Goceri E (2018) A method for leukocyte segmentation using modified gram-schmidt orthogonalization and expectation-maximization. In: International conference on applied analysis and mathematical modeling ICAAMM18, Istanbul, Turkey

  • Goceri E (2019a) Analysis of deep networks with residual blocks and different activation functions: classification of skin diseases. In: 2019 ninth international conference on image processing theory, tools and applications (IPTA), pp 1–6

  • Goceri E (2019b) Challenges and recent solutions for image segmentation in the era of deep learning. In: 2019 ninth international conference on image processing theory, tools and applications (IPTA), pp 1–6

  • Goceri E (2019c) Diagnosis of Alzheimer’s disease with Sobolev gradient-based optimization and 3D convolutional neural network. Int J Numer Methods Biomed Eng. https://doi.org/10.1002/cnm.3225

    Article  MathSciNet  Google Scholar 

  • Goceri E, Dura E (2015a) Artificial neural network based abdominal organ segmentations: a review. In: 2015 IEEE 14th international conference on machine learning and applications (ICMLA), pp 1191–1194. https://doi.org/10.1109/ICMLA.2015.231

  • Goceri N, Goceri E (2015b) A neural network based kidney segmentation from MR images. In: 2015 IEEE 14th international conference on machine learning and applications (ICMLA), pp 1195–1198

  • Goceri E, Songül C (2017a) Automated detection and extraction of skull from mr head images: preliminary results. In: 2017 international conference on computer science and engineering (UBMK), pp 171–176

  • Goceri E, Songul C (2017b) Computer-based segmentation, change detection and quantification for lesions in multiple sclerosis. In: Adali E (ed) 2017 International conference on computer science and engineering (UBMK), pp 177–182

  • Goceri E, Songul C (2018) Biomedical information technology: image based computer aided diagnosis systems. In: International conference on advanced technologies, Antalya

  • Goceri E, Unlu MZ, Dicle O (2015a) A comparative performance evaluation of various approaches for liver segmentation from SPIR images. Turk J Electr Eng Comput Sci 23(3):741–768. https://doi.org/10.3906/elk-1304-36

    Article  Google Scholar 

  • Goceri E, Shah ZK, Gurcan MN (2017b) Vessel segmentation from abdominal magnetic resonance images: adaptive and reconstructive approach. Int J Numer Methods Biomed Eng. https://doi.org/10.1002/cnm.2811

    Article  Google Scholar 

  • Habba M, Ameur M, Jabrane Y (2018) A novel Gini index based evaluation criterion for image segmentation. Optik 168:446–457. https://doi.org/10.1016/j.ijleo.2018.04.045

    Article  Google Scholar 

  • Henderson P, Ferrari V (2017) End-to-end training of object class detectors for mean average precision. In: Computer vision—ACCV 2016 PT V, vol 10115, pp 198–213. https://doi.org/10.1007/978-3-319-54193-8_13

  • Hoang HS, Phuong Pham C, Franklin D, van Walsum T, Ha Luu M (2019) An evaluation of CNN-based liver segmentation methods using multi-types of ct abdominal images from multiple medical centers. In: 2019 19th international symposium on communications and information technologies (ISCIT), pp 20–25

  • Huang C, Wu Q, Meng F (2016) Qualitynet: Segmentation quality evaluation with deep convolutional networks. In: 2016 visual communications and image processing (VCIP), pp 1–4

  • Jianqing Liu, Yee-Hong Yang (1994) Multiresolution color image segmentation. IEEE Trans Pattern Anal Mach Intell 16:689–700

    Article  Google Scholar 

  • Jinping L, Weihua G, Qing C, Zhaohui T, Chunhua Y (2013) An unsupervised method for flotation froth image segmentation evaluation base on image gray-level distribution. In: 2013 32nd Chinese control conference (CCC), pp 4018–4022

  • Johnson B, Xie Z (2011) Unsupervised image segmentation evaluation and refinement using a multi-scale approach. ISPRS J Photogramm Remote Sens 66(4):473–483. https://doi.org/10.1016/j.isprsjprs.2011.02.006

    Article  Google Scholar 

  • Jordan J, Angelopoulou E (2012) Supervised multispectral image segmentation with power watersheds. In: 2012 19th IEEE international conference on image processing, pp 1585–1588

  • Karimi S, Jiang X, Cosman P, Martz H (2014) Flexible methods for segmentation evaluation: results from CT-based luggage screening. J X-Ray Sci Technol 22(2):175–195. https://doi.org/10.3233/XST-140418

    Article  Google Scholar 

  • Kaya B, Goceri E, Becker A, Elder B, Puduvalli V, Winter J, Gurcan M, Otero JJ (2017) Automated fluorescent miscroscopic image analysis of PTBP1 expression in glioma. PLoS ONE 12(3):e0170991. https://doi.org/10.1371/journal.pone.0170991

    Article  Google Scholar 

  • Khan JF, Bhuiyan SM (2014) Weighted entropy for segmentation evaluation. Opt Laser Technol 57(SI):236–242. https://doi.org/10.1016/j.optlastec.2013.07.012

    Article  Google Scholar 

  • Khan J, Bhuiyan S (2011) Evaluation of the number of segments using weighted entropy. In: Proceedings SSST 2011: 43rd IEEE southeastern symposium on system theory, pp 173–178

  • Kirillov A, He K, Girshick R, Rother C, Dollár P (2019) Panoptic segmentation. In: 2019 IEEE/CVF conference on computer vision and pattern recognition (CVPR), pp 9396–9405

  • Kubassova O, Boesen M, Bliddal H (2008) General framework for unsupervised evaluation of quality of segmentation results. In: 2008 15th IEEE international conference on image processing, vol 1–5, pp 3036–3039. https://doi.org/10.1109/ICIP.2008.4712435

  • Laurent P, Cresson T, Vazquez C, Hagemeister N, de Guise JA (2016) A multi-criteria evaluation platform for segmentation algorithms. In: 2016 38th annual international conference of the IEEE engineering in medicine and biology society (EMBC), pp 6441–6444

  • Ledig C, Shi W, Bai W, Rueckert D (2014) Patch-based evaluation of image segmentation. In: 2014 IEEE conference on computer vision and pattern recognition (CVPR), pp 3065–3072. https://doi.org/10.1109/CVPR.2014.392

  • Levine M, Nazif A (1985) Dynamic measurement of computer generated image segmentations. IEEE Trans Pattern Anal Mach Intell 7(2):155–164. https://doi.org/10.1109/TPAMI.1985.4767640

    Article  Google Scholar 

  • Li Peijun, Xiao Xiaobai (2004) Evaluation of multiscale morphologicala segmentation of multispectral imagery for land cover classification. IGARSS 2004. In: 2004 IEEE international geoscience and remote sensing symposium, vol 4, pp 2676–2679

  • Li H, Zhao X, Su A, Zhang H, Liu J, Gu G (2020) Color space transformation and multi-class weighted loss for adhesive white blood cell segmentation. IEEE Access 8:24808–24818

    Article  Google Scholar 

  • Liu H, Peng C, Yu C, Wang J, Liu X, Yu G, Jiang W (2019) An end-to-end network for panoptic segmentation. In: 2019 IEEE/CVF conference on computer vision and pattern recognition (CVPR), pp 6165–6174

  • Lukac P, Hudec R, Benco M, Kamencay P, Dubcova Z, Zacharasova M (2011) Simple comparison of image segmentation algorithms based on evaluation criterion. In: Proceedings of the 21st international conference—radioelektronika 2011, pp 233–236. https://doi.org/10.1109/RADIOELEK.2011.5936406

  • Luu HM, Klink C, Moelker A, Niessen W, van Walsum T (2015) Quantitative evaluation of noise reduction and vesselness filters for liver vessel segmentation on abdominal CTA images. Phys Med Biol 60(10):3905–3926. https://doi.org/10.1088/0031-9155/60/10/3905

    Article  Google Scholar 

  • Lu Y, Wan Y, Li G (2016) Notice of removal:scale-constrained unsupervised evaluation method for multi-scale image segmentation. In: 2016 IEEE international conference on image processing (ICIP), pp 2559–2563

  • Mageswari SU, Mala C (2014) Analysis and performance evaluation of various image segmentation methods. In: 2014 international conference on contemporary computing and informatics (IC3I), pp 469–474

  • Malladi SRSP, Ram S, Rodriguez JJ (2018) A ground-truth fusion method for image segmentation evaluation. In: 2018 IEEE southwest symposium on image analysis and interpretation (SSIAI), pp 137–140

  • Mantilla SCL, Yari Y (2017) Multispectral images segmentation using fuzzy probabilistic local cluster for unsupervised clustering. In: 2017 IEEE Latin American conference on computational intelligence (LA-CCI), pp 1–5

  • Marpu PR, Neubert M, Herold H, Niemeyer I (2010) Enhanced evaluation of image segmentation results. J Spatial Sci 55(1):55–68. https://doi.org/10.1080/14498596.2010.487850

    Article  Google Scholar 

  • Mendrik AM, Vincken KL, Kuijf HJ (2015) MRBrainS challenge: online evaluation framework for brain image segmentation in 3T MRI scans. Comput Intell Neurosci. https://doi.org/10.1155/2015/813696

    Article  Google Scholar 

  • Monteiro FC, Campilho AC (2012) Distance measures for image segmentation evaluation. In: Numerical analysis and applied mathematics (ICNAAM 2012), volume A and B. American Institute of Physics, vol 1479, pp 794–797. https://doi.org/10.1063/1.4756257

  • Nogueira K, Dalla Mura M, Chanussot J, Schwartz WR, dos Santos JA (2019) Dynamic multicontext segmentation of remote sensing images based on convolutional networks. IEEE Trans Geosci Remote Sens 57(10):7503–7520

    Article  Google Scholar 

  • Pal N, Bhandari D (1993) Image thresholding: some new techniques. Signal Process 33(2):139–158. https://doi.org/10.1016/0165-1684(93)90107-L

    Article  MATH  Google Scholar 

  • Pare S, Kumar A, Singh GK, Bajaj V (2020) Image segmentation using multilevel thresholding: a research review. Iran J Sci Technol Trans Electr Eng 44(1):1–29. https://doi.org/10.1007/s40998-019-00251-1

    Article  Google Scholar 

  • Peng B, Li T (2013) A probabilistic measure for quantitative evaluation of image segmentation. IEEE Signal Process Lett 20(7):689–692. https://doi.org/10.1109/LSP.2013.2262938

    Article  Google Scholar 

  • Peng R, Varshney PK (2015) On performance limits of image segmentation algorithms. Comput Vis Image Underst 132:24–38. https://doi.org/10.1016/j.cviu.2014.11.004

    Article  Google Scholar 

  • Peng B, Wang X, Yang Y (2016) Region based exemplar references for image segmentation evaluation. IEEE Signal Process Lett 23(4):459–462. https://doi.org/10.1109/LSP.2016.2517101

    Article  Google Scholar 

  • Peng B, Zhang L, Mou X, Yang M-H (2017) Evaluation of segmentation quality via adaptive composition of reference segmentations. IEEE Trans Pattern Anal Mach Intell 39(10):1929–1941. https://doi.org/10.1109/TPAMI.2016.2622703

    Article  Google Scholar 

  • Peng B, Simfukwe M, Li T (2018) Region-based image segmentation evaluation via perceptual pooling strategies. Mach Vis Appl 29(3):477–488. https://doi.org/10.1007/s00138-017-0903-x

    Article  Google Scholar 

  • Peng C, Li Y, Jiao L, Chen Y, Shang R (2019) Densely based multi-scale and multi-modal fully convolutional networks for high-resolution remote-sensing image semantic segmentation. IEEE J Sel Top Appl Earth Observ Remote Sens 12(8):2612–2626

    Article  Google Scholar 

  • Philipp-Foliguet S, Guigues L (2006) New criteria for evaluating image segmentation results. In: 2006 IEEE international conference on acoustics, speech and signal processing, vol 1–13, pp 1357–1360

  • Pont-Tuset J, Marques F (2016) Supervised evaluation of image segmentation and object proposal techniques. IEEE Trans Pattern Anal Mach Intell 38(7):1465–1478. https://doi.org/10.1109/TPAMI.2015.2481406

    Article  Google Scholar 

  • Pont-Tuset J, Marques F (2013) Measures and meta-measures for the supervised evaluation of image segmentation. In: 2013 IEEE conference on computer vision and pattern recognition (CVPR), pp 2131–2138. https://doi.org/10.1109/CVPR.2013.277

  • Poudel P, Illanes A, Sheet D, Friebe M (2018) Evaluation of commonly used algorithms for thyroid ultrasound images segmentation and improvement using machine learning approaches. J Healthc Eng. https://doi.org/10.1155/2018/8087624

    Article  Google Scholar 

  • Prabha DS, Kumar JS (2016) Performance evaluation of image segmentation using objective methods. Indian J Sci Technol. https://doi.org/10.17485/ijst/2016/v9i8/87907

    Article  Google Scholar 

  • Qaddoura R, Faris H, Aljarah I (2020) An efficient clustering algorithm based on the k-nearest neighbors with an indexing ratio. Int J Mach Learn Cybern 11(3):675–714. https://doi.org/10.1007/s13042-019-01027-z

    Article  Google Scholar 

  • Roman-Roldan R, Gomez-Lopera J, Atae-Allah C, Martinez-Aroza J, Luque-Escamilla P (2001) A measure of quality for evaluating methods of segmentation and edge detection. Pattern Recognit 34(5):969–980. https://doi.org/10.1016/S0031-3203(00)00052-2

    Article  MATH  Google Scholar 

  • Rosenberger C, Chehdi K (2000) Genetic fusion: application to multi-components image segmentation. In: 2000 IEEE international conference on acoustics, speech, and signal processing, vol 4, pp 2223–2226

  • Sahoo P, Soltani S, Wong A, Chen Y (1988) A survey of thresholding techniques. Comput Vis Graph Image Process 41(2):233–260. https://doi.org/10.1016/0734-189X(88)90022-9

    Article  Google Scholar 

  • Saqui D, Saito JH, de Lima DC, Jorge LADC, Ferreira EJ, Ataky STM, Fambrini F (2019) Nsga2-based method for band selection for supervised segmentation in hyperspectral imaging. In: 2019 IEEE international conference on systems, man and cybernetics (SMC), pp 3580–3585

  • Shan P (2018) Image segmentation method based on K-mean algorithm. EURASIP J Image Video Process. https://doi.org/10.1186/s13640-018-0322-6

    Article  Google Scholar 

  • Sharma NK, Ronak S, Nema MK, Rakshit S (2010) Statistical evaluation of image segmentation. In: 2010 IEEE 2nd international advance computing conference, pp 101–105. https://doi.org/10.1109/IADCC.2010.5423030

  • Shi R, Ngan KN, Li S, Paramesran R, Li H (2015) Visual quality evaluation of image object segmentation: subjective assessment and objective measure. IEEE Trans Image Process 24(12):5033–5045. https://doi.org/10.1109/TIP.2015.2473099

    Article  MathSciNet  MATH  Google Scholar 

  • Shi W, Meng F, Wu Q (2017) Segmentation quality evaluation based on multi-scale convolutional neural networks. In: 2017 IEEE visual communications and image processing (VCIP), pp 1–4

  • Shi R, Ngan KN, Li S (2014) Jaccard index compensation for object segmentation evaluation. In: 2014 IEEE international conference on image processing (ICIP), pp 4457–4461

  • Shi R, Ngan KN, Li S (2017) Objectness based unsupervised object segmentation quality evaluation. In: 2017 seventh international conference on information science and technology (ICIST2017), pp 256–258

  • Skalski A, Jakubowski J, Drewniak T (2018) LEFMIS: locally-oriented evaluation framework for medical image segmentation algorithms. Phys Med Biol 63(16):2018. https://doi.org/10.1088/1361-6560/aad316

    Article  Google Scholar 

  • Srubar S (2012) Quality measurement of image segmentation evaluation methods. In: 8th international conference on signal image technology & internet based systems (SITIS 2012), pp 254–258

  • Strasters K, Gerbrands J (1991) Three-dimensional image segmentation using a split, merge and group approach. Pattern Recognit Lett 12(5):307–325. https://doi.org/10.1016/0167-8655(91)90414-H

    Article  Google Scholar 

  • Su T (2018) An improved unsupervised image segmentation evaluation approach based on under- and over- segmentation aware. Ann Photogramm Remote Sens Spatial Inf Sci 4:197–204

    Article  Google Scholar 

  • Su T, Zhang S (2017) Local and global evaluation for remote sensing image segmentation. ISPRS J Photogramm Remote Sens 130:256–276. https://doi.org/10.1016/j.isprsjprs.2017.06.003

    Article  Google Scholar 

  • Sundara SM, Aarthi R (2019) Segmentation and evaluation of white blood cells using segmentation algorithms. In: 2019 3rd international conference on trends in electronics and informatics (ICOEI), pp 1143–1146

  • Taha AA, Hanbury A, del Toro OAJ (2014) A formal method for selecting evaluation metrics for image segmentation. In: 2014 IEEE international conference on image processing (ICIP), pp 932–936

  • Taha AA, Hanbury A (2015) Metrics for evaluating 3D medical image segmentation: analysis, selection, and tool. BMC Med Imaging. https://doi.org/10.1186/s12880-015-0068-x

    Article  Google Scholar 

  • Tang Y, Zhao L, Ren L (2019) Different versions of entropy rate superpixel segmentation for hyperspectral image. In: 2019 IEEE 4th international conference on signal and image processing (ICSIP), pp 1050–1054

  • Unnikrishnan R, Pantofaru C, Hebert M (2007) Toward objective evaluation of image segmentation algorithms. IEEE Trans Pattern Anal Mach Intell 29(6):929–944. https://doi.org/10.1109/TPAMI.2007.1046

    Article  Google Scholar 

  • Vedaldi A, Lenc K (2015) MatConvNet convolutional neural networks for MATLAB. In: MM’15: proceedings of the 2015 acm multimedia conference, pp 689–692. https://doi.org/10.1145/2733373.2807412

  • Wang Y, Qi Q, Liu Y (2018) Unsupervised segmentation evaluation using area-weighted variance and jeffries–Matusita distance for remote sensing images. Remote Sens 10(8):2018. https://doi.org/10.3390/rs10081193

    Article  Google Scholar 

  • Wang Y, Qi Q, Jiang L, Liu Y (2020) Hybrid remote sensing image segmentation considering intrasegment homogeneity and intersegment heterogeneity. IEEE Geosci Remote Sens Lett 17(1):22–26

    Article  Google Scholar 

  • Wiesmann V, Bergler M, Palmisano R, Prinzen M, Franz D, Wittenberg T (2017) Using simulated fluorescence cell micrographs for the evaluation of cell image segmentation algorithms. BMC Bioinform. https://doi.org/10.1186/s12859-017-1591-2

    Article  Google Scholar 

  • Wu J, Li B, Ni W, Yan W, Zhang H (2019) Optimal segmentation scale selection for object-based change detection in remote sensing images using Kullback–Leibler divergence. IEEE Geosci Remote Sens Lett. https://doi.org/10.1109/LGRS.2019.2943406

    Article  Google Scholar 

  • Xia Y, Zhang B, Coenen F (2016) Face occlusion detection using deep convolutional neural networks. Int J Pattern Recognit Artif Intell. https://doi.org/10.1142/S0218001416600107

    Article  Google Scholar 

  • Yan Z, Yang X, Cheng K-T (2018) Joint segment-level and pixel-wise losses for deep learning based retinal vessel segmentation. IEEE Trans Biomed Eng 65(9):1912–1923. https://doi.org/10.1109/TBME.2018.2828137

    Article  Google Scholar 

  • Yang J, Li P, He Y (2014) A multi-band approach to unsupervised scale parameter selection for multi-scale image segmentation. ISPRS J Photogramm Remote Sens 94:13–24. https://doi.org/10.1016/j.isprsjprs.2014.04.008

    Article  Google Scholar 

  • Yang J, He Y, Caspersen J, Jones T (2015) A discrepancy measure for segmentation evaluation from the perspective of object recognition. ISPRS J Photogramm Remote Sens 101:186–192. https://doi.org/10.1016/j.isprsjprs.2014.12.015

    Article  Google Scholar 

  • Ye P, Kumar J, Kang L, Doermann D (2012) Unsupervised feature learning framework for no-reference image quality assessment. In: 2012 IEEE conference on computer vision and pattern recognition (CVPR), pp 1098–1105

  • Zagoruyko S, Komodakis N (2015) Learning to compare image patches via convolutional neural networks. In: 2015 IEEE conference on computer vision and pattern recognition (CVPR), pp 4353–4361

  • Zeng Y, Niu X, Dou Y (2019) Aircraft segmentation from remote sensing image by transferring natual image trained forground extraction CNN model. In: 2019 IEEE 4th international conference on signal and image processing (ICSIP), pp 817–822

  • Zhang Hui, Cholleti S, Goldman SA, Fritts JE (2006) Meta-evaluation of image segmentation using machine learning. In: 2006 IEEE computer society conference on computer vision and pattern recognition (CVPR’06), vol 1, pp 1138–1145

  • Zhang Y (1996) A survey on evaluation methods for image segmentation. Pattern Recognit 29(8):1335–1346. https://doi.org/10.1016/0031-3203(95)00169-7

    Article  Google Scholar 

  • Zhang L, Yang K (2014) Region-of-interest extraction based on frequency domain analysis and salient region detection for remote sensing image. IEEE Geosci Remote Sens Lett 11:916–920

    Article  Google Scholar 

  • Zhang H, Fritts J, Goldman S (2004) An entropy-based objective evaluation method for image segmentation. Storage Retr Methods Appl Multimed 5307(2004):38–49

    Google Scholar 

  • Zhang H, Fritts JE, Goldman SA (2008) Image segmentation evaluation: a survey of unsupervised methods. Comput Vis Image Underst 110(2):260–280. https://doi.org/10.1016/j.cviu.2007.08.003

    Article  Google Scholar 

  • Zhang X, Xiao P, Feng X (2012) An unsupervised evaluation method for remotely sensed imagery segmentation. IEEE Geosci Remote Sens Lett 9(2):156–160. https://doi.org/10.1109/LGRS.2011.2163056

    Article  Google Scholar 

  • Zhang X, Feng X, Xiao P, He G, Zhu L (2015) Segmentation quality evaluation using region-based precision and recall measures for remote sensing images. ISPRS J Photogramm Remote Sens 102:73–84. https://doi.org/10.1016/j.isprsjprs.2015.01.009

    Article  Google Scholar 

  • Zhang L, Li A, Zhang Z, Yang K (2016) Global and local saliency analysis for the extraction of residential areas in high-spatial-resolution remote sensing image. IEEE Trans Geosci Remote Sens 54(7):3750–3763. https://doi.org/10.1109/TGRS.2016.2527044

    Article  Google Scholar 

  • Zhang L, Ma J, Lv X, Chen D (2020) Hierarchical weakly supervised learning for residential area semantic segmentation in remote sensing images. IEEE Geosci Remote Sens Lett 17(1):117–121

    Article  Google Scholar 

  • Zhao Y, Hao K, He H, Tang X, Wei B (2020) A visual long-short-term memory based integrated CNN model for fabric defect image classification. Neurocomputing 380:259–270. https://doi.org/10.1016/j.neucom.2019.10.067

    Article  Google Scholar 

  • Zhao Q, Liu F, Zhang L, Zhang D (2010) A comparative study on quality assessment of high resolution fingerprint images. In: 2010 IEEE international conference on image processing, pp 3089–3092. https://doi.org/10.1109/ICIP.2010.5648800

  • Zhou S, Nie D, Adeli E, Yin J, Lian J, Shen D (2020) High-resolution encoder-decoder networks for low-contrast medical image segmentation. IEEE Trans Image Process 29:461–475

    Article  MathSciNet  Google Scholar 

  • Ziolko B, Emms D, Ziolko M (2018) Fuzzy evaluations of image segmentations. IEEE Trans Fuzzy Syst 26(4):1789–1799. https://doi.org/10.1109/TFUZZ.2017.2752130

    Article  Google Scholar 

Download references

Funding

This study was funded by National Natural Science Foundation of China (Grant No. 61201421).

Author information

Authors and Affiliations

  1. School of Information Science and Engineering, Lanzhou University, Lanzhou, 730000, China

    Zhaobin Wang & E. Wang

  2. Key Laboratory of Microbial Resources Exploitation and Application of Gansu Province, Institute of Biology, Gansu Academy of Sciences, Lanzhou, China

    Ying Zhu

Authors
  1. Zhaobin Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. E. Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Ying Zhu
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding authors

Correspondence to Zhaobin Wang or Ying Zhu.

Ethics declarations

Conflict of interest

The authors declare that they have no conflict of interest.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Wang, Z., Wang, E. & Zhu, Y. Image segmentation evaluation: a survey of methods. Artif Intell Rev 53, 5637–5674 (2020). https://doi.org/10.1007/s10462-020-09830-9

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10462-020-09830-9

Keywords

Search

Navigation