Unsupervised Performance Evaluation of Image Segmentation

  • Sebastien ChabrierEmail author
  • Bruno Emile
  • Christophe Rosenberger
  • Helene Laurent
Open Access
Research Article
Part of the following topical collections:
  1. Performance Evaluation in Image Processing


We present in this paper a study of unsupervised evaluation criteria that enable the quantification of the quality of an image segmentation result. These evaluation criteria compute some statistics for each region or class in a segmentation result. Such an evaluation criterion can be useful for different applications: the comparison of segmentation results, the automatic choice of the best fitted parameters of a segmentation method for a given image, or the definition of new segmentation methods by optimization. We first present the state of art of unsupervised evaluation, and then, we compare six unsupervised evaluation criteria. For this comparative study, we use a database composed of 8400 synthetic gray-level images segmented in four different ways. Vinet's measure (correct classification rate) is used as an objective criterion to compare the behavior of the different criteria. Finally, we present the experimental results on the segmentation evaluation of a few gray-level natural images.


Information Technology Performance Evaluation Evaluation Criterion Quantum Information Image Segmentation 


  1. 1.
    Freixenet J, Muñoz X, Raba D, Marti J, Cufi X: Yet another survey on image segmentation: region and boundary information integration. Proceedings of the European Conference on Computer Vision (ECCV '02), May 2002, Copenhagen, Denmark 408–422.zbMATHGoogle Scholar
  2. 2.
    Haralick RM, Shapiro LG: Image segmentation techniques. Computer Vision, Graphics, & Image Processing 1985, 29(1):100–132. 10.1016/S0734-189X(85)90153-7CrossRefGoogle Scholar
  3. 3.
    Zhang YJ: A survey on evaluation methods for image segmentation. Pattern Recognition 1996, 29(8):1335–1346. 10.1016/0031-3203(95)00169-7CrossRefGoogle Scholar
  4. 4.
    Nasab NM, Analoui M, Delp EJ: Robust and efficient image segmentation approaches using Markov random field models. Journal of Electronic Imaging 2003, 12(1):50–58. 10.1117/1.1525280CrossRefGoogle Scholar
  5. 5.
    Baddeley AJ: An error metric for binary images. In Robust Computer Vision. Wichmann, Karlsruhe, Germany; 1992:59–78.Google Scholar
  6. 6.
    Vinet L: Segmentation et mise en correspondance de régions de paires d'images stéréoscopiques, M.S. thesis. Université de Paris IX Dauphine, Paris, France; 1991.Google Scholar
  7. 7.
    Huttenlocher DP, Rucklidge WJ: Multi-resolution technique for comparing images using the Hausdorff distance. Proceedings of IEEE Computer Vision and Pattern Recognition (CVPR '93), June 1993, New York, NY, USA 705–706.CrossRefGoogle Scholar
  8. 8.
    Martin D, Fowlkes C, Tal D, Malik J: A database of human segmented natural images and its application to evaluating segmentation algorithms and measuring ecological statistics. Proceedings of the IEEE International Conference on Computer Vision (ICCV '01), July 2001, Vancouver, BC, Canada 2: 416–423.Google Scholar
  9. 9.
    Weszka JS, Rosenfeld A: Threshold evaluation techniques. IEEE Transactions on Systems, Man and Cybernetics 1978, 8(8):622–629.CrossRefGoogle Scholar
  10. 10.
    Levine MD, Nazif AM: Dynamic measurement of computer generated image segmentations. IEEE Transactions on Pattern Analysis and Machine Intelligence 1985, 7(2):155–164.CrossRefGoogle Scholar
  11. 11.
    Sezgin M, Sankur B: Survey over image thresholding techniques and quantitative performance evaluation. Journal of Electronic Imaging 2004, 13(1):146–168. 10.1117/1.1631315CrossRefGoogle Scholar
  12. 12.
    Cochran WG:Some methods for strengthening the commonOpen image in new window tests. Biometrics 1954, 10: 417–451. 10.2307/3001616MathSciNetzbMATHGoogle Scholar
  13. 13.
    Pal NR, Pal SK: Entropic thresholding . Signal Processing 1989, 16(2):97–108. 10.1016/0165-1684(89)90090-XMathSciNetCrossRefGoogle Scholar
  14. 14.
    Rosenberger C: Mise en oeuvre d'un système adaptatif de segmentation d'images, M.S. thesis. Université de Rennes 1, Rennes, France; 1999.Google Scholar
  15. 15.
    Borsotti M, Campadelli P, Schettini R: Quantitative evaluation of color image segmentation results. Pattern Recognition Letters 1998, 19(8):741–747. 10.1016/S0167-8655(98)00052-XzbMATHCrossRefGoogle Scholar
  16. 16.
    Liu J, Yang Y-H: Multiresolution color image segmentation. IEEE Transactions on Pattern Analysis and Machine Intelligence 1994, 16(7):689–700. 10.1109/34.297949CrossRefGoogle Scholar
  17. 17.
    Zeboudj R: Filtrage, seuillage automatique, contraste et contours: du pré-traitement à l'analyse d'image, M.S. thesis. Université de Saint Etienne, Saint Etienne, France; 1988.Google Scholar
  18. 18.
    Chabrier S, Rosenberger C, Laurent H, Emile B, Marché P: Evaluating the segmentation result of a gray-level image. Proceedings of 12th European Signal Processing Conference (EUSIPCO '04), September 2004, Vienna, Austria 953–956.Google Scholar
  19. 19.
    Chabrier S, Emile B, Laurent H, Rosenberger C, Marché P: Unsupervised evaluation of image segmentation application to multi-spectral images. Proceedings of International Conference on Pattern Recognition (ICPR '04), August 2004, Cambridge, UK 1: 576–579.CrossRefGoogle Scholar
  20. 20.
    Krishnapuram R, Keller JM:PossibilisticOpen image in new window-means algorithm: insights and recommendations. IEEE Transactions on Fuzzy Systems 1996, 4(3):385–393. 10.1109/91.531779Google Scholar
  21. 21.
    Comaniciu D, Meer P: Mean shift: a robust approach toward feature space analysis. IEEE Transactions on Pattern Analysis and Machine Intelligence 2002, 24(5):603–619. 10.1109/34.1000236CrossRefGoogle Scholar
  22. 22.
    Monawer HA: Image vector quantization using a modified LBG algorithm with approximated centroids. Electronics Letters 1995, 31(3):174–175. 10.1049/el:19950100CrossRefGoogle Scholar
  23. 23.
    Erdem ÇE, Sankur B, Tekalp AM: Performance measures for video object segmentation and tracking. IEEE Transactions on Image Processing 2004, 13(7):937–951. 10.1109/TIP.2004.828427CrossRefGoogle Scholar
  24. 24.
    Nazif AM, Levine MD: Low level image segmentation: an expert system. IEEE Transactions on Pattern Analysis and Machine Intelligence 1984, 6(5):555–577.CrossRefGoogle Scholar
  25. 25.
    Krishnapuram R, Keller JM: Possibilistic approach to clustering. IEEE Transactions on Fuzzy Systems 1993, 1(2):98–110. 10.1109/91.227387CrossRefGoogle Scholar

Copyright information

© Chabrier et al. 2006

Authors and Affiliations

  • Sebastien Chabrier
    • 1
    Email author
  • Bruno Emile
    • 1
  • Christophe Rosenberger
    • 1
  • Helene Laurent
    • 1
  1. 1.Laboratoire Vision et Robotique, UPRES EA 2078,ENSI de BourgesUniversit$#233; d' Orl$#233;ansBourges cedexFrance

Personalised recommendations