Distance Measures for Image Segmentation Evaluation

  • Xiaoyi JiangEmail author
  • Cyril Marti
  • Christophe Irniger
  • Horst Bunke
Open Access
Research Article
Part of the following topical collections:
  1. Performance Evaluation in Image Processing


The task considered in this paper is performance evaluation of region segmentation algorithms in the ground-truth-based paradigm. Given a machine segmentation and a ground-truth segmentation, performance measures are needed. We propose to consider the image segmentation problem as one of data clustering and, as a consequence, to use measures for comparing clusterings developed in statistics and machine learning. By doing so, we obtain a variety of performance measures which have not been used before in image processing. In particular, some of these measures have the highly desired property of being a metric. Experimental results are reported on both synthetic and real data to validate the measures and compare them with others.


Image Processing Information Technology Machine Learning Performance Evaluation Real Data 


  1. 1.
    Jiang X: Performance evaluation of image segmentation algorithms. In Handbook of Pattern Recognition and Computer Vision. 3rd edition. Edited by: Chen CH, Wang PSP. World Scientific, Singapore; 2005:525–542.CrossRefGoogle Scholar
  2. 2.
    Jiang X, Mojon D: Supervised evaluation methodology for curvilinear structure detection algorithms. Proceedings of 16th International Conference on Pattern Recognition (ICPR~'02), August 2002, Quebec, Canada 1: 103–106.CrossRefGoogle Scholar
  3. 3.
    Prieto MS, Allen AR: A similarity metric for edge images. IEEE Transactions on Pattern Analysis and Machine Intelligence 2003, 25(10):1265–1273. 10.1109/TPAMI.2003.1233900CrossRefGoogle Scholar
  4. 4.
    Sezgin M, Sankur B: Survey over image thresholding techniques and quantitative performance evaluation. Journal of Electronic Imaging 2004, 13(1):146–165. 10.1117/1.1631315CrossRefGoogle Scholar
  5. 5.
    Jiang X, Marti C, Irniger C, Bunke H: Image segmentation evaluation by techniques of comparing clusterings. Proceedings of 13th International Conference on Image Analysis and Processing (ICIAP '05), September 2005, Cagliari, ItalyGoogle Scholar
  6. 6.
    Hoover A, Jean-Baptiste G, Jiang X, et al.: An experimental comparison of range image segmentation algorithms. IEEE Transactions on Pattern Analysis and Machine Intelligence 1996, 18(7):673–689. 10.1109/34.506791CrossRefGoogle Scholar
  7. 7.
    Chang KI, Bowyer KW, Sivagurunath M: Evaluation of texture segmentation algorithms. Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR '99), June 1999, Fort Collins, Colo, USA 1: 294–299.CrossRefGoogle Scholar
  8. 8.
    Jiang X: An adaptive contour closure algorithm and its experimental evaluation. IEEE Transactions on Pattern Analysis and Machine Intelligence 2000, 22(11):1252–1265. 10.1109/34.888710CrossRefGoogle Scholar
  9. 9.
    Jiang X, Bowyer KW, Morioka Y, et al.: Some further results of experimental comparison of range image segmentation algorithms. Proceedings of 15th International Conference on Pattern Recognition (ICPR '00), September 2000, Barcelona, Spain 4: 877–881.CrossRefGoogle Scholar
  10. 10.
    Min J, Powell MW, Bowyer KW: Automated performance evaluation of range image segmentation algorithms. IEEE Transactions on Systems, Man and Cybernetics—Part B: Cybernetics 2004, 34(1):263–271. 10.1109/TSMCB.2003.811118CrossRefGoogle Scholar
  11. 11.
    Powell MW, Bowyer KW, Jiang X, Bunke H: Comparing curved-surface range image segmenters. Proceedings of 6th IEEE International Conference on Computer Vision (ICCV '98), January 1998, Bombay, India 286–291.Google Scholar
  12. 12.
    Huang Q, Dom B: Quantitative methods of evaluating image segmentation. Proceedings of International Conference on Image Processing (ICIP '95), October 1995, Washington, DC, USA 3: 53–56.CrossRefGoogle Scholar
  13. 13.
    Freixenet J, Muñoz X, Raba D, Martí J, Cufí X: Yet another survey on image segmentation: region and boundary information integration. Proceedings of 7th European Conference on Computer Vision-Part III (ECCV '02), May 2002, Copenhagen, Denmark 408–422.zbMATHGoogle Scholar
  14. 14.
    Martin D, Fowlkes C, Tal D, Malik J: A database of human segmented natural images and its application to evaluating segmentation algorithms and measuring ecological statistics. Proceedings of 8th IEEE International Conference on Computer Vision (ICCV '01), July 2001, Vancouver, BC, Canada 2: 416–423.Google Scholar
  15. 15.
    Jain AK, Murty MN, Flynn PJ: Data clustering: a review. ACM Computing Surveys (CSUR) 1999, 31(3):264–323. 10.1145/331499.331504CrossRefGoogle Scholar
  16. 16.
    Rand WM: Objective criteria for the evaluation of clustering methods. Journal of the American Statistical Association 1971, 66(336):846–850. 10.2307/2284239CrossRefGoogle Scholar
  17. 17.
    Fowlkes EB, Mallows CL: A Method for comparing two hierarchical clusterings. Journal of the American Statistical Association 1983, 78(383):553–569. 10.2307/2288117CrossRefGoogle Scholar
  18. 18.
    Ben-Hur A, Elisseeff A, Guyon I: A stability based method for discovering structure in clustered data. Proceedings of 7th Pacific Symposium on Biocomputing (PSB '02), January 2002, Lihue, Hawaii, USA 7: 6–17.Google Scholar
  19. 19.
    van Dongen S: Performance criteria for graph clustering and Markov cluster experiments. In Tech. Rep. INS-R0012. Centrum voor Wiskunde en Informatica (CWI), Amsterdam, The Netherlands; 2000.Google Scholar
  20. 20.
    Khuller S, Raghavachari B: Advanced combinatorial algorithms. In Algorithms and Theory of Computation Handbook. Edited by: Atallah MJ. CRC Press, Boca Raton, Fla, USA; 1999:1–23. chapter 7zbMATHGoogle Scholar
  21. 21.
    Cover TM, Thomas JA: Elements of Information Theory. John Wiley & Sons, Chichester, UK; 1991.CrossRefGoogle Scholar
  22. 22.
    Strehl A, Ghosh J, Mooney R: Impact of similarity measures on web-page clustering. Proceedings of 17th National Conference on Artificial Intelligence: Workshop of Artificial Intelligence for Web Search (AAAI '00), July 2000, Austin, Tex, USA 58–64.Google Scholar
  23. 23.
    Meila M: Comparing clusterings by the variation of information. Proceedings of 16th Annual Conference on Computational Learning Theory and 7th Workshop on Kernel Machines (COLT/Kernel '03), August 2003, Washington, DC, USA 173–187.CrossRefGoogle Scholar
  24. 24.
    Cinque L, Levialdi S, Pignalberi G, Cucchiara R, Martinz S: Optimal range segmentation parameters through genetic algorithms. Proceedings of 15th International Conference on Pattern Recognition (ICPR '00), September 2000, Barcelona, Spain 1: 474–477.CrossRefGoogle Scholar

Copyright information

© Jiang et al. 2006

Authors and Affiliations

  • Xiaoyi Jiang
    • 1
    Email author
  • Cyril Marti
    • 2
  • Christophe Irniger
    • 2
  • Horst Bunke
    • 2
  1. 1.Computer Vision and Pattern Recognition Group, Department of Computer ScienceUniversity of MünsterMünsterGermany
  2. 2.Institute of Computer Science and Applied MathematicsUniversity of BernBernSwitzerland

Personalised recommendations