Speed Comparison of Segmentation Evaluation Methods

  • Stepan Srubar
Part of the Lecture Notes in Computer Science book series (LNCS, volume 8466)


Segmentation algorithms are widely used in image processing and there is a definite need for good quality segmentation algorithms. In order to assess which segmentation algorithms are good for our tasks, we need to measure their quality. This is done by evaluation methods. Still, we have the same problem. There are several evaluation methods, but which are good and fast enough? This article measures the quality and speed of some evaluation methods and shows that there are large differences between them.


Segmentation Evaluation Quality Speed 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Ben-Hur, A., Elisseeff, A., Guyon, I.: A stability based method for discovering structure in clustered data. In: Pacific Symposium on Biocomputing, pp. 6–17 (2002),
  2. 2.
    Fowlkes, E.B., Mallows, C.L.: A Method for Comparing Two Hierarchical Clusterings. Journal of the American Statistical Association 78(383), 553–569 (1983), CrossRefzbMATHGoogle Scholar
  3. 3.
    Huang, Q., Dom, B.: Quantitative methods of evaluating image segmentation. In: Proceedings of the International Conference on Image Processing, vol. 3, pp. 53–56 (1995),
  4. 4.
    Jiang, X., Marti, C., Irniger, C., Bunke, H.: Distance measures for image segmentation evaluation. EURASIP J. Appl. Signal Process. 2006, 209 (2006), CrossRefGoogle Scholar
  5. 5.
    Larsen, B., Aone, C.: Fast and effective text mining using linear-time document clustering. In: Proceedings of the Fifth ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, KDD 1999, pp. 16–22. ACM, New York (1999),
  6. 6.
    Martin, D.R.: An empirical approach to grouping and segmentation. Phd dissertation, University of California, Berkeley (June-August 2002)Google Scholar
  7. 7.
    Martin, D.R., Fowlkes, C., Tal, D., Malik, J.: A database of human segmented natural images and its application to evaluating segmentation algorithms and measuring ecological statistics. Tech. Rep. UCB/CSD-01-1133, EECS Department, University of California, Berkeley (January 2001),
  8. 8.
    Meilă, M.: Comparing Clusterings by the Variation of Information. In: Schölkopf, B., Warmuth, M.K. (eds.) COLT/Kernel 2003. LNCS (LNAI), vol. 2777, pp. 173–187. Springer, Heidelberg (2003)CrossRefGoogle Scholar
  9. 9.
    Meilă, M., Heckerman, D.: An experimental comparison of model-based clustering methods. Mach. Learn. 42, 9–29 (2001), CrossRefzbMATHGoogle Scholar
  10. 10.
    Mirkin, B.G.: Mathematical Classification and Clustering. Kluwer Academic Publishers, Dordrecht (1996)CrossRefzbMATHGoogle Scholar
  11. 11.
    Monteiro, F.C., Campilho, A.C.: Performance evaluation of image segmentation. In: Campilho, A., Kamel, M. (eds.) ICIAR 2006. LNCS, vol. 4141, pp. 248–259. Springer, Heidelberg (2006), CrossRefGoogle Scholar
  12. 12.
    Pal, N.R., Bhandari, D.: Image thresholding: some new techniques. Signal Process. 33, 139–158 (1993), Google Scholar
  13. 13.
    Pratt, W.K.: Digital Image Processing. John Wiley & Sons, Inc., New York (1978)Google Scholar
  14. 14.
    Rand, W.M.: Objective Criteria for the Evaluation of Clustering Methods. Journal of the American Statistical Association 66(336), 846–850 (1971), CrossRefGoogle Scholar
  15. 15.
    dos Santos Cardoso, J., Corte-Real, L.: Toward a generic evaluation of image segmentation. IEEE Transactions on Image Processing 14(11), 1773–1782 (2005), CrossRefGoogle Scholar
  16. 16.
    Strasters, K.C., Gerbrands, J.J.: Three-dimensional image segmentation using a split, merge and group approach. Pattern Recogn. Lett. 12, 307–325 (1991), CrossRefGoogle Scholar
  17. 17.
    Strehl, A., Ghosh, J., Mooney, R.: Impact of Similarity Measures on Web-page Clustering. In: Proceedings of the 17th National Conference on Artificial Intelligence: Workshop of Artificial Intelligence for Web Search (AAAI 2000), Austin, Texas, USA, July 30-31, pp. 58–64. AAAI (July 2000)Google Scholar
  18. 18.
    Unnikrishnan, R., Pantofaru, C., Hebert, M.: A measure for objective evaluation of image segmentation algorithms. In: Proceedings of the 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR 2005) - Workshops, vol. 03, p. 34+. IEEE Computer Society, Washington, DC (2005)
  19. 19.
    Unnikrishnan, R., Pantofaru, C., Hebert, M.: Toward Objective Evaluation of Image Segmentation Algorithms. IEEE Transactions on Pattern Analysis and Machine Intelligence 29(6), 929–944 (2007), CrossRefGoogle Scholar
  20. 20.
    Van Dongen, S.: Performance criteria for graph clustering and markov cluster experiments. Tech. rep., National Research Institute for Mathematics and Computer Science, Amsterdam, Netherlands (2000)Google Scholar
  21. 21.
    Šrubař, V.: Quality measurement of image segmentation evaluation methods. In: 8th International Conference on Signal Image Technology and Internet Based Systems, SITIS 2012r, pp. 254–258 (2012),
  22. 22.
    Wallace, D.L.: A Method for Comparing Two Hierarchical Clusterings: Comment. Journal of the American Statistical Association 78(383), 569–576 (1983)Google Scholar
  23. 23.
    Yasnoff, W.A., Bacus, J.W.: Scene segmentation algorithm development using error measures. AOCH 9, 45–58 (1984)Google Scholar
  24. 24.
    Yasnoff, W.A., Mui, J.K., Bacus, J.W.: Error measures for scene segmentation. Pattern Recognition 9(4), 217–231 (1977), CrossRefGoogle Scholar

Copyright information

© Springer International Publishing Switzerland 2014

Authors and Affiliations

  • Stepan Srubar
    • 1
  1. 1.VŠB-Technical University of OstravaCzech Republic

Personalised recommendations