Performance Evaluation of Image Segmentation

  • Fernando C. Monteiro
  • Aurélio C. Campilho
Part of the Lecture Notes in Computer Science book series (LNCS, volume 4141)


In spite of significant advances in image segmentation techniques, evaluation of these methods thus far has been largely subjective. Typically, the effectiveness of a new algorithm is demonstrated only by the presentation of a few segmented images that are evaluated by some method, or it is otherwise left to subjective evaluation by the reader. We propose a new approach for evaluation of segmentation that takes into account not only the accuracy of the boundary localization of the created segments but also the under-segmentation and over-segmentation effects, regardless to the number of regions in each partition. In addition, it takes into account the way humans perceive visual information. This new metric can be applied both to automatically provide a ranking among different segmentation algorithms and to find an optimal set of input parameters of a given algorithm.


Ground Truth Image Segmentation Segmentation Algorithm Segmentation Evaluation Segmentation Mask 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Cardoso, J.S., Corte-Real, L.: Toward a generic evaluation of image segmentation. IEEE Transactions on Image Processing 14(11), 1773–1782 (2005)CrossRefGoogle Scholar
  2. 2.
    Gelasca, E.D., Ebrahimi, T., Farias, M.C.Q., Carli, M., Mitra, S.K.: Towards perceptually driven segmentation evaluation metrics. In: Proc. IEEE Computer Vision and Pattern Recognition Workshop, vol. 4, p. 52 (2004)Google Scholar
  3. 3.
    Huang, Q., Dom, Byron: Quantitative methods of evaluating image segmentation. In: Proc. IEEE International Conference on Image Processing, vol. III, pp. 53–56 (1995)Google Scholar
  4. 4.
    Levine, M.D., Nazif, A.M.: Dynamic measurement of computer generated image segmentations. Trans. Pattern Analysis and Machine Intelligence 7(2), 155–164 (1985)CrossRefGoogle Scholar
  5. 5.
    Martin, D., Fowlkes, C.: The Berkeley segmentation database and benchmark, online at,
  6. 6.
    Martin, D.: An empirical approach to grouping and segmentation, Ph.D dissertation, University of California, Berkeley (2002)Google Scholar
  7. 7.
    Mezaris, V., Kompatsiaris, I., Strintzis, M.G.: Still image objective segmentation evaluation using ground truth. In: Proc. of 5th COST 276 Workshop, pp. 9–14 (2003)Google Scholar
  8. 8.
    Odet, C., Belaroussi, B., Cattin, H.B.: Scalable discrepancy measures for segmentation evaluation. In: Proc. Intern. Conf. on Image Processing, vol. I, pp. 785–788 (2002)Google Scholar
  9. 9.
    Peleg, S., Werman, M., Rom, H.: A unified approach to the change of resolution: Space and gray-level. IEEE Transactions on Pattern Analysis and Machine Intelligence 11, 739–742 (1989)CrossRefGoogle Scholar
  10. 10.
    Raghavan, V., Bollmann, P., Jung, G.: A critical investigation of recall and precision as measures of retrieval system performance. ACM Transactions on Information Systems 7(3), 205–229 (1989)CrossRefGoogle Scholar
  11. 11.
    Rubner, Y., Tomasi, C., Guibas, L.J.: The Earth Mover’s Distance as a metric for image retrieval. International Journal of Computer Vision 40(2), 99–121 (2000)CrossRefzbMATHGoogle Scholar
  12. 12.
    Sahoo, P.K., Soltani, S., Wang, A.K.C.: A survey of thresholding techniques. Computer Vision, Graphics and Image Processing 41(2), 233–260 (1988)CrossRefGoogle Scholar
  13. 13.
    Yasnoff, W.A., Mui, J.K., Bacus, J.W.: Error measures in scene segmentation. Pattern Recognition 9(4), 217–231 (1977)CrossRefGoogle Scholar
  14. 14.
    Zhang, Y.J.: A survey on evaluation methods for image segmentation. Pattern Recognition 29(8), 1335–1346 (1996)CrossRefGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2006

Authors and Affiliations

  • Fernando C. Monteiro
    • 1
    • 2
  • Aurélio C. Campilho
    • 1
    • 3
  1. 1.INEB – Instituto de Engenharia Biomédica 
  2. 2.Escola Superior de Tecnologia e de Gestão de BragançaBragançaPortugal
  3. 3.FEUP – Faculdade de EngenhariaUniversidade do Porto, Rua Dr. Roberto FriasPortoPortugal

Personalised recommendations