Skip to main content

Advertisement

SpringerLink
Log in

Benchmarking Image Segmentation Algorithms

  • Published:
International Journal of Computer Vision Aims and scope Submit manuscript

Abstract

We present a thorough quantitative evaluation of four image segmentation algorithms on images from the Berkeley Segmentation Database. The algorithms are evaluated using an efficient algorithm for computing precision and recall with regard to human ground-truth boundaries. We test each segmentation method over a representative set of input parameters, and present tuning curves that fully characterize algorithm performance over the complete image database. We complement the evaluation on the BSD with segmentation results on synthetic images. The results reported here provide a useful benchmark for current and future research efforts in image segmentation.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  • Belongie, S., & Malik, J. (1998). Finding boundaries in natural images: A new method using point descriptors and area completion. In European conference on computer vision (pp. 751–766).

  • Belongie, S., Fowlkes, C., Chung, F., & Malik, J. (2002). Spectral partitioning with indefinite kernels using the Nyström extension. In European conference on computer vision (pp. 21–31).

  • Blake, A., Rother, A., Brown, M., Perez, P., & Torr, P. (2004). Interactive image segmentation using an adaptive GMMRF model. In European conference on computer vision (pp. 428–441).

  • Borra, S., & Sarkar, S. (1997). A framework for performance characterization of intermediate-level grouping modules. IEEE Transactions on Pattern Analysis and Machine Intelligence, 19(11), 1306–1312.

    Article  Google Scholar 

  • Boykov, Y. Y., & Jolly, M.-P. (2001). Interactive Graph Cuts for optimal boundary & region segmentation of objects in N-D images. In IEEE international conference on computer vision (pp. 105–112).

  • Boykov, Y., & Kolmogorov, V. (2001). An experimental comparison of min-cut/max-flow algorithms for energy minimization in vision. In International workshop on energy minimization methods in computer vision and pattern recognition. Lecture Notes in Computer Science (pp. 359–374).

  • Boykov, Y., Veksler, O., & Zabih, R. (2001). Fast approximate energy minimization via graph cuts. IEEE Transactions on Pattern Analysis and Machine Intelligence, 23(11), 1222–1239.

    Article  Google Scholar 

  • Cheng, Y. (1995). Mean shift, mode seeking, and clustering. IEEE Transactions on Pattern Analysis and Machine Intelligence, 17(8), 790–799.

    Article  Google Scholar 

  • Cho, K., & Meer, P. (1997). Image segmentation from consensus information. Computer Vision and Image Understanding, 68(1), 72–89.

    Article  Google Scholar 

  • Comaniciu, D., & Meer, P. (2002). Mean shift: A robust approach toward feature space analysis. IEEE Transactions on Pattern Analysis and Machine Intelligence, 24(5), 603–619.

    Article  Google Scholar 

  • Cour, T., Yu, S., & Shi, J. (2006). Normalized cuts Matlab code. Computer and Information Science, Penn State University. code available at http://www.seas.upenn.edu/~timothee/software_ncut/software.html.

  • Estrada, F. J. (2005). Advances in computational image segmentation and perceptual grouping. PhD thesis, University of Toronto, Department of Computer Science.

  • Estrada, F. J., & Jepson, A. D. (2004). Spectral embedding and min-cut for image segmentation. In British machine vision conference (pp. 317–326).

  • Estrada, F. J., & Jepson, A. D. (2005). Quantitative evaluation of a novel image segmentation algorithm. In IEEE international conference on computer vision and pattern recognition.

  • Felzenszwalb, P., & Huttenlocher, D. (2004). Image segmentation by local variation code. MIT Artificial Intelligence Lab. Code available at http://people.cs.uchicago.edu/~pff/segment/.

  • Felzenszwalb, P. F., & Huttenlocher, D. P. (1998). Image segmentation using local variation. In IEEE conference on computer vision and pattern recognition (pp. 98–104).

  • Fowlkes, C., Belongie, S., & Malik, J. (2001). Efficient spatiotemporal grouping using the Nyström method. In IEEE conference on computer vision and pattern recognition (pp. 231–238).

  • Fowlkes, C., Belongie, S., Chung, F., & Malik, J. (2004). Spectral grouping using the Nyström method. IEEE Transactions on Pattern Analysis and Machine Intelligence, 26(2), 214–225.

    Article  Google Scholar 

  • Gdalyahu, Y., Weinshall, D., & Werman, M. (2001). Self-organization in vision: Stochastic clustering for image segmentation, perceptual grouping, and image database organization. IEEE Transactions on Pattern Analysis and Machine Intelligence, 23(10), 1053–1074.

    Article  Google Scholar 

  • Georgescu, B., & Christoudias, C. M. (2003). The Edge Detection and Image segmentation (EDISON) system. Robust Image Understanding Laboratory, Rutgers University. Code available at http://web.archive.org/web/20060617020918/www.caip.rutgers.edu/riul/research/code/EDISON/index.html.

  • Ishikawa, H., & Geiger, D. (1998). Segmentation by grouping junctions. In IEEE conference on computer vision and pattern recognition (pp. 125–131).

  • Kaufhold, J., & Hoogs, A. (2004). Learning to segment images using region-based perceptual properties. In IEEE international conference on computer vision and pattern recognition (vol. 2, pp. 954–961).

  • Malik, J., Belongie, S., Leung, T., & Shi, J. (1999). Textons, contours and regions: Cue integration in image segmentation. In IEEE international conference on computer vision (pp. 918–925).

  • Malik, J., Belongie, S., Leung, T., & Shi, J. (2001). Contour and texture analysis for image segmentation. International Journal of Computer Vision, 43(1), 7–27.

    Article  MATH  Google Scholar 

  • Martin, D. (2002). An empirical approach to grouping and segmentation. PhD thesis, University of California, Berkeley.

  • Martin, D., & Fowlkes, C. (2001). The Berkeley segmentation database and benchmark. Computer Science Department, Berkeley University. http://www.eecs.berkeley.edu/Research/Projects/CS/vision/bsds/.

  • Martin, D., Fowlkes, C., Tal, D., & Malik, J. (2001). A database of human segmented natural images and its application to evaluating segmentation algorithms and measuring ecological statistics. In IEEE international conference on computer vision (pp. 416–425).

  • Martin, D. R., Fowlkes, C. C., & Malik, J. (2002). Learning to detect natural image boundaries using brightness and texture. In Neural information processing systems.

  • Martin, D., Fowlkes, C., & Malik, J. (2004). Learning to detect natural image boundaries using local brightness, color, and texture cues. IEEE Transactions on Pattern Analysis and Machine Intelligence, 26(5), 530–549.

    Article  Google Scholar 

  • Meila, M., & Shi, J. (2000). Learning segmentation by random walks. In NIPS (pp. 873–879).

  • Sharon, E., Brandt, A., & Basri, R. (2000). Fast multiscale image segmentation. In IEEE conference on computer vision and pattern recognition (pp. 70–77).

  • Sharon, E., Brandt, A., & Basri, R. (2001). Segmentation and boundary detection using multiscale intensity measurements. In IEEE conference on computer vision and pattern recognition (pp. 469–476).

  • Shental, N., Zomet, A., Hertz, T., & Weiss, Y. (2003). Learning and inferring image segmentations using the GBP typical cut algorithm. In IEEE international conference on computer vision (pp. 1243–1250).

  • Shi, J., & Malik, J. (2000). Normalized cuts and image segmentation. IEEE Transactions on Pattern Analysis and Machine Intelligence, 22(8), 888–905.

    Article  Google Scholar 

  • Veksler, O. (2000). Image segmentation by nested cuts. In IEEE conference on computer vision and pattern recognition (pp. 339–344).

  • Wang, S., & Siskind, J. M. (2001). Image segmentation with minimum mean cut. In IEEE international conference on computer vision (pp. 517–524).

  • Wang, S., & Siskind, J. M. (2003). Image segmentation with ratio cut. IEEE Transactions on Pattern Analysis and Machine Intelligence, 25(6), 675–690.

    Article  Google Scholar 

  • Wu, Z., & Leahy, R. (1993). An optimal graph theoretic approach to data clustering: Theory and its application to image segmentation. IEEE Transactions on Pattern Analysis and Machine Intelligence, 15(11), 1101–1113.

    Article  Google Scholar 

  • Yu, S. X. (2004). Segmentation using multiscale cues. In IEEE international conference on computer vision and pattern recognition (pp. 247–254).

  • Yu, S. X. (2005). Segmentation induced by scale invariance. In IEEE international conference on computer vision and pattern recognition.

  • Yu, S. X., & Shi, J. (2003). Multiclass spectral clustering. In IEEE International Conference on Computer Vision (pp. 313–319).

Download references

Author information

Authors and Affiliations

  1. Department of Computer Science, University of Toronto, 6 King’s College Road, M5S 3G4, Toronto, ON, Canada

    Francisco J. Estrada & Allan D. Jepson

Authors
  1. Francisco J. Estrada
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Allan D. Jepson
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Francisco J. Estrada.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Estrada, F.J., Jepson, A.D. Benchmarking Image Segmentation Algorithms. Int J Comput Vis 85, 167–181 (2009). https://doi.org/10.1007/s11263-009-0251-z

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11263-009-0251-z

Keywords

Search

Navigation