Machine Vision and Applications

, Volume 29, Issue 3, pp 477–488 | Cite as

Region-based image segmentation evaluation via perceptual pooling strategies

  • Bo Peng
  • Macmillan Simfukwe
  • Tianrui Li
Original Paper


Image segmentation is an essential step for many computer vision tasks. Evaluating the quality of image segmentations becomes indispensable for choosing an appropriate output of the image segmentation algorithms. To quantitatively evaluate the segmentation quality, various evaluation measures have been proposed to produce a quality map, and a spatial pooling algorithm is followed to combine the quality map into a single quality score. In this paper, we propose two pooling strategies instead of using the conventional spatial average operation. By assigning perceptual meaningful weights to the quality maps, we obtain evaluation measures that are correlated with the human perception of segmentation quality. Specifically, a quality-based and a visual importance-based pooling strategies are designed and tested on some popular evaluation measures, respectively. To the best of our knowledge, this is the first work that applies perceptual pooling strategies for segmentation evaluation. Extensive experiments are conducted on a subjective evaluation benchmark and the Berkeley Segmentation Dataset (BSDS500). The results indicate that the proposed strategies can improve the performance of existing evaluation measures and produce a more perceptually meaningful judgment on the segmentation quality.


Image segmentation evaluation pooling strategies visual importance 


  1. 1.
    Arbelaez, P., Maire, M., Fowlkes, C.C., Malik, J.: Contour detection and hierarchical image segmentation. IEEE Trans. Pattern Anal. Mach. Intell. 33(5), 898–916 (2011)CrossRefGoogle Scholar
  2. 2.
    Borsotti, M., Campadelli, P., Schettini, R.: Quantitative evaluation of color image segmentation results. Pattern Recognit. Lett. 19(8), 741–747 (1998)CrossRefMATHGoogle Scholar
  3. 3.
    Bruce, N., Tsotsos, J.: Saliency, attention, and visual search: an information theoretic approach. J. Vis. 9(3), 1–24 (2009)CrossRefGoogle Scholar
  4. 4.
    Chen, P., Krim, H., Mendoza, O.: Multiphase joint segmentation–registration and object tracking for layered images. IEEE Trans. Image Process. 19(7), 1706–1719 (2010)MathSciNetCrossRefMATHGoogle Scholar
  5. 5.
    Christensen, H., Phillips, P.: Empirical Evaluation Methods in Computer Vision. World Scientific Publishing Company, Singapore (2002)CrossRefMATHGoogle Scholar
  6. 6.
    Comanicu, D., Meer, P.: Mean shift: a robust approach toward feature space analysis. IEEE Trans. Pattern Anal. Mach. Intell. 24, 603–619 (2002)CrossRefGoogle Scholar
  7. 7.
    Felzenszwalb, P., Huttenlocher, D.: Efficient graph based image segmentation. Int. J. Comput. Vis. 59(2), 167–181 (2004)CrossRefGoogle Scholar
  8. 8.
    Hou, X., Harel, J., Koch, C.: Image signature: highlighting sparse salient regions. IEEE Trans. Pattern Anal. Mach. Intell. 34(1), 194–201 (2012)CrossRefGoogle Scholar
  9. 9.
    Hou, X., Zhang, L.: Saliency detection: a spectral residual approach. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 2336–2343 (2007)Google Scholar
  10. 10.
    Itti, L., Koch, C., Niebur, E.: A model of saliency-based visual attention for rapid scene analysis. IEEE Trans. Pattern Anal. Mach. Intell. 20(11), 1254–1259 (1998)CrossRefGoogle Scholar
  11. 11.
    Jiang, X., Marti, C., Irniger, C., Bunke, H.: Distance measures for image segmentation evaluation. EURASIP J. Appl. Signal Process. 2006, 209 (2006)Google Scholar
  12. 12.
    Kanungo, T., Dom, B., Niblack, W., Steele, D.: A fast algorithm for mdl-based multi-band image segmentation. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 609–616 (1994)Google Scholar
  13. 13.
    Martin, D.: An empirical approach to grouping and segmentation. Ph.D. thesis, EECS Department, University of California, Berkeley (2002)Google Scholar
  14. 14.
    Martin, D., Fowlkes, C., Tal, D., Malik, J.: A database of human segmented natural images and its application to evaluating segmentation algorithms and measuring ecological statistics. In: IEEE International Conference on Computer Vision, pp. 416–424 (2001)Google Scholar
  15. 15.
    Meila, M.: Comparing clusterings: an axiomatic view. In: International Conference on Machine Learning, pp. 577–584 (2005)Google Scholar
  16. 16.
    Moorthy, A.K., Bovik, A.C.: Visual importance pooling for image quality assessment. IEEE J. Image Inf. Vis. Qual. 3(2), 193–C201 (2009)Google Scholar
  17. 17.
    Pantofaru, C., Schmid, C., Hebert, M.: Object recognition by integrating multiple image segmentations. In: European Conference on Computer Vision, pp. 481–494 (2008)Google Scholar
  18. 18.
    Peng, B., Simfukwe, M., Yang, Y., Li., T.: Perceptual pooling strategies for image segmentation quality evaluation. In: The 12th conference on uncertainty modeling in knowledge engineering and decision making (FLINS), pp. 918–923 (2016)Google Scholar
  19. 19.
    Peng, B., Veksler, O.: Parameter selection for graph cut based image segmentation. In: British Machine Vision Conference, pp. 153–162 (2008)Google Scholar
  20. 20.
    Pont-Tuset, J., Marques, F.: Measures and meta-measures for the supervised evaluation of image segmentation. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 2131–2138 (2013)Google Scholar
  21. 21.
    Rao, S., Mobahi, H., Yang, A., Sastry, S., Ma, Y.: Natural image segmentation with adaptive texture and boundary encoding. In: Asian Conference of Computer Vision, pp. 135–146 (2009)Google Scholar
  22. 22.
    Ren, X., Malik, J.: Learning a classification model for segmentation. In: IEEE International Conference on Computer Vision, pp. 10–17 (2003)Google Scholar
  23. 23.
    Smistad, E., Falch, T., Bozorgi, M., Elster, A., Lindseth, F.: Medical image segmentation on gpus—a comprehensive review. Med. Image Anal. 20(1), 1–18 (2015)CrossRefGoogle Scholar
  24. 24.
    Tong, N., Lu, H., Zhang, L., Ruan, X.: Saliency detection with multi-scale superpixels. IEEE Signal Process. Lett. 21(9), 1035–1039 (2014)CrossRefGoogle Scholar
  25. 25.
    Unnikrishnan, R., Pantofaru, C., Hebert, M.: Toward objective evaluation of image segmentation algorithms. IEEE Trans. Pattern Anal. Mach. Intell. 29(6), 929–944 (2007)CrossRefGoogle Scholar
  26. 26.
    Wang, Z., Bovik, A.: Modern Image Quality Assessment. Morgan and Claypool Publishing Company, New York (2006)Google Scholar
  27. 27.
    Yang, A., Wright, J., Ma, Y., Sastry, S.: Unsupervised segmentation of natural images via lossy data compression. Comput. Vis. Image Underst. 11(2), 212–225 (2008)CrossRefGoogle Scholar
  28. 28.
    Yang, C., Zhang, L., Lu, H., Ruan, X., Yan, M.: Saliency detection via graph-based manifold ranking. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 3166–3173 (2013)Google Scholar
  29. 29.
    Zhang, H., Fritts, J., Goldman, S.: An entropy-based objective segmentation evaluation method for image segmentation. In: SPIE Sotrage and Retrieval Methods and Applicaitons for Multimedia, pp. 38–49 (2004)Google Scholar
  30. 30.
    Zhang, H., Fritts, J., Goldman, S.: A co-evaluation framework for improving segmentation evaluation. In: SPIE Signal Processing and Target Recognition, pp. 420–430 (2005)Google Scholar

Copyright information

© Springer-Verlag GmbH Germany, part of Springer Nature 2018

Authors and Affiliations

  1. 1.School of Information Science and TechnologySouthwest Jiaotong UniversityChengduChina

Personalised recommendations