Sensing and Imaging

, 20:5 | Cite as

Pointwise Multi-resolution Feature Descriptor for Spectral Segmentation

  • JingMao Zhang
  • YanXia ShenEmail author
Original Paper


In the process of spectral segmentation, it is crucial to compute a reliable affinity matrix with different features of an image. In this paper, we present a method of constructing the affinity matrix based on multi-resolution features extracted from the original features. A pointwise multi-resolution feature descriptor (PMFD) is designed based on spectral graph wavelets, which characterize the topology of the image centered at different pixels. After choosing the scales of interest in our descriptor, a new affinity matrix is constructed based on the extracted features. For large-size affinity matrixes, it is difficult to compute the proposed PMFD for all pixels of an image. Therefore, an approximate algorithm is proposed to compute the PMFD. To demonstrate the effectiveness of our method, a series of experiments on the Berkeley image segmentation dataset are implemented using the PMFD-based spectral segmentation algorithm. A comparison with other image segmentation techniques demonstrates that our method offers significantly improved pointwise spectral segmentation performance.


Pointwise feature descriptor Multi-resolution Spectral graph wavelet Approximate algorithm Spectral segmentation 



This work was supported by National Nature Science Foundation (Grant Nos. 61573167, 61572237).the Fundamental Research Funds for the Central Universities (Grant Nos. JUSRP31106, JUSRP51510), Postgraduate Research and Practice Innovation Program of Jiangsu Province (Grant No. KYCX17_1454).


  1. 1.
    Kim, S., Nowozin, S., Kohli, P., et al. (2011). Higher-order correlation clustering for image segmentation. Advances in Neural Information Processing Systems, 1530–1538.Google Scholar
  2. 2.
    Mobahi, H., Rao, S. R., Yang, A. Y., et al. (2011). Segmentation of natural images by texture and boundary compression. International Journal of Computer Vision, 95(1), 86–98.CrossRefGoogle Scholar
  3. 3.
    Comaniciu, D., & Meer, P. (2002). Mean shift: A robust approach toward feature space analysis. IEEE Transactions on Pattern Analysis and Machine Intelligence, 24(5), 603–619.CrossRefGoogle Scholar
  4. 4.
    Carson, C., Belongie, S., Greenspan, H., et al. (2002). Blobworld: Image segmentation using expectation-maximization and its application to image querying. IEEE Transactions on Pattern Analysis and Machine Intelligence, 24(8), 1026–1038.CrossRefGoogle Scholar
  5. 5.
    Shi, J., & Malik, J. (2000). Normalized cuts and image segmentation. IEEE Transactions on Pattern Analysis and Machine Intelligence, 22(8), 888–905.CrossRefGoogle Scholar
  6. 6.
    Rohkohl C, Engel K. (2007). Efficient image segmentation using pairwise pixel similarities. In Joint pattern recognition symposium. Berlin: Springer (pp. 254–263).Google Scholar
  7. 7.
    Li, X., Jin, L., Song, E., et al. (2016). An integrated similarity metric for graph-based color image segmentation. Multimedia Tools and Applications, 75(6), 2969–2987.CrossRefGoogle Scholar
  8. 8.
    Pont-Tuset, J., Arbelaez, P., Barron, J. T., et al. (2017). Multiscale combinatorial grouping for image segmentation and object proposal generation. IEEE Transactions on Pattern Analysis and Machine Intelligence, 39(1), 128–140.CrossRefGoogle Scholar
  9. 9.
    Arbelaez, P., Maire, M., Fowlkes, C., et al. (2011). Contour detection and hierarchical image segmentation. IEEE Transactions on Pattern Analysis and Machine Intelligence, 33(5), 898–916.CrossRefGoogle Scholar
  10. 10.
    Cour T, Benezit F, Shi J. (2005). Spectral segmentation with multiscale graph decomposition. In IEEE computer society conference on computer vision and pattern recognition, CVPR (vol. 2, pp. 1124–1131).Google Scholar
  11. 11.
    Kim, T. H., Lee, K. M., & Lee, S. U. (2013). Learning full pairwise affinities for spectral segmentation. IEEE Transactions on Pattern Analysis and Machine Intelligence, 35(7), 1690–1703.CrossRefGoogle Scholar
  12. 12.
    Wang, X., Tang, Y., Masnou, S., et al. (2015). A global/local affinity graph for image segmentation. IEEE Transactions on Image Processing, 24(4), 1399–1411.MathSciNetCrossRefGoogle Scholar
  13. 13.
    Zhang, J. M., & Shen, Y. X. (2017). Spectral segmentation via minimum barrier distance[J]. Multimedia Tools and Applications, 76(24), 25713–25729.CrossRefGoogle Scholar
  14. 14.
    Aytekin, Ç., Ozan, E. C., Kiranyaz, S., et al. (2017). Extended quantum cuts for unsupervised salient object extraction. Multimedia Tools and Applications, 76(8), 10443–10463.CrossRefGoogle Scholar
  15. 15.
    Felzenszwalb, P. F., & Huttenlocher, D. P. (2004). Efficient graph-based image segmentation. International Journal of Computer Vision, 59(2), 167–181.CrossRefGoogle Scholar
  16. 16.
    Estrada, F. J., & Jepson, A. D. (2009). Benchmarking image segmentation algorithms[J]. International Journal of Computer Vision, 85(2), 167–181.CrossRefGoogle Scholar
  17. 17.
    Hammond, D. K., Vandergheynst, P., & Gribonval, R. (2011). Wavelets on graphs via spectral graph theory. Applied and Computational Harmonic Analysis, 30(2), 129–150.MathSciNetCrossRefGoogle Scholar
  18. 18.
    Pham, M. T., Mercier, G., & Michel, J. (2015). Pointwise graph-based local texture characterization for very high resolution multispectral image classification. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 8(5), 1962–1973.CrossRefGoogle Scholar
  19. 19.
    Tremblay, N., & Borgnat, P. (2014). Graph wavelets for multiscale community mining. IEEE Transactions on Signal Processing, 62(20), 5227–5239.MathSciNetCrossRefGoogle Scholar
  20. 20.
    Hwa Kim W, Ravi S. N, Johnson S. C, et al. (2015). On statistical analysis of neuroimages with imperfect registration. In Proceedings of the IEEE international conference on computer vision. (pp. 666–674).Google Scholar
  21. 21.
    Kim W. H., Pachauri, D., Hatt, C., et al. (2012). Wavelet based multi-scale shape features on arbitrary surfaces for cortical thickness discrimination. In Advances in Neural Information Processing Systems. (pp. 1241–1249).Google Scholar
  22. 22.
    Li, C., & Hamza, A. B. (2014). Spatially aggregating spectral descriptors for nonrigid 3D shape retrieval: a comparative survey. Multimedia Systems, 20(3), 253–281.CrossRefGoogle Scholar
  23. 23.
    Li, C., & Hamza, A. B. (2013). A multiresolution descriptor for deformable 3D shape retrieval[J]. The Visual Computer, 29(6–8), 513–524.CrossRefGoogle Scholar
  24. 24.
    Unnikrishnan, R., Pantofaru, C., & Hebert, M. (2007). Toward objective evaluation of image segmentation algorithms. IEEE Transactions on Pattern Analysis and Machine Intelligence, 6, 929–944.CrossRefGoogle Scholar
  25. 25.
    Meilǎ, M. (2005). Comparing clusterings: an axiomatic view. In Proceedings of the 22nd international conference on machine learning. New York: ACM (pp. 577–584).Google Scholar
  26. 26.
    Sharon, E., Galun, M., Sharon, D., et al. (2006). Hierarchy and adaptivity in segmenting visual scenes. Nature, 442(7104), 810.CrossRefGoogle Scholar
  27. 27.
    Sarkar, S., Das, S., & Chaudhuri, S. S. (2015). A multilevel color image thresholding scheme based on minimum cross entropy and differential evolution. Pattern Recognition Letters, 54, 27–35.CrossRefGoogle Scholar
  28. 28.
    Mignotte, M. (2014). A label field fusion model with a variation of information estimator for image segmentation. Information Fusion, 20, 7–20.CrossRefGoogle Scholar
  29. 29.
    Khelifi, L., & Mignotte, M. (2017). EFA-BMFM: A multi-criteria framework for the fusion of colour image segmentation. Information Fusion, 38, 104–121.CrossRefGoogle Scholar
  30. 30.
    Díaz-Pernas, F. J., Antón-Rodríguez, M., Martínez-Zarzuela, M., et al. (2013). Multi-nature hierarchical approach for natural image segmentation with pattern refinement feedback. Neurocomputing, 99, 325–338.CrossRefGoogle Scholar

Copyright information

© Springer Science+Business Media, LLC, part of Springer Nature 2019

Authors and Affiliations

  1. 1.The Engineering Research Center of IoT Technology and Application of the Ministry of EducationJiangnan UniversityWuXiChina

Personalised recommendations