Finding Semantic Structures in Image Hierarchies Using Laplacian Graph Energy

  • Yi-Zhe Song
  • Pablo Arbelaez
  • Peter Hall
  • Chuan Li
  • Anupriya Balikai
Part of the Lecture Notes in Computer Science book series (LNCS, volume 6314)


Many segmentation algorithms describe images in terms of a hierarchy of regions. Although such hierarchies can produce state of the art segmentations and have many applications, they often contain more data than is required for an efficient description. This paper shows Laplacian graph energy is a generic measure that can be used to identify semantic structures within hierarchies, independently of the algorithm that produces them. Quantitative experimental validation using hierarchies from two state of art algorithms show we can reduce the number of levels and regions in a hierarchy by an order of magnitude with little or no loss in performance when compared against human produced ground truth. We provide a tracking application that illustrates the value of reduced hierarchies.


Regular Graph Graph Complexity Maximally Stable Extremal Region Graph Energy Benchmark Score 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

Supplementary material

978-3-642-15561-1_50_MOESM1_ESM.avi (118 kb)
Electronic Supplementary Material (119 KB)
978-3-642-15561-1_50_MOESM2_ESM.avi (943 kb)
Electronic Supplementary Material (943 KB)
978-3-642-15561-1_50_MOESM3_ESM.avi (691 kb)
Electronic Supplementary Material (691 KB)
978-3-642-15561-1_50_MOESM4_ESM.avi (258 kb)
Electronic Supplementary Material (259 KB)


  1. 1.
    Ahuja, N.: A transform for multiscale image segmentation by integrated edge and region detection. TPAMI 18(12), 1211–1235 (1996)MathSciNetGoogle Scholar
  2. 2.
    Ahuja, N., Todorovic, S.: Connected segmentation tree - a joint representation of region layout and hierarchy. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 1–8 (June 2008)Google Scholar
  3. 3.
    Arbeláez, P., Maire, M., Fowlkes, C., Malik, J.: From contours to regions: An empitical evaluation. In: Computer Vision and Pattern Recognition (2009)Google Scholar
  4. 4.
    Bangham, J.A., Harvey, R.W., Ling, P.D., Aldridge, R.V.: Morphological scale-space preserving transforms in many dimensions. Journal of Electronic Imaging 5, 283–299 (1996)CrossRefGoogle Scholar
  5. 5.
    Comaniciu, D., Meer, P.: Mean shift: a robust approach toward feature space analysis. TPAMI 24(5), 603–614 (2002)Google Scholar
  6. 6.
    Cour, T., Benezit, F., Shi, J.: Spectral segmentation with multiscale graph decomposition. In: IEEE Computer Society Conference on Computer Vision and Pattern Recognition, CVPR 2005, vol. 2, pp. 1124–1131 (June 2005)Google Scholar
  7. 7.
    Day, J., So, W.: Singular value inequality and graph energy. Electronic Journal of Linear Algebra 16, 291–299 (2007)zbMATHMathSciNetGoogle Scholar
  8. 8.
    Escolano, F., Hancock, E.R., Lozano, M.A.: Polytopal graph complexity, matrix permanentsm, and embedding. In: da Vitoria Lobo, N., Kasparis, T., Roli, F., Kwok, J.T., Georgiopoulos, M., Anagnostopoulos, G.C., Loog, M. (eds.) S+SSPR 2008. LNCS, vol. 5342, pp. 237–246. Springer, Heidelberg (2008)CrossRefGoogle Scholar
  9. 9.
    Fisher, D.: Iterative optimization and simplification of hierarchical clusterings. J. Artif. Int. Res. 4(1), 147–179 (1996)zbMATHGoogle Scholar
  10. 10.
    Gutman, I., Zhou, B.: Laplacian energy of a graph. Linear Algebra and its applications 414, 29–37 (2006)zbMATHCrossRefMathSciNetGoogle Scholar
  11. 11.
    Kokkinos, I., Yuille, A.: Hop: Hierarchical object parsing. In: IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2009, pp. 802–809 (June 2009)Google Scholar
  12. 12.
    Liu, J., Yang, Y.: Multiresolution color image segmentation. IEEE Transactions on Pattern Analysis and Machine Intelligence 16, 689–700 (1994)CrossRefGoogle Scholar
  13. 13.
    Lucas, B.D., Kanade, T.: An iterative image registration technique with an application to stereo vision. In: Proceedings of the 7th International Joint Conference on Artificial Intelligence (IJCAI 1981), pp. 674–679 (April 1981)Google Scholar
  14. 14.
    Martin, D., Fowlkes, C., Malik, K.: Learning to detect natural image boundaries using local brightness, color and texture cues. TPAMI 26(5), 530–549 (2004)Google Scholar
  15. 15.
    Martin, D., Fowlkes, C., Tal, D., Malik, J.: A database of human segmented natural images and its application to evaluating segmentation algorithms and measuring ecological statistics. In: Proc. 8th Int’l Conf. Computer Vision., vol. 2, pp. 416–423 (2001)Google Scholar
  16. 16.
    Matas, J., Chum, O., Martin, U., Pajdla, T.: Robust wide basline stereo from maximally stable extremal regions. In: BMVC, pp. 384–393 (2002)Google Scholar
  17. 17.
    Maire, M.: Arbeláez, P., Fowlkes, C., Malik, J.: Using contours to detect and localize juncations in natural images. In: CVPR (2008)Google Scholar
  18. 18.
    Paris, S., Durand, F.: A topological approach to hierarchical segmentation using mean shift. In: CVPR, pp. 1–8. IEEE Computer Society Press, Los Alamitos (2007)Google Scholar
  19. 19.
    Ren, X.: Multi-scale improves boundary detection in natural images. In: Forsyth, D., Torr, P., Zisserman, A. (eds.) ECCV 2008, Part III. LNCS, vol. 5304, pp. 533–545. Springer, Heidelberg (2008)CrossRefGoogle Scholar
  20. 20.
    Robles-Kelly, A., Hancock, E.R.: A riemannian approach to graph embedding. Pattern Recognition 40, 1042–1056 (2007)zbMATHCrossRefGoogle Scholar
  21. 21.
    Shi, J., Malik, J.: Normalized cuts and image segmentation. TPAMI, 888–905 (2000)Google Scholar
  22. 22.
    Shokoufandeh, A., Dickinson, S.J., Siddiqi, K., Zucker, S.W.: Indexing using a spectral encoding of topological structure. In: International Conference on Pattern Recognition, pp. 491–497 (1999)Google Scholar
  23. 23.
    Tu, P., Saxena, T., Hartley, R.: Recognizing objects using color-annotated adjacency graphs. In: Shape, Contour and Grouping in Computer Vision, pp. 246–263 (1999)Google Scholar
  24. 24.
    White, D., Wilson, R.: Mixing spectral representations of graphs. In: International Conference on Pattern Recognition (2006)Google Scholar
  25. 25.
    Worthington, P., Hancock, E.: Region-based object recognition using shape-from-shading. In: Vernon, D. (ed.) ECCV 2000. LNCS, vol. 1842, pp. 455–471. Springer, Heidelberg (2000)CrossRefGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2010

Authors and Affiliations

  • Yi-Zhe Song
    • 1
  • Pablo Arbelaez
    • 2
  • Peter Hall
    • 1
  • Chuan Li
    • 1
  • Anupriya Balikai
    • 1
  1. 1.MTRCUniversity of BathBath
  2. 2.University of California at BerkeleyBerkeley

Personalised recommendations