Journal of Mathematical Imaging and Vision

, Volume 60, Issue 4, pp 479–502 | Cite as

Hierarchical Segmentations with Graphs: Quasi-flat Zones, Minimum Spanning Trees, and Saliency Maps

  • Jean CoustyEmail author
  • Laurent Najman
  • Yukiko Kenmochi
  • Silvio Guimarães


Hierarchies of partitions are generally represented by dendrograms (direct representation). They can also be represented by saliency maps or minimum spanning trees. In this article, we precisely study the links between these three representations. In particular, we provide a new bijection between saliency maps and hierarchies based on quasi-flat zones as often used in image processing and we characterize saliency maps and minimum spanning trees as solutions to constrained minimization problems where the constraint is quasi-flat zones preservation. In practice, these results make up a toolkit for designing new hierarchical methods where one can choose the most convenient representation. They also invite us to process non-image data with morphological hierarchies. More precisely, we show the practical interest of the proposed framework for: (i) hierarchical watershed image segmentations, (ii) combinations of different hierarchical segmentations, (iii) hierarchicalizations of some non-hierarchical image segmentation methods based on regional dissimilarities, and (iv) hierarchical analysis of geographic data.


Mathematical morphology Hierarchy of partitions Hierarchical image segmentation Watershed Saliency maps Minimum spanning trees Hierarchical classification 



The research leading to these results has received funding from the French Agence Nationale de la Recherche (Contract ANR-2010-BLAN-0205-03), the French Committee for the Evaluation of Academic and Scientific Cooperation with Brazil, and the Brazilian Federal Agency of Support and Evaluation of Postgraduate Education (Program CAPES/PVE: Grant 064965/2014-01, and Program CAPES/COFECUB: Grant 592/08).


  1. 1.
    Achanta, R., Shaji, A., Smith, K., Lucchi, A., Fua, P., Süsstrunk, S.: Slic superpixels compared to state-of-the-art superpixel methods. IEEE Trans. Pattern Anal. Mach. Intell. 34(11), 2274–2282 (2012)CrossRefGoogle Scholar
  2. 2.
    Alpert, S., Galun, M., Brandt, A., Basri, R.: Image segmentation by probabilistic bottom-up aggregation and cue integration. IEEE Trans. Pattern Anal. Mach. Intell. 34(2), 315–327 (2012)CrossRefGoogle Scholar
  3. 3.
    Arbelaez, P., Maire, M., Fowlkes, C., Malik, J.: Contour detection and hierarchical image segmentation. IEEE Trans. Pattern Anal. Mach. Intell. 33(5), 898–916 (2011)CrossRefGoogle Scholar
  4. 4.
    Arbeláez, P., Cohen, L.: A metric approach to vector-valued image segmentation. Int. J. Comput. Vis. 69(1), 119–126 (2006)CrossRefGoogle Scholar
  5. 5.
    Bender, M., Farach-Colton, M.: The LCA problem revisited. In: Latin American Theoretical INformatics, pp. 88–94 (2000)Google Scholar
  6. 6.
    Bertrand, G.: On topological watersheds. J. Math. Imaging Vis. 22(2–3), 217–230 (2005)MathSciNetCrossRefGoogle Scholar
  7. 7.
    Beucher, S.: Watershed, hierarchical segmentation and waterfall algorithm. In: Serra, J., Soille, P. (eds.) Mathematical Morphology and Its Applications to Image and Signal Processing, pp. 69–76. Kluwer (1994)Google Scholar
  8. 8.
    Cormen, T.H., Leiserson, C.E., Rivest, R.L., Stein, C., et al.: Introduction to Algorithms, 2nd edn. MIT Press, Cambridge (2001)zbMATHGoogle Scholar
  9. 9.
    Couprie, M., Najman, L., Bertrand, G.: Quasi-linear algorithms for the topological watershed. J. Math. Imaging Vis. 22(2–3), 231–249 (2005)MathSciNetCrossRefzbMATHGoogle Scholar
  10. 10.
    Cousty, J., Bertrand, G., Najman, L., Couprie, M.: Watershed cuts: minimum spanning forests and the drop of water principle. IEEE Trans. Pattern Anal. Mach. Intell. 31(8), 1362–1374 (2009)CrossRefGoogle Scholar
  11. 11.
    Cousty, J., Najman, L.: Incremental algorithm for hierarchical minimum spanning forests and saliency of watershed cuts. Math. Morphol. Its Appl. Image Signal Process. Lect. Notes Comput. Sci. 6671, 272–283 (2011)MathSciNetzbMATHGoogle Scholar
  12. 12.
    Cousty, J., Najman, L., Kenmochi, Y., Guimarães, S.: New characterizations of minimum spanning trees and of saliency maps based on quasi-flat zones. In: Benediktsson, J.A., Chanussot, J., Najman, L., Talbot, H. (eds.) Mathematical Morphology and Its Applications to Signal and Image Processing, Lecture Notes in Computer Science, vol. 9082, pp. 205–216. Springer, UK (2015). doi: 10.1007/978-3-319-18720-4_18
  13. 13.
    Cousty, J., Najman, L., Perret, B.: Constructive links between some morphological hierarchies on edge-weighted graphs. In: Luengo Hendriks, C.L., Borgefors, G., Strand, R. (eds.) Mathematical Morphology and Its Applications to Image and Signal Processing, pp. 86–97. Springer, Berlin (2013)Google Scholar
  14. 14.
    Cousty, J., Najman, L., Serra, J.: Raising in watershed lattices. In: 15th IEEE International Conference on Image Processing, 2008 (ICIP 2008), pp. 2196–2199. IEEE (2008)Google Scholar
  15. 15.
    Felzenszwalb, P., Huttenlocher, D.: Efficient graph-based image segmentation. Int. J. Comput. Vis. 59, 167–181 (2004)CrossRefGoogle Scholar
  16. 16.
    Gordon, A.D.: A review of hierarchical classification. J. R. Stat. Soc. Ser. A (Gen.) 150, 119–137 (1987)MathSciNetCrossRefzbMATHGoogle Scholar
  17. 17.
    Guigues, L., Cocquerez, J.P., Men, H.L.: Scale-sets image analysis. Int. J. Comput. Vis. 68(3), 289–317 (2006)CrossRefGoogle Scholar
  18. 18.
    Guimarães, S.J.F., Cousty, J., Kenmochi, Y., Najman, L.: A hierarchical image segmentation algorithm based on an observation scale. In: SSPR/SPR, pp. 116–125. Springer, Berlin (2012)Google Scholar
  19. 19.
    Guimarães, S., Kenmochi, Y., Cousty, J., Patrocínio, Z., Jr., Najman, L.: Hierarchizing Graph-Based Image Segmentation Algorithms Relying on Region Dissimilarity the Case of the Felzenszwalb–Huttenlocher Method (2016). Research report on HAL
  20. 20.
    Guimarães, S.J.F., do Patrocínio, Z.K.G.J.: A graph-based hierarchical image segmentation method based on a statistical merging predicate. In: Petrosino, A. (ed.) Image Analysis and Processing—ICIAP 2013, Lecture Notes in Computer Science, vol. 8156, pp. 11–20. Springer, Berlin (2013)Google Scholar
  21. 21.
    Guimarães, S.J.F., do Patrocínio, Z.K.G.J., Kenmochi, Y., Cousty, J., Najman, L.: Hierarchical image segmentation relying on a likelihood ratio test. In: Murino, V., Puppo, E. (eds.) Image Analysis and Processing—ICIAP 2015, Lecture Notes in Computer Science, vol. 9280, pp. 25–35. Springer. Berlin (2015)Google Scholar
  22. 22.
    Kiran, B.R., Serra, J.: Fusion of ground truths and hierarchies of segmentations. Pattern Recognit. Lett. 47, 63–71 (2014). Advances in Mathematical MorphologyCrossRefGoogle Scholar
  23. 23.
    Kiran, B.R., Serra, J., et al.: Global-local optimizations on hierarchies of segmentations. Pattern Recognit. 47(1), 12–24 (2014)CrossRefzbMATHGoogle Scholar
  24. 24.
    Kovalevsky, V.A.: Finite topology as applied to image analysis. Comput. Vis. Graph. Image Process. 46(2), 141–161 (1989)CrossRefGoogle Scholar
  25. 25.
    Leclerc, B.: Description combinatoire des ultramétriques. Mathématiques et Sciences humaines 73, 5–37 (1981)zbMATHGoogle Scholar
  26. 26.
    Malmberg, F., Hendriks, C.L.L.: An efficient algorithm for exact evaluation of stochastic watersheds. Pattern Recognit. Lett. 47, 80–84 (2014). Advances in Mathematical MorphologyCrossRefGoogle Scholar
  27. 27.
    Martin, D.R., Fowlkes, C.C., Malik, J.: Learning to detect natural image boundaries using local brightness, color, and texture cues. IEEE Trans. Pattern Anal. Mach. Intell. 26(5), 530–549 (2004)CrossRefGoogle Scholar
  28. 28.
    Meyer, F.: The dynamics of minima and contours. In: Maragos, P., Schafer, R., Butt, M. (eds.) Mathematical Morphology and Its Applications to Image and Signal Processing, pp. 329–336. Kluwer Academic Publisher (1996)Google Scholar
  29. 29.
    Meyer, F., Maragos, P.: Morphological scale-space representation with levelings. In: Nielsen, M., Johansen, P., Olsen, O.F., Weickert, J. (eds.) Scale-Space Theories in Computer Vision, Lecture Notes in Computer Science, vol. 1682, pp. 187–198. Springer, Berlin (1999)Google Scholar
  30. 30.
    Meyer, F., Najman, L.: Segmentation, Minimum Spanning Tree and Hierarchies. Mathematical Morphology: From Theory to Applications, pp. 229–261. Wiley (2013)Google Scholar
  31. 31.
    Monasse, P., Guichard, F.: Fast computation of a contrast-invariant image representation. IEEE Trans. Image Process. 9(5), 860–872 (2000)CrossRefGoogle Scholar
  32. 32.
    NetworkX Examples: Knuth Miles. Accessed 21 Sept 2015
  33. 33.
    Nagao, M., Matsuyama, T., Ikeda, Y.: Region extraction and shape analysis in aerial photographs. Comput. Graph. Image Process. 10(3), 195–223 (1979)CrossRefGoogle Scholar
  34. 34.
    Najman, L.: On the equivalence between hierarchical segmentations and ultrametric watersheds. J. Math. Imaging Vis. 40(3), 231–247 (2011)MathSciNetCrossRefzbMATHGoogle Scholar
  35. 35.
    Najman, L., Cousty, J., Perret, B.: Playing with Kruskal: algorithms for morphological trees in edge-weighted graphs. In: Luengo Hendriks, C.L., Borgefors, G., Strand, R. (eds.) Mathematical Morphology and Its Applications to Image and Signal Processing, pp. 135–146. Springer, Berlin (2013)Google Scholar
  36. 36.
    Najman, L., Schmitt, M.: Geodesic saliency of watershed contours and hierarchical segmentation. IEEE Trans. Pattern Anal. Mach. Intell. 18(12), 1163–1173 (1996)CrossRefGoogle Scholar
  37. 37.
    Nakache, J.P., Confais, J.: Approche pragmatique de la classification: arbres hiérarchiques, partitionnements. Editions Technip (2004)Google Scholar
  38. 38.
    Nock, R., Nielsen, F.: Statistical region merging. IEEE Trans. Pattern Anal. Mach. Intell. 26(11), 1452–1458 (2004)CrossRefGoogle Scholar
  39. 39.
    Pavlidis, T.: Structural Pattern Recognition, vol. 2. Springer, New York (1977)zbMATHGoogle Scholar
  40. 40.
    Peng, B., Zhang, L., Zhang, D.: Automatic image segmentation by dynamic region merging. IEEE Trans. Image Process. 20(12), 3592–3605 (2011)MathSciNetCrossRefzbMATHGoogle Scholar
  41. 41.
    Perret, B., Cousty, J., Guimarães, S., Maya, D.: Evaluation of Hierarchical Watersheds (2016). (submitted)
  42. 42.
    Philipp-Foliguet, S., Jordan, M., Najman, L., Cousty, J.: Artwork 3D model database indexing and classification. Pattern Recognit. 44(3), 588–597 (2011)CrossRefzbMATHGoogle Scholar
  43. 43.
    Ronse, C.: Ordering partial partitions for image segmentation and filtering: merging, creating and inflating blocks. J. Math. Imaging Vis. 49(1), 202–233 (2014)MathSciNetCrossRefzbMATHGoogle Scholar
  44. 44.
    Rother, C., Kolmogorov, V., Blake, A.: GrabCut: interactive foreground extraction using iterated graph cuts. ACM Trans. Graph. 23(3), 309–314 (2004)CrossRefGoogle Scholar
  45. 45.
    Salembier, P., Garrido, L.: Binary partition tree as an efficient representation for image processing, segmentation, and information retrieval. IEEE Trans. Image Process. 9(4), 561–576 (2000)CrossRefGoogle Scholar
  46. 46.
    Salembier, P., Oliveras, A., Garrido, L.: Antiextensive connected operators for image and sequence processing. IEEE Trans. Image Process. 7(4), 555–570 (1998)CrossRefGoogle Scholar
  47. 47.
    Silla Jr., C.N., Freitas, A.A.: A survey of hierarchical classification across different application domains. Data Min. Knowl. Discov. 22(1–2), 31–72 (2011)MathSciNetCrossRefzbMATHGoogle Scholar
  48. 48.
    Skupin, A.: The world of geography: visualizing a knowledge domain with cartographic means. Proc. Natl. Acad. Sci. 101(suppl 1), 5274–5278 (2004)CrossRefGoogle Scholar
  49. 49.
    Skupin, A., Fabrikant, S.I.: Spatialization methods: a cartographic research agenda for non-geographic information visualization. Cartogr. Geogr. Inf. Sci. 30(2), 99–119 (2003)CrossRefGoogle Scholar
  50. 50.
    Soille, P.: Constrained connectivity for hierarchical image partitioning and simplification. IEEE Trans. Pattern Anal. Mach. Intell. 30(7), 1132–1145 (2008)CrossRefGoogle Scholar
  51. 51.
    Soille, P., Najman, L.: On morphological hierarchical representations for image processing and spatial data clustering. In: Köthe, U., Montanvert, A., Soille, P. (eds.) Applications of Discrete Geometry and Mathematical Morphology: First International Workshop, WADGMM 2010, Istanbul, Turkey, August 22, 2010, Revised Selected Papers, pp. 43–67. Springer, Berlin (2012)CrossRefGoogle Scholar
  52. 52.
    Urbach, E., Roerdink, J., Wilkinson, M.: Connected shape-size pattern spectra for rotation and scale-invariant classification of gray-scale images. IEEE Trans. Pattern Anal. Mach. Intell. 29(2), 272–285 (2007)CrossRefGoogle Scholar
  53. 53.
    Vachier, C., Meyer, F.: Extinction value: a new measurement of persistence. In: IEEE Workshop on Nonlinear Signal and Image Processing, pp. 254–257 (1995)Google Scholar

Copyright information

© Springer Science+Business Media, LLC 2017

Authors and Affiliations

  • Jean Cousty
    • 1
    Email author
  • Laurent Najman
    • 1
  • Yukiko Kenmochi
    • 1
  • Silvio Guimarães
    • 1
    • 2
  1. 1.Université Paris-Est, Laboratoire d’Informatique Gaspard-Monge, A3SI, ESIEE Paris, CNRSMarne-la-ValléeFrance
  2. 2.PUC Minas - ICEI - DCC - VIPLABBelo HorizonteBrazil

Personalised recommendations