International Journal of Computer Vision

, Volume 68, Issue 3, pp 289–317 | Cite as

Scale-Sets Image Analysis

  • Laurent GuiguesEmail author
  • Jean Pierre Cocquerez
  • Hervé Le Men


This paper introduces a multi-scale theory of piecewise image modelling, called the scale-sets theory, and which can be regarded as a region-oriented scale-space theory. The first part of the paper studies the general structure of a geometrically unbiased region-oriented multi-scale image description and introduces the scale-sets representation, a representation which allows to handle such a description exactly. The second part of the paper deals with the way scale-sets image analyses can be built according to an energy minimization principle. We consider a rather general formulation of the partitioning problem which involves minimizing a two-term-based energy, of the form λ C + D, where D is a goodness-of-fit term and C is a regularization term. We describe the way such energies arise from basic principles of approximate modelling and we relate them to operational rate/distorsion problems involved in lossy compression problems. We then show that an important subset of these energies constitutes a class of multi-scale energies in that the minimal cut of a hierarchy gets coarser and coarser as parameter λ increases. This allows us to devise a fast dynamic-programming procedure to find the complete scale-sets representation of this family of minimal cuts. Considering then the construction of the hierarchy from which the minimal cuts are extracted, we end up with an exact and parameter-free algorithm to build scale-sets image descriptions whose sections constitute a monotone sequence of upward global minima of a multi-scale energy, which is called the “scale climbing” algorithm. This algorithm can be viewed as a continuation method along the scale dimension or as a minimum pursuit along the operational rate/distorsion curve. Furthermore, the solution verifies a linear scale invariance property which allows to completely postpone the tuning of the scale parameter to a subsequent stage. For computational reasons, the scale climbing algorithm is approximated by a pair-wise region merging scheme: however the principal properties of the solutions are kept. Some results obtained with Mumford-Shah’s piece-wise constant model and a variant are provided and different applications of the proposed multi-scale analyses are finally sketched.


image segmentation multi-scale modelling energy minimization hierarchy 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. Alvarez, L., Guichard, F., Lions, P.-L. and Morel, J.-M. 1993. Axioms and Fundamental Equations of Image Processing. Arch. Rational Mech. Anal., 123:199–257.MathSciNetCrossRefzbMATHGoogle Scholar
  2. Badaud, J., Witkin, A.P., Baudin, M., and Duda, R.O. Uniqueness of the gaussian kernel for scale-space filtering. 1986. IEEE trans. Pattern Anal. Mach. Intell., 8:26–33.Google Scholar
  3. Ballester, C., Caselles, V., and González, M. 1994. Affine Invariant Segmentation by Variational Method. In Proc. of 9th RFIA, Paris, pp. 379–390.Google Scholar
  4. Beucher, S. and Lantuejoul, C. 1979. Use of watersheds in contour detection. In Proceedings of Int. Workshop on Image Processing, Real-time Edge and Motion Detection/Estimation, Rennes, (France).Google Scholar
  5. Beucher, S. and Meyer, F. 1993. The morphological approach to segmentation : the watershed transformation. In Dougherty, (ed.), Mathematical Morphology in Image Marcel Dekker, Processing, chapter 12, pp. 433–481.Google Scholar
  6. Blake, A. and Zisserman, A. 1987. Visual Reconstruction. MIT Press, Cambridge, MA.Google Scholar
  7. Bosworth, J. H. and Acton, S.T. 2003. Morphological scale-space in image processing. Digital Signal Processing, 13(3).Google Scholar
  8. Boykov, Y., Veksler, O., and Zabih, R. 1999. Fast approximate energy minimization via graph cuts: ICCV, Seventh International Conference on Computer Vision (ICCV′99), Vol. 1, p. 377. IEEE trans. on Pattern Analysis and Machine Intelligence.Google Scholar
  9. Breiman, L., Friedman, J.H., Olshen, R.A., and Stone, C.J. 1984. Classification and Regression Trees. Wadsworth and Brooks.Google Scholar
  10. Chehata, N., Jung F., Pierrot-Deseilligny, M., and Stamon, G. 2003. A region based approach for 3D-roof reconstruction form HR satellite stereo pairs. In DICTA 03, Vol II, Sydney, Australia, pp. 889–898.Google Scholar
  11. Deriche, R. 1987. Using Canny’s criteria to derive a recursively implemented optimal edge detector. International Journal of Computer Vision, 1(2):167–187 .CrossRefGoogle Scholar
  12. Donoho, D. 1997. CART and best-ortho-basis selection : A connection. Annals of statistics, 25:1870–1911.zbMATHMathSciNetCrossRefGoogle Scholar
  13. Fuchs, F. and Le Men, H. 1999. Detecting planar patches in urban scenes. In Proceedings of SPIE, Visual Information Processing VIII, Orlando, USA, volume 3716, pp. 167–176, avril .Google Scholar
  14. Garrido, L., and Salembier, P. 1999. Representing and retrieving regions using binary partition trees. In IEEE Int. Conference on Image Processing ICIP′99, Kobe, Japan, pp. 25–28.Google Scholar
  15. Geman S., and Geman, D. 1984. Stochastic relaxation, gibbs distributions, and the bayesian restoration of images. IEEE Transactions on Pattern Analysis and Machine Intelligence, 6(6):721–741.zbMATHGoogle Scholar
  16. Guigues, L. 2003. Modéles multi-échelles pour la segmentation d’images (Multi-scale models for image segmentation). PhD thesis, Univ. de Cergy-Pontoise, France.Google Scholar
  17. Guigues, L., Le Men, H., and Cocquerez, J.-P. 2001. Segmentation d’image par minimisation d’un critére mdl dans une pyramide de segmentations. In actes du XVIIIéme colloque GRETSI sur le Traitement des Images et du Signal, Toulouse, France.Google Scholar
  18. Guigues L., Men Hervé Le, and Cocquerez Jean-Pierre. 2003. The hierarchy of the cocoons of a graph and its application to image segmentation. Pattern Recognition Letters, 24(3):1024–1066.Google Scholar
  19. Haralick and Shapiro 1991. Glossary of computer vision terms. Pattern Recognition, 24(1):69–93 .CrossRefGoogle Scholar
  20. Haris, K., Estradiadis, S.N., Maglaveras, N., and Katsaggelos, A.K. 1998. Hybrid image segmentation using watersheds and fast region merging. IEEE Trans. on Image Processing, 7(12):1684–1699.CrossRefGoogle Scholar
  21. Horowitz, S.L. and Pavlidis, Theodosios. 1976. Picture Segmentation by a Tree Traversal Algorithm. J. of the Assoc. for Comp. Mach., 23(2):368–388.zbMATHGoogle Scholar
  22. Jackay, P.T., Boles, W.W., and Deriche, M. 1994. Morphological fingerprints and their use in object recognition in range images. In Proc. of IEEE Conf. on Acoustics, Speech and Signal Processing, Adelaide, Australia, pp. V5–V8.Google Scholar
  23. Jackay, P. T. and Deriche, M. 1987. Scale-space properties of the multiscale morphological dilation-erosion. IEEE trans. Pattern Anal. Mach. Intell., 9:532–550.CrossRefGoogle Scholar
  24. Jolion, J.-M. and Montanvert, A. 1992. The adaptive pyramid, a framework for 2d image analysis. CVGIP: Image Understanding, 55(3):339–348.CrossRefzbMATHGoogle Scholar
  25. Kanungo, T., Dom, B., Niblack, W., Steele, D., and Sheinvald, J. 1995. MDL-Based Multi-Band Image Segmentation Using a Fast Region Merging Scheme. Technical Report RJ 9960 (87919), IBM Research Division.Google Scholar
  26. Koenderink, J.J. 1984. The structure of images. Biol. Cybern., 50:363–370.zbMATHMathSciNetCrossRefGoogle Scholar
  27. Koepfler, G., Lopez, C. and Morel, J.M. 1994 A Multiscale Algorithm for Image Segmentation by Variational Method. SIAM Journal on Numerical Analysis, 31(1):282–299.MathSciNetCrossRefzbMATHGoogle Scholar
  28. Kurita, T. 1995. A Efficient Clustering Algorithm for Region Merging. IEICE Trans. on Information and Systems, E78-D(12):1546–1551.Google Scholar
  29. Lance and Williams 1967. A general theory of classificatory sorting strategies. The Computer Journal, 9:373–380.Google Scholar
  30. Leclerc, Y.G. 1989. Constructing Simple Stable Descriptions for Image Partitioning. International Journal of Computer Vision, 3(1):73–102.CrossRefGoogle Scholar
  31. Lindeberg, T. 1994. Scale-space Theory in Computer Vision. Kluwer Academic.Google Scholar
  32. Marcotegui, B., Zanoguera, F., Correia, P., Rosa, R., Marqués, F., Mech, R., and Wollborn, M. 1999. A video object generation tool allowing friendly user interaction. In Proc. of ICIP′99, Kobe (Japan).Google Scholar
  33. Marr, D. 1982. Vision: A computational Investigation Into the Human Representation and Processing of Visual Information. W.H. Freeman and Company, NY 29–61.Google Scholar
  34. Meyer, F. 1999a. Graph Based Morphological Segmentation. In Proc. 2nd IAPR TC-15 workshop on Graph based Representations, Haindorf, Austria, pp. 51–60.Google Scholar
  35. Meyer, F. 1999b. Morphological multiscale and interactive segmentation. In Proc. of IEEE-EURASIP Workshop on Nonlinear Signal and Image Processing, Antalya, Turkey.Google Scholar
  36. Meyer, F. 2001. An overview of morphological segmentation. Int. Journal of Pattern Recognition and Artificial Intelligence, 15(7):1089–1118.CrossRefGoogle Scholar
  37. Montanvert, A., Meer, P., and Rosenfeld, A. 1991. Hierarchical image analysis using irregular tesselations. IEEE Transactions on Pattern Analysis and Machine Intelligence, 13(4):307–316.CrossRefGoogle Scholar
  38. Morel, J.-M. and Solimini, S. 1995. Variational methods in image segmentation. Progress in Nonlinear Differential Equations and Their Applications, vol. 14, Birkhaüser, Boston.Google Scholar
  39. Mumford, D., and Shah, J. 1989. Optimal approximations by piecewise smooth functions and associated variational problems. Comm. on Pure and Applied Math., 17(4):577–685.MathSciNetGoogle Scholar
  40. Park, K.-R. and Lee, C.-N. 1996. Scale-space using mathematical morphology. IEEE trans. Pattern Anal. Mach. Intell., 8:1121–1126.CrossRefGoogle Scholar
  41. Perona, P. and Malik, J. 1990. Scale space and edge detection using anisotropic diffusion. IEEE trans. on PAMI, 12(7):629–639.Google Scholar
  42. Poggi, G. and Ragozini, A.R.P. 1999. Image segmentation by a tree-structured markov random field. IEEE Signal Processing Letters, 6:155–157.CrossRefGoogle Scholar
  43. Salembier, P. and Garrido, L. 2000a. Binary partition tree as an Efficient Representation for Image Processing, Segmentation, and Information Retrieval. IEEE Trans. on Image Processing, 9(4):561–576.CrossRefGoogle Scholar
  44. Salembier, P. and Garrido, L. 2000b. Visual segments tree creation for MPEG-7 description schemes. In IEEE Int. Conference on Multimedia and Exposition ICME′2000, NY city, NY USA.Google Scholar
  45. Salembier, P., Oliveras, Albert., and Garrido, Luis. 1998. Anti-extensive Connected Operators for Image and Sequence Processing. IEEE Trans. on Image Processing, 7(4):555–570.CrossRefGoogle Scholar
  46. Salembier, P. and Serra, J. 1995. Flat Zones Filtering, Connected Operators, and Filters by Reconstruction. IEEE trans. on Image Processing, 4(8):1153–1160.CrossRefGoogle Scholar
  47. Sapiro, G. and Tannenbaum, A. 1993. Affine Invariant Scale Space. Int. J. of Computer Vision, 11:24–44.Google Scholar
  48. Serra J. and Salembier, P. 1993. Connected operators and pyramids. In SPIE conf. on Image Algebra and Mathematical Morphology, San Diego, CA, volume 2030, pp. 65–76.Google Scholar
  49. Shannon, C. 1959. Coding theorems for a discrete source with a fidelity criterion. IRE Natl. Conv. Record., Part 4:142–163.Google Scholar
  50. Shi, J. and Malik, J. 2000. Normalized Cuts and Image Segmentation. IEEE Transactions on Pattern Analysis and Machine Intelligence, 22(8):888–905.CrossRefGoogle Scholar
  51. Taillandier, F., Guigues, L., and Deriche, R. 2003. A framework for constrained multi-scale range image segmentation. In Proc. of IEEE Int. Conf. on Image Processing (ICIP′03), Barcelona, Spain.Google Scholar
  52. Roger, T.-S. and Marc, P.-D. 2004. An edge-based method for registering a graph onto an image with application to cadastre registration. In Proc. of the 2004 International Conference on Pattern Recognition (ICPR 2004), Cambridge, UK.Google Scholar
  53. Wang, J.-P. 1998. Stochastic Relaxation on Partitions With Connected Components and its Application to Image Segmentation. IEEE Transactions on Pattern Analysis and Machine Intelligence, 20(6):619–636.CrossRefGoogle Scholar
  54. Witkin, A.W. 1983. Scale space filtering. In Proc. of 8th Int. Joint Conf. on Artificial Intell., Karlsruhe, West Germany, pp. 1019–1021.Google Scholar
  55. Zanoguera, F., Marcotegui, B., and Meyer, F. 1999. A toolbox for interactive segmentation based on nested partitions. In proc. of ICIP′99, Kobe (Japan).Google Scholar

Copyright information

© Springer Science + Business Media, LLC 2006

Authors and Affiliations

  • Laurent Guigues
    • 1
    Email author
  • Jean Pierre Cocquerez
    • 2
  • Hervé Le Men
    • 1
  1. 1.Institut Géographique National, MATIS Lab.Saint-Mandé CedexFrance
  2. 2.Université de Technologie de Compiègne, HEUDIASYC Lab., UMR CNRS 6599Compiègne CedexFrance

Personalised recommendations