Soft Image Segmentation: On the Clustering of Irregular, Weighted, Multivariate Marked Networks

  • Raphaël CeréEmail author
  • François Bavaud
Conference paper
Part of the Communications in Computer and Information Science book series (CCIS, volume 936)


The contribution exposes and illustrates a general, flexible formalism, together with an associated iterative procedure, aimed at determining soft memberships of marked nodes in a weighted network. Gathering together spatial entities which are both spatially close and similar regarding their features is an issue relevant in image segmentation, spatial clustering, and data analysis in general. Unoriented weighted networks are specified by an “exchange matrix”, determining the probability to select a pair of neighbors. We present a family of membership-dependent free energies, whose local minimization specifies soft clusterings. The free energy additively combines a mutual information, as well as various energy terms, concave or convex in the memberships: within-group inertia, generalized cuts (extending weighted Ncut and modularity), and membership discontinuities (generalizing Dirichlet forms). The framework is closely related to discrete Markov models, random walks, label propagation and spatial autocorrelation (Moran’s I), and can express the Mumford-Shah approach. Four small datasets illustrate the theory.


Free energy Image segmentation Iterative clustering Soft K-means Laplacian Modularity Moran’s I Mumford-shah functional Multivariate features Ncut Soft membership Spatial autocorrelation Spatial clustering 


  1. 1.
    Shi, J., Malik, J.: Normalized cuts and image segmentation. IEEE Trans. Pattern Anal. Mach. Intell. 22, 888–905 (2000)CrossRefGoogle Scholar
  2. 2.
    Grady, L., Schwartz, E.L.: Isoperimetric graph partitioning for image segmentation. IEEE Trans. Pattern Anal. Mach. Intell. 28, 469–475 (2006)CrossRefGoogle Scholar
  3. 3.
    Newman, M.E.: Modularity and community structure in networks. Proc. Natl. Acad. Sci. 103, 8577–8582 (2006)CrossRefGoogle Scholar
  4. 4.
    Berger, J., Snell, J.L.: On the concept of equal exchange. Syst. Res. Behav. Sci. 2, 111–118 (1957)MathSciNetCrossRefGoogle Scholar
  5. 5.
    Ceré, R., Bavaud, F.: Multi-labelled image segmentation in irregular, weighted networks: a spatial autocorrelation approach. In: Proceedings of the 3rd International Conference on Geographical Information Systems Theory, Applications and Management, Scitepress, pp. 62–69 (2017)Google Scholar
  6. 6.
    Zhu, X., Ghahramani, Z.: Learning from labeled and unlabeled data with label propagation. Technical report CMU-CALD-02-107, Carnegie Mellon University (2002)Google Scholar
  7. 7.
    Raghavan, U.N., Albert, R., Kumara, S.: Near linear time algorithm to detect community structures inlarge-scale networks. Phys. Rev. E 76, 036106 (2007)CrossRefGoogle Scholar
  8. 8.
    von Luxburg, U.: A tutorial on spectral clustering. Stat. Comput. 17, 395–416 (2007)MathSciNetCrossRefGoogle Scholar
  9. 9.
    Bavaud, F.: Aggregation invariance in general clustering approaches. Adv. Data Anal. Classif. 3, 205–225 (2009)MathSciNetCrossRefGoogle Scholar
  10. 10.
    Bavaud, F.: On the Schoenberg transformations in data analysis: theory and illustrations. J. Classif. 28, 297–314 (2011)MathSciNetCrossRefGoogle Scholar
  11. 11.
    Rose, K., Gurewitz, E., Fox, G.: A deterministic annealing approach to clustering. Pattern Recogn. Lett. 11, 589–594 (1990)CrossRefGoogle Scholar
  12. 12.
    Mumford, D., Shah, J.: Optimal approximations by piecewise smooth functions and associated variational problems. Commun. Pure Appl. Math. 42, 577–685 (1989)MathSciNetCrossRefGoogle Scholar
  13. 13.
    Petitot, J.: An introduction to the Mumford-Shah segmentation model. J. Physiol.-Paris 97, 335–342 (2003)CrossRefGoogle Scholar
  14. 14.
    Vitti, A.: The Mumford-Shah variational model for image segmentation: an overview of the theory, implementation and use. ISPRS J. Photogramm. Remote. Sens. 69, 50–64 (2012)CrossRefGoogle Scholar
  15. 15.
    Gaumer, P., Moliterni, C.: Dictionnaire mondial de la bande dessinée. Larousse, Paris (1994)Google Scholar
  16. 16.
    Couprie, C., Grady, L., Najman, L., Talbot, H.: Power watershed: a unifying graph-based optimization framework. IEEE Trans. Pattern Anal. Mach. Intell. 33, 1384–1399 (2011)CrossRefGoogle Scholar
  17. 17.
    Fouss, F., Saerens, M., Shimbo, M.: Algorithms and Models for Network Data and Link Analysis. Cambridge University Press, Cambridge (2016)CrossRefGoogle Scholar
  18. 18.
    Bavaud, F., Guex, G.: Interpolating between random walks and shortest paths: a path functional approach. In: Aberer, K., Flache, A., Jager, W., Liu, L., Tang, J., Guéret, C. (eds.) SocInfo 2012. LNCS, vol. 7710, pp. 68–81. Springer, Heidelberg (2012). Scholar
  19. 19.
    Françoisse, K., Kivimäki, I., Mantrach, A., Rossi, F., Saerens, M.: A bag-of-paths framework for network data analysis. arXiv preprint arXiv:1302.6766 (2013)
  20. 20.
    Devooght, R., Mantrach, A., Kivimäki, I., Bersini, H., Jaimes, A., Saerens, M.: Random walks based modularity: application to semi-supervised learning. In: Proceedings of the 23rd International Conference on World Wide Web, pp. 213–224. ACM (2014)Google Scholar
  21. 21.
    Sinop, A.K., Grady, L.: A seeded image segmentation framework unifying graph cuts and random walker which yields a new algorithm. In: 2007 IEEE 11th International Conference on Computer Vision. ICCV 2007, pp. 1–8. IEEE (2007)Google Scholar
  22. 22.
    Guex, G.: Interpolating between random walks and optimal transportation routes: flow with multiple sources and targets. Phys. A: Stat. Mech. Appl. 450, 264–277 (2016)MathSciNetCrossRefGoogle Scholar
  23. 23.
    Doyle, P.G., Snell, J.L.: Random Walks and Electric Networks. Mathematical Association of America, Washington D.C. (1984)zbMATHGoogle Scholar
  24. 24.
    Grady, L.: Random walks for image segmentation. IEEE Trans. Pattern Anal. Mach. Intell. 28, 1768–1783 (2006)CrossRefGoogle Scholar
  25. 25.
    Critchley, F., Fichet, B.: The partial order by inclusion of the principal classes of dissimilarity on a finite set, and some of their basic properties. In: Van Cutsem, B. (ed.) Classification and Dissimilarity Analysis. LNS, vol. 93, pp. 5–65. Springer, New York (1994). Scholar
  26. 26.
    Bivand, R.S., Pebesma, E.J., Gomez-Rubio, V., Pebesma, E.J.: ApplieD Spatial Data Analysis with R, vol. 747248717. Springer, New York (2008). Scholar
  27. 27.
    LeSage, J.P.: An introduction to spatial econometrics. Revue d’économie industrielle 19–44 (2008). Field number 123Google Scholar
  28. 28.
    Arbia, G.: Spatial Data Configuration in Statistical Analysis of rEgional Economic and Related Problems, vol. 14. Springer, Dordrecht (2012). Scholar
  29. 29.
    Anselin, L.: Spatial Econometrics: Methods and Models, vol. 4. Springer, Dordrech (2013). Scholar
  30. 30.
    Griffith, D.A.: Spatial Autocorrelation and Spatial Filtering: Gaining Understanding Through Theory and Scientific Visualization. Springer, Heidelberg (2013). Scholar
  31. 31.
    Besag, J.: On the statistical analysis of dirty pictures. J. R. Stat. Soc. Ser. B (Methodol.) 48, 259–302 (1986)MathSciNetzbMATHGoogle Scholar
  32. 32.
    Greig, D.M., Porteous, B.T., Seheult, A.H.: Exact maximum a posteriori estimation for binary images. J. R. Stat. Soc. Ser. B (Methodol.) 51, 271–279 (1989)Google Scholar
  33. 33.
    Anselin, L.: Local indicators of spatial association - LISA. Geogr. Anal. 27, 93–115 (1995)CrossRefGoogle Scholar
  34. 34.
    White, S., Smyth, P.: A spectral clustering approach to finding communities in graphs. In: Proceedings of the 2005 SIAM International Conference on Data Mining, pp. 274–285. SIAM (2005)Google Scholar
  35. 35.
    Ng, A.Y., Jordan, M.I., Weiss, Y.: On spectral clustering: analysis and an algorithm. In: Advances in Neural Information Processing Systems, pp. 849–856 (2002)Google Scholar
  36. 36.
    Lebichot, B., Saerens, M.: An experimental study of graph-based semi-supervised classification with additional node information. arXiv preprint arXiv:1705.08716 (2017)
  37. 37.
    Yen, L., Saerens, M., Mantrach, A., Shimbo, M.: A family of dissimilarity measures between nodes generalizing both the shortest-path and the commute-time distances. In: Proceedings of the 14th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. KDD 2008, pp. 785–793. ACM, New York (2008)Google Scholar
  38. 38.
    Bavaud, F.: Euclidean distances, soft and spectral clustering on weighted graphs. In: Balcázar, J.L., Bonchi, F., Gionis, A., Sebag, M. (eds.) ECML PKDD 2010. LNCS (LNAI), vol. 6321, pp. 103–118. Springer, Heidelberg (2010). Scholar
  39. 39.
    Kivimäki, I., Shimbo, M., Saerens, M.: Developments in the theory of randomized shortest paths with a comparison of graph node distances. Phys. A: Stat. Mech. Appl. 393, 600–616 (2014)CrossRefGoogle Scholar
  40. 40.
    Bavaud, F.: Testing spatial autocorrelation in weighted networks: the modes permutation test. J. Geogr. Syst. 3, 233–247 (2013)CrossRefGoogle Scholar
  41. 41.
    Cliff, A.D., Ord, J.K.: Spatial Processes: Models & Applications. Taylor & Francis, Didcot (1981)zbMATHGoogle Scholar

Copyright information

© Springer Nature Switzerland AG 2019

Authors and Affiliations

  1. 1.Department of Geography and SustainabilityUniversity of LausanneLausanneSwitzerland

Personalised recommendations