A Spatially-Constrained Normalized Gamma Process for Data Clustering

  • Sotirios P. Chatzis
  • Dimitrios Korkinof
  • Yiannis Demiris
Part of the IFIP Advances in Information and Communication Technology book series (IFIPAICT, volume 381)


In this work, we propose a novel nonparametric Bayesian method for clustering of data with spatial interdependencies. Specifically, we devise a novel normalized Gamma process, regulated by a simplified (pointwise) Markov random field (Gibbsian) distribution with a countably infinite number of states. As a result of its construction, the proposed model allows for introducing spatial dependencies in the clustering mechanics of the normalized Gamma process, thus yielding a novel nonparametric Bayesian method for spatial data clustering. We derive an efficient truncated variational Bayesian algorithm for model inference. We examine the efficacy of our approach by considering an image segmentation application using a real-world dataset. We show that our approach outperforms related methods from the field of Bayesian nonparametrics, including the infinite hidden Markov random field model, and the Dirichlet process prior.


  1. 1.
    Walker, S., Damien, P., Laud, P., Smith, A.: Bayesian nonparametric inference for random distributions and related functions. J. Roy. Statist. Soc. B 61(3), 485–527 (1999)MathSciNetzbMATHCrossRefGoogle Scholar
  2. 2.
    Neal, R.: Markov chain sampling methods for Dirichlet process mixture models. J. Comput. Graph. Statist. 9, 249–265 (2000)MathSciNetGoogle Scholar
  3. 3.
    Muller, P., Quintana, F.: Nonparametric Bayesian data analysis. Statist. Sci. 19(1), 95–110 (2004)MathSciNetCrossRefGoogle Scholar
  4. 4.
    Antoniak, C.: Mixtures of Dirichlet processes with applications to Bayesian nonparametric problems. The Annals of Statistics 2(6), 1152–1174 (1974)MathSciNetzbMATHCrossRefGoogle Scholar
  5. 5.
    Blei, D., Jordan, M.: Variational methods for the Dirichlet process. In: 21st Int. Conf. Machine Learning, New York, NY, USA, pp. 12–19 (July 2004)Google Scholar
  6. 6.
    Orbanz, P., Buhmann, J.: Nonparametric Bayes image segmentation. International Journal of Computer Vision 77, 25–45 (2008)CrossRefGoogle Scholar
  7. 7.
    Zhang, J.: The mean field theory in EM procedures for Markov random fields. IEEE Transactions on Image Processing 2(1), 27–40 (1993)CrossRefGoogle Scholar
  8. 8.
    Celeux, G., Forbes, F., Peyrard, N.: EM procedures using mean field-like approximations for Markov model-based image segmentation. Pattern Recognition 36(1), 131–144 (2003)zbMATHCrossRefGoogle Scholar
  9. 9.
    Chatzis, S.P., Tsechpenakis, G.: The infinite hidden Markov random field model. In: Proc. 12th International IEEE Conference on Computer Vision (ICCV), Kyoto, Japan, pp. 654–661 (September 2009)Google Scholar
  10. 10.
    Chatzis, S.P., Tsechpenakis, G.: The infinite hidden Markov random field model. IEEE Transactions on Neural Networks 21(6), 1004–1014 (2010)CrossRefGoogle Scholar
  11. 11.
    Ferguson, T.: A Bayesian analysis of some nonparametric problems. The Annals of Statistics 1, 209–230 (1973)MathSciNetzbMATHCrossRefGoogle Scholar
  12. 12.
    Blackwell, D., MacQueen, J.: Ferguson distributions via Pólya urn schemes. The Annals of Statistics 1(2), 353–355 (1973)MathSciNetzbMATHCrossRefGoogle Scholar
  13. 13.
    Maroquin, J., Mitte, S., Poggio, T.: Probabilistic solution of ill-posed problems in computational vision. Journal of the American Statistical Assocation 82, 76–89 (1987)CrossRefGoogle Scholar
  14. 14.
    Geman, S., Geman, D.: Stochastic relaxation, Gibbs distributions and the Bayesian restoration of images. IEEE Transactions on Pattern Analysis and Machine Intelligence 6, 721–741 (1984)zbMATHCrossRefGoogle Scholar
  15. 15.
    Clifford, P.: Markov random fields in statistics. In: Grimmett, G., Welsh, D. (eds.) Disorder in Physical Systems. A volume in Honour of John M. Hammersley on the Occasion of His 70th Birthday. Oxford Science Publication, Clarendon Press, Oxford (1990)Google Scholar
  16. 16.
    Chatzis, S.P., Varvarigou, T.A.: A fuzzy clustering approach toward hidden Markov random field models for enhanced spatially constrained image segmentation. IEEE Transactions on Fuzzy Systems 16(5), 1351–1361 (2008)CrossRefGoogle Scholar
  17. 17.
    Qian, W., Titterington, D.: Estimation of parameters in hidden Markov models. Philosophical Transactions of the Royal Society of London A 337, 407–428 (1991)zbMATHCrossRefGoogle Scholar
  18. 18.
    Blei, D.M., Jordan, M.I.: Variational inference for Dirichlet process mixtures. Bayesian Analysis 1(1), 121–144 (2006)MathSciNetCrossRefGoogle Scholar
  19. 19.
    Jordan, M., Ghahramani, Z., Jaakkola, T., Saul, L.: An introduction to variational methods for graphical models. In: Jordan, M. (ed.) Learning in Graphical Models, pp. 105–162. Kluwer, Dordrecht (1998)CrossRefGoogle Scholar
  20. 20.
    Martin, D., Fowlkes, C., Tal, D., Malik, J.: A database of human segmented natural images and its application to evaluating segmentation algorithms and measuring ecological statistics. In: Proc. 8th Int’l Conf. Computer Vision, Vancouver, Canada, pp. 416–423 (July 2001)Google Scholar
  21. 21.
    Unnikrishnan, R., Pantofaru, C., Hebert, M.: A measure for objective evaluation of image segmentation algorithms. In: Proc. IEEE Conf. Computer Vision and Pattern Recognition, San Diego, CA, USA, pp. 34–41 (June 2005)Google Scholar
  22. 22.
    Mori, G.: Guiding model search using segmentation. In: Proc. 10th IEEE Int. Conf. on Computer Vision, ICCV (2005)Google Scholar
  23. 23.
    Varma, M., Zisserman, A.: Classifying Images of Materials: Achieving Viewpoint and Illumination Independence. In: Heyden, A., Sparr, G., Nielsen, M., Johansen, P. (eds.) ECCV 2002, Part III. LNCS, vol. 2352, pp. 255–271. Springer, Heidelberg (2002)CrossRefGoogle Scholar
  24. 24.
    Nikou, C., Galatsanos, N., Likas, A.: A class-adaptive spatially variant mixture model for image segmentation. IEEE Transactions on Image Processing 16(4), 1121–1130 (2007)MathSciNetCrossRefGoogle Scholar

Copyright information

© IFIP International Federation for Information Processing 2012

Authors and Affiliations

  • Sotirios P. Chatzis
    • 1
  • Dimitrios Korkinof
    • 2
  • Yiannis Demiris
    • 2
  1. 1.Cyprus University of TechnologyLimassolCyprus
  2. 2.Imperial College LondonUK

Personalised recommendations