Non-linear Bayesian Image Modelling

  • Christopher M. Bishop
  • John M. Winn
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 1842)


In recent years several techniques have been proposed for modelling the low-dimensional manifolds, or ‘subspaces’, of natural images. Examples include principal component analysis (as used for instance in ‘eigen-faces’), independent component analysis, and auto-encoder neural networks. Such methods suffer from a number of restrictions such as the limitation to linear manifolds or the absence of a probablistic representation. In this paper we exploit recent developments in the fields of variational inference and latent variable models to develop a novel and tractable probabilistic approach to modelling manifolds which can handle complex non-linearities. Our framework comprises a mixture of sub-space components in which both the number of components and the effective dimensionality of the subspaces are determined automatically as part of the Bayesian inference procedure. We illustrate our approach using two classical problems: modelling the manifold of face images and modelling the manifolds of hand-written digits.


Face Image Independent Component Analysis Latent Variable Model Principal Component Analysis Model Variational Inference 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


  1. 1.
    H. Attias. Learning parameters and structure of latent variables by variational Bayes, 1999. Submitted to UAI.Google Scholar
  2. 2.
    C. M. Bishop. Bayesian PCA. In S. A. Solla M. S. Kearns and D. A. Cohn, editors, Advances in Neural Information Processing Systems, volume 11, pages 382–388. MIT Press, 1999.Google Scholar
  3. 3.
    C. M. Bishop and M. E. Tipping. A hierarchical latent variable model for data visualization. IEEE Transactions on Pattern Analysis and Machine Intelligence, 20(3):281–293, 1998.CrossRefGoogle Scholar
  4. 4.
    M. J. Black and Y. Yacoob. Recognizing facial expressions under rigid and non-rigid facial motions. In International Workshop on Automatic Face and Gesture Recognition, Zurich, pages 12–17, 1995.Google Scholar
  5. 5.
    C. Bregler and S.M. Omohundro. Nonlinear manifold learning for visual speech recognition. In Fifth International Conference on Computer Vision, pages 494–499, Boston, Jun 1995.Google Scholar
  6. 6.
    T. F. Cootes, C. J. Taylor, D. H. Cooper, and J. Graham. Active shape models-their training and application. In Computer vision, graphics and image understanding, volume 61, pages 38–59, 1995.CrossRefGoogle Scholar
  7. 7.
    B. Frey and N. Jojic. Transformed component analysis: joint estimation of spatial transformations and image components. In Seventh International Conference on Computer Vision, pages 1190–1196, 1999.Google Scholar
  8. 8.
    Z. Ghahramani and M. J. Beal. Variational inference for Bayesian mixture of factor analysers. In Advances in Neural Information Processing Systems, volume 12, 1999.Google Scholar
  9. 9.
    T. Heap and D. Hogg. Wormholes in shape space: Tracking through discontinuous changes in shape. In Sixth International Conference on Computer Vision, pages 344–349, 1998.Google Scholar
  10. 10.
    M. I. Jordan, Z. Gharamani, T. S. Jaakkola, and L. K. Saul. An introduction to variational methods for graphical models. In M. I. Jordan, editor, Learning in Graphical Models, pages 105–162. Kluwer, 1998.Google Scholar
  11. 11.
    B. Moghaddam. Principal manifolds and Bayesian subspaces for visual recognition. In Seventh International Conference on Computer Vision, pages 1131–1136, 1999.Google Scholar
  12. 12.
    B. Moghaddam and A. Pentland. Probabilistic visual learning for object recognition. IEEE Transactions on Pattern Analysis and Machine Intelligence, 19(7):696–710, 1997.CrossRefGoogle Scholar
  13. 13.
    M. E. Tipping and C. M. Bishop. Mixtures of probabilistic principal component analyzers. Neural Computation, 11(2):443–482, 1999.CrossRefGoogle Scholar
  14. 14.
    M. E. Tipping and C. M. Bishop. Probabilistic principal component analysis. Journal of the Royal Statistical Society, Series B, 21(3):611–622, 1999.MathSciNetGoogle Scholar
  15. 15.
    M. Turk and A. Pentland. Eigenfaces for recognition. Journal of Cognitive Neuroscience, 3(1):71–86, 1991.CrossRefGoogle Scholar
  16. 16.
    N. Ueda, R. Nakano, Z. Ghahramani, and G. E. Hinton. SMEM algorithm for mixture models. In Advances in Neural Information Processing Systems, volume 11, 1999.Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2000

Authors and Affiliations

  • Christopher M. Bishop
    • 1
  • John M. Winn
    • 2
  1. 1.Microsoft ResearchCambridgeUK
  2. 2.Department of EngineeringUniversity of CambridgeCambridgeUK

Personalised recommendations