Bayesian Non-negative Matrix Factorization

  • Mikkel N. Schmidt
  • Ole Winther
  • Lars Kai Hansen
Part of the Lecture Notes in Computer Science book series (LNCS, volume 5441)


We present a Bayesian treatment of non-negative matrix factorization (NMF), based on a normal likelihood and exponential priors, and derive an efficient Gibbs sampler to approximate the posterior density of the NMF factors. On a chemical brain imaging data set, we show that this improves interpretability by providing uncertainty estimates. We discuss how the Gibbs sampler can be used for model order selection by estimating the marginal likelihood, and compare with the Bayesian information criterion. For computing the maximum a posteriori estimate we present an iterated conditional modes algorithm that rivals existing state-of-the-art NMF algorithms on an image feature extraction problem.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Paatero, P., Tapper, U.: Positive matrix factorization: A nonnegative factor model with optimal utilization of error-estimates of data values. Environmetrics 5(2), 111–126 (1994)CrossRefGoogle Scholar
  2. 2.
    Lee, D.D., Seung, H.S.: Learning the parts of objects by non-negative matrix factorization. Nature 401(6755), 788–791 (1999)CrossRefGoogle Scholar
  3. 3.
    Sajda, P., Du, S., Parra, L.: Recovery of constituent spectra using non-negative matrix factorization. In: Wavelets: Applications in Signal and Image Processing, Proceedings of SPIE, vol. 5207, pp. 321–331 (2003)Google Scholar
  4. 4.
    Gaussier, E., Goutte, C.: Relation between PLSA and NMF and implication. In: Proceedings of the International SIGIR Conference on Research and Development in Information Retrieval, pp. 601–602 (2005)Google Scholar
  5. 5.
    Schmidt, M.N., Olsson, R.K.: Single-channel speech separation using sparse non-negative matrix factorization. In: ISCA International Conference on Spoken Language Processing, (INTERSPEECH) (2006)Google Scholar
  6. 6.
    Ochs, M.F., Stoyanova, R.S., Arias-Mendoza, F., Brown, T.R.: A new method for spectral decomposition using a bilinear bayesian approach. Journal of Magnetic Resonance 137, 161–176 (1999)CrossRefGoogle Scholar
  7. 7.
    Moussaoui, S., Brie, D., Mohammad-Djafari, A., Carteret, C.: Separation of non-negative mixture of non-negative sources using a Bayesian approach and MCMC sampling. IEEE Transactions on Signal Processing 54(11), 4133–4145 (2006)CrossRefGoogle Scholar
  8. 8.
    Chib, S.: Marginal likelihood from the Gibbs output. Journal of the American Statistical Association 90(432), 1313–1321 (1995)MathSciNetCrossRefzbMATHGoogle Scholar
  9. 9.
    Besag, J.: On the statistical analysis of dirty pictures. Journal of the Royal Statistical Society 48(3), 259–302 (1986)MathSciNetzbMATHGoogle Scholar
  10. 10.
    Sajda, P., Du, S., Brown, T.R., Stoyanova, R., Shungu, D.C., Mao, X., Parra, L.C.: Nonnegative matrix factorization for rapid recovery of constituent spectra in magnetic resonance chemical shift imaging of the brain. IEEE Transactions on Medical Imaging 23(12), 1453–1465 (2004)CrossRefGoogle Scholar
  11. 11.
    Schmidt, M.N., Laurberg, H.: Nonnegative matrix factorization with gaussian process priors. Computational Intelligence and Neuroscience (February 2008)Google Scholar
  12. 12.
    Graham, D.B., Allinson, N.M.: Characterizing virtual eigensignatures for general purpose face recognition. In: Face Recognition: From Theory to Applications. Computer and Systems Sciences, vol. 163, pp. 446–456 (1998)Google Scholar
  13. 13.
    Lee, D.D., Seung, H.S.: Algorithms for non-negative matrix factorization. In: Advances in Neural Information Processing Systems (NIPS), pp. 556–562 (2000)Google Scholar
  14. 14.
    Kim, D., Sra, S., Dhillon, I.S.: Fast Newton-type methods for the least squares nonnegative matrix approximation problem. In: Proceedings of SIAM Conference on Data Mining (2007)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2009

Authors and Affiliations

  • Mikkel N. Schmidt
    • 1
  • Ole Winther
    • 2
  • Lars Kai Hansen
    • 2
  1. 1.Department of EngineeringUniversity of CambridgeUK
  2. 2.DTU InformaticsTechnical University of DenmarkDenmark

Personalised recommendations