Skip to main content
SpringerLink
Log in

Pseudo-marginal Markov Chain Monte Carlo for Nonnegative Matrix Factorization

  • Published:
Neural Processing Letters Aims and scope Submit manuscript

Abstract

A pseudo-marginal Markov chain Monte Carlo (PMCMC) method is proposed for nonnegative matrix factorization (NMF). The sampler jointly simulates the joint posterior distribution for the nonnegative matrices and the matrix dimensions which indicate the number of the nonnegative components in the NMF model. We show that the PMCMC sampler is a generalization of a version of the reversible jump Markov chain Monte Carlo. An illustrative synthetic data was used to demonstrate the ability of the proposed PMCMC sampler in inferring the nonnegative matrices and as well as the matrix dimensions. The proposed sampler was also applied to a nuclear magnetic resonance spectroscopy data to infer the number of nonnegative components.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4

Similar content being viewed by others

References

  1. Andrieu C, Roberts GO (2009) The pseudo-marginal approach for efficient Monte Carlo computations. Ann Stat 37(2):697–725

    Article  MathSciNet  MATH  Google Scholar 

  2. Andrieu C, Doucet A, Holenstein R (2010) Particle Markov chain Monte Carlo methods (with discussion). J R Stat Soc B 72(3):269–342

    Article  MathSciNet  Google Scholar 

  3. Boutsidis C, Zouzias A, Mahoney MW, Drineas P (2015) Randomized dimensionality reduction for-means clustering. IEEE Trans Inf Theory 61(2):1045–1062

    Article  MathSciNet  Google Scholar 

  4. Calderhead B, Girolami M (2009) Estimating Bayes factors via thermodynamic integration and population MCMC. Comput Stat Data Anal 53:4028–4045

    Article  MathSciNet  MATH  Google Scholar 

  5. Chib S (1995) Marginal likelihood from the Gibbs output. J Am Stat Assoc 90:1313–1321

    Article  MathSciNet  MATH  Google Scholar 

  6. Ding C, Li T, Peng W, Park H (2006) Orthogonal nonnegative matrix t-factorizations for clustering. In: Proceedings of the 12th ACM SIGKDD international conference on Knowledge discovery and data mining, pp 126–135

  7. Dunn WB, Bailey N, Johnson HE (2005) Measuring the metabolome: current analytical technologies. Analyst 130:606–625

    Article  Google Scholar 

  8. Friel N, Pettitt AN (2008) Marginal likelihood estimation via power posteriors. J R Stat Soc B 70:589–607

    Article  MathSciNet  MATH  Google Scholar 

  9. Green P (1995) Reversible jump Markov chain Monte Carlo computation and Bayesian model determination. Biometrika 82:711–732

    Article  MathSciNet  MATH  Google Scholar 

  10. Liu JS (2001) Monte Carlo strategies in scientific computing. Springer, Berlin

    MATH  Google Scholar 

  11. Liu T, Tao D (2015) Classification with noisy labels by importance reweighting. IEEE Trans Pattern Anal Mach Intell 38:447–461

    Article  Google Scholar 

  12. Liu T, Tao D (2015) On the performance of manhattan nonnegative matrix factorization. IEEE Trans Neural Netw Learn Syst 99:1

    Article  Google Scholar 

  13. Manganas L, Zhang X, Li Y, Hazel RD, Smith SD, Wagshul ME, Henn F, Benveniste H, Djuric PM, Enikolopov G, Maletic-Savatic M (2007) Magnetic resonance spectroscopy identifies neural progenitor cells in the live human brain. Science 318:980–985

    Article  Google Scholar 

  14. Rubin D (1987) Comment on the calculation of posterior distributions by data augmentation by Tanner, M.A. and Wong, W. H. J Am Stat Assoc 82:543

    Google Scholar 

  15. Schachtner R, Po G, Tomé AM, Puntonet CG, Lang EW et al (2014) A new bayesian approach to nonnegative matrix factorization: uniqueness and model order selection. Neurocomputing 138:142–156

    Article  Google Scholar 

  16. Schmidt MN, Winther O, Hansen LK (2009) Bayesian non-negative matrix factorization. In: International conference on independent component analysis and signal separation. Lecture Notes in Computer Science (LNCS), vol 5441. Springer, Berlin, pp 540–547

  17. Sun M, Zhang X et al (2015) A stable approach for model order selection in nonnegative matrix factorization. Pattern Recognit Lett 54:97–102

    Article  Google Scholar 

  18. Tan V, Fevotte C (2012) Automatic relevance determination in nonnegative matrix factorization with the beta-divergence. IEEE Trans Pattern Anal Mach Intell 35:1592–1605

    Article  Google Scholar 

  19. Xu C, Tao D, Xu C, Rui Y (2014) Large-margin weakly supervised dimensionality reduction. In: Proceedings of the 31st international conference on machine learning (ICML-14), pp 865–873

  20. Zhong M, Girolami M (2009) Reversible jump MCMC for non-negative matrix factorization. In: Proceedings of the 12th international conference on artificial intelligence and statistics (AISTATS), FL, pp 663–670

  21. Zhong M, Girolami M, Faulds K, Graham D (2011) Bayesian methods to detect dye-labelled DNA oligonucleotides in multiplexed Raman spectra. J R Stat Soc C (Appl Stat) 60(2):187–206

    Article  MathSciNet  Google Scholar 

Download references

Acknowledgments

This work is supported by the fundamental research funds for the central universities in China.

Author information

Authors and Affiliations

  1. School of Science, Dalian Ocean University, Dalian, 116023, People’s Republic of China

    Junfu Du

  2. Department of Biomedical Engineering, Dalian University of Technology, Dalian, 116024, People’s Republic of China

    Mingjun Zhong

Authors
  1. Junfu Du
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Mingjun Zhong
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Mingjun Zhong.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Du, J., Zhong, M. Pseudo-marginal Markov Chain Monte Carlo for Nonnegative Matrix Factorization. Neural Process Lett 45, 553–562 (2017). https://doi.org/10.1007/s11063-016-9542-x

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11063-016-9542-x

Keywords

Search

Navigation