Skip to main content
SpringerLink
Log in

NMF versus ICA for blind source separation

  • Regular Article
  • Published:
Advances in Data Analysis and Classification Aims and scope Submit manuscript

Abstract

Blind source separation (BSS) is a problem of recovering source signals from signal mixtures without or very limited information about the sources and the mixing process. From literatures, nonnegative matrix factorization (NMF) and independent component analysis (ICA) seem to be the mainstream techniques for solving the BSS problems. Even though the using of NMF and ICA for BSS is well studied, there is still a lack of works that compare the performances of these techniques. Moreover, the nonuniqueness property of NMF is rarely mentioned even though this property actually can make the reconstructed signals vary significantly, and thus introduces the difficulty on how to choose the representative reconstructions from several possible outcomes. In this paper, we compare the performances of NMF and ICA as BSS methods using some standard NMF and ICA algorithms, and point out the difficulty in choosing the representative reconstructions originated from the nonuniqueness property of NMF.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12

Similar content being viewed by others

Notes

  1. There is actually a study that considers the effect of the nonuniqueness property of NMF to the reconstruction results as it displays the average signal-to-noise values over some trials (Plaza et al. 2012). However the authors do not mention the difficulty in choosing the representative reconstructions.

  2. The auxiliary function for proving the nonincreasing property in any MUR based NMF algorithm is also the Lyapunov function, so that the stability of any MUR based NMF algorithm can be shown directly by using the result presented in Badeau (2010).

  3. MVCNMF has a justification as a BSS method as it looks for estimate source vectors that span a simplex that circumscribes the observed data.

References

  • Anttila P et al (1995) Source identification of bulk wet deposition in Finland by positive matrix factorization. Atmos Environ 29(14):1705–1718

    Article  Google Scholar 

  • Arngren M, Schmidt MN, Larsen J (2011) Unmixing of hyperspectral images using Bayesian non-negative matrix factorization. J Signal Process Syst 65:479–496

    Article  Google Scholar 

  • Badeau R (2010) Stability analysis of multiplicative update algorithms and application to nonnegative matrix factorization. IEEE Trans Neural Netw 21(12):1869–1881

    Article  Google Scholar 

  • Berry M, Brown M, Langville A, Pauca P, Plemmons RJ (2007) Algorithms and applications for approximate nonnegative matrix factorization. Comput Stat Data Anal 52(1):155–173

    Article  MathSciNet  MATH  Google Scholar 

  • Bertin N et al (2009) A tempering approach for Itakura–Saito non-negative matrix factorization. With application to music transcription. In: Proceedings of IEEE international conference on acoustics, speech and signal processing, pp 1545–1548

  • Bertin N et al (2010) Enforcing harmonicity and smoothness in Bayesian non-negative matrix factorization applied to polyphonic music transcription. IEEE Trans Audio Speech Lang Process 18(3):538–549

    Article  Google Scholar 

  • Bertrand A, Moonen M (2010) Blind separation of non-negative source signals using multiplicative updates and subspace projection. Signal Process 90(10):2877–2890

    Article  MATH  Google Scholar 

  • Brunet JP et al (2003) Metagenes and molecular pattern discovery using matrix factorization. Proc Natl Acad Sci USA 101(12):4164–4169

    Article  Google Scholar 

  • Carmona-Saez P et al (2006) Biclustering of gene expression data by non-smooth non-negative matrix factorization. BMC Bioinform 7(78)

  • Cauquy MA, Roggemann M, Schultz T (2004) Approaches for processing spectral measurements of reflected sunlight for space situational awareness. In: Proceedings of SPIE conference on defense and security, vol 5428, pp 48–57

  • Chen Z, Nowrouzian B, Zarowski CJ (2005) An investigation of SNR estimation techniques based on uniform Cramer-Rao lower bound. In: 48th midwest symposium on circuits and systems, pp 215–218

  • Choi S (2008) Algorithms for orthogonal nonnegative matrix factorization. In: Proceedings of IEEE int’l joint conf. on neural networks, pp 1828–1832

  • Cichocki A, Amari S, Zdunek R, Kompass R, Hori G, He Z (2006) Extended SMART algorithms for non-negative matrix factorization. Lect Notes Comput Sci 4029:548–562

    Article  Google Scholar 

  • Craig MD (1994) Minimum-volume transforms for remotely sensed data. IEEE Trans Geosci Remote Sens 32(3):542–552

    Article  Google Scholar 

  • Devarajan K (2008) Nonnegative matrix factorization: an analytical and interpretive tool in computational biology. PLoS Comput Biol 4(7):e1000029

    Article  Google Scholar 

  • Ding C, Li T, Peng W, Park H (2006) Orthogonal nonnegative matrix t-factorizations for clustering. In: Proceedings of 12th ACM SIGKDD int’l conf. on knowledge discovery and data mining, pp 126–135

  • Févotte C et al (2009) Nonnegative matrix factorization with the Itakura–Saito divergence. With application to music analysis. Neural Comput 21(3):793–830

    Article  MATH  Google Scholar 

  • Févotte C, Idier J (2011) Algorithms for nonnegative matrix factorization with the \(\beta \)-divergence. Neural Comput 23(9):2421–2456

    Article  MathSciNet  MATH  Google Scholar 

  • FitzGerald D et al (2009) On the use of the beta divergence for musical source separation. In: Proceedings of the Irish signals and systems conference

  • Fogel P et al (2007) Inferential, robust non-negative matrix factorization analysis of microarray data. Bioinformatics 23(1):44–49

    Article  Google Scholar 

  • Gao Y, Church G (2005) Improving molecular cancer class discovery through sparse non-negative matrix factorization. Bioinformatics 21(21):3970–3975

    Article  Google Scholar 

  • Gillis N, Glineur F (2012) A multilevel approach for nonnegative matrix factorization. J Comput Appl Math 236(7):1708–1723

    Article  MathSciNet  MATH  Google Scholar 

  • Grady PD (2007) Sparse separation of under-determined speech mixtures. Ph.D. thesis, National University of Ireland, Maynooth

  • Grady PD, Pearlmutter BA (2008) Discovering speech phones using convolutive non-negative matrix factorisation with a sparseness constraint. Neurocomputing 72(1–3):88–101

    Google Scholar 

  • Grippo L, Sciandrone M (2000) On the convergence of the block nonlinear Gauss–Seidel method under convex constraints. Oper Res Lett 26(3):127–136

    Article  MathSciNet  MATH  Google Scholar 

  • Hennequin R et al (2010) NMF with time-frequency activations to model non stationary audio events. In: Proceedings of IEEE international conference on acoustics speech and signal processing, pp 445–448

  • Hindi H (2004) A tutorial on convex optimization. In: Proceedings of American control conference, pp 3252–3265

  • Hoyer PO (2004) Non-negative matrix factorization with sparseness constraints. J Mach Learn Res 5:1457–1469

    MathSciNet  MATH  Google Scholar 

  • Hyvrinen A (1999) Fast and robust fixed-point algorithms for independent component analysis. IEEE Trans Neural Netw 10(3):626–634

    Article  Google Scholar 

  • Hyvrinen A, Oja E (2000) Independent component analysis: algorithms and applications. Neural Netw 13:411–430

    Article  Google Scholar 

  • Inamura K et al (2005) Two subclasses of lung squamous cell carcinoma with different gene expression profiles and prognosis identified by hierarchical clustering and non-negative matrix factorization. Oncogene 24:7105–7113

    Article  Google Scholar 

  • Jia S, Qian Y (2009) Constrained nonnegative matrix factorization for hyperspectral unmixing. IEEE Trans Geosci Remote Sens 47(1):161–173

    Article  MATH  Google Scholar 

  • Keshava N, Mustard J (2002) Spectral unmixing. IEEE Signal Process Mag 8:44–57

    Article  Google Scholar 

  • Kim H, Park H (2007) Sparse non-negative matrix factorizations via alternating non-negativity constrained least squares for microarray data analysis. Bioinformatics 23(12):1495–1502

    Article  Google Scholar 

  • Kim H, Park H (2008) Nonnegative matrix factorization based on alternating nonnegativity constrained least squares and active set method. SIAM J Matrix Anal Appl 30(2):713–730

    Article  MathSciNet  MATH  Google Scholar 

  • Kim J, Park H (2008) Sparse nonnegative matrix factorization for clustering. CSE Technical Reports; GT-CSE-08-01, Georgia Institute of Technology

  • Kim J, Park H (2008) Toward faster nonnegative matrix factorization: a new algorithm and comparisons. In: Proceedings of the eighth IEEE international conference on data mining, pp 353–362

  • Lee D, Seung H (1999) Learning the parts of objects by non-negative matrix factorization. Nature 401(6755):788–791

    Article  Google Scholar 

  • Lee D, Seung H (2000) Algorithms for non-negative matrix factorization. In: Proceedings of advances in neural processing information systems, pp 556–562

  • Li SZ et al (2001) Learning spatially localized, parts-based representation. In: Proceedings of IEEE comp. soc. conf. on computer vision and pattern recognition, pp 207–212

  • Li H, Adali T, Wang W, Emge D (2005) Non-negative matrix factorization with orthogonality constraints for chemical agent detection in Raman spectra. In: Proceedings of IEEE workshop on machine learning for signal processing, pp 253–258

  • Lin CJ (2005) Projected gradient methods for non-negative matrix factorization. Technical Report ISSTECH-95-013. Department of CS, National Taiwan University

  • Luu L et al (2003) Object characterization from spectral data. In: Proceedings of AMOS technical conference

  • Masalmah YM (2007) Unsupervised unmixing of hyperspectral imagery using the constrained positive matrix factorization. Ph.D. thesis, Computing and Information Science and Engineering, University of Puerto Rico

  • Miao L, Qi H (2007) Endmember extraction from highly mixed data using minimum volume constrained nonnegative matrix factorization. IEEE Trans Geosci Remote Sens 45(3):765–777

    Article  Google Scholar 

  • Mirzal A (2014) A convergent algorithm for orthogonal nonnegative matrix factorization. J Comput Appl Math 260:149–166

    Article  MathSciNet  MATH  Google Scholar 

  • Nascimento JMP, Dias JMB (2005) Does independent component analysis play a role in unmixing hyperspectral data? IEEE Trans Geosci Remote Sens 43(1):175–187

    Article  Google Scholar 

  • Oja E, Plumbley MD (2004) Blind separation of positive sources by globally convergent gradient search. Neural Comput 16(9):1811–1825

    Article  MATH  Google Scholar 

  • Oja E, Plumbley MD (2003) Blind separation of positive sources using non-negative PCA. In: Proceedings of the 4th international symposium on independent component analysis and blind signal separation, warning for areas of moderate seismicity, pp 11–16

  • Paatero P, Tapper U (1994) Positive matrix factorization: a non-negative factor model with optimal utilization of error estimates of data values. Environmetrics 5:111–126

    Article  Google Scholar 

  • Pascual-Montano A et al (2006) Nonsmooth nonnegative matrix factorization. IEEE Trans Pattern Anal Mach Intell 28(3):403–415

    Article  Google Scholar 

  • Pauca VP, Piper J, Plemmons RJ (2006) Nonnegative matrix factorization for spectral data analysis. Linear Algebra Appl 416(1):29–47

    Article  MathSciNet  MATH  Google Scholar 

  • Piper J et al (2004) Object characterization from spectral data using nonnegative matrix factorization. In: Proceedings of AMOS technical conference

  • Plaza J, Hendrix EMT, Garcia I, Martin G, Plaza A (2012) On endmember identification in hyperspectral images without pure pixels: a comparison of algorithms. J Math Imaging Vis 42:163–175

    Article  MathSciNet  MATH  Google Scholar 

  • Plumbey MD (2002) Conditions for nonnegative independent component analysis. IEEE Signal Process Lett 9(6):177–180

    Article  Google Scholar 

  • Plumbey MD (2003) Algorithms for nonnegative independent component analysis. IEEE Trans Neural Netw 14(3):534–543

    Article  Google Scholar 

  • Plumbey MD, Oja E (2004) A nonnegative PCA algorithm for independent component analysis. IEEE Trans Neural Netw 15(1):66–76

    Article  Google Scholar 

  • Ren G (2009) SNR estimation algorithm based on the preamble for OFDM systems in frequency selective channels. IEEE Trans Commun 57(8):2230–2234

    Article  Google Scholar 

  • Shahnaz F et al (2006) Document clustering using nonnegative matrix factorization. Inf Process Manag 42(2):373–386

    Article  MATH  Google Scholar 

  • Sinha P (2002a) Identifying perceptually significant features for recognizing faces. In: Proceedings of the SPIE electronic imaging symposium

  • Sinha P (2002b) Recognizing complex patterns. Nat Neurosci 5(suppl.):1093–1097

  • Sinha P et al (2006) Face recognition by humans: nineteen results all computer vision researchers should know about. Proc IEEE 94(11):1948–1962

  • Stögbauer H, Kraskov A, Astakhov SA, Grassberger P (2004) Least dependent component analysis based on mutual information. Phys Rev E 70(6):066123

    Article  Google Scholar 

  • Vincent E et al (2010) Adaptive harmonic spectral decomposition for multiple pitch estimation. IEEE Trans Audio Speech Lang Process 18:528–537

    Article  Google Scholar 

  • Virtanen T et al (2008) Bayesian extensions to non-negative matrix factorisation for audio signal modelling. In: Proceedings of IEEE international conference on acoustics, speech and signal processing, pp 1825–1828

  • Wang G et al (2006) LS-NMF: a modified non-negative matrix factorization algorithm utilizing uncertainty estimates. BMC Bioinform 7(175)

  • Wang JJY et al (2013) Non-negative matrix factorization by maximizing correntropy for cancer clustering. BMC Bioinform 14(107)

  • Wang D, Lu H (2013) On-line learning parts-based representation via incremental orthogonal projective non-negative matrix factorization. Signal Process 93(6):1608–1623

    Article  Google Scholar 

  • Xu W et al (2003) Document clustering based on non-negative matrix factorization. In: Proceedings of ACM SIGIR, pp 267–273

  • Xu X et al (2006) Subspace-based noise variance and SNR estimation for MIMO OFDM systems. J Electron (China) 23(2):176–180

    Article  Google Scholar 

  • Yoo J, Choi S (2010) Orthogonal nonnegative matrix tri-factorization for co-clustering: multiplicative updates on Stiefel manifolds. Inf Process Manag 46(5):559–570

    Article  Google Scholar 

  • Yoo J, Choi S (2008) Orthogonal nonnegative matrix factorization: multiplicative updates on Stiefel manifolds. In: Proceedings of the 9th int’l conf. intelligent data engineering and automated learning, pp 140–147

  • Yuvaraj N, Vivekanandan P (2013) An efficient SVM based tumor classification with symmetry non-negative matrix factorization using gene expression data. In: Int’l conf. on information communication and embedded systems, pp 761–768

  • Zarowski CJ (2002) Limitations on SNR estimator accuracy. IEEE Trans Signal Process 50(9):2368–2372

    Article  Google Scholar 

  • Zheng CH et al (2009) Tumor clustering using nonnegative matrix factorization with gene selection. IEEE Trans Inf Technol Biomed 13(4):599–607

    Article  Google Scholar 

  • Zhou G et al (2011) Online blind source separation using incremental nonnegative matrix factorization with volume constraint. IEEE Trans Neural Netw 22(4):550–560

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Computer Science Department, College of Arts and Applied Sciences, Dhofar University, Salalah, Oman

    Andri Mirzal

Authors
  1. Andri Mirzal
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Andri Mirzal.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Mirzal, A. NMF versus ICA for blind source separation. Adv Data Anal Classif 11, 25–48 (2017). https://doi.org/10.1007/s11634-014-0192-4

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11634-014-0192-4

Keywords

Mathematics Subject Classification

Search

Navigation