Nonparametric Orthogonal NMF and its Application in Cancer Clustering

  • Andri Mirzal
Conference paper
Part of the Lecture Notes in Electrical Engineering book series (LNEE, volume 285)


Orthogonal nonnegative matrix factorization (NMF) is an NMF objective function that enforces orthogonality constraint on its factor. There are two challenges in optimizing this objective function: the first is how to design an algorithm that has convergence guarantee, and the second is how to automatically choose the regularization parameter. In our previous work, we have been able to develop a convergent algorithm for this objective function. However, the second challenge remains unsolved. In this paper, we provide an attempt to answer the second challenge. The proposed method is based on the L-curve approach and has a simple form which is preferable since it introduces only a small additional computational cost. This method transforms the algorithm into nonparametric, and is also extendable to other NMF objective functions as long as the functions are differentiable with respect to the corresponding regularization parameters. Numerical results are then provided to evaluate the feasibility of the method in choosing the appropriate regularization parameter values by utilizing it in cancer clustering tasks.


Cancer clustering Gene expression Nonnegative matrix factorization Nonparametric learning Orthogonality constraint 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.



The author would like to thank the reviewers for useful comments. This research was supported by Ministry of Higher Education of Malaysia and Universiti Teknologi Malaysia under Exploratory Research Grant Scheme R.J130000.7828.4L095.


  1. 1.
    Paatero, P., Tapper, U.: Positive matrix factorization: A non-negative factor model with optimal utilization of error estimates of data values. Environmetrics 5, 111–126(1994)Google Scholar
  2. 2.
    Anttila, P., et al.: Source identification of bulk wet deposition in Finland by positive matrix factorization. Atmospheric Environment 29(14), 1705–1718 (1995)Google Scholar
  3. 3.
    Lee, D., Seung, H.: Learning the parts of objects by non-negative matrix factorization. Nature 401(6755), 788–791 (1999)Google Scholar
  4. 4.
    Lee, D., Seung, H.: Algorithms for non-negative matrix factorization. In Proc. Advances in Neural Processing Information Systems, 556–562 (2000)Google Scholar
  5. 5.
    Xu, W., et al.: Document clustering based on non-negative matrix factorization. In Proc. ACM SIGIR, 267–273 (2003)Google Scholar
  6. 6.
    Shahnaz, F., et al.: Document clustering using nonnegative matrix factorization. Information Processing & Management 42(2), 373–386 (2006)Google Scholar
  7. 7.
    Pauca, V.P., et al.: Nonnegative matrix factorization for spectral data analysis. Linear Algebra and Its Applications 416(1), 29–47 (2006)Google Scholar
  8. 8.
    Jia, S., Qian, Y.: Constrained nonnegative matrix factorization for hyperspectral unmixing. IEEE Transactions on Geoscience and Remote Sensing 47(1), 161–173 (2009)Google Scholar
  9. 9.
    Li, S.Z., et al.: Learning spatially localized, parts-based representation. In Proc. IEEE Comp. Soc. Conf. on Computer Vision and Pattern Recognition, 207–212 (2001)Google Scholar
  10. 10.
    Hoyer, P.O.: Non-negative matrix factorization with sparseness constraints. The Journal of Machine Learning Research 5, 1457{1469 (2004)Google Scholar
  11. 11.
    Wang, D., Lu, H.: On-line learning parts-based representation via incremental orthogonal projective non-negative matrix factorization. Signal Processing 93(6), 1608–1623 (2013)Google Scholar
  12. 12.
    Pascual-Montano, A., et al.: Nonsmooth nonnegative matrix factorization. IEEE Transactions on Pattern Analysis and Machine Intelligence 28(3), 403{415 (2006)Google Scholar
  13. 13.
    Gillis, N., Glineur, F.: A multilevel approach for nonnegative matrix factorization. J. Computational and Applied Mathematics 236(7), 1708–1723 (2012)Google Scholar
  14. 14.
    Cichocki, A., et al.: Extended SMART algorithms for non-negative matrix factorization. LNCS 4029, 548–562, Springer (2006)Google Scholar
  15. 15.
    Zhou, G., et al.: Online blind source separation using incremental nonnegative matrix factorization with volume constraint. IEEE Transactions on Neural Networks 22(4), 550–560 (2011)Google Scholar
  16. 16.
    Bertin, N., et al.: Enforcing harmonicity and smoothness in bayesian non-negative matrix factorization applied to polyphonic music transcription. IEEE Transactions on Audio, Speech, and Language Processing 18(3), 538–549 (2010)Google Scholar
  17. 17.
    Bertrand, A., Moonen, M.: Blind separation of non-negative source signals using multiplicative updates and subspace projection. Signal Processing 90(10), 2877–2890 (2010)Google Scholar
  18. 18.
    Virtanen, T., et al.: Bayesian extensions to non-negative matrix factorisation for audio signal modelling. In IEEE Int’l Conf. on Acoustics, Speech and Signal Processing, 1825–1828 (2008)Google Scholar
  19. 19.
    Brunet, J.P., et al.: Metagenes and molecular pattern discovery using matrix factorization. Proc. Natl Acad. Sci. USA 101(12), 4164–4169 (2003)Google Scholar
  20. 20.
    Gao, Y., Church, G.: Improving molecular cancer class discovery through sparse non-negative matrix factorization. Bioinformatics 21(21), 3970–3975 (2005)Google Scholar
  21. 21.
    Kim, H., Park, H.: Sparse non-negative matrix factorizations via alternating non-negativity constrained least squares for microarray data analysis. Bioinformatics 23(12), 1495–1502 (2007)Google Scholar
  22. 22.
    Devarajan, K.: Nonnegative matrix factorization: an analytical and interpretive tool in computational biology. PLoS Computational Biology 4(7), e1000029 (2008)Google Scholar
  23. 23.
    Kim, H., Park, H.: Nonnegative matrix factorization based on alternating non-negativity constrained least squares and active set method. SIAM J. Matrix Anal. Appl. 30(2), 713–730 (2008)Google Scholar
  24. 24.
    Carmona-Saez, et al.: Biclustering of gene expression data by non-smooth non-negative matrix factorization. BMC Bioinformatics 7(78) (2006)Google Scholar
  25. 25.
    Inamura, K., et al.: Two subclasses of lung squamous cell carcinoma with different gene expression profiles and prognosis identified by hierarchical clustering and non-negative matrix factorization. Oncogene (24), 7105–7113 (2005)Google Scholar
  26. 26.
    Fogel, P., et al.: Inferential, robust non-negative matrix factorization analysis of microarray data. Bioinformatics 23(1), 44–49 (2007)Google Scholar
  27. 27.
    Zheng, C.H., et al.: Tumor clustering using nonnegative matrix factorization with gene selection. IEEE Transactions on Information Technology in Biomedicine 13(4), 599–607 (2009)Google Scholar
  28. 28.
    Wang, G., et al.: LS-NMF: A modified non-negative matrix factorization algorithm utilizing uncertainty estimates. BMC Bioinformatics 7(175) (2006)Google Scholar
  29. 29.
    Wang, J.J.Y., et al.: Non-negative matrix factorization by maximizing correntropy for cancer clustering. BMC Bioinformatics 14(107) (2013)Google Scholar
  30. 30.
    Yuvaraj, N., Vivekanandan, P.: An efficient SVM based tumor classification with symmetry non-negative matrix factorization using gene expression data. In Int’l Conf. on Information Communication and Embedded Systems, 761–768 (2013)Google Scholar
  31. 31.
    Ding, C., et al.: Orthogonal nonnegative matrix t-factorizations for clustering. In 12th ACM SIGKDD Int’l Conf. on Knowledge Discovery and Data Mining, 126–135 (2006)Google Scholar
  32. 32.
    Mirzal, A.: A convergent algorithm for orthogonal nonnegative matrix factorization. To appear in J. Computational and Applied Mathematics.Google Scholar
  33. 33.
    Hansen, P.C.: Analysis of discrete ill-posed problems by means of the L-curve. SIAM Review 34(4), 561–580 (1992)Google Scholar
  34. 34.
    Lin, C.J.: On the convergence of multiplicative update algorithms for non-negative matrix factorization. IEEE Transactions on Neural Networks 18(6), 1589–1596 (2007)Google Scholar
  35. 35.
    Lin, C.J.: Projected gradient methods for non-negative matrix factorization. Technical Report ISSTECH-95-013, Department of CS, National Taiwan University (2005)Google Scholar
  36. 36.
    Rand, W.M.: Objective criteria for the evaluation of clustering methods. Journal of the American Statistical Association 66(336), 846–850 (1971)Google Scholar
  37. 37.
    Hubert, L., Arabie, P.: Comparing partitions. Journal of Classification 2(1), 193–218 (1985)Google Scholar
  38. 38.
    Vinh, N.X., et al.: Information theoretic measures for clustering comparison: Is a correction for chance necessary? In 26th Annual Int’l Conf. on Machine Learning, pp. 1073–1080 (2009)Google Scholar
  39. 39.
    Souto, M.C.P., et al.: Clustering cancer gene expression data: a comparative study. BMC Bioinformatics 9(497) (2008)Google Scholar

Copyright information

© Springer Science+Business Media Singapore 2014

Authors and Affiliations

  1. 1.Faculty of Computing, N28-439-03Universiti Teknologi Malaysia UTMJohor BahruMalaysia

Personalised recommendations