Chinese Science Bulletin

, Volume 51, Issue 1, pp 7–18 | Cite as

Nonnegative matrix factorization and its applications in pattern recognition

  • Liu Weixiang 
  • Zheng Nanning 
  • You Qubo 


Matrix factorization is an effective tool for large-scale data processing and analysis. Nonnegative matrix factorization (NMF) method, which decomposes the nonnegative matrix into two nonnegative factor matrices, provides a new way for matrix factorization. NMF is significant in intelligent information processing and pattern recognition. This paper firstly introduces the basic idea of NMF and some new relevant methods. Then we discuss the loss functions and relevant algorithms of NMF in the framework of probabilistic models based on our researches, and the relationship between NMF and information processing of perceptual process. Finally, we make use of NMF to deal with some practical questions of pattern recognition and point out some open problems for NMF.


nonnegative data feature extraction NMF intrusion detection digital watermarking EEG signal analysis 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Hubert, L. J., Meulman, J. J., Heiser, W. J., Two purposes for matrix factorization: A historical appraisal, SIAM Review, 2000, 42: 68–82.CrossRefGoogle Scholar
  2. 2.
    Duda, R. O., Hart, P. E., Stork, D. G., Pattern Classification, 2nd ed., New York: John Wiley & Sons, 2001.Google Scholar
  3. 3.
    Lee, D. D., Seung, H. S., Learning the parts of objects with nonnegative matrix factorization, Nature, 1999, 401: 788–791.Google Scholar
  4. 4.
    Paatero, P., Tapper, U., Positive matrix factorization: A non-negative factor model with optimal utilization of error estimates of data values, Environmetrics, 1994, 5: 111–126.Google Scholar
  5. 5.
    Lee, D. D., Seung, H. S., Algorithms for nonnegative matrix factorization, Advances in Neural Information Processing Systems 13 (eds. Leen, T., Dietterich, T., Tresp, V.), Cambridge: MIT Press, 2000.Google Scholar
  6. 6.
    Sajda, P., Du, S., Parra, L. et al., Recovery of constituent spectra using non-negative matrix factorization, Proc. SPIE, 2003, 5207: 321–331.Google Scholar
  7. 7.
    Cichocki, A., Amari, S., Adaptive Blind Signal and Image Processing: Learning Algorithms and Applications, New York: John Wiley & Sons, 2002.Google Scholar
  8. 8.
    Liu, W. X., Zheng, N. N., Learning sparse features for classification by mixture models, Pattern Recognition Letters, 2004, 25(2): 155–161.Google Scholar
  9. 9.
    Li, S. Z., Hou, X. W., Zhang, H. J. et al., Learning spatially localized, parts-based representation, Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2001, 1: 207–212.Google Scholar
  10. 10.
    Olshausen, B. A., Field, D. J., Emergence of simple-cell receptive field properties by learning a sparse code for natural images, Nature, 1996, 381: 607–609.CrossRefGoogle Scholar
  11. 11.
    Hoyer, P. O., Non-negative sparse coding, in Proc. Neural Net-works for Signal Processing, 2002, 557-565.Google Scholar
  12. 12.
    Hoyer, P. O., Modeling receptive fields with non-negative sparse coding, Neurocomputing, 2003, (52-54): 547-552.Google Scholar
  13. 13.
    Liu, W. X., Zheng, N. N., Lu, X. F., Non-negative matrix factorization for visual coding, Proceedings of the IEEE International Conference on Acoustics, Speech, and Signal Processing, 2003, 3: 293–296.Google Scholar
  14. 14.
    Hoyer, P. O., Non-negative matrix factorization with sparseness constraints, Journal of Machine Learning Research, 2004, 5: 1457–1469.Google Scholar
  15. 15.
    Guillamet, D., Vitriã, J., Schiele, B., Introducing a weighted non-negative matrix factorization for image classification, Pattern Recognition Letters, 2003, 24(14): 2447–2454.CrossRefGoogle Scholar
  16. 16.
    Paatero, P., Least squares formulation of robust non-negative factor analysis, Chemometrics and Intelligent Laboratory Systems, 1997, 37(1): 23–35.CrossRefGoogle Scholar
  17. 17.
    Welling, M., Weber, M., Positive tensor factorization, Pattern Recognition Letters, 2001, 22(12): 1255–1261.CrossRefGoogle Scholar
  18. 18.
    Wang, Y., Jia, Y., Hu, C. et al., Fisher non-negative matrix factorization for learning local features, Asian Conference on Computer Vision, Korea, January 27-30, 2004.Google Scholar
  19. 19.
    Ahn, J. H., Choi, S., Oh, J. H., A multiplicative up-propagation algorithm, in Proc. Interal Conference on Machine Learning, 2004, 17-24.Google Scholar
  20. 20.
    Lee, D. D., Seung, H. S., Learning in intelligent embedded systems, in Proceedings of the Embedded Systems Workshop, March 29-31, 1999, Cambridge, Massachusetts, USA. seung/papers/lee_es99.pdf.Google Scholar
  21. 21.
    Lee, J. S., Lee, D. D., Choi, S. et al., Non-negative matrix factorization of dynamic images in nuclear medicine, in Proceedings of IEEE Nuclear Science Symposium Conference Record, San Diego, California, November 4-10, 2001, 4: 2027–2030.Google Scholar
  22. 22.
    Ramanath, R., Kuehni, R. G., Snyder, W. E. et al., Spectral spaces and color spaces, Color Research and Application, 2004, 29(1): 29–37.Google Scholar
  23. 23.
    Kawamoto, T., Hotta, K., Mishima, T. et al., Estimation of single tones from chord sounds using non-negative matrix factorization, Neural Network World, 2000, 3: 429–436.Google Scholar
  24. 24.
    Cho, Y. C., Choi, S., Bang, S. Y., Non-negative component parts of sound for classification, in Proc. IEEE Int’l Symp. Signal Processing and Information Technology, Darmstadt, Germany, December 14-17, 2003.Google Scholar
  25. 25.
    Novak, M., Mammone, R., Improvement of non-negative matrix factorization based language model using exponential models, IEEE Workshop on Automatic Speech Recognition and Understanding, 2001, 190-193.Google Scholar
  26. 26.
    Xu, W., Liu, X., Gong, Y., Document-clustering based on non-negative matrix factorization, Proceedings of SIGIR’03, July 28-August 1, Toronto, CA, 2003, 267-273.Google Scholar
  27. 27.
    Seppanen, J. K., Hollmén, J. E., Bingham, E. et al., Nonnegative matrix factorization on gene expression data, Bioinformatics, 2002, poster 49, Bergen, April 2002.Google Scholar
  28. 28.
    Brunet, J. P., Tamayo, P., Golub, T. R. et al., Metagenes and molecular pattern discovery using matrix factorization, Proceedings of the National Academy of Sciences, 2004, 101: 4164–4169.CrossRefGoogle Scholar
  29. 29.
    Dai Yingxia et al., System Safety and Intrusion Detection, Beijing: Tsinghua University Publishing Company, 2002.Google Scholar
  30. 30.
    Forrest, S., Hofmeyr, S. A., Somayaji, A., Longstaff, T. A., A sense of self for unix processes, Proceedings of IEEE Symposium on Computer Security and Privacy [C], 1996, 120-128.Google Scholar
  31. 31.
    Warrender, C., Forrest, S., Pearlmutter, B., Detecting intrusions using system calls: Alternative data models, in Proceedings of IEEE Symposium on Security and Privacy [C], 1999, 133-145.Google Scholar
  32. 32.
    Cox, I. J., Kilian, J., Leighton, T. et al., Secure spread spectrum watermarking for multimedia, IEEE Trans. on Image Processing, 1997, 6(12): 1673–1687.CrossRefGoogle Scholar
  33. 33.
    Liu, W. X., Zheng, N. N., Li, X., Nonnegative matrix factorization for EEG signal classification, ISNN, 2004, (2): 470-475.Google Scholar
  34. 34.
    Wolpaw, J. R., Birbaumer, N., McFarland, D. J. et al., Brain computer interfaces for communication and control, Clinical Neurophysiology, 2002, 113(6): 767–791.CrossRefGoogle Scholar
  35. 35.
    Keirn, Z. A., Alternative modes of communication between man and machine [D], Masters Thesis, Electrical Engineering, Purdue University, 1988.Google Scholar
  36. 36.
    Keirn, Z. A., Aunon, J. I., A new model of communication between man and his surroundings, IEEE Transactions on Biomedical Engineering, 1990, 37: 1209–1214.CrossRefGoogle Scholar
  37. 37.
    Anderson, C. W., Kirby, M., EEG subspace representations and feature selection for brain computer interfaces, in Proceedings of the 1st IEEE Workshop on Computer Vision and Pattern Recognition for Human Computer Interaction, 2003, 475-483.Google Scholar
  38. 38.
    Garrett, G., Peterson, D. A., Anderson, C. W. et al., Comparison of linear and nonlinear methods for EEG signal classification, IEEE Transactions on Neural Systems and Rehabilitative Engineering, 2003, 11(2): 141–144.Google Scholar
  39. 39.
    Lin Chen, Where does perception come from? Academic Report of the 12th Academicians Conference of Chinese Academy of Sciences. Scholar
  40. 40.
    Zheng Nanning, Cognition and computer vision, Academic Report of Ten Year Anniversary of the National Outstanding Youth Foundation of China, 2004.Google Scholar
  41. 41.
    Saito, N., Larson, B. M., Benichou, B., Sparsity vs. statistical independence from a best-basis viewpoint, in Proc. SPIE Wavelet Applications in Signal and Image Processing VIII (eds. Aldroubi, A., Laine, A. F., Unser, M. A.), vol. 4119, 2000, 474–486.Google Scholar
  42. 42.
    Saul, L. K., Sha, F., Lee, D. D., Statistical signal processing with nonnegativity constraints, Proceedings of the Eighth European Conference on Speech Communication and Technology, vol. 2, Geneva, Switzerland, 2003, 1001-1004.Google Scholar
  43. 43.
    Liu, W. G., Yi, G. L., Existing and New Algorithms for Nonnegative Matrix Factorization. 383CProject/CS 383C Project.htm.Google Scholar
  44. 44.
    Liu, W. X., Zheng, N. N., Li, X., Relative gradient speeding up additive updates for nonnegative matrix factorization, Neurocomputing, 2004, 57: 493–499.CrossRefGoogle Scholar
  45. 45.
    Wild, S. M., Curry, J., Dougherty, A., Improving non-negative matrix factorizations through structured initialization, Pattern Recognition, 2004, 37: 2217–2232.CrossRefGoogle Scholar
  46. 46.
    Donoho, D., Stodden, V., When does non-negative matrix factorization give a correct decomposition into parts? Tech. Report, Department of Statistics, Stanford University, 2003.Google Scholar
  47. 47.
    Chu, M., Diele, F., Plemmons, R. et al., Optimality, computation, and interpretations of nonnegative (non-negative) matrix factorizations, Submitted to the SIAM Journal on Matrix Analysis, October 2004.Google Scholar
  48. 48.
    Plumbley, M. D., Conditions for non-negative independent component analysis, IEEE Signal Processing Letters, 2002, 9(6): 177–180.CrossRefGoogle Scholar
  49. 49.
    Downs, O. B., MacKay, D., Lee, D. D., The nonnegative Boltzmann machine, in Advances in Neural Information Processing Systems 12 (eds. Solla, S. A., Leen, T. K., Muller, K.-R.), 2000, 428-434.Google Scholar
  50. 50.
    Ross, D., Zemel, R., Multiple cause vector quantization, in Advances in Neural Information Processing Systems 15 (eds. Becker, S., Thrun, S., Obermayer, K.), Cambridge: MIT Press, 2003.Google Scholar
  51. 51.
    Ge, X. J., Iwata, S., Learning the parts of objects by auto-association, Neural Networks, 2002, 15: 285–295.CrossRefGoogle Scholar
  52. 52.
    Wang Peng, Nonnegative matrix factorization: Wonderful power of math, Computer Education, 2004, 10: 38–40.Google Scholar

Copyright information

© Science in China Press 2006

Authors and Affiliations

  • Liu Weixiang 
    • 1
  • Zheng Nanning 
    • 1
  • You Qubo 
    • 1
  1. 1.Institute of Artificial Intelligence and Robotics, Xi’an Jiaotong UniversityXi’anChina

Personalised recommendations