Skip to main content
SpringerLink
Log in

Nonnegative matrix factorization and its applications in pattern recognition

  • Reviews
  • Published:
Chinese Science Bulletin

Abstract

Matrix factorization is an effective tool for large-scale data processing and analysis. Nonnegative matrix factorization (NMF) method, which decomposes the nonnegative matrix into two nonnegative factor matrices, provides a new way for matrix factorization. NMF is significant in intelligent information processing and pattern recognition. This paper firstly introduces the basic idea of NMF and some new relevant methods. Then we discuss the loss functions and relevant algorithms of NMF in the framework of probabilistic models based on our researches, and the relationship between NMF and information processing of perceptual process. Finally, we make use of NMF to deal with some practical questions of pattern recognition and point out some open problems for NMF.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. Hubert, L. J., Meulman, J. J., Heiser, W. J., Two purposes for matrix factorization: A historical appraisal, SIAM Review, 2000, 42: 68–82.

    Article  Google Scholar 

  2. Duda, R. O., Hart, P. E., Stork, D. G., Pattern Classification, 2nd ed., New York: John Wiley & Sons, 2001.

    Google Scholar 

  3. Lee, D. D., Seung, H. S., Learning the parts of objects with nonnegative matrix factorization, Nature, 1999, 401: 788–791.

    Google Scholar 

  4. Paatero, P., Tapper, U., Positive matrix factorization: A non-negative factor model with optimal utilization of error estimates of data values, Environmetrics, 1994, 5: 111–126.

    Google Scholar 

  5. Lee, D. D., Seung, H. S., Algorithms for nonnegative matrix factorization, Advances in Neural Information Processing Systems 13 (eds. Leen, T., Dietterich, T., Tresp, V.), Cambridge: MIT Press, 2000.

    Google Scholar 

  6. Sajda, P., Du, S., Parra, L. et al., Recovery of constituent spectra using non-negative matrix factorization, Proc. SPIE, 2003, 5207: 321–331.

    Google Scholar 

  7. Cichocki, A., Amari, S., Adaptive Blind Signal and Image Processing: Learning Algorithms and Applications, New York: John Wiley & Sons, 2002.

    Google Scholar 

  8. Liu, W. X., Zheng, N. N., Learning sparse features for classification by mixture models, Pattern Recognition Letters, 2004, 25(2): 155–161.

    Google Scholar 

  9. Li, S. Z., Hou, X. W., Zhang, H. J. et al., Learning spatially localized, parts-based representation, Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2001, 1: 207–212.

    Google Scholar 

  10. Olshausen, B. A., Field, D. J., Emergence of simple-cell receptive field properties by learning a sparse code for natural images, Nature, 1996, 381: 607–609.

    Article  Google Scholar 

  11. Hoyer, P. O., Non-negative sparse coding, in Proc. Neural Net-works for Signal Processing, 2002, 557-565.

  12. Hoyer, P. O., Modeling receptive fields with non-negative sparse coding, Neurocomputing, 2003, (52-54): 547-552.

  13. Liu, W. X., Zheng, N. N., Lu, X. F., Non-negative matrix factorization for visual coding, Proceedings of the IEEE International Conference on Acoustics, Speech, and Signal Processing, 2003, 3: 293–296.

    Google Scholar 

  14. Hoyer, P. O., Non-negative matrix factorization with sparseness constraints, Journal of Machine Learning Research, 2004, 5: 1457–1469.

    Google Scholar 

  15. Guillamet, D., Vitriã, J., Schiele, B., Introducing a weighted non-negative matrix factorization for image classification, Pattern Recognition Letters, 2003, 24(14): 2447–2454.

    Article  Google Scholar 

  16. Paatero, P., Least squares formulation of robust non-negative factor analysis, Chemometrics and Intelligent Laboratory Systems, 1997, 37(1): 23–35.

    Article  Google Scholar 

  17. Welling, M., Weber, M., Positive tensor factorization, Pattern Recognition Letters, 2001, 22(12): 1255–1261.

    Article  Google Scholar 

  18. Wang, Y., Jia, Y., Hu, C. et al., Fisher non-negative matrix factorization for learning local features, Asian Conference on Computer Vision, Korea, January 27-30, 2004.

  19. Ahn, J. H., Choi, S., Oh, J. H., A multiplicative up-propagation algorithm, in Proc. Interal Conference on Machine Learning, 2004, 17-24.

  20. Lee, D. D., Seung, H. S., Learning in intelligent embedded systems, in Proceedings of the Embedded Systems Workshop, March 29-31, 1999, Cambridge, Massachusetts, USA. http://hebb.mit.edu/people/ seung/papers/lee_es99.pdf.

  21. Lee, J. S., Lee, D. D., Choi, S. et al., Non-negative matrix factorization of dynamic images in nuclear medicine, in Proceedings of IEEE Nuclear Science Symposium Conference Record, San Diego, California, November 4-10, 2001, 4: 2027–2030.

    Google Scholar 

  22. Ramanath, R., Kuehni, R. G., Snyder, W. E. et al., Spectral spaces and color spaces, Color Research and Application, 2004, 29(1): 29–37.

    Google Scholar 

  23. Kawamoto, T., Hotta, K., Mishima, T. et al., Estimation of single tones from chord sounds using non-negative matrix factorization, Neural Network World, 2000, 3: 429–436.

    Google Scholar 

  24. Cho, Y. C., Choi, S., Bang, S. Y., Non-negative component parts of sound for classification, in Proc. IEEE Int’l Symp. Signal Processing and Information Technology, Darmstadt, Germany, December 14-17, 2003.

  25. Novak, M., Mammone, R., Improvement of non-negative matrix factorization based language model using exponential models, IEEE Workshop on Automatic Speech Recognition and Understanding, 2001, 190-193.

  26. Xu, W., Liu, X., Gong, Y., Document-clustering based on non-negative matrix factorization, Proceedings of SIGIR’03, July 28-August 1, Toronto, CA, 2003, 267-273.

  27. Seppanen, J. K., Hollmén, J. E., Bingham, E. et al., Nonnegative matrix factorization on gene expression data, Bioinformatics, 2002, poster 49, Bergen, April 2002.

  28. Brunet, J. P., Tamayo, P., Golub, T. R. et al., Metagenes and molecular pattern discovery using matrix factorization, Proceedings of the National Academy of Sciences, 2004, 101: 4164–4169.

    Article  Google Scholar 

  29. Dai Yingxia et al., System Safety and Intrusion Detection, Beijing: Tsinghua University Publishing Company, 2002.

    Google Scholar 

  30. Forrest, S., Hofmeyr, S. A., Somayaji, A., Longstaff, T. A., A sense of self for unix processes, Proceedings of IEEE Symposium on Computer Security and Privacy [C], 1996, 120-128.

  31. Warrender, C., Forrest, S., Pearlmutter, B., Detecting intrusions using system calls: Alternative data models, in Proceedings of IEEE Symposium on Security and Privacy [C], 1999, 133-145.

  32. Cox, I. J., Kilian, J., Leighton, T. et al., Secure spread spectrum watermarking for multimedia, IEEE Trans. on Image Processing, 1997, 6(12): 1673–1687.

    Article  Google Scholar 

  33. Liu, W. X., Zheng, N. N., Li, X., Nonnegative matrix factorization for EEG signal classification, ISNN, 2004, (2): 470-475.

  34. Wolpaw, J. R., Birbaumer, N., McFarland, D. J. et al., Brain computer interfaces for communication and control, Clinical Neurophysiology, 2002, 113(6): 767–791.

    Article  Google Scholar 

  35. Keirn, Z. A., Alternative modes of communication between man and machine [D], Masters Thesis, Electrical Engineering, Purdue University, 1988.

  36. Keirn, Z. A., Aunon, J. I., A new model of communication between man and his surroundings, IEEE Transactions on Biomedical Engineering, 1990, 37: 1209–1214.

    Article  Google Scholar 

  37. Anderson, C. W., Kirby, M., EEG subspace representations and feature selection for brain computer interfaces, in Proceedings of the 1st IEEE Workshop on Computer Vision and Pattern Recognition for Human Computer Interaction, 2003, 475-483.

  38. Garrett, G., Peterson, D. A., Anderson, C. W. et al., Comparison of linear and nonlinear methods for EEG signal classification, IEEE Transactions on Neural Systems and Rehabilitative Engineering, 2003, 11(2): 141–144.

    Google Scholar 

  39. Lin Chen, Where does perception come from? Academic Report of the 12th Academicians Conference of Chinese Academy of Sciences. http://www.casad.ac.cn/2005-4/2005418101856.htm

  40. Zheng Nanning, Cognition and computer vision, Academic Report of Ten Year Anniversary of the National Outstanding Youth Foundation of China, 2004.

  41. Saito, N., Larson, B. M., Benichou, B., Sparsity vs. statistical independence from a best-basis viewpoint, in Proc. SPIE Wavelet Applications in Signal and Image Processing VIII (eds. Aldroubi, A., Laine, A. F., Unser, M. A.), vol. 4119, 2000, 474–486.

  42. Saul, L. K., Sha, F., Lee, D. D., Statistical signal processing with nonnegativity constraints, Proceedings of the Eighth European Conference on Speech Communication and Technology, vol. 2, Geneva, Switzerland, 2003, 1001-1004.

  43. Liu, W. G., Yi, G. L., Existing and New Algorithms for Nonnegative Matrix Factorization. http://www.cs.utexas.edu/users/liuwg/ 383CProject/CS 383C Project.htm.

  44. Liu, W. X., Zheng, N. N., Li, X., Relative gradient speeding up additive updates for nonnegative matrix factorization, Neurocomputing, 2004, 57: 493–499.

    Article  Google Scholar 

  45. Wild, S. M., Curry, J., Dougherty, A., Improving non-negative matrix factorizations through structured initialization, Pattern Recognition, 2004, 37: 2217–2232.

    Article  Google Scholar 

  46. Donoho, D., Stodden, V., When does non-negative matrix factorization give a correct decomposition into parts? Tech. Report, Department of Statistics, Stanford University, 2003.

  47. Chu, M., Diele, F., Plemmons, R. et al., Optimality, computation, and interpretations of nonnegative (non-negative) matrix factorizations, Submitted to the SIAM Journal on Matrix Analysis, October 2004.

  48. Plumbley, M. D., Conditions for non-negative independent component analysis, IEEE Signal Processing Letters, 2002, 9(6): 177–180.

    Article  Google Scholar 

  49. Downs, O. B., MacKay, D., Lee, D. D., The nonnegative Boltzmann machine, in Advances in Neural Information Processing Systems 12 (eds. Solla, S. A., Leen, T. K., Muller, K.-R.), 2000, 428-434.

  50. Ross, D., Zemel, R., Multiple cause vector quantization, in Advances in Neural Information Processing Systems 15 (eds. Becker, S., Thrun, S., Obermayer, K.), Cambridge: MIT Press, 2003.

    Google Scholar 

  51. Ge, X. J., Iwata, S., Learning the parts of objects by auto-association, Neural Networks, 2002, 15: 285–295.

    Article  Google Scholar 

  52. Wang Peng, Nonnegative matrix factorization: Wonderful power of math, Computer Education, 2004, 10: 38–40.

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Institute of Artificial Intelligence and Robotics, Xi’an Jiaotong University, Xi’an, 710049, China

    Liu Weixiang, Zheng Nanning & You Qubo

Authors
  1. Liu Weixiang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Zheng Nanning
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. You Qubo
    View author publications

    You can also search for this author in PubMed Google Scholar

About this article

Cite this article

Liu, W., Zheng, N. & You, Q. Nonnegative matrix factorization and its applications in pattern recognition. CHINESE SCI BULL 51, 7–18 (2006). https://doi.org/10.1007/s11434-005-1109-6

Download citation

  • Received:

  • Accepted:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11434-005-1109-6

Keywords

Search

Navigation