Skip to main content

Online Projective Nonnegative Matrix Factorization for Large Datasets

  • Conference paper

Part of the Lecture Notes in Computer Science book series (LNTCS,volume 7665)

Abstract

Projective Nonnegative Matrix Factorization (PNMF) is one of the recent methods for computing low-rank approximations to data matrices. It is advantageous in many practical application domains such as clustering, graph partitioning, and sparse feature extraction. However, up to now a scalable implementation of PNMF for large-scale machine learning problems has been lacking. Here we provide an online algorithm for fast PNMF learning with low memory cost. The new algorithm simply applies multiplicative update rules iteratively on small subsets of the data, with historical data naturally accumulated. Consequently users do not need extra efforts to tune any optimization parameters such as learning rates or the history weight. In addition to scalability and convenience, empirical studies on synthetic and real-world datasets indicate that our online algorithm runs much faster than the existing batch version.

Keywords

  • Online learning
  • PNMF
  • NMF
  • large-scale datasets

Supported by the Academy of Finland in the project Finnish Center of Excellence in Computational Inference Research (COIN).

This is a preview of subscription content, access via your institution.

Buying options

Chapter
USD   29.95
Price excludes VAT (USA)
  • DOI: 10.1007/978-3-642-34487-9_35
  • Chapter length: 6 pages
  • Instant PDF download
  • Readable on all devices
  • Own it forever
  • Exclusive offer for individuals only
  • Tax calculation will be finalised during checkout
eBook
USD   99.00
Price excludes VAT (USA)
  • ISBN: 978-3-642-34487-9
  • Instant PDF download
  • Readable on all devices
  • Own it forever
  • Exclusive offer for individuals only
  • Tax calculation will be finalised during checkout
Softcover Book
USD   131.00
Price excludes VAT (USA)

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. Cichocki, A., Zdunek, R., Phan, A.H., Amari, S.: Nonnegative Matrix and Tensor Factorizations: Applications to Exploratory Multi-way Data Analysis. John Wiley (2009)

    Google Scholar 

  2. Ding, C., Li, T., Jordan, M.I.: Convex and semi-nonnegative matrix factorizations. IEEE Transactions on Pattern Analysis and Machine Intelligence 32(1), 45–55 (2010)

    CrossRef  Google Scholar 

  3. Févotte, C., Bertin, N., Durrieu, J.L.: Nonnegative matrix factorization with the Itakura-Saito divergence: with application to music analysis. Neural Computation 21(3), 793–830 (2009)

    MATH  CrossRef  Google Scholar 

  4. Kushner, H.J., Clark, D.S.: Stochastic Approximation Methods for Constrained and Unconstrained Systems. Springer, New York (1978)

    CrossRef  Google Scholar 

  5. Lakshminarayanan, B., Raich, R.: Non-negative matrix factorization for parameter estimation in hidden markov models. In: Proceedings of IEEE International Workshop on Machine Learning for Signal Processing, pp. 89–94 (2010)

    Google Scholar 

  6. Lee, D.D., Seung, H.S.: Learning the parts of objects by non-negative matrix factorization. Nature 401, 788–791 (1999)

    CrossRef  Google Scholar 

  7. Lin, C.J.: Projected gradient methods for non-negative matrix factorization. Neural Computation 19, 2756–2779 (2007)

    MathSciNet  MATH  CrossRef  Google Scholar 

  8. Liu, C., Yang, H., Fan, J., He, L., Wang, Y.: Distributed nonnegative matrix factorization for web-scale dyadic data analysis on MapReduce. In: Proceedings of 19th International World Wide Web Conference (2010)

    Google Scholar 

  9. Mairal, J., Bach, F., Ponce, J., Sapiro, G.: Online learning for matrix factorization and sparse coding. The Journal of Machine Learning Research 11, 19–60 (2010)

    MathSciNet  MATH  Google Scholar 

  10. Phillips, P.J., Moon, H., Rizvi, S.A., Rauss, P.J.: The FERET evaluation methodology for face recognition algorithms. IEEE Trans. Pattern Analysis and Machine Intelligence 22, 1090–1104 (2000)

    CrossRef  Google Scholar 

  11. Woodard, D., Flynn, P.: Finger surface as a biometric identifier. Computer Vision and Image Understanding 100(3), 357–384 (2005)

    CrossRef  Google Scholar 

  12. Yang, Z., Oja, E.: Linear and nonlinear projective nonnegative matrix factorization. IEEE Transaction on Neural Networks 21(5), 734–749 (2010)

    CrossRef  Google Scholar 

  13. Yang, Z., Oja, E.: Unified development of multiplicative algorithms for linear and quadratic nonnegative matrix factorization. IEEE Transactions on Neural Networks 22(12), 1878–1891 (2011)

    CrossRef  Google Scholar 

Download references

Author information

Authors and Affiliations

Authors

Editor information

Editors and Affiliations

Rights and permissions

Reprints and Permissions

Copyright information

© 2012 Springer-Verlag Berlin Heidelberg

About this paper

Cite this paper

Yang, Z., Zhang, H., Oja, E. (2012). Online Projective Nonnegative Matrix Factorization for Large Datasets. In: Huang, T., Zeng, Z., Li, C., Leung, C.S. (eds) Neural Information Processing. ICONIP 2012. Lecture Notes in Computer Science, vol 7665. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-34487-9_35

Download citation

  • DOI: https://doi.org/10.1007/978-3-642-34487-9_35

  • Publisher Name: Springer, Berlin, Heidelberg

  • Print ISBN: 978-3-642-34486-2

  • Online ISBN: 978-3-642-34487-9

  • eBook Packages: Computer ScienceComputer Science (R0)