Online Projective Nonnegative Matrix Factorization for Large Datasets
Projective Nonnegative Matrix Factorization (PNMF) is one of the recent methods for computing low-rank approximations to data matrices. It is advantageous in many practical application domains such as clustering, graph partitioning, and sparse feature extraction. However, up to now a scalable implementation of PNMF for large-scale machine learning problems has been lacking. Here we provide an online algorithm for fast PNMF learning with low memory cost. The new algorithm simply applies multiplicative update rules iteratively on small subsets of the data, with historical data naturally accumulated. Consequently users do not need extra efforts to tune any optimization parameters such as learning rates or the history weight. In addition to scalability and convenience, empirical studies on synthetic and real-world datasets indicate that our online algorithm runs much faster than the existing batch version.
KeywordsOnline learning PNMF NMF large-scale datasets
Unable to display preview. Download preview PDF.
- 1.Cichocki, A., Zdunek, R., Phan, A.H., Amari, S.: Nonnegative Matrix and Tensor Factorizations: Applications to Exploratory Multi-way Data Analysis. John Wiley (2009)Google Scholar
- 5.Lakshminarayanan, B., Raich, R.: Non-negative matrix factorization for parameter estimation in hidden markov models. In: Proceedings of IEEE International Workshop on Machine Learning for Signal Processing, pp. 89–94 (2010)Google Scholar
- 8.Liu, C., Yang, H., Fan, J., He, L., Wang, Y.: Distributed nonnegative matrix factorization for web-scale dyadic data analysis on MapReduce. In: Proceedings of 19th International World Wide Web Conference (2010)Google Scholar