Online Kernel Matrix Factorization

  • Andrés Esteban Páez-TorresEmail author
  • Fabio A. González
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 9423)


Matrix factorization (MF) has shown to be a competitive machine learning strategy for many problems such as dimensionality reduction, latent topic modeling, clustering, dictionary learning and manifold learning, among others. In general, MF is a linear modeling method, so different strategies, most of them based on kernel methods, have been proposed to extend it to non-linear modeling. However, as with many other kernel methods, memory requirements and computing time limit the application of kernel-based MF methods in large-scale problems. In this paper, we present a new kernel MF (KMF). This method uses a budget, a set of representative points of size \(p\ll n\), where n is the size of the training data set, to tackle the memory problem, and uses stochastic gradient descent to tackle the computation time and memory problems. The experimental results show a performance, in particular tasks, comparable to other kernel matrix factorization and clustering methods, and a competitive computing time in large-scale problems.


Feature space factorization Kernel matrix factorization Large-scale learning 


  1. 1.
    An, S., Yun, J.M., Choi, S.: Multiple kernel nonnegative matrix factorization. In: 2011 ICASSP Conference, pp. 1976–1979. IEEE (2011)Google Scholar
  2. 2.
    Bottou, L.: Large-scale machine learning with stochastic gradient descent. In: COMPSTAT 2010, pp. 177–186. Springer (2010)Google Scholar
  3. 3.
    Chen, X., Cai, D.: Large-scale spectral clustering with landmark-based representation. In: Twenty-Fifth AAAI Conference (2011)Google Scholar
  4. 4.
    Crammer, K., Kandola, J., Singer, Y.: Online classification on a budget. In: Advances in Neural Information Processing Systems, pp. 225–232 (2004)Google Scholar
  5. 5.
    Dhillon, I.S., Guan, Y., Kulis, B.: Kernel k-means: spectral clustering and normalized cuts. In: Tenth ACM SIGKDD Conference, pp. 551–556. ACM (2004)Google Scholar
  6. 6.
    Ding, C., Li, T., Jordan, M.I.: Convex and semi-nonnegative matrix factorizations. IEEE Transactions on Pattern Analysis and Machine Intelligence 32(1), 45–55 (2010)CrossRefGoogle Scholar
  7. 7.
    Hyvärinen, A., Karhunen, J., Oja, E.: Independent component analysis, vol. 46. John Wiley & Sons (2004)Google Scholar
  8. 8.
    Jun, Y., Jintao, M., Xiaoxu, L.: A kernel based non-negative matrix factorization. In: Second IITA-GRS Conference, vol. 1, pp. 376–379. IEEE (2010)Google Scholar
  9. 9.
    Lee, D.D., Seung, H.S.: Learning the parts of objects by non-negative matrix factorization. Nature 401(6755), 788–791 (1999)CrossRefGoogle Scholar
  10. 10.
    Li, P., Chen, C., Bu, J.: Clustering analysis using manifold kernel concept factorization. Neurocomputing 87, 120–131 (2012)CrossRefGoogle Scholar
  11. 11.
    Li, Y., Ngom, A.: A new kernel non-negative matrix factorization and its application in microarray data analysis. In: 2012 CIBCB Symposium, pp. 371–378. IEEE (2012)Google Scholar
  12. 12.
    Pan, B., Lai, J., Chen, W.S.: Nonlinear nonnegative matrix factorization based on mercer kernel construction. Pattern Recognition 44(10), 2800–2810 (2011)CrossRefzbMATHGoogle Scholar
  13. 13.
    Rendle, S., Schmidt-Thieme, L.: Online-updating regularized kernel matrix factorization models for large-scale recommender systems. In: 2008 ACM RecSys Conference, pp. 251–258. ACM (2008)Google Scholar
  14. 14.
    Schölkopf, B., Smola, A., Müller, K.-R.: Kernel principal component analysis. In: Gerstner, W., Hasler, M., Germond, A., Nicoud, J.-D. (eds.) ICANN 1997. LNCS, vol. 1327, pp. 583–588. Springer, Heidelberg (1997) Google Scholar
  15. 15.
    Sculley, D.: Web-scale k-means clustering. In: WWW 2010 Conference, pp. 1177–1178. ACM (2010)Google Scholar
  16. 16.
    Shawe-Taylor, J., Cristianini, N.: Kernel methods for pattern analysis. Cambridge University Press (2004)Google Scholar
  17. 17.
    Wang, L., Rege, M., Dong, M., Ding, Y.: Low-rank kernel matrix factorization for large-scale evolutionary clustering. IEEE Transactions on Knowledge and Data Engineering 24(6), 1036–1050 (2012)CrossRefGoogle Scholar
  18. 18.
    Xia, Z., Ding, C., Chow, E.: Robust kernel nonnegative matrix factorization. In: 12th ICDMW Conference, pp. 522–529. IEEE (2012)Google Scholar
  19. 19.
    Zhang, D., Zhou, Z.-H., Chen, S.: Non-negative Matrix Factorization on Kernels. In: Yang, Q., Webb, G. (eds.) PRICAI 2006. LNCS (LNAI), vol. 4099, pp. 404–412. Springer, Heidelberg (2006) CrossRefGoogle Scholar

Copyright information

© Springer International Publishing Switzerland 2015

Authors and Affiliations

  • Andrés Esteban Páez-Torres
    • 1
    Email author
  • Fabio A. González
    • 1
  1. 1.MindLab Research GroupUniversidad Nacional de ColombiaBogotáColombia

Personalised recommendations