Parallelized Preconditioned Model Building Algorithm for Matrix Factorization

  • Kamer KayaEmail author
  • Ş. İlker Birbil
  • M. Kaan Öztürk
  • Amir Gohari
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 10710)


Matrix factorization is a common task underlying several machine learning applications such as recommender systems, topic modeling, or compressed sensing. Given a large and possibly sparse matrix A, we seek two smaller matrices W and H such that their product is as close to A as possible. The objective is minimizing the sum of square errors in the approximation. Typically such problems involve hundreds of thousands of unknowns, so an optimizer must be exceptionally efficient. In this study, a new algorithm, Preconditioned Model Building is adapted to factorize matrices composed of movie ratings in the MovieLens data sets with 1, 10, and 20 million entries. We present experiments that compare the sequential MATLAB implementation of the PMB algorithm with other algorithms in the minFunc package. We also employ a lock-free sparse matrix factorization algorithm and provide a scalable shared-memory parallel implementation. We show that (a) the optimization performance of the PMB algorithm is comparable to the best algorithms in common use, and (b) the computational performance can be significantly increased with parallelization.


Preconditioned model building Matrix factorization Multicore parallelism 


  1. 1.
    Berry, M.W., Browne, M., Langville, A.N., Pauca, V.P., Plemmons, R.J.: Algorithms and applications for approximate nonnegative matrix factorization. Comput. Stat. Data Anal. 52(1), 155–173 (2007)MathSciNetCrossRefGoogle Scholar
  2. 2.
    Bertsekas, D.P.: Nonlinear Programming. Athena Scientific, Belmont (1999)zbMATHGoogle Scholar
  3. 3.
    Duff, I.S., Erisman, A.M., Reid, J.K.: Direct Methods for Sparse Matrices. Oxford University Press Inc., New York (1986)zbMATHGoogle Scholar
  4. 4.
    Harper, F.M., Konstan, J.A.: The movielens datasets: history and context. ACM Trans. Interact. Intell. Syst. 5(4), 19:1–19:19 (2015)CrossRefGoogle Scholar
  5. 5.
    Hernando, A., Bobadilla, J., Ortega, F.: A non negative matrix factorization for collaborative filtering recommender systems based on a bayesian probabilistic model. Knowl.-Based Syst. 97, 188–202 (2016)CrossRefGoogle Scholar
  6. 6.
    Koren, Y., Bell, R., Volinsky, C.: Matrix factorization techniques for recommender systems. Computer 42(8), 30–37 (2009)CrossRefGoogle Scholar
  7. 7.
    Lee, D.D., Seung, H.S.: Learning the parts of objects by non-negative matrix factorization. Nature 401(6755), 788–791 (1999)CrossRefGoogle Scholar
  8. 8.
    Liu, D.C., Nocedal, J.: On the limited-memory BFGS method for large scale optimization. Math. Program. 45, 503–528 (1989)MathSciNetCrossRefGoogle Scholar
  9. 9.
    Öztoprak, F.: Parallel Algorithms for Nonlinear Optimization. Ph.D. thesis. Sabancı University (2011)Google Scholar
  10. 10.
    Öztoprak, F., Birbil, S.I.: An alternative globalization strategy for unconstrained optimization. arXiv preprint, arXiv:1705.05158 (2017) (To appear in Optimization)MathSciNetCrossRefGoogle Scholar
  11. 11.
    Schmidt, M.: minFunc: unconstrained differentiable multivariate optimization in matlab, Accessed 22 March 2017

Copyright information

© Springer International Publishing AG 2018

Authors and Affiliations

  • Kamer Kaya
    • 1
    Email author
  • Ş. İlker Birbil
    • 1
  • M. Kaan Öztürk
    • 1
  • Amir Gohari
    • 1
  1. 1.Faculty of Engineering and Natural SciencesSabancı UniversityIstanbulTurkey

Personalised recommendations