Convex Factorization Machines

  • Mathieu Blondel
  • Akinori Fujino
  • Naonori Ueda
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 9285)


Factorization machines are a generic framework which allows to mimic many factorization models simply by feature engineering. In this way, they combine the high predictive accuracy of factorization models with the flexibility of feature engineering. Unfortunately, factorization machines involve a non-convex optimization problem and are thus subject to bad local minima. In this paper, we propose a convex formulation of factorization machines based on the nuclear norm. Our formulation imposes fewer restrictions on the learned model and is thus more general than the original formulation. To solve the corresponding optimization problem, we present an efficient globally-convergent two-block coordinate descent algorithm. Empirically, we demonstrate that our approach achieves comparable or better predictive accuracy than the original factorization machines on 4 recommendation tasks and scales to datasets with 10 million samples.


Factorization machines Feature interactions Recommender systems Nuclear norm 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Abernethy, J., Bach, F., Evgeniou, T., Vert, J.P.: A new approach to collaborative filtering: Operator estimation with spectral regularization. J. Mach. Learn. Res. 10, 803–826 (2009)Google Scholar
  2. 2.
    Bertsekas, D.P.: Nonlinear programming. Athena scientific Belmont (1999)Google Scholar
  3. 3.
    Buitinck, L., Louppe, G., Blondel, M., Pedregosa, F., Mueller, A., Grisel, O., Niculae, V., Prettenhofer, P., Gramfort, A., Grobler, J., Layton, R., VanderPlas, J., Joly, A., Holt, B., Varoquaux, G.: API design for machine learning software: experiences from the scikit-learn project. In: ECML PKDD Workshop: Languages for Data Mining and Machine Learning, pp. 108–122 (2013)Google Scholar
  4. 4.
    Dudik, M., Harchaoui, Z., Malick, J.: Lifted coordinate descent for learning with trace-norm regularization. In: AISTATS, vol. 22, pp. 327–336 (2012)Google Scholar
  5. 5.
    Fazel, M., Hindi, H., Boyd, S.P.: A rank minimization heuristic with application to minimum order system approximation. American Control Conference 6, 4734–4739 (2001)CrossRefGoogle Scholar
  6. 6.
    Grippo, L., Sciandrone, M.: On the convergence of the block nonlinear gauss-seidel method under convex constraints. Operations Research Letters 26(3), 127–136 (2000)MathSciNetCrossRefGoogle Scholar
  7. 7.
    Hsieh, C.J., Olsen, P.: Nuclear norm minimization via active subspace selection. In: ICML, pp. 575–583 (2014)Google Scholar
  8. 8.
    Koren, Y.: Factorization meets the neighborhood: a multifaceted collaborative filtering model. In: KDD, pp. 426–434 (2008)Google Scholar
  9. 9.
    Koren, Y.: Collaborative filtering with temporal dynamics. Communications of the ACM 53(4), 89–97 (2010)CrossRefGoogle Scholar
  10. 10.
    Loni, B., Shi, Y., Larson, M., Hanjalic, A.: Cross-domain collaborative filtering with factorization machines. In: de Rijke, M., Kenter, T., de Vries, A.P., Zhai, C.X., de Jong, F., Radinsky, K., Hofmann, K. (eds.) ECIR 2014. LNCS, vol. 8416, pp. 656–661. Springer, Heidelberg (2014) CrossRefGoogle Scholar
  11. 11.
    Recht, B., Fazel, M., Parrilo, P.A.: Guaranteed minimum-rank solutions of linear matrix equations via nuclear norm minimization. SIAM Review 52(3), 471–501 (2010)MathSciNetCrossRefzbMATHGoogle Scholar
  12. 12.
    Rendle, S.: Factorization machines. In: ICDM, pp. 995–1000. IEEE (2010)Google Scholar
  13. 13.
    Rendle, S.: Factorization machines with libfm. ACM Transactions on Intelligent Systems and Technology (TIST) 3(3), 57–78 (2012)Google Scholar
  14. 14.
    Rendle, S.: Scaling factorization machines to relational data. In: VLDB, vol. 6, pp. 337–348 (2013)Google Scholar
  15. 15.
    Rendle, S., Gantner, Z., Freudenthaler, C., Schmidt-Thieme, L.: Fast context-aware recommendations with factorization machines. In: SIGIR, pp. 635–644 (2011)Google Scholar
  16. 16.
    Rendle, S., Schmidt-Thieme, L.: Pairwise interaction tensor factorization for personalized tag recommendation. In: WSDM, pp. 81–90. ACM (2010)Google Scholar
  17. 17.
    Shalev-Shwartz, S., Gonen, A., Shamir, O.: Large-scale convex minimization with a low-rank constraint. In: ICML, pp. 329–336 (2011)Google Scholar
  18. 18.
    Srebro, N., Rennie, J., Jaakkola, T.S.: Maximum-margin matrix factorization. In: Advances in Neural Information Processing Systems, pp. 1329–1336 (2004)Google Scholar
  19. 19.
    Tomioka, R., Hayashi, K., Kashima, H.: Estimation of low-rank tensors via convex optimization. arXiv preprint arXiv:1010.0789 (2010)

Copyright information

© Springer International Publishing Switzerland 2015

Authors and Affiliations

  1. 1.NTT Communication Science LaboratoriesKyotoJapan

Personalised recommendations