Fast Nonnegative Matrix Factorization and Completion Using Nesterov Iterations

  • Clément Dorffer
  • Matthieu Puigt
  • Gilles Delmaire
  • Gilles Roussel
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 10169)

Abstract

In this paper, we aim to extend Nonnegative Matrix Factorization with Nesterov iterations (Ne-NMF)—well-suited to large-scale problems—to the situation when some entries are missing in the observed matrix. In particular, we investigate the Weighted and Expectation-Maximization strategies which both provide a way to process missing data. We derive their associated extensions named W-NeNMF and EM-W-NeNMF, respectively. The proposed approaches are then tested on simulated nonnegative low-rank matrix completion problems where the EM-W-NeNMF is shown to outperform state-of-the-art methods and the W-NeNMF technique.

Keywords

Low-rank matrix completion Nonnegative matrix factorization Nesterov iterations Gradient descent 

References

  1. 1.
    Candès, E.J., Plan, Y.: Matrix completion with noise. Proc. IEEE 98(6), 925–936 (2010)CrossRefGoogle Scholar
  2. 2.
    Candès, E.J., Recht, B.: Exact matrix completion via convex optimization. Found. Comput. Math. 9(6), 717–772 (2009)MathSciNetCrossRefMATHGoogle Scholar
  3. 3.
    Choo, J., Lee, C., Reddy, C.K., Park, H.: Weakly supervised nonnegative matrix factorization for user-driven clustering. Data Min. Knowl. Disc. 29(6), 1598–1621 (2015)MathSciNetCrossRefGoogle Scholar
  4. 4.
    Chreiky, R., Delmaire, G., Puigt, M., Roussel, G., Courcot, D., Abche, A.: Split gradient method for informed non-negative matrix factorization. In: Vincent, E., Yeredor, A., Koldovský, Z., Tichavský, P. (eds.) Proceedings of LVA/ICA, pp. 376–383 (2015)Google Scholar
  5. 5.
    Dorffer, C., Puigt, M., Delmaire, G., Roussel, G.: Blind calibration of mobile sensors using informed nonnegative matrix factorization. In: Vincent, E., Yeredor, A., Koldovský, Z., Tichavský, P. (eds.) Proceedings of LVA/ICA, pp. 497–505 (2015)Google Scholar
  6. 6.
    Dorffer, C., Puigt, M., Delmaire, G., Roussel, G.: Blind mobile sensor calibration using an informed nonnegative matrix factorization with a relaxed rendezvous model. In: Proceedings of ICASSP, pp. 2941–2945 (2016)Google Scholar
  7. 7.
    Dorffer, C., Puigt, M., Delmaire, G., Roussel, G.: Nonlinear mobile sensor calibration using informed semi-nonnegative matrix factorization with a Vandermonde factor. In: Proceedings of SAM (2016)Google Scholar
  8. 8.
    Fazel, M.: Matrix rank minimization with applications. Ph.D. thesis, Stanford University (2002)Google Scholar
  9. 9.
    Guan, N., Tao, D., Luo, Z., Yuan, B.: NeNMF: an optimal gradient method for nonnegative matrix factorization. IEEE Trans. Signal Proc. 60(6), 2882–2898 (2012)MathSciNetCrossRefGoogle Scholar
  10. 10.
    Guillamet, D., Vitrià, J., Schiele, B.: Introducing a weighted non-negative matrix factorization for image classification. Pattern Recogn. Lett. 24(14), 2447–2454 (2003)CrossRefMATHGoogle Scholar
  11. 11.
    Haldar, J.P., Hernando, D.: Rank-constrained solutions to linear matrix equations using powerfactorization. IEEE Signal Process. Lett. 16(7), 584–587 (2009)CrossRefGoogle Scholar
  12. 12.
    Hamon, R., Emiya, V., Févotte, C.: Convex nonnegative matrix factorization with missing data. In: Procceedings of MLSP (2016)Google Scholar
  13. 13.
    Ho, N.D.: Nonnegative matrix factorizations algorithms and applications. Ph.D. thesis, Université Catholique de Louvain (2008)Google Scholar
  14. 14.
    Hoyer, P.O.: Non-negative matrix factorization with sparseness constraints. J. Mach. Learn. Res. 5, 1457–1469 (2004)MathSciNetMATHGoogle Scholar
  15. 15.
    Kim, H., Park, H.: Nonnegative matrix factorization based on alternating nonnegativity constrained least squares and active set method. SIAM J. Matrix Anal. Appl. 30(2), 713–730 (2008)MathSciNetCrossRefMATHGoogle Scholar
  16. 16.
    Kim, Y.D., Choi, S.: Weighted nonnegative matrix factorization. In: Proceedings of ICASSP, pp. 1541–1544, April 2009Google Scholar
  17. 17.
    Kumar, R., Da Silva, C., Akalin, O., Aravkin, A.Y., Mansour, H., Recht, B., Herrmann, F.J.: Efficient matrix completion for seismic data reconstruction. Geophysics 80(5), V97–V114 (2015)CrossRefGoogle Scholar
  18. 18.
    Lantéri, H., Theys, C., Richard, C., Févotte, C.: Split gradient method for nonnegative matrix factorization. In: Proceedings of EUSIPCO (2010)Google Scholar
  19. 19.
    Lee, D.D., Seung, H.S.: Algorithms for non-negative matrix factorization. In: NIPS, pp. 556–562 (2001)Google Scholar
  20. 20.
    Limem, A., Delmaire, G., Puigt, M., Roussel, G., Courcot, D.: Non-negative matrix factorization under equality constraints–a study of industrial source identification. Appl. Numer. Math. 85, 1–15 (2014)MathSciNetCrossRefMATHGoogle Scholar
  21. 21.
    Limem, A., Puigt, M., Delmaire, G., Roussel, G., Courcot, D.: Bound constrained weighted NMF for industrial source apportionment. In: Proceedings of MLSP (2014)Google Scholar
  22. 22.
    Lin, C.: Projected gradient methods for nonnegative matrix factorization. Neural Comput. 19(10), 2756–2779 (2007)MathSciNetCrossRefMATHGoogle Scholar
  23. 23.
    Liu, C., Yang, H.C., Fan, J., He, L.W., Wang, Y.M.: Distributed nonnegative matrix factorization for web-scale dyadic data analysis on MapReduce. In: Proceedings of WWW Conference, April 2010Google Scholar
  24. 24.
    Mairal, J., Bach, F., Ponce, J., Sapiro, G.: Online learning for matrix factorization and sparse coding. J. Mach. Learn. Res. 11, 19–60 (2010)MathSciNetMATHGoogle Scholar
  25. 25.
    Nesterov, Y.: A method of solving a convex programming problem with convergence rate O(1/k2). Sov. Math. Doklady 27, 372–376 (1983)MATHGoogle Scholar
  26. 26.
    Savvaki, S., Tsagkatakis, G., Panousopoulou, A., Tsakalides, P.: Application of matrix completion on water treatment data. In: Proceedings of CySWater, pp. 3:1–3:6 (2015)Google Scholar
  27. 27.
    Sobral, A., Bouwmans, T., Zahzah, E.: LRSLibrary: low-rank and sparse tools for background modeling and subtraction in videos. In: Robust Low-Rank and Sparse Matrix Decomposition: Applications in Image and Video Processing. CRC Press, Taylor and Francis GroupGoogle Scholar
  28. 28.
    Srebro, N., Jaakkola, T.: Weighted low-rank approximations. In: Proceedings of ICML (2003)Google Scholar
  29. 29.
    Tepper, M., Sapiro, G.: Compressed nonnegative matrix factorization is fast and accurate. IEEE Trans. Signal Process. 64(9), 2269–2283 (2016)MathSciNetCrossRefGoogle Scholar
  30. 30.
    Wang, Y.X., Zhang, Y.J.: Nonnegative matrix factorization: a comprehensive review. IEEE Trans. Knowl. Data Eng. 25(6), 1336–1353 (2013)CrossRefGoogle Scholar
  31. 31.
    Wen, Z., Yin, W., Zhang, Y.: Solving a low-rank factorization model for matrix completion by a nonlinear successive over-relaxation algorithm. Math. Program. Comput. 4(4), 333–361 (2012)MathSciNetCrossRefMATHGoogle Scholar
  32. 32.
    Zhang, S., Wang, W., Ford, J., Makedon, F.: Learning from incomplete ratings using non-negative matrix factorization, chap. 58, pp. 549–553 (2006)Google Scholar
  33. 33.
    Zhou, G., Cichocki, A., Xie, S.: Fast nonnegative matrix/tensor factorization based on low-rank approximation. IEEE Trans. Signal Process. 60(6), 2928–2940 (2012)MathSciNetCrossRefGoogle Scholar

Copyright information

© Springer International Publishing AG 2017

Authors and Affiliations

  • Clément Dorffer
    • 1
  • Matthieu Puigt
    • 1
  • Gilles Delmaire
    • 1
  • Gilles Roussel
    • 1
  1. 1.Univ. Littoral Côte d’Opale, LISIC – EA 4491CalaisFrance

Personalised recommendations