Skip to main content
SpringerLink
Log in

Inductive Matrix Completion with Feature Selection

  • GENERAL NUMERICAL METHODS
  • Published:
Computational Mathematics and Mathematical Physics Aims and scope Submit manuscript

Abstract

We consider the problem of inductive matrix completion, i.e., the reconstruction of a matrix using side features of its rows and columns. In numerous applications, however, side information of this kind includes redundant or uninformative features, so feature selection is required. An approach based on matrix factorization with group LASSO regularization on the coefficients of the side features is proposed, which combines feature selection with matrix completion. It is proved that the theoretical sample complexity for the proposed approach is lower than for methods without sparsifying. A computationally efficient iterative procedure for simultaneous matrix completion and feature selection is proposed. Experiments on synthetic and real-world data demonstrate that, due to the feature selection procedure, the proposed approach outperforms other methods.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1.
Fig. 2.

Similar content being viewed by others

REFERENCES

  1. J. D. M. Rennie and N. Srebro, “Fast maximum margin matrix factorization for collaborative prediction,” Proceedings of the 22nd International Conference on Machine Learning (2005), pp. 713–719.

  2. Y. Koren, R. Bell, and C. Volinsky, “Matrix factorization techniques for recommender systems,” Computer 42 (8), 30–37 (2009).

    Article  Google Scholar 

  3. J. Yi, T. Yang, R. Jin, A. K. Jain, and M. Mahdavi, “Robust ensemble clustering by matrix completion,” 2012 IEEE 12th International Conference on Data Mining (ICDM) (2012), pp. 1176–1181.

  4. A. Argyriou, T. Evgeniou, and M. Pontil, “Convex multi-task feature learning,” Mach. Learn. 73 (3), 243–272 (2008).

    Article  Google Scholar 

  5. R. S. Cabral, F. Torre, J. P. Costeira, and A. Bernardino, “Matrix completion for multi-label image classification,” Advances in Neural Information Processing Systems (2011), pp. 190–198.

  6. Z. Weng and X. Wang, “Low-rank matrix completion for array signal processing,” 2012 IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP) (2012), pp. 2697–2700.

  7. P. Chen and D. Suter, “Recovering the missing components in a large noisy low-rank matrix: Application to SFM,” IEEE Trans. Pattern Anal. Mach. Intell. 26 (8), 1051–1063 (2004).

    Article  Google Scholar 

  8. E. J. Candès and B. Recht, “Exact matrix completion via convex optimization,” Found. Comput. Math. 9 (6), 717–772 (2009).

    Article  MathSciNet  Google Scholar 

  9. E. J. Candès and T. Tao, “The power of convex relaxation: Near-optimal matrix completion,” IEEE Trans. Inf. Theory 56 (5), 2053–2080 (2010).

    Article  MathSciNet  Google Scholar 

  10. O. Shamir and S. Shalev-Shwartz, “Matrix completion with the trace norm: Learning, bounding, and transducing,” J. Mach. Learn. Res. 15 (1), 3401–3423 (2014).

    MathSciNet  MATH  Google Scholar 

  11. J. Hannon, M. Bennett, and B. Smyth, “Recommending twitter users to follow using content and collaborative filtering approaches,” Proceedings of the Fourth ACM Conference on Recommender Systems (2010), pp. 199–206.

  12. M. Xu, R. Jin, and Z.-H. Zhou, “Speedup matrix completion with side information: Application to multi-label learning,” Advances in Neural Information Processing Systems (2013), pp. 2301–2309.

  13. N. Natarajan and I. S. Dhillon, “Inductive matrix completion for predicting gene-disease associations,” Bioinformatics 30 (12), i60–i68 (2014).

    Article  Google Scholar 

  14. K.-Y. Chiang, C.-J. Hsieh, and I. S. Dhillon, “Matrix completion with noisy side information,” Proceedings of the 28th International Conference on Neural Information Processing Systems (2015), Vol. 2, pp. 3447–3455.

  15. S. Si, K.-Y. Chiang, C.-J. Hsieh, N. Rao, and I. S. Dhillon, “Goal-directed inductive matrix completion,” Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (2016), pp. 1165–1174.

  16. J. Lu, G. Liang, J. Sun, and J. Bi, “A sparse interactive model for matrix completion with side information,” Advances in Neural Information Processing Systems 29 (2016), pp. 4071–4079.

    Google Scholar 

  17. Y. Guo, “Convex co-embedding for matrix completion with predictive side information,” Proceedings of the Thirty-First AAAI Conference on Artificial Intelligence (AAAI-17) (2017), pp. 1955–1961.

  18. X. Zhang, S. Du, and Q. Gu, “Fast and sample efficient inductive matrix completion via multi-phase Procrustes flow,” Proceedings of the 35th International Conference on Machine Learning (2018), pp. 5756–5765.

  19. A. Soni, T. Chevalier, and S. Jain, “Noisy inductive matrix completion under sparse factor models,” 2017 IEEE Intern. Symposium on Information Theory (ISIT) (2017), pp. 2990–2994.

  20. P. Jain, P. Netrapalli, and S. Sanghavi, “Low-rank matrix completion using alternating minimization,” Proceedings of the 45 Annual ACM Symposium on Theory of Computing (2013), pp. 665–674.

  21. Q. Berthet and N. Baldin, “Statistical and computational rates in graph logistic regression,” International Conference on Artificial Intelligence and Statistics (2020), pp. 2719–2730.

  22. D. P. Bertsekas and J. N. Tsitsiklis, Parallel and Distributed Computation: Numerical Methods (Prentice Hall, Englewood Cliffs, 1989).

    MATH  Google Scholar 

  23. S. Boyd, N. Parikh, E. Chu, B. Peleato, and J. Eckstein, “Distributed optimization and statistical learning via the alternating direction method of multipliers,” Found. Trends Mach. Learn. 3 (1), 1–122 (2011).

    Article  Google Scholar 

  24. A. Maurer and M. Pontil, “Structured sparsity and generalization,” J. Mach. Learn. Res. 13, 671–690 (2012).

    MathSciNet  MATH  Google Scholar 

  25. P. L. Bartlett and S. Mendelson, “Rademacher and Gaussian complexities: Risk bounds and structural results,” J. Mach. Learn. Res. 3, 463–482 (2002).

    MathSciNet  MATH  Google Scholar 

  26. M. Mohri, A. Rostamizadeh, and A. Talwalkar, Foundations of Machine Learning (MIT Press, US, 2012).

    MATH  Google Scholar 

  27. R. Glowinski and A. Marroco, “Sur l’approximation, par éléments finis d’ordre un, et la résolution, par pénalisation: Dualité d’une classe de problémes de Dirichlet non linкires,” ESAIM: Math. Model. Numer. Anal. 9, 41–76 (1975).

    MATH  Google Scholar 

  28. D. Gabay and B. Mercier, “A dual algorithm for the solution of nonlinear variational problems via finite element approximation,” Comput. Math. Appl. 2 (1), 17–40 (1976).

    Article  Google Scholar 

  29. D. Gabay, “Chapter 9: Applications of the method of multipliers to variational inequalities,” in Augmented Lagrangian Methods: Applications to the Numerical Solution of Boundary-Value Problems (North-Holland, Amsterdam, 1983), pp. 299–331.

    Google Scholar 

  30. J. Eckstein and D. P. Bertsekas, “On the Douglas–Rachford splitting method and the proximal point algorithm for maximal monotone operators,” Math. Progr. 55 (1), 293–318 (1992).

    Article  MathSciNet  Google Scholar 

  31. H.-F. Yu, P. Jain, P. Kar, and I. S. Dhillon, “Large-scale multi-label learning with missing labels,” Proceedings of the 31st International Conference on Machine Learning (2014), pp. 593–601.

  32. C.-J. Lin, R. C. Weng, and S. S. Keerthi, “Trust region newton method for logistic regression,” J. Mach. Learn. Res. 9, 627–650 (2008).

    MathSciNet  MATH  Google Scholar 

  33. N. Simon, J. Friedman, T. Hastie, and R. Tibshirani, “A sparse-group lasso,” J. Comput. Graphic. Stat. 22 (2), 231–245 (2013).

    Article  MathSciNet  Google Scholar 

  34. C.-C. Chang and C.-J. Lin, “LIBSVM: A library for support vector machines,” ACM Trans. Intell. Syst. Technol. 2 (3), 1–27 (2011).

    Article  Google Scholar 

  35. M. R. Farhat, B. J. Shapiro, K. J. Kieser, et al., “Genomic analysis identifies targets of convergent positive selection in drug-resistant mycobacterium tuberculosis,” Nat. Genet. 45, 1183–1189 (2013).

    Article  Google Scholar 

  36. T. M. Walker, T. A. Kohl, S. V. Omar, et al., “Whole-genome sequencing for prediction of mycobacterium tuberculosis drug susceptibility and resistance: A retrospective cohort study,” Lancet Infect. Dis. Appl. 15 (10), 1193–1202 (2015).

    Article  Google Scholar 

  37. L. J. Pankhurst, C. Elias, A. A. Votintseva, et al., “Rapid, comprehensive, and affordable mycobacterial diagnosis with whole-genome sequencing: A prospective study,” Lancet Respir. Med. 4 (1), 49–58 (2016).

    Article  Google Scholar 

  38. F. Coll, J. Phelan, G. A. Hill-Cawthorne, et al., “Genome-wide analysis of multi- and extensively drug-resistant mycobacterium tuberculosis,” Nat. Genet. 50 (2), 307–316 (2018).

    Article  Google Scholar 

Download references

Funding

This work was supported by the Russian Foundation for Basic Research, project no. 18-37-00489.

Author information

Authors and Affiliations

  1. Skolkovo Institute of Science and Technology (Skoltech), 121205, Moscow, Russia

    I. Nazarov, M. Panov & B. Shirokikh

  2. Institute for Information Transmission Problems of the Russian Academy of Sciences (Kharkevich Institute), 127051, Moscow, Russia

    G. Fedonin & B. Shirokikh

  3. Moscow Institute of Physics and Technology (National Research University), 141700, Dolgoprudnyi, Moscow oblast, Russia

    M. Burkina, G. Fedonin & B. Shirokikh

  4. Central Research Institute of Epidemiology, 111123, Moscow, Russia

    G. Fedonin

Authors
  1. M. Burkina
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. I. Nazarov
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. M. Panov
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. G. Fedonin
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. B. Shirokikh
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding authors

Correspondence to I. Nazarov or M. Panov.

Additional information

Translated by I. Ruzanova

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Burkina, M., Nazarov, I., Panov, M. et al. Inductive Matrix Completion with Feature Selection. Comput. Math. and Math. Phys. 61, 719–732 (2021). https://doi.org/10.1134/S0965542521050079

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1134/S0965542521050079

Keywords:

Search

Navigation