Advertisement

Deep collective matrix factorization for augmented multi-view learning

  • Ragunathan Mariappan
  • Vaibhav RajanEmail author
Article
  • 26 Downloads
Part of the following topical collections:
  1. Special Issue of the ECML PKDD 2019 Journal Track

Abstract

Learning by integrating multiple heterogeneous data sources is a common requirement in many tasks. Collective Matrix Factorization (CMF) is a technique to learn shared latent representations from arbitrary collections of matrices. It can be used to simultaneously complete one or more matrices, for predicting the unknown entries. Classical CMF methods assume linearity in the interaction of latent factors which can be restrictive and fails to capture complex non-linear interactions. In this paper, we develop the first deep-learning based method, called dCMF, for unsupervised learning of multiple shared representations, that can model such non-linear interactions, from an arbitrary collection of matrices. We address optimization challenges that arise due to dependencies between shared representations through multi-task Bayesian optimization and design an acquisition function adapted for collective learning of hyperparameters. Our experiments show that dCMF significantly outperforms previous CMF algorithms in integrating heterogeneous data for predictive modeling. Further, on two tasks—recommendation and prediction of gene-disease association—dCMF outperforms state-of-the-art matrix completion algorithms that can utilize auxiliary sources of information.

Keywords

Collective Matrix Factorization Deep learning Augmented multi-view learning Bayesian optimization Recommendation Gene-disease prioritization 

Notes

References

  1. Andrew, G., Arora, R., Bilmes, J., & Livescu, K. (2013). Deep canonical correlation analysis. In Proceedings of the 30th international conference on machine learning, pp. 1247–1255.Google Scholar
  2. Bengio, Y., Courville, A., & Vincent, P. (2013). Representation learning: A review and new perspectives. IEEE Transactions on Pattern Analysis and Machine Intelligence, 35(8), 1798–1828.Google Scholar
  3. Bergstra, J. S., Bardenet, R., Bengio, Y., & Kégl, B. (2011). Algorithms for hyper-parameter optimization. In Proceedings of the 24th international conference on neural information processing systems, pp. 2546–2554.Google Scholar
  4. Bergstra, J., & Bengio, Y. (2012). Random search for hyper-parameter optimization. Journal of Machine Learning Research, 13, 281–305.MathSciNetzbMATHGoogle Scholar
  5. Bonilla, E. V., Chai, K. M., & Williams, C. (2007). Multi-task Gaussian process prediction. In Proceedings of the 20th international conference on neural information processing systems, pp. 153–160.Google Scholar
  6. Bouchard, G., Yin, D., & Guo, S. (2013). Convex collective matrix factorization. In Proceedings of the sixteenth international conference on artificial intelligence and statistics, pp. 144–152.Google Scholar
  7. Boutros, M., & Ahringer, J. (2008). The art and design of genetic screens: RNA interference. Nature Reviews Genetics, 9(7), 554.Google Scholar
  8. Chang, S., Han, W., Tang, J., Qi, G.-J., Aggarwal, C. C., & Huang, T. S. (2015). Heterogeneous network embedding via deep architectures. In Proceedings of the 21th ACM SIGKDD international conference on knowledge discovery and data mining, pp. 119–128. ACM.Google Scholar
  9. Chen, M., Xu, Z., Weinberger, K., & Sha, F. (2012). Marginalized denoising autoencoders for domain adaptation. In Proceedings of the 29th international conference on machine learning, pp. 1627–1634.Google Scholar
  10. Coburn, T. C. (2000). Geostatistics for natural resources evaluation. Technometrics, 42(4), 437–438.Google Scholar
  11. Cui, P., Wang, X., Pei, J., & Zhu, W. (2019). A survey on network embedding. IEEE Transactions on Knowledge and Data Engineering, 31(5), 833–852.Google Scholar
  12. Demšar, J. (2006). Statistical comparisons of classifiers over multiple data sets. Journal of Machine Learning Research, 7, 1–30.MathSciNetzbMATHGoogle Scholar
  13. Dong, X., Yu, L., Wu, Z., Sun, Y., Yuan, L., & Zhang, F. (2017). A hybrid collaborative filtering model with deep structure for recommender systems. In Proceedings of the thirty-first AAAI conference on artificial intelligence, pp. 1309–1315.Google Scholar
  14. Frayling, T. M. (2007). Genome-wide association studies provide new insights into type 2 diabetes aetiology. Nature Reviews Genetics, 8(9), 657.Google Scholar
  15. Goodfellow, I., Bengio, Y., Courville, A., & Bengio, Y. (2016). Deep learning (Vol. 1). Cambridge: MIT Press.zbMATHGoogle Scholar
  16. Guo, X., Gao, L., Liu, X., & Yin, J. (2017). Improved deep embedded clustering with local structure preservation. In Proceedings of the twenty-sixth international joint conference on artificial intelligence, pp. 1753–1759.Google Scholar
  17. Han, X., Shi, C., Wang, S., Philip, S. Y., & Song, L. (2018). Aspect-level deep collaborative filtering via heterogeneous information networks. In Proceedings of the twenty-seventh international joint conference on artificial intelligence, pp. 3393–3399.Google Scholar
  18. Hardoon, D. R., Szedmak, S., & Shawe-Taylor, J. (2004). Canonical correlation analysis: An overview with application to learning methods. Neural Computation, 16(12), 2639–2664.zbMATHGoogle Scholar
  19. Hinton, G. E., & Salakhutdinov, R. R. (2006). Reducing the dimensionality of data with neural networks. Science, 313(5786), 504–507.MathSciNetzbMATHGoogle Scholar
  20. Hotelling, H. (1936). Relations between two sets of variates. Biometrika, 28(3/4), 321–377.zbMATHGoogle Scholar
  21. Hu, Y., Zhang, D., Ye, J., Li, X., & He, X. (2013). Fast and accurate matrix completion via truncated nuclear norm regularization. IEEE Transactions on Pattern Analysis and Machine Intelligence, 35(9), 2117–2130.Google Scholar
  22. Jones, D. R. (2001). A taxonomy of global optimization methods based on response surfaces. Journal of Global Optimization, 21(4), 345–383.MathSciNetzbMATHGoogle Scholar
  23. Kingma, D. P., & Welling, M. (2014). Auto-encoding variational Bayes. In International conference on learning representations.Google Scholar
  24. Klami, A., Bouchard, G., & Tripathi, A. (2014). Group-sparse embeddings in collective matrix factorization. In International conference on learning representations.Google Scholar
  25. Knowles, J. (2006). ParEGO: A hybrid algorithm with on-line landscape approximation for expensive multiobjective optimization problems. IEEE Transactions on Evolutionary Computation, 10(1), 50–66.Google Scholar
  26. Kolker, E., et al. (2015). Finding text-supported gene-to-disease co-appearances with MOPED-Digger. Omics: A Journal of Integrative Biology, 19(12), 754–756.Google Scholar
  27. Koren, Y., Bell, R., & Volinsky, C. (2009). Matrix factorization techniques for recommender systems. Computer, 42(8), 30–37.Google Scholar
  28. Lan, C., Wang, J., & Huan, J. (2016). Towards a theoretical understanding of negative transfer in collective matrix factorization. In Proceedings of the thirty-second conference on uncertainty in artificial intelligence, pp. 367–376.Google Scholar
  29. Lee, I., Blom, U. M., Wang, P. I., Shim, J. E., & Marcotte, E. M. (2011). Prioritizing candidate disease genes by network-based boosting of genome-wide association data. Genome Research, 21(7), 1109–1121.Google Scholar
  30. Li, X., & She, J. (2017). Collaborative variational autoencoder for recommender systems. In Proceedings of the 23rd ACM SIGKDD international conference on knowledge discovery and data mining, pp. 305–314.Google Scholar
  31. Li, S., Kawale, J., & Fu, Y. (2015). Deep collaborative filtering via marginalized denoising auto-encoder. In Proceedings of the 24th ACM international conference on information and knowledge management, pp. 811–820.Google Scholar
  32. Liu, J., Wang, D., & Ding, Y. (2017). PHD: A probabilistic model of hybrid deep collaborative filtering for recommender systems. In Proceedings of the ninth Asian conference on machine learning, pp. 224–239.Google Scholar
  33. Loguercio, S., Good, B. M., & Su, A. I. (2013). Dizeez: An online game for human gene-disease annotation. PLoS ONE, 8(8), 71171.Google Scholar
  34. Natarajan, N., & Dhillon, I. S. (2014). Inductive matrix completion for predicting gene-disease associations. Bioinformatics, 30(12), 60–68.Google Scholar
  35. Ngiam, J., Khosla, A., Kim, M., Nam, J., Lee, H., & Ng, A. Y. (2011). Multimodal deep learning. In Proceedings of the 28th international conference on machine learning, pp. 689–696.Google Scholar
  36. Opap, K., & Mulder, N. (2017). Recent advances in predicting gene-disease associations. F1000Research 6.Google Scholar
  37. Pazzani, M., & Billsus, D. (1997). Learning and revising user profiles: The identification of interesting web sites. Machine Learning, 27(3), 313–331.Google Scholar
  38. Pers, T. H., et al. (2011). Meta-analysis of heterogeneous data sources for genome-scale identification of risk genes in complex phenotypes. Genetic Epidemiology, 35(5), 318–332.Google Scholar
  39. Piñero, J., et al. (2016). DisGeNET: A comprehensive platform integrating information on human disease-associated genes and variants. Nucleic Acids Research, 45(D1), 833–839.Google Scholar
  40. Piro, R. M., & Di Cunto, F. (2012). Computational approaches to disease-gene prediction: Rationale, classification and successes. The FEBS Journal, 279(5), 678–696.Google Scholar
  41. Schuyler, P. L., Hole, W. T., Tuttle, M. S., & Sherertz, D. D. (1993). The UMLS metathesaurus: Representing different views of biomedical concepts. Bulletin of the Medical Library Association, 81(2), 217.Google Scholar
  42. Seyyedrazzagi, E., & Navimipour, N. J. (2017). Disease genes prioritizing mechanisms: A comprehensive and systematic literature review. Network Modeling Analysis in Health Informatics and Bioinformatics, 6(1), 13.Google Scholar
  43. Shi, C., Hu, B., Zhao, W. X., & Philip, S. Y. (2019). Heterogeneous information network embedding for recommendation. IEEE Transactions on Knowledge and Data Engineering, 31(2), 357–370.Google Scholar
  44. Shi, C., & Philip, S. Y. (2017). Heterogeneous information network analysis and applications. Berlin: Springer.Google Scholar
  45. Singh, A. P., & Gordon, G. J. (2008). Relational learning via collective matrix factorization. In Proceedings of the 14th ACM SIGKDD international conference on knowledge discovery and data mining, pp. 650–658.Google Scholar
  46. Singh-Blom, U. M., et al. (2013). Prediction and validation of gene-disease associations using methods inspired by social network analyses. PLoS ONE, 8(5), 58977.Google Scholar
  47. Snoek, J., Larochelle, H., & Adams, R. P. (2012). Practical Bayesian optimization of machine learning algorithms. In Proceedings of the 25th international conference on neural information processing systems, pp. 2951–2959.Google Scholar
  48. Srebro, N., & Shraibman, A. (2005). Rank, trace-norm and max-norm. In International conference on computational learning theory, pp. 545–560.Google Scholar
  49. Swersky, K., Snoek, J., & Adams, R. P. (2013). Multi-task Bayesian optimization. In Proceedings of the 26th international conference on neural information processing systems, pp. 2004–2012.Google Scholar
  50. Vincent, P., et al. (2010). Stacked denoising autoencoders: Learning useful representations in a deep network with a local denoising criterion. Journal of Machine Learning Research, 11, 3371–3408.MathSciNetzbMATHGoogle Scholar
  51. Wang, W., Arora, R., Livescu, K., & Bilmes, J. (2015). On Deep Multi-view Representation Learning. In Proceedings of the 32nd international conference on machine learning, pp. 1083–1092.Google Scholar
  52. Wang, Q., Sun, M., Zhan, L., Thompson, P., Ji, S., & Zhou, J. (2017). Multi-modality disease modeling via collective deep matrix factorization. In Proceedings of the 23rd ACM SIGKDD international conference on knowledge discovery and data mining, pp. 1155–1164.Google Scholar
  53. Wang, H., Wang, N., & Yeung, D.-Y. (2015). Collaborative deep learning for recommender systems. In Proceedings of the 21th ACM SIGKDD international conference on knowledge discovery and data mining, pp. 1235–1244.Google Scholar
  54. Weinstein, J. N., et al. (2013). The Cancer Genome Atlas Pan-Cancer analysis project. Nature Genetics, 45(10), 1113–1120.Google Scholar
  55. Zeng, X., Liao, Y., Liu, Y., & Zou, Q. (2017). Prediction and validation of disease genes using HeteSim scores. IEEE/ACM Transactions on Computational Biology and Bioinformatics, 14(3), 687–695.Google Scholar
  56. Zhou, H., & Skolnick, J. (2016). A knowledge-based approach for predicting gene-disease associations. Bioinformatics, 32(18), 2831–2838.Google Scholar

Copyright information

© The Author(s), under exclusive licence to Springer Science+Business Media LLC, part of Springer Nature 2019

Authors and Affiliations

  1. 1.School of ComputingNational University of SingaporeSingaporeSingapore

Personalised recommendations