Abstract
We introduce a Bayesian extension of the tensor factorization problem to multiple coupled tensors. For a single tensor it reduces to standard PARAFAC-type Bayesian factorization, and for two tensors it is the first Bayesian Tensor Canonical Correlation Analysis method. It can also be seen to solve a tensorial extension of the recent Group Factor Analysis problem. The method decomposes the set of tensors to factors shared by subsets of the tensors, and factors private to individual tensors, and does not assume orthogonality. For a single tensor, the method empirically outperforms existing methods, and we demonstrate its performance on multiple tensor factorization tasks in toxicogenomics and functional neuroimaging.
Chapter PDF
Similar content being viewed by others
Keywords
- Canonical Correlation Analysis
- Link Prediction
- Group Factor Analysis
- Core Tensor
- Automatic Relevance Determination
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.
References
Kolda, T., Bader, B.: Tensor decompositions and applications. SIAM Review 51(3), 455–500 (2009)
Carroll, J.D., Chang, J.J.: Analysis of individual differences in multidimensional scaling via an n-way generalization of Eckart-Young decomposition. Psychometrika 35(3), 283–319 (1970)
Harshman, R.A.: Foundations of the parafac procedure: models and conditions for an explanatory multimodal factor analysis. UCLA Working Papers in Phonetics 16, 1–84 (1970)
Tucker, L.R.: Some mathematical notes on three-mode factor analysis. Psychometrika 31(3), 279–311 (1966)
Xiong, L., Chen, X., Huang, T.K., Schneider, J., Carbonell, J.G.: Temporal collaborative filtering with bayesian probabilistic tensor factorization. In: Proceedings of SIAM Data Mining, vol. 10, pp. 211–222 (2010)
Chu, W., Ghahramani, Z.: Probabilistic models for incomplete multi-dimensional arrays. In: Proceedings of AISTATS. JMLR W&CP, vol. 5, pp. 89–96 (2009)
Lee, S.H., Choi, S.: Two-dimensional canonical correlation analysis. IEEE Sig. Proc. Letters 14(10), 735–738 (2007)
Kim, T.K., Cipolla, R.: Canonical correlation analysis of video volume tensors for action categorization and detection. IEEE Transactions on Pattern Analysis and Machine Intelligence 31(8), 1415–1428 (2009)
Yan, J., Zheng, W., Zhou, X., Zhao, Z.: Sparse 2-d canonical correlation analysis via low rank matrix approximation for feature extraction. IEEE Sig. Proc. Letters 19(1), 51–54 (2012)
Lu, H.: Learning canonical correlations of paired tensor sets via tensor-to-vector projection. In: Proceedings of IJCAI, pp. 1516–1522 (2013)
Hardoon, D.R., Szedmak, S., Shawe-Taylor, J.: Canonical correlation analysis: An overview with application to learning methods. Neural Computation 16(12), 2639–2664 (2004)
Klami, A., Virtanen, S., Kaski, S.: Bayesian canonical correlation analysis. Journal of Machine Learning Research 14, 965–1003 (2013)
Virtanen, S., Klami, A., Khan, S.A., Kaski, S.: Bayesian group factor analysis. In: Proceedings of AISTATS. JMLR W&CP, vol. 22, pp. 1269–1277 (2012)
Cattell, R.B.: Parallel proportional profiles and other principles for determining the choice of factors by rotation. Psychometrika 9(4), 267–283 (1944)
Kruskal, J.B.: Three-way arrays: rank and uniqueness of trilinear decompositions, with application to arithmetic complexity and statistics. Linear Algebra and its Applications 18(2), 95–138 (1977)
Neal, R.M.: Bayesian learning for neural networks. Springer (1996)
Mitchell, T.J., Beauchamp, J.J.: Bayesian variable selection in linear regression. Journal of the American Statistical Association 83(404), 1023–1032 (1988)
Krijnen, W., Dijkstra, T., Stegeman, A.: On the non-existence of optimal solutions and the occurrence of degeneracy in the candecomp/parafac model. Psychometrika 73(3), 431–439 (2008)
Srensen, M., Lathauwer, L., Comon, P., Icart, S., Deneire, L.: Canonical polyadic decomposition with a columnwise orthonormal factor matrix. SIAM Journal on Matrix Analysis and Applications 33(4), 1190–1213 (2012)
Mørup, M., Hansen, L.K.: Automatic relevance determination for multiway models. Journal of Chemometrics 23(7-8), 352–363 (2009)
Hoff, P.D.: Hierarchical multilinear models for multiway data. Computational Statistics & Data Analysis 55(1), 530–543 (2011)
Hayashi, K., Takenouchi, T., Shibata, T., Kamiya, Y., Kato, D., Kunieda, K., Yamada, K., Ikeda, K.: Exponential family tensor factorization: an online extension and applications. Knowledge and Information Systems 33(1), 57–88 (2012)
Xu, Z., Yan, F., Qi, A.: Infinite tucker decomposition: Nonparametric bayesian models for multiway data analysis. In: Proceedings of ICML, pp. 1023–1030 (2012)
Acar, E., Rasmussen, M.A., Savorani, F., Naes, T., Bro, R.: Understanding data fusion within the framework of coupled matrix and tensor factorizations. Chemometrics and Intelligent Laboratory Systems 129, 53–63 (2013)
Ermis, B., Acar, E., Cemgil, A.T.: Link prediction in heterogeneous data via generalized coupled tensor factorization. Data Mining and Knowledge Discovery, 1–34 (2013)
Yilmaz, K.Y., Cemgil, A.T., Simsekli, U.: Generalised coupled tensor factorisation. In: Proceedings of NIPS, pp. 2151–2159 (2011)
Andersson, C.A., Bro, R.: The N-way toolbox for MATLAB. Chemometrics and Intelligent Laboratory Systems 52(1), 1–4 (2000)
Lundy, M.E., Harshman, R.A., Kruskal, J.B.: A two-stage procedure incorporating good features of both trilinear and quadrilinear models. Multiway Data Analysis, 123–130 (1989)
Hartung, T., Vliet, E.V., Jaworska, J., Bonilla, L., Skinner, N., Thomas, R.: Food for thought.. systems toxicology. ALTEX 29(2), 119–128 (2012)
Lamb, J., et al.: The connectivity map: Using gene-expression signatures to connect small molecules, genes, and disease. Science 313(5795), 1929–1935 (2006)
Shoemaker, R.H.: The nci60 human tumour cell line anticancer drug screen. Nature Reviews Cancer 6(10), 813–823 (2006)
Kamal, A., et al.: A high-affinity conformation of HSP90 confers tumour selectivity on HSP90 inhibitors. Nature 425(6956), 407–410 (2003)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2014 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Khan, S.A., Kaski, S. (2014). Bayesian Multi-view Tensor Factorization. In: Calders, T., Esposito, F., Hüllermeier, E., Meo, R. (eds) Machine Learning and Knowledge Discovery in Databases. ECML PKDD 2014. Lecture Notes in Computer Science(), vol 8724. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-662-44848-9_42
Download citation
DOI: https://doi.org/10.1007/978-3-662-44848-9_42
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-662-44847-2
Online ISBN: 978-3-662-44848-9
eBook Packages: Computer ScienceComputer Science (R0)