Abstract
We present a Bayesian non-negative tensor factorization model for count-valued tensor data, and develop scalable inference algorithms (both batch and online) for dealing with massive tensors. Our generative model can handle overdispersed counts as well as infer the rank of the decomposition. Moreover, leveraging a reparameterization of the Poisson distribution as a multinomial facilitates conjugacy in the model and enables simple and efficient Gibbs sampling and variational Bayes (VB) inference updates, with a computational cost that only depends on the number of nonzeros in the tensor. The model also provides a nice interpretability for the factors; in our model, each factor corresponds to a “topic”. We develop a set of online inference algorithms that allow further scaling up the model to massive tensors, for which batch inference methods may be infeasible. We apply our framework on diverse real-world applications, such as multiway topic modeling on a scientific publications database, analyzing a political science data set, and analyzing a massive household transactions data set.
Keywords
- Tensor factorization
- Bayesian learning
- Latent factor models
- Count data
- Online bayesian inference
Download conference paper PDF
References
Bazerque, J.A., Mateos, G., Giannakis, G.B.: Inference of poisson count processes using low-rank tensor data. In: ICASSP (2013)
Beutel, A., Kumar, A., Papalexakis, E.E., Talukdar, P.P., Faloutsos, C., Xing, E.P.: Flexifact: Scalable flexible factorization of coupled tensors on hadoop. In: SDM (2014)
Cappé, O., Moulines, E.: On-line expectation-maximization algorithm for latent data models. Journal of the Royal Statistical Society: Series B (Statistical Methodology) 71(3), 593–613 (2009)
Chi, E.C., Kolda, T.G.: On tensors, sparsity, and nonnegative factorizations. SIAM Journal on Matrix Analysis and Applications 33(4), 1272–1299 (2012)
Cichocki, A., Zdunek, R., Phan, A.H., Amari, S.: Nonnegative matrix and tensor factorizations: applications to exploratory multi-way data analysis and blind source separation. John Wiley & Sons (2009)
Dunson, D.B., Herring, A.H.: Bayesian latent variable models for mixed discrete outcomes. Biostatistics 6(1), 11–25 (2005)
Guhaniyogi, R., Qamar, S., Dunson, D.B.: Bayesian conditional density filtering. arXiv preprint arXiv:1401.3632 (2014)
Heinrich, G., Goesele, M.: Variational Bayes for Generic Topic Models. In: Mertsching, B., Hund, M., Aziz, Z. (eds.) KI 2009. LNCS, vol. 5803, pp. 161–168. Springer, Heidelberg (2009)
Hoffman, M.D., Blei, D.M., Wang, C., Paisley, J.: Stochastic variational inference. The Journal of Machine Learning Research 14(1), 1303–1347 (2013)
Inah, J., Papalexakis, E.E., Kang, U., Faloutsos, C.: Haten2: Billion-scale tensor decompositions. In: ICDE (2015)
Johndrow, J.E., Battacharya, A., Dunson, D.B.: Tensor decompositions and sparse log-linear models. arXiv preprint arXiv:1404.0396 (2014)
Jordan, M.I., Ghahramani, Z., Jaakkola, T.S., Saul, L.K.: An introduction to variational methods for graphical models. Machine Learning 37(2), 183–233 (1999)
Kang, U., Papalexakis, E., Harpale, A., Faloutsos, C.: Gigatensor: scaling tensor analysis up by 100 times-algorithms and discoveries. In: KDD (2012)
Kolda, T.G., Bader, B.W.: Tensor decompositions and applications. SIAM Review 51(3), 455–500 (2009)
Kozubowski, T.J., Podgórski, K.: Distributional properties of the negative binomial Lévy process. Centre for Mathematical Sciences, Faculty of Engineering, Lund University, Mathematical Statistics (2008)
Leetaru, K., Schrodt, P.A.: Gdelt: Global data on events, location, and tone, 1979–2012. ISA Annual Convention 2, 4 (2013)
Papalexakis, E., Faloutsos, C., Sidiropoulos, N.: Parcube: Sparse parallelizable candecomp-parafac tensor decompositions. ACM Transactions on Knowledge Discovery from Data (2015)
Rai, P., Wang, Y., Guo, S., Chen, G., Dunson, D., Carin, L.: Scalable bayesian low-rank decomposition of incomplete multiway tensors. In: ICML (2014)
Schein, A., Paisley, J., Blei, D.M., Wallach, H.: Inferring polyadic events with poisson tensor factorization. In: NIPS Workshop (2014)
Schmidt, M., Mohamed, S.: Probabilistic non-negative tensor factorisation using markov chain monte carlo. In: 17th European Signal Processing Conference (2009)
Shashua, A., Hazan, T.: Non-negative tensor factorization with applications to statistics and computer vision. In: ICML (2005)
Zhao, Q., Zhang, L., Cichocki, A.: Bayesian cp factorization of incomplete tensors with automatic rank determination
Zhou, G., Cichocki, A., Xie, S.: Fast nonnegative matrix/tensor factorization based on low-rank approximation. IEEE Transactions on Signal Processing 60(6), 2928–2940 (2012)
Zhou, M., Hannah, L.A., Dunson, D., Carin, L.: Beta-negative binomial process and poisson factor analysis. In: AISTATS (2012)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2015 Springer International Publishing Switzerland
About this paper
Cite this paper
Hu, C., Rai, P., Chen, C., Harding, M., Carin, L. (2015). Scalable Bayesian Non-negative Tensor Factorization for Massive Count Data. In: Appice, A., Rodrigues, P., Santos Costa, V., Gama, J., Jorge, A., Soares, C. (eds) Machine Learning and Knowledge Discovery in Databases. ECML PKDD 2015. Lecture Notes in Computer Science(), vol 9285. Springer, Cham. https://doi.org/10.1007/978-3-319-23525-7_4
Download citation
DOI: https://doi.org/10.1007/978-3-319-23525-7_4
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-23524-0
Online ISBN: 978-3-319-23525-7
eBook Packages: Computer ScienceComputer Science (R0)