Scalable Bayesian Non-negative Tensor Factorization for Massive Count Data

  • Changwei Hu
  • Piyush RaiEmail author
  • Changyou Chen
  • Matthew Harding
  • Lawrence Carin
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 9285)


We present a Bayesian non-negative tensor factorization model for count-valued tensor data, and develop scalable inference algorithms (both batch and online) for dealing with massive tensors. Our generative model can handle overdispersed counts as well as infer the rank of the decomposition. Moreover, leveraging a reparameterization of the Poisson distribution as a multinomial facilitates conjugacy in the model and enables simple and efficient Gibbs sampling and variational Bayes (VB) inference updates, with a computational cost that only depends on the number of nonzeros in the tensor. The model also provides a nice interpretability for the factors; in our model, each factor corresponds to a “topic”. We develop a set of online inference algorithms that allow further scaling up the model to massive tensors, for which batch inference methods may be infeasible. We apply our framework on diverse real-world applications, such as multiway topic modeling on a scientific publications database, analyzing a political science data set, and analyzing a massive household transactions data set.


Tensor factorization Bayesian learning Latent factor models Count data Online bayesian inference 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Bazerque, J.A., Mateos, G., Giannakis, G.B.: Inference of poisson count processes using low-rank tensor data. In: ICASSP (2013)Google Scholar
  2. 2.
    Beutel, A., Kumar, A., Papalexakis, E.E., Talukdar, P.P., Faloutsos, C., Xing, E.P.: Flexifact: Scalable flexible factorization of coupled tensors on hadoop. In: SDM (2014)Google Scholar
  3. 3.
    Cappé, O., Moulines, E.: On-line expectation-maximization algorithm for latent data models. Journal of the Royal Statistical Society: Series B (Statistical Methodology) 71(3), 593–613 (2009)MathSciNetCrossRefzbMATHGoogle Scholar
  4. 4.
    Chi, E.C., Kolda, T.G.: On tensors, sparsity, and nonnegative factorizations. SIAM Journal on Matrix Analysis and Applications 33(4), 1272–1299 (2012)MathSciNetCrossRefzbMATHGoogle Scholar
  5. 5.
    Cichocki, A., Zdunek, R., Phan, A.H., Amari, S.: Nonnegative matrix and tensor factorizations: applications to exploratory multi-way data analysis and blind source separation. John Wiley & Sons (2009)Google Scholar
  6. 6.
    Dunson, D.B., Herring, A.H.: Bayesian latent variable models for mixed discrete outcomes. Biostatistics 6(1), 11–25 (2005)CrossRefzbMATHGoogle Scholar
  7. 7.
    Guhaniyogi, R., Qamar, S., Dunson, D.B.: Bayesian conditional density filtering. arXiv preprint arXiv:1401.3632 (2014)
  8. 8.
    Heinrich, G., Goesele, M.: Variational Bayes for Generic Topic Models. In: Mertsching, B., Hund, M., Aziz, Z. (eds.) KI 2009. LNCS, vol. 5803, pp. 161–168. Springer, Heidelberg (2009) CrossRefGoogle Scholar
  9. 9.
    Hoffman, M.D., Blei, D.M., Wang, C., Paisley, J.: Stochastic variational inference. The Journal of Machine Learning Research 14(1), 1303–1347 (2013)MathSciNetzbMATHGoogle Scholar
  10. 10.
    Inah, J., Papalexakis, E.E., Kang, U., Faloutsos, C.: Haten2: Billion-scale tensor decompositions. In: ICDE (2015)Google Scholar
  11. 11.
    Johndrow, J.E., Battacharya, A., Dunson, D.B.: Tensor decompositions and sparse log-linear models. arXiv preprint arXiv:1404.0396 (2014)
  12. 12.
    Jordan, M.I., Ghahramani, Z., Jaakkola, T.S., Saul, L.K.: An introduction to variational methods for graphical models. Machine Learning 37(2), 183–233 (1999)CrossRefGoogle Scholar
  13. 13.
    Kang, U., Papalexakis, E., Harpale, A., Faloutsos, C.: Gigatensor: scaling tensor analysis up by 100 times-algorithms and discoveries. In: KDD (2012)Google Scholar
  14. 14.
    Kolda, T.G., Bader, B.W.: Tensor decompositions and applications. SIAM Review 51(3), 455–500 (2009)MathSciNetCrossRefGoogle Scholar
  15. 15.
    Kozubowski, T.J., Podgórski, K.: Distributional properties of the negative binomial Lévy process. Centre for Mathematical Sciences, Faculty of Engineering, Lund University, Mathematical Statistics (2008)Google Scholar
  16. 16.
    Leetaru, K., Schrodt, P.A.: Gdelt: Global data on events, location, and tone, 1979–2012. ISA Annual Convention 2, 4 (2013)Google Scholar
  17. 17.
    Papalexakis, E., Faloutsos, C., Sidiropoulos, N.: Parcube: Sparse parallelizable candecomp-parafac tensor decompositions. ACM Transactions on Knowledge Discovery from Data (2015)Google Scholar
  18. 18.
    Rai, P., Wang, Y., Guo, S., Chen, G., Dunson, D., Carin, L.: Scalable bayesian low-rank decomposition of incomplete multiway tensors. In: ICML (2014)Google Scholar
  19. 19.
    Schein, A., Paisley, J., Blei, D.M., Wallach, H.: Inferring polyadic events with poisson tensor factorization. In: NIPS Workshop (2014)Google Scholar
  20. 20.
    Schmidt, M., Mohamed, S.: Probabilistic non-negative tensor factorisation using markov chain monte carlo. In: 17th European Signal Processing Conference (2009)Google Scholar
  21. 21.
    Shashua, A., Hazan, T.: Non-negative tensor factorization with applications to statistics and computer vision. In: ICML (2005)Google Scholar
  22. 22.
    Zhao, Q., Zhang, L., Cichocki, A.: Bayesian cp factorization of incomplete tensors with automatic rank determinationGoogle Scholar
  23. 23.
    Zhou, G., Cichocki, A., Xie, S.: Fast nonnegative matrix/tensor factorization based on low-rank approximation. IEEE Transactions on Signal Processing 60(6), 2928–2940 (2012)Google Scholar
  24. 24.
    Zhou, M., Hannah, L.A., Dunson, D., Carin, L.: Beta-negative binomial process and poisson factor analysis. In: AISTATS (2012)Google Scholar

Copyright information

© Springer International Publishing Switzerland 2015

Authors and Affiliations

  • Changwei Hu
    • 1
  • Piyush Rai
    • 1
    Email author
  • Changyou Chen
    • 1
  • Matthew Harding
    • 2
  • Lawrence Carin
    • 1
  1. 1.Department of Electrical and Computer EngineeringDuke UniversityDurhamUSA
  2. 2.Sanford School of Public Policy and Department of EconomicsDuke UniversityDurhamUSA

Personalised recommendations