Structurally Regularized Non-negative Tensor Factorization for Spatio-Temporal Pattern Discoveries

Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 10534)


Understanding spatio-temporal activities in a city is a typical problem of spatio-temporal data analysis. For this analysis, tensor factorization methods have been widely applied for extracting a few essential patterns into latent factors. Non-negative Tensor Factorization (NTF) is popular because of its capability of learning interpretable factors from non-negative data, simple computation procedures, and dealing with missing observation. However, since existing NTF methods are not fully aware of spatial and temporal dependencies, they often fall short of learning latent factors where a large portion of missing observation exist in data. In this paper, we present a novel NTF method for extracting smooth and flat latent factors by leveraging various kinds of spatial and temporal structures. Our method incorporates a unified structured regularizer into NTF that can represent various kinds of auxiliary information, such as an order of timestamps, a daily and weekly periodicity, distances between sensor locations, and areas of locations. For the estimation of the factors for our model, we present a simple and efficient optimization procedure based on the alternating direction method of multipliers. In missing value interpolation experiments of traffic flow data and bike-sharing system data, we demonstrate that our proposed method improved interpolation performances from existing NTF, especially when a large portion of missing values exists.



The part of this work was supported by JSPS KAKENHI Grant Numbers JP16H01548 and JP26280086, and NICT “Research and Development on Fundamental and Utilization Technologies for Social Big Data”.


  1. 1.
    Ali, M.I., Gao, F., Mileo, A.: CityBench: a configurable benchmark to evaluate RSP engines using smart city datasets. In: Arenas, M., Corcho, O., Simperl, E., Strohmaier, M., d’Aquin, M., Srinivas, K., Groth, P., Dumontier, M., Heflin, J., Thirunarayan, K., Staab, S. (eds.) ISWC 2015. LNCS, vol. 9367, pp. 374–389. Springer, Cham (2015). CrossRefGoogle Scholar
  2. 2.
    Bach, F.R.: Structured sparsity-inducing norms through submodular functions. In: Proceedings of NIPS, pp. 118–126 (2010)Google Scholar
  3. 3.
    Barbero, A., Sra, S.: Fast Newton-type methods for total variation regularization. In: Proceedings of ICML, pp. 313–320 (2011)Google Scholar
  4. 4.
    Boyd, S., Parikh, N., Chu, E., Peleato, B., Eckstein, J.: Distributed optimization and statistical learning via the alternating direction method of multipliers. Found. Trends Mach. Learn. 3(1), 1–122 (2011)CrossRefzbMATHGoogle Scholar
  5. 5.
    Boykov, Y., Kolmogorov, V.: An experimental comparison of min-cut/max-flow algorithms for energy minimization in vision. IEEE Trans. Pattern Anal. Mach. Intell. 26(9), 1124–1137 (2004)CrossRefzbMATHGoogle Scholar
  6. 6.
    Cai, D., He, X., Han, J., Huang, T.S.: Graph regularized nonnegative matrix factorization for data representation. IEEE Trans. Pattern Anal. Mach. Intell. 33(8), 1548–1560 (2011)CrossRefGoogle Scholar
  7. 7.
    Chambolle, A., Darbon, J.: On total variation minimization and surface evolution using parametric maximum flows. Int. J. Comput. Vis. 84(3), 288 (2009)CrossRefzbMATHGoogle Scholar
  8. 8.
    Cichocki, A., Zdunek, R., Phan, A.H., Amari, S.-I.: Nonnegative Matrix and Tensor Factorizations: Applications to Exploratory Multi-way Data Analysis and Blind Source Separation. Wiley, Hoboken (2009)CrossRefGoogle Scholar
  9. 9.
    Dhillon, I.S., Sra, S.: Generalized nonnegative matrix approximations with Bregman divergences. In: Proceedings of NIPS, vol. 18 (2005)Google Scholar
  10. 10.
    Fan, Z., Song, X., Shibasaki, R.: CitySpectrum: a non-negative tensor factorization approach. In: Proceedings of UbiComp, pp. 213–223 (2014)Google Scholar
  11. 11.
    Fujishige, S.: Submodular Functions and Optimization, vol. 58. Elsevier, Amsterdam (2005)zbMATHGoogle Scholar
  12. 12.
    Gallo, G., Grigoriadis, M.D., Tarjan, R.E.: A fast parametric maximum flow algorithm and applications. SIAM J. Comput. 18(1), 30–55 (1989)MathSciNetCrossRefzbMATHGoogle Scholar
  13. 13.
    Gillis, N.: The why and how of nonnegative matrix factorization. In: Regularization, Optimization, Kernels, and Support Vector Machines, pp. 257–291. Chapman and Hall/CRC (2014)Google Scholar
  14. 14.
    Han, Y., Moutarde, F.: Analysis of large-scale traffic dynamics in an urban transportation network using non-negative tensor factorization. Int. J. Intell. Transp. Syst. Res. 14(1), 36–49 (2016)Google Scholar
  15. 15.
    Huang, K., Sidiropoulos, N.D., Liavas, A.P.: A flexible and efficient algorithmic framework for constrained matrix and tensor factorization. IEEE Trans. Sig. Process. 64(19), 5052–5065 (2016)MathSciNetCrossRefGoogle Scholar
  16. 16.
    Kimura, T., Ishibashi, K., Mori, T., Sawada, H., Toyono, T., Nishimatsu, K., Watanabe, A., Shimoda, A., Shiomoto, K.: Spatio-temporal factorization of log data for understanding network events. In: Proceedings of INFOCOM, pp. 610–618 (2014)Google Scholar
  17. 17.
    Kolda, T.G., Bader, B.W.: Tensor decompositions and applications. SIAM Rev. 51(3), 455–500 (2009)MathSciNetCrossRefzbMATHGoogle Scholar
  18. 18.
    Lee, D.D., Seung, H.S.: Learning the parts of objects by non-negative matrix factorization. Nature 401(6755), 788–791 (1999)CrossRefzbMATHGoogle Scholar
  19. 19.
    Lovász, L.: Submodular functions and convexity. In: Bachem, A., Korte, B., Grötschel, M. (eds.) Mathematical Programming The State of the Art, pp. 235–257. Springer, Heidelberg (1983). CrossRefGoogle Scholar
  20. 20.
    Nagano, K., Kawahara, Y., Aihara, K.: Size-constrained submodular minimization through minimum norm base. In: Proceedings of ICML, pp. 977–984 (2011)Google Scholar
  21. 21.
    Rao, N., Yu, H.-F., Ravikumar, P.K., Dhillon, I.S.: Collaborative filtering with graph information: consistency and scalable methods. In: Proceedings of NIPS, pp. 2107–2115 (2015)Google Scholar
  22. 22.
    Smola, A.J., Kondor, R.: Kernels and regularization on graphs. In: Schölkopf, B., Warmuth, M.K. (eds.) COLT-Kernel 2003. LNCS (LNAI), vol. 2777, pp. 144–158. Springer, Heidelberg (2003). CrossRefGoogle Scholar
  23. 23.
    Sun, L., Axhausen, K.W.: Understanding urban mobility patterns with a probabilistic tensor factorization framework. Transp. Res. Part B: Methodol. 91, 511–524 (2016)CrossRefGoogle Scholar
  24. 24.
    Takeuchi, K., Kawahara, Y., Iwata, T.: Higher order fused regularization for supervised learning with grouped parameters. In: Appice, A., Rodrigues, P.P., Santos Costa, V., Soares, C., Gama, J., Jorge, A. (eds.) ECML PKDD 2015. LNCS (LNAI), vol. 9284, pp. 577–593. Springer, Cham (2015). CrossRefGoogle Scholar
  25. 25.
    Takeuchi, K., Tomioka, R., Ishiguro, K., Kimura, A., Sawada, H.: Non-negative multiple tensor factorization. In: Proceedings of ICDM, pp. 1199–1204 (2013)Google Scholar
  26. 26.
    Tandon, R., Sra, S.: Sparse nonnegative matrix approximation: new formulations and algorithms. Rapp. Tech. 193, 38–42 (2010)Google Scholar
  27. 27.
    Tibshirani, R., Saunders, M., Rosset, S., Zhu, J., Knight, K.: Sparsity and smoothness via the fused lasso. J. R. Stat. Soc. Ser. B (Stat. Methodol.) 67(1), 91–108 (2005)MathSciNetCrossRefzbMATHGoogle Scholar
  28. 28.
    Wang, Y.-X., Sharpnack, J., Smola, A., Tibshirani, R.J.: Trend filtering on graphs. J. Mach. Learn. Res. 17(105), 1–41 (2016)MathSciNetzbMATHGoogle Scholar
  29. 29.
    Witten, D.M., Tibshirani, R., Hastie, T.: A penalized matrix decomposition with applications to sparse principal components and canonical correlation analysis. Biostatistics 10, 515–534 (2009)CrossRefGoogle Scholar
  30. 30.
    Xin, B., Kawahara, Y., Wang, Y., Gao, W.: Efficient generalized fused lasso with its application to the diagnosis of Alzheimer’s disease. In: Proceedings of AAAI, pp. 2163–2169 (2014)Google Scholar
  31. 31.
    Yangyang, X., Yin, W.: A block coordinate descent method for regularized multiconvex optimization with applications to nonnegative tensor factorization and completion. SIAM J. Imaging Sci. 6(3), 1758–1789 (2013)MathSciNetCrossRefzbMATHGoogle Scholar
  32. 32.
    Yang, D., Zhang, D., Zheng, V.W., Yu, Z.: Modeling user activity preference by leveraging user spatial temporal characteristics in LBSNs. IEEE Trans. Syst. Man Cybern.: Syst. 45(1), 129–142 (2015)CrossRefGoogle Scholar

Copyright information

© Springer International Publishing AG 2017

Authors and Affiliations

  1. 1.NTT Communication Science LaboratoriesKyotoJapan
  2. 2.The Institute of Scientific and Industrial Research (ISIR)Osaka UniversityOsakaJapan
  3. 3.Department of Intelligence Science and TechnologyKyoto UniversityKyotoJapan
  4. 4.Center for Advanced Intelligence ProjectRIKENTokyoJapan

Personalised recommendations