Abstract
Anomaly detection in multidimensional data is a challenging task. Detecting anomalous mobility patterns in a city needs to take spatial, temporal, and traffic information into consideration. Although existing techniques are able to extract spatiotemporal features for anomaly analysis, few systematic analysis about how different factors contribute to or affect the anomalous patterns has been proposed. In this paper, we propose a novel technique to localize spatiotemporal anomalous events based on tensor decomposition. The proposed method employs a spatial-feature-temporal tensor model and analyzes latent mobility patterns through unsupervised learning. We first train the model based on historical data and then use the model to capture the anomalies, i.e., the mobility patterns that are significantly different from the normal patterns. The proposed technique is evaluated based on the yellow-cab dataset collected from New York City. The results show several interesting latent mobility patterns and traffic anomalies that can be deemed as anomalous events in the city, suggesting the effectiveness of the proposed anomaly detection method.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Bahadori MT, Yu QR, Liu Y (2014) Fast multivariate spatio-temporal analysis via low rank tensor learning. In: Advances in neural information processing systems, pp 3491–3499
Bai Y, Tezcan J, Cheng Q, Cheng J (2013) A multiway model for predicting earthquake ground motion. In: ACIS international conference on software engineering, artificial intelligence, networking and parallel/distributed computing (SNPD), pp 219–224
Breunig MM, Kriegel HP, Ng RT, Sander J (2000) Lof: identifying density-based local outliers. ACM Sigmod Rec 29:93–104
Chandola V, Banerjee A, Kumar V (2009) Anomaly detection: a survey. ACM Comput Surv (CSUR) 41(3):15
Chen Y, Zhou XS, Huang TS (2001) One-class svm for learning in image retrieval. IEEE Image Process 1:34–37
Cichocki A, Zdunek R, Phan AH, Amari SI (2009) Nonnegative matrix and tensor factorizations: applications to exploratory multi-way data analysis and blind source separation. Wiley, New York
Fanaee-T H, Gama J (2015) Eigenevent: an algorithm for event detection from complex data streams in syndromic surveillance. Intell Data Anal 19(3):597–616
Fanaee-T H, Gama J (2016a) Tensor-based anomaly detection: an interdisciplinary survey. Knowl Based Syst 98:130–147
Fanaee-T H, Gama J (2016b) Event detection from traffic tensors: a hybrid model. Neurocomputing 203:22–33
Fanaee-T H, Gama J (2014) An eigenvector-based hotspot detection. arXiv preprint arXiv:1406.3191
Gauvin L, Panisson A, Cattuto C (2014) Detecting the community structure and activity patterns of temporal networks: a non-negative tensor factorization approach. PloS ONE 9(1):e86028
Jiang M, Cui P, Faloutsos C (2016) Suspicious behavior detection: current trends and future directions. IEEE Intell Syst 31(1):31–39
Kim J, He Y, Park H (2014) Algorithms for nonnegative matrix and tensor factorizations: a unified view based on block coordinate descent framework. J Global Optim 58(2):285–319
Kolda TG, Bader BW (2009) Tensor decompositions and applications. SIAM Rev 51(3):455–500
Kotsia I, Guo W, Patras I (2012) Higher rank support tensor machines for visual recognition. Pattern Recogn 45(12):4192–4203
Liu S, Cui W, Wu Y, Liu M (2014) A survey on information visualization: recent advances and challenges. Visual Comput 30(12):1373–1393
Liu D, Weng D, Li Y, Bao J, Zheng Y, Qu H, Wu Y (2017) SmartAdP: Visual analytics of large-scale taxi trajectories for selecting billboard locations. IEEE Trans. Vis. Comput. Graphics 23(1):1–10
Liu Y, Zhou B, Chen F, Cheung DW (2016) Graph topic scan statistic for spatial event detection. In: Proceedings of the 25th ACM International on Conference on Information and Knowledge Management. ACM, pp 489–498
Mao HH, Wu CJ, Papalexakis EE, Faloutsos C, Lee KC, Kao TC (2014) Malspot: Multi2 malicious network behavior patterns analysis. In: Pacific-Asia conference on knowledge discovery and data mining. Springer, pp 1–14
Matsubara Y, Sakurai Y, Faloutsos C, Iwata T, Yoshikawa M (2012) Fast mining and forecasting of complex time-stamped events. In: Proceedings of the ACM SIGKDD international conference on Knowledge discovery and data mining, pp 271–279
Nomikos P, MacGregor JF (1994) Monitoring batch processes using multiway principal component analysis. AIChE J 40(8):1361–1375
Papalexakis EE, Faloutsos C, Sidiropoulos ND (2012) Parcube: sparse parallelizable tensor decompositions. In: Joint European conference on machine learning and knowledge discovery in databases. Springer, pp 521–536
Papalexakis E, Pelechrinis K, Faloutsos C (2014) Spotting misbehaviors in location-based social networks using tensors. In: Proceedings of the international conference on world wide web. ACM, pp 551–552
Paschalidis IC, Smaragdakis G (2009) Spatio-temporal network anomaly detection by assessing deviations of empirical measures. IEEE/ACM Trans Netw (TON) 17(3):685–697
Prada MA, Dominguez M, Barrientos P, Garcia S (2012a) Dimensionality reduction for damage detection in engineering structures. Int J Mod Phys B 26(25):1246004
Prada MA, Toivola J, Kullaa J, HollméN J (2012b) Three-way analysis of structural health monitoring data. Neurocomputing 80:119–128
Rendle S (2012) Factorization machines with libfm. ACM Trans Intell Syst Technol (TIST) 3(3):57
Rozenshtein P, Anagnostopoulos A, Gionis A, Tatti N (2014) Event detection in activity networks. In: Proceedings of the 20th ACM SIGKDD international conference on Knowledge discovery and data mining. ACM, pp 1176–1185
Shi L, Gangopadhyay A, Janeja VP (2015) Stensr: spatio-temporal tensor streams for anomaly detection and pattern discovery. Knowl Inf Syst 43(2):333
Sun GD, Liang R, Qu H, Wu Y (2017a) Embedding spatiotemporal information into maps by route-zooming. IEEE Trans. Vis. Comput. Graphics 23(5):1506–1519
Sun G, Tang T, Peng TQ, Liang R, Wu Y (2017b) Socialwave: visual analysis of spatio-temporal diffusion of information on social media. ACM Trans Intell Syst Technol 9(2):15
Sun J, Tao D, Papadimitriou S, Yu PS, Faloutsos C (2008) Incremental tensor analysis: Theory and applications. ACM Trans Knowl Discov Data (TKDD) 2(3):11
Sun J, Tao D, Faloutsos C (2006) Beyond streams and graphs: dynamic tensor analysis. In: Proceedings of the 12th ACM SIGKDD international conference on Knowledge discovery and data mining. ACM, pp 374–383
Sun G, Wu YC, Liang RH, Liu SX (2013) A survey of visual analytics techniques and applications: state-of-the-art research and future challenges. J Comput Sci Tech 28(5):852–867
Tao D, Li X, Hu W, Maybank S, Wu X (2005) Supervised tensor learning. In: IEEE international conference on data mining
Thai-Nghe N, Horváth T, Schmidt-Thieme L (2010) Factorization models for forecasting student performance. In: Educational Data Mining 2011
Tian X, Zhang X, Deng X, Chen S (2009) Multiway kernel independent component analysis based on feature samples for batch process monitoring. Neurocomputing 72(7):1584–1596
Tork HF, Oliveira M, Gama J, Malinowski S, Morla R (2012) Event and anomaly detection using tucker3 decomposition. In: Workshop on ubiquitous data mining, p 8
Wang XR, Lizier JT, Obst O, Prokopenko M, Wang P (2008) Spatiotemporal anomaly detection in gas monitoring sensor networks. In: Wireless sensor networks: 5th European conference, EWSN 2008. Springer, pp 90–105
Wang J, Gao F, Cui P, Li C, Xiong Z (2014) Discovering urban spatio-temporal structure from time-evolving traffic networks. In: Asia-Pacific web conference. Springer, pp 93–104
Weng D, Zhu H, Bao J, Zheng Y, Wu Y (2018) Homefinder revisited: finding ideal homes with reachability centric multi-criteria decision making. In Proceedings of ACM CHI
Wu Y, Lan J, Shu X, Ji C, Zhao K, Wang J, Zhang H (2018) ITTVIS: Interactive visualization of table tennis data. IEEE Trans Visualization and Comp Graphics 24(1):709–718
Xia J, Chen W, Hou Y, Hu W, Huang X, Ebertk DS (2016) DimScanner: A relation-based visual exploration approach towards data dimension inspection. In: IEEE conference on visual analytics science and technology (VAST). pp 81–90
Xie C, Chen W, Huang X, Hu Y, Barlowe S, Yang J (2014) VAET: A visual analytics approach for e-transactions time-series. IEEE Trans. Vis. Comput. Graphics 20(12):1743–1752
Xu P, Mei H, Ren L, Chen W (2017) ViDX: Visual diagnostics of assembly line performance in smart factories. IEEE Trans. Vis. Comput. Graphics 23(1):291–300
Young WC, Blumenstock JE, Fox EB, McCormick TH (2014) Detecting and classifying anomalous behavior in spatiotemporal network data. In: Proceedings of KDD workshop on learning about emergencies from social information (KDD-LESI 2014), pp 29–33
Yuan J, Zheng Y, Xie X (2012) Discovering regions of different functions in a city using human mobility and pois. In: Proceedings of the ACM SIGKDD international conference on Knowledge discovery and data mining, pp 186–194
Zhang T, Wang X, Li Z, Guo F, Ma Y, Chen W (2017) A survey of network anomaly visualization. Sc China Infor Sci 60(12):121101
Zhao Z, Cheng Z, Hong L, Chi EH (2015) Improving user topic interest profiles by behavior factorization. In: Proceedings of the international conference on world wide web. ACM, pp 1406–1416
Zheng Y, Liu T, Wang Y, Zhu Y, Liu Y, Chang E (2014) Diagnosing New York city’s noises with ubiquitous data. In: Proceedings of the ACM international joint conference on pervasive and ubiquitous computing, pp 715–725
Author information
Authors and Affiliations
Corresponding author
Additional information
Responsible editor: Jieping Ye.
Rights and permissions
About this article
Cite this article
Lin, C., Zhu, Q., Guo, S. et al. Anomaly detection in spatiotemporal data via regularized non-negative tensor analysis. Data Min Knowl Disc 32, 1056–1073 (2018). https://doi.org/10.1007/s10618-018-0560-3
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10618-018-0560-3