Data Mining and Knowledge Discovery

, Volume 29, Issue 1, pp 137–167 | Cite as

Sequential network change detection with its applications to ad impact relation analysis

  • Yu Hayashi
  • Kenji Yamanishi


We are concerned with the issue of tracking changes of variable dependencies from multivariate time series. Conventionally, this issue has been addressed in the batch scenario where the whole data set is given at once, and the change detection must be done in a retrospective way. This paper addresses this issue in a sequential scenario where multivariate data are sequentially input and the detection must be done in a sequential fashion. We propose a new method for sequential tracking of variable dependencies. In it we employ a Bayesian network as a representation of variable dependencies. The key ideas of our method are: (1) we extend the theory of dynamic model selection, which has been developed in the batch-learning scenario, into the sequential setting, and apply it to our issue, (2) we conduct the change detection sequentially using dynamic programming per a window where we employ the Hoeffding’s bound to automatically determine the window size. We empirically demonstrate that our proposed method is able to perform change detection more efficiently than a conventional batch method. Further, we give a new framework of an application of variable dependency change detection, which we call Ad Impact Relation analysis (AIR). In it, we detect the time point when a commercial message advertisement has given an impact on the market and effectively visualize the impact through network changes. We employ real data sets to demonstrate the validity of AIR.


Network change detection Minimum description length principle Dynamic model selection Bayesian network Information theory Marketing 



This work was partially supported by MEXT KAKENHI 23240019, Aihara Project, the FIRST program from JSPS, initiated by CSTP, Hakuhodo Corporation, NTT Corporation, and Microsoft Corporation (CORE6 Project). Specifically we thank HAKUHODO Inc. for providing us data sets for AIR and valuable comments.


  1. Akaike H (1974) A new look at the statistical model identification. IEEE Trans Autom Control 19(6):716–723CrossRefzbMATHMathSciNetGoogle Scholar
  2. Cheng J, Bell DA, Liu W (1997) An algorithm for bayesian belief network construction from data. In: Proceedings of international workshop on artificial intelligence and statistics, pp 83–90Google Scholar
  3. Fearnhead P, Liu Z (2007) Online inference for multiple changepoint problems. J R Stat Soc B 69(4):589–605CrossRefMathSciNetGoogle Scholar
  4. Friedman J, Hastie T, Tibshirani R (2008) Sparse inverse covariance estimation with the graphical lasso. Biostatistics 9(3):432–441CrossRefzbMATHGoogle Scholar
  5. Geiger D, Heckerman D (1994) Learning Gaussian networks. Technical report, Microsoft research, mSR-TR-94-10Google Scholar
  6. Grünwald PD (2007) The minimum description length principle. MIT Press, CambridgeGoogle Scholar
  7. Guo F, Hanneke S, Fu W, Xing EP (2007) Recovering temporally rewiring networks: a model-based approach. In: Proceedings of the 24th international conference on machine learning, pp 321–328Google Scholar
  8. Hayashi Y, Yamanishi K (2012) Sequential network change detection with its applications to ad impact relation analysis. In: Proceedings of the 12th IEEE international conference on data mining, pp 280–289Google Scholar
  9. Hirai S, Yamanishi K (2011) Efficient computation of normalized maximum likelihood coding for Gaussian mixtures with its applications to optimal clustering. In: Proceedings of the 2011 IEEE international symposium on information theory, pp 1031–1035Google Scholar
  10. Hirai S, Yamanishi K (2012) Detecting changes of clustering structures using normalized maximum likelihood coding. In: Proceedings of the 18th ACM SIGKDD international conference on knowledge discovery and data mining, pp 343–351Google Scholar
  11. Hirose S, Yamanishi K, Nakata T, Fujimaki R (2009) Network anomaly detection based on eigen equation compression. In: Proceedings of the 15th ACM SIGKDD conference on knowledge discovery and data mining, pp 1185–1194Google Scholar
  12. Hoeffding W (1963) Probability inequalities for sums of bounded random variables. J Am Stat Assoc 58(301):13–30CrossRefzbMATHMathSciNetGoogle Scholar
  13. Ide T, Kashima H (2004) Eigenspace-based anomaly detection in computer systems. In: Proceedings of the 10th ACM SIGKDD international conference on knowledge discovery and data mining, pp 440–449Google Scholar
  14. Ide T, Lozano AC, Abe N, Liu Y (2009) Proximity-based anomaly detection using sparse structure learning. In: Proceedings of the 10th ACM SIGKDD international conference on knowledge discovery and data mining, pp 97–108Google Scholar
  15. Krichevsky RE, Trofimov VK (1981) The performance of universal encoding. IEEE Trans Inf Theory 27(2):199–206CrossRefzbMATHMathSciNetGoogle Scholar
  16. Rissanen J (2000) MDL denoising. IEEE Trans Inf Theory 46(7):2537–2543CrossRefzbMATHMathSciNetGoogle Scholar
  17. Rissanen J (2007) Information and complexity in statistical modeling. Springer, New YorkzbMATHGoogle Scholar
  18. Schwarz G (1978) Estimating the dimension of a model. Ann Stat 6(2):461–464CrossRefzbMATHGoogle Scholar
  19. Shtarkov YM (1987) Universal sequential coding of single messages. Transl Probl Inf Transm 23(3):3–17MathSciNetGoogle Scholar
  20. Silander T, Myllymäki P (2006) A simple approach for finding the globally optimal Bayesian network structure. In: Proceedings of the 22nd conference on uncertainty in artificial intelligence, pp 445–452Google Scholar
  21. Silander T, Roos T, Kontkanen P, Myllymäki P (2008) Factorized normalized maximum likelihood criterion for learning Bayesian network structures. In: Proceedings of 4th European workshop on probabilistic, graphical models, pp 257–264Google Scholar
  22. Talih M, Hengartner N (2005) Structural learning with time-varying components: tracking the cross-section of financial time series. J R Stat Soc B 67(3):321–341CrossRefzbMATHMathSciNetGoogle Scholar
  23. Viterbi A (1967) Error bounds for convolutional codes and an asymptotically optimum decoding algorithm. IEEE Trans Inf Theory 13(2):260–269CrossRefzbMATHGoogle Scholar
  24. Xuan X, Murphy K (2007) Modeling changing dependency structure in multivariate time series. In: Proceedings of the 24th international conference on machine learning, pp 1055–1062Google Scholar
  25. Yamanishi K, Maruyama Y (2005) Dynamic syslog mining for network failure monitoring. In: Proceedings of the 11th ACM SIGKDD international conference on knowledge discovery and data mining, pp 499–508Google Scholar
  26. Yamanishi K, Maruyama Y (2007) Dynamic model selection with its applications to novelty detection. IEEE Trans Inf Theory 53(6):2180–2189CrossRefMathSciNetGoogle Scholar

Copyright information

© The Author(s) 2013

Authors and Affiliations

  1. 1.Graduate School of Information Science and TechnologyThe University of TokyoTokyoJapan

Personalised recommendations