Advertisement

Statistics and Computing

, Volume 22, Issue 2, pp 485–496 | Cite as

Distributed detection/localization of change-points in high-dimensional network traffic data

  • A. Lung-Yut-Fong
  • C. Lévy-Leduc
  • O. Cappé
Article

Abstract

We propose a novel approach for distributed statistical detection of change-points in high-volume network traffic. We consider more specifically the task of detecting and identifying the targets of Distributed Denial of Service (DDoS) attacks. The proposed algorithm, called DTopRank, performs distributed network anomaly detection by aggregating the partial information gathered in a set of network monitors. In order to address massive data while limiting the communication overhead within the network, the approach combines record filtering at the monitor level and a nonparametric rank test for doubly censored time series at the central decision site. The performance of the DTopRank algorithm is illustrated both on synthetic data as well as from a traffic trace provided by a major Internet service provider.

Keywords

Distributed detection Change-point detection Rank test Censored data Network anomaly detection 

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. Basseville, M., Nikiforov, I.V.: Detection of Abrupt Changes: Theory and Applications. Prentice Hall, New York (1993) Google Scholar
  2. Billingsley, P.: Convergence of Probability Measures. Wiley, New York (1968) zbMATHGoogle Scholar
  3. Brodsky, B.E., Darkhovsky, B.S.: Nonparametric Methods in Change-Point Problems. Kluwer Academic, Dordrecht (1993) Google Scholar
  4. Csörgő, M., Horváth, L.: Limit Theorems in Change-Point Analysis. Wiley, New York (1997) Google Scholar
  5. Dijkstra, E.: A note on two problems in connexion with graphs. Numer. Math. 1(1), 269–271 (1959) MathSciNetzbMATHCrossRefGoogle Scholar
  6. Erdős, P., Rényi, A.: On random graphs. I. Publ. Math. (Debr.) 6, 290–297 (1959) Google Scholar
  7. Gombay, E., Liu, S.: A nonparametric test for change in randomly censored data. Can. J. Stat. 28(1), 113–121 (2000) MathSciNetzbMATHCrossRefGoogle Scholar
  8. Huang, L., Nguyen, X., Garofalakis, M., Jordan, M.I., Joseph, A., Taft, N.: In-network PCA and anomaly detection. In: Schölkopf, B., Platt, J., Hoffman, T. (eds.) Advances in Neural Information Processing Systems, vol. 19, pp. 617–624. MIT Press, Cambridge (2007) Google Scholar
  9. Krishnamurthy, B., Sen, S., Zhang, Y., Chen, Y.: Sketch-based change detection: methods, evaluation, and applications. In: IMC ’03: Proceedings of the 3rd ACM SIGCOMM Conference on Internet Measurement, pp. 234–247. ACM, New York (2003) CrossRefGoogle Scholar
  10. Lakhina, A., Crovella, M., Diot, C.: Diagnosing network-wide traffic anomalies. In: SIGCOMM ’04: Proceedings of the 2004 Conference on Applications, Technologies, Architectures, and Protocols for Computer Communications, pp. 219–230. ACM, New York (2004) CrossRefGoogle Scholar
  11. Lévy-Leduc, C., Roueff, F.: Detection and localization of change-points in high-dimensional network traffic data. Ann. Appl. Stat. 3(2), 637–662 (2009) MathSciNetzbMATHCrossRefGoogle Scholar
  12. Nucci, A., Sridharan, A., Taft, N.: The problem of synthetically generating IP traffic matrices: initial recommendations. ACM SIGCOMM Comput. Commun. Rev. 35(3), 19–32 (2005) CrossRefGoogle Scholar
  13. Park, C., Hernandez-Campos, F., Marron, J., Smith, F.D.: Long-range dependence in a changing Internet traffic mix. Comput. Netw. 48(3), 401–422 (2005) CrossRefGoogle Scholar
  14. Siris, V.A., Papagalou, F.: Application of anomaly detection algorithms for detecting SYN flooding attacks. Comput. Commun. 29(9), 1433–1442 (2006). iCON 2004—12th IEEE International Conference on Network 2004 CrossRefGoogle Scholar
  15. Susitaival, R., Juva, I., Peuhkuri, M., Aalto, S.: Characteristics of origin-destination pair traffic in Funet. Telecommun. Syst. 33, 67–88 (2006) CrossRefGoogle Scholar
  16. Tartakovsky, A., Rozovskii, B., Blazek, R., Kim, H.: Detection of intrusion in information systems by sequential change-point methods. Stat. Methodol. 3(3), 252–340 (2006) MathSciNetCrossRefGoogle Scholar
  17. van der Vaart, A.W.: Asymptotic Statistics. Cambridge Series in Statistical and Probabilistic Mathematics, vol. 3. Cambridge University Press, Cambridge (1998) zbMATHGoogle Scholar
  18. Wang, H., Zhang, D., Shin, K.G.: Detecting SYN flooding attacks. In: INFOCOM 2002 Twenty-First Annual Joint Conference of the IEEE Computer and Communications Societies Proceedings IEEE, vol. 3, pp. 1530–1539 (2002) CrossRefGoogle Scholar

Copyright information

© Springer Science+Business Media, LLC 2011

Authors and Affiliations

  1. 1.Institut Telecom & CNRS/LTCI/Telecom ParisTechParis Cédex 13France

Personalised recommendations