Skip to main content
SpringerLink
Log in

Distributed detection/localization of change-points in high-dimensional network traffic data

  • Published:
Statistics and Computing Aims and scope Submit manuscript

Abstract

We propose a novel approach for distributed statistical detection of change-points in high-volume network traffic. We consider more specifically the task of detecting and identifying the targets of Distributed Denial of Service (DDoS) attacks. The proposed algorithm, called DTopRank, performs distributed network anomaly detection by aggregating the partial information gathered in a set of network monitors. In order to address massive data while limiting the communication overhead within the network, the approach combines record filtering at the monitor level and a nonparametric rank test for doubly censored time series at the central decision site. The performance of the DTopRank algorithm is illustrated both on synthetic data as well as from a traffic trace provided by a major Internet service provider.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  • Basseville, M., Nikiforov, I.V.: Detection of Abrupt Changes: Theory and Applications. Prentice Hall, New York (1993)

    Google Scholar 

  • Billingsley, P.: Convergence of Probability Measures. Wiley, New York (1968)

    MATH  Google Scholar 

  • Brodsky, B.E., Darkhovsky, B.S.: Nonparametric Methods in Change-Point Problems. Kluwer Academic, Dordrecht (1993)

    Google Scholar 

  • Csörgő, M., Horváth, L.: Limit Theorems in Change-Point Analysis. Wiley, New York (1997)

    Google Scholar 

  • Dijkstra, E.: A note on two problems in connexion with graphs. Numer. Math. 1(1), 269–271 (1959)

    Article  MathSciNet  MATH  Google Scholar 

  • Erdős, P., Rényi, A.: On random graphs. I. Publ. Math. (Debr.) 6, 290–297 (1959)

    Google Scholar 

  • Gombay, E., Liu, S.: A nonparametric test for change in randomly censored data. Can. J. Stat. 28(1), 113–121 (2000)

    Article  MathSciNet  MATH  Google Scholar 

  • Huang, L., Nguyen, X., Garofalakis, M., Jordan, M.I., Joseph, A., Taft, N.: In-network PCA and anomaly detection. In: Schölkopf, B., Platt, J., Hoffman, T. (eds.) Advances in Neural Information Processing Systems, vol. 19, pp. 617–624. MIT Press, Cambridge (2007)

    Google Scholar 

  • Krishnamurthy, B., Sen, S., Zhang, Y., Chen, Y.: Sketch-based change detection: methods, evaluation, and applications. In: IMC ’03: Proceedings of the 3rd ACM SIGCOMM Conference on Internet Measurement, pp. 234–247. ACM, New York (2003)

    Chapter  Google Scholar 

  • Lakhina, A., Crovella, M., Diot, C.: Diagnosing network-wide traffic anomalies. In: SIGCOMM ’04: Proceedings of the 2004 Conference on Applications, Technologies, Architectures, and Protocols for Computer Communications, pp. 219–230. ACM, New York (2004)

    Chapter  Google Scholar 

  • Lévy-Leduc, C., Roueff, F.: Detection and localization of change-points in high-dimensional network traffic data. Ann. Appl. Stat. 3(2), 637–662 (2009)

    Article  MathSciNet  MATH  Google Scholar 

  • Nucci, A., Sridharan, A., Taft, N.: The problem of synthetically generating IP traffic matrices: initial recommendations. ACM SIGCOMM Comput. Commun. Rev. 35(3), 19–32 (2005)

    Article  Google Scholar 

  • Park, C., Hernandez-Campos, F., Marron, J., Smith, F.D.: Long-range dependence in a changing Internet traffic mix. Comput. Netw. 48(3), 401–422 (2005)

    Article  Google Scholar 

  • Siris, V.A., Papagalou, F.: Application of anomaly detection algorithms for detecting SYN flooding attacks. Comput. Commun. 29(9), 1433–1442 (2006). iCON 2004—12th IEEE International Conference on Network 2004

    Article  Google Scholar 

  • Susitaival, R., Juva, I., Peuhkuri, M., Aalto, S.: Characteristics of origin-destination pair traffic in Funet. Telecommun. Syst. 33, 67–88 (2006)

    Article  Google Scholar 

  • Tartakovsky, A., Rozovskii, B., Blazek, R., Kim, H.: Detection of intrusion in information systems by sequential change-point methods. Stat. Methodol. 3(3), 252–340 (2006)

    Article  MathSciNet  Google Scholar 

  • van der Vaart, A.W.: Asymptotic Statistics. Cambridge Series in Statistical and Probabilistic Mathematics, vol. 3. Cambridge University Press, Cambridge (1998)

    MATH  Google Scholar 

  • Wang, H., Zhang, D., Shin, K.G.: Detecting SYN flooding attacks. In: INFOCOM 2002 Twenty-First Annual Joint Conference of the IEEE Computer and Communications Societies Proceedings IEEE, vol. 3, pp. 1530–1539 (2002)

    Chapter  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Institut Telecom & CNRS/LTCI/Telecom ParisTech, 46, rue Barrault, 75634, Paris Cédex 13, France

    A. Lung-Yut-Fong, C. Lévy-Leduc & O. Cappé

Authors
  1. A. Lung-Yut-Fong
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. C. Lévy-Leduc
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. O. Cappé
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to A. Lung-Yut-Fong.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Lung-Yut-Fong, A., Lévy-Leduc, C. & Cappé, O. Distributed detection/localization of change-points in high-dimensional network traffic data. Stat Comput 22, 485–496 (2012). https://doi.org/10.1007/s11222-011-9240-5

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11222-011-9240-5

Keywords

Search

Navigation