Beyond Shannon: Characterizing Internet Traffic with Generalized Entropy Metrics

  • Bernhard Tellenbach
  • Martin Burkhart
  • Didier Sornette
  • Thomas Maillart
Part of the Lecture Notes in Computer Science book series (LNCS, volume 5448)


Tracking changes in feature distributions is very important in the domain of network anomaly detection. Unfortunately, these distributions consist of thousands or even millions of data points. This makes tracking, storing and visualizing changes over time a difficult task. A standard technique for capturing and describing distributions in a compact form is the Shannon entropy analysis. Its use for detecting network anomalies has been studied in-depth and several anomaly detection approaches have applied it with considerable success. However, reducing the information about a distribution to a single number deletes important information such as the nature of the change or it might lead to overlooking a large amount of anomalies entirely. In this paper, we show that a generalized form of entropy is better suited to capture changes in traffic features, by exploring different moments. We introduce the Traffic Entropy Spectrum (TES) to analyze changes in traffic feature distributions and demonstrate its ability to characterize the structure of anomalies using traffic traces from a large ISP.


Generalize Entropy Shannon Entropy Anomaly Detection Tsallis Entropy Border Router 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Barford, P., Kline, J., Plonka, D., Ron, A.: A signal analysis of network traffic anomalies. In: IMW 2002: Proceedings of the 2nd ACM SIGCOMM Workshop on Internet measurment, pp. 71–82. ACM, New York (2002)CrossRefGoogle Scholar
  2. 2.
    Scherrer, A., Larrieu, N., Owezarski, P., Borgnat, P., Abry, P.: Non-gaussian and long memory statistical characterizations for internet traffic with anomalies. IEEE Transactions on Dependable and Secure Computing 4(1), 56–70 (2007)CrossRefGoogle Scholar
  3. 3.
    Dubendorfer, T., Plattner, B.: Host behaviour based early detection of worm outbreaks in internet backbones. In: 14th IEEE WET ICE, pp. 166–171 (2005)Google Scholar
  4. 4.
    Cisco Systems Inc.: Netflow services solutions guide,
  5. 5.
    Quittek, J., Zseby, T., Claise, B., Zander, S.: Rfc 3917: Requirements for ip flow information export (ipfix) (October 2004)Google Scholar
  6. 6.
    Wagner, A., Plattner, B.: Entropy based worm and anomaly detection in fast ip networks. In: 14th IEEE WET ICE, Linköping, Sweden (June 2005)Google Scholar
  7. 7.
    Lakhina, A., Crovella, M., Diot, C.: Diagnosing network-wide traffic anomalies. In: ACM SIGCOMM, Portland (August 2004)Google Scholar
  8. 8.
    Li, X., Bian, F., Crovella, M., Diot, C., Govindan, R., Iannaccone, G., Lakhina, A.: Detection and identification of network anomalies using sketch subspaces. In: Internet Measurement Conference (IMC), Rio de Janeriro, Brazil, pp. 147–152. ACM, New York (2006)Google Scholar
  9. 9.
    Ziviani, A., Monsores, M.L., Rodrigues, P.S.S., Gomes, A.T.A.: Network anomaly detection using nonextensive entropy. IEEE Communications Letters 11(12) (2007)Google Scholar
  10. 10.
    Shannon, C.: Prediction and entropy of printed english. Bell System Tech. Jour. (January 1951)Google Scholar
  11. 11.
    Tsallis, C.: Possible generalization of boltzmann-gibbs statistics. J. Stat. Phys. 52 (1988)Google Scholar
  12. 12.
    Tsallis, C.: Nonextensive statistics: theoretical, experimental and computational evidences and connections. Brazilian Journal of Physics (January 1999)Google Scholar
  13. 13.
    Tsallis, C.: Entropic nonextensivity: a possible measure of complexity. Chaos (January 2002)Google Scholar
  14. 14.
    Dauxois, T.: Non-gaussian distributions under scrutiny. J. Stat. Mech. (January 2007)Google Scholar
  15. 15.
    Wilk, G., Wlodarczyk, Z.: Example of a possible interpretation of tsallis entropy. arXiv cond-mat.stat-mech (November 2007)Google Scholar
  16. 16.
    Willinger, W., Paxson, V., Taqqu, M.S.: Self-similarity and heavy tails: Structural modeling of network traffic. In: Statistical Techniques and Applications (1998)Google Scholar
  17. 17.
    Kohler, E., Li, J., Paxson, V., Shenker, S.: Observed structure of addresses in ip traffic. In: Proceedings of the SIGCOMM Internet Measurement Workshop, pp. 253–266. ACM, New York (2002)CrossRefGoogle Scholar
  18. 18.
    SWITCH: The swiss education and research network,
  19. 19.
    Gu, Y., McCallum, A., Towsley, D.: Detecting anomalies in network traffic using maximum entropy estimation. In: IMC 2005, pp. 1–6. ACM, New York (2005)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2009

Authors and Affiliations

  • Bernhard Tellenbach
    • 1
  • Martin Burkhart
    • 1
  • Didier Sornette
    • 2
  • Thomas Maillart
    • 2
  1. 1.Computer Engineering and Networks LaboratoryETH ZurichSwitzerland
  2. 2.Department of Management, Technology and EconomicsETH ZurichSwitzerland

Personalised recommendations