Advertisement

Scalable and Robust Decentralized IP Traffic Flow Collection and Analysis (SCRIPT)

Chapter

Abstract

As the IP traffic observed on network operator’s backbones keeps increasing year by year, the analysis of NetFlow data metered for this traffic becomes a burden for centralized traffic monitoring solutions. Thus, SCRIPT proposes a decentralized accounting architecture and framework for NetFlow storage and analysis, which is flexible to allow for the development of distributed traffic analysis applications. SCRIPT mechanisms organize multiple PCs or AXP (Application Extension Platform) cards in an analysis network and route NetFlow records according to rules imposed by the analysis application. In turn, the evaluation of the prototype has shown that (a) this approach allows for a linear increase of the number of NetFlow records, which can be processed with the number of nodes in the SCRIPT deployment network, and (b) deploying SCRIPT on router-embedded AXP cards is improving an already existing infrastructure with the capability of storage and processing of NetFlow records.

Keywords

Hash Function Application Program Interface Traffic Analysis Stream Control Transmission Protocol Flow Record 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

Notes

Acknowledgements

This work was supported in part by the Cisco University Research Program Fund, Grant No. 2008-02735, in part by the DaSAHIT project funded by the Swiss National Science Foundations, Contract No. 200021-118128/1, and the IST Network of Excellence EMANICS funded by the European Union, Contract No. FP6-2004-IST-026854-NoE. The authors would like to express many thanks to Ralf Wolter, Benoit Claise, and David Hausheer for their valuable support and inspiring discussions as well as Alexander Clemm for his detailed feedback, which helped to improve this chapter.

References

  1. 1.
    Bailey MD, Cooke E, Jahanian F, Nazario J (2005) The Internet motion sensor: A distributed blackhole monitoring system. In: 12th annual network and distributed system security symposium (NDSS’05), San Diego, Feb 2005Google Scholar
  2. 2.
    Brauckhoff D, Tellenbach B, Wagner A, May M, Lakhina A (2006) Impact of packet sampling on anomaly detection metrics. In: 6th ACM SIGCOMM Conference on Internet Measurements, Rio de Janeiro, Brazil, 17–25 Oct 2006Google Scholar
  3. 3.
    Claise B (ed) (2004) Cisco systems NetFlow services export version 9; Internet engineering task force, Internet engineering task force RFC 3954, Oct 2004Google Scholar
  4. 4.
    Claise B (ed) (2008) Specification of the IP Flow Information Export (IPFIX) Protocol for the Exchange of IP Traffic Flow Information; Internet Engineering Task Force RFC 5101, Jan 2008Google Scholar
  5. 5.
    Duffield N, Lund C, Thorup M (2001) Charging from sampled network usage. In: 1st ACM SIGCOMM Workshop on Internet Measurements, San Francisco, Nov 2001Google Scholar
  6. 6.
    FIPS 180-2 (2002) Secure Hash Standard (SHS), National Institute of Standards and Technology, Aug 2002, amended Feb 2004Google Scholar
  7. 7.
    Han SH, Kim MS, Ju HT, Hong JWK (2002) The architecture of NG-MON: a passive network monitoring system for high-speed IP networks. In 13th IFIP/IEEE International Workshop on Distributed Systems: Operations and Management (DSOM’02), Montreal, Canada, Oct 2002Google Scholar
  8. 8.
    Henke C, Schmoll C, Zseby T (2008) Empirical evaluation of hash functions for multipoint measurements. ACM Computer Communication Review 38(3): 39–50Google Scholar
  9. 9.
    Jimenez R, Osmani F, Knutsson B (2011) Sub-second lookups on a large-scale Kademlia based overlay. In: 11th IEEE International Conference on Peer-to-Peer Computing 2011, Kyoto, Japan, Aug 2011Google Scholar
  10. 10.
    Kitatsuji Y, Yamazaki K (2004) A distributed real-time tool for IP-flow measurement. In: international symposium on applications and the Internet, Tokyo, Japan, Jan 2004Google Scholar
  11. 11.
    Maymounkov P, Mazières D (2002) Kademlia: a Peer-to-Peer information system based on the XOR metric. IPTPS, CambridgeGoogle Scholar
  12. 12.
    Mao Y, Chen K, Wang D, Zheng W (2001) Cluster-based online monitoring system of web traffic. In: 3rd International Workshop on Web Information and Data Management, Atlanta, Georgia, USA, Nov 2001Google Scholar
  13. 13.
    Morariu C, Racz P, Stiller B (2009) Design and implementation of a distributed platform for sharing IP flow records. In: 20th IFIP/IEEE International Workshop on Distributed Systems: Operations and Management (DSOM’09), Venice, Italy, Oct 2009Google Scholar
  14. 14.
    Morariu C, Kramis T, Stiller B (2008) DIPStorage: distributed storage of IP flow records. In: 16th IEEE workshop on local and metropolitan area networks, Cluj-Napoca Romania, Sept 2008Google Scholar
  15. 15.
    Morariu C, Racz P, Stiller B (2010) SCRIPT: a framework for scalable real-time IP flow record analysis. In: 12th IEEE/IFIP Network Operations and Management Symposium (NOMS 2010), IEEE, Osaka, Japan, April 2010Google Scholar
  16. 16.
    Postel J (1980) User datagram protocol. Internet Engineering Task Force, RFC 768, August 1980Google Scholar
  17. 17.
    Rivest R (1992) The MD5 message-digest algorithm. Internet Engineering Task Force RFC 1321, April 1992Google Scholar
  18. 18.
    Schulzrinne H, Casner S, Frederick R, Jacobson V (2003) RTP: a transport protocol for real-time applications. Internet Engineering Task Force RFC 3550, July 2003Google Scholar
  19. 19.
    Stewart R, Xie Q, Morneault K, Sharp C, Schwarzbauer H, Taylor T, Rytina I, Kalla M, Zhang L, Paxson V (2000) Stream control transmission protocol. Internet Engineering Task Force RFC 2960, Oct 2000Google Scholar
  20. 20.
    Wikipedia (2011) NetFlow. http://en.wikipedia.org/wiki/Netflow
  21. 21.
    Zseby T, Boschi E, Brownlee N, Claise B (2007) IPFIX applicability. Internet Engineering Task Force, Internet Draft, http://www.ietf.org/internet-drafts/draft-ietf-ipfix-as-12.txt

Copyright information

© Springer Science+Business Media New York 2013

Authors and Affiliations

  • Burkhard Stiller
    • 1
  • Cristian Morariu
    • 1
  • Peter Racz
    • 1
  1. 1.Communication Systems Group CSG, Department of Informatics IFIUniversity of ZürichZürichSwitzerland

Personalised recommendations