Journal of Network and Systems Management

, Volume 23, Issue 3, pp 401–419 | Cite as

An Autonomic Traffic Classification System for Network Operation and Management

  • Valentín Carela-Español
  • Pere Barlet-Ros
  • Oriol Mula-Valls
  • Josep Solé-Pareta
Article

Abstract

Traffic classification is an important aspect in network operation and management, but challenging from a research perspective. During the last decade, several works have proposed different methods for traffic classification. Although most proposed methods achieve high accuracy, they present several practical limitations that hinder their actual deployment in production networks. For example, existing methods often require a costly training phase or expensive hardware, while their results have relatively low completeness. In this paper, we address these practical limitations by proposing an autonomic traffic classification system for large networks. Our system combines multiple classification techniques to leverage their advantages and minimize the limitations they present when used alone. Our system can operate with Sampled NetFlow data making it easier to deploy in production networks to assist network operation and management tasks. The main novelty of our system is that it can automatically retrain itself in order to sustain a high classification accuracy along time. We evaluate our solution using a 14-day trace from a large production network and show that our system can sustain an accuracy <96 %, even in presence of sampling, during long periods of time. The proposed system has been deployed in production in the Catalan Research and Education network and it is currently being used by network managers of more than 90 institutions connected to this network.

Keywords

Network monitoring Machine learning Deep Packet Inspection Application identification Self-adaptative system 

Notes

Acknowledgments

The authors want to thank ipoque for kindly providing access to their PACE software and Tatsuya Mori for sharing with us the list of IPs presented in [10]. We would also like to thank UPCnet and CESCA for the traffic traces provided for this study. This research was funded by the Spanish Ministry of Economy and Competitiveness under contract TEC2011-27474 (NOMADS project) and by the Comissionat per a Universitats i Recerca del DIUE de la Generalitat de Catalunya (Ref. 2009SGR-1140).

References

  1. 1.
    Internet Assigned Numbers Authority (IANA): http://www.iana.org/assignments/port-numbers
  2. 2.
    Moore, A., Papagiannaki, K.: Toward the accurate identification of network applications. In: Proceedings of Passive and Active Measurement Conference (PAM), pp. 41–54 (2005)Google Scholar
  3. 3.
    Dainotti, A., Gargiulo, F., Kuncheva, L., Pescape, A., Sansone, C.: Identification of traffic flows hiding behind tcp port 80. In: IEEE International Conference on Communications (ICC), pp. 1–6 (2009)Google Scholar
  4. 4.
    Karagiannis, T., Papagiannaki, K., Faloutsos, M.: BLINC: multilevel traffic classification in the dark. In: Proceedings of ACM Annual Conference of the Special Interest Group on Data Communication (SIGCOMM), pp. 229–240 (2005)Google Scholar
  5. 5.
    Jiang, H., Moore, A., Ge, Z., Jin, S., Wang, J.: Lightweight application classification for network management. In: Proceedings of the ACM SIGCOMM Workshop on Internet Network Management (INM), pp. 299–304 (2007)Google Scholar
  6. 6.
    Nguyen, T., Armitage, G.: A survey of techniques for internet traffic classification using machine learning. IEEE Commun. Surv. Tutor. 10(4), 56–76 (2008)CrossRefGoogle Scholar
  7. 7.
    Yoon, S., Park, J., Park, J., Oh, Y., Kim, M.: Internet application traffic classification using fixed IP-port. Manag. Enabling Future Internet Chang. Bus. New Comput. Serv. 5787, 21–30 (2009)CrossRefGoogle Scholar
  8. 8.
    Carela-Espanol, V., Barlet-Ros, P., Sole-Simo, M., Dainotti, A., de Donato, W., Pescape, A.: K-dimensional trees for continuous traffic classification. In: Proceedings of Traffic Monitoring and Analysis (TMA) pp. 141–155 (2010)Google Scholar
  9. 9.
    Li, J., Zhang, S., Li, C., Yan, J.: Composite lightweight traffic classification system for network management. Int. J. Netw. Manag. 20(2), 85–105 (2010)Google Scholar
  10. 10.
    Mori, T., Kawahara, R., Hasegawa, H., Shimogawa, S.: Characterizing traffic flows originating from large-scale video sharing services. In: Proceedings of Traffic Monitoring and Analysis (TMA) pp. 17–31 (2010)Google Scholar
  11. 11.
    Carela-Espanol, V., Barlet-Ros, P., Cabellos-Aparicio, A., Sole-Pareta, J.: Analysis of the impact of sampling on NetFlow traffic classification. Comput. Netw. 55(5), 1083–1099 (2011)CrossRefGoogle Scholar
  12. 12.
    Dainotti, A., Pescapé, A., Sansone, C.: Early classification of network traffic through multi-classification. In: Proceedings of Traffic Monitoring and Analysis (TMA) pp. 122–135 (2011)Google Scholar
  13. 13.
    Lee, S., Kim, H., Barman, D., Lee, S., Kim, C., Kwon, T., Choi, Y.: NeTraMark: a network traffic classification benchmark. ACM SIGCOMM Comput. Commun. Rev. 41(1), 22–30 (2011)CrossRefGoogle Scholar
  14. 14.
    Williams, N., Zander, S., Armitage, G.: A preliminary performance comparison of five machine learning algorithms for practical IP traffic flow classification. ACM SIGCOMM Comput. Commun. Rev. 36(5), 5–16 (2006)CrossRefGoogle Scholar
  15. 15.
    Crotti, M., Gringoli, F.: Traffic classification through simple statistical fingerprinting. ACM SIGCOMM Comput. Commun. Rev. 37(1), 5–16 (2007)CrossRefGoogle Scholar
  16. 16.
    Li, W., Canini, M., Moore, A., Bolla, R.: Efficient application identification and the temporal and spatial stability of classification schema. Comput. Netw. 53(6), 790–809 (2009)MATHCrossRefGoogle Scholar
  17. 17.
    Sen, S., Spatscheck, O., Wang, D.: Accurate, scalable in-network identification of p2p traffic using application signatures. In: Proceedings of ACM International World Wide Web Conference (WWW), pp. 512–521 (2004)Google Scholar
  18. 18.
    Karagiannis, T., Broido, A., Faloutsos, M.: Transport layer identification of P2P traffic. In: Proceedings of ACM Internet Measurement Conference (IMC), pp. 121–134 (2004)Google Scholar
  19. 19.
    Xu, K., Zhang, Z., Bhattacharyya, S.: Profiling internet backbone traffic: behavior models and applications. In: Proceedings of ACM Annual Conference of the Special Interest Group on Data Communication (SIGCOMM), pp. 169–180 (2005)Google Scholar
  20. 20.
    Karagiannis, T., Papagiannaki, K., Taft, N., Faloutsos, M.: Profiling the end host. In: Proceedings of Passive and Active Measurement Conference (PAM), pp. 186–196. Springer (2007)Google Scholar
  21. 21.
    Kim, H., Claffy, K., Fomenkov, M., Barman, D., Faloutsos, M., Lee, K.: Internet traffic classification demystified: myths, caveats, and the best practices. In: Proceedings of ACM International Conference on Emerging Networking EXperiments and Technologies (CoNEXT), p. 11 (2008)Google Scholar
  22. 22.
    L7-filter, Application Layer Packet Classifier for Linux: http://l7-filter.clearfoundation.com/
  23. 23.
    OpenDPI, The Open Source Deep Packet Inspection Engine: http://www.opendpi.org/
  24. 24.
    PACE, ipoque’s Protocol and Application Classification Engine: http://www.ipoque.com/en/products/pace
  25. 25.
  26. 26.
    Iannaccone, G.: Fast prototyping of network data mining applications. In: Proceedings of Passive and Active Measurement Conference (PAM), pp. 41–50 (2006)Google Scholar
  27. 27.
  28. 28.
    Barlet-Ros, P., Sole-Pareta, J., Barrantes, J., Codina, E., Domingo-Pascual, J.: SMARTxAC: a passive monitoring and analysis system for high-speed networks. Campus-Wide Inf. Syst. 23(4), 283–296 (2006)CrossRefGoogle Scholar
  29. 29.
    Quinlan, J.R.: C4.5: Programs for Machine Learning. The Morgan Kaufmann Series in Machine Learning. Morgan Kaufmann, San Mateo, CA (1993)Google Scholar
  30. 30.
    Lim, Y., Kim, H., Jeong, J., Kim, C., Kwon, T., Choi, Y.: Internet traffic classification demystified: on the sources of the discriminative power. In: Proceedings of ACM International Conference on Emerging Networking Experiments and Technologies (CoNEXT), p. 9 (2010)Google Scholar
  31. 31.
    Is See5/C5.0 Better Than C4.5?: http://rulequest.com/see5-comparison.html
  32. 32.
    Cohen, J.: A coefficient of agreement for nominal scales. Educ. Psychol. Meas. 20(1), 37–46 (1960)CrossRefGoogle Scholar
  33. 33.
    Alcock, S., Nelson, R.: Libprotoident: Traffic Classification Using Lightweight Packet Inspection. Technical Report. University of Waikato (2012). http://www.wand.net.nz/publications/lpireport
  34. 34.
    nDPI, Open and Extensible GPLv3 Deep Packet Inspection Library: http://www.ntop.org/products/ndpi/
  35. 35.
    Zhang, J., Chen, C., Xiang, Y., Zhou, W., Vasilakos, A.: An effective network traffic classification method with unknown flow detection. IEEE Trans. Netw. Serv. Manag. 10(2), 133–147 (2013)CrossRefGoogle Scholar

Copyright information

© Springer Science+Business Media New York 2013

Authors and Affiliations

  • Valentín Carela-Español
    • 1
  • Pere Barlet-Ros
    • 1
  • Oriol Mula-Valls
    • 1
  • Josep Solé-Pareta
    • 1
  1. 1.Dept. Arquitectura de ComputadorsUPC BarcelonaTechBarcelonaSpain

Personalised recommendations