Distributed Detection of Zero-Day Network Traffic Flows

  • Yuantian Miao
  • Lei Pan
  • Sutharshan Rajasegarar
  • Jun Zhang
  • Christopher Leckie
  • Yang Xiang
Conference paper
First Online:
Part of the Communications in Computer and Information Science book series (CCIS, volume 845)

Abstract

Zero-day (or unknown) traffic brings about challenges for network security and management tasks, in terms of identifying the occurrence of those events in the network in an accurate and timely manner. In this paper, we propose a distributed mechanism to detect such unknown traffic in a timely manner. We compare our distributed scheme with a centralized system, where all the network flow data are used as a whole to perform the detection. We combined supervised and unsupervised learning mechanisms to discover and classify the unknown traffic efficiently, using clustering and Random Forest (RF) based schemes for this purpose. Further, we incorporated the correlation information in the traffic flows to improve the accuracy of detection, by means of using a Bag of Flows (BoFs) based method. Evaluation on real traces reveal that our distributed approach achieves a comparable detection performance to that of a centralized scheme. Further, the distributed scheme that incorporates unknown sample sharing in the framework shows improvement in the zero-day traffic detection performance. Moreover, the classifier used with the combination of BoF and RF shows improved detection accuracy, compared with not using BoFs.

Keywords

Traffic classification Machine learning Unknown flow detection Zero-day traffic 
This is a preview of subscription content, log in to check access

Notes

Acknowledgement

This work was supported by the National Natural Science Foundation of China under Grant 61401371.

References

  1. 1.
    Nguyen, T.T., Armitage, G.: A survey of techniques for internet traffic classification using machine learning. IEEE Commun. Surv. Tutor. 10(4), 56–76 (2008)CrossRefGoogle Scholar
  2. 2.
    Finamore, A., Mellia, M., Meo, M., Rossi, D.: KISS: stochastic packet inspection classifier for UDP traffic. IEEE/ACM Trans. Netw. (TON) 18(5), 1505–1515 (2010)CrossRefGoogle Scholar
  3. 3.
    Juvonen, A., Sipola, T.: Adaptive framework for network traffic classification using dimensionality reduction and clustering. In: 2012 4th International Congress on Ultra Modern Telecommunications and Control Systems and Workshops (ICUMT), pp. 274–279. IEEE (2012)Google Scholar
  4. 4.
    Kim, H., Claffy, K.C., Fomenkov, M., Barman, D., Faloutsos, M., Lee, K.: Internet traffic classification demystified: myths, caveats, and the best practices. In: Proceedings of the 2008 ACM CoNEXT Conference, p. 11. ACM (2008)Google Scholar
  5. 5.
    Alazab, M., Venkatraman, S., Watters, P., Alazab, M.: Zero-day malware detection based on supervised learning algorithms of API call signatures. In: Proceedings of the Ninth Australasian Data Mining Conference, vol. 121, pp. 171–182. Australian Computer Society, Inc. (2011)Google Scholar
  6. 6.
    Este, A., Gringoli, F., Salgarelli, L.: Support vector machines for TCP traffic classification. Comput. Netw. 53(14), 2476–2490 (2009)CrossRefGoogle Scholar
  7. 7.
    Finamore, A., Mellia, M., Meo, M.: Mining unclassified traffic using automatic clustering techniques. In: Domingo-Pascual, J., Shavitt, Y., Uhlig, S. (eds.) TMA 2011. LNCS, vol. 6613, pp. 150–163. Springer, Heidelberg (2011).  https://doi.org/10.1007/978-3-642-20305-3_13CrossRefGoogle Scholar
  8. 8.
    Criminisi, A., Shotton, J., Konukoglu, E., et al.: Decision forests: a unified framework for classification, regression, density estimation, manifold learning and semi-supervised learning. Found. Trends Comput. Graph. Vis. 7(2–3), 81–227 (2012)MATHGoogle Scholar
  9. 9.
    Hastie, T., Tibshirani, R., Friedman, J.: Unsupervised learning. In: Hastie, T., Tibshirani, R., Friedman, J. (eds.) The Elements of Statistical Learning. Springer Series in Statistics, pp. 485–585. Springer, New York (2009).  https://doi.org/10.1007/978-0-387-84858-7_14CrossRefMATHGoogle Scholar
  10. 10.
    Zhang, J., Chen, X., Xiang, Y., Zhou, W., Wu, J.: Robust network traffic classification. IEEE/ACM Trans. Netw. (TON) 23(4), 1257–1270 (2015)CrossRefGoogle Scholar
  11. 11.
    Miao, Y., Ruan, Z., Pan, L., Zhang, J., Xiang, Y., Wang, Y.: Comprehensive analysis of network traffic data. In: 2016 IEEE International Conference on Computer and Information Technology (CIT), pp. 423–430. IEEE (2016)Google Scholar
  12. 12.
    Han, Y., Chan, J., Alpcan, T., Leckie, C.: Using virtual machine allocation policies to defend against co-resident attacks in cloud computing. IEEE Trans. Dependable Secure Comput. 14(1), 95–108 (2017)Google Scholar
  13. 13.
    Rajasegarar, S., Leckie, C., Palaniswami, M.: Hyperspherical cluster based distributed anomaly detection in wireless sensor networks. J. Parallel Distrib. Comput. 74(1), 1833–1847 (2014)CrossRefGoogle Scholar
  14. 14.
    Ling, Z., Luo, J., Wu, K., Yu, W., Fu, X.: Torward: discovery of malicious traffic over Tor. In: 2014 Proceedings IEEE INFOCOM, pp. 1402–1410. IEEE (2014)Google Scholar
  15. 15.
    Conti, M., Mancini, L.V., Spolaor, R., Verde, N.V.: Analyzing android encrypted network traffic to identify user actions. IEEE Trans. Inf. Forensics Secur. 11(1), 114–125 (2016)CrossRefGoogle Scholar
  16. 16.
    Kharraz, A., Robertson, W., Balzarotti, D., Bilge, L., Kirda, E.: Cutting the gordian knot: a look under the hood of ransomware attacks. In: Almgren, M., Gulisano, V., Maggi, F. (eds.) DIMVA 2015. LNCS, vol. 9148, pp. 3–24. Springer, Cham (2015).  https://doi.org/10.1007/978-3-319-20550-2_1CrossRefGoogle Scholar
  17. 17.
    Zhang, J., Xiang, Y., Wang, Y., Zhou, W., Xiang, Y., Guan, Y.: Network traffic classification using correlation information. IEEE Trans. Parallel Distrib. Syst. 24(1), 104–117 (2013)CrossRefGoogle Scholar
  18. 18.
    Zhang, J., Chen, C., Xiang, Y., Zhou, W., Vasilakos, A.V.: An effective network traffic classification method with unknown flow detection. IEEE Trans. Netw. Serv. Manag. 10(2), 133–147 (2013)CrossRefGoogle Scholar
  19. 19.
    Erman, J., Mahanti, A., Arlitt, M.: QRP05-4: internet traffic identification using machine learning. In: IEEE GLOBECOM 2006, pp. 1–6, November 2006Google Scholar
  20. 20.
    Wang, Y., Xiang, Y., Yu, S.Z.: An automatic application signature construction system for unknown traffic. Concurr. Comput.: Pract. Exp. 22(13), 1927–1944 (2010)CrossRefGoogle Scholar
  21. 21.
    Zhang, J., Chen, X., Xiang, Y., Zhou, W.: Zero-day traffic identification. In: Wang, G., Ray, I., Feng, D., Rajarajan, M. (eds.) CSS 2013. LNCS, vol. 8300, pp. 213–227. Springer, Cham (2013).  https://doi.org/10.1007/978-3-319-03584-0_16CrossRefGoogle Scholar

Copyright information

© Springer Nature Singapore Pte Ltd. 2018

Authors and Affiliations

  • Yuantian Miao
    • 1
  • Lei Pan
    • 2
  • Sutharshan Rajasegarar
    • 2
  • Jun Zhang
    • 1
  • Christopher Leckie
    • 3
  • Yang Xiang
    • 1
  1. 1.School of Software and Electrical EngineeringSwinburne University of TechnologyHawthornAustralia
  2. 2.School of ITDeakin UniversityGeelongAustralia
  3. 3.Department of Computing and Information SystemsUniversity of MelbourneMelbourneAustralia

Personalised recommendations