Distributed Detection of Zero-Day Network Traffic Flows
Abstract
Zero-day (or unknown) traffic brings about challenges for network security and management tasks, in terms of identifying the occurrence of those events in the network in an accurate and timely manner. In this paper, we propose a distributed mechanism to detect such unknown traffic in a timely manner. We compare our distributed scheme with a centralized system, where all the network flow data are used as a whole to perform the detection. We combined supervised and unsupervised learning mechanisms to discover and classify the unknown traffic efficiently, using clustering and Random Forest (RF) based schemes for this purpose. Further, we incorporated the correlation information in the traffic flows to improve the accuracy of detection, by means of using a Bag of Flows (BoFs) based method. Evaluation on real traces reveal that our distributed approach achieves a comparable detection performance to that of a centralized scheme. Further, the distributed scheme that incorporates unknown sample sharing in the framework shows improvement in the zero-day traffic detection performance. Moreover, the classifier used with the combination of BoF and RF shows improved detection accuracy, compared with not using BoFs.
Keywords
Traffic classification Machine learning Unknown flow detection Zero-day trafficNotes
Acknowledgement
This work was supported by the National Natural Science Foundation of China under Grant 61401371.
References
- 1.Nguyen, T.T., Armitage, G.: A survey of techniques for internet traffic classification using machine learning. IEEE Commun. Surv. Tutor. 10(4), 56–76 (2008)CrossRefGoogle Scholar
- 2.Finamore, A., Mellia, M., Meo, M., Rossi, D.: KISS: stochastic packet inspection classifier for UDP traffic. IEEE/ACM Trans. Netw. (TON) 18(5), 1505–1515 (2010)CrossRefGoogle Scholar
- 3.Juvonen, A., Sipola, T.: Adaptive framework for network traffic classification using dimensionality reduction and clustering. In: 2012 4th International Congress on Ultra Modern Telecommunications and Control Systems and Workshops (ICUMT), pp. 274–279. IEEE (2012)Google Scholar
- 4.Kim, H., Claffy, K.C., Fomenkov, M., Barman, D., Faloutsos, M., Lee, K.: Internet traffic classification demystified: myths, caveats, and the best practices. In: Proceedings of the 2008 ACM CoNEXT Conference, p. 11. ACM (2008)Google Scholar
- 5.Alazab, M., Venkatraman, S., Watters, P., Alazab, M.: Zero-day malware detection based on supervised learning algorithms of API call signatures. In: Proceedings of the Ninth Australasian Data Mining Conference, vol. 121, pp. 171–182. Australian Computer Society, Inc. (2011)Google Scholar
- 6.Este, A., Gringoli, F., Salgarelli, L.: Support vector machines for TCP traffic classification. Comput. Netw. 53(14), 2476–2490 (2009)CrossRefGoogle Scholar
- 7.Finamore, A., Mellia, M., Meo, M.: Mining unclassified traffic using automatic clustering techniques. In: Domingo-Pascual, J., Shavitt, Y., Uhlig, S. (eds.) TMA 2011. LNCS, vol. 6613, pp. 150–163. Springer, Heidelberg (2011). https://doi.org/10.1007/978-3-642-20305-3_13CrossRefGoogle Scholar
- 8.Criminisi, A., Shotton, J., Konukoglu, E., et al.: Decision forests: a unified framework for classification, regression, density estimation, manifold learning and semi-supervised learning. Found. Trends Comput. Graph. Vis. 7(2–3), 81–227 (2012)MATHGoogle Scholar
- 9.Hastie, T., Tibshirani, R., Friedman, J.: Unsupervised learning. In: Hastie, T., Tibshirani, R., Friedman, J. (eds.) The Elements of Statistical Learning. Springer Series in Statistics, pp. 485–585. Springer, New York (2009). https://doi.org/10.1007/978-0-387-84858-7_14CrossRefMATHGoogle Scholar
- 10.Zhang, J., Chen, X., Xiang, Y., Zhou, W., Wu, J.: Robust network traffic classification. IEEE/ACM Trans. Netw. (TON) 23(4), 1257–1270 (2015)CrossRefGoogle Scholar
- 11.Miao, Y., Ruan, Z., Pan, L., Zhang, J., Xiang, Y., Wang, Y.: Comprehensive analysis of network traffic data. In: 2016 IEEE International Conference on Computer and Information Technology (CIT), pp. 423–430. IEEE (2016)Google Scholar
- 12.Han, Y., Chan, J., Alpcan, T., Leckie, C.: Using virtual machine allocation policies to defend against co-resident attacks in cloud computing. IEEE Trans. Dependable Secure Comput. 14(1), 95–108 (2017)Google Scholar
- 13.Rajasegarar, S., Leckie, C., Palaniswami, M.: Hyperspherical cluster based distributed anomaly detection in wireless sensor networks. J. Parallel Distrib. Comput. 74(1), 1833–1847 (2014)CrossRefGoogle Scholar
- 14.Ling, Z., Luo, J., Wu, K., Yu, W., Fu, X.: Torward: discovery of malicious traffic over Tor. In: 2014 Proceedings IEEE INFOCOM, pp. 1402–1410. IEEE (2014)Google Scholar
- 15.Conti, M., Mancini, L.V., Spolaor, R., Verde, N.V.: Analyzing android encrypted network traffic to identify user actions. IEEE Trans. Inf. Forensics Secur. 11(1), 114–125 (2016)CrossRefGoogle Scholar
- 16.Kharraz, A., Robertson, W., Balzarotti, D., Bilge, L., Kirda, E.: Cutting the gordian knot: a look under the hood of ransomware attacks. In: Almgren, M., Gulisano, V., Maggi, F. (eds.) DIMVA 2015. LNCS, vol. 9148, pp. 3–24. Springer, Cham (2015). https://doi.org/10.1007/978-3-319-20550-2_1CrossRefGoogle Scholar
- 17.Zhang, J., Xiang, Y., Wang, Y., Zhou, W., Xiang, Y., Guan, Y.: Network traffic classification using correlation information. IEEE Trans. Parallel Distrib. Syst. 24(1), 104–117 (2013)CrossRefGoogle Scholar
- 18.Zhang, J., Chen, C., Xiang, Y., Zhou, W., Vasilakos, A.V.: An effective network traffic classification method with unknown flow detection. IEEE Trans. Netw. Serv. Manag. 10(2), 133–147 (2013)CrossRefGoogle Scholar
- 19.Erman, J., Mahanti, A., Arlitt, M.: QRP05-4: internet traffic identification using machine learning. In: IEEE GLOBECOM 2006, pp. 1–6, November 2006Google Scholar
- 20.Wang, Y., Xiang, Y., Yu, S.Z.: An automatic application signature construction system for unknown traffic. Concurr. Comput.: Pract. Exp. 22(13), 1927–1944 (2010)CrossRefGoogle Scholar
- 21.Zhang, J., Chen, X., Xiang, Y., Zhou, W.: Zero-day traffic identification. In: Wang, G., Ray, I., Feng, D., Rajarajan, M. (eds.) CSS 2013. LNCS, vol. 8300, pp. 213–227. Springer, Cham (2013). https://doi.org/10.1007/978-3-319-03584-0_16CrossRefGoogle Scholar