Distributed Detection of Zero-Day Network Traffic Flows
Zero-day (or unknown) traffic brings about challenges for network security and management tasks, in terms of identifying the occurrence of those events in the network in an accurate and timely manner. In this paper, we propose a distributed mechanism to detect such unknown traffic in a timely manner. We compare our distributed scheme with a centralized system, where all the network flow data are used as a whole to perform the detection. We combined supervised and unsupervised learning mechanisms to discover and classify the unknown traffic efficiently, using clustering and Random Forest (RF) based schemes for this purpose. Further, we incorporated the correlation information in the traffic flows to improve the accuracy of detection, by means of using a Bag of Flows (BoFs) based method. Evaluation on real traces reveal that our distributed approach achieves a comparable detection performance to that of a centralized scheme. Further, the distributed scheme that incorporates unknown sample sharing in the framework shows improvement in the zero-day traffic detection performance. Moreover, the classifier used with the combination of BoF and RF shows improved detection accuracy, compared with not using BoFs.
KeywordsTraffic classification Machine learning Unknown flow detection Zero-day traffic
This work was supported by the National Natural Science Foundation of China under Grant 61401371.
- 3.Juvonen, A., Sipola, T.: Adaptive framework for network traffic classification using dimensionality reduction and clustering. In: 2012 4th International Congress on Ultra Modern Telecommunications and Control Systems and Workshops (ICUMT), pp. 274–279. IEEE (2012)Google Scholar
- 4.Kim, H., Claffy, K.C., Fomenkov, M., Barman, D., Faloutsos, M., Lee, K.: Internet traffic classification demystified: myths, caveats, and the best practices. In: Proceedings of the 2008 ACM CoNEXT Conference, p. 11. ACM (2008)Google Scholar
- 5.Alazab, M., Venkatraman, S., Watters, P., Alazab, M.: Zero-day malware detection based on supervised learning algorithms of API call signatures. In: Proceedings of the Ninth Australasian Data Mining Conference, vol. 121, pp. 171–182. Australian Computer Society, Inc. (2011)Google Scholar
- 11.Miao, Y., Ruan, Z., Pan, L., Zhang, J., Xiang, Y., Wang, Y.: Comprehensive analysis of network traffic data. In: 2016 IEEE International Conference on Computer and Information Technology (CIT), pp. 423–430. IEEE (2016)Google Scholar
- 12.Han, Y., Chan, J., Alpcan, T., Leckie, C.: Using virtual machine allocation policies to defend against co-resident attacks in cloud computing. IEEE Trans. Dependable Secure Comput. 14(1), 95–108 (2017)Google Scholar
- 14.Ling, Z., Luo, J., Wu, K., Yu, W., Fu, X.: Torward: discovery of malicious traffic over Tor. In: 2014 Proceedings IEEE INFOCOM, pp. 1402–1410. IEEE (2014)Google Scholar
- 16.Kharraz, A., Robertson, W., Balzarotti, D., Bilge, L., Kirda, E.: Cutting the gordian knot: a look under the hood of ransomware attacks. In: Almgren, M., Gulisano, V., Maggi, F. (eds.) DIMVA 2015. LNCS, vol. 9148, pp. 3–24. Springer, Cham (2015). https://doi.org/10.1007/978-3-319-20550-2_1CrossRefGoogle Scholar
- 19.Erman, J., Mahanti, A., Arlitt, M.: QRP05-4: internet traffic identification using machine learning. In: IEEE GLOBECOM 2006, pp. 1–6, November 2006Google Scholar