Probabilistic Transition-Based Approach for Detecting Application-Layer DDoS Attacks in Encrypted Software-Defined Networks

  • Elena Ivannikova
  • Mikhail Zolotukhin
  • Timo Hämäläinen
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 10394)


With the emergence of cloud computing, many attacks, including Distributed Denial-of-Service (DDoS) attacks, have changed their direction towards cloud environment. In particular, DDoS attacks have changed in scale, methods, and targets and become more complex by using advantages provided by cloud computing. Modern cloud computing environments can benefit from moving towards Software-Defined Networking (SDN) technology, which allows network engineers and administrators to respond quickly to the changing business requirements. In this paper, we propose an approach for detecting application-layer DDoS attacks in cloud environment with SDN. The algorithm is applied to statistics extracted from network flows and, therefore, is suitable for detecting attacks that utilize encrypted protocols. The proposed detection approach is comprised of the extraction of normal user behavior patterns and detection of anomalies that significantly deviate from these patterns. The algorithm is evaluated using DDoS detection system prototype. Simulation results show that intermediate application-layer DDoS attacks can be properly detected, while the number of false alarms remains low.


DDoS attack Anomaly detection SDN Clustering Behavior pattern Probabilistic model 



This research was supported by the Nokia Foundation Scholarship funded by Nokia, Finland.


  1. 1.
    Chen, P.J., Chen, Y.W.: Implementation of SDN based network intrusion detection and prevention system. In: 2015 International Carnahan Conference on Security Technology (ICCST) (2015).
  2. 2.
    Chen, R., Wei, J.Y., Yu, H.F.: An improved grey self-organizing map based dos detection. In: IEEE Conference on Cybernetics and Intelligent Systems, pp. 497–502 (2008).
  3. 3.
    Chwalinski, P., Belavkin, R., Cheng, X.: Detection of application layer DDoS attacks with clustering and Bayes factors. In: 2013 IEEE International Conference on Systems, Man, and Cybernetics (SMC), pp. 156–161 (2013).
  4. 4.
    Dotcenko, S., Vladyko, A., Letenko, I.: A fuzzy logic-based information security management for software-defined networks. In: 16th ICACT, pp. 167–171 (2014).
  5. 5.
    Guha, S., Rastogi, R., Shim, K.: Cure: an efficient clustering algorithm for large databases. Inf. Syst. 26(1), 35–58 (2001). doi: 10.1016/S0306-4379(01)00008-4 CrossRefMATHGoogle Scholar
  6. 6.
    Hastie, T.J., Tibshirani, R.J., Friedman, J.H.: The Elements of Statistical Learning: Data Mining, Inference, and Prediction. Springer Series in Statistics. Springer, New York (2009). doi: 10.1007/978-0-387-84858-7 CrossRefMATHGoogle Scholar
  7. 7.
    Jackson, K.: OpenStack Cloud Computing Cookbook. Packt Publishing, Birmingham (2012)Google Scholar
  8. 8.
    Ke-Xin, Y., Jian-qi, Z.: A novel dos detection mechanism. In: International Conference on Mechatronic Science, Electric Engineering and Computer (MEC), pp. 296–298 (2011).
  9. 9.
    Knorr, E.: Opendaylight: A big step toward the software-defined data center. InfoWorld (2013)Google Scholar
  10. 10.
    Le, A., Dinh, P., Le, H., Tran, N.C.: Flexible network-based intrusion detection and prevention system on software-defined networks. In: 2015 ACOMP, pp. 106–111 (2015).
  11. 11.
    Lim, S., Ha, J., Kim, H., Kim, Y., Yang, S.: A SDN-oriented DDoS blocking scheme for botnet-based attacks. In: 2014 6th International Conference on Ubiquitous and Future Networks (ICUFN), pp. 63–68 (2014).
  12. 12.
    Lloyd, S.: Least squares quantization in PCM. IEEE Trans. Inf. Theor. 28(2), 129–137 (2006). MathSciNetCrossRefMATHGoogle Scholar
  13. 13.
    Macqueen, J.: Some methods for classification and analysis of multivariate observations. In: 5th Berkeley Symposium on Mathematical Statistics and Probability, pp. 281–297 (1967)Google Scholar
  14. 14.
    Mills, K., Yuan, J.: Monitoring the macroscopic effect of DDoS flooding attacks. IEEE Trans. Dependable Secure Comput. 2, 324–335 (2005). CrossRefGoogle Scholar
  15. 15.
    Mirkovic, J., Reiher, P.: A taxonomy of DDoS attack and DDoS defense mechanisms. SIGCOMM Comput. Commun. Rev. 34(2), 39–53 (2004). CrossRefGoogle Scholar
  16. 16.
    Mohammadi, N.B., Barna, C., Shtern, M., Khazaei, H., Litoiu, M.: CAAMP: completely automated DDoS attack mitigation platform in hybrid clouds. In: 12th International CNSM, pp. 136–143 (2016).
  17. 17.
    Pfaff, B., Pettit, J., Koponen, T., Jackson, E.J., Zhou, A., Rajahalme, J., Gross, J., Wang, A., Stringer, J., Shelar, P., Amidon, K., Casado, M.: The design and implementation of open vswitch. In: 12th USENIX Conference on Networked Systems Design and Implementation (NSDI), pp. 117–130 (2015)Google Scholar
  18. 18.
    Phan, T.V., Bao, N.K., Park, M.: A novel hybrid flow-based handler with DDoS attacks in software-defined networking. In: 2016 IEEE UIC/ATC/ScalCom/CBDCom/IoP/SmartWorld (2016).
  19. 19.
  20. 20.
    Somani, G., Gaur, M.S., Sanghi, D., Conti, M., Buyya, R.: DDoS attacks in cloud computing: issues, taxonomy, and future directions. ACM Comput. Surv. 1(1), 1–44 (2015)Google Scholar
  21. 21.
    Stevanovic, D., Vlajic, N.: Next generation application-layer DDoS defences: applying the concepts of outlier detection in data streams with concept drift. In: 13th ICMLA, pp. 456–462 (2014).
  22. 22.
    Xiao, P., Li, Z., Qi, H., Qu, W., Yu, H.: An efficient DDoS detection with bloom filter in SDN. In: 2016 IEEE Trustcom/BigDataSE/ISPA, pp. 1–6 (2016).
  23. 23.
    Xu, C., Zhao, G., Xie, G., Yu, S.: Detection on application layer DDoS using random walk model. In: IEEE International Conference on Communications (ICC), pp. 707–712 (2014).
  24. 24.
    Zolotukhin, M., Hämäläinen, T., Kokkonen, T., Siltanen, J.: Increasing web service availability by detecting application-layer DDoS attacks in encrypted traffic. In: 23rd ICT, pp. 1–6 (2016).
  25. 25.
    Zolotukhin, M., Kokkonen, T., Hämäläinen, T., Siltanen, J.: On application-layer DDoS attack detection in high-speed encrypted networks. Int. J. Digital Content Tech. Appl. 10(5), 14–33 (2016)Google Scholar

Copyright information

© Springer International Publishing AG 2017

Authors and Affiliations

  • Elena Ivannikova
    • 1
  • Mikhail Zolotukhin
    • 1
  • Timo Hämäläinen
    • 1
  1. 1.Department of Mathematical Information TechnologyUniversity of JyväskyläJyväskyläFinland

Personalised recommendations