Defending DDoS Attacks Using Hidden Markov Models and Cooperative Reinforcement Learning

  • Xin Xu
  • Yongqiang Sun
  • Zunguo Huang
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 4430)


In recent years, distributed denial of service (DDoS) attacks have brought increasing threats to the Internet since attack traffic caused by DDoS attacks can consume lots of bandwidth or computing resources on the Internet and the availability of DDoS attack tools has become more and more easy. However, due to the similarity between DDoS attack traffic and transient bursts of normal traffic, it is very difficult to detect DDoS attacks accurately and quickly. In this paper, a novel DDoS detection approach based on Hidden Markov Models (HMMs) and cooperative reinforcement learning is proposed, where a distributed cooperation detection scheme using source IP address monitoring is employed. To realize earlier detection of DDoS attacks, the detectors are distributed in the mediate network nodes or near the sources of DDoS attacks and HMMs are used to establish a profile for normal traffic based on the frequencies of new IP addresses. A cooperative reinforcement learning algorithm is proposed to compute optimized strategies of information exchange among the distributed multiple detectors so that the detection accuracies can be improved without much load on information communications among the detectors. Simulation results on distributed detection of DDoS attacks generated by TFN2K tools illustrate the effectiveness of the proposed method.


Communication Cost Intrusion Detection Anomaly Detection Observation Sequence Normal Traffic 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


  1. 1.
    Denning, D.: An Intrusion-Detection Model. IEEE Transactions on Software Engineering 13(2), 222–232 (1987)CrossRefGoogle Scholar
  2. 2.
    Lee, W.K., Stolfo, S.J.: A Data Mining Framework for Building Intrusion Detection Model. In: Gong, L., Reiter, M.K. (eds.) Proceedings of the IEEE Symposium on Security and Privacy, Oakland, CA, pp. 120–132. IEEE Computer Society Press, Los Alamitos (1999)Google Scholar
  3. 3.
    Mirkovic, J., Reiher, P.: A Taxonomy of DDoS Attack and DDoS Defense Mechanisms. ACM SIGCOMM Computer Communications Review 34(2), 39–54 (2004)CrossRefGoogle Scholar
  4. 4.
    Chang, R.K.C.: Defending against flooding-based, distributed denial-of-service attacks: A tutorial. IEEE Communications Magazine 40(10), 42–51 (2002)CrossRefGoogle Scholar
  5. 5.
    Feinstein, L., Schnackenberg, D.: Statistical Approaches to DDoS Attack Detection and Response. In: Proceedings of the DARPA Information Survivability Conference and Expostion(DISCEX’03), pp. 303–314 (2003)Google Scholar
  6. 6.
    Noh, S., et al.: Using Inductive Learning for the Detection of Distributed Denial of Service Attacks. In: Liu, J., Cheung, Y.-m., Yin, H. (eds.) IDEAL 2003. LNCS, vol. 2690, pp. 286–295. Springer, Heidelberg (2003)Google Scholar
  7. 7.
    Jin, S., Yeung, D.S.: A Covariance Analysis Model for DDoS Attack Detection. In: Proc. of the Int’l Conf. on Communications, pp. 1882–1886. IEEE Computer Society Press, Los Alamitos (2004)Google Scholar
  8. 8.
    Li, M.: An Approach to Reliably Identifying Signs of DDoS Flood Attacks Based on LRD Traffic Pattern Recognition. Computer and Security 23(7), 549–558 (2004)CrossRefGoogle Scholar
  9. 9.
    Seo, J., Lee, C., Moon, J.: Defending DDoS Attacks Using Network Traffic Analysis and Probabilistic Packet Drop. In: Jin, H., et al. (eds.) GCC 2004. LNCS, vol. 3252, pp. 390–397. Springer, Heidelberg (2004)Google Scholar
  10. 10.
    Mirkovic, J., Prier, G., Reiher, P.: Attacking DDoS at the Source. In: Proceedings of International Conference on Network Protocols, Paris, France, pp. 312–321 (2002)Google Scholar
  11. 11.
    Peng, T., Leckie, C., Kotagiri, R.: Proactively Detecting Distributed Denial of Service Attacks Using Source IP Ad-dress Monitoring. In: Mitrou, N.M., et al. (eds.) NETWORKING 2004. LNCS, vol. 3042, pp. 771–782. Springer, Heidelberg (2004)Google Scholar
  12. 12.
    Rabiner, L.R.: A Tutorial on Hidden Markov Models and Selected Applications in Speech Recognition. Proceedings of the IEEE 77(2), 257–286 (1986)CrossRefGoogle Scholar
  13. 13.
    Paxson, V.: An Analysis of Using Reflectors for Distributed Denial-of-Service Attacks. Computer Communication Review 31(3), 76–89 (2001)CrossRefGoogle Scholar
  14. 14.
    Peng, T., Leckie, C., Kotagiri, R.: Detecting Distributed Denial of Service Attacks by Sharing Distributed Beliefs. In: Safavi-Naini, R., Seberry, J. (eds.) ACISP 2003. LNCS, vol. 2727, pp. 214–225. Springer, Heidelberg (2003)CrossRefGoogle Scholar
  15. 15.
    Jung, J., Krishnamurthy, B., Rabinovich, M.: Flash Crowds and Denial of Service Attacks: Characterization and Implications for CDNs and Web Sites. In: Proceeding of 11th Word Wide Web conference, Honolulu, Hawaii, USA (2002)Google Scholar
  16. 16.
    Sutton, R., Barto, A.G.: Reinforcement Learning: An Introduction. MIT Press, Cambridge (1998)Google Scholar
  17. 17.
    Hu, J., Wellman, M.P.: Multiagent Reinforcement Learning: Theoretical Framework and an Algorithm. In: 15th Intl Conference on Machine Learning, pp. 242–250 (1998)Google Scholar
  18. 18.
    Xu, X.: A Reinforcement Learning Approach for Host-Based Intrusion Detection Using Sequences of System Calls. In: Huang, D.-S., Zhang, X.-P., Huang, G.-B. (eds.) ICIC 2005. LNCS, vol. 3644, pp. 995–1003. Springer, Heidelberg (2005)CrossRefGoogle Scholar

Copyright information

© Springer Berlin Heidelberg 2007

Authors and Affiliations

  • Xin Xu
    • 1
  • Yongqiang Sun
    • 2
  • Zunguo Huang
    • 2
  1. 1.Institute of Automation, National University of Defense Technology, 410073, ChangshaP.R. China
  2. 2.School of Computer, National University of Defense Technology, 410073, ChangshaP.R. China

Personalised recommendations