Skip to main content

A Hybrid Deep Reinforcement Learning Routing Method Under Dynamic and Complex Traffic with Software Defined Networking

  • Conference paper
  • First Online:
Advanced Information Networking and Applications (AINA 2024)

Abstract

Software-defined networking (SDN) routing based on reinforcement learning (RL) is a very promising research topic in recent years, achieving better solutions comparing to traditional network routing based on mathematical models. However, with continuous increase of network complexity and scale, the RL methods show a slow convergence speed and insufficient adaptability to network changes. This leads to the major drawbacks of existing RL algorithms in modern large-scale networks, especially with complexity and dynamics features. Therefore, this paper proposed a novel RL method based on pre-trained data called PRLR, a pre-trained reinforcement learning based SDN routing method, which can effectively improve the QoS of SDN routing and improve the convergence speed of reinforcement learning. The experimental results demonstrate that our proposed PRLR outperforms the benchmarking methods in terms of multiple metrics, such as network delays, bandwidth availability, goodput ratio, as well as the convergence efficiency and works well in dynamic routing topologies.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 149.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 279.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Moy, J.: “RFC2328: OSPF Version 2.” (1998)

    Google Scholar 

  2. Fadlullah, Z.Md., et al.: State-of-the-art deep learning: evolving machine intelligence toward tomorrow’s intelligent network traffic control systems. IEEE Commun. Surv. Tutor. 19(4), 2432–2455 (2017)

    Google Scholar 

  3. Swain, P., Kamalia, U., Bhandarkar, R., Modi, T.: CoDRL: intelligent packet routing in SDN using convolutional deep reinforcement learning. In: 2019 IEEE International Conference on Advanced Networks and Telecommunications Systems (ANTS), pp. 1–6. IEEE (2019)

    Google Scholar 

  4. Xie, J., et al.: A survey of machine learning techniques applied to software defined networking (SDN): research issues and challenges. IEEE Commun. Surv. Tutor. 21(1), 393–430 (2018)

    Article  Google Scholar 

  5. Amin, R., Rojas, E., Aqdus, A., Ramzan, S., Casillas-Perez, D., Arco, J.M.: A survey on machine learning techniques for routing optimization in SDN. IEEE Access (2021)

    Google Scholar 

  6. Zhang, H., Yan, J.: Performance of SDN routing in comparison with legacy routing protocols. In: 2015 International Conference on Cyber-Enabled Distributed Computing and Knowledge Discovery, pp. 491–494. IEEE (2015)

    Google Scholar 

  7. Caria, M., Jukan, A., Hoffmann, M.: SDN partitioning: a centralized control plane for distributed routing protocols. IEEE Trans. Netw. Serv. Manage. 13(3), 381–393 (2016)

    Article  Google Scholar 

  8. Moy, J.T.: OSPF: Anatomy of an Internet Routing Protocol. Addison-Wesley Professional, Reading (1998)

    Google Scholar 

  9. Dobrijevic, O., Santl, M., Matijasevic, M.: Ant colony optimization for QoE-centric flow routing in software-defined networks. In: 2015 11th International Conference on Network and Service Management (CNSM), pp. 274–278. IEEE (2015)

    Google Scholar 

  10. Gao, Y., Cheng, L., Sang, L., Yang, D.: Spectrum sharing for LTE and WiFi coexistence using decision tree and game theory. In: 2016 IEEE Wireless Communications and Networking Conference, pp. 1–6. IEEE (2016)

    Google Scholar 

  11. Fernández-Fernández, A., Cervelló-Pastor, C., Ochoa-Aday, L.: A multi-objective routing strategy for QoS and energy awareness in software-defined networks. IEEE Commun. Lett. 21(11), 2416–2419 (2017)

    Article  Google Scholar 

  12. Assefa, B.G., Ozkasap, O.: HyMER: a hybrid machine learning framework for energy efficient routing in SDN. arXiv preprint arXiv:1909.08074 (2019)

  13. Kim, S., Son, J., Talukder, A., Hong, C.S.: Congestion prevention mechanism based on Q-leaning for efficient routing in SDN. In: 2016 International Conference on Information Networking (ICOIN), pp. 124–128. IEEE (2016)

    Google Scholar 

  14. Chavula, J., Densmore, M., Suleman, H.: Using SDN and reinforcement learning for traffic engineering in UbuntuNet Alliance. In: 2016 International Conference on Advances in Computing and Communication Engineering (ICACCE), pp. 349–355. IEEE (2016)

    Google Scholar 

Download references

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Ziyang Zhang .

Editor information

Editors and Affiliations

Rights and permissions

Reprints and permissions

Copyright information

© 2024 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Zhang, Z., Guan, L., Meng, Q. (2024). A Hybrid Deep Reinforcement Learning Routing Method Under Dynamic and Complex Traffic with Software Defined Networking. In: Barolli, L. (eds) Advanced Information Networking and Applications. AINA 2024. Lecture Notes on Data Engineering and Communications Technologies, vol 204. Springer, Cham. https://doi.org/10.1007/978-3-031-57942-4_19

Download citation

Publish with us

Policies and ethics