Skip to main content
Log in

Intelligent routing method based on Dueling DQN reinforcement learning and network traffic state prediction in SDN

  • Published:
Wireless Networks Aims and scope Submit manuscript


The traditional routing method makes use of limited information on the network links to make routing decisions, which makes it difficult to adapt to the dynamic and complex network and adjust the router’s forward strategy. To address these issues, this paper proposes an intelligent routing method based on the Software Defined Network (SDN), Dueling DQN (a Deep Reinforcement Learning algorithm) and network traffic state prediction. First, the global network awareness information is obtained with the SDN network measurement mechanism, which is converted into a traffic matrix consisting of multiple network link status information such as bandwidth and delay, etc. Then, the optimal forwarding route under the current network state is generated by predicting the network traffic matrix and the Dueling DQN. The experimental results show that: (1) compared with the traditional Dijkstra and OSPF routing methods, the proposed method significantly improves the network throughput and effectively reduces the network delay and packet loss rate; (2) comparing with the reinforcement learning algorithms DDPG and PPO, the proposed approach achieves a faster convergence state, which improves the efficiency of network routing.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Subscribe and save

Springer+ Basic
EUR 32.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or Ebook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15
Fig. 16
Fig. 17

Similar content being viewed by others


  1. Nunes, B. A. A., Mendonca, M., Nguyen, X. N., Obraczka, K., & Turletti, T. (2014). A survey of software-defined networking: Past, present, and future of programmable networks. IEEE Communications Surveys and Tutorials, 16(3), 1617–1634.

    Article  Google Scholar 

  2. Sun, P., Yu, M., Freedman, M. J., Rexford, J., & Walker, D. (2015). Hone: Joint host-network traffic management in software-defined networks. Journal of Network and Systems Management, 23(2), 374–399.

    Article  Google Scholar 

  3. Guerin, R. A., Orda, A., & Williams, D. (1997). QoS routing mechanisms and OSPF extensions. In GLOBECOM 97. IEEE Global Telecommunications Conference, pp. 1903–1908. IEEE.

  4. Verma, A., & Bhardwaj, N. (2016). A review on routing information protocol (RIP) and open shortest path first (OSPF) routing protocol. International Journal of Future Generation Communication and Networking, 9(4), 161–170.

    Article  Google Scholar 

  5. Ni, W., Huang, C., Wu, J., & Savoie, M. (2013). Availability of survivable Valiant load balancing (VLB) networks over optical networks. Optical Switching and Networking, 10(3), 274–289.

    Article  Google Scholar 

  6. Ibarz, J., Tan, J., Finn, C., Kalakrishnan, M., Pastor, P., & Levine, S. (2021). How to train your robot with deep reinforcement learning: Lessons we have learned. The International Journal of Robotics Research, 40(4–5), 698–721.

    Article  Google Scholar 

  7. LeCun, Y., Bengio, Y., & Hinton, G. (2015). Deep learning. Nature, 521(7553), 436–444.

    Article  Google Scholar 

  8. Botvinick, M., Ritter, S., Wang, J. X., Kurth-Nelson, Z., Blundell, C., & Hassabis, D. (2019). Reinforcement learning, fast and slow. Trends in Cognitive Sciences, 23(5), 408–422.

    Article  Google Scholar 

  9. Sherstinsky, A. (2020). Fundamentals of recurrent neural network (RNN) and long short-term memory (LSTM) network. Physica D: Nonlinear Phenomena, 404, 132306.

    Article  MathSciNet  Google Scholar 

  10. Ahn, C. W., & Ramakrishna, R. S. (2002). A genetic algorithm for shortest path routing problem and the sizing of populations. IEEE Transactions on Evolutionary Computation, 6(6), 566–579.

    Article  Google Scholar 

  11. Derbel, H., Jarboui, B., Hanafi, S., & Chabchoub, H. (2012). Genetic algorithm with iterated local search for solving a location-routing problem. Expert Systems with Applications, 39(3), 2865–2871.

    Article  Google Scholar 

  12. Zhang, D. G., Liu, S., Liu, X. H., Zhang, T., & Cui, Y. Y. (2018). Novel dynamic source routing protocol (DSR) based on genetic algorithm-bacterial foraging optimization (GA-BFO). International Journal of Communication Systems, 31(18), 1–20.

    Article  Google Scholar 

  13. Parsaei, M. R., Mohammadi, R., & Javidan, R. (2017). A new adaptive traffic engineering method for telesurgery using ACO algorithm over software defined networks. European Research in Telemedicine/La Recherche Europeenne en Telemedecine, 6(3–4), 173–180.

    Article  Google Scholar 

  14. Jing, S., Muqing, W., Yong, B., & Min, Z. (2017). An improved GAC routing algorithm based on SDN. IEEE International Conference on Computer and Communications (ICCC), pp. 173–176.

  15. Lin, C., Wang, K., & Deng, G. (2017). A QoS-aware routing in SDN hybrid networks. Procedia Computer Science, 110, 242–249.

    Article  Google Scholar 

  16. Truong Dinh, K., Kukliński, S., Osiński, T., & Wytrębowicz, J. (2020). Heuristic traffic engineering for SDN. Journal of Information and Telecommunication, 4(3), 251–266.

    Article  Google Scholar 

  17. Ke, C. K., Wu, M. Y., Hsu, W. H., & Chen, C. Y. (2019). Discover the optimal IoT packets routing path of software-defined network via artificial bee colony algorithm. In International Wireless Internet Conference, pp. 147–162. Springer, Cham.

  18. Shokouhifar, M. (2021). FH-ACO: Fuzzy heuristic-based ant colony optimization for joint virtual network function placement and routing. Applied Soft Computing, 107, 107401.

    Article  Google Scholar 

  19. Zhang, L., & Lei, Y. (2021). Particle swarm optimization-based information-centric networking intra-domain routing strategy. Internet Technology Letters, 4(1), e196.

    Article  Google Scholar 

  20. Valadarsky, A., Schapira, M., Shahaf, D., & Tamar, A. (2017). Learning to route. In Proceedings of the 16th ACM workshop on hot topics in networks, pp. 185–191.

  21. Sharma, D. K., Dhurandher, S. K., Woungang, I., Srivastava, R. K., Mohananey, A., & Rodrigues, J. J. (2016). A machine learning-based protocol for efficient routing in opportunistic networks. IEEE Systems Journal, 12(3), 2207–2213.

    Article  Google Scholar 

  22. Li, W., Li, G., & Yu, X. (2015). A fast traffic classification method based on SDN network. In The 4th International Conference on Electronics, Communications and Networks, pp. 223–229. Beijing, China.

  23. Zhou, X., Su, M., Liu, Z., Hu, Y., Sun, B., & Feng, G. (2020). Smart tour route planning algorithm based on naïve Bayes interest data mining machine learning. ISPRS International Journal of Geo-Information, 9(2), 112.

    Article  Google Scholar 

  24. Yanjun, L., Xiaobo, L., & Osamu, Y. (2014). Traffic engineering framework with machine learning based meta-layer in software-defined networks. In 2014 4th IEEE International Conference on Network Infrastructure and Digital Content, pp. 121–125. IEEE.

  25. Tang, F., Mao, B., Fadlullah, Z. M., Kato, N., Akashi, O., Inoue, T., & Mizutani, K. (2017). On removing routing protocol from future wireless networks: A real-time deep learning approach for intelligent traffic control. IEEE Wireless Communications, 25(1), 154–160.

    Article  Google Scholar 

  26. Mao, B., Tang, F., Fadlullah, Z. M., & Kato, N. (2019). An intelligent route computation approach based on real-time deep learning strategy for software defined communication systems. IEEE Transactions on Emerging Topics in Computing, 9(3), 1554–1565.

    Article  Google Scholar 

  27. Kato, N., Fadlullah, Z. M., Mao, B., Tang, F., Akashi, O., Inoue, T., & Mizutani, K. (2016). The deep learning vision for heterogeneous network traffic control: Proposal, challenges, and future perspective. IEEE Wireless Communications, 24(3), 146–153.

    Article  Google Scholar 

  28. Hendriks, T., Camelo, M., & Latré, S. (2018). Q 2-routing: A Qos-aware Q-routing algorithm for wireless ad hoc networks. In 2018 14th International Conference on Wireless and Mobile Computing, Networking and Communications (WiMob), pp. 108–115. IEEE.

  29. Chen, T., Gao, X., Liao, T., & Chen, G. (2019). Pache: A packet management scheme of cache in data center networks. IEEE Transactions on Parallel and Distributed Systems, 31(2), 253–265.

    Article  Google Scholar 

  30. Casas-Velasco, D. M., Rendon, O. M. C., & da Fonseca, N. L. (2020). Intelligent routing based on reinforcement learning for software-defined networking. IEEE Transactions on Network and Service Management, 18(1), 870–881.

    Article  Google Scholar 

  31. Jin, Z., Zang, W., Jiang, Y., & Lan, J. (2019). A QLearning based business differentiating routing mechanism in SDN architecture. Journal of Physics: Conference Series, 1168(2), 022025.

    Article  Google Scholar 

  32. Yin, Y., Huang, C., Wu, D. F., Huang, S., Ashraf, M., & Guo, Q. (2021). Reinforcement learning-based routing algorithm in satellite-terrestrial integrated networks. Wireless Communications and Mobile Computing.

    Article  Google Scholar 

  33. Zhao, L., Wang, J., Liu, J., & Kato, N. (2019). Routing for crowd management in smart cities: A deep reinforcement learning perspective. IEEE Communications Magazine, 57(4), 88–93.

    Article  Google Scholar 

  34. Chen, Y. R., Rezapour, A., Tzeng, W. G., & Tsai, S. C. (2020). Rl-routing: An sdn routing algorithm based on deep reinforcement learning. IEEE Transactions on Network Science and Engineering, 7(4), 3185–3199.

    Article  Google Scholar 

  35. Zhang, J., Ye, M., Guo, Z., Yen, C. Y., & Chao, H. J. (2020). CFR-RL: Traffic engineering with reinforcement learning in SDN. IEEE Journal on Selected Areas in Communications, 38(10), 2249–2259.

    Article  Google Scholar 

  36. Fu, Q., Sun, E., Meng, K., Li, M., & Zhang, Y. (2020). Deep Q-learning for routing schemes in SDN-based data center networks. IEEE Access, 8, 103491–103499.

    Article  Google Scholar 

  37. Liu, W. X., Cai, J., Chen, Q. C., & Wang, Y. (2021). DRL-R: Deep reinforcement learning approach for intelligent routing in software-defined data-center networks. Journal of Network and Computer Applications, 177, 102865.

    Article  Google Scholar 

  38. Hossain, M. B., & Wei, J. (2019). Reinforcement learning-driven QoS-aware intelligent routing for software-defined networks. In 2019 IEEE global conference on signal and information processing (GlobalSIP) , pp. 1–5. IEEE.

  39. Yu, C., Lan, J., Guo, Z., & Hu, Y. (2018). DROM: Optimizing the routing in software-defined networks with deep reinforcement learning. IEEE Access, 6, 64533–64539.

    Article  Google Scholar 

  40. Zhang, D., & Kabuka, M. R. (2018). Combining weather condition data to predict traffic flow: A GRU-based deep learning approach. IET Intelligent Transport Systems, 12(7), 578–585.

    Article  Google Scholar 

  41. Hochreiter, S., & Schmidhuber, J. (1997). Long short-term memory. Neural Computation, 9(8), 1735–1780.

    Article  Google Scholar 

  42. Clark, D. D., Partridge, C., Ramming, J. C., & Wroclawski, J. T. (2003). A knowledge plane for the internet. In Proceedings of the 2003 conference on Applications, technologies, architectures, and protocols for computer communications, pp. 3–10.

  43. Mestres, A., Rodriguez-Natal, A., Carner, J., Barlet-Ros, P., Alarcón, E., Solé, M., Muntés-Mulero, V., Meyer, D., Barkai, S., Hibbett, M. J., & Estrada, G. (2017). Knowledge-defined networking. ACM SIGCOMM Computer Communication Review., 47(3), 2–10.

    Article  Google Scholar 

  44. Xue, X., & Huang, Q. (2022). Generative adversarial learning for optimizing ontology alignment. Expert Systems.

    Article  Google Scholar 

  45. Al Shalabi, L., & Shaaban, Z. (2006). Normalization as a preprocessing engine for data mining and the approach of preference matrix. In 2006 International conference on dependability of computer systems, pp. 207–214. IEEE.

  46. Casas-Velasco, D. M., Rendon, O. M. C., & da Fonseca, N. L. (2021). DRSIR: A deep reinforcement learning approach for routing in software-defined networking. IEEE Transactions on Network and Service Management.

    Article  Google Scholar 

  47. Ban, T. W. (2020). An autonomous transmission scheme using dueling DQN for D2D communication networks. IEEE Transactions on Vehicular Technology, 69(12), 16348–16352.

    Article  Google Scholar 

  48. White, S. R., Hanson, J. E., Whalley, I., Chess, D. M., & Kephart, J. O. (2004). An architectural approach to autonomic computing. In International Conference on Autonomic Computing, 2004. Proceedings, pp. 2–9. IEEE.

  49. Mininet. Accessed: Jan. 5, 2021. [Online]. Available:

  50. Ryu. Accessed: Dec. 31, 2020. [Online]. Available:

  51. IPerf. Accessed: Jan. 5, 2021. [Online]. Available:

  52. New York Metro IBX data center data sheet. Accessed: Dec. 31, 2020[Online]Available:

  53. Li, Y., Cai, Z. P., & Xu, H. (2018). LLMP: Exploiting LLDP for latency measurement in software-defined data center networks. Journal of Computer Science and Technology, 33(2), 277–285.

    Article  Google Scholar 

Download references


This work was supported in part by the National Natural Science Foundation of China under Grant No. 62161006, No. 61861013 and No. 61662018, in part by the Science and Technology Major Project of Guangxi No. AA18118031, in part by Guangxi Natural Science Foundation of China under Grant No. 2018GXNSFAA050028, in part by Director Fund project of Key Laboratory of Cognitive Radio and Information Processing of Ministry of Education under Grant No. CRKL190102, and in part by Guangxi Key Laboratory of Wireless Wide band Communication and Signal Processing No. GXKL06220110.

Source code

The experimental code can be accessed at

Author information

Authors and Affiliations


Corresponding author

Correspondence to Miao Ye.

Ethics declarations

Conflict of interest

The authors declare no conflict of interest.

Data availability

The dataset generated during this study by the SDN multi-threaded measurement mechanism designed in this paper through the flow measurement, which includes 1616 flow matrices, can be obtained from the author or accessed at

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Huang, L., Ye, M., Xue, X. et al. Intelligent routing method based on Dueling DQN reinforcement learning and network traffic state prediction in SDN. Wireless Netw 30, 4507–4525 (2024).

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: