Abstract
The rising number of users and the demand for more diverse and specialized applications have led to a tremendous increase in network traffic. Managing diverse traffic demands from numerous applications is a challenging task for the existing traditional networking architecture. Hybrid software defined network is widely used to simplify operations providing flexible traffic management and automation. However, managing dynamic traffic demands and routing traffic flows is a challenging task. Therefore, in this paper, a centrality based Q-learning routing traffic engineering method for congestion detection and optimized traffic routing is proposed. The proposed method uses the reinforcement Q-learning algorithm to find an optimal path for routing the traffic. The centrality measures of the nodes are computed and ranked using the simple additive weighted method to detect the top k high-risk nodes. Simulations are carried out under different network scenarios for various traffic profiles. The results show that the proposed method outperformed the existing methods in terms of path length, delay, link utilization, throughput and computation time.
Similar content being viewed by others
References
Ahvar, E., Ahvar, S., Raza, S.M., Vilchez, J.M.S., Lee, G.M.: Next generation of SDN in cloud-fog for 5G and beyond-enabled applications: opportunities and challenges. Network 1, 28–49 (2021). https://doi.org/10.3390/network1010004
Singh, S., Jha, R.K.: A survey on software defined networking: architecture for next generation network. J. Netw. Syst. Manag. 25, 321–374 (2017). https://doi.org/10.1007/s10922-016-9393-9
Vissicchio, S., Vanbever, L., Bonaventure, O.: Opportunities and research challenges of hybrid software defined networks. ACM SIGCOMM Comput. Commun. Rev. 44, 70–75 (2014). https://doi.org/10.1145/2602204.2602216
Kreutz, D., Ramos, F.M., Veríssimo, P.E., Rothenberg, C.E., Azodolmolky, S., Uhlig, S.: Software-defined networking : a comprehensive survey. Proc. IEEE. 103, 14–76 (2015). https://doi.org/10.1109/JPROC.2014.2371999
Ahmad, S., Mir, A.H.: Scalability, consistency, reliability and security in SDN controllers: a survey of diverse SDN controllers. J. Netw. Syst. Manag. (2021). https://doi.org/10.1007/s10922-020-09575-4
Sanvito, D., Filippini, I., Capone, A., Paris, S., Leguay, J.: Clustered robust routing for traffic engineering in software-defined networks. Comput. Commun. 144, 175–187 (2019). https://doi.org/10.1016/j.comcom.2019.06.002
Chen, F., Wu, C., Hong, X., Lu, Z., Wang, Z., Lin, C.: Engineering traffic uncertainty in the OpenFlow data plane. Proc. IEEE INFOCOM. (2016). https://doi.org/10.1109/INFOCOM.2016.752461
Takahashi, Y., Ishibashi, K., Tsujino, M., Kamiyama, N., Shiomoto. K., Otoshi, T., Ohsita, Y., Murata, M.: Separating predictable and unpredictable flows via dynamic flow mining for effective traffic engineering. IEEE Int. Conf. Commun. (2016). https://doi.org/10.1109/ICC.2016.7511371
Özçevik, M.E., Canberk, B., Duong, T.Q.: End to end delay modeling of heterogeneous traffic flows in software defined 5G networks. Ad Hoc Netw. 60, 26–39 (2017). https://doi.org/10.1016/j.adhoc.2017.02.006
Kim, H., Feamster, N.: Improving network management with software defined networking. IEEE Commun. Mag. 51, 114–119 (2013). https://doi.org/10.1109/MCOM.2013.6461195
Amin, R., Reisslein, M., Shah, N.: Hybrid SDN networks: a survey of existing approaches. IEEE Commun. Surv. Tutor. 20(4), 3259–3306 (2018). https://doi.org/10.1109/COMST.2018.2837161
Vissicchio, S., Vanbever, L., Cittadini, L., Xie, G., Bonaventure, O.: Safe updates of hybrid SDN networks. IEEE/ACM Trans. Netw. 25(3), 1649–1662 (2017). https://doi.org/10.1109/TNET.2016.2642586
Casas-Velasco, D.M., Rendon, O.M.C., da Fonseca, N.L.S.: Intelligent routing based on reinforcement learning for software-defined networking. IEEE Trans. Netw. Manag. 18(1), 870–881 (2021). https://doi.org/10.1109/TNSM.2020.3036911
Wang, J., Yang, Y., Xiao, L., Nahrstedt, K.: Edge-based traffic engineering for OSPF networks. Comput. Networks. 48(4), 605–625 (2005). https://doi.org/10.1016/j.comnet.2004.11.008
Chiesa, M., Kindler, G., Schapira, M.: Traffic engineering with equal-cost-multipath: an algorithmic perspective. IEEE/ACM Trans. Netw. 25(2), 779–792 (2017). https://doi.org/10.1109/TNET.2016.2614247
Guo, Y., Wang, Z., Yin, X., Shi, X., Wu, J.: Traffic Engineering in SDN/OSPF Hybrid Network. In: Proc. - IEEE 22nd Int. Conf. on Netw. Protoc. (2014) https://doi.org/10.1109/ICNP.2014.90
Buriol, L.S., Resende, M.G.C., Ribeiro, C.C., Thorup, M.: A hybrid genetic algorithm for the weight setting problem in OSPF/IS-IS routing. Netw. 46(1), 36–56 (2005). https://doi.org/10.1002/net.20070
Fortz, B., Thorup, M.: Internet traffic engineering by optimizing OSPF weights. Internet traffic engineering by optimizing OSPF weights. Proc. IEEE INFOCOM. 2, 519–528 (2000). https://doi.org/10.1109/INFCOM.2000.832225.
Isravel, D.P., Silas, S., Rajsingh, E.B.: Preferential multi-attribute rank based forwarding node selection in software defined networks. In: Proc.- 3rd Int. Conf. Comput. Commun. Technol., pp. 73–79 (2019). https://doi.org/10.1109/ICCCT2.2019.8824798
Mishra, A., Sahoo, A., Dalvi, B., Zhu, T.: WOSPF: a traffic engineering solution for OSPF networks. In: Proc.- IEEE Glob. Commun. Conf. GLOBECOM (2016). https://doi.org/10.1109/GLOCOM.2016.7842393
Shirmarz, A., Ghaffari, A.: An adaptive greedy flow routing algorithm for performance improvement in software—defined network. Int. J. Numer. Model. (2019). https://doi.org/10.1002/jnm.2676
Majidi, A., Jahanbakhsh, N., Gao, X., Zheng, J., Chen, G.: DC-ECN: a machine-learning based dynamic threshold control scheme for ECN marking in DCN. Comput. Commun. (2019). https://doi.org/10.1016/j.comcom.2019.10.028
Wang, H., Xu, H., Huang, L., Wang, J., Yang, X.: Load-balancing routing in software defined networks with multiple controllers. Comput. Netw. 141, 82–91 (2018). https://doi.org/10.1016/j.comnet.2018.05.012
Nasimi, M., Habibi, M.A., Han, B., Schotten, H.D.: Edge-assisted congestion control mechanism for 5G network using software-defined networking. In: Proc. -Int. Symp. Wirel. Commun. Syst. 1–5, (2018). https://doi.org/10.1109/ISWCS.2018.8491233
Handigol, N., Seetharaman, S., McKeown, N., Johari, R.: Plug-n-Serve: load-balancing web traffic using OpenFlow. ACM Sigcomm Demo (2009).
Chu, C.Y., Xi, K., Luo, M., Chao, H.J.: Congestion-aware single link failure recovery in hybrid SDN networks. Proc. IEEE INFOCOM. 26, 1086–1094 (2015). https://doi.org/10.1109/INFOCOM.2015.7218482
Zakia, U., Yedder, H.B.: Dynamic load balancing in SDN-based data center networks. In: IEEE Annu. Inf. Techno., Electron. and Mob. Commun. Conf. (IEMCON), pp. 242–247 (2017). https://doi.org/10.1109/IEMCON.2017.8117206
Tajiki, M.M., Akbari, B., Shojafar, M., Mokari, N.: Joint QoS and congestion control based on traffic prediction in SDN. Appl. Sci. 7(12), 1–15 (2017). https://doi.org/10.3390/app7121265
Le, D.C., Zincir-Heywood, N.: A frontier: dependable, reliable and secure machine learning for network/system management. J. Netw. Syst. Manag. 28, 827–849 (2020). https://doi.org/10.1007/s10922-020-09512-5
Xie, J., Yu, F.R., Huang, T., Xie, R., Liu, J., Wang, C., Liu, Y.: A survey of machine learning techniques applied to software defined networking (SDN): research issues and challenges. IEEE Commun. Surv. Tutorials. 21(1), 393–430 (2019). https://doi.org/10.1109/COMST.2018.2866942
Dong, M., Li, Q., Zarchy, D., Godfrey, P.B., Schapira, M.: PCC: Re-architecting congestion control for consistent high performance. In: Proc. 12th USENIX Symp. Networked Syst. Des. Implement. NSDI '15, pp. 395–408 (2015).
Dong, M., Tong, M., Zarchy, D., Arslan, E., Gilad, Y., Godfrey, B., Schapira, M.: PCC Vivace : online-learning congestion control. In: Proc. 15th USENIX Symp. Networked Syst. Des. Implement. (NSDI '18), pp. 343–356 (2018).
Nguyen, T.T.T., Armitage, G.: A survey of techniques for internet traffic classification using machine learning. IEEE Commun. Surv. Tutorials. 10(4), 56–76 (2008). https://doi.org/10.1109/SURV.2008.080406
Azzouni, A., Boutaba, R., Pujolle, G.: NeuRoute : Predictive dynamic routing for software-defined networks. In: Proc. - 13th Int. Conf. Netw. Serv. Manag. (CNSM), pp. 1–6 (2017). https://doi.org/10.23919/CNSM.2017.8256059
Assefa, B.G., Ozkasap, O.: MER-SDN : machine learning framework for traffic aware energy efficient routing in SDN. In: IEEE 16th Intl Conf on Dependable, Autonomic and Secure Computing, 16th Intl Conf on Pervasive Intelligence and Computing, 4th Intl Conf on Big Data Intelligence and Computing and Cyber Science and Technology Congress(DASC/PiCom/DataCom/CyberSciTech), pp. 74–980 (2018). https://doi.org/10.1109/DASC/PiCom/DataCom/CyberSciTec.2018.000-1
Choi, S.P.M., Yeung, D.-Y.: Predictive Q-routing: a memory-based reinforcement learning approach to adaptive traffic control. Adv. Neural Inf. Process. Syst. 8, 945–951 (1996)
Chavula, J., Densmore, M., Suleman, H.: Using SDN and reinforcement learning for traffic engineering in ubuntunet alliance. In: Proc. Int. Conf. Adv. Comput. Commun. Eng. (ICACCE), pp. 349–355 (2016). https://doi.org/10.1109/ICACCE.2016.8073774
Kong, Y., Zang, H., Ma, X.: Improving TCP congestion control with machine intelligence. In: Proc. Work. Network Meets AI and ML, pp. 60–66 (2018). https://doi.org/10.1145/3229543.3229550
Lin, S., Akyildiz, I. F., Wang, P., Luo, M.: QoS-aware adaptive routing in multi-layer hierarchical software defined networks : a reinforcement learning approach. In: IEEE Int. Conf. Serv. Comput., pp. 25–33 (2016). https://doi.org/10.1109/SCC.2016.12
Yu, C., Lan, J., Guo, Z., Hu, Y.: DROM: optimizing the routing in software-defined networks with deep reinforcement learning. IEEE Access 6, 64533–64539 (2018). https://doi.org/10.1109/ACCESS.2018.2877686
Tessler, C., Shpigelman, Y., Dalal, G., Mandelbaum, A., Kazakov, D.H., Fuhrer, B., Chechik, G., Mannor, S.: Reinforcement learning for datacenter congestion control (2021). arXiv:2102.09337
Bachl, M., Zseby, T., Fabini, J.: Rax: deep reinforcement learning for congestion control. IEEE Int. Conf. Commun. (2019). https://doi.org/10.1109/ICC.2019.8761187
Lei, K., Liang, Y., Li, W.: Congestion control in SDN-based networks via multi-task deep reinforcement learning. IEEE Netw. 34, 28–34 (2020). https://doi.org/10.1109/MNET.011.1900408
Zhang, J., Ye, M., Guo, Z., Yen, C.-Y., Choa, H.J.: CFR-RL: Traffic engineering with reinforcement learning in SDN. IEEE J. Sel. Areas Commun. 38(10), 2249–2259 (2020). https://doi.org/10.1109/JSAC.2020.3000371
Agryzkov, T., Oliver, J.L., Tortosa, L., Vicent, J.: A new betweenness centrality measure based on an algorithm for ranking the nodes of a network. Appl. Math. Comput. 244, 467–478 (2014). https://doi.org/10.1016/j.amc.2014.07.026
Saxena, A., Gera, R., Iyengar, S.R.S.: A faster method to estimate closeness centrality ranking (2017). arXiv:1706.02083
Liben-Nowell, D., Kleinberg, J.: The link-prediction problem for social networks. J. Am. Society Inf. Sci. Technol. 58(7), 1019–1031 (2007)
Moradi, P., Shiri, M.E., Entezari, N.: Automatic skill acquisition in reinforcement learning agents using agents using connection bridge centrality. In: Kim, T., Vasilakos, T., Sakurai, K., Xiao, Y., Zhao, G., Ślęzak, D. (eds) Communication and Networking. FGCN 2010. Commun. Comput. Inf. Sci. 120, pp. 51–62. Springer, Berlin (2010). https://doi.org/10.1007/978-3-642-17604-3
Zhang, J., Luo, Y.: Degree centralityIn, betweenness centrality , and closeness centrality in social network. In: Proc. - 2nd Int. Conf. Model. Simul. Appl. Math., vol. 132, pp. 300–303 (2017). https://doi.org/10.2991/msam-17.2017.68
Okamoto, K., Chen, W., Li, X.Y.: Ranking of closeness centrality for large-scale social networks. In: Preparata, F.P., Wu, X., Yin, J. (eds) Frontiers in Algorithmics. FAW 2008. Lecture Notes in Comput. Sci. 5059, pp. 186–195. Springer, Berlin. https://doi.org/10.1007/978-3-540-69311-6_21
Benzi, M., Klymko, C.: On the limiting behavior of parameter-dependent network centrality measures. SIAM J. Matrix Anal. Appl. 36(2), 686–706 (2015). https://doi.org/10.1137/130950550
Kaliszewski, I., Podkopaev, D.: Simple additive weighting—a metamodel for multiple criteria decision analysis methods. Expert Syst. Appl. 54, 155–161 (2016). https://doi.org/10.1016/j.eswa.2016.01.042
Bhattacharyya, R., Bura, A., Rengarajan, D., Rumuly, M., Shakkottai, S., Kalathil, D., Mok, R.K.P., Dhamdhere, A.: QFlow : A reinforcement learning approach to high QoE video streaming over wireless networks. In: Proc. - 20th ACM Int. Symp. Mobile Ad Hoc Netw. Comput., pp. 251–260 (2019). https://doi.org/10.1145/3323679.3326523
Author information
Authors and Affiliations
Corresponding author
Ethics declarations
Conflict of interest
The authors declare no conflict of interest.
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
About this article
Cite this article
Isravel, D.P., Silas, S. & Rajsingh, E.B. Centrality Based Congestion Detection Using Reinforcement Learning Approach for Traffic Engineering in Hybrid SDN. J Netw Syst Manage 30, 2 (2022). https://doi.org/10.1007/s10922-021-09627-3
Received:
Revised:
Accepted:
Published:
DOI: https://doi.org/10.1007/s10922-021-09627-3