Skip to main content
SpringerLink
Log in

Intelligent and resizable control plane for software defined vehicular network: a deep reinforcement learning approach

  • Published:
Telecommunication Systems Aims and scope Submit manuscript

Abstract

Software-defined networking (SDN) has become one of the most promising paradigms to manage large scale networks. Distributing the SDN control proved its performance in terms of resiliency and scalability. However, the choice of the number of controllers to use remains problematic. A large number of controllers may be oversized inducing an overhead in the investment cost and the synchronization cost in terms of delay and traffic load. However, a small number of controllers may be insufficient to achieve the objective of the distributed approach. So, the number of used controllers should be tuned in function of the traffic charge and application requirements. In this paper, we present an intelligent and resizable control plane for software defined vehicular network architecture, where SDN capabilities coupled with deep reinforcement learning (DRL) allow achieving better QoS for vehicular applications. Interacting with SDVN, DRL agent decides the optimal number of distributed controllers to deploy according to the network environment (number of vehicles, load, speed etc.). To the best of our knowledge, this is the first work that adjusts the number of controllers by learning from the vehicular environment dynamicity. Experimental results proved that our proposed system outperforms static distributed SDVN architecture in terms of end-to-end delay and packet loss.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14

Similar content being viewed by others

References

  1. Bannour, F., Souihi, S., & Mellouk, A. (2018). Distributed SDN control: Survey, taxonomy, and challenges. IEEE Communications Surveys & Tutorials, 20(1), 333–354. https://doi.org/10.1109/COMST.2017.2782482.

    Article  Google Scholar 

  2. Fan, Y., & Zhang, N. (2017). A survey on software-defined vehicular networks. Journal of Computers, 28(4), 236–244. https://doi.org/10.3966/199115592017062803025.

    Article  Google Scholar 

  3. Smida, K., Tounsi, H., Frikha, M., & Song, Y. (2019). Software Defined Internet of Vehicles: a survey from QoS and scalability perspectives. In 15th international wireless communications mobile computing conference (IWCMC) (pp. 1349–1354). https://doi.org/10.1109/IWCMC.2019.8766647.

  4. Kaul, A., Obraczka, K., Santos, M. A. S., Rothenberg, C. E., & Turletti, T. (2017). Dynamically distributed network control for message dissemination in ITS. In IEEE/ACM 21st international symposium on distributed simulation and real time applications (DS-RT) (pp. 1–9). https://doi.org/10.1109/DISTRA.2017.8167677.

  5. Smida, K. , Tounsi, H., Frikha, M., & Song, Y. (2019). Delay study in multi-controller software defined vehicular network using OpenDaylight for emergency applications. In 15th international wireless communications mobile computing conference (IWCMC) (pp. 615–620). https://doi.org/10.1109/IWCMC.2019.8766633.

  6. Kumari, A., & Sairam, A. S. (2021). Controller placement problem in software-defined networking: A survey. Networks. https://doi.org/10.1002/net.22016

  7. Liu, W., Wang, Y., Zhang, J., Liao, H., Liang, Z., & Liu, X. (2020). AAMcon: An adaptively distributed SDN controller in data center networks. Frontiers of Computer Science, 14(1), 146–161. https://doi.org/10.1007/s11704-019-7266-6.

    Article  Google Scholar 

  8. Dixit, A. A., Hao, F., Mukherjee, S., Lakshman, T. V., & Kompella, R. (2014). ElastiCon: An elastic distributed SDN controller. In Proceedings of the tenth ACM/IEEE symposium on Architectures for networking and communications systems—ANCS ’14, Los Angeles, California, USA (pp. 17–28). https://doi.org/10.1145/2658260.2658261.

  9. Boutaba, R., et al. (2018). A comprehensive survey on machine learning for networking: Evolution, applications and research opportunities. Journal of Internet Services and Applications, 9(1), 16. https://doi.org/10.1186/s13174-018-0087-2.

    Article  Google Scholar 

  10. Arulkumaran, K., Deisenroth, M. P., Brundage, M., & Bharath, A. A. (2017). Deep reinforcement learning: A brief survey. IEEE Signal Processing Magazine, 34(6), 26–38. https://doi.org/10.1109/MSP.2017.2743240.

    Article  Google Scholar 

  11. Ye, H., & Li, G. Y. (2018). Deep reinforcement learning for resource allocation in V2V communications. In IEEE international conference on communications (ICC) (pp. 1–6). https://doi.org/10.1109/ICC.2018.8422586.

  12. Zhang, Z., Ma, L., Poularakis, K., Leung, K., & Wu, L. (2019). DQ scheduler: Deep reinforcement learning based controller synchronization in distributed SDN. arXiv:1812.00852, 1-7. https://doi.org/10.1109/ICC.2019.8761183.

  13. Open Networking Foundation. Software-Defined Networking (SDN) Definition. Retrieved mai 10, 2020, from https://www.opennetworking.org/sdn-definition/.

  14. Open Networking Foundation. OpenFlow protocol Archives. Retrieved May 09, 2020, from https://www.opennetworking.org/tag/openflow-protocol/.

  15. IBM Cloud Education. (2021). REST APIs , Retrieved August 02, 2021, from https://www.ibm.com/cloud/learn/rest-apis.

  16. Zhang, T., Bianco, A., & Giaccone, P. (2016). The role of inter-controller traffic in SDN controllers placement. In IEEE conference on network function virtualization and software defined networks (NFV-SDN) (pp. 87–92). https://doi.org/10.1109/NFV-SDN.2016.7919481.

  17. Brewer, E. (2012). Pushing the CAP: Strategies for consistency and availability. Computers, 2(45), 23–29. https://doi.org/10.1109/MC.2012.37.

    Article  Google Scholar 

  18. OpenDaylight. Retrieved July 26, 2020, from https://www.opendaylight.org/.

  19. Open Network Operating System (ONOS) SDN Controller for SDN/NFV Solutions, Open Networking Foundation. Retrieved July 26, 2020, from https://www.opennetworking.org/onos/.

  20. Ongaro, D., & Ousterhout, J. (2014). In Search of an Understandable Consensus Algorithm. In Proceedings of the 2014 USENIX conference on USENIX Annual Technical Conference (pp. 305–320).

  21. Sakic, E., & Kellerer, W. (2018). Response time and availability study of RAFT consensus in distributed SDN control plane. IEEE Transactions on Network and Service Management, 15(1), 304–318. https://doi.org/10.1109/TNSM.2017.2775061.

    Article  Google Scholar 

  22. Network Topology State - ONOS - Wiki. Retrieved July 29, 2020, from https://wiki.onosproject.org/display/ONOS/.

  23. Jiacheng, C., Haibo, Z., Ning, Z., Peng, Y., Lin, G., & Xuemin, S. (2016). Software defined Internet of vehicles: Architecture, challenges and solutions. Journal of Communications and Information Networks, 1, 14–26. https://doi.org/10.1007/BF03391543.

    Article  Google Scholar 

  24. Erickson, D. (2013). The Beacon OpenFlow Controller (p. 18). https://doi.org/10.1145/2491185.2491189.

  25. Voellmy, A., & Wang, J. (2012). Scalable software defined network controllers. ACM SIGCOMM Computer Communication Review, 42, 289–290. https://doi.org/10.1145/2377677.2377735.

    Article  Google Scholar 

  26. Kalupahana Liyanage, K. S., Ma, M., & Joo Chong, P. H. (2018). Controller placement optimization in hierarchical distributed software defined vehicular networks. Computer Networks, 135, 226–239. https://doi.org/10.1016/j.comnet.2018.02.022.

    Article  Google Scholar 

  27. Toufga, S., Abdellatif, S., Assouane, H. T., Owezarski, P., & Villemur, T. (2020). Towards dynamic controller placement in software defined vehicular networks. Sensors, 20(6), 1701. https://doi.org/10.3390/s20061701.

    Article  Google Scholar 

  28. An Overview of USDOT Connected Vehicle Roadside Unit Research Activities. (2017).Retrieved jul 28, 2020, from https://connectedautomateddriving.eu/wp-content/uploads/2017/08/USDOT.pdf.

  29. Chai, R., Yuan, Q., Zhu, L., & Chen, Q. (2019). Control plane delay minimization-based capacitated controller placement algorithm for SDN. EURASIP Journal on Wireless Communications and Networking, 1, 282. https://doi.org/10.1186/s13638-019-1607-x.

    Article  Google Scholar 

  30. Wang, G., Zhao, Y., Huang, J., Duan, Q., & Li, J. (2016). A K-means-based network partition algorithm for controller placement in software defined network. In IEEE International Conference on Communications (ICC), (pp. 1–6). https://doi.org/10.1109/ICC.2016.7511441.

  31. Alenazi, M. (2019). Distributed SDN deployment in backbone networks for low-delay and high-reliability applications. International Journal of Advanced Computer Science and Applications. https://doi.org/10.14569/IJACSA.2019.0101274.

    Article  Google Scholar 

  32. Huang, V., Chen, G., Fu, Q., & Wen, E. (2019). Optimizing controller placement for software-defined networks. In IFIP/IEEE symposium on integrated network and service management (IM) (pp. 224–232).

  33. Alowa, A., & Fevens, T. (2020). Towards minimum inter-controller delay time in software defined networking. Procedia Computer Science, 175, 395–402. https://doi.org/10.1016/j.procs.2020.07.056.

    Article  Google Scholar 

  34. Zhang, Z., Ma, L., Poularakis, K., Leung, K. K., Tucker, J., & Swami, A. (2019). MACS: Deep reinforcement learning based SDN controller synchronization policy design. Arxiv190909063 Cs. Retrieved jul 17, 2020, from arXiv:1909.09063.

  35. Xie, J., et al. (2019). A survey of machine learning techniques applied to software defined networking (SDN): Research issues and challenges. IEEE Communications Surveys & Tutorials, 21(1), 393–430. https://doi.org/10.1109/COMST.2018.2866942.

    Article  Google Scholar 

  36. Mestres, A., et al. (2017). Knowledge-defined networking. ACM SIGCOMM Computer Communication Review, 47(3), 2–10. https://doi.org/10.1145/3138808.3138810.

    Article  Google Scholar 

  37. Yu, C., Lan, J., Guo, Z., & Hu, Y. (2018). DROM: Optimizing the routing in software-defined networks with deep reinforcement learning. IEEE Access, PP, 1-1. https://doi.org/10.1109/ACCESS.2018.2877686.

  38. Guo, X., Lin, H., Li, Z., & Peng, M. (2019). Deep reinforcement learning based QoS-aware secure routing for SDN-IoT. IEEE Internet Things Journal. https://doi.org/10.1109/JIOT.2019.2960033.

    Article  Google Scholar 

  39. Zhang, D., Yu, F. R., Yang, R., & Tang, H. (2018). A Deep Reinforcement Learning-based Trust Management Scheme for Software-defined Vehicular Networks. In Proceedings of the 8th ACM symposium on design and analysis of intelligent vehicular networks and applications, Montreal, QC, Canada, (pp. 1–7). https://doi.org/10.1145/3272036.3272037.

  40. Latah, M., & Toker, L. (2019). Artificial intelligence enabled software defined networking: A comprehensive overview. IET Network, 8(2), 79–99. https://doi.org/10.1049/iet-net.2018.5082.

    Article  Google Scholar 

  41. Watkins, C. J. C. H., & Dayan, P. (1992). Q-learning. Machine Learning, 8(3), 279–292. https://doi.org/10.1007/BF00992698.

    Article  Google Scholar 

  42. Arulkumaran, K., Deisenroth, M. P., Brundage, M., & Bharath, A. A. (2017). A brief survey of deep reinforcement learning. ArXiv. https://doi.org/10.1109/MSP.2017.2743240

  43. Suh, D., Jang, S., Han, S., Pack, S., Kim, T., & Kwak, J. (2016). On performance of OpenDaylight clustering, In IEEE NetSoft Conference and Workshops (NetSoft) (pp. 407–410). https://doi.org/10.1109/NETSOFT.2016.7502476.

  44. Smida, K., Tounsi, H., Frikha, M., & Song, Y. Q. (2020). Efficient SDN controller for safety applications in SDN-Based vehicular networks: POX, floodlight, ONOS or OpenDaylight?. In IEEE eighth international conference on communications and networking (ComNet) (pp. 1–6). https://doi.org/10.1109/ComNet47917.2020.9306095.

  45. Mnih, V., et al. (2015). Human-level control through deep reinforcement learning. Nature, 518, 7540. https://doi.org/10.1038/nature14236.

    Article  Google Scholar 

  46. Neghabi, A. A., Jafari Navimipour, N., Hosseinzadeh, M., & Rezaee, A. (2018). Load balancing mechanisms in the software defined networks: A systematic and comprehensive review of the literature. IEEE Access, 6, 14159–14178. https://doi.org/10.1109/ACCESS.2018.2805842.

    Article  Google Scholar 

  47. Mininet-WiFi: Emulating software-defined wireless networks. Retrieved Feb 23, 2019, from https://www.researchgate.net/publication/295861311_Mininet-WiFi_Emulating_software defined_wireless_networks.

  48. Behrisch, M., Bieker, L., Erdmann, J., & Krajzewicz, D. (2011). SUMO—Simulation of urban mobility an overview. In The third international conference on advances in system simulation.

  49. Iperf2. SourceForge. (2019). Retrieved Feb 23, 2019, from https://sourceforge.net/projects/iperf2/.

  50. Empowering App Development for Developers | Docker. Retrieved jul 29, 2020, from https://www.docker.com/.

  51. Service requirements for V2X services. (2017). Retrieved Jan 22, 2021, from https://www.etsi.org/deliver/etsi_ts/122100_122199/122185/14.03.00_60/ts_122185v140300p.pf.

  52. Intelligent Transport Systems (ITS); Vehicular Communications; Basic Set of Applications;(2013). Retrieved Aug 11, 2020, from https://www.etsi.org/deliver/etsi_en/302600_302699/30263703/01.02.00_20/en_30263703v010200a.pdf.

Download references

Author information

Authors and Affiliations

  1. Carthage University, Sup’com, Carthage, Tunisia

    Karima Smida, Hajer Tounsi & Mounir Frikha

  2. King Faisal University (KFU), Al Hofuf, Saudi Arabia

    Mounir Frikha

Authors
  1. Karima Smida
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Hajer Tounsi
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Mounir Frikha
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding authors

Correspondence to Karima Smida, Hajer Tounsi or Mounir Frikha.

Ethics declarations

Conflict of interest

The authors declare that there are no conflicts of interest and competing interests.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Smida, K., Tounsi, H. & Frikha, M. Intelligent and resizable control plane for software defined vehicular network: a deep reinforcement learning approach. Telecommun Syst 79, 163–180 (2022). https://doi.org/10.1007/s11235-021-00838-2

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11235-021-00838-2

Keywords

Search

Navigation