Skip to main content
SpringerLink
Log in

Security Aware Virtual Network Embedding Algorithm Based on Reinforcement Learning

  • Chapter
  • First Online:
QoS-Aware Virtual Network Embedding

  • 226 Accesses

Abstract

VNE algorithm is always the key problem in NV technology. At present, the research in this field still has the following problems. The traditional way to solve VNE problem is to use heuristic algorithm. However, this method relies on manual embedding rules, which does not accord with the actual situation of VNE. In addition, as the use of intelligent learning algorithm to solve the problem of VNE has become a trend, this method is gradually outdated. At the same time, there are some security problems in VNE. However, there is no intelligent algorithm to solve the security problem of VNE. For this reason, this chapter proposes a security aware VNE algorithm based on RL. In the training phase, we use a policy network as a learning agent and take the extracted attributes of the substrate nodes to form a feature matrix as input. The learning agent is trained in this environment to get the mapping probability of each substrate node. In the test phase, we map nodes according to the mapping probability and use the breadth-first strategy (BFS) to map links. For the security problem, we add security requirements level constraint for each virtual node and security level constraint for each substrate node. Virtual nodes can only be embedded on substrate nodes that are not lower than the level of security requirements. Experimental results show that the proposed algorithm is superior to other typical algorithms in terms of long-term average return, long-term revenue consumption ratio, and VNR acceptance rate.

â’¸ [2020] IEEE. Reprinted, with permission, from ref. [4].

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 129.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 169.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD 169.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. X. Fu, F.R. Yu, J. Wang, Q. Qi, J. Liao, Service function chain embedding for NFV-enabled IoT based on deep reinforcement learning. IEEE Commun. Mag. 57(1), 102–108 (2019)

    Google Scholar 

  2. J. Du, C. Jiang, Z. Han, H. Zhang, S. Mumtaz, Y. Ren, Contract mechanism and performance analysis for data transaction in mobile social networks. IEEE Trans. Netw. Sci. Eng. 6(2), 103–115 (2019)

    Google Scholar 

  3. P. Zhang, H. Yao, M. Li, Y. Liu, Virtual network embedding based on modified genetic algorithm. Peer-to-Peer Netw. Appl. 2, 1–12 (2017)

    Google Scholar 

  4. P. Zhang, C. Wang, C. Jiang, A. Benslimane, Security-aware virtual network embedding algorithm based on reinforcement learning. IEEE Trans. Netw. Sci. Eng. 1–1 (2020)

    Google Scholar 

  5. P. Zhang, H. Yao, Y. Liu, Virtual network embedding based on computing, network, and storage resource constraints. IEEE Int. Things J. 5(5), 3298–3304 (2018)

    Google Scholar 

  6. H. Cao, S. Wu, Y. Hu, Y. Liu, L. Yang, A survey of embedding algorithm for virtual network embedding. China Commun. 16(12), 1–33 (2019)

    Google Scholar 

  7. C. Jiang, X. Ma, C. Chen, J. Ma, Y. Ren, On searching available channels with asynchronous MAC-layer spectrum sensing. IEICE Trans. Commun. E93-B(8), 2113–2125 (2010)

    Google Scholar 

  8. B. Deng, C. Jiang, H. Yao, S. Guo, S. Zhao, The next generation heterogeneous satellite communication networks: integration of resource management and deep reinforcement learning. IEEE Wirel. Commun. 27(2), 105–111 (2020)

    Google Scholar 

  9. Y. Liu, B. Zhao, P. Zhao, P. Fan, H. Liu, A survey: typical security issues of software-defined networking. China Commun. 16(7), 13–31 (2019)

    Google Scholar 

  10. P. Zhang, C. Wang, Z. Qin, H. Cao, A multidomain virtual network embedding algorithm based on multiobjective optimization for Internet of Drones architecture in Industry 4.0. Softw. Pract. Expert. 1–19 (2020). https://doi.org/10.1002/spe.2815

  11. X. Zhu, C. Jiang, L. Kuang, Z. Zhao, S. Guo, Two-layer game based resource allocation in cloud based integrated terrestrial-satellite networks. IEEE Trans. Cogn. Commun. Netw. 6(2), 509–522 (2020)

    Google Scholar 

  12. P. Zhang, Incorporating energy and load balance into virtual network embedding process. Comput. Commun. 129, 80–88 (2018)

    Google Scholar 

  13. J. Wang, C. Jiang, H. Zhang, Y. Ren, K-C. Chen, L. Hanzo, Thirty years of machine learning: the road to Pareto-Optimal Wireless Networks. IEEE Commun. Surv. Tutorials 22(3), 1472–1514 (2020)

    Google Scholar 

  14. C. Jiang, H. Zhang, Y. Ren, Z. Han, K. Chen, L. Hanzo, Machine learning paradigms for next-generation wireless networks. IEEE Wirel. Commun. 24(2), 98–105 (2017)

    Google Scholar 

  15. S. Xu, Overview of deep reinforcement learning. Comput. Knowl. Technol. 15(3), 193–194 (2019)

    Google Scholar 

  16. K.T.D. Nguyen, C. Huang, An intelligent parallel algorithm for online virtual network embedding, in 2019 International Conference on Computer, Information and Telecommunication Systems (CITS) (2019), pp. 1–5

    Google Scholar 

  17. P. Zhang, H. Li, Y. Ni, F. Gong, M. Wang, F. Wang, Security aware virtual network embedding algorithm using information entropy TOPSIS. J. Netw. Syst. Manag. 28(1), 35–57 (2020)

    Google Scholar 

  18. N.M.M.K. Chowdhury, M.R. Rahman, R. Boutaba, Virtual network embedding with coordinated node and link mapping, in Proceedings of the IEEE INFOCOM[C], Rio de Janeiro (2009), pp. 783–791

    Google Scholar 

  19. S. Shanbhag, A.R. Kandoor, C. Wang, et al., VHub: single-stage virtual network mapping through hub location. Comput. Netw. 77, 169–180 (2015)

    Google Scholar 

  20. M. Yu, Y. Yi, J. Rexford, M. Chiang, Rethinking virtual network embedding: substrate support for path splitting and migration. ACM Sigcomm. Comput. Commun. 38(2), 17–29 (2008)

    Google Scholar 

  21. X. Liu, B. Wang, S. Liu, Z. Yang, Z. Zhao, Heuristic algorithm for secure virtual network embedding. Syst. Eng. Elect. 40(3), 676–681 (2018)

    Google Scholar 

  22. P. Zhang, C. Jiang, X. Pang, Y. Qian, STEC-IoT: a security tactic by virtualizing edge computing on IoT. IEEE Internet Things J. 8(4), 2459–2467 (2021)

    Google Scholar 

  23. S. Gong, J. Chen, C. Huang, Q. Zhu, Trust-aware secure virtual network embedding algorithm. J. Commun. 36(11), 1–10 (2015)

    Google Scholar 

  24. S. Haeri, L. Trajkovic, Virtual network embedding via Monte Carlo tree search. IEEE Trans. Cybern. 48(2), 510–521 (2018)

    Google Scholar 

  25. R. Mijumbi, J.L. Gorricho, J. Serrat, Neural network-based autonomous allocation of resources in virtual networks, in Networks and Communications (EuCNC), 2014 European Conference (2014), pp. 1–6

    Google Scholar 

  26. Y. Kawamoto, H. Takagi, H. Nishiyama, Efficient resource allocation utilizing Q-learning in multiple UA communications. IEEE Trans. Netw. Sci. Eng. (2018) Y. Kawamoto, H. Takagi, H. Nishiyama, N. Kato, Efficient Resource Allocation Utilizing Q-Learning in Multiple UA Communications, IEEE Trans. Netw. Sci. Eng. 6(3), 293–302 (2019). https://doi.org/10.1109/TNSE.2018.2842246

  27. H. Yao, X. Chen, M. Li, P. Zhang, L. Wang, A novel reinforcement learning algorithm for virtual network embedding. Neurocomputing 284, 1–9 (2018)

    Google Scholar 

  28. Q. Li, C. Li, J. Zhang, H. Chen, S. Wang, A review of deep neural network compression. Comput. Sci. 46(9), 1–14 (2019)

    Google Scholar 

  29. J. Du, E. Gelenbe, C. Jiang, H. Zhang, Y. Ren, Contract design for traffic offloading and resource allocation in software defined ultra-dense networks. IEEE J. Select. Areas Commun. 35(11), 2457–2467 (2017)

    Google Scholar 

  30. S. Messaoud, A. Bradai, E. Moulay, Online GMM clustering and mini-batch gradient descent based optimization for industrial IoT 4.0. IEEE Trans. Indust. Inform. 16(2), 1427–1435 (2020)

    Google Scholar 

  31. M. Thomas, E.W. Zegura, Generation and analysis of random graphs to model internetworks. Coll. Comput 63(4), 413–442 (1994)

    Google Scholar 

  32. P. Zhang, S. Wu, M. Wang, H. Yao, Y. Liu, Topology based reliable virtual network embedding from a QoE perspective. China Commun. 15(10), 38–50 (2018)

    Google Scholar 

  33. Q. Yang, T. Jiang, N.C. Beaulieu, J. Wang, C. Jiang, S. Mumtaz, Z. Zhou, Heterogeneous semi-blind interference alignment in finite-snr networks with fairness consideration. IEEE Trans. Wirel. Commun. 19(4), 2472–2488 (2020)

    Google Scholar 

  34. H. Yao, H. Liu, P. Zhang, S. Wu, C. Jiang, S. Guo, A learning-based approach to intra-domain QoS routing. IEEE Trans. Veh. Technol. 69(6), 6718–6730 (2020)

    Google Scholar 

  35. M. Abadi, P. Barham, J. Chen, et al., Tensorflow: a system for large-scale machine learning. OSDI 16, 265–283 (2016)

    Google Scholar 

  36. X. Cheng, S. Su, Z. Zhang, H. Wang, F. Yang, Y. Luo, J. Wang, Virtual network embedding through topology-aware node ranking. ACM SIGCOMM Comput. Commun. 41(2), 38–47 (2011)

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. School of Information Science and Technology, Tsinghua University, Beijing, China

    Chunxiao Jiang

  2. College of Computer Science and Technology, China University of Petroleum (East China), Qingdao, Shandong, China

    Peiying Zhang

Authors
  1. Chunxiao Jiang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Peiying Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

Rights and permissions

Reprints and permissions

Copyright information

© 2021 The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd.

About this chapter

Check for updates. Verify currency and authenticity via CrossMark

Cite this chapter

Jiang, C., Zhang, P. (2021). Security Aware Virtual Network Embedding Algorithm Based on Reinforcement Learning. In: QoS-Aware Virtual Network Embedding. Springer, Singapore. https://doi.org/10.1007/978-981-16-5221-9_4

Download citation

  • DOI: https://doi.org/10.1007/978-981-16-5221-9_4

  • Published:

  • Publisher Name: Springer, Singapore

  • Print ISBN: 978-981-16-5220-2

  • Online ISBN: 978-981-16-5221-9

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation