Smart Resource Allocation Using Reinforcement Learning in Content-Centric Cyber-Physical Systems

  • Keke Gai
  • Meikang Qiu
  • Meiqin Liu
  • Hui Zhao
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 10699)


The exponential growing rate of the networking technologies has led to a dramatical large scope of the connected computing environment. As a novel computing deployment, Cyber-Physical Systems (CPSs) are considered an alternative for achieving high performance by the enhanced capabilities in system controls, resource allocations, data exchanges, and flexible adoptions. However, current CPS is encountering the bottleneck concerning the resource allocation due to the mismatching networking service quality and complicated service offering environments. The concept of Quality of Experience (QoE) in networks further increases the demand for intensifying intelligent resource allocations to satisfy distinct user groups in a dynamic manner. This paper concentrates on the issue of resource allocations in CPS and also considers the satisfactory of QoE in content-centric computing systems. A novel approach is proposed by this work, which utilizes the mechanism of reinforcement learning to obtain high accurate QoE in resource allocations. The assessments of the proposed approach were processed by both theoretical proofs and experimental evaluations.


Reinforcement learning Resource allocation Content-centric Cyber-Physical System Smart computing 


  1. 1.
    Xin, S., Guo, Q., Sun, H., Zhang, B., Wang, J., Chen, C.: Cyber-physical modeling and cyber-contingency assessment of hierarchical control systems. IEEE Trans. Smart Grid 6(5), 2375–2385 (2015)CrossRefGoogle Scholar
  2. 2.
    Liu, R., Vellaithurai, C., Biswas, S., Gamage, T., Srivastava, A.: Analyzing the cyber-physical impact of cyber events on the power grid. IEEE Trans. Smart Grid 6(5), 2444–2453 (2015)CrossRefGoogle Scholar
  3. 3.
    Gai, K., Qiu, M., Sun, X.: A survey on FinTech. J. Netw. Comput. Appl. PP, 1 (2017)Google Scholar
  4. 4.
    Qiu, M., Zhong, M., Li, J., Gai, K., Zong, Z.: Phase-change memory optimization for green cloud with genetic algorithm. IEEE Trans. Comput. 64(12), 3528–3540 (2015)MathSciNetCrossRefzbMATHGoogle Scholar
  5. 5.
    Liu, X., Aeron, S., Aggarwal, V., Wang, X., Wu, M.: Adaptive sampling of RF fingerprints for fine-grained indoor localization. IEEE Trans. Mob. Comput. 15(10), 2411–2423 (2016)CrossRefGoogle Scholar
  6. 6.
    Liu, X., Zhu, Y., Kong, L., Liu, C., Gu, Y., Vasilakos, A., Wu, M.: CDC: compressive data collection for wireless sensor networks. IEEE Trans. Parallel Distrib. Syst. 26(8), 2188–2197 (2015)CrossRefGoogle Scholar
  7. 7.
    Gai, K., Qiu, M., Liu, M., Xiong, Z.: In-memory big data analytics under space constraints using dynamic programming. Future Gener. Comput. Syst. PP, 1 (2018)Google Scholar
  8. 8.
    Gai, K., Qiu, M.: Blend arithmetic operations on tensor-based fully homomorphic encryption over real numbers. IEEE Trans. Ind. Inform. PP(99), 1 (2018)Google Scholar
  9. 9.
    Gai, K., Qiu, M., Zhao, H., Sun, X.: Resource management in sustainable cyber-physical systems using heterogeneous cloud computing. IEEE Trans. Sustain. Comput. PP(99), 1–13 (2017)CrossRefGoogle Scholar
  10. 10.
    Gai, K., Qiu, M., Zhao, H.: Privacy-preserving data encryption strategy for big data in mobile cloud computing. IEEE Trans. Big Data PP(99), 1–11 (2017)CrossRefGoogle Scholar
  11. 11.
    Gavalas, D., Konstantopoulos, C., Mastakas, K., Pantziou, G.: A survey on algorithmic approaches for solving tourist trip design problems. J. Heuristics 20(3), 291–328 (2014)CrossRefGoogle Scholar
  12. 12.
    Venugopalan, S., Sinnen, O.: ILP formulations for optimal task scheduling with communication delays on parallel systems. IEEE Trans. Parallel Distrib. Syst. 26(1), 142–151 (2015)CrossRefGoogle Scholar
  13. 13.
    Wang, W., Zhu, K., Ying, L., Tan, J., Zhang, L.: A throughput optimal algorithm for map task scheduling in MapReduce with data locality. ACM SIGMETRICS Perform. Eval. Rev. 40(4), 33–42 (2013)CrossRefGoogle Scholar
  14. 14.
    Ghemawat, S., Gobioff, H., Leung, S.: The Google file system. ACM SIGOPS Oper. Syst. Rev. 37(5), 29–43 (2003)CrossRefGoogle Scholar
  15. 15.
    Gai, K., Qiu, M., Zhao, H.: Cost-aware multimedia data allocation for heterogeneous memory using genetic algorithm in cloud computing. IEEE Trans. Cloud Comput. PP(99), 1–11 (2016)CrossRefGoogle Scholar
  16. 16.
    Gai, K., Qiu, M., Zhao, H.: Energy-aware task assignment for mobile cyber-enabled applications in heterogeneous cloud computing. J. Parallel Distrib. Comput. 111, 126–135 (2018)CrossRefGoogle Scholar
  17. 17.
    Gai, K., Qiu, M., Zhao, H., Tao, L., Zong, Z.: Dynamic energy-aware cloudlet-based mobile cloud computing model for green computing. J. Netw. Comput. Appl. 59, 46–54 (2015)CrossRefGoogle Scholar
  18. 18.
    Liu, Y., Tang, L., Tong, S., Chen, C., Li, D.: Reinforcement learning design-based adaptive tracking control with less learning parameters for nonlinear discrete-time MIMO systems. IEEE Trans. Neural Netw. Learn. Syst. 26(1), 165–176 (2015)MathSciNetCrossRefGoogle Scholar
  19. 19.
    Wei, C., Zhang, Z., Qiao, W., Qu, L.: Reinforcement-learning-based intelligent maximum power point tracking control for wind energy conversion systems. IEEE Trans. Ind. Electron. 62(10), 6360–6370 (2015)CrossRefGoogle Scholar
  20. 20.
    Yin, J., Chen, D., Li, L.: Intelligent train operation algorithms for subway by expert system and reinforcement learning. IEEE Trans. Intell. Transp. Syst. 15(6), 2561–2571 (2014)CrossRefGoogle Scholar

Copyright information

© Springer International Publishing AG 2018

Authors and Affiliations

  1. 1.School of Computer Science and TechnologyBeijing Institute of TechnologyBeijingChina
  2. 2.Department of Computer SciencePace UniversityNew York CityUSA
  3. 3.Shenzhen UniversityGuangdongChina
  4. 4.College of Electrical EngineeringZhejiang UniversityHangzhouChina
  5. 5.Institute of Intelligent Network SystemHenan UniversityKaifengChina

Personalised recommendations