Skip to main content

Advertisement

SpringerLink
Log in

On optimizing the charging trajectory of mobile chargers in wireless sensor networks: a deep reinforcement learning approach

  • Original Paper
  • Published:
Wireless Networks Aims and scope Submit manuscript

Abstract

Wireless rechargeable sensor networks (WRSNs) are broadly utilized in numerous areas. However, the limited battery capacity of sensor nodes (SNs) is considered as a critical issue. To extend the battery life of SNs, mobile chargers (MCs) equipped with wireless power transfer (WPT) technology have been proposed as a key solution for charging SNs. Using directional antennas to focus energy within a specific area, as opposed to an omnidirectional antenna, increases the energy efficiency of an MC. In this paper, we focus on the travel path charging scheduling problem with a directional MC in on-demand WRSNs. Our goals are to develop a mechanism to reduce the changing delay time and boost the energy efficiency of MC. In this case, the MC receives the charging requests of SNs and responds to them by selecting appropriate stopping points (SPs) and the charging orientation angles in each SP. We propose a mobile directional charging scheduling (MDCS) solution based on a deep reinforcement learning technique. The simulation results demonstrate the superior performance of our method to existing studies in terms of the energy consumption of the MC, the number of dead SNs, and charging delay time.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8

Similar content being viewed by others

References

  1. Rajasekaran, M., Yassine, A., Hossain, M. S., Alhamid, M. F., & Guizani, M. (2019). Autonomous monitoring in healthcare environment: Reward-based energy charging mechanism for IoMT wireless sensing nodes. Future Generation Computer Systems, 98, 565–576.

    Article  Google Scholar 

  2. Sumi, F., Dutta, L., & Sarker, F. (2018). Future with Wireless Power Transfer Technology. J Electr Electron Syst, 7(279), 2332–2796.

    Google Scholar 

  3. Tashtarian, F., Sohraby, K., & Varasteh, A. (2017). Multihop data gathering in wireless sensor networks with a mobile sink. International Journal of Communication Systems, 30(12), e3264.

    Article  Google Scholar 

  4. Lu, X., Wang, P., Niyato, D., Kim, D. I., & Han, Z. (2015). Wireless charging technologies: Fundamental standards and network applications. IEEE Communications Surveys & Tutorials, 18(2), 1413–1452.

    Article  Google Scholar 

  5. Engmann, F., Katsriku, F. A., Abdulai, J.-D., Adu-Manu, K. S., & Banaseka, F. K. (2018). Prolonging the lifetime of wireless sensor networks: a review of current techniques. Wireless Communications and Mobile Computing. https://doi.org/10.1155/2018/8035065

    Article  Google Scholar 

  6. Kurs, A., Karalis, A., Moffatt, R., Joannopoulos, J. D., Fisher, P., & Soljačić, M. (2007). Wireless power transfer via strongly coupled magnetic resonances. Science, 317(5834), 83–86.

    Article  MathSciNet  Google Scholar 

  7. Peng, Y., Li, Z., Zhang, W., & Qiao, D. (2010). Prolonging sensor network lifetime through wireless charging. In 2010 31st IEEE real-time systems Symposium, (pp. 129–139).

  8. Xie, L., Shi, Y., Hou, Y. T., & Lou, A. (2013). Wireless power transfer and applications to sensor networks. IEEE Wireless Communications, 20(4), 140–145.

    Article  Google Scholar 

  9. Xie, L., Shi, Y., Hou, Y. T., & Sherali, H. D. (2012). Making sensor networks immortal: An energy-renewal approach with wireless power transfer. IEEE/ACM Transactions on Networking, 20(6), 1748–1761.

    Article  Google Scholar 

  10. Kaswan, A., Tomar, A., & Jana, P. K. (2018). An efficient scheduling scheme for mobile charger in on-demand wireless rechargeable sensor networks. Journal of Network and Computer Applications, 114, 123–134.

    Article  Google Scholar 

  11. Ding, Z., Zhong, C., Ng, D. W. K., Peng, M., Suraweera, H. A., Schober, R., & Poor, H. V. (2015). Application of smart antenna technologies in simultaneous wireless information and power transfer. IEEE Communications Magazine, 53(4), 86–93.

    Article  Google Scholar 

  12. Xu, X., Chen, L., & Cheng, Z. (2019). Optimizing charging efficiency and maintaining sensor network perpetually in mobile directional charging. Sensors, 19(12), 2657.

    Article  Google Scholar 

  13. Lin, C., Zhou, Y., Ma, F., Deng, J., Wang, L., and Wu, G. ( 2019). Minimizing charging delay for directional charging in wireless rechargeable sensor networks. In IEEE INFOCOM Conference on computer communications, (pp. 1819–1827).

  14. Wang, X., Dai, H., Huang, H., Liu, Y., Chen, G., and Dou, W. (2019). Robust scheduling for wireless charger networks. In IEEE INFOCOM Conference on computer communications.(pp. 2323–2331).

  15. Ding, X., Wang, Y., Sun, G., Luo, C., Li, D., Chen, W., & Hu, Q. (2020). Optimal charger placement for wireless power transfer. Computer Networks, 170, 107123.

    Article  Google Scholar 

  16. Yu, N., Dai, H., Liu, A. X., and Tian, B. (2018). Placement of connected wireless chargers. In IEEE INFOCOM 2018-IEEE Conference on computer communications. (pp. 387–395).

  17. Lin, C., Yang, Z., Dai, H., Cui, L., Wang, L., & Wu, G. (2021). Minimizing charging delay for directional charging. IEEE/ACM Transactions on Networking., 29(6), 2478–2493.

    Article  Google Scholar 

  18. Nowrozian, N., & Tashtarian, F. (2021). A mobile charger based on wireless power transfer technologies: a survey of concepts, techniques, challenges, and applications on rechargeable wireless sensor networks. Journal of AI and Data Mining, 9(3), 383–402.

    Google Scholar 

  19. Kaswan, A., Jana, P. K., & Das, S. K. (2022). A survey on mobile charging techniques in wireless rechargeable sensor networks. IEEE Communications Surveys & Tutorials, 24(3), 1750–1779.

    Article  Google Scholar 

  20. He, L., Zhuang, Y., Pan, J., and Xu, J. (2010). Evaluating on-demand data collection with mobile elements in wireless sensor networks. In 2010 IEEE 72nd Vehicular Technology Conference-Fall, (pp. 1–5).

  21. He, L., Kong, L., Gu, Y., Pan, J., & Zhu, T. (2014). Evaluating the on-demand mobile charging in wireless sensor networks. IEEE Transactions on Mobile Computing, 14(9), 1861–1875.

    Article  Google Scholar 

  22. Ye, X., & Liang, W. (2017). Charging utility maximization in wireless rechargeable sensor networks. Wireless Networks, 23(7), 2069–2081.

    Article  Google Scholar 

  23. Ma, Y., Liang, W., & Xu, W. (2018). Charging utility maximization in wireless rechargeable sensor networks by charging multiple sensors simultaneously. IEEE/ACM Transactions on Networking, 26(4), 1591–1604.

    Article  Google Scholar 

  24. Huang, H., Lin, S., Chen, L., Gao, J., Mamat, A., & Wu, J. (2015). Dynamic mobile charger scheduling in heterogeneous wireless sensor networks. In 2015 IEEE 12th International Conference on mobile ad hoc and sensor systems, (pp. 379–387).

  25. Lyu, Z., Wei, Z., Pan, J., Chen, H., Xia, C., Han, J., & Shi, L. (2019). Periodic charging planning for a mobile WCE in wireless rechargeable sensor networks based on hybrid PSO and GA algorithm. Applied Soft Computing, 75, 388–403.

    Article  Google Scholar 

  26. Tomar, A., & Jana, P. K. (2019). Mobile charging of wireless sensor networks for internet of things: a multi-attribute decision making approach. In International Conference on Distributed Computing and Internet Technology, (pp. 309–324). Cham: Springer.

  27. Cormen, T. H. (2001). Introduction to algorithms. The MIT Press.

    Google Scholar 

  28. Shannon, C. E. (1948). A mathematical theory of communication. The Bell System Technical Journal, 27(3), 379–423.

    Article  MathSciNet  Google Scholar 

  29. Hwang, C. L., & Yoon, K. (1981). Methods for multiple attribute decision making. Multiple attribute decision making (pp. 58–191). Springer.

    Chapter  Google Scholar 

  30. Cao, X., Xu, W., Liu, X., Peng, J., & Liu, T. (2021). A deep reinforcement learning-based on-demand charging algorithm for wireless rechargeable sensor networks. Ad Hoc Networks, 110, 102278.

    Article  Google Scholar 

  31. Wei, Z., Liu, F., Lyu, Z., Ding, X., Shi, L., & Xia, C. (2018). Reinforcement learning for a novel mobile charging strategy in wireless rechargeable sensor networks. In International Conference on Wireless Algorithms, Systems, and Applications (pp. 485–496). Cham: Springer

  32. Wei, Z., Li, M., Wei, Z., Cheng, L., Lyu, Z., & Liu, F. (2020). A novel on-demand charging strategy based on swarm reinforcement learning in WRSNs. IEEE Access, 8, 84258–84271.

    Article  Google Scholar 

  33. Le Nguyen, P., Nguyen, T. H., & Nguyen, K. (2020). Qlearning-based, optimized on-demand charging algorithm in WRSN. In 2020 IEEE 19th International Symposium on Network Computing and Applications (NCA). (pp. 1–8)

  34. Nguyen, P. L., La, V. Q., Nguyen, A. D., Nguyen, T. H., & Nguyen, K. (2021). An on-demand charging for connected target coverage in WRSNs using fuzzy logic and Q-Learning. Sensors, 21(16), 5520.

    Article  Google Scholar 

  35. Banoth, S. P. R., Donta, P. K., & Amgoth, T. (2021). Dynamic mobile charger scheduling with partial charging strategy for WSNs using deep-Qnetworks. Neural Computing and Applications, 33(22), 15267–15279.

    Article  Google Scholar 

  36. Lee, C., Na, W., Jang, G., Lee, C., & Cho, S. (2020). Energy-efficient and delay-minimizing charging method with a multiple directional mobile charger. IEEE Internet of Things Journal, 8(10), 8291–8303.

    Article  Google Scholar 

  37. Riccardo Bonetto, V.L., (2020) Chapter 8–Machine learning. In Computing in Communication Networks.

  38. Zeng, X. (2019). Reinforcement learning based approach for the navigation of a pipe-inspection robot at sharp pipe corners. Master’s thesis, University of Twente.

  39. He, S., Chen, J., Jiang, F., Yau, D. K., Xing, G., & Sun, Y. (2012). Energy provisioning in wireless rechargeable sensor networks. IEEE Transactions on Mobile Computing, 12(10), 1931–1942.

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Computer Engineering, Mashhad Branch, Islamic Azad University, Mashhad, Iran

    Newsha Nowrozian, Farzad Tashtarian & Yahya Forghani

Authors
  1. Newsha Nowrozian
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Farzad Tashtarian
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Yahya Forghani
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Farzad Tashtarian.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Nowrozian, N., Tashtarian, F. & Forghani, Y. On optimizing the charging trajectory of mobile chargers in wireless sensor networks: a deep reinforcement learning approach. Wireless Netw 30, 421–436 (2024). https://doi.org/10.1007/s11276-023-03384-8

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11276-023-03384-8

Keywords

Search

Navigation