Abstract
In order to improve the energy efficiency of environmental monitoring for energy harvesting wireless sensor networks (EH-WSNs) in remote areas and achieve energy-neutral operation, an adaptive monitoring and energy management optimization method of EH-WSNs based on deep Q network (DQN) algorithm is proposed. In this paper, aiming at EH-WSNs with single-hop cluster structure, we first present a more realistic energy model established by combining different climate characteristics. Then, the optimization model of maximizing long-term monitoring utility is formulated based on harvested energy constraints. We use deep Q network (DQN) to learn random and dynamic solar energy harvesting process on solar-powered sensor nodes and optimize the monitored performance of EH-WSNs through the replay memory mechanism and freezing parameter mechanism. Finally, we present an adaptive monitoring optimization method based DQN to achieve the long-term utility. Through simulation verification and comparative analysis, in different rainy weather environments, the proposed optimization algorithm has greatly improved in terms of average monitoring reward, monitoring interruption rate and energy overflow rate. Moreover, it also indicates that the proposed algorithm has effective adaptation to the random and dynamic solar energy arrival.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Lombardo, L., Corbellini, S., Parvis, M., Elsayed, A., Angelini, E., Grassini, S.: Wireless sensor network for distributed environmental monitoring. IEEE Trans. Instrum. Meas. 67(5), 1214–1222 (2017)
Muduli, L., Mishra, D.P., Jana, P.K.: Application of wireless sensor network for environmental monitoring in underground coal mines: a systematic review. J. Netw. Comput. Appl. 106, 48–67 (2018)
Cao, Y., Ji, R., Ji, L., Lei, G., Wang, H., Shao, X.: l2-MPTCP: a learning-driven latency-aware multipath transport scheme for industrial internet applications. IEEE Transactions on Industrial Informatics (2022)
Cao, Y., Ji, R., Huang, X., Lei, G., Shao, X., You, I.: Empirical Mode Decomposition-empowered Network Traffic Anomaly Detection for Secure Multipath TCP Communications, Mobile Networks and Applications (2022)
Antony, S.M., Indu, S., Pandey, R.: An efficient solar energy harvesting system for wireless sensor network nodes. J. Inf. Optim. Sci. 41(1), 39–50 (2020)
Sun, W., Ding, Z., Qin, Z., Chu, F., Han, Q.: Wind energy harvesting based on fluttering double-flag type triboelectric nanogenerators. Nano Energy 70, 104526 (2020)
Sharma, H., Haque, A., Jaffery, Z.A.: Modeling and optimisation of a solar energy harvesting system for wireless sensor network nodes. J. Sens. Actuator Netw. 7(3), 40 (2018)
Sharma, H., Haque, A., Jaffery, Z.A.: Maximization of wireless sensor network lifetime using solar energy harvesting for smart agriculture monitoring. Ad Hoc Netw. 94, 101966 (2019)
Sarang, S., Drieberg, M., Awang, A., Ahmad, R.: A QoS MAC protocol for prioritized data in energy harvesting wireless sensor networks. Comput. Netw. 144, 141–153 (2018)
Lakshmi, P.S., Jibukumar, M.G., Neenu, V.S.: Network lifetime enhancement of multi-hop wireless sensor network by RF energy harvesting. In: Proceedings of the 2018 International Conference on Information Networking, pp. 738–743 (2018)
Nguyen, H.S., Ly, T.T.H., Nguyen, T.S., Huynh, V.V., Nguyen, T.L., Voznak, M.: Outage performance analysis and SWIPT optimization in energy-harvesting wireless sensor network deploying NOMA. Sensors 19(3), 613 (2019)
Ren, Q., Yao, G.: An energy-efficient cluster head selection scheme for energy-harvesting wireless sensor networks. Sensors 20(1), 187 (2020)
Xiong, Y., Chen, G., Lu, M., Wan, X., Wu, M., She, J.: A two-phase lifetime-enhancing method for hybrid energy-harvesting wireless sensor network. IEEE Sens. J. 20(4), 1934–1946 (2019)
Bengheni, A., Didi, F., Bambrik, I.: EEM-EHWSN: enhanced energy management scheme in energy harvesting wireless sensor networks. Wireless Netw. 25(6), 3029–3046 (2019)
Qiu, C., Hu, Y., Chen, Y., Zeng, B.: Lyapunov optimization for energy harvesting wireless sensor communications. IEEE Internet Things J. 5(3), 1947–1956 (2018)
Lee, P., Eu, Z.A., Han, M., Tan, H.: Empirical modeling of a solar-powered energy harvesting wireless sensor node for time-slotted operation. In: Proceedings of the 2011 IEEE Wireless Communications and Networking Conference, pp. 179–184 (2011)
Fraternali, F., Balaji, B., Agarwal, Y., Gupta, R.K.: Aces: automatic configuration of energy harvesting sensors with reinforcement learning. ACM Trans. Sens. Netw. 16(4), 1–31 (2020)
Tekin, N., Gungor, V.C.: The impact of error control schemes on lifetime of energy harvesting wireless sensor networks in industrial environments. Comput. Stand. Interfaces 70, 103417 (2020)
Han, C., Zhang, S., Zhang, B., Zhou, J., Sun, L.: A distributed image compression scheme for energy harvesting wireless multimedia sensor networks. Sensors 20(3), 667 (2020)
Raja, J., Mookhambika, N.: A novel energy harvesting with middle-order weighted probability (EHMoWP) for performance improvement in wireless sensor network (WSN). J. Ambient Intell. Humaniz. Comput. 1–12 (2021)
Zairi, S., Zouari, B., Niel, E., Dumitrescu, E.: Nodes self-scheduling approach for maximising wireless sensor network lifetime based on remaining energy. IET Wirel. Sens. Syst. 2(1), 52–62 (2012)
Sahoo, J., Sahoo, B.: Solving target coverage problem in wireless sensor networks using greedy approach. In: Proceedings of the 2020 International Conference on Computer Science, Engineering and Applications, pp. 1–4 (2020)
Acknowledgements
This research was supported by the National Natural Science Foundation of China (Grant No. 61961026, 61962036), Natural Science Foundation of Jiangxi Province, China (Grant No. 20202BABL202003), China Postdoctoral Science Foundation (Grant No. 2020M671556), Major science and technology projects in Jiangxi province (20213AAG01012).
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2023 ICST Institute for Computer Sciences, Social Informatics and Telecommunications Engineering
About this paper
Cite this paper
Bao, X., Bian, P., Tan, W., Xu, X., Nie, J. (2023). Adaptive Monitoring Optimization Based on Deep-Q-Network for Energy Harvesting Wireless Sensor Networks. In: Cao, Y., Shao, X. (eds) Mobile Networks and Management. MONAMI 2022. Lecture Notes of the Institute for Computer Sciences, Social Informatics and Telecommunications Engineering, vol 474. Springer, Cham. https://doi.org/10.1007/978-3-031-32443-7_23
Download citation
DOI: https://doi.org/10.1007/978-3-031-32443-7_23
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-32442-0
Online ISBN: 978-3-031-32443-7
eBook Packages: Computer ScienceComputer Science (R0)