Abstract
Wireless sensor network (WSN) is an evergreen research area, which always looks for energy efficiency. The main challenge for attaining energy efficiency in WSN is data transmission. It is well-explored that a good routing algorithm helps in achieving energy efficiency and contributes to enhance the network lifetime. Inspired by the potential of reinforcement learning (RL), this article presents an RL based routing algorithm for WSN, which frames routes by considering the current status of the network. This results in the detection of optimal routes, such that the transmission delay is minimized and the reliability is increased by the choice of reward functions. Understanding the importance of reward functions, this work selects three trustworthy reward functions for the computation of Q-value. The performance of the proposed work is analyzed and compared with the existing approaches to justify the effectiveness of the proposed routing technique in terms of packet delivery rate, average latency, energy consumption and network lifetime.
Similar content being viewed by others
Data availability
Data sharing is not applicable to this article as no new data were created or analyzed in this study.
References
Zhang, Y., Qiu, M., Tsai, C. W., Hassan, M. M., & Alamri, A. (2017). Health-CPS: Healthcare cyber-physical system assisted by cloud and big data. IEEE Systems Journal, 11(1), 88–95.
Younus, M. U., Islam, S. U., Ali, I., Khan, S., & Khan, M. K. (2019). A survey on software dened networking enabled smart buildings. Architecture, challenges and use cases. Journal of Network and Computer Applications, 137, 62–77.
Al-Aubidy, K. M., Derbas, A. M., & Al-Mutairi, A. W. (Mar. 2016) Real-time patient health monitoring and alarming using wireless-sensor-network. In Proc. 13th Int. Multi-Conf. Syst., Signals Devices (SSD), pp. 416–423.
Usman, M. (2018). Analysis of the impact of different parameter settings on wireless sensor network lifetime. International Journal of Advanced Computer Science and Applications, 9(3), 16–21.
Chen, Y. R., Rezapour, A., Tzeng, W. G., & Tsai, S. C. (2020). RL-Routing: An SDN routing algorithm based on deep reinforcement learning. IEEE Transactions on Network Science and Engineering, 7(4), 3185–3199.
Mohajer, A., Daliri, M. S., Mirzaei, A., Ziaeddini, A., Nabipour, M., & Bavaghar, M. (2022). Heterogeneous computational resource allocation for NOMA: Toward green mobile edge-computing systems. IEEE Transactions on Services Computing. https://doi.org/10.1109/TSC.2022.3186099
Shao, Y., Rezaee, A., Liew, S. C., & Chan, V. W. (2020). Significant sampling for shortest path routing: A deep reinforcement learning solution. IEEE Journal on Selected Areas in Communications, 38(10), 2234–2248.
Younus, M. U., Khan, M. K., Anjum, M. R., Afridi, S., Arain, Z. A., & Jamali, A. A. (2020). Optimizing the lifetime of software defined wireless sensor network via reinforcement learning. IEEE Access., 9, 259–272.
Ding, R., Xu, Y., Gao, F., Shen, X., & Wu, W. (2019). Deep reinforcement learning for router selection in network with heavy traffic. IEEE Access, 7, 37109–37120.
Yu, C., Lan, J., Guo, Z., & Hu, Y. (2018). DROM: Optimizing the routing in software-defined networks with deep reinforcement learning. IEEE Access, 6, 64533–64539.
Tsai, K. C., Zhuang, Z., Lent, R., Wang, J., Qi, Q., Wang, L. C., & Han, Z. (2021). Tensor-based reinforcement learning for network routing. IEEE Journal of Selected Topics in Signal Processing, 15(3), 617–629.
Casas-Velasco, D. M., Rendon, O. M. C., & da Fonseca, N. L. (2020). Intelligent routing based on reinforcement learning for software-defined networking. IEEE Transactions on Network and Service Management, 18(1), 870–881.
Luong, N. C., Hoang, D. T., Gong, S., Niyato, D., Wang, P., Liang, Y. C., & Kim, D. I. (2019). Applications of deep reinforcement learning in communications and networking: A survey. IEEE Communications Surveys & Tutorials, 21(4), 3133–3174.
Suárez-Varela, J., Mestres, A., Yu, J., Kuang, L., Feng, H., Cabellos-Aparicio, A., & Barlet-Ros, P. (2019). Routing in optical transport networks with deep reinforcement learning. Journal of Optical Communications and Networking, 11(11), 547–558.
Al-Tous, H., & Barhumi, I. (2020). Reinforcement learning framework for delay sensitive energy harvesting wireless sensor networks. IEEE Sensors Journal, 21(5), 7103–7113.
Nikjoo, F., Mirzaei, A., & Mohajer, A. (2018). A novel approach to efficient resource allocation in NOMA heterogeneous networks: Multi-criteria green resource management. Applied Artificial Intelligence, 32(7–8), 583–612.
Jha, A., Singh, K.K., Devi, K.V., & Manjula, V. (2021). Reinforcement learning based weighted multipath routing for datacenter networks. Materials Today: Proceedings.
Wang, D., Liu, J., Yao, D., & Member, I. E. E. E. (2020). An energy-efficient distributed adaptive cooperative routing based on reinforcement learning in wireless multimedia sensor networks. Computer Networks, 178, 107313.
Silva, M. A. L., de Souza, S. R., Souza, M. J. F., & Bazzan, A. L. C. (2019). A reinforcement learning-based multi-agent framework applied for solving routing and scheduling problems. Expert Systems with Applications, 131, 148–171.
Rolla, V. G., & Curado, M. (2013). A reinforcement learning-based routing for delay tolerant networks. Engineering Applications of Artificial Intelligence, 26(10), 2243–2250.
Liu, W. X., Cai, J., Chen, Q. C., & Wang, Y. (2021). DRL-R: Deep reinforcement learning approach for intelligent routing in software-defined data-center networks. Journal of Network and Computer Applications, 177, 102865.
Chettibi, S., & Chikhi, S. (2016). Dynamic fuzzy logic and reinforcement learning for adaptive energy efficient routing in mobile ad-hoc networks. Applied Soft Computing, 38, 321–328.
Qu, C., Gai, W., Zhong, M., & Zhang, J. (2020). A novel reinforcement learning based grey wolf optimizer algorithm for unmanned aerial vehicles (UAVs) path planning. Applied Soft Computing, 89, 106099.
Mohajer, A., Sorouri, F., Mirzaei, A., Ziaeddini, A., Rad, K. J., & Bavaghar, M. (2022). Energy-aware hierarchical resource management and Backhaul traffic optimization in heterogeneous cellular networks. IEEE Systems Journal. https://doi.org/10.1109/JSYST.2022.3154162
Mahmood, T., Li, J., Pei, Y., Akhtar, F., Butt, S. A., Ditta, A., & Qureshi, S. (2022). An intelligent fault detection approach based on reinforcement learning system in wireless sensor network. The Journal of Supercomputing, 78(3), 3646–3675.
Yun, W. K., & Yoo, S. J. (2021). Q-learning-based data-aggregation-aware energy-efficient routing protocol for wireless sensor networks. IEEE Access, 9, 10737–10750.
Senthil, T., & Kannapiran, B. (2017). ECTMRA: Energy conserving trustworthy multipath routing algorithm based on cuckoo search algorithm. Wireless Personal Communications, 94, 2239–2258. https://doi.org/10.1007/s11277-016-3378-6
Ragavan, P. S., & Ramasamy, K. (2020). Software defined networking approach based efficient routing in multihop and relay surveillance using Lion Optimization algorithm. Computer Communications, 150, 764–770.
Yasotha, S., & Gopalakrishnan, V. (2021). Reliable energy preserving cluster-based routing policy with optimal route selection for wireless sensor networks. International Journal of Enterprise Network Management, 12(3), 221–238.
Funding
None.
Author information
Authors and Affiliations
Corresponding author
Ethics declarations
Conflict of interest
The authors declare that they have no conflict of interest.
Ethical approval
This article does not contain any studies with human participants.
Human or animal rights
This article does not contain any studies with human or animal subjects performed by any of the authors.
Informed Consent
Informed consent was obtained from all individual participants included in the study.
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.
About this article
Cite this article
Prabhu, D., Alageswaran, R. & Miruna Joe Amali, S. Multiple agent based reinforcement learning for energy efficient routing in WSN. Wireless Netw 29, 1787–1797 (2023). https://doi.org/10.1007/s11276-022-03198-0
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11276-022-03198-0