Skip to main content

A novel algorithm for wireless sensor network routing protocols based on reinforcement learning

Abstract

Major challenging problems for wireless sensor networks are the utilization of energy and lifetime routing maximization in the network layer. In wireless sensor network protocols are more critical over data routing in the network. Energy awareness has been described in the context of data-centric, spatial based and categorized protocols. This research paper presents energy consumption analytical analysis based on adoptable routing algorithms based on reinforcement learning using Q-Learning algorithms. Performance comparisons with distributed routing algorithms in the context of the rate of delivery, energy consumption, flow rate, number of packets lost and lifetime of the system were evaluated.

This is a preview of subscription content, access via your institution.

Fig. 1
Fig. 2
Fig. 3

References

  1. Altman E, Kherani AA, Michiardi P, Molva R (2004) Non-cooperative forwarding in ad- hoc networks. In: Proceedings of the 15th IEEE international symposium on personal, indoor and mobile radio communications

  2. Arjunan S, Pothula S (2016) A  survey on unequal clustering protocols in wireless sensor networks. J King Saud Univ Comput Inf Sci

  3. Chand S, Kumar R, Kumar B, Singh S, Malik A (2016) NEECP: novel energy-efficient clustering protocol for prolonging lifetime of WSNs IET Wirel. Sens Syst 6(5):151–157

    Google Scholar 

  4. Chettibi S, Chikhi S (2011) A survey of reinforcement learning based routing proto- cols for mobile Ad-Hoc networks, recent trends in wireless and mobile networks, 162. Communications in Computer and information Science, Springer, pp. 1–13

  5. Chettibi S, Chikhi S (2012) An adaptive energy-aware routing protocol for MANETs Us- ing the SARSA reinforcement lea ing algorithm. Evolving and Adaptive Intelligent Systems (EAIS), IEEE Conference on, 84–89

  6. Chettibi S, Chikhi S (2013) FEA-OLSR: an adaptive energy aware routing protocol for manets using zero-order sugeno fuzzy system. Int J Comput Sci Issues 10(2):136–141

    Google Scholar 

  7. Chettibi S, Chikhi S (2014) Adaptive maximum-lifetime routing in mobile ad-hoc networks using temporal difference reinforcement learning. Evol Syst 5, Springer: Berlin, Heidelberg

  8. Chettibi S, Chikhi S (2016) Dynamic fuzzy (local routing) logic and reinforcement learning for adaptive energy efficient routing in mobile ad-hoc networks. Applied Soft Compu- ting 38:321–328

    Article  Google Scholar 

  9. Cho W, Kim SL (2002) A fully distributed routing algorithm for maximizing life time of a wireless ad hoc network. In: Proc IEEE 4 th Int workshop-mobile & wireless commun. Network, pp. 670–674

  10. Das SK, Tripathi S (2018) Intelligent energy-aware efficient routing for MANET. Wirel Netw 24(4):1139–1159

    Article  Google Scholar 

  11. Devika R, Santhi B, Sivasubramanian T (2013) Survey on routing protocol in wireless sensor network I jet, 5(1):350–356

  12. Heinzelman WR (2000) Application-Specific Protocol Architectures for Wireless Networks, Ph.D. thesis, Massachusetts Institute of Technology

  13. Giordano S (2002) Mobile ad hoc networks. Handbook of wireless networks and mobile computing. pp. 325–346

  14. Kim D, Aceves JJGL, Obraczka, Cano JC, Manzoni P (2002) Power-aware routing based on the energy drain rate for mobile ad-hoc networks. In: 11th international confer-ence on computer communications and networks

  15. Lin H, Chen P, Wang L (2014) Fan-shaped clustering for large-scale sensor networks Proc– 2014 Int Conf Cyber-Enabled Distrib Comput Knowl Discov CyberC. pp. 361–364

  16. Lindsey S, Raghavendra CS (2002) PEGASIS: power-efficient gathering in sensor Information system. In: Proceedings IEEE aerospace conference, 3, Big Sky, MT, pp. 1125–1130

  17. Manjeshwar A, Agrawal DP (2001) TEEN: a protocol for enhanced efficiency in wireless sensor networks. In: The proceedings of the 1st international workshop on parallel and distributed computing issues in wireless networks and mobile computing, San Francisco, CA

  18. Naruephiphat W, Usaha W (2008) Balancing tradeoffs for energy- efficient routing in MA- NETs based on reinforcement learning. In: The IEEE 67th vehicular technology con-ference

  19. Nurmi P (2007) Reinforcement learning for routing in ad-hoc networks. In: Proceedings of the fifth international symposium on modeling and optimization in mobile, Ad-Hoc, and Wireless Networks (WiOpt)

  20. NS (2004) The UCB/LBNL/VINT Network Simulator (NS), http://www.isi.edu/nsnam/ns/

  21. Perkins C, Belding-Royer E, Das S (2003) Ad hoc on-demand distance vector (AODV) Routing. Network Working Group

  22. Rabiner W, Kulik J, Balakrishnan H (1999) Adaptive protocols for information dissemination in wireless sensor networks. In: Proceedings of the fifth annual international conference on mobile computing and networking (MOBICOM), Seattle, WA, USA, pp. 74–185

  23. Ravi G, Kashwan KR (2015) A new routing protocol for energy efficient mobile applica- tions for ad hoc networks. Comput Electr Eng 48:77–85

    Article  Google Scholar 

  24. Srinivasan V, Nuggehalli P, Chiasserini CF, Rao RR (2003) Cooperation in wireless ad hoc networks. In: Proceedings of the 22nd annual joint conference of the IEEE computer and communications societies (INFOCOM). IEEE Computer Society, pp. 808– 817

  25. Sutton R, Barto A (1998) Reinforcement learning. MIT Press: Cambridge, MA

  26. Sutton RS, Barto AG (2014) Reinforcement Learning, Second edition, in progress, MIT Press

  27. Taheri H, Neamatollahi P, Younis OM, Naghibzadeh S, Yaghmaee MH (2012) An energy-aware distributed clustering protocol in wireless sensor networks using fuzzy logic Ad Hoc Netw., 10(7):1469–1481

  28. Vassileva N, Barcelo-Arroyo F (2008) A survey of routing protocols for maximizing the lifetime of Ad Hoc wireless networks. Int J Softw Eng Appl 2(3):77–79

    Google Scholar 

  29. Watkins CJ, Dayan P (1992) Qlearnin. Mach Learn 8: 279–292

    Google Scholar 

  30. Xu Y, Heidemann J, Estrin DD (2001) Geography informed energy conservation for Ad- Hoc routing. In: Proceedings of 7th annual international conference on mobile computing and networking, pp. 70–84

  31. Younis O, Fahmy S (2002) Distributed custering in Ad-hoc sensor networks: a hybrid, energy-efficient approach

  32. Younis O, Fahmy S (2004) Heed: a hybrid, energy-efficient, distributed clustering approach for Ad-hoc networks. IEEE Trans Mob Comput 3(4):366–369

    Article  Google Scholar 

Download references

Funding

The authors did not receive support from any organization for the submitted work.

Author information

Affiliations

Authors

Corresponding author

Correspondence to Purushottam Sharma.

Ethics declarations

Conflict of interest

The authors declare that they have no conflicts of interest to report regarding the present study.

Human and animals participants

This is an observational study. This research includes No involvement of Human and Animals, so no ethical approval is required.

Informed consent

The studies are conducted on already available data for which consent not required.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Yadav, A.K., Sharma, P. & Yadav, R.K. A novel algorithm for wireless sensor network routing protocols based on reinforcement learning. Int J Syst Assur Eng Manag (2021). https://doi.org/10.1007/s13198-021-01414-2

Download citation

Keywords

  • Q-learning
  • Reinforcement learning
  • Wireless sensor networks
  • Routing Protocol