Skip to main content

Advertisement

SpringerLink
Log in

EQRSRL: an energy-aware and QoS-based routing schema using reinforcement learning in IoMT

  • Original Paper
  • Published:
Wireless Networks Aims and scope Submit manuscript

Abstract

Internet of Medical Thing (IoMT) is an emerging technology in healthcare that can be used to realize a wide variety of medical applications. It improves people’s quality of life and makes it easier to care for the sick individuals in an efficient and safe manner. To do this, IoMT leverages the capabilities of some new technologies including IoT, Artificial Intelligence, cloud computing, computer networks and medicine. Combining these technologies to monitor the patient’s health conditions in real-time or semi-real-time is a critical challenge in IoMT. In this regard, one of the most crucial components of IoMT are network communication protocols that should provide a fast and reliable communication path between a connected biosensor to a patient and cloud computing environment. In this paper, we propose EQRSRL as an efficient routing mechanism for different types of IoMT applications. The aim of EQRSRL is to provide a reasonable level of Quality of Service (QoS) for IoMT traffics. To achieve this goal, it categorizes the network traffic into three classes and treats them differently concerning their QoS requirements. Moreover, EQRSRL divides the network environment into multiple zones to decrease the number of message exchange between the nodes. In order to compute optimal paths between the nodes, it considers QoS and energy metrics, and makes use of a reinforcement learning approach in path computation process. Simulation results show that the implementation of EQRSRL in IoMT is practical and leads to improvement of 82% in average energy consumption, 25% in end-to-end delay and 7% packet delivery ration in compared to the state-of-the-art routing techniques.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12

Similar content being viewed by others

References

  1. Sisinni, E., Saifullah, A., Han, S., Jennehag, U., & Gidlund, M. (2018). Industrial internet of things: Challenges, opportunities, and directions. IEEE transactions on industrial informatics, 14(11), 4724–4734.

    Google Scholar 

  2. Malik, P. K., Sharma, R., Singh, R., Gehlot, A., Satapathy, S. C., Alnumay, W. S., Pelusi, D., Ghosh, U., & Nayak, J. (2021). Industrial internet of things and its applications in industry 4.0: State of the art. Computer Communications, 166, 125–139.

    Google Scholar 

  3. Ali, A., Zhu, Y., & Zakarya, M. (2021). A data aggregation based approach to exploit dynamic spatio-temporal correlations for citywide crowd flows prediction in fog computing. Multimedia Tools and Applications, 80, 31401–31433.

    Google Scholar 

  4. Zhu, L., Majumdar, S., & Ekenna, C. (2021). An invisible warfare with the internet of battlefield things: A literature review. Human Behavior and Emerging Technologies, 3(2), 255–260.

    Google Scholar 

  5. Xu, G., Shi, Y., Sun, X., & Shen, W. (2019). Internet of things in marine environment monitoring: A review. Sensors, 19(7), 1711.

    Google Scholar 

  6. Nazari, A., Tavassolian, F., Abbasi, M., Mohammadi, R., & Yaryab, P. (2022). An intelligent sdn-based clustering approach for optimizing iot power consumption in smart homes. Wireless Communications and Mobile Computing. https://doi.org/10.1155/2022/8783380

    Article  Google Scholar 

  7. Mehmood, Y., Ahmad, F., Yaqoob, I., Adnane, A., Imran, M., & Guizani, S. (2017). Internet-of-things-based smart cities: Recent advances and challenges. IEEE Communications Magazine, 55(9), 16–24.

    Google Scholar 

  8. Ali, A., Zhu, Y., & Zakarya, M. (2021). Exploiting dynamic spatio-temporal correlations for citywide traffic flow prediction using attention based neural networks. Information Sciences, 577, 852–870.

    MathSciNet  Google Scholar 

  9. Jalili Marandi, S., Golsorkhtabaramiri, M., Hosseinzadeh, M., & Jafarali Jassbi, S. (2022). Iot based thermal aware routing protocols in wireless body area networks: Survey: Iot based thermal aware routing in wban. IET Communications, 16(15), 1753–1771.

    Google Scholar 

  10. Mohammadi, R., Nazari, A., Nassiri, M., & Conti, M. (2021). An sdn-based framework for qos routing in internet of underwater things. Telecommunication Systems, 78(2), 253–266.

    Google Scholar 

  11. Yuehong, Y., Zeng, Y., Chen, X., & Fan, Y. (2016). The internet of things in healthcare: An overview. Journal of Industrial Information Integration, 1, 3–13.

    Google Scholar 

  12. Rahmani, A. M., Bayramov, S., & Kiani Kalejahi, B. (2022). Internet of things applications: Opportunities and threats. Wireless Personal Communications, 122(1), 451–476.

    Google Scholar 

  13. Kumar, R. (2020). Energy efficient dynamic cluster head and routing path selection strategy for wbans. Wireless Personal Communications, 113(1), 33–58.

    Google Scholar 

  14. Singla, R., Kaur, N., Koundal, D., Lashari, S. A., Bhatia, S., & Rahmani, M. K. I. (2021). Optimized energy efficient secure routing protocol for wireless body area network. IEEE Access, 9, 116745–116759.

    Google Scholar 

  15. Movassaghi, S., Abolhasan, M., & Lipman, J. (2013). A review of routing protocols in wireless body area networks. Journal of Networks, 8, 559.

    Google Scholar 

  16. Al Barazanchi, I., Abdulshaheed, H. R., Safiah, M., & Sidek, B. (2020). A survey: Issues and challenges of communication technologies in wban. Sustainable Engineering and Innovation, 1(2), 84–97.

    Google Scholar 

  17. Ahmed, G., Mahmood, D., & Islam, S. (2019). Thermal and energy aware routing in wireless body area networks. International Journal of Distributed Sensor Networks, 15(6), 1550147719854974.

    Google Scholar 

  18. Abdullah, M., & Ehsan, A. (2014). Routing protocols for wireless sensor networks: Classifications and challenges. Journal of Electronics and Communication Engineering Research, 2(2), 05–15.

    Google Scholar 

  19. Yessad, N., Omar, M., Tari, A., & Bouabdallah, A. (2018). Qos-based routing in wireless body area networks: A survey and taxonomy. Computing, 100(3), 245–275.

    MathSciNet  Google Scholar 

  20. Khan, Z. A., Sivakumar, S., Phillips, W., & Robertson, B. (2013). A qos-aware routing protocol for reliability sensitive data in hospital body area networks. Procedia Computer Science, 19, 171–179.

    Google Scholar 

  21. Nadeem, Q., Javaid, N., Mohammad, S.N., Khan, M., Sarfraz, S., & Gull, M. (2013). Simple: Stable increased-throughput multi-hop protocol for link efficiency in wireless body area networks. In 2013 Eighth International Conference on Broadband and Wireless Computing, Communication and Applications, pp. 221–226 . IEEE

  22. Djenouri, D., & Balasingham, I. (2009). New qos and geographical routing in wireless biomedical sensor networks. In 2009 Sixth International Conference on Broadband Communications, Networks, and Systems, pp. 1–8 . IEEE

  23. Razzaque, M. A., Hong, C. S., & Lee, S. (2011). Data-centric multiobjective qos-aware routing protocol for body sensor networks. Sensors, 11(1), 917–937.

    Google Scholar 

  24. Khan, Z., Sivakumar, S., Phillips, W., & Robertson, B. (2012). Qprd: Qos-aware peering routing protocol for delay sensitive data in hospital body area network communication. In 2012 Seventh International Conference on Broadband, Wireless Computing, Communication and Applications, pp. 178–185. IEEE

  25. Liang, X., Balasingham, I., & Byun, S.-S. (2008). A reinforcement learning based routing protocol with qos support for biomedical sensor networks. In 2008 First International Symposium on Applied Sciences on Biomedical and Communication Technologies, pp. 1–5. IEEE

  26. Ahmad, N., Awan, M. D., Khiyal, M. S. H., Babar, M. I., Abdelmaboud, A., Ibrahim, H. A., & Hamed, N. O. (2022). Improved qos aware routing protocol (im-qrp) for wban based healthcare monitoring system. IEEE Access, 10, 121864–121885.

    Google Scholar 

  27. Memon, S., Wang, J., Bhangwar, A. R., Fati, S. M., Rehman, A., Xu, T., & Zhang, L. (2021). Temperature and reliability-aware routing protocol for wireless body area networks. IEEE Access, 9, 140413–140423.

    Google Scholar 

  28. Tang, Q., Tummala, N., Gupta, S.K., & Schwiebert, L. (2005). Tara: Thermal-aware routing algorithm for implanted sensor networks. In International Conference on Distributed Computing in Sensor Systems, pp. 206–217. Springer

  29. Bag, A., & Bassiouni, M.A. (2006). Energy efficient thermal aware routing algorithms for embedded biomedical sensor networks. In 2006 IEEE International Conference on Mobile Ad Hoc and Sensor Systems, pp. 604–609. IEEE

  30. Sodhro, A. H., Li, Y., & Shah, M. A. (2016). Energy-efficient adaptive transmission power control for wireless body area networks. IET Communications, 10(1), 81–90.

    Google Scholar 

  31. Takahashi, D., Xiao, Y., & Hu, F. (2007). Ltrt: Least total-route temperature routing for embedded biomedical sensor networks. In GLOBECOM 2007-IEEE Global Telecommunications Conference,

  32. Ahmed, G., Mehmood, D., Shahzad, K., & Malick, R. A. S. (2021). An efficient routing protocol for internet of medical things focusing hot spot node problem. International Journal of Distributed Sensor Networks, 17(2), 1550147721991706.

    Google Scholar 

  33. Ahmed, O., Ren, F., Hawbani, A., & Al-Sharabi, Y. (2020). Energy optimized congestion control-based temperature aware routing algorithm for software defined wireless body area networks. IEEE Access, 8, 41085–41099.

    Google Scholar 

  34. Selem, E., Fatehy, M., Abd El-Kader, S. M., & Nassar, H. (2019). The (temperature heterogeneity energy) aware routing protocol for iot health application. IEEE Access, 7, 108957–108968.

    Google Scholar 

  35. Selem, E., Fatehy, M., & Abd El-Kader, S. M. (2021). mobthe (mobile temperature heterogeneity energy) aware routing protocol for wban iot health application. IEEE Access, 9, 18692–18705.

    Google Scholar 

  36. Sharma, R., Ryait, H. S., & Gupta, A. K. (2015). Clustering based routing protocol to increase the stability in wban. International Journal of Innovations in Engineering and Technology, 6(1), 119–125.

    Google Scholar 

  37. Singh, K., & Singh, R.K. (2015). An energy efficient fuzzy based adaptive routing protocol for wireless body area network. In 2015 IEEE UP Section Conference on Electrical Computer and Electronics (UPCON) pp. 1–6. IEEE

  38. Watteyne, T., Augé-Blum, I., Dohler, M., & Barthel, D. (2007). Anybody: A self-organization protocol for body area networks. In: BODYNETS, p. 6.

  39. Mu, J., Stewart, R., Han, L., & Crawford, D. (2018). A self-organized dynamic clustering method and its multiple access mechanism for multiple wbans. IEEE Internet of Things Journal, 6(4), 6042–6051.

    Google Scholar 

  40. Anguraj, D. K., & Smys, S. (2019). Trust-based intrusion detection and clustering approach for wireless body area networks. Wireless Personal Communications, 104(1), 1–20.

    Google Scholar 

  41. Arafat, M. Y., Pan, S., & Bak, E. (2023). Distributed energy-efficient clustering and routing for wearable IoT enabled wireless body area networks. IEEE Access, 11, 5047.

    Google Scholar 

  42. Navya, V., & Deepalakshmi, P. (2018). Energy efficient routing for critical physiological parameters in wireless body area networks under mobile emergency scenarios. Computers & Electrical Engineering, 72, 512–525.

    Google Scholar 

  43. Ahmed, S., Javaid, N., Akbar, M., Iqbal, A., Khan, Z.A., & Qasim, U. (2014). Laeeba: Link aware and energy efficient scheme for body area networks. In 2014 IEEE 28th International Conference on Advanced Information Networking and Applications, pp. 435–440. IEEE

  44. Javaid, N., Ahmad, A., Nadeem, Q., Imran, M., & Haider, N. (2015). im-simple: Improved stable increased-throughput multi-hop link efficient routing protocol for wireless body area networks. Computers in Human Behavior, 51, 1003–1011.

    Google Scholar 

  45. Ullah, F., Khan, M. Z., Faisal, M., Rehman, H. U., Abbas, S., & Mubarek, F. S. (2021). An energy efficient and reliable routing scheme to enhance the stability period in wireless body area networks. Computer Communications, 165, 20–32.

    Google Scholar 

  46. Jabbar, A. H., & Alshawi, I. S. (2021). Spider monkey optimization routing protocol for wireless sensor networks. International Journal of Electrical & Computer Engineering, 11(3), 2432.

    Google Scholar 

  47. Halgamuge, M. N., Zukerman, M., Ramamohanarao, K., & Vu, H. L. (2009). An estimation of sensor energy consumption. Progress in Electromagnetics Research B, 12, 259–295.

    Google Scholar 

  48. Tang, W., Ma, X., Wei, J., & Wang, Z. (2019). Measurement and analysis of near-ground propagation models under different terrains for wireless sensor networks. Sensors, 19(8), 1901.

    Google Scholar 

  49. Kurt, S., & Tavli, B. (2017). Path-loss modeling for wireless sensor networks: A review of models and comparative evaluations. IEEE Antennas and Propagation Magazine, 59(1), 18–37.

    Google Scholar 

  50. Jin, Y., Kulkarni, P., Wilcox, J., & Sooriyabandara, M. (2016). A centralized scheduling algorithm for ieee 802.15. 4e tsch based industrial low power wireless networks. In 2016 IEEE Wireless Communications and Networking Conference, pp. 1–6 . IEEE

  51. Choi, K.-H., & Chung, S.-H. (2016). A new centralized link scheduling for 6tisch wireless industrial networks. In Internet of Things, Smart Spaces, and Next Generation Networks and Systems, pp. 360–371. Springer.

  52. Kotsiou, V., Papadopoulos, G. Z., Chatzimisios, P., & Theoleyre, F. (2019). Whitelisting without collisions for centralized scheduling in wireless industrial networks. IEEE Internet of Things Journal, 6(3), 5713–5721.

    Google Scholar 

  53. Mammeri, Z. (2019). Reinforcement learning based routing in networks: Review and classification of approaches. Ieee Access, 7, 55916–55950.

    Google Scholar 

  54. Albertsen, C. M. (2019). Generalizing the first-difference correlated random walk for marine animal movement data. Scientific Reports, 9(1), 1–14.

    Google Scholar 

  55. https://github.com/aminnazari91/eqrsrl.git

Download references

Author information

Authors and Affiliations

  1. Department of Computer Engineering, Bu-ali Sina University, Ahmadi Roshan, Hamedan, Hamedan, 65, Iran

    Amin Nazari, Mojtaba Kordabadi & Reza Mohammadi

  2. Department of Mathematics, University of Delft, Mekelweg 5, 2628, Delft, The Netherlands

    Chhagan Lal

Authors
  1. Amin Nazari
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Mojtaba Kordabadi
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Reza Mohammadi
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Chhagan Lal
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

AN: Conceptualization, Methodology, Software, Original draft preparation, MK: Software, Original draft preparation, RM and CL: Supervision—Conceptualization, Methodology—Reviewing and Editing

Corresponding author

Correspondence to Reza Mohammadi.

Ethics declarations

Conflicts of interest

The authors declare no competing interests.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Nazari, A., Kordabadi, M., Mohammadi, R. et al. EQRSRL: an energy-aware and QoS-based routing schema using reinforcement learning in IoMT. Wireless Netw 29, 3239–3253 (2023). https://doi.org/10.1007/s11276-023-03367-9

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11276-023-03367-9

Keywords

Search

Navigation