Skip to main content
SpringerLink
Log in

RI-RPL: a new high-quality RPL-based routing protocol using Q-learning algorithm

  • Published:
The Journal of Supercomputing Aims and scope Submit manuscript

Abstract

The lack of a central controller, severe resource constraints, and multi-path data routing have turned data exchanges into one of the fundamental challenges of the Internet of Things. Despite numerous research efforts on various aspects of routing and data exchanges, some fundamental challenges such as the instant negative impacts of selecting the best possible path and the absence of measures to observe the dynamic conditions of nodes still exist. This study introduces a method called RI-RPL, based on the development of the RPL routing protocol, along with the use of reinforcement learning to address these challenges effectively. To achieve this, RI-RPL is designed in three general stages. In the first stage, routers are aligned with optimizing the RPL protocol with a focus on the Q-learning algorithm. In the second stage, based on learning and convergence, changes in the parents’ learning in different network conditions are supported. In the third stage, control and management changes are coordinated. The reason for choosing this algorithm is its ability to address the desired challenges effectively without wasting network resources for calculations. Simulation results using the Cooja software show that the proposed RI-RPL method, compared to similar recent methods such as ELBRP, RLQRPL, and RPL, has improved successful delivery rates by 4.03%, 13.26%, and 28.87%, respectively, for end-to-end delay by 3.04%, 9.82%, and 13.12%, respectively, for energy consumption optimization by 10.43%, 28.91%, and 36.35%, respectively, for throughput by 10.23%, 28.45%, and 46.88%, respectively, and for network data loss rate by 15.06%, 34.95%, and 49.66%, respectively.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15
Fig. 16
Fig. 17
Fig. 18
Fig. 19
Fig. 20
Fig. 21
Fig. 22
Fig. 23
Fig. 24
Fig. 25
Fig. 26
Fig. 27
Fig. 28
Fig. 29
Fig. 30
Fig. 31
Fig. 32
Fig. 33
Fig. 34
Fig. 35
Fig. 36

Similar content being viewed by others

Notes

  1. Routing Protocol for Low-Power and Lossy Networks.

  2. Routing Over Low Power and Lossy.

  3. Internet Engineering Task Force.

  4. Energy-Aware Grid-based Data Aggregation Scheme in Routing.

  5. Energy-Aware RPL.

  6. A New Load Balancing Objective Function for Low-Power and Lossy Networks.

  7. Composite Routing Technique in IoT Application Networks.

  8. Lightweight Load Balancing and Route Minimizing Solution for RPL.

  9. Forwarding Traffic Consciousness Objective Function for RPL Routing Protocol.

  10. QoS‑Centric Fault‑Resilient Routing Protocol for Mobile‑WSN-Based Low-Power and Lossy Networks.

  11. Weighted Random Forward RPL for High Traffic and Energy Demanding Scenarios.

  12. Rank Remain Energy RPL.

  13. Enhance-Minimum Rank with Hysteresis Objective Function.

  14. RPL Powered by Laplacian Energy for Stable Path Selection During Link Failures in an Internet of Things Network.

  15. Fuzzy Logic Approach for Routing in Internet of Things Network.

  16. Congestion and QoS Aware RPL for IoT Applications Under Heavy Traffic.

  17. Improvement of Minimum Rank Hysteresis Objective Function.

  18. Reliable Link Quality-Based RPL Routing.

  19. Link Quality-Based Objective Function.

  20. Energy-Efficient Priority-Based Multi-Objective QoS Routing.

  21. Fuzzy Logic Objective Function.

  22. An Effective Routing Algorithm for Low-Power and Lossy Networks Using Multi-Criteria Decision-Making Techniques.

  23. Vlse Kriterijumsk Optimizacija Kompromisno Resenje.

  24. Analytical Hierarchy Process.

  25. Energy and Load Balancing Routing Protocol for IoT.

  26. Reinforcement Learning-Based RPL Routing Protocol.

  27. Point-To-Point.

  28. Point-To-Multipoint.

  29. Multipoint-To-Point.

References

  1. Osorio A, Calle M, Soto JD, Candelo-Becerra JE (2020) Routing in LoRaWAN: overview and challenges. IEEE Commun Mag 58(6):72–76

    Article  Google Scholar 

  2. Bhuiyan MN, Rahman MM, Billah MM, Saha D (2021) Internet of things (IoT): a review of its enabling technologies in healthcare applications, standards protocols, security, and market opportunities. IEEE Internet Things J 8(13):10474–10498

    Article  Google Scholar 

  3. Tightiz L, Yang H (2020) A comprehensive review on IoT protocols’ features in smart grid communication. Energies (Basel) 13(11):2762

    Article  Google Scholar 

  4. Oleiwi HW, Al-Raweshidy H (2023) Cooperative Hybrid-NOMA/Dynamic SWIPT-Pairing Mechanism for 6G THz Communications, in 2023 5th Global Power, Energy and Communication Conference (GPECOM), IEEE, pp 524–529

  5. Hassan R, Qamar F, Hasan MK, Aman AHM, Ahmed AS (2020) Internet of Things and its applications: a comprehensive survey. Symmetry (Basel) 12(10):1674

    Article  Google Scholar 

  6. Aslam S, Michaelides MP, Herodotou H (2020) Internet of ships: a survey on architectures, emerging applications, and challenges. IEEE Internet Things J 7(10):9714–9727

    Article  Google Scholar 

  7. Souri A, Hussien A, Hoseyninezhad M, Norouzi M (2022) A systematic review of IoT communication strategies for an efficient smart environment. Trans Emerg Telecommun Technol 33(3):e3736

    Article  Google Scholar 

  8. Chandnani N, Khairnar CN (2020) A comprehensive review and performance evaluation of recent trends for data aggregation and routing techniques in IoT networks. Social Networking and Computational Intelligence: Proceedings of SCI-2018, pp 467–484

  9. Oleiwi HW, Al-Raweshidy H (2022) SWIPT-Pairing mechanism for channel-aware cooperative H-NOMA in 6G Terahertz communications. Sensors 22(16):6200

    Article  Google Scholar 

  10. Marietta J, Chandra Mohan B (2020) A review on routing in internet of things. Wirel Pers Commun 111:209–233

    Article  Google Scholar 

  11. Gopika D, Panjanathan R (2020) Withdrawn: energy efficient routing protocols for WSN based IoT applications: a review. Elsevier

    Book  Google Scholar 

  12. Sari RF, Rosyidi L, Susilo B, Asvial M (2021) A comprehensive review on network protocol design for autonomic internet of things. Information 12(8):292

    Article  Google Scholar 

  13. Dey AJ, Sarma HKD (2020) Routing techniques in internet of things: a review. Trends in Communication, Cloud, and Big Data: Proceedings of 3rd National Conference on CCB, 2018, Springer, pp 41–50

  14. Gaddour O, Koubâa A (2012) RPL in a nutshell: a survey. Comput Netw 56(14):3163–3178

    Article  Google Scholar 

  15. Kushalnagar N, Montenegro G, Schumacher C (2007) IPv6 over low-power wireless personal area networks (6LoWPANs): overview, assumptions, problem statement, and goals

  16. Shah Z, Levula A, Khurshid K, Ahmed J, Ullah I, Singh S (2021) Routing protocols for mobile Internet of things (IoT): a survey on challenges and solutions. Electronics (Basel) 10(19):2320

    Google Scholar 

  17. Lamaazi H, Benamar N (2020) A comprehensive survey on enhancements and limitations of the RPL protocol: a focus on the objective function. Ad Hoc Netw 96:102001

    Article  Google Scholar 

  18. Thubert P (2012) Objective function zero for the routing protocol for low-power and lossy networks (RPL)

  19. Gnawali O, Levis P (2012) The minimum rank with hysteresis objective function

  20. Pancaroglu D, Sen S (2021) Load balancing for RPL-based internet of things: a review. Ad Hoc Netw 116:102491

    Article  Google Scholar 

  21. Sankar S, Srinivasan P, Luhach AK, Somula R, Chilamkurti N (2020) Energy-aware grid-based data aggregation scheme in routing protocol for agricultural internet of things. Sustain Comput Inform Syst 28:100422

    Google Scholar 

  22. Touzene A, Al Kalbani A, Day K, Al Zidi N (2020) Performance analysis of a new energy-aware RPL routing objective function for internet of things. 2020 International Conference on Electrical, Communication, and Computer Engineering (ICECCE), IEEE, pp 1–6

  23. Rana PJ, Bhandari KS, Zhang K, Cho G (2020) EBOF: a new load balancing objective function for low-power and lossy networks. IEIE Trans Smart Process Comput 9(3):244–251

    Article  Google Scholar 

  24. Mishra SN, Elappila M, Chinara S (2020) Eha-rpl: a composite routing technique in iot application networks. First International Conference on Sustainable Technologies for Computational Intelligence: Proceedings of ICTSCI 2019, Springer, pp 645–657

  25. Seyfollahi A, Ghaffari A (2020) A lightweight load balancing and route minimizing solution for routing protocol for low-power and lossy networks. Comput Netw 179:107368

    Article  Google Scholar 

  26. Hassani AE, Sahel A, Badri A (2021) FTC-OF: forwarding traffic consciousness objective function for RPL routing protocol. Int J Electr Electron Eng Telecommun 10:168–175

    Google Scholar 

  27. Eloudrhiri Hassani A, Sahel A, Badri A (2021) IRH-OF: a new objective function for RPL routing protocol in IoT applications. Wirel Pers Commun 119:673–689

    Article  Google Scholar 

  28. Acevedo PD, Jabba D, Sanmartín P, Valle S, Nino-Ruiz ED (2021) WRF-RPL: weighted random forward RPL for high traffic and energy demanding scenarios. IEEE Access 9:60163–60174

    Article  Google Scholar 

  29. Wang H, Fan Z, He X, Li P, Zhang C (2021) Improvement of RPL routing strategy based on 6LoWPAN. Sensor Networks and Signal Processing: Proceedings of the 2nd Sensor Networks and Signal Processing (SNSP 2019), 19–22 November 2019, Hualien, Taiwan, Springer, pp 21–35

  30. Zarzoor AR (2021) Optimizing RPL performance based on the selection of best route between child and root node using E-MHOF method. Int J Electr Comput Eng 11(1):224–231

    Google Scholar 

  31. Pushpalatha M, Anusha T, Rao TR, Venkataraman R (2021) L-RPL: RPL powered by laplacian energy for stable path selection during link failures in an internet of things network. Comput Netw 184:107697

    Article  Google Scholar 

  32. Pingale R, Shinde SN (2021) fuzzy logic approach for routing in internet of things network. Tehnički glasnik 15(1):18–24

    Article  Google Scholar 

  33. Kaviani F, Soltanaghaei M (2022) CQARPL: congestion and QoS-aware RPL for IoT applications under heavy traffic. J Supercomput 78(14):16136–16166

    Article  Google Scholar 

  34. Hassani AE, Sahel A, Badri A (2022) Towards an enhanced minimum rank hysteresis objective function for RPL IoT routing protocol. WITS 2020: Proceedings of the 6th International Conference on Wireless Technologies, Embedded, and Intelligent Systems, Springer, pp 483–493

  35. Charles ASJ, Kalavathi P (2022) A reliable link quality-based RPL routing for internet of things. Soft comput 26(1):123–135

    Article  Google Scholar 

  36. Thenmozhi R, Sakthivel P, Kulothungan K (2022) Hybrid multi-objective-optimization algorithm for energy efficient priority-based QoS routing in IoT networks. Wirel Netw. https://doi.org/10.1007/s11276-021-02848-z

    Article  Google Scholar 

  37. Darabkh KA, Al-Akhras M, Ala’F K, Jafar IF, Jubair F (2022) An innovative RPL objective function for broad range of IoT domains utilizing fuzzy logic and multiple metrics. Expert Syst Appl 205:117593

    Article  Google Scholar 

  38. Fazli F, Mansubbassiri M (2022) V-RPL: an effective routing algorithm for low power and lossy networks using multi-criteria decision-making techniques. Ad Hoc Netw 132:102868

    Article  Google Scholar 

  39. Kalantar S, Jafari M, Hashemipour M (2023) Energy and load balancing routing protocol for IoT. Int J Commun Syst 36(2):e5371

    Article  Google Scholar 

  40. Seyfollahi A, Taami T, Ghaffari A (2023) Towards developing a machine learning-metaheuristic-enhanced energy-sensitive routing framework for the internet of things. Microprocess Microsyst 96:104747

    Article  Google Scholar 

  41. Sutton RS, Barto AG (1998) Introduction to reinforcement learning, vol 135. MIT press, Cambridge

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Faculty of Computer Engineering, Najafabad Branch, Islamic Azad University, Najafabad, Iran

    Niloofar Zahedy, Behrang Barekatain & Alfonso Ariza Quintana

  2. Big Data Research Center, Najafabad Branch, Islamic Azad University, Najafabad, Iran

    Behrang Barekatain

  3. E.T.S.I. Telecomunicación, Universidad de Málaga, Málaga, Spain

    Alfonso Ariza Quintana

Authors
  1. Niloofar Zahedy
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Behrang Barekatain
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Alfonso Ariza Quintana
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

The first two authors worked on all aspects of the paper and the third one checked the simulation process as well as the idea.

Corresponding author

Correspondence to Behrang Barekatain.

Ethics declarations

Competing interests

The authors declare no competing interests.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Zahedy, N., Barekatain, B. & Quintana, A.A. RI-RPL: a new high-quality RPL-based routing protocol using Q-learning algorithm. J Supercomput 80, 7691–7749 (2024). https://doi.org/10.1007/s11227-023-05724-z

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11227-023-05724-z

Keywords

Search

Navigation