Skip to main content
SpringerLink
Log in

TCP-RLLD: TCP with reinforcement learning based loss differentiation for mobile adhoc networks

  • OriginalPaper
  • Published:
Wireless Networks Aims and scope Submit manuscript

Abstract

TCP-Transmission Control Protocol provides connection oriented and reliable communication. TCP’s default consideration of any loss as a cause of network congestion degrades the end-to-end performance in wireless networks specially in MANETs-Mobile Adhoc Networks. TCP should identify various non congestion losses such as channel loss and route failure loss to act accordingly. Over the years, researchers have proposed Machine Learning based network protocols for accurate and efficient decision making. Reinforcement learning suits better for the dynamic networks with unpredictable traffic and topology. TCP-RLLD, TCP with Reinforcement Learning based Loss Differentiation is an end-to-end transport layer solution to predict cause of a packet loss. TCP’s default consideration of any loss as a congestion loss is overruled by TCP-RLLD to avoid unnecessary reduction of the transmission rate. TCP-RLLD is evaluated with multiple TCP variants for the Mobile Adhoc Networks. The extensive evaluation is performed with NS-3 simulator. This paper discusses TCP-RLLD architecture along with the detail of performance improvement.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14

Similar content being viewed by others

References

  1. Mast, N., & Owens, T. J. (2011). A survey of performance enhancement of transmission control protocol (TCP) in wireless adhoc networks. EURASIP Journal on Wireless Communications and Networking, 2011, 1–23.

    Article  Google Scholar 

  2. Al-Jubari, A. M., Othman, M., Mohd Ali, B., & Abdul Hamid, N. A. W. (2011). TCP performance in multi-hop wireless ad hoc networks challenges and solution. EURASIP Journal on Wireless Communications and Networking, 2011, 1–25.

    Article  Google Scholar 

  3. Tsaoussidis, V., & Matta, I. (2002). Open issues on TCP for mobile computing. Journal on Wireless Communications and Mobile Computing, 2, 3–20.

    Article  Google Scholar 

  4. Wang, S., Chaovalitwongse, W., & Babuska, R. (2012). Machine learning algorithms in bipedal robot control. IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews), 42(5), 728–743.

    Article  Google Scholar 

  5. Kotsiantis, S. B. (2007). Supervised machine learning: A review of classification techniques. In Proceedings of the 2007 Conference on Emerging Artificial Intelligence Applications in Computer Engineering: Real Word AI Systems with Applications in eHealth, HCI, Information Retrieval and Pervasive Technologies) (pp. 3–24).

  6. Woergoetter, F., & Porr, B. (2008). Reinforcement learning. Scholarpedia, 3(3), 1448.

    Article  Google Scholar 

  7. Wang, M., Cui, Y., Wang, X., Xiao, S., & Jiang, J. (2018). Machine learning for networking: Workflow, advances and opportunities. IEEE Network, 32(2), 92–99.

    Article  Google Scholar 

  8. Boutaba, Raouf, Salahuddin, Mohammad A., Limam, Noura, Ayoubi, Sara, Shahriar, Nashid, Estrada-Solano, Felipe, & Caicedo, Oscar M. (2018). A comprehensive survey on machine learning for networking: evolution, applications and research opportunities. Journal of Internet Services and Applications, 9, 1–99.

    Article  Google Scholar 

  9. Molia, H. K., & Kothari, A. D. (2019). TCP with machine learning—advances and opportunities. International Journal of Advanced Trends in Computer Science and Engineering, 8, 3526–3534.

    Article  Google Scholar 

  10. Shafiq, M., Tian, Z., Bashir, A. K., Du, X., & Guizani, M. (2021). Corrauc: A malicious bot-IoT traffic detection method in IoT network using machine-learning techniques. IEEE Internet of Things Journal, 8(5), 3242–3254.

    Article  Google Scholar 

  11. Shafiq, M., Tian, Z., Bashir, A. K., Du, X., & Guizani, M. (2020). IoT malicious traffic identification using wrapper-based feature selection mechanisms. Computers and Security, 94, 101863.

    Article  Google Scholar 

  12. Shafiq, M., Tian, Z., Bashir, A. K., Jolfaei, A., & Yu, X. (2020). Data mining and machine learning methods for sustainable smart cities traffic classification: A survey. Sustainable Cities and Society, 60, 102177.

    Article  Google Scholar 

  13. Shafiq, M., Tian, Z., Sun, Y., Xiaojiang, D., & Guizani, M. (2020). Selection of effective machine learning algorithm and bot-IoT attacks traffic identification for internet of things in smart city. Future Generation Computer Systems, 107, 433–442.

    Article  Google Scholar 

  14. Fonseca, N., & Crovella, M. (2005). Bayesian packet loss detection for TCP. IEEE 24th Annual Joint Conference of the IEEE Computer and Communications Societies., 3, 1826–1837.

    Google Scholar 

  15. El Khayat, I., Geurts, P., & Leduc, G. (2010). Enhancement of TCP over wired/wireless networks with packet loss classifiers inferred by supervised learning. Wireless Networks, 16, 273–290.

    Article  Google Scholar 

  16. Winstein, K., & Balakrishnan, H. (2013). Tcp ex machina: Computer-generated congestion control. ACM SIGCOMM Computer Communication Review, 43(4), 123–134.

    Article  Google Scholar 

  17. Kong, Y., Zang, H., & Ma, X. (2018). Improving TCP congestion control with machine intelligence. In Proceedings of the 2018 Workshop on Network Meets AI & ML (pp. 60–66).

  18. Arouche Nunes, B. A., Veenstra, K., Ballenthin, W., Lukin, S., & Obraczka, K. (2014). A machine learning framework for TCP round-trip time estimation. EURASIP Journal on Wireless Communications and Networking, 2014, 1–22.

    Article  Google Scholar 

  19. Alim Al Islam, A. B. M., & Raghunathan, V. (2015). iTCP: An intelligent TCP with neural network based end-to-end congestion control for ad-hoc multi-hop wireless mesh networks. Wireless Networks, 21, 581–610.

  20. Li, W., Zhou, F., Meleis, W. and Chowdhury, K., (2016). Learning-based and data-driven TCP design for memory-constrained IoT. In 2016 International conference on distributed computing in sensor systems (DCOSS) (pp. 199–205). IEEE.

  21. Jiang, H., Luo, Y., Zhang, Q. Y., Yin, M. Y., & Chun, W. (2017). TCP-Gvegas with prediction and adaptation in multi-hop ad hoc networks. Wireless Networks, 23, 1535–1548.

    Article  Google Scholar 

  22. Kumar, P., Tripathi, S., & Pal, P. (2018). Neural network based reliable transport layer protocol for manet. In 4th International conference on recent advances in information technology IIT(ISM), Dhanbad-IEEE Explore (pp. 1–6).

  23. Li, W., Zhou, F., Chowdhury, K. R., & Meleis, W. (2018). QTCP: Adaptive congestion control with reinforcement learning. IEEE Transactions on Network Science and Engineering, 6(3), 445–458.

    Article  Google Scholar 

  24. Molia, H.K. & Kothari, A.D., (2020). Appropriateness of machine learning techniques for TCP with MANETs. In Proceedings of the international conference on communication and intelligent systems, Lecture Notes in Networks and Systems (pp. 503–511). Springer.

  25. Sutton, R. S., & Barto, A. G. (2011). Reinforcement learning: An introduction. MIT Press.

  26. Ana Aguiar. CRAWDAD dataset it/vr2marketbaiaotrial (v. 2019-09-16). Downloaded from https://crawdad.org/it/vr2marketbaiaotrial/20190916, September 2019.

Download references

Author information

Authors and Affiliations

  1. Government Engineering College, Rajkot, Gujarat, India

    Hardik K. Molia

  2. Accenture, Pune, Maharashtra, India

    Amit D. Kothari

Authors
  1. Hardik K. Molia
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Amit D. Kothari
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Hardik K. Molia.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Molia, H.K., Kothari, A.D. TCP-RLLD: TCP with reinforcement learning based loss differentiation for mobile adhoc networks. Wireless Netw 29, 1937–1948 (2023). https://doi.org/10.1007/s11276-023-03254-3

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11276-023-03254-3

Keywords

Search

Navigation