Skip to main content
SpringerLink
Log in

Analysis of Virtualized Congestion Control in Applications Based on Hadoop MapReduce

  • Conference paper
  • First Online:
High Performance Computing Systems (WSCAD 2018)

Part of the book series: Communications in Computer and Information Science ((CCIS,volume 1171))

Included in the following conference series:

Abstract

Among the existing applications for processing massive volumes of data, the Hadoop MapReduce (HMR) is widely used in clouds, having above all internal network flows of different volume and periodicity. In this regard, providers have the challenge of managing data centers with a wide range of operating systems and features. The diversity of algorithms and parameters related to TCP constitutes a heterogeneous communication scenario prone to degradation of communication-intensive applications. Due to total control in the data center, providers can apply the Virtualized Congestion Control (VCC) to generate optimized algorithms. From the tenant’s perspective, virtualization is a transparently performed. Some technologies have made possible to develop such virtualization. Explicit Congestion Notification (ECN) is a technique for congestion identification which acts by monitoring the queues occupancy. Although promising, the specialized literature lacks on a deep analysis of the VCC impact on the applications. Our work characterizes the VCC impact on HMR on scenarios in which there are present applications competing for network resources using optimized and non-optimized TCP stacks. We identified the HMR has its performance substantially influenced by the data volume according to the employed TCP stack. Moreover, we highlight some VCC limitations.

Supported by UDESC/FAPESC, developed on LabP2D.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 39.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Notes

  1. 1.

    This paper is a revised and expanded version of [14].

References

  1. Alizadeh, M., et al.: Data center TCP (DCTCP). SIGCOMM Comput. Commun. Rev. 40(4), 63–74 (2010)

    Article  Google Scholar 

  2. Alizadeh, M., Javanmard, A., Prabhakar, B.: Analysis of DCTCP: stability, convergence, and fairness. In: Proceedings of the ACM SIGMETRICS Joint International Conference on Measurement and Modeling of Computer Systems. SIGMETRICS 2011, pp. 73–84. ACM (2011)

    Google Scholar 

  3. Chiu, D.M., Jain, R.: Analysis of the increase and decrease algorithms for congestion avoidance in computer networks. Comput. Netw. ISDN Syst. 17(1), 1–14 (1989)

    Article  Google Scholar 

  4. Chowdhury, M., Zaharia, M., Ma, J., Jordan, M.I., Stoica, I.: Managing data transfers in computer clusters with orchestra. SIGCOMM Comput. Commun. Rev. 41(4), 98–109 (2011)

    Article  Google Scholar 

  5. Cronkite-Ratcliff, B., et al.: Virtualized congestion control. In: Proceedings of the 2016 ACM SIGCOMM Conference. SIGCOMM 2016, pp. 230–243. ACM (2016)

    Google Scholar 

  6. Floyd, S.: TCP and explicit congestion notification. SIGCOMM Comput. Commun. Rev. 24(5), 8–23 (1994)

    Article  MathSciNet  Google Scholar 

  7. Ha, S., Rhee, I., Xu, L.: CUBIC: a new TCP-friendly high-speed TCP variant. SIGOPS Oper. Syst. Rev. 42(5), 64–74 (2008)

    Article  Google Scholar 

  8. He, K., et al.: AC/DC TCP: virtual congestion control enforcement for datacenter networks. In: Proceedings of the 2016 SIGCOMM Conference. SIGCOMM 2016, pp. 244–257. ACM (2016)

    Google Scholar 

  9. Jacobson, V.: Congestion avoidance and control. SIGCOMM Comput. Commun. Rev. 18(4), 314–329 (1988)

    Article  Google Scholar 

  10. Judd, G.: Attaining the promise and avoiding the pitfalls of TCP in the datacenter. In: Proceedings of the 12th USENIX Conference on Networked Systems Design and Implementation. NSDI 2015, pp. 145–157. Berkeley (2015)

    Google Scholar 

  11. Kühlewind, M., Neuner, S., Trammell, B.: On the state of ECN and TCP options on the Internet. In: Roughan, M., Chang, R. (eds.) PAM 2013. LNCS, vol. 7799, pp. 135–144. Springer, Heidelberg (2013). https://doi.org/10.1007/978-3-642-36516-4_14

    Chapter  Google Scholar 

  12. Kumar, P., et al.: : PicNIC: predictable virtualized NIC. In: Proceedings of the ACM Special Interest Group on Data Communication. SIGCOMM 2019, pp. 351–366. ACM (2019)

    Google Scholar 

  13. Lantz, B., Heller, B., McKeown, N.: A network in a laptop: rapid prototyping for software-defined networks. In: Proceedings of the 9th ACM SIGCOMM Workshop on Hot Topics in Networks. HotNets-IX, pp. 19:1–19:6. ACM (2010)

    Google Scholar 

  14. Moro, V., Pillon, M.A., Miers, C., Koslovski, G.: Análise da virtualização do controle de congestionamento na execução de aplicações hadoop mapreduce. In: Simpósio de Sistemas Computacionais de Alto Desempenho - WSCAD, October 2018

    Google Scholar 

  15. Neves, M.V., De Rose, C.A.F., Katrinis, K.: MRemu: an emulation-based framework for datacenter network experimentation using realistic MapReduce traffic. In: Proceedings of the 2015 IEEE 23rd International Symposium on Modeling, Analysis, and Simulation of Computer and Telecommunication Systems. MASCOTS 2015, pp. 174–177 (2015)

    Google Scholar 

  16. Pfaff, B., et al.: The design and implementation of Open vSwitch. In: Proceedings of the 12th USENIX Conference on Networked Systems Design and Implementation. NSDI 2015,pp. 117–130 (2015)

    Google Scholar 

  17. Popa, L., Kumar, G., Chowdhury, M., Krishnamurthy, A., Ratnasamy, S., Stoica, I.: FairCloud: sharing the network in cloud computing. In: Proceedings of the ACM SIGCOMM 2012 Conference on Applications, Technologies, Architectures, and Protocols for Computer Communication. SIGCOMM 2012, pp. 187–198. ACM (2012)

    Google Scholar 

  18. Roy, A., Zeng, H., Bagga, J., Porter, G., Snoeren, A.C.: Inside the social network’s (datacenter) network. SIGCOMM Comput. Commun. Rev. 45(4), 123–137 (2015)

    Article  Google Scholar 

  19. de Souza, F.R., Miers, C.C., Fiorese, A., Koslovski, G.P.: Qos-aware virtual infrastructures allocation on SDN-based clouds. In: Proceedings of the 17th IEEE/ACM International Symposium on Cluster, Cloud and Grid Computing. CCGrid 2017, pp. 120–129. IEEE Press, Piscataway (2017)

    Google Scholar 

  20. Primet, P.V.-B., Anhalt, F., Koslovski, G.: Exploring the virtual infrastructure service concept in Grid’5000. In: 20th ITC Specialist Seminar on Network Virtualization. Hoi An, May 2009

    Google Scholar 

  21. Wu, H., Ju, J., Lu, G., Guo, C., Xiong, Y., Zhang, Y.: Tuning ECN for data center networks. In: Proceedings of the 8th International Conference on Emerging Networking Experiments and Technologies. CoNEXT 2012, pp. 25–36. ACM (2012)

    Google Scholar 

  22. Zahavi, E., Shpiner, A., Rottenstreich, O., Kolodny, A., Keslassy, I.: Links as a service (LaaS): guaranteed tenant isolation in the shared cloud. In: Proceedings of the 2016 Symposium on Architectures for Networking and Communications Systems. ANCS 2016, pp. 87–98. ACM (2016)

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Graduate Program in Applied Computing, Santa Catarina State University, Florianópolis, Brazil

    Vilson Moro, Maurício Aronne Pillon, Charles Christian Miers & Guilherme Piêgas Koslovski

Authors
  1. Vilson Moro
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Maurício Aronne Pillon
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Charles Christian Miers
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Guilherme Piêgas Koslovski
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Guilherme Piêgas Koslovski .

Editor information

Editors and Affiliations

  1. Mackenzie Presbyterian University, São Paulo, Brazil

    Calebe Bianchini

  2. National Laboratory for Scientific Computing, Rio de Janeiro, Brazil

    Carla Osthoff

  3. University of São Paulo, São Paulo, Brazil

    Paulo Souza

  4. Federal University of Minas Gerais, Belo Horizonte, Brazil

    Renato Ferreira

Rights and permissions

Reprints and permissions

Copyright information

© 2020 Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Moro, V., Pillon, M.A., Miers, C.C., Koslovski, G.P. (2020). Analysis of Virtualized Congestion Control in Applications Based on Hadoop MapReduce. In: Bianchini, C., Osthoff, C., Souza, P., Ferreira, R. (eds) High Performance Computing Systems. WSCAD 2018. Communications in Computer and Information Science, vol 1171. Springer, Cham. https://doi.org/10.1007/978-3-030-41050-6_3

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-41050-6_3

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-41049-0

  • Online ISBN: 978-3-030-41050-6

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation