Skip to main content
SpringerLink
Log in

Approximate Multicast Coflow Scheduling in Reconfigurable Data Center Networks

  • Conference paper
  • First Online:
Algorithms and Architectures for Parallel Processing (ICA3PP 2023)

Part of the book series: Lecture Notes in Computer Science ((LNCS,volume 14489))

  • 171 Accesses

Abstract

The emerging optical circuit technology, capable of establishing circuit connections among switches, has been proposed as a promising paradigm for data center networks. This paper investigates the problem of minimizing the completion time of multicast coflow in optical circuit switches (OCS)-based data center networks. The existing works either only focused on multicast coflow scheduling or focused solely on circuit scheduling in OCS-based networks, which greatly limits their performance. Hence, in this paper, we study how to reduce the completion time of multicast flows by considering circuit scheduling and coflow scheduling simultaneously. Firstly, We formulate the problem of multicast coflow scheduling, and prove it to be NP-hard. We propose a delay-efficient multicast coflow scheduling algorithm by integrating coflow scheduling with circuit scheduling. The proposed algorithm is proved to have an approximate ratio of at most \(2\sqrt{n}\), where n represents the number of optical circuit switches. Through extensive simulations, it is shown that the proposed algorithm can achieve high performance compared to state-of-the-art methods.

This research was supported by China University Industry University Research Innovation Fund under Grant No. 2021FNA02010, the NSFC under Grant No. U20A6003, U2001201, 62372118, the Guangdong Basic and Applied Basic Research Foundation under Grant No. 2022A1515011032, the Guangzhou Science and Technology Plan under Grant 2023A04J1701, and the Guangdong Provincial Key Laboratory of Cyber-Physical System under Grant 2020B1212060069.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 59.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 79.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Ajibade, L.S., Bakar, K.A., Aliyu, A., Danish, T.: Straggler mitigation in hadoop mapreduce framework: a review. Int. J. Adv. Comput. Sci. Appl. 13(8) (2022)

    Google Scholar 

  2. Chen, Q., Cai, Z., Cheng, L., Gao, H.: Low latency broadcast scheduling for battery-free wireless networks without predetermined structures. In: 2020 IEEE 40th International Conference on Distributed Computing Systems (ICDCS), pp. 245–255 (2020). https://doi.org/10.1109/ICDCS47774.2020.00052

  3. Chen, Q., Gao, H., Cheng, S., Fang, X., Cai, Z., Li, J.: Centralized and distributed delay-bounded scheduling algorithms for multicast in duty-cycled wireless sensor networks. IEEE/ACM Trans. Netw. 25(6), 3573–3586 (2017)

    Article  Google Scholar 

  4. Chen, Q., et al.: Latency-optimal pyramid-based joint communication and computation scheduling for distributed edge computing. In: IEEE Conference on Computer Communications, IEEE INFOCOM 2023, pp. 1–10 (2023). https://doi.org/10.1109/INFOCOM53939.2023.10228964

  5. Farrington, N., Porter, G., Fainman, Y., Papen, G., Vahdat, A.: Hunting mice with microsecond circuit switches. In: Proceedings of the 11th ACM Workshop on Hot Topics in Networks, pp. 115–120 (2012)

    Google Scholar 

  6. Jia, S., Jin, X., Ghasemiesfeh, G., Ding, J., Gao, J.: Competitive analysis for online scheduling in software-defined optical wan. In: IEEE Conference on Computer Communications, IEEE INFOCOM 2017, pp. 1–9. IEEE (2017)

    Google Scholar 

  7. Jin, X., et al.: Optimizing bulk transfers with software-defined optical WAN. In: Proceedings of the 2016 ACM SIGCOMM Conference, pp. 87–100 (2016)

    Google Scholar 

  8. Li, J., et al.: Digital twin-assisted, SFC-enabled service provisioning in mobile edge computing. IEEE Trans. Mob. Comput. 1–16 (2022). https://doi.org/10.1109/TMC.2022.3227248

  9. Li, W., Chen, S., Li, K., Qi, H., Xu, R., Zhang, S.: Efficient online scheduling for coflow-aware machine learning clusters, vol. 10, pp. 2564–2579. IEEE (2020)

    Google Scholar 

  10. Li, W., Chen, S., Li, K., Qi, H., Xu, R., Zhang, S.: Efficient online scheduling for coflow-aware machine learning clusters. IEEE Trans. Cloud Comput. 10(4), 2564–2579 (2020)

    Article  Google Scholar 

  11. Li, W., et al.: CoMan: managing bandwidth across computing frameworks in multiplexed datacenters. IEEE Trans. Parallel Distrib. Syst. 29(5), 1013–1029 (2017)

    Article  MathSciNet  Google Scholar 

  12. Li, W., Yuan, X., Li, K., Qi, H., Zhou, X.: Leveraging endpoint flexibility when scheduling coflows across geo-distributed datacenters. In: IEEE Conference on Computer Communications, IEEE INFOCOM 2018, pp. 873–881. IEEE (2018)

    Google Scholar 

  13. Li, W., Yuan, X., Li, K., Qi, H., Zhou, X., Xu, R.: Endpoint-flexible coflow scheduling across geo-distributed datacenters. IEEE Trans. Parallel Distrib. Syst. 31(10), 2466–2481 (2020)

    Article  Google Scholar 

  14. Li, W., et al.: Efficient coflow transmission for distributed stream processing. In: IEEE Conference on Computer Communications, IEEE INFOCOM 2020, pp. 1319–1328. IEEE (2020)

    Google Scholar 

  15. Luo, L., Foerster, K.T., Schmid, S., Yu, H.: Splitcast: optimizing multicast flows in reconfigurable datacenter networks. In: IEEE Conference on Computer Communications, IEEE INFOCOM 2020, pp. 2559–2568. IEEE (2020)

    Google Scholar 

  16. Luo, L., Foerster, K.T., Schmid, S., Yu, H.: Optimizing multicast flows in high-bandwidth reconfigurable datacenter networks. J. Netw. Comput. Appl. 203, 103399 (2022)

    Article  Google Scholar 

  17. Mitsuya, T., Ochiai, T., Kuno, T., Mori, Y., Hasegawa, H., Sato, K.: Highly reliable and large-scale optical circuit switch for intra-datacentre networks. In: 2022 European Conference on Optical Communication (ECOC), pp. 1–4. IEEE (2022)

    Google Scholar 

  18. Mosheiov, G.: Multi-machine scheduling with linear deterioration. INFOR: Inf. Syst. Oper. Res. 36(4), 205–214 (1998)

    Google Scholar 

  19. Peng, Y., Chen, K., Wang, G., Bai, W., Ma, Z., Gu, L.: HadoopWatch: a first step towards comprehensive traffic forecasting in cloud computing. In: IEEE Conference on Computer Communications, IEEE INFOCOM 2014, pp. 19–27. IEEE (2014)

    Google Scholar 

  20. Perelló, J., et al.: All-optical packet/circuit switching-based data center network for enhanced scalability, latency, and throughput. IEEE Netw. 27(6), 14–22 (2013)

    Article  Google Scholar 

  21. Porter, G., et al.: Integrating microsecond circuit switching into the data center. ACM SIGCOMM Comput. Commun. Rev. 43(4), 447–458 (2013)

    Article  Google Scholar 

  22. Shafiee, M., Ghaderi, J.: Scheduling coflows with dependency graph. IEEE/ACM Trans. Netw. 30(1), 450–463 (2021)

    Article  Google Scholar 

  23. Sun, X.S., Ng, T.E.: When creek meets river: exploiting high-bandwidth circuit switch in scheduling multicast data. In: 2017 IEEE 25th International Conference on Network Protocols (ICNP), pp. 1–6. IEEE (2017)

    Google Scholar 

  24. Tang, S., He, B., Yu, C., Li, Y., Li, K.: A survey on spark ecosystem: big data processing infrastructure, machine learning, and applications, vol. 34, pp. 71–91 (2022). https://doi.org/10.1109/TKDE.2020.2975652

  25. Tang, Y., Yuan, T., Liu, B., Xiao, C.: Effective*-flow schedule for optical circuit switching based data center networks: a comprehensive survey, vol. 197, p. 108321. Elsevier (2021)

    Google Scholar 

  26. Wang, G., et al.: C-through: part-time optics in data centers. In: Proceedings of the ACM SIGCOMM 2010 Conference, pp. 327–338 (2010)

    Google Scholar 

  27. Wang, H., Xu, H., Huang, L., Wang, J., Yang, X.: Load-balancing routing in software defined networks with multiple controllers. Comput. Netw. 141, 82–91 (2018)

    Article  Google Scholar 

  28. Wang, H., Yu, X., Xu, H., Fan, J., Qiao, C., Huang, L.: Integrating coflow and circuit scheduling for optical networks. IEEE Trans. Parallel Distrib. Syst. 30(6), 1346–1358 (2019). https://doi.org/10.1109/TPDS.2018.2889251

    Article  Google Scholar 

  29. Xia, Y., Ng, T.E., Sun, X.S.: Blast: accelerating high-performance data analytics applications by optical multicast. In: 2015 IEEE Conference on Computer Communications (INFOCOM), pp. 1930–1938. IEEE (2015)

    Google Scholar 

  30. Xu, H., Li, X.Y., Huang, L., Deng, H., Huang, H., Wang, H.: Incremental deployment and throughput maximization routing for a hybrid SDN. IEEE/ACM Trans. Netw. 25(3), 1861–1875 (2017)

    Article  Google Scholar 

  31. Xu, R., Li, W., Li, K., Zhou, X., Qi, H.: Scheduling mix-coflows in datacenter networks. IEEE Trans. Netw. Serv. Manag. 18(2), 2002–2015 (2020)

    Article  Google Scholar 

  32. Yang, H., Zhu, Z.: Topology configuration scheme for accelerating coflows in a hyper-flex-lion, vol. 14, pp. 805–814. Optica Publishing Group (2022)

    Google Scholar 

  33. Yao, B., Gao, H., Chen, Q., Li, J.: Energy-adaptive and bottleneck-aware many-to-many communication scheduling for battery-free WSNs. IEEE Internet Things J. 8(10), 8514–8529 (2021). https://doi.org/10.1109/JIOT.2020.3045979

    Article  Google Scholar 

  34. Zeng, Y., Ye, B., Tang, B., Guo, S., Qu, Z.: Scheduling coflows of multi-stage jobs under network resource constraints. Comput. Netw. 184, 107686 (2021)

    Article  Google Scholar 

  35. Zhao, Y., et al.: Rapier: integrating routing and scheduling for coflow-aware data center networks. In: 2015 IEEE Conference on Computer Communications (INFOCOM), pp. 424–432. IEEE (2015)

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. School of Computer Science and Technology, Guangdong University of Technology, Guangzhou, China

    Yuhang Wu, Quan Chen, Jianglong Liu, Fulong Li & Lianglun Cheng

Authors
  1. Yuhang Wu
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Quan Chen
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Jianglong Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Fulong Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Lianglun Cheng
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Quan Chen .

Editor information

Editors and Affiliations

  1. Royal Melbourne Institute of Technology, Melbourne, VIC, Australia

    Zahir Tari

  2. Tianjin University, Tianjin, China

    Keqiu Li

  3. University of Arizona, Tucson, AZ, USA

    Hongyi Wu

Rights and permissions

Reprints and permissions

Copyright information

© 2024 The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd.

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Wu, Y., Chen, Q., Liu, J., Li, F., Cheng, L. (2024). Approximate Multicast Coflow Scheduling in Reconfigurable Data Center Networks. In: Tari, Z., Li, K., Wu, H. (eds) Algorithms and Architectures for Parallel Processing. ICA3PP 2023. Lecture Notes in Computer Science, vol 14489. Springer, Singapore. https://doi.org/10.1007/978-981-97-0798-0_9

Download citation

  • DOI: https://doi.org/10.1007/978-981-97-0798-0_9

  • Published:

  • Publisher Name: Springer, Singapore

  • Print ISBN: 978-981-97-0797-3

  • Online ISBN: 978-981-97-0798-0

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation