Skip to main content
Book cover

International Conference on High Performance Computing

ISC High Performance 2021: High Performance Computing pp 57–76Cite as

A Hierarchical Task Scheduler for Heterogeneous Computing

Part of the Lecture Notes in Computer Science book series (LNTCS,volume 12728)

Abstract

Heterogeneous computing is one of the future directions of HPC. Task scheduling in heterogeneous computing must balance the challenge of optimizing the application performance and the need for an intuitive interface with the programming run-time to maintain programming portability. The challenge is further compounded by the varying data communication time between tasks. This paper proposes RANGER, a hardware-assisted task-scheduling framework. By integrating RISC-V cores with accelerators, the RANGER scheduling framework divides scheduling into global and local levels. At the local level, RANGER further partitions each task into fine-grained subtasks to reduce the overall makespan. At the global level, RANGER maintains the coarse granularity of the task specification, thereby maintaining programming portability. The extensive experimental results demonstrate that RANGER achieves a \(12.7\times \) performance improvement on average, while only requires \(2.7\%\) of area overhead.

Keywords

  • Extreme heterogeneity
  • Accelerators
  • HPC system architecture
  • Challenges in programming for massive scale

This manuscript has been co-authored by UT-Battelle, LLC under Contract No. DE-AC05-00OR22725 with the U.S. Department of Energy. The United States Government retains and the publisher, by accepting the article for publication, acknowledges that the United States Government retains a non-exclusive, paid-up, irrevocable, worldwide license to publish or reproduce the published form of this manuscript, or allow others to do so, for United States Government purposes. The Department of Energy will provide public access to these results of federally sponsored research in accordance with the DOE Public Access Plan.

This is a preview of subscription content, access via your institution.

Chapter
USD   29.95
Price excludes VAT (USA)
  • DOI: 10.1007/978-3-030-78713-4_4
  • Chapter length: 20 pages
  • Instant PDF download
  • Readable on all devices
  • Own it forever
  • Exclusive offer for individuals only
  • Tax calculation will be finalised during checkout
eBook
USD   84.99
Price excludes VAT (USA)
  • ISBN: 978-3-030-78713-4
  • Instant PDF download
  • Readable on all devices
  • Own it forever
  • Exclusive offer for individuals only
  • Tax calculation will be finalised during checkout
Softcover Book
USD   109.99
Price excludes VAT (USA)
Learn about institutional subscriptions
Fig. 1.
Fig. 2.
Fig. 3.
Fig. 4.
Fig. 5.
Fig. 6.
Fig. 7.
Fig. 8.
Fig. 9.

References

  1. Arabnejad, H., Barbosa, J.G.: List scheduling algorithm for heterogeneous systems by an optimistic cost table. IEEE Trans. Parallel Distrib. Syst. 25(3), 682–694 (2013)

    CrossRef  Google Scholar 

  2. ARM Corp.: AMBA: the standard for on-chip communication. https://www.arm.com/products/silicon-ip-system/embedded-system-design/amba-specifications. Accessed 10 Dec 2020

  3. Arnold, O., Noethen, B., Fettweis, G.: Instruction set architecture extensions for a dynamic task scheduling unit. In: 2012 IEEE Computer Society Annual Symposium on VLSI, pp. 249–254. IEEE (2012)

    Google Scholar 

  4. Asanovic, K., et al.: The Rocket chip generator. EECS Department, University of California, Berkeley, Technical report UCB/EECS-2016-17 (2016)

    Google Scholar 

  5. Binkert, N., et al.: The GEM5 simulator. ACM SIGARCH Comput. Archit. News 39(2), 1–7 (2011)

    CrossRef  Google Scholar 

  6. Blumofe, R.D., Joerg, C.F., Kuszmaul, B.C., Leiserson, C.E., Randall, K.H., Zhou, Y.: Cilk: an efficient multithreaded runtime system. J. Parallel Distrib. Comput. 37(1), 55–69 (1996)

    CrossRef  Google Scholar 

  7. Canon, L.C., Marchal, L., Simon, B., Vivien, F.: Online scheduling of task graphs on heterogeneous platforms. IEEE Trans. Parallel Distrib. Syst. 31, 721–732 (2019)

    CrossRef  Google Scholar 

  8. Dallou, T., Engelhardt, N., Elhossini, A., Juurlink, B.: Nexus#: a distributed hardware task manager for task-based programming models. In: 2015 IEEE International Parallel and Distributed Processing Symposium, pp. 1129–1138. IEEE (2015)

    Google Scholar 

  9. Frigo, M., Leiserson, C.E., Randall, K.H.: The implementation of the Cilk-5 multithreaded language. In: Proceedings of the ACM SIGPLAN 1998 Conference on Programming Language Design and Implementation, pp. 212–223 (1998)

    Google Scholar 

  10. He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 770–778 (2016)

    Google Scholar 

  11. Huang, T.W., Lin, C.X., Guo, G., Wong, M.: Cpp-Taskflow: fast task-based parallel programming using modern C++. In: 2019 IEEE International Parallel and Distributed Processing Symposium (IPDPS), pp. 974–983. IEEE (2019)

    Google Scholar 

  12. Intel Corp.: Coffee lake - microarchitecture - intel. https://en.wikichip.org/wiki/intel/microarchitectures/coffee_lake. Accessed 10 Dec 2020

  13. Johnston, B., Milthorpe, J.: AIWC: OpenCL-based architecture-independent workload characterization. In: 2018 IEEE/ACM 5th Workshop on the LLVM Compiler Infrastructure in HPC (LLVM-HPC), pp. 81–91. IEEE (2018)

    Google Scholar 

  14. Kale, L.V., Krishnan, S.: Charm++: parallel programming with message-driven objects. In: Wilson, G.V., Lu, P. (eds.) Parallel Programming Using C++, vol. 1, pp. 175–213. MIT Press, Cambridge (1996)

    Google Scholar 

  15. Kaleem, R., Barik, R., Shpeisman, T., Hu, C., Lewis, B.T., Pingali, K.: Adaptive heterogeneous scheduling for integrated GPUs. In: 2014 23rd International Conference on Parallel Architecture and Compilation Techniques (PACT), pp. 151–162. IEEE (2014)

    Google Scholar 

  16. Khronos Group: OpenCL: the open standard for parallel programming of heterogeneous systems (2019)

    Google Scholar 

  17. Kukanov, A., Voss, M.J.: The foundations for scalable multi-core software in Intel Threading Building Blocks. Intel Technol. J. 11(4) (2007)

    Google Scholar 

  18. Liu, F., Miniskar, N.R., Chakraborty, D., Vetter, J.S.: DEFFE: a data-efficient framework for performance characterization in domain-specific computing. In: Proceedings of the 17th ACM International Conference on Computing Frontiers, pp. 182–191 (2020)

    Google Scholar 

  19. Ma, Z., Catthoor, F., Vounckx, J.: Hierarchical task scheduler for interleaving subtasks on heterogeneous multiprocessor platforms. In: Proceedings of the 2005 Asia and South Pacific Design Automation Conference, pp. 952–955 (2005)

    Google Scholar 

  20. Morais, L., et al.: Adding tightly-integrated task scheduling acceleration to a RISC-V multi-core processor. In: Proceedings of the 52nd Annual IEEE/ACM International Symposium on Microarchitecture, pp. 861–872 (2019)

    Google Scholar 

  21. Nickolls, J., Buck, I.: NVIDIA CUDA software and GPU parallel computing architecture. In: Microprocessor Forum (2007)

    Google Scholar 

  22. OpenACC: OpenACC: directives for accelerators (2015)

    Google Scholar 

  23. OpenMP: OpenMP reference (1999)

    Google Scholar 

  24. Robison, A.D.: Composable parallel patterns with Intel Cilk Plus. Comput. Sci. Eng. 15(2), 66–71 (2013)

    CrossRef  Google Scholar 

  25. Ronneberger, O., Fischer, P., Brox, T.: U-Net: convolutional networks for biomedical image segmentation. In: Navab, N., Hornegger, J., Wells, W.M., Frangi, A.F. (eds.) MICCAI 2015. LNCS, vol. 9351, pp. 234–241. Springer, Cham (2015). https://doi.org/10.1007/978-3-319-24574-4_28

    CrossRef  Google Scholar 

  26. Shao, Y.S., Xi, S.L., Srinivasan, V., Wei, G.Y., Brooks, D.: Co-designing accelerators and SoC interfaces using gem5-Aladdin. In: 2016 49th Annual IEEE/ACM International Symposium on Microarchitecture (MICRO), pp. 1–12. IEEE (2016)

    Google Scholar 

  27. Sijstermans, F.: The NVIDIA deep learning accelerator. In: Proceedings Hot Chips: A Symposium on High Performance Chips, August 2018

    Google Scholar 

  28. Simonyan, K., Zisserman, A.: Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv:1409.1556 (2014)

  29. Sinnen, O.: Task Scheduling for Parallel Systems, vol. 60. Wiley, Hoboken (2007)

    Google Scholar 

  30. Själander, M., Terechko, A., Duranton, M.: A look-ahead task management unit for embedded multi-core architectures. In: 2008 11th EUROMICRO Conference on Digital System Design Architectures, Methods and Tools, pp. 149–157. IEEE (2008)

    Google Scholar 

  31. Slaughter, E., et al.: Task bench: a parameterized benchmark for evaluating parallel runtime performance, pp. 1–30 (2020)

    Google Scholar 

  32. Szegedy, C., Vanhoucke, V., Ioffe, S., Shlens, J., Wojna, Z.: Rethinking the inception architecture for computer vision. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2818–2826 (2016)

    Google Scholar 

  33. Topcuoglu, H., Hariri, S., Wu, M.Y.: Performance-effective and low-complexity task scheduling for heterogeneous computing. IEEE Trans. Parallel Distrib. Syst. 13(3), 260–274 (2002)

    CrossRef  Google Scholar 

  34. Ullman, J.D.: NP-complete scheduling problems. J. Comput. Syst. Sci. 10(3), 384–393 (1975)

    CrossRef  MathSciNet  Google Scholar 

  35. Vetter, J.S., Brightwell, R., et al.: Extreme heterogeneity 2018: DOE ASCR basic research needs workshop on extreme heterogeneity (2018). https://doi.org/10.2172/1473756

  36. Waterman, A., Lee, Y., Avizienis, R., Cook, H., Patterson, D.A., Asanovic, K.: The RISC-V instruction set. In: Hot Chips Symposium, p. 1 (2013)

    Google Scholar 

  37. Western Digital Corp.: RISC-V: accelerating next-generation compute requirements. https://www.westerndigital.com/company/innovations/risc-v. Accessed 10 Dec 2020

Download references

Acknowledgments

This material is based upon work supported by the US Department of Energy (DOE) Office of Science, Office of Advanced Scientific Computing Research under contract number DE-AC05-00OR22725.

This research was supported in part by the DOE Advanced Scientific Computing Research Program Sawtooth Project and the Laboratory Directed Research and Development Program of Oak Ridge National Laboratory, managed by UT-Battelle LLC for DOE.

Author information

Authors and Affiliations

  1. Advanced Computing Systems Research Section, Oak Ridge National Laboratory, Oak Ridge, TN, 37831, USA

    Narasinga Rao Miniskar, Frank Liu, Aaron R. Young & Jeffrey S. Vetter

  2. Rowan University, 201 Mullica Hill Road, Glassboro, NJ, 08028, USA

    Dwaipayan Chakraborty

Authors
  1. Narasinga Rao Miniskar
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Frank Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Aaron R. Young
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Dwaipayan Chakraborty
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Jeffrey S. Vetter
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Narasinga Rao Miniskar .

Editor information

Editors and Affiliations

  1. Hewlett Packard Enterprise, Seattle, WA, USA

    Bradford L. Chamberlain

  2. University of Amsterdam, Amsterdam, The Netherlands

    Ana-Lucia Varbanescu

  3. Extreme Computing Research Center, Thuwal Jeddah, Saudi Arabia

    Hatem Ltaief

  4. The University of Tennessee, Knoxville, Knoxville, TN, USA

    Piotr Luszczek

Rights and permissions

Reprints and Permissions

Copyright information

© 2021 UT-Battelle, LLC

About this paper

Verify currency and authenticity via CrossMark

Cite this paper

Miniskar, N.R., Liu, F., Young, A.R., Chakraborty, D., Vetter, J.S. (2021). A Hierarchical Task Scheduler for Heterogeneous Computing. In: Chamberlain, B.L., Varbanescu, AL., Ltaief, H., Luszczek, P. (eds) High Performance Computing. ISC High Performance 2021. Lecture Notes in Computer Science(), vol 12728. Springer, Cham. https://doi.org/10.1007/978-3-030-78713-4_4

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-78713-4_4

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-78712-7

  • Online ISBN: 978-3-030-78713-4

  • eBook Packages: Computer ScienceComputer Science (R0)