A Compiler-Assisted Runtime-Prefetching Scheme for Heterogeneous Platforms

  • Li Chen
  • Baojiang Shou
  • Xionghui Hou
  • Lei Huang
Part of the Lecture Notes in Computer Science book series (LNCS, volume 7312)


GPGPU has been widely used in recent years in both academia and industry. Many research for benchmarks on GPUs were reported to achieve over 100 times speedup, however, due to the high overhead of data transfer between GPU and CPU in real-world applications, the achievements are dramatically limited. In the case of using multiple GPUs, the situation is even worse. Another difficulty raised by the GPGPUs is the programming productivity.

In this work, we introduce a new language extension to the easy-to-use programming model OpenMP, implement a runtime and a prefetching mechanism to further extend our work in support of OpenMP on heterogeneous platforms. The new language extension allows the OpenUH compiler to generate efficient code for heterogeneous platforms with multiple GPUs included. To improve the performance of applications with lots of data transfer, we implement runtime inter-thread dataflow analysis and a runtime-prefetching mechanism with the help of compiler analysis, making the data transfer overlap with the computation as much as possible. We have evaluated our prefetching system using benchmarks including NPB SP, kmeans and nbody. In these experiments, we achieve speedups of 1.23, 1.4 and 1.32 respectively compared with the versions without prefetching support.


Data Transfer Parallel Loop Language Extension Heterogeneous Platform Multiple GPUs 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Augonnet, C., Thibault, S., Namyst, R., Wacrenier, P.A.: Starpu: a unified platform for task scheduling on heterogeneous multicore architectures. Concurr. Comput.: Pract. Exper. 23, 187–198 (2011)CrossRefGoogle Scholar
  2. 2.
    Barrachina, S., Castillo, M., Igual, F., Mayo, R., Quintana-Orti, E.: Evaluation and tuning of the level 3 cublas for graphics processors. In: IEEE International Symposium on Parallel and Distributed Processing, IPDPS 2008, pp. 1–8 (April 2008)Google Scholar
  3. 3.
    Becchi, M., Byna, S., Cadambi, S., Chakradhar, S.: Data-aware scheduling of legacy kernels on heterogeneous platforms with distributed memory. In: Proceedings of the 22nd ACM Symposium on Parallelism in Algorithms and Architectures, SPAA 2010, pp. 82–91. ACM, New York (2010), CrossRefGoogle Scholar
  4. 4.
    Che, S., Sheaffer, J.W., Skadron, K.: Dymaxion: optimizing memory access patterns for heterogeneous systems. In: Proceedings of 2011 International Conference for High Performance Computing, Networking, Storage and Analysis, SC 2011, pp. 13:1–13:11. ACM, New York (2011)Google Scholar
  5. 5.
    Grewe, D., O’Boyle, M.F.P.: A Static Task Partitioning Approach for Heterogeneous Systems Using OpenCL. In: Knoop, J. (ed.) CC 2011. LNCS, vol. 6601, pp. 286–305. Springer, Heidelberg (2011)CrossRefGoogle Scholar
  6. 6.
    Gelado, I., Kelm, J.H., Ryoo, S., Lumetta, S.S., Navarro, N., Hwu, W.M.W.: Cuba: an architecture for efficient cpu/co-processor data communication. In: Proceedings of the 22nd Annual International Conference on Supercomputing, ICS 2008, pp. 299–308. ACM, New York (2008)CrossRefGoogle Scholar
  7. 7.
    Group, K.O.W.: The opencl specification (2011),
  8. 8.
    Han, T.D., Abdelrahman, T.S.: /hi/cuda: a high-level directive-based language for gpu programming. In: GPGPU-2: Proceedings of 2nd Workshop on General Purpose Processing on Graphics Processing Units, pp. 52–61. ACM, New York (2009)CrossRefGoogle Scholar
  9. 9.
    Jablin, T.B., Prabhu, P., Jablin, J.A., Johnson, N.P., Beard, S.R., August, D.I.: Automatic cpu-gpu communication management and optimization. In: Proceedings of the 32nd ACM SIGPLAN Conference on Programming Language Design and Implementation, PLDI 2011, pp. 142–151. ACM, New York (2011)Google Scholar
  10. 10.
    Kim, J., Kim, H., Lee, J.H., Lee, J.: Achieving a single compute device image in opencl for multiple gpus. In: Proceedings of the 16th ACM Symposium on Principles and Practice of Parallel Programming, PPoPP 2011, pp. 277–288. ACM, New York (2011)Google Scholar
  11. 11.
    Lee, V.W., Kim, C., Chhugani, J., Deisher, M., Kim, D., Nguyen, A.D., Satish, N., Smelyanskiy, M., Chennupaty, S., Hammarlund, P., Singhal, R., Dubey, P.: Debunking the 100x gpu vs. cpu myth: an evaluation of throughput computing on cpu and gpu. In: Proceedings of the 37th Annual International Symposium on Computer Architecture, ISCA 2010, pp. 451–460. ACM, New York (2010)CrossRefGoogle Scholar
  12. 12.
    Linderman, M.D., Collins, J.D., Wang, H., Meng, T.H.: Merge: a programming model for heterogeneous multi-core systems. In: Proceedings of the 13th International Conference on Architectural Support for Programming Languages and Operating Systems, ASPLOS XIII, pp. 287–296. ACM, New York (2008)CrossRefGoogle Scholar
  13. 13.
    Luk, C.K., Hong, S., Kim, H.: Qilin: exploiting parallelism on heterogeneous multiprocessors with adaptive mapping. In: Proceedings of the 42nd Annual IEEE/ACM International Symposium on Microarchitecture, MICRO 42, pp. 45–55. ACM, New York (2009)CrossRefGoogle Scholar
  14. 14.
    Meng, J., Skadron, K.: Performance modeling and automatic ghost zone optimization for iterative stencil loops on gpus. In: Proceedings of the 23rd International Conference on Supercomputing, ICS 2009, pp. 256–265. ACM, New York (2009)CrossRefGoogle Scholar
  15. 15.
    Org., O.S.: The openacc application programming interface (2011),
  16. 16.
    Planas, J., Badia, R.M., Ayguadé, E., Labarta, J.: Hierarchical task-based programming with starss. Int. J. High Perform. Comput. Appl. 23, 284–299 (2009), CrossRefGoogle Scholar
  17. 17.
    Prasad, A., Anantpur, J., Govindarajan, R.: Automatic compilation of matlab programs for synergistic execution on heterogeneous processors. In: Proceedings of the 32nd ACM SIGPLAN Conference on Programming Language Design and Implementation, PLDI 2011, pp. 152–163. ACM, New York (2011), Google Scholar
  18. 18.
    Strengert, M., Müller, C., Dachsbacher, C., Ertl, T.: Cudasa: Compute unified device and systems architecture. In: Favre, J.M., Ma, K.L. (eds.) EGPGV, pp. 49–56. Eurographics Association (2008)Google Scholar
  19. 19.
    Sung, I.J., Stratton, J.A., Hwu, W.M.W.: Data layout transformation exploiting memory-level parallelism in structured grid many-core applications. In: Proceedings of the 19th International Conference on Parallel Architectures and Compilation Techniques, PACT 2010, pp. 513–522. ACM, New York (2010)CrossRefGoogle Scholar
  20. 20.
    The Portland Group: PGI Fortran & C Accelator Programming Model. White Paper (2010)Google Scholar
  21. 21.
    Ueng, S.-Z., Lathara, M., Baghsorkhi, S.S., Hwu, W.-m.W.: CUDA-Lite: Reducing GPU Programming Complexity. In: Amaral, J.N. (ed.) LCPC 2008. LNCS, vol. 5335, pp. 1–15. Springer, Heidelberg (2008)CrossRefGoogle Scholar
  22. 22.
    Venkatasubramanian, S., Vuduc, R.W., none, n.: Tuned and wildly asynchronous stencil kernels for hybrid cpu/gpu systems. In: Proceedings of the 23rd International Conference on Supercomputing, ICS 2009, pp. 244–255. ACM, New York (2009)CrossRefGoogle Scholar
  23. 23.
    Vineet, V., Harish, P., Patidar, S., Narayanan, P.J.: Fast minimum spanning tree for large graphs on the gpu. In: Proceedings of the Conference on High Performance Graphics 2009, HPG 2009, pp. 167–171. ACM, New York (2009)CrossRefGoogle Scholar
  24. 24.
    White, L.: OpenMP Extensions for Heterogeneous Architectures. In: Chapman, B.M., Gropp, W.D., Kumaran, K., Müller, M.S. (eds.) IWOMP 2011. LNCS, vol. 6665, pp. 94–107. Springer, Heidelberg (2011), CrossRefGoogle Scholar
  25. 25.
    Yang, C., Wang, F., Du, Y., Chen, J., Liu, J., Yi, H., Lu, K.: Adaptive optimization for petascale heterogeneous cpu/gpu computing. In: Proceedings of the 2010 IEEE International Conference on Cluster Computing, CLUSTER 2010, pp. 19–28. IEEE Computer Society, Washington, DC (2010)CrossRefGoogle Scholar
  26. 26.
    Yang, Y., Xiang, P., Kong, J., Zhou, H.: A gpgpu compiler for memory optimization and parallelism management. In: Proceedings of the 2010 ACM SIGPLAN Conference on Programming Language Design and Implementation, PLDI 2010, pp. 86–97. ACM, New York (2010)CrossRefGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2012

Authors and Affiliations

  • Li Chen
    • 1
  • Baojiang Shou
    • 1
  • Xionghui Hou
    • 1
  • Lei Huang
    • 2
  1. 1.State Key Laboratory of Computer Architecture, Institute of Computing TechnologyChinese Academy of SciencesBeijingChina
  2. 2.Department of Computer SciencePrairie View A&M UniversityPrairie ViewUSA

Personalised recommendations