Advertisement

Springer Nature is making SARS-CoV-2 and COVID-19 research free. View research | View latest news | Sign up for updates

Toward a transparent and efficient GPU cloudification architecture

Abstract

The cloud model allows the access to a vast amount of computational resources, alleviating the need for acquisition and maintenance costs on a pay-per-use basis. However, other resources, such as (GPUs), have not been fully adapted to this model. Many areas would benefit from suitable cloud solutions based on GPUs: video encoding, sequencing in bioinformatics, scene rendering in remote gaming, or machine learning. Cloud providers offer local and exclusive access to GPUs by using PCI passthrough. This limitation can be overcome by integrating new virtual GPUs (vGPUs) in cloud infrastructures or by providing mechanisms to cloudify existing GPUs, cloudified GPUs (cGPUs), which do not support native virtualization. The proposed architecture enables an effective and transparent integration of cGPUs in public cloud infrastructures. Our solution offers several access modes (local/remote and exclusive/shared) and configures autonomously its components by integrating with the message middleware of the cloud infrastructure. A prototype of the proposed architecture has been evaluated in a real cloud deployment. Experiments assess overhead in the infrastructure and performance of GPU-based applications by considering three different programs: matrix multiplication, sequencing read alignment, and Monte-Carlo on multiple GPUs. Results show that our solution introduces low impact both on the infrastructure and the performance of applications.

This is a preview of subscription content, log in to check access.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15
Fig. 16

Notes

  1. 1.

    rCUDA v16.11: http://www.rcuda.net/index.php/94-new-rcuda-version.html.

  2. 2.

    https://github.com/jgutie2r/CgiHTTPServer.

  3. 3.

    https://docs.openstack.org/ocata/networking-guide/.

  4. 4.

    http://developer.download.nvidia.com/compute/cuda/1.1-Beta/x86_website/samples.html#matrixMulDrv.

  5. 5.

    http://developer.download.nvidia.com/compute/cuda/1.1-Beta/x86_website/samples.html#MonteCarloMultiGPU.

  6. 6.

    http://sourceforge.net/projects/iperf.

References

  1. 1.

    Michael A, Armando F, Rean G, Joseph Anthony D, Randy K, Andy K, Gunho L, David P, Ariel R, Ion S, Matei Z (2010) A view of cloud computing. Commun ACM 53(4):50–58. https://doi.org/10.1145/1721654.1721672

  2. 2.

    Mastelic T, Oleksiak A, Claussen H, Brandic I, Pierson J-M, Vasilakos AV (2014) Cloud computing: survey on energy efficiency. ACM Comput Surv 47(2):33:1–33:36. https://doi.org/10.1145/2656204. ISSN 0360-0300

  3. 3.

    Mell P, Grance T (2011) The NIST definition of cloud computing. NIST Pubs (800-154). https://doi.org/10.6028/NIST.SP.800-145

  4. 4.

    Che S, Li J, Sheaffer JW, Skadron K, Lach J (2008) Accelerating compute-intensive applications with GPUs and FPGAs. In: Symposium on Application Specific Processors, pp 101–107. https://doi.org/10.1109/SASP.2008.4570793

  5. 5.

    Rodríguez-Sánchez R, Martínez JL, Fernández-Escribano G, Sánchez JL, Claver JM, Diaz P (2012) Optimizing H.264/AVC interprediction on a GPU-based framework. Concurr Comput Pract Exp 24(14):1607–1624. https://doi.org/10.1002/cpe.1911

  6. 6.

    Yongchao L, Bertil S, Maskell Douglas L (2012) CUSHAW: a CUDA compatible short read aligner to large genomes based on the Burrows–Wheeler transform. Bioinformatics 28(14):1830–1837. https://doi.org/10.1093/bioinformatics/bts276

  7. 7.

    Wei C, Ryan S, Chun-Ying H, Kuan-Ta C, Jiangchuan L, Leung Victor CM, Cheng-Hsin H (2016) A survey on cloud gaming—future of computer games. IEEE Access 4:7605–7620. https://doi.org/10.1109/ACCESS.2016.2590500

  8. 8.

    Temam O (2016) Enabling future progress in machine-learning. In: IEEE Symposium on VLSI Circuits, Digest of Technical Papers, pp 1–3. https://doi.org/10.1109/VLSIC.2016.7573457

  9. 9.

    Amazon Web Services: EC2. http://aws.amazon.com/ec2. [Cited 2018-05-25]

  10. 10.

    Microsoft Azure: GPU optimized virtual machine sizes. https://docs.microsoft.com/en-us/azure/virtual-machines/windows/sizes-gpu/. [Cited 2018-05-25]

  11. 11.

    Google Cloud: GPUs on Compute Engine. https://cloud.google.com/compute/docs/gpus/. [Cited 2018-05-25]

  12. 12.

    NVIDIA GPU Cloud: GPU-Accelerated Containers. https://www.nvidia.com/en-us/gpu-cloud/. [Cited 2018-05-25]

  13. 13.

    Walters JP, Younge AJ, Kang DI, Yao KT, Kang M, Crago SP, Fox G (2014) GPU passthrough performance: a comparison of KVM, Xen, VMWare ESXi, and LXC for CUDA and OpenCL applications. In: IEEE 7th International Conference on Cloud Computing (CLOUD), pp 636–643. IEEE. https://doi.org/10.1109/CLOUD.2014.90

  14. 14.

    Amazon EC2 Elastic GPUs. https://aws.amazon.com/ec2/elastic-gpus/. [Cited 2018-05-25]

  15. 15.

    Hong C-H, Spence I, Nikolopoulos DS (2017) GPU virtualization and scheduling methods—a comprehensive survey. ACM Comput Surv 1(1). https://doi.org/10.1145/3068281

  16. 16.

    OpenStack: The Open Source Cloud Operating System. http://www.openstack.org/software/. [Cited 2018-05-25]

  17. 17.

    Vogel A, Griebler D, Maron CAF, Schepke C, Fernandes LG (2016) Private IaaS clouds: a comparative analysis of OpenNebula, CloudStack and OpenStack. In: Proceedings of the 24th Euromicro International Conference on Parallel, Distributed, and Network-Based Processing, pp 672–679. https://doi.org/10.1109/PDP.2016.75

  18. 18.

    Chirivella-Perez E, Gutierrez-Aguado J, Claver JM, Alcaraz-Calero JM (2015) Hybrid and extensible architecture for cloud infrastructure deployment. In: 15th IEEE International Conference on Computer and Information Technology, pp 611–617. https://doi.org/10.1109/CIT/IUCC/DASC/PICOM.2015.87

  19. 19.

    Habib I (2008) Virtualization with KVM. Linux J 2008(166). http://www.linuxjournal.com/article/9764. ISSN 1075-3583

  20. 20.

    NVIDIA NVLink Fabric, 2017. https://www.nvidia.com/en-us/data-center/nvlink/. [Cited 2018-05-25]

  21. 21.

    NVIDIA. NVIDIA GRID Technology, 2015. http://www.nvidia.com/object/grid-technology.html. [Cited 2018-05-25]

  22. 22.

    Song J, Lv Z, Tian K (2014) KVMGT: a full GPU virtualization solution. https://www.linux-kvm.org/images/f/f3/01x08b-KVMGT-a.pdf. [Cited 2018-05-25]

  23. 23.

    Intel Graphics Virtualization Technology (Intel GVT), 2017. https://01.org/igvt-g/blogs/wangbo85/2017/intel-gvt-g-kvmgt-public-release-q22017. [Cited 2018-05-25]

  24. 24.

    Qi Z, Yao J, Zhang C, Yu M, Yang Z, Guan H (2014) VGRIS: virtualized GPU resource isolation and scheduling in cloud gaming. ACM Trans Archit Code Optim 11(2):17:1–17:25. https://doi.org/10.1145/2632216. ISSN 1544-3566

  25. 25.

    Liang T-Y, Chang Y-W (2011) GridCUDA: A grid-enabled CUDA programming toolkit. In: 25th IEEE International Conference on Advanced Information Networking and Applications Workshops, pp 141–146. https://doi.org/10.1109/WAINA.2011.82

  26. 26.

    Oikawa M, Kawai A, Nomura K, Yasuoka K, Yoshikawa K, Narumi T (Nov 2012) DS-CUDA: a middleware to use many GPUs in the cloud environment. In: High Performance Computing, Networking, Storage and Analysis (SCC), pp 1207–1214. https://doi.org/10.1109/SC.Companion.2012.146

  27. 27.

    Shi L, Chen H, Sun J (May 2009) vCUDA: GPU accelerated high performance computing in virtual machines. In: IEEE International Symposium on Parallel Distributed Processing, pp 1–11. https://doi.org/10.1109/IPDPS.2009.5161020

  28. 28.

    Giunta G, Montella R, Agrillo G, Coviello G (2010) A GPGPU transparent virtualization component for high performance computing clouds. In: European Conference on Parallel Processing, pp 379–391. Springer. https://doi.org/10.1007/978-3-642-15277-1_37

  29. 29.

    Reaño Crlos, Silla F, Shainer G, Schultz S (2015) Local and remote GPUs perform similar with EDR 100G InfiniBand. In: 16th International Middleware Conference, Middleware Industry’15, pp 4:1–4:7. ACM. https://doi.org/10.1145/2830013.2830015. ISBN 978-1-4503-3727-4

  30. 30.

    Reaño C, Silla F (2016) Reducing the performance gap of remote GPU virtualization with infiniband connect-IB. In: 21st IEEE Symposium on Computers and Communications, ISCC’16, pp 920–925. https://doi.org/10.1109/ISCC.2016.7543854

  31. 31.

    Silla F, Iserte S, Reaño C, Prades J (2017) On the benefits of the remote GPU virtualization mechanism: the rCUDA case. Concurrency and Computation: Practice and Experience, pp e4072–e4089. https://doi.org/10.1002/cpe.4072. ISSN 1532-0634

  32. 32.

    Hong CH, Spence I, Nikolopoulos DS (Dec 2017b) Fairgv: fair and fast gpu virtualization. IEEE Trans Parallel Distrib Syst 28(12):3472–3485. https://doi.org/10.1109/TPDS.2017.2717908. ISSN 1045-9219

  33. 33.

    Pérez F, Reaño C, Silla F (2016) Providing CUDA acceleration to KVM virtual machines in infiniband clusters with rCUDA. In: 16th IFIP International Conference on Distributed Applications and Interoperable Systems, DAIS’16, pp 82–95. Springer. https://doi.org/10.1007/978-3-319-39577-7_7. ISBN 978-3-319-39577-7

  34. 34.

    Prades J, Reaño C, Silla F (2016) CUDA acceleration for Xen virtual machines in infiniband clusters with rCUDA. In: Proceedings of the 21st ACM SIGPLAN Symposium on Principles and Practice of Parallel Programming, PPoPP’16, pp 35:1–35:2. ACM, New York, NY, USA. https://doi.org/10.1145/2851141.2851181. ISBN 978-1-4503-4092-2

  35. 35.

    Raffaele M, Giulio G, Giuliano L, Marco L, Carlo P, Carmine F, Valentina P, Cheol-Ho H, Spence Ivor TA, Nikolopoulos Dimitrios S (2017) On the virtualization of CUDA based GPU remoting on ARM and X86 machines in the GVirtuS framework. Int J Parallel Program 45(5):1142–1163. https://doi.org/10.1007/s10766-016-0462-1

  36. 36.

    Diab KM, Rafique MM, Hefeeda M (2013) Dynamic sharing of GPUs in cloud systems. In: IEEE Parallel and Distributed Processing Symposium Workshops PhD Forum (IPDPSW), pp 947–954. https://doi.org/10.1109/IPDPSW.2013.102

  37. 37.

    Jun TJ, Dung VQ, Yoo MH, Kim D, Cho H, Hahm J (2014) GPGPU enabled HPC cloud platform based on OpenStack. In: The International Conference for High Performance Computing, Networking, Storage and Analysis. http://hdl.handle.net/10203/211249

  38. 38.

    Iserte S, Clemente-Castelló FJ, Castelló A, Mayo R, Quintana-Ortí ES (2016) Enabling GPU virtualization in cloud environments. In: Proceedings of the 6th International Conference on Cloud Computing and Services Science, pp 249–256. https://doi.org/10.5220/0005780502490256

  39. 39.

    Popa L, Ratnasamy S, Iannaccone G, Krishnamurthy A, Stoica I (2010) A cost comparison of datacenter network architectures. In: Proceedings of the 6th International Conference, Co-NEXT’10, pp 16:1–16:12. New York, NY, USA. https://doi.org/10.1145/1921168.1921189. ISBN 978-1-4503-0448-1

  40. 40.

    Al-Fares M, Loukissas A, Vahdat A (2008) A scalable, commodity data center network architecture. In: Proceedings of the ACM SIGCOMM 2008 Conference on Data Communication, SIGCOMM’08, pp 63–74. ACM, New York, NY, USA. https://doi.org/10.1145/1402958.1402967. ISBN 978-1-60558-175-0

  41. 41.

    Calero JMA, Aguado JG (2015) MonPaaS: an adaptive Monitoring Platform as a Service for cloud computing infrastructures and services. IEEE Trans Serv Comput 8(1):65–78. https://doi.org/10.1109/TSC.2014.2302810. ISSN 1939-1374

  42. 42.

    Lilja David J (2004) Measuring computer performance. A practitioner’s guide. Cambridge University Press, Cambridge

  43. 43.

    Peña AJ, Reaño C, Silla F, Mayo R, Quintana-Ortí ES, Duato J (2014) A complete and efficient CUDA-sharing solution for HPC clusters. Parallel Comput 40(10):574–588. https://doi.org/10.1016/j.parco.2014.09.011. ISSN 0167-8191

Download references

Author information

Correspondence to Raúl Peña-Ortiz.

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Gutiérrez-Aguado, J., Claver, J.M. & Peña-Ortiz, R. Toward a transparent and efficient GPU cloudification architecture. J Supercomput 75, 3640–3672 (2019). https://doi.org/10.1007/s11227-018-2720-z

Download citation

Keywords

  • Cloud computing
  • Distributed computing
  • Platform virtualization
  • Computer network management
  • GPU cloudification