Skip to main content
Book cover

International Conference on Embedded Computer Systems

SAMOS 2020: Embedded Computer Systems: Architectures, Modeling, and Simulation pp 228–242Cite as

A Quantitative Study of Locality in GPU Caches

Part of the Lecture Notes in Computer Science book series (LNTCS,volume 12471)

Abstract

Traditionally, GPUs only had programmer-managed caches. The advent of hardware-managed caches accelerated the use of GPUs for general-purpose computing. However, as GPU caches are shared by thousands of threads, they are usually a victim of contention and can suffer from thrashing and high miss rate, in particular, for memory-divergent workloads. As data locality is crucial for performance, there have been several efforts focusing on exploiting data locality in GPUs. However, there is a lack of quantitative analysis of data locality and data reuse in GPUs. In this paper, we quantitatively study the data locality and its limits in GPUs. We observe that data locality is much higher than exploited by current GPUs. We show that, on the one hand, the low spatial utilization of cache lines justifies the use of demand-fetched caches. On the other hand, the much higher actual spatial utilization of cache lines shows the lost spatial locality and presents opportunities for further optimizing the cache design.

Keywords

  • Data locality
  • GPU caches
  • Memory divergence

This is a preview of subscription content, access via your institution.

Chapter
USD   29.95
Price excludes VAT (USA)
  • DOI: 10.1007/978-3-030-60939-9_16
  • Chapter length: 15 pages
  • Instant PDF download
  • Readable on all devices
  • Own it forever
  • Exclusive offer for individuals only
  • Tax calculation will be finalised during checkout
eBook
USD   79.99
Price excludes VAT (USA)
  • ISBN: 978-3-030-60939-9
  • Instant PDF download
  • Readable on all devices
  • Own it forever
  • Exclusive offer for individuals only
  • Tax calculation will be finalised during checkout
Softcover Book
USD   99.99
Price excludes VAT (USA)
Learn about institutional subscriptions
Fig. 1.
Fig. 2.
Fig. 3.
Fig. 4.
Fig. 5.
Fig. 6.
Fig. 7.

References

  1. Al-Kiswany, S., Gharaibeh, A., Santos-Neto, E., Yuan, G., Ripeanu, M.: StoreGPU: exploiting graphics processing units to accelerate distributed storage systems. In: Proceedings of the 17th International Symposium on High Performance Distributed Computing, HPDC (2008)

    Google Scholar 

  2. Bakhoda, A., Yuan, G.L., Fung, W.W.L., Wong, H., Aamodt, T.M.: Analyzing CUDA workloads using a detailed GPU simulator. In: Proceedings of the IEEE International Symposium on Performance Analysis of Systems and Software, ISPASS (2009)

    Google Scholar 

  3. Cederman, D., Chatterjee, B., Tsigas, P.: Understanding the performance of concurrent data structures on graphics processors. In: Kaklamanis, C., Papatheodorou, T., Spirakis, P.G. (eds.) Euro-Par 2012. LNCS, vol. 7484, pp. 883–894. Springer, Heidelberg (2012). https://doi.org/10.1007/978-3-642-32820-6_87

    CrossRef  Google Scholar 

  4. Che, S., et al.: Rodinia: a benchmark suite for heterogeneous computing. In: Proceedings of the IEEE International Symposium on Workload Characterization, IISWC (2009)

    Google Scholar 

  5. Chen, X., Chang, L.W., Rodrigues, C.I., Lv, J., Wang, Z., Hwu, W.M.: Adaptive cache management for energy-efficient GPU computing. In: Proceedings of the 47th IEEE/ACM International Symposium on Microarchitecture, MICRO (2014)

    Google Scholar 

  6. Hong, S., Oguntebi, T., Olukotun, K.: Efficient parallel graph exploration on multi-core CPU and GPU. In: International Conference on Parallel Architectures and Compilation Techniques, PACT (2011)

    Google Scholar 

  7. Jia, W., Shaw, K.A., Martonosi, M.: Characterizing and improving the use of demand-fetched caches in GPUs. In: Proceedings of the 26th ACM International Conference on Supercomputing, ICS (2012)

    Google Scholar 

  8. Lal, S., Lucas, J., Andersch, M., Alvarez-Mesa, M., Elhossini, A., Juurlink, B.: GPGPU workload characteristics and performance analysis. In: Proceedings of the 14th International Conference on Embedded Computer Systems: Architectures, Modeling, and Simulation, SAMOS (2014)

    Google Scholar 

  9. Li, A., van den Braak, G.J., Kumar, A., Corporaal, H.: Adaptive and transparent cache bypassing for GPUs. In: Proceedings of the International Conference for High Performance Computing, Networking, Storage and Analysis, SC (2015)

    Google Scholar 

  10. Li, A., Song, S.L., Liu, W., Liu, X., Kumar, A., Corporaal, H.: Locality-aware CTA clustering for modern GPUs. In: Proceedings of the International Conference on Architectural Support for Programming Languages and Operating Systems, ASPLOS (2017)

    Google Scholar 

  11. Li, C., Song, S.L., Dai, H., Sidelnik, A., Hari, S.K.S., Zhou, H.: Locality-driven dynamic GPU cache bypassing. In: Proceedings of the 29th ACM on International Conference on Supercomputing, ICS (2015)

    Google Scholar 

  12. Meng, J., Tarjan, D., Skadron, K.: Dynamic warp subdivision for integrated branch and memory divergence tolerance. In: Proceedings of the 37th Annual International Symposium on Computer Architecture, ISCA (2010)

    Google Scholar 

  13. NVIDIA: CUDA: Compute Unified Device Architecture (2007). http://developer.nvidia.com/object/gpucomputing.html

  14. Reguly, I.Z., Giles, M.: Efficient sparse matrix-vector multiplication on cache-based GPUs. In: Proceedings of the Innovative Parallel Computing, InPar (2012)

    Google Scholar 

  15. Rogers, T.G., O’Connor, M., Aamodt, T.M.: Cache-conscious wavefront scheduling. In: Proceedings of the 45th Annual IEEE/ACM International Symposium on Microarchitecture, MICRO (2012)

    Google Scholar 

  16. Sartori, J., Kumar, R.: Branch and data herding: reducing control and memory divergence for error-tolerant GPU applications. In: Proceedings of the 21st International Conference on Parallel Architectures and Compilation Techniques, PACT (2012)

    Google Scholar 

  17. Tarjan, D., Meng, J., Skadron, K.: Increasing memory miss tolerance for SIMD cores. In: Proceedings of the Conference on High Performance Computing Networking, Storage and Analysis, SC (2009)

    Google Scholar 

  18. Xiao, S., Lin, H., Feng, W.C.: Accelerating protein sequence search in a heterogeneous computing system. In: IEEE International Parallel and Distributed Processing Symposium, IPDPS (2011)

    Google Scholar 

  19. Xie, X., Liang, Y., Wang, Y., Sun, G., Wang, T.: Coordinated static and dynamic cache bypassing for GPUs. In: 2015 IEEE 21st International Symposium on High Performance Computer Architecture, HPCA (2015)

    Google Scholar 

  20. Zhang, E.Z., Jiang, Y., Guo, Z., Tian, K., Shen, X.: On-the-fly elimination of dynamic irregularities for GPU computing. In: Proceedings of the 16th International Conference on Architectural Support for Programming Languages and Operating Systems, ASPLOS (2011)

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Technische Universität Berlin, Berlin, Germany

    Sohan Lal & Ben Juurlink

Authors
  1. Sohan Lal
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Sohan Lal .

Editor information

Editors and Affiliations

  1. Department of Computer Science and Engineering, University of California San Diego, La Jolla, CA, USA

    Prof. Alex Orailoglu

  2. Department of Electrical and Computer Engineering, Fraunhofer IESE, Kaiserslautern, Germany

    Matthias Jung

  3. Department of Computer Science, Friedrich-Alexander University, Erlangen, Germany

    Marc Reichenbach

Rights and permissions

Reprints and Permissions

Copyright information

© 2020 Springer Nature Switzerland AG

About this paper

Verify currency and authenticity via CrossMark

Cite this paper

Lal, S., Juurlink, B. (2020). A Quantitative Study of Locality in GPU Caches. In: Orailoglu, A., Jung, M., Reichenbach, M. (eds) Embedded Computer Systems: Architectures, Modeling, and Simulation. SAMOS 2020. Lecture Notes in Computer Science(), vol 12471. Springer, Cham. https://doi.org/10.1007/978-3-030-60939-9_16

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-60939-9_16

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-60938-2

  • Online ISBN: 978-3-030-60939-9

  • eBook Packages: Computer ScienceComputer Science (R0)