Advertisement

An OpenMP Extension Library for Memory Affinity

  • Dirk Schmidl
  • Tim Cramer
  • Christian Terboven
  • Dieter an Mey
  • Matthias S. Müller
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 8766)

Abstract

OpenMP 4.0 extended affinity support to allow pinning of threads to places. Places are an abstraction of machine locations which in many cases do not require extensive hardware knowledge by the user. For memory affinity, i.e. data initialization and migration on NUMA systems, support is still missing in OpenMP. In this work we present an extension library for OpenMP which implements round-robin memory initialization over places and memory migration, either explicitly or implicitly. The latter is presented with an implementation based on a next-touch algorithm for Linux. We study the overhead of our methods with a simple model that allows to predict if migration is beneficial or not for a certain use case and we demonstrate the correctness of the migration methods and the correctness of our model prediction with the STREAM benchmark and an implementation of a CG method. Finally, we discuss how memory affinity could be integrated in future OpenMP versions.

Keywords

Memory Access High Performance Computing Migration Strategy Memory Access Pattern Data Access Pattern 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. 1.
    Bircsak, J., Craig, P., Crowell, R., Cvetanovic, Z., Harris, J., Nelson, C.A., Offner, C.D.: Extending OpenMP for NUMA Machines. In: Proceedings of the 2000 ACM/IEEE Conference on Supercomputing, SC 2000. IEEE Computer Society, Washington, DC (2000)Google Scholar
  2. 2.
    Broquedis, F., Furmento, N., Goglin, B., Namyst, R., Wacrenier, P.-A.: Dynamic Task and Data Placement over NUMA Architectures: An OpenMP Runtime Perspective. In: Müller, M.S., de Supinski, B.R., Chapman, B.M. (eds.) IWOMP 2009. LNCS, vol. 5568, pp. 79–92. Springer, Heidelberg (2009)CrossRefGoogle Scholar
  3. 3.
    Goglin, B., Furmento, N.: Enabling high-performance memory migration for multithreaded applications on LINUX. In: IEEE International Symposium on Parallel Distributed Processing, IPDPS 2009, pp. 1–9 (May 2009)Google Scholar
  4. 4.
    Corbet, J.: AutoNUMA: The other approach to NUMA scheduling, http://lwn.net/Articles/488709/ (last visited on Sepetmber 05, 2014)
  5. 5.
    Kleen, A.: A NUMA API for LINUX. Technical linux whitepaper, Novell (April 2005)Google Scholar
  6. 6.
    Lankes, S., Bierbaum, B., Bemmerl, T.: Affinity-on-next-touch: An Extension to the Linux Kernel for NUMA Architectures. In: Wyrzykowski, R., Dongarra, J., Karczewski, K., Wasniewski, J. (eds.) PPAM 2009, Part I. LNCS, vol. 6067, pp. 576–585. Springer, Heidelberg (2010)CrossRefGoogle Scholar
  7. 7.
    Laudon, J., Lenoski, D.: The SGI Origin: A ccNUMA Highly Scalable Server. In: Proceedings of the 24th Annual International Symposium on Computer Architecture, ISCA 1997, pp. 241–251. ACM, New York (1997)Google Scholar
  8. 8.
    Löf, H., Holmgren, S.: Affinity-on-next-touch: Increasing the Performance of an Industrial PDE Solver on a cc-NUMA System. In: Proceedings of the 19th Annual International Conference on Supercomputing, ICS 2005, pp. 387–392. ACM, New York (2005)Google Scholar
  9. 9.
    McCalpin, J.D.: STREAM: Sustainable Memory Bandwidth in High Performance Computers (1995)Google Scholar
  10. 10.
    Nikolopoulos, D.S., Papatheodorou, T.S., Polychronopoulos, C.D., Labarta, J., Ayguadé, E.: Leveraging Transparent Data Distribution in OpenMP via User-Level Dynamic Page Migration. In: Valero, M., Joe, K., Kitsuregawa, M., Tanaka, H. (eds.) ISHPC 2000. LNCS, vol. 1940, pp. 415–427. Springer, Heidelberg (2000)CrossRefGoogle Scholar
  11. 11.
    Noordergraaf, L., van der Pas, R.: Performance experiences on sun’s wildfire prototype. In: Proceedings of the 1999 ACM/IEEE Conference on Supercomputing, SC 1999. ACM, New York (1999)Google Scholar
  12. 12.
    OpenMP ARB, OpenMP Application Program Interface, v. 4.0, http://www.openmp.org (last visited on September 05, 2014)
  13. 13.
    Terboven, C., an Mey, D., Schmidl, D., Jin, H., Reichstein, T.: Data and Thread Affinity in OpenMP Programs. In: Proceedings of the 2008 Workshop on Memory Access on Future Processors: A solved Problem? MAW 2008, pp. 377–384. ACM, New York (2008)Google Scholar
  14. 14.
    Terboven, C., Schmidl, D., Cramer, T., an Mey, D.: Assessing OpenMP Tasking Implementations on NUMA Architectures. In: Chapman, B.M., Massaioli, F., Müller, M.S., Rorro, M. (eds.) IWOMP 2012. LNCS, vol. 7312, pp. 182–195. Springer, Heidelberg (2012)CrossRefGoogle Scholar
  15. 15.
    Terboven, C., Schmidl, D., Cramer, T., an Mey, D.: Task-Parallel Programming on NUMA Architectures. In: Kaklamanis, C., Papatheodorou, T., Spirakis, P.G. (eds.) Euro-Par 2012. LNCS, vol. 7484, pp. 638–649. Springer, Heidelberg (2012)CrossRefGoogle Scholar

Copyright information

© Springer International Publishing Switzerland 2014

Authors and Affiliations

  • Dirk Schmidl
    • 1
    • 2
    • 3
  • Tim Cramer
    • 1
    • 2
    • 3
  • Christian Terboven
    • 1
    • 2
    • 3
  • Dieter an Mey
    • 1
    • 2
    • 3
  • Matthias S. Müller
    • 1
    • 2
    • 3
  1. 1.IT CenterRWTH Aachen UniversityAachenGermany
  2. 2.High Performance ComputingRWTH Aachen UniversityAachenGermany
  3. 3.JARA - High-Performance ComputingAachenGermany

Personalised recommendations