Affinity-On-Next-Touch: An Extension to the Linux Kernel for NUMA Architectures

  • Stefan Lankes
  • Boris Bierbaum
  • Thomas Bemmerl
Part of the Lecture Notes in Computer Science book series (LNCS, volume 6067)


For many years now, NUMA architectures are being used in the design of large shared memory computers and they are gaining importance even for smaller-scale systems. On a NUMA machine, the distribution of data has a significant impact on the performance and scalability of data-intensive programs, because of the difference in access speed between local and remote parts of the memory system. Unfortunately, memory access patterns are often very complex and difficult to predict. Affinity-on-next-touch may be a useful page placement strategy to distribute the data in a suitable manner, but support for it is missing from the current Linux kernel. In this paper, we present an extension to the Linux kernel which implements this strategy and compare it with alternative approaches.


Memory Bandwidth Memory Access Pattern Page Cache List Operation Page Frame 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Dagum, L., Menon, R.: OpenMP: An Industry-Standard API for Shared-Memory Programming. IEEE Computational Science & Engineering 5(1), 46–55 (1998)CrossRefGoogle Scholar
  2. 2.
    Dormanns, M., Lankes, S., Bemmerl, T., Bolz, G., Pfeiffle, E.: Parallelization of an Airline Flight-Scheduling Module on a SCI-Coupled NUMA Shared Memory Cluster. In: High Performance Computing Systems and Applications (HPCS), Kingston, Canada (June 1999)Google Scholar
  3. 3.
    Dormanns, M.: Shared-Memory Parallelization of the GROMOS 1996 Molecular Dynamics Code. In: Hellwagner, H., Reinefeld, A. (eds.). SCI: Scalable Coherent Interface. Springer, Heidelberg (1999)Google Scholar
  4. 4.
    Noordergraaf, L., van der Pas, R.: Performance Experiences on Sun’s WildFire Prototype. In: Proceedings of the 1999 ACM/IEEE Conference on Supercomputing, Portland, Oregon, USA (November 1999)Google Scholar
  5. 5.
    Bircsak, J., Craig, P., Crowell, R., Cvetanovic, Z., Harris, J., Nelson, C.A., Offner, C.D.: Extending OpenMP for NUMA machines. In: Proceedings of the 2000 ACM/IEEE Conference on Supercomputing, Dallas, Texas, USA (November 2000)Google Scholar
  6. 6.
    Löf, H., Holmgren, S.: Affinity-on-next-touch: Increasing the Performance of an Industrial PDE Solver on a cc-NUMA System. In: Proceedings of the 19th Annual International Conference on Supercomputing, Cambridge, Massachusetts, USA, pp. 387–392 (June 2005)Google Scholar
  7. 7.
    Terboven, C., an Mey, D., Schmidl, D., Jin, H., Reichstein, T.: Data and Thread Affinity in OpenMP Programs. In: Proceedings of the 2008 Workshop on Memory Access on future Processors: A solved problem? ACM International Conference on Computing Frontiers, Ischia, Italy, pp. 377–384 (May 2008)Google Scholar
  8. 8.
    Love, R.: Linux Kernel Development, 2nd edn. Novell Press (2005)Google Scholar
  9. 9.
    McCalpin, J.D.: Memory Bandwidth and Machine Balance in Current High Performance Computers. In: IEEE Computer Society Technical Committee on Computer Architecture (TCCA) Newsletter, pp. 19–25 (December 1995)Google Scholar
  10. 10.
    Nordén, M., Löf, H., Rantakokko, J., Holmgren, S.: Geographical Locality and Dynamic Data Migration for OpenMP Implementations of Adaptive PDE Solvers. In: Proceedings of the 2nd International Workshop on OpenMP (IWOMP), Reims, France, pp. 382–393 (June 2006)Google Scholar
  11. 11.
    Schermerhorn, L.T.: Automatic page migration for linux (a matter of hygiene). In: 2007 (2007)Google Scholar
  12. 12.
    Goglin, B., Furmento, N.: Enabling High-Performance Memory Migration for Multithreaded Applications on Linux. In: Proceedings of the 23rd IEEE International Parallel and Distributed Processing Symposium (IPDPS 2009), Workshop on Multithreaded Architectures and Applications (MTAAP 2009), Rome, Italy (2009)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2010

Authors and Affiliations

  • Stefan Lankes
    • 1
  • Boris Bierbaum
    • 1
  • Thomas Bemmerl
    • 1
  1. 1.Chair for Operating SystemsRWTH Aachen UniversityAachenGermany

Personalised recommendations