Kerncraft: A Tool for Analytic Performance Modeling of Loop Kernels

  • Julian Hammer
  • Jan Eitzinger
  • Georg Hager
  • Gerhard Wellein
Conference paper


Achieving optimal program performance requires deep insight into the interaction between hardware and software. For software developers without an in-depth background in computer architecture, understanding and fully utilizing modern architectures is close to impossible. Analytic loop performance modeling is a useful way to understand the relevant bottlenecks of code execution based on simple machine models. The Roofline Model and the Execution-Cache-Memory (ECM) model are proven approaches to performance modeling of loop nests. In comparison to the Roofline model, the ECM model can also describes the single-core performance and saturation behavior on a multicore chip.We give an introduction to the Roofline and ECM models, and to stencil performance modeling using layer conditions (LC). We then present Kerncraft, a tool that can automatically construct Roofline and ECM models for loop nests by performing the required code, data transfer, and LC analysis. The layer condition analysis allows to predict optimal spatial blocking factors for loop nests. Together with the models it enables an ab-initio estimate of the potential benefits of loop blocking optimizations and of useful block sizes. In cases where LC analysis is not easily possible, Kerncraft supports a cache simulator as a fallback option. Using a 25-point long-range stencil we demonstrate the usefulness and predictive power of the Kerncraft tool.


Loop Nest Cache Size Cache Line Memory Hierarchy Layer Condition 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.



This work was in part funded by the German Academic Exchange Service’s (DAAD) FITweltweit program and the Federal Ministry of Education and Research (BMBF) SKAMPY grant.


  1. 1.
    Djoudi, L., Barthou, D., Carribault, P., Lemuet, C., Acquaviva, J.T., Jalby, W., et al.: MAQAO: modular assembler quality analyzer and optimizer for itanium 2. In: The 4th Workshop on EPIC architectures and compiler technology, San Jose (2005).
  2. 2.
    Evans, C., Ingerson, B., Ben-Kiki, O.: YAML Ain’t Markup Language (2001). Google Scholar
  3. 3.
    Grosser, T., Groesslinger, A., Lengauer, C.: Polly – performing polyhedral optimizations on a low-level intermediate representation. Parallel Process. Lett. 22 (04), 1250010 (2012). doi: 10.1142/S0129626412500107 MathSciNetCrossRefGoogle Scholar
  4. 4.
    Hammer, J.: Layer conditions (2016). Google Scholar
  5. 5.
    Hammer, J.: pycachesim – a single-core cache hierarchy simulator written in python (2015).
  6. 6.
    Hammer, J., Hager, G., Eitzinger, J., Wellein, G.: Automatic loop kernel analysis and performance modeling with kerncraft. In: Proceedings of the 6th International Workshop on Performance Modeling, Benchmarking, and Simulation of High Performance Computing Systems, PMBS ’15, pp. 4:1–4:11. ACM, New York (2015). doi: 10.1145/2832087.2832092
  7. 7.
    Hockney, R.W., Curington, I.J.: f 1∕2: a parameter to characterize memory and communication bottlenecks. Parallel Comput. 10 (3), 277–286 (1989). doi: 10.1016/0167-8191(89)90100-2
  8. 8.
    Hofmann, J., Fey, D., Riedmann, M., Eitzinger, J., Hager, G., Wellein, G.: Performance analysis of the Kahan-enhanced scalar product on current multi-core and many-core processors. Concurr. Comput. Pract. Exper. (2016). doi: 10.1002/cpe.3921 Google Scholar
  9. 9.
  10. 10.
    ISO: ISO C Standard 1999. Technical Report (1999). ISO/IEC 9899:1999 draft
  11. 11.
    Kerncraft toolkit (2015).
  12. 12.
    Kreutzer, M., Thies, J., Röhrig-Zöllner, M., Pieper, A., Shahzad, F., Galgon, M., Basermann, A., Fehske, H., Hager, G., Wellein, G.: GHOST: building blocks for high performance sparse linear algebra on heterogeneous systems. Int. J. Parallel Prog. 1–27 (2016). doi: 10.1007/s10766-016-0464-z
  13. 13.
    Lo, Y., Williams, S., Van Straalen, B., Ligocki, T., Cordery, M., Wright, N., Hall, M., Oliker, L.: Roofline model toolkit: a practical tool for architectural and program analysis. In: Jarvis, S.A., Wright, S.A., Hammond, S.D. (eds.) High Performance Computing Systems. Performance Modeling, Benchmarking, and Simulation. Lecture Notes in Computer Science, vol. 8966, pp. 129–148. Springer International Publishing, Berlin (2015). doi: 10.1007/978-3-319-17248-4_7Google Scholar
  14. 14.
    McCalpin, J.D.: STREAM: sustainable memory bandwidth in high performance computers. Technical Report, University of Virginia, Charlottesville, VA (1991–2007). A continually updated technical report
  15. 15.
    Narayanan, S.H.K., Norris, B., Hovland, P.D.: Generating performance bounds from source code. In: 2010 39th International Conference on Parallel Processing Workshops (ICPPW), pp. 197–206 (2010). doi: 10.1109/ICPPW.2010.37
  16. 16.
    Rivera, G., Tseng, C.W.: Tiling optimizations for 3D scientific computations. In: Supercomputing, ACM/IEEE 2000 Conference, pp. 32–32 (2000). doi: 10.1109/SC.2000.10015
  17. 17.
    Stengel, H., Treibig, J., Hager, G., Wellein, G.: Quantifying performance bottlenecks of stencil computations using the execution-cache-memory model. In: Proceedings of the 29th ACM International Conference on Supercomputing, ICS ’15, pp. 207–216. ACM, New York (2015). doi: 10.1145/2751205.2751240
  18. 18.
    SymPy Development Team: SymPy: python library for symbolic mathematics (2016).
  19. 19.
    Treibig, J., Hager, G., Wellein, G.: Likwid: a lightweight performance-oriented tool suite for x86 multicore environments. In: Proceedings of PSTI2010, the First International Workshop on Parallel Software Tools and Tool Infrastructures, San Diego, CA (2010)Google Scholar
  20. 20.
    Unat, D., Chan, C., Zhang, W., Williams, S., Bachan, J., Bell, J., Shalf, J.: ExaSAT: an exascale co-design tool for performance modeling. Int. J. High Perform. Comput. Appl. 29 (2), 209–232 (2015). doi:10.1177/1094342014568690CrossRefGoogle Scholar
  21. 21.
    Williams, S., Waterman, A., Patterson, D.: Roofline: an insightful visual performance model for multicore architectures. Commun. ACM 52 (4), 65–76 (2009). doi: 10.1145/1498765.1498785 CrossRefGoogle Scholar
  22. 22.
    Wittmann, M., Hager, G., Zeiser, T., Treibig, J., Wellein, G.: Chip-level and multi-node analysis of energy-optimized lattice Boltzmann CFD simulations. Concurrency Comput. Pract. Exper. 28 (7), 2295–2315 (2016). doi: 10.1002/cpe.3489 CrossRefGoogle Scholar

Copyright information

© Springer International Publishing AG 2017

Authors and Affiliations

  • Julian Hammer
    • 1
  • Jan Eitzinger
    • 1
  • Georg Hager
    • 1
  • Gerhard Wellein
    • 1
  1. 1.Erlangen Regional Computing CenterErlangenGermany

Personalised recommendations