Computer Science - Research and Development

, Volume 28, Issue 2, pp 241–251

Towards I/O analysis of HPC systems and a generic architecture to collect access patterns

Authors

    • Universität Hamburg—Deutsches Klimarechenzentrum GmbH
  • Julian M. Kunkel
    • Universität Hamburg—Deutsches Klimarechenzentrum GmbH
  • Michaela Zimmer
    • Universität Hamburg—Deutsches Klimarechenzentrum GmbH
  • Thomas Ludwig
    • Universität Hamburg—Deutsches Klimarechenzentrum GmbH
  • Michael Resch
    • High Performance Computing Center Stuttgart (HLRS)Universität Stuttgart
  • Thomas Bönisch
    • High Performance Computing Center Stuttgart (HLRS)Universität Stuttgart
  • Xuan Wang
    • High Performance Computing Center Stuttgart (HLRS)Universität Stuttgart
  • Andriy Chut
    • High Performance Computing Center Stuttgart (HLRS)Universität Stuttgart
  • Alvaro Aguilera
    • Zentrum für Informationsdienste und HochleistungsrechnenTechnische Universität Dresden
  • Wolfgang E. Nagel
    • Zentrum für Informationsdienste und HochleistungsrechnenTechnische Universität Dresden
  • Michael Kluge
    • Zentrum für Informationsdienste und HochleistungsrechnenTechnische Universität Dresden
  • Holger Mickler
    • Zentrum für Informationsdienste und HochleistungsrechnenTechnische Universität Dresden
Special Issue Paper

DOI: 10.1007/s00450-012-0221-5

Cite this article as:
Wiedemann, M.C., Kunkel, J.M., Zimmer, M. et al. Comput Sci Res Dev (2013) 28: 241. doi:10.1007/s00450-012-0221-5

Abstract

In high-performance computing applications, a high-level I/O call will trigger activities on a multitude of hardware components. These are massively parallel systems supported by huge storage systems and internal software layers. Their complex interplay currently makes it impossible to identify the causes for and the locations of I/O bottlenecks. Existing tools indicate when a bottleneck occurs but provide little guidance in identifying the cause or improving the situation.

We have thus initiated Scalable I/O for Extreme Performance to find solutions for this problem. To achieve this goal in SIOX, we will build a system to record access information on all layers and components, to recognize access patterns, and to characterize the I/O system. The system will ultimately be able to recognize the causes of the I/O bottlenecks and propose optimizations for the I/O middleware that can improve I/O performance, such as throughput rate and latency. Furthermore, the SIOX system will be able to support decision making while planning new I/O systems.

In this paper, we introduce the SIOX system and describe its current status: We first outline our approach for collecting the required access information. We then provide the architectural concept, the methods for reconstructing the I/O path and an excerpt of the interface for data collection. This paper focuses especially on the architecture, which collects and combines the relevant access information along the I/O path, and which is responsible for the efficient transfer of this information. An abstract modelling approach allows us to better understand the complexity of the analysis of the I/O activities on parallel computing systems, and an abstract interface allows us to adapt the SIOX system to various HPC file systems.

Keywords

I/O analysis I/O path Causality tree

Copyright information

© Springer-Verlag 2012