Skip to main content
SpringerLink
Log in
Book cover

International Conference on High Performance Computing

ISC High Performance 2017: High Performance Computing pp 15–29Cite as

On the Scalability of Data Reduction Techniques in Current and Upcoming HPC Systems from an Application Perspective

  • Conference paper
  • First Online:

Part of the book series: Lecture Notes in Computer Science ((LNTCS,volume 10524))

Abstract

We implement and benchmark parallel I/O methods for the fully-manycore driven particle-in-cell code PIConGPU. Identifying throughput and overall I/O size as a major challenge for applications on today’s and future HPC systems, we present a scaling law characterizing performance bottlenecks in state-of-the-art approaches for data reduction. Consequently, we propose, implement and verify multi-threaded data-transformations for the I/O library ADIOS as a feasible way to trade underutilized host-side compute potential on heterogeneous systems for reduced I/O latency.

This project has received funding from the European Unions Horizon 2020 research and innovation programme under grant agreement No. 654220. An award of computer time was provided by the Innovative and Novel Computational Impact on Theory and Experiment (INCITE) program. This research used resources of the Oak Ridge Leadership Computing Facility at the Oak Ridge National Laboratory, which is supported by the Office of Science of the U.S. Department of Energy under Contract No. DE-AC05-00OR22725.

This is a preview of subscription content, log in via an institution.

Notes

  1. 1.

    An experimental development preview with compression support in parallel HDF5 was announced after our measurements in February 2017.

References

  1. Abbasi, H., Wolf, M., Eisenhauer, G., Klasky, S., Schwan, K., Zheng, F.: Datastager: scalable data staging services for petascale applications. Clust. Comput. 13(3), 277–290 (2010). doi:10.1007/s10586-010-0135-6

    Article  Google Scholar 

  2. Alted, F.: blosc 1.11.4-dev, March 2017. https://github.com/Blosc/c-blosc

  3. Ayachit, U., Bauer, A., Geveci, B., O’Leary, P., Moreland, K., Fabian, N., Mauldin, J.: ParaView catalyst: enabling in situ data analysis and visualization. In: Proceedings of the First Workshop on In Situ Infrastructures for Enabling Extreme-Scale Analysis and Visualization, ISAV2015, pp. 25–29. ACM (2015). doi:10.1145/2828612.2828624

  4. Bhimji, W., Bard, D., Romanus, M., Paul, D., Ovsyannikov, A., Friesen, B., Bryson, M., Correa, J., Lockwood, G.K., Tsulaia, V., et al.: Accelerating science with the NERSC burst buffer early user program. In: Proceedings of Cray Users Group (2016)

    Google Scholar 

  5. Birdsall, C., Langdon, A.: Plasma physics via computer simulation. The Adam Hilger series on plasma physics. McGraw-Hill, New York (1985). ISBN 9780070053717

    Google Scholar 

  6. Burau, H., Widera, R., Honig, W., Juckeland, G., Debus, A., Kluge, T., Schramm, U., Cowan, T.E., Sauerbrey, R., Bussmann, M.: PIConGPU: a fully relativistic particle-in-cell code for a gpu cluster. IEEE Trans. Plasma Sci. 38(10), 2831–2839 (2010)

    Article  Google Scholar 

  7. Burtscher, M., Mukka, H., Yang, A., Hesaaraki, F.: Real-time synthesis of compression algorithms for scientific data. In: SC16: International Conference for High Performance Computing, Networking, Storage and Analysis, pp. 264–275, November 2016. doi:10.1109/SC.2016.22

  8. Bussmann, M., Burau, H., Cowan, T.E., Debus, A., Huebl, A., Juckeland, G., Kluge, T., Nagel, W.E., Pausch, R., Schmitt, F., Schramm, U., Schuchart, J., Widera, R.: Radiative signatures of the relativistic Kelvin-Helmholtz instability. In: Proceedings of the International Conference on High Performance Computing, Networking, Storage and Analysis, SC 2013, pp. 5:1–5:12. ACM (2013). doi:10.1145/2503210.2504564

  9. Collet, Y., Skibinski, P., Terrell, N., Purcell, S.: Contributors: Zstandard (zstd) 1.1.4 - fast real-time compression algorithm, March 2017. https://github.com/facebook/zstd

  10. Docan, C., Parashar, M., Klasky, S.: DataSpaces: an interaction and coordination framework or coupled simulation workflows. In: Proceedings of 19th International Symposium on High Performance and Distributed Computing (HPDC 2010), June 2010. doi:10.1007/s10586-011-0162-y

  11. Eckert, C.H.J.: Enhancements of the massively parallel memory allocator scatteralloc and its adaption to the general interface mallocMC, October 2014. doi:10.5281/zenodo.34461

  12. Grismayer, T., Alves, E., Fonseca, R., Silva, L.: dc-magnetic-field generation in unmagnetized shear flows. Phys. Rev. Lett. 111, 015005 (2013). doi:10.1103/PhysRevLett.111.015005

    Article  Google Scholar 

  13. Gunderson, S.H., Evlogimenos, A.: Contributors: Snappy 1.1.1 - a fast compressor/decompressor (2011). https://github.com/google/snappy

  14. Hockney, R., Eastwood, J.: Computer Simulation Using Particles. Taylor & Francis, Bristol (1988). ISBN: 9780852743928

    Google Scholar 

  15. Huebl, A., Lehe, R., Vay, J.L., Grote, D.P., Sbalzarini, I., Kuschel, S., Bussmann, M.: openPMD 1.0.0: a meta data standard for particle and mesh based data, November 2015. doi:10.5281/zenodo.33624

  16. Huebl, A., Schmitt, F., Widera, R., Grund, A., Schumann, C., Eckert, C., Bukva, A., Pausch, R.: libSplash: 1.6.0: SerialDataCollector filename API, October 2016. doi:10.5281/zenodo.163609

  17. Huebl, A., Widera, R., Grund, A., Pausch, R., Burau, H., Debus, A., Garten, M., Worpitz, B., Zenker, E., Winkler, F., Eckert, C., Tietze, S., Schneider, B., Knespel, M., Bussmann, M.: PIConGPU 0.2.4: Charge of bound electrons, openPMD axis range, manipulate by position, March 2017. doi:10.5281/zenodo.346005

  18. Huebl, A., et al.: Supplementary materials: On the scalability of data reduction techniques in current and upcoming HPC systems from an application perspective, April 2017. doi:10.5281/zenodo.545780

  19. Lindstrom, P.: Fixed-rate compressed floating-point arrays. IEEE Trans. Vis. Comput. Graph. 20(12), 2674–2683 (2014). doi:10.1109/TVCG.2014.2346458

    Article  Google Scholar 

  20. Liu, Q., Logan, J., Tian, Y., Abbasi, H., Podhorszki, N., Choi, J.Y., Klasky, S., Tchoua, R., Lofstead, J., Oldfield, R., et al.: Hello ADIOS: the challenges and lessons of developing leadership class I/O frameworks. Concurr. Comput. Pract. Exp. 26(7), 1453–1473 (2014)

    Article  Google Scholar 

  21. Matthes, A., Huebl, A., Widera, R., Grottel, S., Gumhold, S., Bussmann, M.: In situ, steerable, hardware-independent and data-structure agnostic visualization with ISAAC. Supercomputing Frontiers and Innovations 3(4) (2016). http://superfri.org/superfri/article/view/114

  22. McLay, R., James, D., Liu, S., Cazes, J., Barth, W.: A user-friendly approach for tuning parallel file operations. In: Proceedings of the International Conference for High Performance Computing, Networking, Storage and Analysis, SC 2014, pp. 229–236. IEEE Press (2014). doi:10.1109/SC.2014.24, https://github.com/TACC/t3pio

  23. Meredith, J.S., Ahern, S., Pugmire, D., Sisneros, R.: EAVL: The extreme-scale analysis and visualization library. In: Childs, H., Kuhlen, T., Marton, F. (eds.) Eurographics Symposium on Parallel Graphics and Visualization. The Eurographics Association (2012). doi:10.2312/EGPGV/EGPGV12/021-030

  24. Meuer, H.W., Strohmaier, E., Dongarra, J., Simon, H., Meuer, M.: November 2016 — TOP500 Supercomputer Sites, June 2016. https://www.top500.org/lists/2016/11/. Accessed 22 Mar 2017

  25. Corporation, N.: NVIDIA IndeX 1.4. https://developer.nvidia.com/index

  26. Tao, D., Sheng, D., Chen, Z., Cappello, F.: Significantly improving lossy compression for scientific data sets based on multidimensional prediction and error-controlled quantization. In: IPDPS 2017: Proceedings of the 31th IEEE International Parallel and Distributed Processing Symposium, May 2017

    Google Scholar 

  27. The HDF Group: Hierarchical data format version 5 (C-API: 1.8.14) (2000–2017). http://www.hdfgroup.org/HDF5

  28. Whitlock, B., Favre, J.M., Meredith, J.S.: Parallel in situ coupling of simulation with a fully featured visualization system. In: Kuhlen, T., Pajarola, R., Zhou, K. (eds.) Eurographics Symposium on Parallel Graphics and Visualization. The Eurographics Association (2011). doi:10.2312/EGPGV/EGPGV11/101-109

  29. Zenker, E., Widera, R., Huebl, A., Juckeland, G., Knüpfer, A., Nagel, W.E., Bussmann, M.: Performance-portable many-core plasma simulations: porting PIConGPU to openpower and beyond. In: Taufer, M., Mohr, B., Kunkel, J.M. (eds.) ISC High Performance 2016. LNCS, vol. 9945, pp. 293–301. Springer, Cham (2016). doi:10.1007/978-3-319-46079-6_21

    Chapter  Google Scholar 

  30. Zenker, E., Worpitz, B., Widera, R., Huebl, A., Juckeland, G., Knüpfer, A., Nagel, W.E., Bussmann, M.: Alpaka-an abstraction library for parallel kernel acceleration. In: 2016 IEEE International on Parallel and Distributed Processing Symposium Workshops, pp. 631–640. IEEE (2016)

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Helmholtz-Zentrum Dresden – Rossendorf, Dresden, Germany

    Axel Huebl, René Widera, Alexander Matthes & Michael Bussmann

  2. Technische Universität Dresden, Dresden, Germany

    Axel Huebl, Felix Schmitt & Alexander Matthes

  3. NVIDIA ARC GmbH, Berlin, Germany

    Felix Schmitt

  4. Oak Ridge National Laboratory, Oak Ridge, TN, USA

    Norbert Podhorszki, Jong Youl Choi & Scott Klasky

Authors
  1. Axel Huebl
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. René Widera
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Felix Schmitt
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Alexander Matthes
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Norbert Podhorszki
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Jong Youl Choi
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Scott Klasky
    View author publications

    You can also search for this author in PubMed Google Scholar

  8. Michael Bussmann
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding authors

Correspondence to Axel Huebl or Michael Bussmann .

Editor information

Editors and Affiliations

  1. Deutsches Klimarechenzentrum (DKRZ), Hamburg, Hamburg, Germany

    Julian M. Kunkel

  2. TITECH, Tokyo, Japan

    Rio Yokota

  3. Department of Computer Science, University of Delaware, Newark, Delaware, USA

    Michela Taufer

  4. Lawrence Berkeley National Laboratory, Berkeley, California, USA

    John Shalf

Rights and permissions

Reprints and permissions

Copyright information

© 2017 Springer International Publishing AG

About this paper

Cite this paper

Huebl, A. et al. (2017). On the Scalability of Data Reduction Techniques in Current and Upcoming HPC Systems from an Application Perspective. In: Kunkel, J., Yokota, R., Taufer, M., Shalf, J. (eds) High Performance Computing. ISC High Performance 2017. Lecture Notes in Computer Science(), vol 10524. Springer, Cham. https://doi.org/10.1007/978-3-319-67630-2_2

Download citation

  • DOI: https://doi.org/10.1007/978-3-319-67630-2_2

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-319-67629-6

  • Online ISBN: 978-3-319-67630-2

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation