Abstract
With this work, we explore the feasibility of using in situ data binning techniques to achieve significant data reductions for particle data, and study the associated errors for several post-hoc analysis techniques. We perform an application study in collaboration with fusion simulation scientists on data sets up to 489 GB per time step. We consider multiple ways to carry out the binning, and determine which techniques work the best for this simulation. With the best techniques we demonstrate reduction factors as large as 109x with low error percentage.
The U.S. government retains certain licensing rights. This is a U.S. government work and certain licensing rights apply.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Ahern, S., et al.: Scientific discovery at the exascale. In: Report from the DOE ASCR 2011 Workshop on Exascale Data Management (2011)
Bauer, A.C., et al.: In Situ Methods, Infrastructures, and Applications on High Performance Computing Platforms, a State-of-the-art (STAR) Report. In: Computer Graphics Forum, Proceedings of Eurovis 2016, vol. 35, no. 3, June 2016. lBNL-1005709
Chang, C., et al.: Compressed ion temperature gradient turbulence in diverted tokamak edge. Phys. Plasmas (1994-Present) 16(5), 056108 (2009)
Childs, H., et al.: Extreme scaling of production visualization software on diverse architectures. IEEE Comput. Graph. Appl. 30(3), 22–31 (2010)
Childs, H., et al.: Visualization at extreme scale concurrency. In: Bethel, E.W., Childs, H., Hansen, C. (eds.) High Performance Visualization: Enabling Extreme-Scale Scientific Insight. CRC Press, Boca Raton (2012)
Fabian, N., et al.: The paraview coprocessing library: a scalable, general purpose in situ visualization library. In: 2011 IEEE Symposium on Large Data Analysis and Visualization (LDAV), pp. 89–96. IEEE (2011)
Kress, J., Churchill, R.M., Klasky, S., Kim, M., Childs, H., Pugmire, D.: Preparing for in situ processing on upcoming leading-edge supercomputers. Supercomput. Front. Innov. 3(4), 49–65 (2016)
Kress, J., Pugmire, D., Klasky, S., Childs, H.: Visualization and analysis requirements for in situ processing for a large-scale fusion simulation code. In: Proceedings of the 2nd Workshop on In Situ Infrastructures for Enabling Extreme-scale Analysis and Visualization, pp. 45–50. IEEE Press (2016)
Liu, Q., et al.: Hello adios: the challenges and lessons of developing leadership class i/o frameworks. Concurr. Comput.: Pract. Exp. 26(7), 1453–1473 (2014). https://doi.org/10.1002/cpe.3125
Lo, L., Sewell, C., Ahrens, J.P.: Piston: a portable cross-platform framework for data-parallel visualization operators. In: EGPGV, pp. 11–20 (2012)
Lofstead, J.F., Klasky, S., Schwan, K., Podhorszki, N., Jin, C.: Flexible io and integration for scientific codes through the adaptable io system (adios). In: Proceedings of the 6th International Workshop on Challenges of Large Applications in Distributed Environments, CLADE 2008, pp. 15–24. ACM, New York (2008). https://doi.org/10.1145/1383529.1383533
Meredith, J.S., Ahern, S., Pugmire, D., Sisneros, R.: EAVL: the extreme-scale analysis and visualization library. In: Eurographics Symposium on Parallel Graphics and Visualization, pp. 21–30. The Eurographics Association (2012)
Moreland, K., Ayachit, U., Geveci, B., Ma, K.L.: Dax toolkit: a proposed framework for data analysis and visualization at extreme scale. In: 2011 IEEE Symposium on Large Data Analysis and Visualization (LDAV), pp. 97–104, October 2011
Moreland, K., et al.: VTK-m: accelerating the visualization toolkit for massively threaded architectures. IEEE Comput. Graph. Appl. (CG&A) 36(3), 48–58 (2016)
Neuroth, T., Sauer, F., Wang, W., Ethier, S., Ma, K.L.: Scalable visualization of discrete velocity decompositions using spatially organized histograms. In: 2015 IEEE 5th Symposium on Large Data Analysis and Visualization (LDAV), pp. 65–72. IEEE (2015)
Oldfield, R.A., Widener, P., Maccabe, A.B., Ward, L., Kordenbrock, T.: Effcient data-movement for lightweight i/o. In: 2006 IEEE International Conference on Cluster Computing, pp. 1–9, September 2006. https://doi.org/10.1109/CLUSTR.2006.311897
Pugmire, D., Kress, J., Meredith, J., Podhorszki, N., Choi, J., Klasky, S.: Towards scalable visualization plugins for data staging workows. In: Big Data Analytics: Challenges and Opportunities (BDAC 2014) Workshop at Supercomputing Conference, November 2014
Reach, C., North, C.: Bandlimited olap cubes for interactive big data visualization. In: 2015 IEEE 5th Symposium on Large Data Analysis and Visualization (LDAV), pp. 107–114. IEEE (2015)
Schatz, K., Müller, C., Krone, M., Schneider, J., Reina, G., Ertl, T.: Interactive visual exploration of a trillion particles. In: 2016 IEEE 6th Symposium on Large Data Analysis and Visualization (LDAV), pp. 56–64. IEEE (2016)
Tchoua, R., et al.: Adios visualization schema: a first step towards improving interdisciplinary collaboration in high performance computing. In: 2013 IEEE 9th International Conference on eScience (eScience), pp. 27–34. IEEE (2013)
Vishwanath, V., Hereld, M., Papka, M.: Toward simulation-time data analysis and i/o acceleration on leadership-class systems. In: 2011 IEEE Symposium on Large Data Analysis and Visualization (LDAV), pp. 9–14 (2011). https://doi.org/10.1109/LDAV.2011.6092178
Whitlock, B., Favre, J.M., Meredith, J.S.: Parallel in situ coupling of simulation with a fully featured visualization system. In: Proceedings of the 11th Eurographics conference on Parallel Graphics and Visualization, pp. 101–109. Eurographics Association (2011)
Ye, Y.C., et al.: In situ generated probability distribution functions for interactive post hoc visualization and analysis. In: 2016 IEEE 6th Symposium on Large Data Analysis and Visualization (LDAV), pp. 65–74. IEEE (2016)
Acknowledgements
This research used resources of the Oak Ridge Leadership Computing Facility, which is a DOE Office of Science User Facility supported under Contract DE-AC05-00OR22725.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2018 Springer Nature Switzerland AG
About this paper
Cite this paper
Kress, J., Choi, J., Klasky, S., Churchill, M., Childs, H., Pugmire, D. (2018). Binning Based Data Reduction for Vector Field Data of a Particle-In-Cell Fusion Simulation. In: Yokota, R., Weiland, M., Shalf, J., Alam, S. (eds) High Performance Computing. ISC High Performance 2018. Lecture Notes in Computer Science(), vol 11203. Springer, Cham. https://doi.org/10.1007/978-3-030-02465-9_15
Download citation
DOI: https://doi.org/10.1007/978-3-030-02465-9_15
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-02464-2
Online ISBN: 978-3-030-02465-9
eBook Packages: Computer ScienceComputer Science (R0)