High Performance Computing for Analyzing PB-Scale Data in Nuclear Experiments and Simulations

Abstract

By performance improvement of computers and expansion of experiment facilities, output data having became huge. In near future, the output data will become petabyte (PB)-scale. It will become increasingly important how huge data is analyzed efficiently and derive useful information. To analysis huge data efficiently, we are constructing large-scale data integrated analysis system which treats terabytes-petabytes data. In this system, two elemental technologies, i.e., heterogeneous processor and distributed parallel computing framework with fault-tolerance are implemented. The former and the latter are effective for computation dominant processes and data I/O dominant processes, respectively. First, we have applied acceleration by the heterogeneous processor to experimental data and estimated its performance. The processor accelerated experimental data processing substantially. Next, then we have constructed a prototype of distributed parallel computing system for simulation data and carried out processing test. We have found the notice points for application these elemental techniques.