Lightweight and Accurate Silent Data Corruption Detection in Ordinary Differential Equation Solvers
Silent data corruptions (SDCs) are errors that corrupt the system or falsify results while remaining unnoticed by firmware or operating systems. In numerical integration solvers, SDCs that impact the accuracy of the solver are considered significant. Detecting SDCs in high-performance computing is necessary because results need to be trustworthy and the increase of the number and complexity of components in emerging large-scale architectures makes SDCs more likely to occur. Until recently, SDC detection methods consisted in replicating the processes of the execution or in using checksums (for example algorithm-based fault tolerance). Recently, new detection methods have been proposed relying on mathematical properties of numerical kernels or performing data analysis of the results modified by the application. None of those methods, however, provide a lightweight solution guaranteeing that all significant SDCs are detected. We propose a new method called Hot Rod as a solution to this problem. It checks and potentially corrects the data produced by numerical integration solvers. Our theoretical model shows that all significant SDCs can be detected. We present two detectors and conduct experiments on streamline integration from the WRF meteorology application. Compared with the algorithmic detection methods, the accuracy of our first detector is increased by \(52\,\%\) with a similar false detection rate. The second detector has a false detection rate one order of magnitude lower than these detection methods while improving the detection accuracy by \(23\,\%\). The computational overhead is lower than \(5\,\%\) in both cases. The model has been developed for an explicit Runge-Kutta method, although it can be generalized to other solvers.
KeywordsResilience Fault tolerance Runge-kutta Numerical integration solvers HPC SDC
We express our gratitude to Julie Bessac for assistance with the algorithm and Gail Pieper for comments that greatly improved the manuscript. We also gratefully acknowledge the use of the services and facilities of the Decaf project at Argonne National Laboratory, supported by U.S. Department of Energy, Office of Science, Advanced Scientific Computing Research, under Contract DE-AC02-06CH11357, program manager Lucy Nowell. We also thank the anonymous reviewers for their helpful comments.
- 1.Bagatin, M., Gerardin, S.: Ionizing Radiation Effects in Electronics: From Memories to Imagers. Devices, Circuits, and Systems. CRC Press, Cleveland, Boca Raton (2015)Google Scholar
- 6.Ceschino, F., Kuntzmann, J.: Numerical solution of initial value problems (1966)Google Scholar
- 7.Chen, L., Avizienis, A.: N-version programming: A fault-tolerance approach to reliability of software operation. In: Digest of Papers FTCS-8, pp. 3–9 (1978)Google Scholar
- 8.Di, S., Cappello, F.: Adaptive impact-driven detection of silent data corruption for HPC applications. In: IEEE Transactions on Parallel and Distributed Systems (2016)Google Scholar
- 11.Guo, H., He, W., Peterka, T., Shen, H.W., Collis, S.M., Helmus, J.J.: Finite-time lyapunov exponents and lagrangian coherent structures in uncertain unsteady flows. In: IEEE TVCG (Proceedings of the PacificVis 16) 22, to appear (2016)Google Scholar
- 13.Hwang, A.A., Stefanovici, I.A., Schroeder, B.: Cosmic rays don’t strike twice: understanding the nature of DRAM errors and the implications for system design. ACM SIGPLAN Not. 47, 111–122 (2012)Google Scholar
- 16.McLoughlin, T., Laramee, R.S., Peikert, R., Post, F.H., Chen, M.: Over two decades of integration-based, geometric flow visualization. In: Eurographics 2009 State of the Art Report, pp. 73–92. Munich, Germany (2009)Google Scholar
- 17.Peterka, T., Ross, R., Nouanesengsy, B., Lee, T.Y., Shen, H.W., Kendall, W., Huang, J.: A study of parallel particle tracing for steady-state and time-varying flow fields. In: IPDPS, pp. 580–591. IEEE (2011)Google Scholar