CORRFUNC: Blazing Fast Correlation Functions with AVX512FSIMD Intrinsics
Correlation functions are widely used in extra-galactic astrophysics to extract insights into how galaxies occupy dark matter halos and in cosmology to place stringent constraints on cosmological parameters. A correlation function fundamentally requires computing pair-wise separations between two sets of points and then computing a histogram of the separations. Corrfunc is an existing open-source, high-performance software package for efficiently computing a multitude of correlation functions. In this paper, we will discuss the SIMD AVX512F kernels within Corrfunc, capable of processing 16 floats or 8 doubles at a time. The latest manually implemented Corrfunc AVX512F kernels show a speedup of up to \(\sim \)4\(\times \) relative to compiler-generated code for double-precision calculations. The AVX512F kernels show \(\sim \)1.6\(\times \) speedup relative to the AVX kernels and compares favorably to a theoretical maximum of \(2\times \). In addition, by pruning pairs with too large of a minimum possible separation, we achieve a \(\sim \)5–10% speedup across all the SIMD kernels. Such speedups highlight the importance of programming explicitly with SIMD vector intrinsics for complex calculations that can not be efficiently vectorized by compilers. Corrfunc is publicly available at https://github.com/manodeep/Corrfunc/.
KeywordsCorrelation functions AVX512 SIMD intrinsics Molecular dynamics Spatial distance histograms Applications
MS was primarily supported by NSF Career Award (AST-1151650) during main Corrfunc design and development. MS was also supported by the by the Australian Research Council Laureate Fellowship (FL110100072) awarded to Stuart Wyithe and by funds for the Theoretical Astrophysical Observatory (TAO). TAO is part of the All-Sky Virtual Observatory and is funded and supported by Astronomy Australia Limited, Swinburne University of Technology, and the Australian Government. The latter is provided though the Commonwealth’s Education Investment Fund and National Collaborative Research Infrastructure Strategy (NCRIS), particularly the National eResearch Collaboration Tools and Resources (NeCTAR) project. Parts of this research were conducted by the Australian Research Council Centre of Excellence for All Sky Astrophysics in 3 Dimensions (ASTRO 3D), through project number CE170100013.
- 1.Chhugani, J., et al.: Billion-particle SIMD-friendly two-point correlation on large-scale HPC cluster systems. In: Proceedings of the International Conference on High Performance Computing, Networking, Storage and Analysis, SC 2012, pp. 1:1–1:11. IEEE Computer Society Press, Los Alamitos (2012). http://dl.acm.org/citation.cfm?id=2388996.2388998
- 2.Gonnet, P.: A simple algorithm to accelerate the computation of non-bonded interactions in cell-based molecular dynamics simulations. J. Comput. Chem. 28(2), 570–573 (2007). https://doi.org/10.1002/jcc.20563. https://onlinelibrary.wiley.com/doi/abs/10.1002/jcc.20563CrossRefGoogle Scholar
- 3.Hockney, R., Goel, S., Eastwood, J.: Quiet high-resolution computer models of a plasma. J. Comput. Phys. 14(2), 148–158 (1974). https://doi.org/10.1016/0021-9991(74)90010-2. http://www.sciencedirect.com/science/article/pii/0021999174900102CrossRefGoogle Scholar
- 4.Lindahl, E., Hess, B., van der Spoel, D.: GROMACS 3.0: a package for molecular simulation and trajectory analysis. Mol. Model. Annu. 7, 306–317 (2001). https://doi.org/10.1007/s008940100045. https://link.springer.com/article/10.1007/s008940100045CrossRefGoogle Scholar
- 5.Peebles, P.J.E.: The Large-Scale Structure of the Universe. Princeton University Press, Princeton (1980)Google Scholar
- 6.Quentrec, B., Brot, C.: New method for searching for neighbors in molecular dynamics computations. J. Comput. Phys. 13(3), 430–432 (1973). https://doi.org/10.1016/0021-9991(73)90046-6. http://www.sciencedirect.com/science/article/pii/0021999173900466CrossRefGoogle Scholar
- 8.Sinha, M., Lehman, G.: Corrfunc—a suite of blazing fast correlation functions on the CPU. MNRAS (2019). (Submitted to MNRAS)Google Scholar
- 9.Willis, J.S., Schaller, M., Gonnet, P., Bower, R.G., Draper, P.W.: An efficient SIMD implementation of pseudo-Verlet lists for neighbour interactions in particle-based codes. ArXiv e-prints, April 2018Google Scholar