Estimation of Correlations in Large Samples

Abstract.

Astrophysics provides prime examples of discrete and continuous random fields, such as the distribution of galaxies on the sky, and Cosmic Microwave Background (CMB) maps. The clustering of such fields encodes information about the underlying theory of structure formation, such as the initial conditions, subsequent linear and non-linear gravitational amplification, as well as the physical processes of galaxy formation and can be characterized by their two-point and higher order correlation functions. Recently, new algorithms based on a hierarchical or KDtree representation of data revolutionized the measurement of correlation functions and related quantities, allowing analyses of the future wide field galaxy surveys, as well as megapixel CMB maps. While optimal lossless analysis appears to be infeasible even with the new algorithms, it was shown recently that high precision, but slightly suboptimal, estimators help. We present several examples, where this approach solves analysis problems intractable with traditional, non-hierarchical, optimal methods.