Abstract
This chapter introduces the key statistical concepts that are necessary to understand and analyze high-energy astronomical data. Our goal is to present enough information such that a reader may learn to judge the quality of their inferences and properly evaluate claims made in the literature. We first describe the role played by statistical methodology in analysis in Sect. 6.1. We then introduce the Poisson likelihood, which governs how counts fluctuations are described, in the context of several other useful distributions, in Sect. 6.2. Methods to determine and propagate uncertainties are discussed in Sect. 6.3. This is followed by a discussion of model fitting in Sect. 6.4, followed by hypothesis tests and model comparisons in Sect. 6.5. We list some targeted references from astronomical literature, and some books for further reading, in Sect. 6.6.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
Notes
- 1.
Matsumoto 1997; http://www.math.sci.hiroshima-u.ac.jp/~m-mat/MT/emt.html.
- 2.
Statisticians use Greek letters for variables that describe model parameters and Roman letters for quantities that describe the data. In particular, they use \(\lambda \) as the symbol to represent brightness or strength of a source. This is sometimes also called intensity, but always has units [count].
- 3.
In this, it is similar to the Weibull distribution, which has an \(\frac{1}{\alpha }~x^\beta \) form in the exponential instead of \(x\beta \).
- 4.
This bias is sometimes called the Eddington Bias, though strictly speaking the Eddington Bias also includes the effects of population characteristics. That is, the measured source strengths are affected by both the Type M bias as well as there being more weaker sources that have upward fluctuations in the measurements than stronger sources that have downward fluctuations.
References
Papers and Monographs
Y. Avni, Energy spectra of X-ray clusters of galaxies. ApJ 210, 642 (1976), https://ui.adsabs.harvard.edu/abs/1976ApJ...210..642A/abstract
L. Bretthorst, Bayesian Fourier analysis (1988), https://bayes.wustl.edu/glb/book.pdf
W. Cash, Parameter estimation in astronomy through application of the likelihood ratio. ApJ 228, 939 (1979), https://ui.adsabs.harvard.edu/abs/1979ApJ...228..939C/abstract
D.W. Hogg, Data analysis recipes: probability calculus for inference (2012), arXiv:1205.4446, https://arxiv.org/pdf/1205.4446.pdf
D.W. Hogg, J. Bovy, D. Lang, Data analysis recipes: fitting a model to data (2010), arXiv:1008.4686, https://arxiv.org/pdf/1008.4686.pdf
D.W. Hogg, D. Foreman-Mackey, Data analysis recipes: using Markov Chain Monte Carlo. ApJS 236, 11 (2018), https://ui.adsabs.harvard.edu/abs/2018ApJS..236...11H/abstract
E.D. Feigelson, P.I. Nelson, Statistical methods for astronomical data with upper limits. I. Univariate distributions. ApJ 293, 192 (1985), https://ui.adsabs.harvard.edu/abs/1985ApJ...293..192F/abstract
P.E. Freeman, V. Kashyap, R. Rosner, D.Q. Lamb, A wavelet-based algorithm for the spatial analysis of Poisson data. ApJS 138, 185 (2002), https://ui.adsabs.harvard.edu/abs/2002ApJS..138..185F/abstract
N. Gehrels, Confidence limits for small numbers of events in astrophysical data. ApJ 303, 336 (1986), https://ui.adsabs.harvard.edu/abs/1986ApJ...303..336G/abstract
A. Gelman, J. Carlin, Beyond power calculations: assessing type S (sign) and type M (magnitude) errors. Perspect. Psychol. Sci. 9, 641 (2014), https://doi.org/10.1177/1745691614551642
T. Isobe, E.D. Feigelson, P.I. Nelson, Statistical methods for astronomical data with upper limits. II. Correlation and regression. ApJ 306, 490 (1986), https://ui.adsabs.harvard.edu/abs/1986ApJ...306..490I/abstract
J.S. Kaastra, On the use of C-stat in testing models for X-ray spectra. A&A 605, 51 (2017), https://ui.adsabs.harvard.edu/abs/2017A&A...605A..51K/abstract
V.L. Kashyap, D.A. van Dyk, A. Connors, P.E. Freeman, A. Siemiginowska, X. Jin, A. Zezas, On computing upper limits to source intensities. ApJ 719, 900 (2010), https://ui.adsabs.harvard.edu/abs/2010ApJ...719..900K/abstract
K. Kearns, F. Primini, D. Alexander, Bias-free parameter estimation with few counts, by iterative chi-squared minimization. ASPC 77, 331 (1995), https://ui.adsabs.harvard.edu/abs/1995ASPC...77..331K/abstract
B.C. Kelly, Some aspects of measurement error in linear regression of astronomical data. ApJ 665, 1489 (2007), https://ui.adsabs.harvard.edu/abs/2007ApJ...665.1489K/abstract
T. Loredo, From Laplace to supernova SN 1987 A: Bayesian inference in astrophysics (1990), http://hosting.astro.cornell.edu/staff/loredo/bayes/L90-LaplaceToSN1987A-scan.pdf
M. Matsumoto, T. Nishimura, Mersenne twister: a 623-dimensionally equidistributed uniform pseudo-random number generator. ACM Trans. Model. Comput. Simul. 8, 1 (1998), https://doi.org/10.1145/272991.272995
R. Neal, Probabilistic inference using Markov Chain Monte Carlo methods. Techincal Report CRG-TR-93-1 (Department of Computer Science, University of Toronto, 1993), https://www.cs.toronto.edu/~radford/review.abstract.html
S. Nieuwenhaus, B.U. Forstmann, E.-J. Wagenmakers, Erroneous analyses of interactions in neuroscience: a problem of significance. Nat. Neurosci. 14, 1105 (2011), https://doi.org/10.1038/nn.2886
T. Park, V.L. Kashyap, A. Siemiginowska, D.A. van Dyk, A. Zezas, C. Heinke, B. Wargelin, Bayesian estimation of hardness ratios: modeling and computations. ApJ 652, 610 (2006), https://ui.adsabs.harvard.edu/abs/2006ApJ...652..610P/abstract
F.A. Primini, V.L. Kashyap, Determining X-ray source intensity and confidence bounds in crowded fields. ApJ 796, 24 (2014), https://ui.adsabs.harvard.edu/abs/2014ApJ...796...24P/abstract
R. Protassov, D.A. van Dyk, A. Connors, V.L. Kashyap, A. Siemiginowska, Statistics, handle with care: detecting multiple model components with the likelihood ratio test. ApJ 571, 545 (2002), https://ui.adsabs.harvard.edu/abs/2002ApJ...571..545P/abstract
J.D. Scargle, J.P. Norris, B. Jackson, J. Chiang, Studies in astronomical time series analysis. VI. Bayesian block representations. ApJ 764, 167 (2013), https://ui.adsabs.harvard.edu/abs/2013ApJ...764..167S/abstract
J.H.M.M. Schmitt, Statistical analysis of astronomical data containing upper bounds: general methods and examples drawn from X-ray astronomy. ApJ 293, 178 (1985), https://ui.adsabs.harvard.edu/abs/1985ApJ...293..178S/abstract
J.H.M.M. Schmitt, T. Maccacaro, Number-counts slope estimation in the presence of Poisson noise. ApJ 310, 334 (1986), https://ui.adsabs.harvard.edu/abs/1986ApJ...310..334S/abstract
D.A. van Dyk, A. Connors, V.L. Kashyap, A. Siemiginowska, Analysis of energy spectra with low photon counts via Bayesian posterior simulation. ApJ 548, 224 (2001), https://ui.adsabs.harvard.edu/abs/2001ApJ...548..224V/abstract
Books
J. Babu, E. Feigelson, Astrostatistics (1996), https://www.crcpress.com/Astrostatistics/Babu-Feigelson/p/book/9780412983917
P.R. Bevington, D.K. Robinson, Data Reduction and Error Analysis for the Physical Sciences, 3rd edn. (2003), http://hosting.astro.cornell.edu/academics/courses/astro3310/Books/Bevington_opt.pdf
L. Wasserman, All of Non-Parametric Statistics (2006), http://www.stat.cmu.edu/~larry/all-of-nonpar/
C.K. Rasmussen, C.E. Williams, Gaussian Processes for Machine Learning (2006), http://www.gaussianprocess.org/gpml/
W.H. Press, S.A. Teukolsky, W.T. Vetterling, B.P. Flannery, Numerical Recipes: The Art of Scientific Computing, 3rd edn. (2007), http://numerical.recipes
E. Feigelson, J. Babu, Modern Statistical Methods for Astronomy with R Applications (2012), https://astrostatistics.psu.edu/MSMA/
K. Arnaud, R. Smith, A. Siemiginowska, Handbook of X-Ray Astronomy (2011), http://hea-www.cfa.harvard.edu/~rsmith/xrayastronomyhandbook/
P. Gregory, Bayesian Logical Data Analysis for Physical Sciences (2012), https://doi.org/10.1017/CBO9780511791277
A. Gelman, J.B. Carlin, H.S. Stern, D.B. Dunson, A. Vehtari, D.B. Rubin, Bayesian Data Analysis, 3rd edn. (2013), http://www.stat.columbia.edu/~gelman/book/
E. Robinson, Data Analysis for Scientists and Engineers (2016), https://press.princeton.edu/titles/10911.html
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2020 Springer Nature Singapore Pte Ltd.
About this chapter
Cite this chapter
Kashyap, V.L. (2020). Basics of Astrostatistics. In: Bambi, C. (eds) Tutorial Guide to X-ray and Gamma-ray Astronomy. Springer, Singapore. https://doi.org/10.1007/978-981-15-6337-9_6
Download citation
DOI: https://doi.org/10.1007/978-981-15-6337-9_6
Published:
Publisher Name: Springer, Singapore
Print ISBN: 978-981-15-6336-2
Online ISBN: 978-981-15-6337-9
eBook Packages: Physics and AstronomyPhysics and Astronomy (R0)