Asymptotic formulae for likelihood-based tests of new physics

  • Glen Cowan
  • Kyle Cranmer
  • Eilam Gross
  • Ofer VitellsEmail author
Open Access
Special Article - Tools for Experiment and Theory


We describe likelihood-based statistical tests for use in high energy physics for the discovery of new phenomena and for construction of confidence intervals on model parameters. We focus on the properties of the test procedures that allow one to account for systematic uncertainties. Explicit formulae for the asymptotic distributions of test statistics are derived using results of Wilks and Wald. We motivate and justify the use of a representative data set, called the “Asimov data set”, which provides a simple method to obtain the median experimental sensitivity of a search or measurement as well as fluctuations about this expectation.


Monte Carlo Simulation Systematic Uncertainty Strength Parameter Nuisance Parameter Error Band 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


  1. 1.
    S.S. Wilks, The large-sample distribution of the likelihood ratio for testing composite hypotheses. Ann. Math. Stat. 9, 60–62 (1938) CrossRefzbMATHGoogle Scholar
  2. 2.
    A. Wald, Tests of statistical hypotheses concerning several parameters when the number of observations is large. Trans. Am. Math. Soc. 54(3), 426–482 (1943) CrossRefzbMATHMathSciNetGoogle Scholar
  3. 3.
    I. Asimov, Franchise, in Isaac Asimov: The Complete Stories, vol. 1 (Broadway Books, New York, 1990) Google Scholar
  4. 4.
    V. Bartsch, G. Quast, Expected signal observability at future experiments, CMS Note 2005/004 (2003), (available on CMS information server) Google Scholar
  5. 5.
    ATLAS Collaboration, Expected performance of the ATLAS experiment, detector, trigger and physics. CERN-OPEN-2008-020, Geneva (2008). e-print: arXiv:0901.0512
  6. 6.
    ALEPH, DELPHI, and L3 and OPAL Collaborations, Search for the standard model higgs boson at LEP. Phys. Lett. B 565, 61–75 (2003). CERN-EP/2003-011 CrossRefADSGoogle Scholar
  7. 7.
    A. Stuart, J.K. Ord, S. Arnold, Kendall’s Advanced Theory of Statistics, Classical Inference and the Linear Model, vol. 2A, 6th edn. (Oxford University Press, London, 1999), and earlier editions by Kendall and Stuart Google Scholar
  8. 8.
    R.D. Cousins, G.J. Feldman, Phys. Rev. D 57, 3873 (1998) CrossRefADSGoogle Scholar
  9. 9.
    H. Chernoff, On the distribution of the likelihood ratio. Ann. Math. Stat. 25, 573–578 (1954) CrossRefzbMATHMathSciNetGoogle Scholar
  10. 10.
    M. Abramowitz, I.A. Stegun, Handbook of Mathematical Functions (Dover, New York, 1972). Sect. 26.4.25 zbMATHGoogle Scholar
  11. 11.
    Wikipedia, The Free Encyclopedia, Noncentral chi-square distribution. Wikimedia Foundation, Inc., 6 July 2010 Google Scholar
  12. 12.
    T. Aaltonen et al., Phys. Rev. Lett. 104, 061802 (2010). e-Print: arXiv:1001.4162 [hep-ex] CrossRefADSGoogle Scholar
  13. 13.
    K. Cranmer, Frequentist hypothesis testing with background uncertainty, in Proceedings of PHYSTAT 2003, SLAC, ed. by L. Lyons et al., Stanford, California, 8–11 September 2003, pp. 261–264 Google Scholar
  14. 14.
    T. Junk, Nucl. Instrum. Methods Phys. Res., Sect. A 434, 435 (1999) CrossRefADSGoogle Scholar
  15. 15.
    A.L. Read, J. Phys. G 28, 2693 (2002) CrossRefADSMathSciNetGoogle Scholar
  16. 16.
    R.D. Cousins, J.T. Linnemann, J. Tucker, Nucl. Instrum. Methods Phys. Res., Sect. A 595, 480–501 (2008). e-Print: arXiv:physics/0702156v4 [] CrossRefGoogle Scholar
  17. 17.
    E. Gross, O. Vitells, Trial factors or the look elsewhere effect in high energy physics. arXiv:1005.1891 [] (2010)
  18. 18.
    L. Moneta, K. Belasco, K. Cranmer et al., The RooStats project, in Proceedings of ACAT, Jaipur, India (2010). arXiv:1009.1003 []. Google Scholar
  19. 19.
    R. Brun, F. Rademakers, ROOT: An object oriented data analysis framework,. Nucl. Instrum. Methods A 389, 81–86 (1997) CrossRefADSGoogle Scholar
  20. 20.
    W. Verkerke, D.P. Kirkby, The RooFit toolkit for data modeling, in Proceedings for CHEP03 (2003). physics/0306116 Google Scholar
  21. 21.
    F. James, M. Roos, Minuit: a system for function minimization and analysis of the parameter errors and correlations. Comput. Phys. Commun. 10, 343–367 (1975) CrossRefADSGoogle Scholar

Copyright information

© The Author(s) 2011

Authors and Affiliations

  • Glen Cowan
    • 1
  • Kyle Cranmer
    • 2
  • Eilam Gross
    • 3
  • Ofer Vitells
    • 3
    Email author
  1. 1.Physics DepartmentRoyal Holloway, University of LondonEghamUK
  2. 2.Physics DepartmentNew York UniversityNew YorkUSA
  3. 3.Weizmann Institute of ScienceRehovotIsrael

Personalised recommendations