Skip to main content
SpringerLink
Log in

Comparison between different methods of model selection in cosmology

  • Regular Article
  • Published:
The European Physical Journal Plus Aims and scope Submit manuscript

Abstract

There are several methods for model selection in cosmology which have at least two major goals, that of finding the correct model or predicting well. In this work, we discuss through a study of well-known model selection methods like Akaike information criterion (AIC), Bayesian information criterion (BIC), deviance information criterion (DIC) and Bayesian evidence, how these different goals are pursued in each paradigm. We also apply another method for model selection which is less seen in cosmological literature, the cross-validation method. Using these methods, we will compare two different scenarios in cosmology: \(\Lambda \)CDM model and dynamical dark energy. We show that each of the methods tends to different results in model selection. While BIC and Bayesian evidence overrule the dynamical dark energy scenarios with 2 or 3 extra degree of freedom, the DIC and cross-validation method prefer these dynamical models to \(\Lambda \)CDM model. Assuming the numerical results of different analysis and combining cosmological and statistical aspects of the subject, we propose cross-validation as an interesting method for model selection in cosmology that can lead to different results in comparison with usual methods of model selection.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3

Similar content being viewed by others

Data Availability Statement

This manuscript has associated data in a data repository [Authors’ comment: All observational data used in this paper are available upon request by contacting the corresponding author.]

References

  1. A.G. Riess, A.V. Filippenko, P. Challis et al., AJ 116, 1009 (1998)

    Article  ADS  Google Scholar 

  2. S. Perlmutter, G. Aldering, G. Goldhaber et al., ApJ 517, 565 (1999)

    Article  ADS  Google Scholar 

  3. S.M. Carroll, V. Duvvuri, M. Trodden, M.S. Turner, Phys. Rev. D 70, 043528 (2004)

    Article  ADS  Google Scholar 

  4. M. Kowalski, D. Rubin, G. Aldering et al., ApJ 686, 749 (2008)

    Article  ADS  Google Scholar 

  5. S. Nojiri, S.D. Odintsov, Phys. Rep. arXiv:1011.0544 [gr-qc])

    Article  ADS  MathSciNet  Google Scholar 

  6. C. Bennett et al. (WMAP Collaboration) ApJS 148, 1 (2003)

  7. H.V. Peiris et al. (WMAP), Astrophys. J. Suppl. 148, 213 (2003), arXiv:astro-ph/0302225 [astro-ph]

  8. D. Spergel et al. (WMAP Collaboration), ApJS. 148, 175 (2003)

  9. S. Weinberg, Rev. Mod. Phys. 61, 1 (1989)

    Article  ADS  Google Scholar 

  10. T. Padmanabhan, Phys. Rep. 380, 235 (2003)

    Article  ADS  MathSciNet  Google Scholar 

  11. E.J. Copeland, M. Sami, S. Tsujikawa, Int. J. Mod. Phys. D 15, 1753 (2006)

    Article  ADS  Google Scholar 

  12. T. Delubac et al. (BOSS), Astron. Astrophys. arXiv:1404.1801 [astro-ph.CO])

  13. E. Macaulay, I.K. Wehus, H.K. Eriksen, Phys. Rev. Lett. arXiv:1303.6583 [astro-ph.CO])

    Article  ADS  Google Scholar 

  14. W.L. Freedman, Nat. Astron. arXiv:1706.02739 [astro-ph.CO])

    Article  ADS  Google Scholar 

  15. N. Aghanim et al. (Planck) (2018). arXiv:1807.06209 [astro-ph.CO]

  16. A.G. Riess, S. Casertano, W. Yuan, L.M. Macri, D. Scolnic, Astrophys. J. arXiv:1903.07603 [astro-ph.CO])

    Article  ADS  Google Scholar 

  17. M. Rezaei, M. Malekjani, Phys. Rev. D 96, 063519 (2017)

    Article  ADS  Google Scholar 

  18. T. Yang, A. Banerjee, E. Colgáin, (2019), arXiv:1911.01681 [astro-ph.CO]

  19. N. Khadka, B. Ratra (2019). arXiv:1909.01400 [astro-ph.CO]

  20. E. Lusso, E. Piedipalumbo, G. Risaliti, M. Paolillo, S. Bisogni, E. Nardini, L. Amati, Astron. Astrophys. arXiv:1907.07692 [astro-ph.CO])

    Article  ADS  Google Scholar 

  21. M. Benetti, S. Capozziello, JCAP arXiv:1910.09975 [astro-ph.CO])

    Article  ADS  Google Scholar 

  22. G. Veneziano, Nucl. Phys. B 159, 213 (1979)

    Article  ADS  Google Scholar 

  23. J.K. Erickson, R. Caldwell, P.J. Steinhardt, C. Armendariz-Picon, V.F. Mukhanov, Phys. Rev. Lett. 88, 121301 (2002)

    Article  ADS  Google Scholar 

  24. S. Thomas, Phys. Rev. Lett. 89, 081301 (2002)

    Article  ADS  MathSciNet  Google Scholar 

  25. C. Armendariz-Picon, V.F. Mukhanov, P.J. Steinhardt, Phys. Rev. D arXiv:0006373 [astroph])

    Article  ADS  Google Scholar 

  26. R.R. Caldwell, Phys. Lett. B 545, 23 (2002)

    Article  ADS  Google Scholar 

  27. T. Padmanabhan, Phys. Rev. D 66, 021301 (2002)

    Article  ADS  Google Scholar 

  28. M. Gasperini, F.P.G. Veneziano, Phys. Rev. D 65, 023508 (2002)

    Article  ADS  Google Scholar 

  29. E. Elizalde, S. Nojiri, S.D. Odintsov, Phys. Rev. D arXiv:0405034 [hep-th])

    Article  ADS  Google Scholar 

  30. A. Gomez-Valent, J. Sola, Mon. Not. R. Astron. Soc. arXiv:1412.3785 [astro-ph.CO])

    Article  ADS  Google Scholar 

  31. Lloyd Knox, Marius Millea, Phys. Rev. D 101(4), 043533 (2020)

    Article  ADS  Google Scholar 

  32. H. Akaike, IEEE Trans. Autom. Control 19, 716 (1974)

    Article  ADS  Google Scholar 

  33. G. Schwarz, Ann. Stat. 6, 461 (1978)

    Article  Google Scholar 

  34. D.J. Spiegelhalter, N.G. Best, B.P. Carlin, A. Van Der Linde, J. R. Stat. Soc. B64(4), 583 (2002)

    Article  Google Scholar 

  35. Andrew R. Liddle, Mon. Not. R. Astron. Soc. 377, L74–L78 (2007)

    Article  ADS  Google Scholar 

  36. M. Trashorras, S. Nesseris, J.G. Bellido, Phys. Rev. D 94, 063511 (2016)

    Article  ADS  Google Scholar 

  37. M. Malekjani, S. Basilakos, Z. Davari, A. Mehrabi, M. Rezaei, Mon. Not. R. Astron. Soc. arXiv:1609.01998 [astro-ph.CO])

    Article  ADS  Google Scholar 

  38. M. Rezaei, M. Malekjani, S. Basilakos, A. Mehrabi, D.F. Mota, Astrophys. J. arXiv:1706.02537 [astro-ph.CO])

    Article  ADS  Google Scholar 

  39. M. Malekjani, M. Rezaei, I.A. Akhlaghi, Phys. Rev. D arXiv:1809.08792 [gr-qc])

    Article  ADS  MathSciNet  Google Scholar 

  40. M. Rezaei, Mon. Not. R. Astron. Soc. arXiv:1902.04776 [gr-qc])

    Article  ADS  Google Scholar 

  41. W. Lin, K.J. Mack, L. Hou (2019). arXiv:1910.02978 [astro-ph.CO]

  42. M. Rezaei, M. Malekjani, J. Sola, Phys. Rev. D arXiv:1905.00100 [gr-qc])

    Article  ADS  MathSciNet  Google Scholar 

  43. M. Rezaei, T. Naderi, M. Malekjani, A. Mehrabi, Eur. Phys. J. C arXiv:2004.08168 [astro- ph.CO])

    Article  ADS  Google Scholar 

  44. M. Rezaei, S.P. Ojaghi, M. Malekjani, Astrophys. J. arXiv:2008.03092 [astro-ph.CO])

    Article  ADS  Google Scholar 

  45. R.D. Cousins, Phys. Rev. Lett. arXiv:0807.1330 [hep-ph])

    Article  ADS  Google Scholar 

  46. R.D. Cousins, Synthese arXiv:1310.3791 [physics.data-an])

    Article  MathSciNet  Google Scholar 

  47. G. Efstathiou, Mon. Not. R. Astron. Soc. arXiv:0802.3185 [astro-ph])

    ADS  Google Scholar 

  48. S. Gariazzo, EPJC 80, 552 (2020). arXiv:1910.06646

  49. B. Dabbs, B. Junker, arXiv preprint arXiv:1605.03000 (2016)

  50. J. Aldrich, Int. Stat. Rev. 66(1), 61 (2007)

    Article  Google Scholar 

  51. W. Ralph, The combinatorics of occam’s razor (2015). arXiv:1504.07441 [math.CO]

  52. R. Shibata, Biometrika 71, 43 (1984)

    Article  MathSciNet  Google Scholar 

  53. J. Shao, Stat. Sin. 7, 221 (1997)

    Google Scholar 

  54. R. Dutta, M. Bogdan, J.K. Ghosh, Model selection and multiple testing—a Bayesian and empirical Bayes overview and some new results (2015), arXiv:1510.00547 [math.ST]

  55. A. Kurek, M. Szydlowski, Astrophys. J. arXiv:astro-ph/0702484 [astro-ph])

    Article  ADS  Google Scholar 

  56. F. Arevalo, A. Cid, J. Moya, Eur. Phys. J. C arXiv:1610.09330 [astro-ph.CO])

    Article  ADS  Google Scholar 

  57. R. Trotta (2017) arXiv:1701.01467 [astro-ph.CO]

  58. M.C. March, G.D. Starkman, R. Trotta, P.M. Vaudrevange, Mon. Not. R. Astron. Soc. arXiv:1005.3655 [astro-ph.CO])

    Article  ADS  Google Scholar 

  59. S. Wager, Cross-validation, risk estimation, and model selection (2019). arXiv:1909.11696 [stat.ME]

  60. B. Ghojogh, M. Crowley, The theory behind overfitting, cross validation, regularization, bagging, and boosting: tutorial (2019). arXiv:1905.12787 [stat.ML]

  61. S. Arlot, A. Celisse, A survey of cross-validation procedures for model selection. Stat. Surv. 4, 40–79 (2010)

    Article  MathSciNet  MATH  Google Scholar 

  62. A. Vehtari, A. Gelman, J. Gabry, Practical Bayesian model evaluation using leave-one-out cross-validation and WAIC, arXiv:1507.04544

  63. R. Kohavi, A study of cross-validation and bootstrap for accuracy estimation and model selection, in Proceedings of the 14th International Joint Conference on Artificial Intelligence, Vol. 2, Canada (1995)

  64. Zhang et al., Cross-validation for selecting a model selection procedure. J. Econom. 187(1), 95 (2015)

    Article  MathSciNet  MATH  Google Scholar 

  65. A. Mehrabi, S. Basilakos, F. Pace, MNRAS arXiv:1504.01262 [astro-ph.CO])

    Article  ADS  Google Scholar 

  66. I. Maor, R. Brustein, P.J. Steinhardt, Phys. Rev. Lett. 86, 6 (2001). [Erratum: Phys. Rev. Lett.87,049901(2001)], arXiv:astro-ph/0007297 [astro-ph]

  67. A.G. Riess et al. (Supernova Search Team), ApJ 607, 665 (2004)

  68. B.A. Bassett, M. Brownstone, A. Cardoso, M. Cortes, Y. Fantaye, R. Hlozek, J. Kotze, P. Okouma, JCAP arXiv:0709.0526 [astro-ph])

    Article  ADS  Google Scholar 

  69. G. Efstathiou, Mon. Not. R. Astron. Soc. arXiv:astro-ph/9904356 [astro-ph])

    Article  ADS  Google Scholar 

  70. H.K. Jassal, J.S. Bagla, T. Padmanabhan, Mon. Not. R. Astron. Soc. arXiv:astroph/0404378 [astro-ph])

    Article  ADS  Google Scholar 

  71. E.M. Barboza, J.S. Alcaniz, Z.H. Zhu, R. Silva, Phys. Rev. D arXiv:0905.4052 [astro-ph.CO])

    Article  ADS  Google Scholar 

  72. H. Pade, Ann. Sci. Ecole Norm. Sup. 9(3), 1 (1892)

    Google Scholar 

  73. A. Baker, P. Graves-Morris, Pade Approximants (Cambridge University Press, Cambridge, 1996).

    Book  MATH  Google Scholar 

  74. M. Adachi, M. Kasai, Prog. Theor. Phys. arXiv:1111.6396 [astro-ph.CO])

    Article  ADS  Google Scholar 

  75. M. Rezaei, Mon. Not. R. Astron. Soc. arXiv:1904.02785 [gr-qc])

    Article  ADS  Google Scholar 

  76. D.M. Scolnic et al., Astrophys. J. arXiv:1710.00845 [astro-ph.CO])

    Article  ADS  Google Scholar 

  77. O. Farooq, F.R. Madiyar, S. Crandall, B. Ratra, Astrophys. J. arXiv:1607.03537 [astro-ph.CO])

    Article  ADS  Google Scholar 

  78. S. Alam et al., BOSS. Mon. Not. R. Astron. Soc. 470, 2617 (2017)

    Article  ADS  Google Scholar 

  79. J.E. Bautista et al., Astron. Astrophys. 603, A12 (2017)

    Article  Google Scholar 

  80. F. Beutler, C. Blake, M. Colless, D.H. Jones, L.S. Smith et al., MNRAS 416, 3017 (2011)

    Article  ADS  Google Scholar 

  81. A.J. Ross et al., Mon. Not. R. Astron. Soc. 449, 835 (2015)

    Article  ADS  Google Scholar 

  82. H. Gil-Marn et al., Mon. Not. R. Astron. Soc. 477(2), 1604 (2018)

    Article  ADS  Google Scholar 

  83. T.M.C. Abbott et al. (DES), Mon. Not. R. Astron. Soc. 483, 4866 (2019)

  84. A.R. Liddle, Information criteria for astrophysical model selection. Mon. Not. R. Astron. Soc. 377, L74 (2007)

    Article  ADS  Google Scholar 

  85. F.K. Anagnostopoulos, S. Basilakos, E.N. Saridakis, Observational constraints on Myrzakulov gravity, arXiv:2012.06524 [gr-qc]

  86. H. Jeffreys, Theory of Probability, ed. by H. Jeffreys (Oxford University Press, Cambridge, 1961)

  87. S. Nesseris, J. Garcia-Bellido, JCAP 1308, 036 (2013)

    Article  ADS  Google Scholar 

  88. R.E. Kass, A.E. Raftery, J. Am. Stat. Assoc. 90, 773 (1995)

    Article  Google Scholar 

  89. S. Pan, W. Yang, E. Di Valentino, A. Shafieloo, S. Chakraborty (2019). arXiv:1907.12551 [astro-ph.CO]

Download references

Acknowledgements

The authors gratefully thank the Referee for the constructive comments and recommendations which definitely help to improve the readability and quality of the paper.

Author information

Authors and Affiliations

  1. Department of Physics, Bu Ali Sina University, Hamedan, 65178, Iran

    Mehdi Rezaei & Mohammad Malekjani

  2. Iran Meteorological Organization, Hamedan Research Center for Applied Meteorology, Hamedan, Iran

    Mehdi Rezaei

Authors
  1. Mehdi Rezaei
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Mohammad Malekjani
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Mehdi Rezaei.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Rezaei, M., Malekjani, M. Comparison between different methods of model selection in cosmology. Eur. Phys. J. Plus 136, 219 (2021). https://doi.org/10.1140/epjp/s13360-021-01200-w

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1140/epjp/s13360-021-01200-w

Search

Navigation