Is climate change controversy good for science? IPCC and contrarian reports in the light of bibliometrics


Debate and controversy concerning the issue of climate change generally results in the hindering and obstruction of social and governmental action on this issue. This paper analyses the scientific background, i.e. the reference list of the IPCC Fifth Assessment Report “The Physical Science Basis” and an alternative climate change report of a US think tank institute “Climate Change Reconsidered II. Physical Science”. We compared these two reports to the antecedent reports from 2007 (IPCC AR4 WGI) and 2009 (Climate Change Reconsidered). For the purposes of the study, we developed a database containing all the references collected from the four reports. The bibliometric analysis focused on the distribution of references among peer reviewed scientific journals and the most frequently cited lead authors that created the basis for the evaluation of their different scientific emphasis. Our findings underline that there is still no convergence between the scientific literature of the IPCC and the contrarian reports; however, the remarkable quantitative development on both sides and the qualitative progress of the IPCC report allows us to draw somewhat surprising conclusions in the context of climate change science. Contrary to expectations, controversy is beneficial to the science of climate change as it fosters the review process on both sides of the debate.

This is a preview of subscription content, access via your institution.


  1. Abraham, J. P., Cook, J., Fasullo, J. T., Jacobs, P. H., Mandia, S. A., & Nuccitelli, D. A. (2014). Review of the consensus and asymmetric quality of research on human-induced climate change. Cosmopolis, 5(1), 3–18.

    Google Scholar 

  2. Benestad, R. E., Nuccitelli, D., Lewandowsky, S., et al. (2016). Learning from mistakes in climate research. Theoretical an Applied Climatology, 126(3), 699–703. doi:10.1007/s00704-015-1597-5.

    Article  Google Scholar 

  3. Bjurström, A., & Polk, M. (2011a). Physical and economic bias in climate change research: A scientometric study of IPCC Third Assessment Report. Climatic Change, 108, 1–22.

    Article  Google Scholar 

  4. Bjurström, A., & Polk, M. (2011b). Climate change and interdisciplinarity: A co-citation analysis of IPCC Third Assessment Report. Scientometrics, 87, 525–550.

    Article  Google Scholar 

  5. Boussalis, C., & Coan, T. G. (2016). Text-mining the signals of climate change doubt. Global Environmental Change, 36, 89–100.

    Article  Google Scholar 

  6. Cook, J., Nuccitelli, D., Green, S. A., Richardson, M., Winkler, B., Painting, R., et al. (2013). Quantifying the consensus on anthropogenic global warming in the scientific literature. Environmental Research Letters, 8(2), 024024. doi:10.1088/1748-9326/8/2/024024.

    Article  Google Scholar 

  7. Dunlap, R. E., & McCright, A. M. (2015). Challenging climate change: The denial countermovement. In R. E. Dunlap & R. J. Brulle (Eds.), Climate change and society: Sociological perspectives (pp. 300–332). Oxford: Oxford University Press.

    Google Scholar 

  8. Grundmann, R. (2015). Climate skepticism. In K. Bäckstrand & E. Lövbrand (Eds.), Research handbook on climate governance (pp. 175–187). Northampton: Edward Elgar Publishing.

    Google Scholar 

  9. Hampel, M. (2016). Have climate sceptics taken the bait? What the deconstruction of instrumental climate records can tell us about the politics of climate change. Area, 48(2), 244–248.

    MathSciNet  Article  Google Scholar 

  10. Idso, C., & Singer, S. F. (2009). Climate change reconsidered. Report of the nongovernmental international panel on climate change (NIPCC). Chicago, IL: The Heartland Institute.

  11. Idso, C. D., Carter, R. M., & Singer, S. F. (Eds.). (2013). Climate change reconsidered II: Physical science (p. 993). Chicago, IL: The Heartland Institute.

    Google Scholar 

  12. IPCC. (2007). Climate change 2007: The physical science basis. Contribution of working group I to the fourth assessment report of the intergovernmental panel on climate change. In S. Solomon, D. Qin, M. Manning, Z. Chen, M. Marquis, K. B. Averyt, M. Tignor, & H. L. Miller (Eds.). Cambridge and New York, NY: Cambridge University Press.

  13. IPCC. (2013). Climate change 2013: The physical science basis. Contribution of working group I to the fifth assessment report of the intergovernmental panel on climate change. In T. F. Stocker, D. Qin, G.-K. Plattner, M. Tignor, S. K. Allen, J. Boschung, A. Nauels, Y. Xia, V. Bex, & P. M. Midgley (Eds.). Cambridge and New York, NY: Cambridge University Press.

  14. Jankó, F., Móricz, N., & Papp-Vancsó, J. (2014). Reviewing the climate change reviewers: Exploring controversy through report references and citations. Geoforum, 56, 17–34. doi:10.1016/j.geoforum.2014.06.004.

    Article  Google Scholar 

  15. Marx, W., Haunschild, R., Thor, A., et al. (2017). Which early works are cited most frequently in climate change research literature? A bibliometric approach based on reference publication year spectroscopy. Scientometrics, 110, 335. doi:10.1007/s11192-016-2177-x.

    Article  Google Scholar 

  16. Medimorec, S., & Pennycook, G. (2015). The language of denial: Text analysis reveals differences in language use between climate change proponents and skeptics. Climatic Change, 133(4), 1–9.

    Article  Google Scholar 

  17. Nordlund, G. (2008). Futures research and the IPCC assessment study on the effects of climate change. Futures, 40, 873–876.

    Article  Google Scholar 

  18. Pearce, W., Brown, B., Nerlich, B., & Koteyko, N. (2015). Communicating climate change: Conduits, content, and consensus. Wiley Interdisciplinary Reviews-Climate Change, 6(6), 613–626.

    Article  Google Scholar 

  19. Ryghaug, M., & Skjølsvold, T. M. (2010). The global warming of climate science: Climategate and the construction of scientific facts. International Studies in the Philosophy of Science, 24(3), 287–307. doi:10.1080/02698595.2010.522411.

    Article  Google Scholar 

  20. Tol, R. S. J. (2016). Comment on ‘quantifying the consensus on anthropogenic global warming in the scientific literature’. Environmental Research Letters, 11, 048001. doi:10.1088/1748-9326/11/4/048001.

    Article  Google Scholar 

  21. Vasileiadou, E., Heimeriks, G., & Petersen, A. C. (2011). Exploring the impact of the IPCC Assessment Reports on science. Environmental Science and Policy, 14, 1052–1061.

    Article  Google Scholar 

Download references


We are thankful for the reviewer’s help and suggestions, which improved our study. Our paper was supported by the János Bolyai Research Scholarship of the Hungarian Academy of Sciences (Jankó). Funding for this study was also provided by a project called Agroclimate. 2 VKSZ_12-1-2013-0034 (Papp Vancsó and Móricz).

Author information



Corresponding author

Correspondence to Ferenc Jankó.

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Jankó, F., Papp Vancsó, J. & Móricz, N. Is climate change controversy good for science? IPCC and contrarian reports in the light of bibliometrics. Scientometrics 112, 1745–1759 (2017).

Download citation


  • Climate change controversy
  • IPCC
  • Climate scepticism
  • Peer reviewed journals
  • Lead authors