Abstract
The results from the first foray into the research quality in Australia, Excellence in Research for Australia (ERA), were released in 2011. This article critically assesses the methodology of ERA and its application to the discipline of political science. It uses two other assessments of research quality in the discipline to confirm that the results for political science provide a credible result, as the rank order of departments is broadly compatible with these earlier exercises. The article then shows how the overall assessments for disciplines reflect the methodology employed, and that the average score for the discipline does not provide a credible benchmark against political science globally. While research funding decisions within disciplines are likely to be made according to within-discipline ranks, the use of descriptors relating the numerical score to ‘world class’ have undervalued the quality of the discipline relative to other disciplines and this risks affecting the perceptions of policymakers and impacting funding for the discipline as a whole.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
Notes
The merits of various kinds of bibliometrics and peer assessment in the political and social sciences have been widely discussed. See Donovan (2009); Giménez-Toledo and Román-Román (2009); Gläser (2004); Johnston (2009); McLean et al (2009); Russell (2009); Weale (2009).
References
Behn, R.D. (1981) ‘Policy analysis and policy politics’, Policy Analysis 7: 199–226.
Benn, C.R. and Sánchez, S.F. (2001) ‘Scientific productivity of large telescopes’, PASP (Publications of the Astronomical Society of the Pacific) 11: 358.
Capaldi, E.D. (2010) ‘International Research Collaborations’, in D. Bruce Johnston (ed.) Higher Education in a Global Society, Cheltenham: Edward Elgar, pp. 70–82.
Crabtree, D. (2009) ‘LRP2010 white paper on the scientific impact of Canadian astronomy’, 11 October 2009. Preprint at, http://www.casca.ca/lrp2010/Docs/LRPReports/Pubs.pdf, accessed 2 February 2011.
Donovan, C. (2009) ‘Gradgrinding the social sciences: The politics of metrics of political science’, Political Studies Review 7: 73–83.
EASE (European Association of Science Editors). (2008) ‘EASE statement on inappropriate use of impact factors’, http://www.ease.org.uk/artman2/uploads/1/EASE_statement_IFs_final.pdf, accessed 26 October 2011.
Giménez-Toledo, E. and Román-Román, A. (2009) ‘Assessment of humanities and social sciences monographs through their publishers: A review and a study towards a model of evaluation’, Research Evaluation 18 (3): 201–213.
Gläser, J. (2004) ‘Why are the most influential books in Australian sociology not necessarily the most highly cited ones?’ Journal of Sociology 40 (3): 261–282.
Grant, J. (2011) ‘Forget the APSR: The politics of political science’, New Political Science 33 (1): 87–91.
Hix, S. (2004) ‘A global ranking of political science departments”, Political Studies Review 2: 293–313.
Johnston, R. (2009) ‘Where there are data … quantifying the unquantifiable’, Political Studies Review 7: 50–62.
Martin, B.R. (1996) ‘The use of multiple indicators in the assessment of basic research’, Scientometrics 36 (3): 343–362.
Martin, B.R. and Irvine, J. (1983) ‘Assessing basic research: Some partial indicators of scientific progress in radio astronomy’, Research Policy 12: 61–90.
McLean, I., Blais, A., Garand, J.C. and Giles, M. (2009) ‘Comparative journal ratings: A survey report’, Political Studies Review 7: 18–38.
Pontille, D. and Torny, D. (2010) ‘The controversial policies of journal ratings: Evaluating social sciences and humanities’, Research Evaluation 19 (5): 347–360.
Russell, A. (2009) ‘Retaining the peers: How peer review triumphs over league tables and faulty accounting in the assessment of political science research’, Political Studies Review 7: 63–72.
Sánchez, S.F. and Benn, C.R. (2004) ‘Impact of astronomical research from different countries’, Astronomische Nachrichten 325 (5): 445–450.
Sharman, J.C. and True, J. (2011) ‘Anglo-American followers or Antipodean iconoclasts? The 2008 TRIP survey of international relations in Australia and New Zealand’, Australian Journal of International Affairs 65 (2): 148–166.
Sharman, J.C. and Weller, P. (2009) ‘Where is the quality? Political science scholarship in Australia’, Australian Journal of Political Science 44 (4): 597–612.
Weale, A. (2009) ‘Metrics versus peer review?’ Political Studies Review 7: 39–49.
Wells, A. (2011) ‘Measuring excellence in Australian research’, Dialogue 30 (1): 4–10.
Author information
Authors and Affiliations
Rights and permissions
About this article
Cite this article
Kellow, A. Assessing Political Science Quality: ‘Excellence in Research for Australia’. Eur Polit Sci 11, 567–580 (2012). https://doi.org/10.1057/eps.2011.70
Published:
Issue Date:
DOI: https://doi.org/10.1057/eps.2011.70