Peer review and panel decisions in the assessment of Australian Research Council project grant applicants: what counts in a highly competitive context?

Abstract

Applications submitted to the Australian Research Council Large Grants Scheme in five discipline groups were examined with a view to determining the extent of influence of biographical and academic “track record” variables on ratings by independent assessors (peers) and on final outcomes as determined by the Council's discipline review panels. Variables considered included age, gender, type and status of position, institutional base, previous grants history and publication records of the applicants. Age and gender of the applicants did not have a significant impact on outcomes. Those in research only positions were rated more highly, and those in more prestigious institutions were more likely to win support, but these factors were partly accounted for by the more impressive publication records of these groups. The academic status of the applicant was, however, found to make a significant contribution to an explanation of the variance in independent assessor's ratings of the strength of the researcher, along with but additionally to that contributed by publication components of their track record. Assessor's ratings of the strength of the proposal were, as would be expected, the major influence on the final decisions made by review panels, although academic status of the applicant did, again, make a small but significant additional contribution to an explanation of variance in outcome. These results lend some support to the idea that the “Matthew effect”, or theory of accumulative advantage, has some impact on peer and panel review of applications within the highly competitive arena of Australian Research Council large grants.