Higher Education

, Volume 35, Issue 4, pp 435–452

Peer review and panel decisions in the assessment of Australian Research Council project grant applicants: what counts in a highly competitive context?

  • Pat Bazeley
Article

Abstract

Applications submitted to the Australian Research Council Large Grants Scheme in five discipline groups were examined with a view to determining the extent of influence of biographical and academic “track record” variables on ratings by independent assessors (peers) and on final outcomes as determined by the Council's discipline review panels. Variables considered included age, gender, type and status of position, institutional base, previous grants history and publication records of the applicants. Age and gender of the applicants did not have a significant impact on outcomes. Those in research only positions were rated more highly, and those in more prestigious institutions were more likely to win support, but these factors were partly accounted for by the more impressive publication records of these groups. The academic status of the applicant was, however, found to make a significant contribution to an explanation of the variance in independent assessor's ratings of the strength of the researcher, along with but additionally to that contributed by publication components of their track record. Assessor's ratings of the strength of the proposal were, as would be expected, the major influence on the final decisions made by review panels, although academic status of the applicant did, again, make a small but significant additional contribution to an explanation of variance in outcome. These results lend some support to the idea that the “Matthew effect”, or theory of accumulative advantage, has some impact on peer and panel review of applications within the highly competitive arena of Australian Research Council large grants.

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. Bazeley, P., Kemp, L., Stevens, K., Asmar, C., Grbich, C., Marsh, H. and Bhathal, R. (1996). Waiting in theWings: A Study of Early Career Academic Researchers in Australia. National Board of Employment Education and Training Commissioned Report No. 50, Canberra: Australian Government Publishing Service.Google Scholar
  2. Brennan, M. H. (1993). ‘Excellence and relevance - two sides of the same coin’, in Wood F.Q and Meek V.L. (eds.), Research Grants Management and Funding. Canberra: Bibliotech.Google Scholar
  3. Biglan, A. (1973). ‘The characteristics of subject matter in different academic areas’, Journal of Applied Psychology 57, 195–203.Google Scholar
  4. Castleman, T., Allen, M., Bastalich, W. and Wright, P. (1995). Limited Access: Womens Disadvantage in Higher Education Employment. Melbourne: National Tertiary Education Union.Google Scholar
  5. Cicchetti, D. V. (1991). ‘The reliability of peer review for manuscript and grant submissions: A cross-disciplinary investigation’, Behavioral and Brain Sciences 14, 119–135.Google Scholar
  6. Clark, S. M. and Corcoran, M. (1986). ‘Perspectives in the professional socialization of women faculty: A case of accumulative disadvantage?’ Journal of Higher Education 57, 20–43.Google Scholar
  7. Cole, J. R. and Cole, S. (1979). ‘Which researcher will get the grant’. Nature 279, 575–576.Google Scholar
  8. Cole, S., Rubin, L. and Cole, J. R. (1977). ‘Peer review and the support of science’, Scientific American 237, 34–41.Google Scholar
  9. Emery, J. A., Meyers, H.W. and Hunter, D. E. (1992). ‘NIHFIRST awards: Testing background factors for funding against peer review’, Journal of the Society of Research Administrators 24(2), 7–15.Google Scholar
  10. Marsh, H. W. and Ball, S. (1981). ‘Interjudgmental reliability of reviews for the Journal of Educational Psychology’, Journal of Educational Psychology 73, 872–880.Google Scholar
  11. Marsh, H. W. and Ball, S. (1989). ‘The peer review process used to evaluate manuscripts submitted to academic journals’, Journal of Experimental Education 57, 151–169.Google Scholar
  12. Marsh, H.W. and Bazeley, P. (1996). Multiple Evaluations of Grant Proposals by Independent Assessors: Confirmatory Factor Analysis Evaluations of Reliability, Validity and Structure. Campbelltown: University of Western Sydney Macarthur.Google Scholar
  13. Merton, R.K. (1968). ‘The Matthew effect in science’, Science 199, 55–63.Google Scholar
  14. NBEET (1993a). ‘Review of Grants Outcomes 7’, Condensed Matter Physics. Canberra: Australian Government Publishing Service.Google Scholar
  15. NBEET (1993b). ‘Review of Grants Outcomes 9’, Mathematical Sciences. Canberra: Australian Government Publishing Service.Google Scholar
  16. NBEET (1993c). ‘Review of Grants Outcomes 8’, Materials and Chemical Engineering. Canberra: Australian Government Publishing Service.Google Scholar
  17. NBEET (1993d). ‘Review of Grants Outcomes 10’, Organic Chemistry. Canberra: Australian Government Publishing Service.Google Scholar
  18. NBEET (1993e). ‘Review of Grants Outcomes 11’, Fluid Mechanics. Canberra: Australian Government Publishing Service.Google Scholar
  19. NBEET (1993f ). ‘Review of Grants Outcomes 12’, Molecular Biology. Canberra: Australian Government Publishing Service.Google Scholar
  20. NBEET (1993g). Higher Education Resesarch Infrastructure. Canberra: Australian Government Publishing Service.Google Scholar
  21. NBEET (1994a). The Strategic Role of Academic Research. Canberra: Australian Government Publishing Service.Google Scholar
  22. NBEET (1994b). ‘Review of Grants Outcomes 14’, Psychology. Canberra: Australian Government Publishing Service.Google Scholar
  23. NBEET (1995a). ‘Review of Grants Outcomes 16’, Inorganic Chemistry. Canberra: Australian Government Publishing Service.Google Scholar
  24. NBEET (1995b). ‘Review of Grants Outcomes 18’, Computer Science. Canberra: Australian Government Publishing Service.Google Scholar
  25. Over, R. (1993). ‘Correlates of career advancement in Australian universities’, Higher Education 26, 313–329.Google Scholar
  26. Over, R. (1995). ‘The Australian Research Council Large Grants Scheme: Problems, concerns and recommendations for change’, Australian Universities Review 38, 32–36.Google Scholar
  27. Over, R. (1996). ‘Perceptions of the Australian Research Council Large Grants Scheme: differences between successful and unsuccessful applicants’, Australian Educational Researcher 23, 17–36.Google Scholar
  28. Ramsden, P. (1992). Research performance, DEET Higher Education Division Higher Education Series, No. 2, December.Google Scholar
  29. Rip, A. (1993). ‘The republic of science in the 1990s’, in Wood F.Q. and Meek V.L. (eds.), Research Grants Management and Funding. Canberra: Bibliotech.Google Scholar
  30. Rosenthal, R. (1991). ‘Some indices of the reliability of peer review’, Behavioral and Brain Sciences 14, 160–161.Google Scholar
  31. Tabachnick, B.G. and Fidell, L.S. (1989). Using Multivariate Statistics. New York: Harper Collins.Google Scholar
  32. Van de Kaa, Dirk J. (1993). ‘Picking the winners by consensus: grant giving practice in the Netherlands’, in Wood F.Q. and Meek V.L. Research Grants Management and Funding. Canberra: Bibliotech.Google Scholar
  33. Wood, F.Q. (1997). The Peer Review Process. National Board of Employment Education and Training Commissioned Report No. 54, Canberra: Australian Government Publishing Service.Google Scholar
  34. Wood, F.Q., Meek, V.L. and Harman, G. (1992). ‘The research grant application process. Learning from failure?’, Higher Education 24, 1–23.Google Scholar

Copyright information

© Kluwer Academic Publishers 1998

Authors and Affiliations

  • Pat Bazeley
    • 1
  1. 1.University of Western Sydney MacarthurCampbelltownAustralia

Personalised recommendations