Scientometrics

, Volume 47, Issue 1, pp 11–23 | Cite as

Does Peer Review Predict the Performance of Research Projects in Health Sciences?

  • L. Erik Clavería
  • Eliseo Guallar
  • Jordi Camí
  • José Conde
  • Roberto Pastor
  • José R. Ricoy
  • Eduardo Rodríguez-Farré
  • Fernando Ruiz-Palomo
  • Emilio Muñoz
Article

Abstract

Peer review is a basic component of the scientific process, but its performance has seldom been evaluated systematically. To determine whether pre-approval characteristics of research projects predicted the performance of projects, we conducted a retrospective cohort study of all 2744 single-centre research projects financed by the Spanish Health Research Fund since 1988 and completed before 1996. Peer review scores of grant applications were significant predictors of performance of funded projects, and the likelihood of production was also higher for projects with a basic research component, longer duration, higher budget or a financed research fellow. Funding agencies should monitor their selection process and assess the performance of funded projects to design future strategies in supporting health sciences research.

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. 1.
    N. J. Birkett, The review process for applied-research grant proposals: suggestions for revision, Canadian Medical Association Journal, 150 (1994) 1227–1229.Google Scholar
  2. 2.
    R. Smith, Problems with peer review and alternatives, British Medical Journal, 296 (1988) 774–777.Google Scholar
  3. 3.
    S. A. Glantz, L. A. Bero, Inappropriate and appropriate selection of ‘peers’ in grant review, The Journal of the American Medical Association, 272 (1994) 114–116.Google Scholar
  4. 4.
    A. T. Evans, R. A. McNutt, S. W. Fletcher, R. H. Fletcher, The characteristics of peer reviewers who produce good-quality reviews, Journal of General Internal Medicine, 8 (1993) 422–428.Google Scholar
  5. 5.
    S. Wessely, Peer review of grant applications: what do we know?, Lancet, 352 (1998) 301–305.Google Scholar
  6. 6.
    National Institutes of Health, NIH peer review of research grant applications, US Department of Health and Human Services, Public Health Service, National Institutes of Health, Bethesda, 1996.Google Scholar
  7. 7.
    Biomedical AND Health Research Programme, Biomed 2 (1994–1998), Guide to the evaluation and the selection of proposals, European Comission, Directorate General XII for Science, Research and Development, Brussels, 1996, 1–22.Google Scholar
  8. 8.
    E. Guallar, J. Conde, M. A. De La Cal, J. M. MartÍn-Moreno, en nombre del Grupo de Evaluación de la Actividad del Fondo de Investigaciones Sanitarias entre 1988 y 1995, Guía para la evaluacion de proyectos de investigación en ciencias de la salud, Medicina Clinica, 108 (1997) 460–471.Google Scholar
  9. 9.
    Committee on rating grant applications, National Institutes of Health, Report of the committee on rating grant applications, World Wide Web http://www.nih.gov/grants/peer/rga.htm, 1996, 1–32.Google Scholar
  10. 10.
    J. R. Ricoy, M. F. Guasch, L. E. ClaverÍa, Fondo de Investigación Sanitaria (1988–1995). Una aproximación al análisis de un Programa I+D, Instituto Nacional de la Salud, Madrid, 1996.Google Scholar
  11. 11.
    J. R. Ricoy, La financiación de la investigación médica, Medicina Clinica, 100 (1993) Suppl. 1, 6–8.Google Scholar
  12. 12.
    P. McCullagh, J. A. Nelder, Generalized linear models, Chapman and Hall, London, 1989.Google Scholar
  13. 13.
    T. J. Hastie, R. J. Tibshirani, Generalized additive models, Chapman and Hall, London, 1990.Google Scholar
  14. 14.
    SAS Institute Inc, SAS language: reference, version 6, SAS Institute Inc, Cary, NC, 1990.Google Scholar
  15. 15.
    Statistical Science Inc, S-PLUS reference manual, version 3.2, MathSoft Inc, Seattle, 1993.Google Scholar
  16. 16.
    R. Smith, Promoting research into peer review [editorial] (see comments), British Medical Journal, 309 (1994) 143–144.Google Scholar
  17. 17.
    C. A. Olsson, W. A. Kennedy, Urology peer review at the National Institutes of Health, Journal of Urology, 154 (1995) 1866–1869.Google Scholar
  18. 18.
    H. J. Sutherland, E. M. Meslin, R. Da Cunha, J. E. Till, Judging clinical research questions: what criteria are used?, Social Science and Medicine, 37 (1993) 1427–1430.Google Scholar
  19. 19.
    K. J. Vener, E. J. Feuer, L. Gorelic, A statistical model validating triage for the peer review process: keeping the competitive applications in the review pipeline, FASEB Journal, 7 (1993) 1312–1319.Google Scholar
  20. 20.
    F. J. Medman, R. A. De Melker, The extent of inter-and intrareviewer agreement on the classification and assessment of designs of single-practice research, Family Practice, 12 (1995) 93–97.Google Scholar
  21. 21.
    R. K. Merton, The Matthew effect in science, Science, 159 (1968) 36–63.Google Scholar
  22. 22.
    R. K. Merton, The Matthew effect in science. II. Cumulative advantages and the symbolism of intellectual property, Isis, 79 (1988) 606–623.Google Scholar
  23. 23.
    National Institutes of Health, PHS grants policy statement, US Department of Health and Human Services, Public Health Service, Washington, 1987.Google Scholar
  24. 24.
    D. F. Horrobin, Referees and research administrators: barriers to scientific research?, British Medical Journal, 2 (1974) 216–218.Google Scholar
  25. 25.
    D. F. Horrobin, Peer review of grant applications: a harbinger of mediocrity in clinical research, Lancet, 348 (1996) 1293–1295.Google Scholar
  26. 26.
    J. P. Kassirer, E. W. Campion, Peer review: crude and understudied, but indispensable, The Journal of the American Medical Association, 272 (1994) 96–97.Google Scholar
  27. 27.
    A. Abbott, Funding cuts put pressure on peer review, Nature, 383 (1996) 567.Google Scholar

Copyright information

© Akadémiai Kiadó, Budapest 2000

Authors and Affiliations

  • L. Erik Clavería
    • 1
  • Eliseo Guallar
    • 2
  • Jordi Camí
    • 3
  • José Conde
    • 4
  • Roberto Pastor
    • 2
  • José R. Ricoy
    • 5
  • Eduardo Rodríguez-Farré
    • 6
  • Fernando Ruiz-Palomo
    • 7
  • Emilio Muñoz
    • 8
  1. 1.Neurology UnitGeneral HospitalSegoviaSpain
  2. 2.Department of Epidemiology and BiostatisticsNational School of Public Health Instituto de Salud Carlos IIIMadridSpain
  3. 3.Institut Municipal d'Investigació Mèdica (IMIM)Universitat Pompeu FabraBarcelonaSpain
  4. 4.Agency for Health Technology AssessmentInstituto de Salud Carlos IIIMadridSpain
  5. 5.Neuropathology UnitMadridSpain
  6. 6.Department of Pharmacology and ToxicologyCSICBarcelonaSpain
  7. 7.Division of Internal MedicineHospital Ramón y CajalMadridSpain
  8. 8.Institute for Advanced Sociologic ResearchMadridSpain

Personalised recommendations