Skip to main content

Blind review of research proposals in Korea: Its effectiveness and factors affecting applicant detection

Abstract

This article addresses the potential effectiveness of blind review in selecting and funding research proposals in a “scientifically small” country. By analyzing 474 responses of the blinded reviewers ever worked for Korea Science and Engineering Fund, it was found that blind review is fairly effective. About two thirds of the blinded reviewers were unable to recognize the applicants accurately. The applicant detection was affected by (1) physical age, (2) professional experience, and (3) geographical location of doctoral education of the applicant, (4) review experience, (5) rank of employing universities of the reviewers, and (6) similirity of research interest between an applicant and a reviewer. It was also found that blind review was more strongly advocated by those who had made a wrong guess or who had given up guessing. Implications of the findings and future research directions were discussed.

This is a preview of subscription content, access via your institution.

References

  1. Bakanic, V., C. McPhail, R. J. Simon, (1987) The manuscript review and decision-making process,American Sociological Review, 52:631.

    Article  Google Scholar 

  2. Bozeman, B. (1993) Peer review and evaluations of R&D impacts, inEvaluating R&D Impacts: Methods and Practice (Eds.) byB. Bozeman, J. Melker, Kluwer Academic Publisher, 79–98, Norwell, Massachusetts, USA.

    Google Scholar 

  3. Ceci, S. J., D. Peters, (1984) How blind is blind review,American Psychologist, 39:1491.

    Article  Google Scholar 

  4. Chubin, D. E., E. J. Hackett, (1990)Peerless Science: Peer Review and U.S. Science Policy, Albany, State University of New York Press.

    Google Scholar 

  5. Chubin, D. E., S. Jasanoff, (1985) Peer review and public policy,Science, Technology and Human Values, 10:3.

    Google Scholar 

  6. Chungangilbosa, (1996),97 ranking of the universities in Korea, Chungagilbosa.

  7. Cicchetti, D. V. (1980) Reliability of reviewers for the American psychologist: A biostatistical assessment of the data.American Psychologist, 35:300.

    Article  Google Scholar 

  8. Cole, S., J. R. Cole, G. A. Simon, (1981) Chance and consensus in peer review,Science, 214:881.

    Google Scholar 

  9. Gillespie, G. W., D. E. Chubin, G. M. Kurzon, (1985) Experience with NIH peer review: Researchers' cynicism and desire for change,Science, Technology and Human Values, 10:44.

    Google Scholar 

  10. Gustafson, T. (1975) The controversy over peer review: Recent studies of the peer review system show that its critics have yet to make their case.Science, 190:1060.

    Google Scholar 

  11. Hargens, L. L. (1988) Scholarly consensus and journal rejection rates,American Sociological Review, 53:139.

    Article  Google Scholar 

  12. Hartmann, I., F. Neidhard, (1990) Peer review at the Deutsche Forschungsgemeinschaft,Scientometrics, 19:419.

    Article  Google Scholar 

  13. Jasanoff, S. (1985) Peer review of the regulatory process,Science, Techology and Human Values, 10:20.

    Google Scholar 

  14. Kalberer, J. T. (1985) Peer review and the consensus development process.Science, Technology and Human Values, 10:63.

    Google Scholar 

  15. Koshland, D. E. (1985) Peer review of peer review.Science, 228:1387.

    Google Scholar 

  16. Lee, M., B. Son, K. Om, (1996) Evaluation of national R&D projects in Korea,Research Policy, 25:805.

    Article  Google Scholar 

  17. Lindsey, D. (1988) Assessing precision in the manuscript review process: A little better than a dice roll,Scientometrics, 14:75.

    Article  Google Scholar 

  18. Maddox, J. (1984) Privacy and the peer-review system,Nature, 312:497.

    Article  Google Scholar 

  19. Merton, R. K., H. Zuckerman, (1973) Institutionalized patterns of evaluation in science, inThe Sociology of Science, byR. K. Merton, Chicago, IL: University of Chicago Press, 460–496.

    Google Scholar 

  20. Peters, D. P., S. J. Ceci, (1982) Peer review practices of psychological journal: The fate of published articles, submitted again.Behavioral and Brain Sciences, 5:187.

    Article  Google Scholar 

  21. Porter, A., F. Rossini, (1985) Peer review of interdisciplinary proposals,Science, Technology and Human Values, 10:34.

    Google Scholar 

  22. Rosenblatt, A., S. A. Kirk, (1980) Recognitions of authors in blind review of manuscripts.Journal of Social Service Research, 3:383.

    Article  Google Scholar 

  23. Roy, R. (1984) Alternative to review by peers: A contribution to the theory of scientific choice.Minerva, 22:316.

    Article  Google Scholar 

  24. Roy, R. (1985) Funding science: The real defects of peer review and an alternative to it,Science, Technology and Human Values, 10:47.

    Google Scholar 

Download references

Author information

Affiliations

Authors

Rights and permissions

Reprints and Permissions

About this article

Cite this article

Lee, M., Om, K. & Koh, J. Blind review of research proposals in Korea: Its effectiveness and factors affecting applicant detection. Scientometrics 45, 17–31 (1999). https://doi.org/10.1007/BF02458466

Download citation

Keywords

  • Detection Accuracy
  • American Sociological Review
  • Research Proposal
  • Scientific Performance
  • Funding Decision