Bias in Peer Review of Organic Farming Grant Applications

Abstract

Peer reviews of 84 organic farming grant applications from Sweden were analyzed to determine whether the reviewers’ affiliation to one of two types of agriculture (i.e., organic and conventional) influenced their reviews. Fifteen reviewers were divided into three groups: (1) scientists with experience in organic farming research; (2) scientists with no experience in organic farming research; and (3) users of organic farming research. The two groups of scientists assessed the societal relevance and scientific quality of the grant applications based on three criteria (i.e., presentation, methodology, qualifications), whereas the user group only assessed societal relevance. The analysis showed that the two groups of scientists provided very different reviews. Scientist reviewers with experience in organic farming research agreed more with the user group on research relevance than did scientist reviewers without such experience, and the assessment of relevance was closely correlated to the assessment of scientific quality within both scientific groups. As both scientific groups did not clearly distinguish between societal relevance and scientific quality, the idea of an objective science is challenged. The contextual values associated with the norms of good agriculture were not clearly distinguished from the constitutive values of science associated with the traditional norms of good science. This raises the question of whether organic and conventional grant applications should be mixed for review regardless of the reviewers.

This is a preview of subscription content, access via your institution.

References

  1. H. F. Alrøe (2000) ArticleTitle“Science as systems learning. Some reflections on the cognitive and communicational aspects of science” Cybernetics and Human Knowing 7 IssueID4 57–78

    Google Scholar 

  2. H. F. Alrøe E. S. Kristensen (2002) ArticleTitle“Towards a systemic research methodology in agriculture: Rethinking the role of values in science” Agriculture and Human Values 19 3–23 Occurrence Handle10.1023/A:1015040009300

    Article  Google Scholar 

  3. R. U. Ayres J. C. J. M. Bergh Particlevan den J. M. Gowdy (2001) ArticleTitle“Strong versus weak sustainability: Economics, natural sciences, and ‘consilience’” Environmental Ethics 23 155–168

    Google Scholar 

  4. P. Bazeley (1998) ArticleTitle“Peer review and panel decisions in the assessment of Australian Research Council project grant applicants: What counts in a highly competitive context?” Higher Education 35 435–452 Occurrence Handle10.1023/A:1003118502318

    Article  Google Scholar 

  5. C. E. Beus R. E. Dunlap (1990) ArticleTitle“Conventional versus alternative agriculture: The paradigmatic roots of the debate” Rural Sociology 55 590–616 Occurrence Handle10.1111/j.1549-0831.1990.tb00699.x

    Article  Google Scholar 

  6. C. E. Beus R. E. Dunlap (1991) ArticleTitle“Measuring adherence to alternative vs. conventional agricultural paradigms: A proposed scale” Rural Sociology 56 432–460 Occurrence Handle10.1111/j.1549-0831.1991.tb00442.x

    Article  Google Scholar 

  7. J. Christensen (1998) Alternativer – Natur – Landbrug (Alternatives – Nature – Agriculture) Akademisk Forlag Copenhagen, Denmark

    Google Scholar 

  8. D. V. Cicchetti (1991a) ArticleTitle“The reliability of peer-review for manuscript and grant submissions – A cross-disciplinary investigation” Behavioral and Brain Sciences 14 119–134

    Google Scholar 

  9. D. V. Cicchetti (1991b) ArticleTitle“Reflections from the peer-review mirror” Behavioral and Brain Sciences 14 167–186

    Google Scholar 

  10. S. Cole J. R. Cole G. A. Simon (1981) ArticleTitle“Change and consensus in peer review” Science 214 881–886

    Google Scholar 

  11. C. P. Egri (1999) ArticleTitle“Attitudes,, backgrounds and information preferences of Canadian organic and conventional farmers: Implications for organic farming advocacy and extension” Journal of Sustainable Agriculture 13 IssueID3 45–72

    Google Scholar 

  12. C. Francis J. King (1997) ArticleTitle“Impact of personal values on agricultural research” Society and Natural Resources 10 273–282 Occurrence Handle10.1080/08941929709381026

    Article  Google Scholar 

  13. C. A. Francis G. Lieblein J. Helenius L. Salomonsson H. Olsen J. Porter (2001) ArticleTitle“Challenges in designing ecological agriculture education: A Nordic perspective on change” American Journal of Alternative Agriculture 16 IssueID2 89–95 Occurrence Handle10.1017/S0889189300008985

    Article  Google Scholar 

  14. R. Harding (Eds) (1998) Environmental Decision-making: The Roles of Scientists, Engineers and the Public Sydney, New South Wales The Federation Press

    Google Scholar 

  15. J. Huckle S. Sterling (1996) Education for Sustainability Earthscan Publications Ltd London, UK

    Google Scholar 

  16. G. Lieblein C. Francis J. King (2000) ArticleTitle“Conceptual framework for structuring future agricultural colleges and universities in industrial countries” The Journal of Agricultural Education and Extension 6 213–222

    Google Scholar 

  17. W. Lockeretz (2000) “Organic farming research, today and tomorrow” T. Alföldi W. Lockeretz U. Niggli (Eds) “The World Grows Organic” Proceedings 13th International IFOAM Scientific Conference VDF Hochschulverlag AG an der ETH Zürich Basel, Switzerland 718–720

    Google Scholar 

  18. H. E. Longino (1990) Science as Social Knowledge. Values and Objectivity in Scientific Inquiry Princeton University Press Princeton, New Jersey

    Google Scholar 

  19. T. Luukkonen (1995) ArticleTitle“The impacts of research field evaluations on research practice” Research Policy 24 349–365 Occurrence Handle10.1016/0048-7333(93)00770-T

    Article  Google Scholar 

  20. H. Marsh P. Bazeley (1999) ArticleTitle“Multiple evaluations of grant proposals by independent reviewers: Confirmatory factor analysis evaluations and reliability, validity, and structure” Multivariate Behavioral Research 34 1–30 Occurrence Handle10.1207/s15327906mbr3401_1

    Article  Google Scholar 

  21. A. Miller (1985) ArticleTitle“Psychosocial origins of conflict over pest-control strategies” Agriculture, Ecosystems and Environment 12 235–251 Occurrence Handle10.1016/0167-8809(85)90115-X

    Article  Google Scholar 

  22. E. Neumayer (1999) Weak Versus Strong Sustainability. Exploring the Limits of Two Opposing Paradigms Edward Elgar Cheltenham, UK

    Google Scholar 

  23. R. Packham N. Sriskandarajah (2005) ArticleTitle“Systemic action research for postgraduate education in agriculture and rural development” Systems Research and Behavioral Science 22 IssueID2 119–130 Occurrence Handle10.1002/sres.679

    Article  Google Scholar 

  24. J. Perkins (1982) Insects, Experts, and the Insecticide Crisis. The Quest for New Pest Management Strategies Plenum Press New York, New York: USA

    Google Scholar 

  25. J. Rasmussen P. Kaltoft (2003) ArticleTitle“Alternative versus conventional values and attitudes in higher agricultural education” Biological Agriculture and Horticulture 20 347–363

    Google Scholar 

  26. P. M. Rothwell C. N. Martyn (2000) ArticleTitle“Reproducibility of peer review in clinical neuroscience. Is agreement between reviewers any greater than would be expected by change alone?” Brain 123 1964–1969 Occurrence Handle10.1093/brain/123.9.1964

    Article  Google Scholar 

  27. W. D. Thompson S. D. Walter (1988) ArticleTitle“A reappraisal of the Kappa coefficient” Journal of Clinical Epidemiology 41 949–958 Occurrence Handle10.1016/0895-4356(88)90031-5

    Article  Google Scholar 

  28. G. D. L. Travis H. M. Collins (1991) ArticleTitle“New light on old boys: Cognitive and institutional particularism in the peer review system” Science, Technology, and Human Values 16 322–341

    Google Scholar 

  29. C. Wennerås A. Wold (1997) ArticleTitle“Nepotism and sexism in peer-review” Nature 387 341–343 Occurrence Handle10.1038/387341a0

    Article  Google Scholar 

  30. S. Wessely (1998) ArticleTitle“Peer review of grant applications: What do we know?” The Lancet 352 301–305 Occurrence Handle10.1016/S0140-6736(97)11129-1

    Article  Google Scholar 

  31. J. R. Wilson (2002) ArticleTitle“Responsible authorship and peer review” Science and Engineering Ethics 8 155–174

    Google Scholar 

Download references

Author information

Affiliations

Authors

Corresponding author

Correspondence to Jesper Rasmussen.

Additional information

Jesper Rasmussenis an Associate Professor in the Department of Agricultural Sciences, The Royal Veterinary and Agricultural University, Denmark. He teaches organic farming, plant production, and physical weed control. His research interests include non-chemical weed management, soil tillage in organic farming systems, and the role of values in teaching and research.

Vibeke Langeris an Associate Professor in the Department of Agricultural Sciences, The Royal Veterinary and Agricultural University, Denmark. She teaches organic farming, farming systems, and pest management in organic agriculture. Her research interests include land use, production, and nature management on organic farms.

Hugo Fjelsted Alrøeis a Senior Scientist at the Danish Research Center for Organic Food and Farming (DARCOF). His research interests are in the philosophy of research and science, with a focus on systemic and transdisciplinary research, systems theory and the role of values in science, and ethics and value inquiry in relation to sustainability, precaution, sustainable agriculture, and organic farming. He also works with communication technologies as tools for research.

Rights and permissions

Reprints and Permissions

About this article

Cite this article

Rasmussen, J., Langer, V. & Alrøe, H.F. Bias in Peer Review of Organic Farming Grant Applications. Agric Hum Values 23, 181–188 (2006). https://doi.org/10.1007/s10460-005-6105-6

Download citation

Keywords

  • Conventional agriculture
  • Objective science
  • Organic agriculture
  • Peer review process
  • Scientific paradigms
  • Scientific quality
  • Societal relevance
  • Sweden