Peer reviews of 84 organic farming grant applications from Sweden were analyzed to determine whether the reviewers’ affiliation to one of two types of agriculture (i.e., organic and conventional) influenced their reviews. Fifteen reviewers were divided into three groups: (1) scientists with experience in organic farming research; (2) scientists with no experience in organic farming research; and (3) users of organic farming research. The two groups of scientists assessed the societal relevance and scientific quality of the grant applications based on three criteria (i.e., presentation, methodology, qualifications), whereas the user group only assessed societal relevance. The analysis showed that the two groups of scientists provided very different reviews. Scientist reviewers with experience in organic farming research agreed more with the user group on research relevance than did scientist reviewers without such experience, and the assessment of relevance was closely correlated to the assessment of scientific quality within both scientific groups. As both scientific groups did not clearly distinguish between societal relevance and scientific quality, the idea of an objective science is challenged. The contextual values associated with the norms of good agriculture were not clearly distinguished from the constitutive values of science associated with the traditional norms of good science. This raises the question of whether organic and conventional grant applications should be mixed for review regardless of the reviewers.
This is a preview of subscription content, access via your institution.
Buy single article
Instant access to the full article PDF.
Tax calculation will be finalised during checkout.
Subscribe to journal
Immediate online access to all issues from 2019. Subscription will auto renew annually.
Tax calculation will be finalised during checkout.
H. F. Alrøe (2000) ArticleTitle“Science as systems learning. Some reflections on the cognitive and communicational aspects of science” Cybernetics and Human Knowing 7 IssueID4 57–78
H. F. Alrøe E. S. Kristensen (2002) ArticleTitle“Towards a systemic research methodology in agriculture: Rethinking the role of values in science” Agriculture and Human Values 19 3–23 Occurrence Handle10.1023/A:1015040009300
R. U. Ayres J. C. J. M. Bergh Particlevan den J. M. Gowdy (2001) ArticleTitle“Strong versus weak sustainability: Economics, natural sciences, and ‘consilience’” Environmental Ethics 23 155–168
P. Bazeley (1998) ArticleTitle“Peer review and panel decisions in the assessment of Australian Research Council project grant applicants: What counts in a highly competitive context?” Higher Education 35 435–452 Occurrence Handle10.1023/A:1003118502318
C. E. Beus R. E. Dunlap (1990) ArticleTitle“Conventional versus alternative agriculture: The paradigmatic roots of the debate” Rural Sociology 55 590–616 Occurrence Handle10.1111/j.1549-0831.1990.tb00699.x
C. E. Beus R. E. Dunlap (1991) ArticleTitle“Measuring adherence to alternative vs. conventional agricultural paradigms: A proposed scale” Rural Sociology 56 432–460 Occurrence Handle10.1111/j.1549-0831.1991.tb00442.x
J. Christensen (1998) Alternativer – Natur – Landbrug (Alternatives – Nature – Agriculture) Akademisk Forlag Copenhagen, Denmark
D. V. Cicchetti (1991a) ArticleTitle“The reliability of peer-review for manuscript and grant submissions – A cross-disciplinary investigation” Behavioral and Brain Sciences 14 119–134
D. V. Cicchetti (1991b) ArticleTitle“Reflections from the peer-review mirror” Behavioral and Brain Sciences 14 167–186
S. Cole J. R. Cole G. A. Simon (1981) ArticleTitle“Change and consensus in peer review” Science 214 881–886
C. P. Egri (1999) ArticleTitle“Attitudes,, backgrounds and information preferences of Canadian organic and conventional farmers: Implications for organic farming advocacy and extension” Journal of Sustainable Agriculture 13 IssueID3 45–72
C. Francis J. King (1997) ArticleTitle“Impact of personal values on agricultural research” Society and Natural Resources 10 273–282 Occurrence Handle10.1080/08941929709381026
C. A. Francis G. Lieblein J. Helenius L. Salomonsson H. Olsen J. Porter (2001) ArticleTitle“Challenges in designing ecological agriculture education: A Nordic perspective on change” American Journal of Alternative Agriculture 16 IssueID2 89–95 Occurrence Handle10.1017/S0889189300008985
R. Harding (Eds) (1998) Environmental Decision-making: The Roles of Scientists, Engineers and the Public Sydney, New South Wales The Federation Press
J. Huckle S. Sterling (1996) Education for Sustainability Earthscan Publications Ltd London, UK
G. Lieblein C. Francis J. King (2000) ArticleTitle“Conceptual framework for structuring future agricultural colleges and universities in industrial countries” The Journal of Agricultural Education and Extension 6 213–222
W. Lockeretz (2000) “Organic farming research, today and tomorrow” T. Alföldi W. Lockeretz U. Niggli (Eds) “The World Grows Organic” Proceedings 13th International IFOAM Scientific Conference VDF Hochschulverlag AG an der ETH Zürich Basel, Switzerland 718–720
H. E. Longino (1990) Science as Social Knowledge. Values and Objectivity in Scientific Inquiry Princeton University Press Princeton, New Jersey
T. Luukkonen (1995) ArticleTitle“The impacts of research field evaluations on research practice” Research Policy 24 349–365 Occurrence Handle10.1016/0048-7333(93)00770-T
H. Marsh P. Bazeley (1999) ArticleTitle“Multiple evaluations of grant proposals by independent reviewers: Confirmatory factor analysis evaluations and reliability, validity, and structure” Multivariate Behavioral Research 34 1–30 Occurrence Handle10.1207/s15327906mbr3401_1
A. Miller (1985) ArticleTitle“Psychosocial origins of conflict over pest-control strategies” Agriculture, Ecosystems and Environment 12 235–251 Occurrence Handle10.1016/0167-8809(85)90115-X
E. Neumayer (1999) Weak Versus Strong Sustainability. Exploring the Limits of Two Opposing Paradigms Edward Elgar Cheltenham, UK
R. Packham N. Sriskandarajah (2005) ArticleTitle“Systemic action research for postgraduate education in agriculture and rural development” Systems Research and Behavioral Science 22 IssueID2 119–130 Occurrence Handle10.1002/sres.679
J. Perkins (1982) Insects, Experts, and the Insecticide Crisis. The Quest for New Pest Management Strategies Plenum Press New York, New York: USA
J. Rasmussen P. Kaltoft (2003) ArticleTitle“Alternative versus conventional values and attitudes in higher agricultural education” Biological Agriculture and Horticulture 20 347–363
P. M. Rothwell C. N. Martyn (2000) ArticleTitle“Reproducibility of peer review in clinical neuroscience. Is agreement between reviewers any greater than would be expected by change alone?” Brain 123 1964–1969 Occurrence Handle10.1093/brain/123.9.1964
W. D. Thompson S. D. Walter (1988) ArticleTitle“A reappraisal of the Kappa coefficient” Journal of Clinical Epidemiology 41 949–958 Occurrence Handle10.1016/0895-4356(88)90031-5
G. D. L. Travis H. M. Collins (1991) ArticleTitle“New light on old boys: Cognitive and institutional particularism in the peer review system” Science, Technology, and Human Values 16 322–341
C. Wennerås A. Wold (1997) ArticleTitle“Nepotism and sexism in peer-review” Nature 387 341–343 Occurrence Handle10.1038/387341a0
S. Wessely (1998) ArticleTitle“Peer review of grant applications: What do we know?” The Lancet 352 301–305 Occurrence Handle10.1016/S0140-6736(97)11129-1
J. R. Wilson (2002) ArticleTitle“Responsible authorship and peer review” Science and Engineering Ethics 8 155–174
Jesper Rasmussenis an Associate Professor in the Department of Agricultural Sciences, The Royal Veterinary and Agricultural University, Denmark. He teaches organic farming, plant production, and physical weed control. His research interests include non-chemical weed management, soil tillage in organic farming systems, and the role of values in teaching and research.
Vibeke Langeris an Associate Professor in the Department of Agricultural Sciences, The Royal Veterinary and Agricultural University, Denmark. She teaches organic farming, farming systems, and pest management in organic agriculture. Her research interests include land use, production, and nature management on organic farms.
Hugo Fjelsted Alrøeis a Senior Scientist at the Danish Research Center for Organic Food and Farming (DARCOF). His research interests are in the philosophy of research and science, with a focus on systemic and transdisciplinary research, systems theory and the role of values in science, and ethics and value inquiry in relation to sustainability, precaution, sustainable agriculture, and organic farming. He also works with communication technologies as tools for research.
About this article
Cite this article
Rasmussen, J., Langer, V. & Alrøe, H.F. Bias in Peer Review of Organic Farming Grant Applications. Agric Hum Values 23, 181–188 (2006). https://doi.org/10.1007/s10460-005-6105-6
- Conventional agriculture
- Objective science
- Organic agriculture
- Peer review process
- Scientific paradigms
- Scientific quality
- Societal relevance