Bias in Peer Review of Organic Farming Grant Applications

Abstract

Peer reviews of 84 organic farming grant applications from Sweden were analyzed to determine whether the reviewers’ affiliation to one of two types of agriculture (i.e., organic and conventional) influenced their reviews. Fifteen reviewers were divided into three groups: (1) scientists with experience in organic farming research; (2) scientists with no experience in organic farming research; and (3) users of organic farming research. The two groups of scientists assessed the societal relevance and scientific quality of the grant applications based on three criteria (i.e., presentation, methodology, qualifications), whereas the user group only assessed societal relevance. The analysis showed that the two groups of scientists provided very different reviews. Scientist reviewers with experience in organic farming research agreed more with the user group on research relevance than did scientist reviewers without such experience, and the assessment of relevance was closely correlated to the assessment of scientific quality within both scientific groups. As both scientific groups did not clearly distinguish between societal relevance and scientific quality, the idea of an objective science is challenged. The contextual values associated with the norms of good agriculture were not clearly distinguished from the constitutive values of science associated with the traditional norms of good science. This raises the question of whether organic and conventional grant applications should be mixed for review regardless of the reviewers.