, 215:126 | Cite as

Influence of experimental design on decentralized, on-farm evaluation of populations: a simulation study

  • Gaëlle van FrankEmail author
  • Isabelle Goldringer
  • Pierre Rivière
  • Olivier David


Participatory plant breeding (PPB) has received much attention in recent decades for its ability to develop varieties adapted to the diversity of farm conditions and to farmers’ needs and practices. Specific methodological issues arise when working with on-farm experiments, one being the implementation of an experimental design that matches farmers’ constraints and objectives, while allowing for accurate statistical analyses of the data. We took the example of a French PPB case on bread wheat, in which farmers, facilitators and researchers have co-constructed an experimental design that meets their needs, but is very unbalanced and required the development of Bayesian statistical models to compare populations on-farm, over environments and analyze their sensitivity to environments. Through a simulation study, we investigated the effects of different characteristics of the experimental design on the behavior of two of these Bayesian models to identify the range of values that are most appropriate and give recommendations for decentralized experiments. We analyzed the estimates obtained by the models using different simulated datasets that differed by the values of the experimental design’s parameters. While within-environment population effects were well estimated even with few replicated controls, replicating populations of interest rather than controls within environments and including enough environments provided more power to detect significant differences. Population effects and sensitivities over environments were mainly impacted by the number of replications of populations across environments, therefore effort should be made to repeat populations in more environments if the aim is to characterize their behavior under various environmental conditions.


Bayesian modeling Decentralized breeding On-farm trials Participatory plant breeding 



We thank all the farmers and facilitators participating to the PPB project, as well as Alexandre Protat for assistance with programming the simulations. We also thank the reviewers for their helpful comments on improving the manuscript. This project has received funding from the European Union’s Horizon 2020 research and innovation programme under Grant Agreement No 633571 (DIVERSIFOOD).

Supplementary material

10681_2019_2447_MOESM1_ESM.pdf (126 kb)
Supplementary material 1 (pdf 127 KB)


  1. Altieri M, Koohafkan P (2013) Strengthening resilience of farming systems: a prerequisite for sustainable agricultural production. Wake up before it is too late: make agriculture truly sustainable now for food security in a changing climate. UNCTAD Trade and Environment Review, Geneva, pp 56–60Google Scholar
  2. Altieri MA, Nicholls CI, Henao A, Lana MA (2015) Agroecology and the design of climate change-resilient farming systems. Agron Sustain Dev 35(3):869–890. CrossRefGoogle Scholar
  3. Annicchiarico P (2007) Wide-versus specific-adaptation strategy for lucerne breeding in northern Italy. Theor Appl Genet 114(4):647–657. CrossRefPubMedGoogle Scholar
  4. Annicchiarico P, Chiapparino E, Perenzin M (2010) Response of common wheat varieties to organic and conventional production systems across Italian locations, and implications for selection. Field Crops Res 116(3):230–238. CrossRefGoogle Scholar
  5. Aw-Hassan A, Mazid A, Salahieh H (2008) The role of informal farmer-to-farmer seed distribution in diffusion of new barley varieties in Syria. Exp Agric 44(3):413–431. CrossRefGoogle Scholar
  6. Azaïs JM, Monod H, Bailey RA (1998) The influence of design on validity and efficiency of neighbour methods. Biometrics 54(4):1374–1387CrossRefGoogle Scholar
  7. Barot S, Allard V, Cantarel A, Enjalbert J, Gauffreteau A, Goldringer I, Lata JC, Le Roux X, Niboyet A, Porcher E (2017) Designing mixtures of varieties for multifunctional agriculture with the help of ecology. A review. Agron Sustain Dev 37(2):13. CrossRefGoogle Scholar
  8. Bellon MR, Reeves J (eds) (2002) Quantitative analysis of data from participatory methods in plant breeding. CIMMYT, TexcocoGoogle Scholar
  9. Besag J, Higdon D (1999) Bayesian analysis of agricultural field experiments. R Stat Soc 61(4):691–746. CrossRefGoogle Scholar
  10. van Bueren EL, Jones S, Tamm L, Murphy K, Myers J, Leifert C, Messmer M (2011) The need to breed crop varieties suitable for organic farming, using wheat, tomato and broccoli as examples: a review. NJAS - Wagening J Life Sci 58(3–4):193–205. CrossRefGoogle Scholar
  11. van Bueren ETL, Struik PC, Jacobsen E (2002) Ecological concepts in organic farming and their consequences for an organic crop ideotype. NJAS - Wagening J Life Sci 50(1):1–26. CrossRefGoogle Scholar
  12. Carson Y, Maria A (1997) Simulation optimization: methods and applications. In: Proceedings of the 29th conference on winter simulation, IEEE Computer Society, Washington, DC, USA, WSC ’97, pp 118–126.
  13. Ceccarelli S (1989) Wide adaptation: how wide? Euphytica 40(3):197–205. CrossRefGoogle Scholar
  14. Ceccarelli S (2012) Plant breeding with farmers—a technical manual. ICARDA, AleppoGoogle Scholar
  15. Ceccarelli S (2015) Efficiency of plant breeding. Crop Sci 55(1):87. CrossRefGoogle Scholar
  16. Ceccarelli S, Grando S (2007) Decentralized-participatory plant breeding: an example of demand driven research. Euphytica 155(3):349–360. CrossRefGoogle Scholar
  17. Chaloner K, Verdinelli I (1995) Bayesian experimental design: a review. Stat Sci 10(3):273–304CrossRefGoogle Scholar
  18. Coomes OT, McGuire SJ, Garine E, Caillon S, McKey D, Demeulenaere E, Jarvis D, Aistara G, Barnaud A, Clouvel P, Emperaire L, Louafi S, Martin P, Massol F, Pautasso M, Violon C, Wencélius J (2015) Farmer seed networks make a limited contribution to agriculture? Four common misconceptions. Food Policy 56:41–50. CrossRefGoogle Scholar
  19. Cotes JM, Crossa J, Sanches A, Cornelius PL (2006) A Bayesian approach for assessing the stability of genotypes. Crop Sci 46(6):2654–2665. CrossRefGoogle Scholar
  20. Cullis BR, Smith AB, Coombes NE (2006) On the design of early generation variety trials with correlated data. J Agric Biol Environ Stat 11(4):381–393. CrossRefGoogle Scholar
  21. David O (1994) Balanced block designs under interactive linear models. J Stat Plan Inference 39(1):33–41CrossRefGoogle Scholar
  22. Desclaux D, Nolot JM, Chiffoleau Y, Gozé E, Leclerc C (2008) Changes in the concept of genotype x environment interactions to fit agriculture diversification and decentralized participatory plant breeding: pluridisciplinary point of view. Euphytica 163(3):533–546. CrossRefGoogle Scholar
  23. Digby PGN (1979) Modified joint regression analysis for incomplete variety x environment data. J Agric Sci 93(01):81. CrossRefGoogle Scholar
  24. Finckh M, Gacek E, Goyeau H, Lannou C, Merz U, Mundt C, Munk L, Nadziak J, Newton A, de Vallavieille-Pope C (2000) Cereal variety and species mixtures in practice, with emphasis on disease resistance. Agronomie 20(7):813–837CrossRefGoogle Scholar
  25. Finlay K, Wilkinson G (1963) The analysis of adaptation in a plant-breeding programme. Aust J Agric Res 14(6):742–754CrossRefGoogle Scholar
  26. Gelman A, Rubin D (1992) Inference from iterative simulation using multiple sequences. Stat Sci 7(4):457–511. CrossRefGoogle Scholar
  27. Humphries S, Rosas JC, Gómez M, Jiménez J, Sierra F, Gallardo O, Avila C, Barahona M (2015) Synergies at the interface of farmer-scientist partnerships: agricultural innovation through participatory research and plant breeding in Honduras. Agric Food Secur 4(1):27. CrossRefGoogle Scholar
  28. Kempton RA, Fox PN, Cerezo M (2012) Statistical methods for plant variety evaluation. Springer, BerlinGoogle Scholar
  29. Kleinknecht K, Möhring J, Laidig F, Meyer U, Piepho H (2016) A simulation-based approach for evaluating the efficiency of multienvironment trial designs. Crop Sci 56(5):2237. CrossRefGoogle Scholar
  30. Kobilinsky A, Monod H, Bailey RA (2017) Automatic generation of generalised regular factorial designs. Comput Stat Data Anal 113:311–329CrossRefGoogle Scholar
  31. Lian L, de los Campos G (2016) Fw: An r package for Finlay-Wilkinson regression that incorporates genomic/pedigree information and covariance structures between environments. G3: Genes Genomes Genet 6(3):589–597. CrossRefGoogle Scholar
  32. Müller P (2005) Optimal design: simulation approaches. In: Dey DK, Rao CR (eds) Handbook of statistics, vol 25. Elsevier, Amsterdam, pp 509–518Google Scholar
  33. Murphy KM, Campbell KG, Lyon SR, Jones SS (2007) Evidence of varietal adaptation to organic farming systems. Field Crops Res 102(3):172–177. CrossRefGoogle Scholar
  34. Nabugoomu F, Kempton RA, Talbot M (1999) Analysis of series of trials where varieties differ in sensitivity to locations. J Agric Biol Environ Stat 4(3):310. CrossRefGoogle Scholar
  35. Omer SO, Abdalla AWH, Mohammed MH, Singh M (2015) Bayesian estimation of genotype-by-environment interaction in sorghum variety trials. Commun Biometry Crop Sci 10(2):82–95Google Scholar
  36. O’Hara RB, Cano JM, Ovaskainen O, Teplitsky C, Alho JS (2008) Bayesian approaches in evolutionary quantitative genetics. J Evol Biol 21(4):949–957CrossRefGoogle Scholar
  37. Pautasso M, Aistara G, Barnaud A, Caillon S, Clouvel P, Coomes OT, Delêtre M, Demeulenaere E, De Santis P, Döring T, Eloy L, Emperaire L, Garine E, Goldringer I, Jarvis D, Joly HI, Leclerc C, Louafi S, Martin P, Massol F, McGuire S, McKey D, Padoch C, Soler C, Thomas M, Tramontini S (2013) Seed exchange networks for agrobiodiversity conservation. A review. Agron Sustain Dev 33(1):151–175. CrossRefGoogle Scholar
  38. Plummer M (2003) Jags: a program for analysis of bayesian graphical models using gibbs samplingGoogle Scholar
  39. Plummer M (2016) rjags: Bayesian graphical models using MCMC., r package version 4-6
  40. R Core Team (2018) R: A Language and Environment for Statistical Computing. R Foundation for Statistical Computing, Vienna, Austria.
  41. Rincent R, Kuhn E, Monod H, Oury FX, Rousset M, Allard V, Le Gouis J (2017) Optimization of multi-environment trials for genomic selection based on crop models. Theor Appl Genet 130(8):1735–1752CrossRefGoogle Scholar
  42. Rivière P (2014) Méthodologie de la sélection décentralisée et participative : un exemple sur le blé tendre. PhD thesis, Paris-SudGoogle Scholar
  43. Rivière P, Pin S, de Oliviera Y, David O, Dawson J, Wanner A, Heckmann R, Obbellianne S, Ronot B, Parizot S, Hyancinthe A, Dalmasso C, Baltassat R, Bochède A, Mailhe G, Caizergue F, Gascuel JS, Gasnier R, Berthellot JF, Baboulène J, Poilly C, Lavoyer R, Hernandez MP, Coulbeaut JM, Peloux F, Mouton A, Mercier F, Ranke O, Wittrish R, de Kochko P, Goldringer I (2013) Mise en place d’une méthodologie de sélection participative sur le blé tendre en France. Innov Agron 32:427–441Google Scholar
  44. Rivière P, Dawson JC, Goldringer I, David O (2015) Hierarchical Bayesian modeling for flexible experiments in decentralized participatory plant breeding. Crop Sci 55(3):1053. CrossRefGoogle Scholar
  45. Rivière P, van Frank G, David O, Muñoz F (2017) PPBstats: an R package to perform analysis found within PPB programmes regarding network of seeds circulation, agronomic trials, organoleptic tests and molecular experiments. Version 0.23. URL
  46. Saltelli A, Ratto M, Andres T, Campolongo F, Cariboni J, Gatelli D, Saisana M, Tarantola S (2008) Global sensitivity analysis: the primer. Wiley, HobokenGoogle Scholar
  47. Silvey SD (1980) Optimal designs: an introduction to the theory for parameter estimation. Chapman and Hall, LondonCrossRefGoogle Scholar
  48. Simmonds NW (1991) Selection for local adaptation in a plant breeding programme. Theor Appl Genet 82(3):363–367CrossRefGoogle Scholar
  49. Singh M, Tavva S, Saharawat YS, Rizvi J (2018) A Bayesian assessment of productivity and risks to achieve target yields from improved Chickpea and Mung Bean varieties using on-farm trials in Afghanistan. Exp Agric 54(03):470–481. CrossRefGoogle Scholar
  50. Sperling L, Ashby JA, Smith ME, Weltzien E, McGuire S (2001) A framework for analyzing participatory plant breeding approaches and results. Euphytica 122(3):439–450CrossRefGoogle Scholar
  51. Sun X, Peng T, Mumm RH (2011) The role and basics of computer simulation in support of critical decisions in plant breeding. Mol Breed 28(4):421–436. CrossRefGoogle Scholar
  52. Tekin E, Sabuncuoglu I (2004) Simulation optimization: a comprehensive review on theory and applications. IIE Trans 36(11):1067–1081. CrossRefGoogle Scholar
  53. Theobald CM, Talbot M, Nabugoomu F (2002) A bayesian approach to regional and local-area prediction from crop variety trials. J Agric Biol Environ Stat 7(3):403–419. CrossRefGoogle Scholar
  54. Thomas M, Dawson JC, Goldringer I, Bonneuil C (2011) Seed exchanges, a key to analyze crop diversity dynamics in farmer-led on-farm conservation. Genet Resources Crop Evol 58(3):321–338. CrossRefGoogle Scholar
  55. de Vallavieille-Pope C (2004) Management of disease resistance diversity of cultivars of a species in single fields: controlling epidemics. Comptes Rendus Biol 327(7):611–620. CrossRefGoogle Scholar
  56. Witcombe J, Yadavendra J (2014) How much evidence is needed before client-oriented breeding (COB) is institutionalised? Evidence from rice and maize in India. Field Crops Res 167:143–152. CrossRefGoogle Scholar
  57. Witcombe JR, Joshi A, Goyal SN (2003) Participatory plant breeding in maize: a case study from Gujarat, India. Euphytica 130(3):413–422CrossRefGoogle Scholar
  58. Østergård H, Finckh MR, Fontaine L, Goldringer I, Hoad SP, Kristensen K, van Bueren ETL, Mascher F, Munk L, Wolfe MS (2009) Time for a shift in crop production: embracing complexity through diversity at all levels. J Sci Food Agric 89(9):1439–1445. CrossRefGoogle Scholar

Copyright information

© Springer Nature B.V. 2019

Authors and Affiliations

  1. 1.GQE – Le Moulon, INRA, Univ. Paris-Sud, CNRS, AgroParisTech, Université Paris-SaclayGif-sur-YvetteFrance
  2. 2.Réseau Semences PaysannesAiguillonFrance
  3. 3.MaIAGE, INRA, Université Paris-SaclayJouy-en-JosasFrance

Personalised recommendations