Diversity of experimentation by farmers engaged in agroecology

Abstract

Agroecology questions the production of generic knowledge. Rather than searching for the best practices for large-scale transfer, it would be more efficient to help farmers find their own solutions. A promising activity for farmers is experimentation because it answers their needs and helps them learn. However, how agroecological practices are tested by farmers in their own experiments is still poorly known. In this study, we examined the short-term experimental activity, i.e., experiments carried out at a yearly scale in pre-defined fields. Seventeen farmers in south eastern France were surveyed. The farmers practiced conventional or organic farming and cultivated either arable or market garden crops. Experiments on agroecological practices were characterized, located along a timeline, and discussed with them. To conduct the interviews with the farmers, each experiment was described in three stages: (1) designing the experiment, (2) managing it in real time, and (3) evaluating the results of the experiment. The data collected in the interviews were first analyzed to build a descriptive framework of farmers’ experiments, after which hierarchical cluster analysis was used to analyze the diversity of the farmers’ experiments. Here, we propose for the first time a generic framework to describe farmers’ experiments at a short time scale based on the consistency between the Design, Management, and Evaluation stages. We used the framework to characterize the diversity of farmers’ experiments and identified four clusters. The originality of this work is both building a descriptive framework resulting from in-depth analyses of farmers’ discourse and using statistical tools to identify and interpret the groups of experiments. Our results provide a better understanding of farmers’ experiments and suggest tools and methods to help them experiment, a major challenge in the promotion of a large-scale agroecological transition.

Introduction

Agroecology is a promising paradigm to reconcile many different agriculture issues (Altieri 1989). However, establishing agroecosystems in which ecological processes and natural regulations are favored calls current agricultural research and development into question. First, knowledge on ecological processes in farming systems is partial and hard to translate into appropriate practices. There are many interactions between ecological processes and their management by humans, and our ability to quantify and valorize them is limited (Mediene et al. 2011; Griffon 2017). As a consequence, using natural regulations that are very sensitive to local pedoclimatic and ecological conditions is uncertain, and it is hard to predict what will happen if agroecological practices that are assessed in one location are transferred to other locations. Second, farmers have to acquire knowledge to deal with these practices and processes in their own conditions. All these factors make it difficult for research and extension services to develop and disseminate new knowledge on agroecological systems (Meynard 2017). Farmers’ innovations and learning abilities have been highlighted for many years, for example, in the “farmer first” paradigm (Chambers et al. 1994), and, in particular, their capacity to learn through on-farm experimentation (Johnson 1972; Sumberg and Okali 1997; Saad 2002). Farmers’ experiments are now considered a useful way to enhance agroecological transition (Kummer et al. 2010; De Tourdonnet et al. 2013) and to develop more sustainable farming systems (Darnhofer et al. 2010). First, experimentation enables farmers to learn. Testing, observing, and drawing conclusions are useful skills when working under uncertainty and changing conditions and help build resilience (Kummer et al. 2012). Second, experimentation helps farmers develop agroecological practices that are adapted to their own farming contexts and motivation. Finally, experiments on practices help identify possible ways to manage agroecosystems. The aim of this article is to describe farmers’ experiments with agroecological practices (Fig. 1). We studied how farmers take their agroecosystems into account when planning experiments and, in return, how they refine agroecological practices through experimentation. One difficulty is that the term farmers’ experiment covers a diversity of processes at different time scales. Farmers’ experiments have been described at annual scale (Stolzenbach 1994; Lyon 1996; Wettasinha et al. 1997; Bhuktan et al. 1999; Quiroz 1999), at multi-annual scale (Scheuermeier 1997; Mak 2001) and at varying time scales (Bentley 2006; Kummer 2011; Leitgeb et al. 2014). In this article, we focus on the short-term dimension of farmers’ experiments: we define an experimental situation as the experimentation of a practice that is clearly delimited in time (from a few days up to one cropping year) and in space (a field, a field strip, or a field and its surrounding ecological elements like a hedge or a strip of flowers). The study outputs are first an analytical framework capable of describing experimental situations in detail. Second, its application to a case study enabled the description and characterization of 10 groups of experimental situations.

Fig. 1

Farmer experimenting wheat cultivation without tillage (here after sorghum). Observation of earthworm presence

Materials and methods

A conceptual model of experimental situations was built based on the literature (section 2.1). The model was used as a guide to survey a set of farmers in a case study in south eastern France and to investigate their experimental situations (section 2.2). In section 2.3, we explain how the farmers’ discourse was analyzed.

A conceptual model to describe the experimental situations

Can the description of experimental situations be inspired by scientific experimentation? It is clear that very few farmers experiment using replications, a reference or strict control of factors, as is usually done in the case of factorial experiments to prove or disprove a hypothesis. There are differences in terms of epistemology as farmers seek to act, based on what works in their conditions, rather than to understand all the cause-effect relationships (Hoffmann et al. 2007). Stolzenbach (1994), quoting Schön (1983), showed that farmers experimenting are not only testing a hypothesis, they are also exploring (i.e., implementing a new practice to see what will happen, with no attempt to predict the results) and adapting their actions to reach objectives (called move testing by Schön). Nevertheless, some authors—taking a broader definition of experiment—consider that farmers’ experiments are close to scientific ones (Millar 1994; Hocdé 1997; Sumberg and Okali 1997). Therefore, we studied farmers’ experiments using a generic pattern shared with scientists, based on broad chronological stages (Millar 1994; Hocdé 1997; Quiroz 1999) starting from the design of the experiment, its management in real time, up to the evaluation of the results. In this article, we use a conceptual model of experimental situations based on three stages, adapted from Catalogna and Navarrete (2016). The Design stage describes the farmers’ goals for the experiment and the physical experimental design (which practices are tested, where, and how). The Management stage focuses on how farmers manage the agroecosystem during the experiment depending on real farming conditions and how they collect information to assess the agroecosystem. In the Evaluation stage, lessons concerning the modalities tested are drawn by farmers and, in some cases, are discussed.

Using the conceptual model to survey farmers in the case study

The conceptual model was applied to a set of farmers in the Rhône-Alpes (France), a region where farmers have been experimenting with organic or agroecological practices for several decades. We compared two production systems (market gardening and arable crops) with specific constraints, e.g., for vegetable crops, more crops to deal with, smaller plots, and more frequent observations are required than for arable crops. The topic of the experiments also differed, e.g., more questions are raised concerning pest control for market gardening, whereas in the case of arable crops, questions concern soil management and crop rotation. We selected both conventional and organic farmers: organic farmers face more constraints and fewer opportunities to correct mistakes while an experiment is underway but usually have more experience in agroecological practices. Most of the farms studied were located in the Drôme Administrative Department, in the lowlands (alluvial plains), where mainly arable crops are cultivated but also market gardening crops. We surveyed 17 farms: seven growing arable crops (three organic, three conventional, and one in the process of converting to organic farming), and 10 growing vegetable crops (seven organic and three conventional). Based on an initial phone interview, we selected farmers who had conducted experiments over the last 10 years and were able to describe their experiments with precision. As “experiment” is a loaded term for practitioners, several terms were used to evaluate if they were relevant for the study: “Have you tested or tried new practices, have you changed your practices in the last 10 years?” No specific agroecological practices were selected before the phone interviews but farmers were questioned on some broad categories of practices frequently associated with agroecology: crop rotation, reduced soil tillage, cover crops, and biological control. If experimental situations were suspected during the phone interview and the farmer agreed, he/she was later interviewed on his/her farm.

Interviews were semi-directive, lasted 2 h on average, and were recorded. Farmers were first asked to briefly describe their farm as well as any recent changes they had made to give us a first impression of the context. We then asked the farmers to talk about any experiments that spontaneously came to mind; these were then positioned along a timeline established during the interview. The timeline was used again later to question farmers about any other experimental situation in which they had been involved. The conceptual model served as a guide for the interview with the farmers: for each experimental situation identified, we discussed the three stages and refocused the discussion if one of the stages was omitted. If appropriate, a visit to the location where the experiment was in progress provided an opportunity to summarize what was understood and, if necessary, correct and complete the survey.

Interviews were first transcribed on Nvivo software. A total of 181 experimental situations with sufficient details were selected (ranging from 1 to 19 per farmer, with a mean of 10 per farmer). The experimental situations were coded in a scalable framework describing several variables for each stage of the model. Based on inductive reasoning, modalities and their corresponding variables were determined incrementally while coding the interviews in Excel. The objective was to build a framework for agricultural scientists and agronomists, capable to characterize the experiments as a way to build knowledge. So the variables were chosen in order to estimate how knowledge can be built and to analyze the consistency between the aims of one experiment and its means and the consistency between the results formulated by a farmer and the means and design of the experiment. A first set of 30 experimental situations set up by seven farmers enabled the creation of a first prototype of the analytical framework. The framework was refined if and when the analysis of a new experimental situation introduced new elements that could not be coded with the current version of the framework. As a consequence, the 13 descriptive variables of the framework are not predetermined but the result of iterations between the coding and the refining steps.

Analysis of the diversity of experimental situations in the case study

To analyze the diversity of the 181 previously identified experimental situations, we first performed multiple factor analysis (MFA), a statistical method used to analyze data sets described by qualitative variables. The modalities of each variable were all disjunctive. Next, hierarchical ascendant classification (HAC) enabled us to identify homogeneous groups of experimental situations (K means were used to improve the clusters) and to reveal linkages between different modalities of variables. HAC analysis helped us to explore how the data were organized. A classification based on expertise would have been difficult in this study due to the large number of experimental situations (181) and variables (13). Two variables related to the information collected in the Management stage and judgment in the Evaluation stage were structured in groups to limit their relative effects on the analysis. The analyses were conducted with R software using the FactomineR package (Lê et al. 2008). This method was inspired by Chantre et al. (2014), who used it to identify groups of farmers’ learning processes.

As the MFA resulted in a large number of dimensions (41), four HAC analyses were performed, with 5, 8, 15, and 21 dimensions, respectively, explaining 35, 50, 75, and 90% of the variance. The analysis based on only five dimensions revealed rough structures of the population whereas the highest numbers of dimensions revealed thinner structures, but some could be due to hazard. The groups of experimental situations resulting from the four analyses were often similar, and sometimes, one group in one analysis was split into two in another. We finally kept the groups obtained with the eight-dimension analysis. The “type of production” (market gardening or arable crops), “production label” (conventional or organic), the “farmer’s name,” and the “type of practices” experimented (Fig. 2) were used as supplementary variables to detect if some groups were specific to one or several modalities of these variables, which is an important information to determine the degree of genericity of the framework.

Fig. 2

Number of experimental situations per types of agroecological practices experimented by farmers (N = 181 experimental situations). Mixed practices mean that two types or more agroecological practices were combined and experimented during the same experimental situation.

Results and discussion

The results are firstly a methodological contribution, an analytical framework that will enable characterization of experimental situations elsewhere, and, secondly, the analysis of the diversity of experimental situations in our case study.

An analytical framework to study experimental situations

The framework is composed of 13 variables (placed between quotes) organized in the three previously identified stages: Design, Management and Evaluation (Table 1).

Table 1 The framework for analyzing an experimental situation

Design stage

During this stage, farmers designed their experimental situations and chose what to test and how to organize the experiment? Two variables were related to the farmers’ goals from the conceptual model: the “objective” of the experimental situation and the “link to previous experimental situations.” Three variables were related to the planned experimental design: the degree of “novelty,” the “spatial scale,” and the existence of “simultaneous comparisons.” The variable “objective” represented what the farmers wished to modify in their farming production process and was composed of three modalities. In the first two, the farmers were aiming at direct improvement of their commercial crops or of the agroecosystem (biological regulations and ecological processes). The third modality was the case in which experimental situations were designed to solve a specific agronomic problem. The variable “link to previous experimental situations” described the relationship between the practice being tested and the practices in the previous experimental situations, with six modalities (the last being absence of linkage): the practice was set up to improve the agroecological processes targeted, economic or work performance, or feasibility compared to the previous experimental situation. One modality consisted in transposing a practice to other crops or to other parts of the farm, targeting similar effects. Another consisted in strictly repeating previous experiments. The variable “novelty” informed on the degree to which the practice being tested was new compared to the practices already implemented by the farmer and comprised three modalities. In the first one, the farmers tested a similar practice, meaning they already knew how to set up the practice and what effect was targeted. In the second one, the farmers tested a new practice based on a previously known logic (e.g., a regulation process through the release of previously tested biocontrol agents). In the third one, the farmers tested a new logic, thereby leading to more uncertainties or more elements to be taken into account, because they were not familiar with either the effects targeted or with the practices that would enable them to reach their target. The variable “spatial scale” informed on the number of plots in which the practices were being tested. This variable had four modalities. In the lowest one, the farmers tested practices in less than one plot (e.g., in strips within a field), thus minimizing risks in the case of failure. In the highest one, the farmers experimented on the whole acreage and the experimental situation was thus close to real farming conditions, including in terms of work organization and economic results. Finally, the last variable called “simultaneous comparison” informed on whether or not the farmers simultaneously compared several technical modalities. Some compared several modalities the same year, in the same plot or in several plots. Others compared the modalities with a reference situation, either planned or identified by opportunity because it made sense.

Management stage

During this stage, the farmers implemented the practices they planned to test, modified them if necessary, and collected information. The stage comprised five variables. The variable “progress” reflected the actions farmers added or transformed during the experimental situation to deal with unpredictable events. Three modalities were possible. Farmers adjusted the experiment while it was underway. Or they stopped it because of bad results. Or they did not change anything. Four variables described the information collected by the farmers and used to make their judgments and possibly to discuss their results. These variables were binary depending on whether the information was collected or not: “information on the setup” referred to the farmers’ technical capacity to set up their experimental situation. “Information on crops” referred to either the development or the yield of commercial crops. “Information on the agroecosystem” gathered all information on the biophysical and ecological aspects of the agroecosystem apart from commercial crops. “Information on work” referred to the workload or to the ergonomics of the practices.

Evaluation stage

This stage described the way farmers evaluate the results of their experimental situations and had three variables. Two represented how farmers judge their results. We distinguished achievement of the goal, i.e., results that were in line with the farmers’ expectations (defined with the variables “objective” and “link to previous experimental situations” in the Design stage) and “discoveries,” which concerned new and unexpected observations made while the experiment was underway. Each variable had three modalities. Splitting judgments between the two variables helped distinguish what farmers could validate/invalidate from what they had recently had to deal with. The variable “discussion of the result” described if and how farmers discussed their results. They could put forward an ex-post hypothesis to explain the success or failure of the experimental situation, or find an explanation for the technical failure. They could identify confounding effects when they considered that results might be due to other factors than those being tested. They could also relativize the results of the experimental situation because particular cropping conditions interacted with the practice being tested. Alternatively, the farmers might not discuss results at all.

The analytical framework we built proved to be efficient in coding the farmers’ discourses in the case study (Table 1, two last columns). Concerning the Design stage, most experimental situations aimed at improving the functioning of the agroecosystem in relation with previous experimental situations, with a limited degree of novelty, on one or two plots, and without comparing several modalities. Concerning the Management stage, the experimental situations were rarely adjusted while they were underway, and most information collected concerned commercial crops and the functioning of the agroecosystem. During the Evaluation stage, most results of the experimental situations achieved the farmers’ goals; few discoveries were made by the farmers; results were mainly not discussed.

These results show that the proposed framework makes it possible to describe experimental situations in detail based on textual analysis of farmers’ discourses. The three stages and 13 variables are sufficiently generic because it was possible to describe the wide range of the 181 experimental situations in the case study concerning 17 farmers and involving 10 types of agroecological practices and two production systems (market gardening and arable crops). The framework is an original contribution to the issue of farmers’ experiments because it focuses on the whole process from motives to evaluation of practices. The framework is focused on farmers’ knowledge production, and its use will help foster exchanges between farmers and agronomists on agroecology. Very few frameworks on experiments are available in the literature. In most cases, each scientist identifies only few key points. For instance, Quiroz (1999) studied farmers’ experiments in terms of the number and size of trial repetitions, location (and size) of trials, religion and other beliefs, and evaluation of experiments (agronomic aspects, yield, and economic benefits). In her review, Saad (2002) focused on steps, use of controls and replications, location, scale, the number of factors, and observation. Elements are described individually but they lack consistency between them. On the other hand, Kummer (2011) built a framework to identify several types of farmers’ experiments. This framework is composed of five variables: motives (personal, economic, problem solving or none), methodological approach, information sources, personal identification, and duration. The first difference between Kummer’s framework and ours is that the author considered information sources and personal identification as part of the experimental activity, which we did not. Conversely, the author did not include evaluation variables that we considered to be part of an experimental activity. Moreover, variables are qualified by five levels of intensity but detailed criteria used to assign a particular level are missing (for instance, how are qualified farmers’ methodological approaches qualified as highly elaborate or less elaborate?). Our framework focuses more on the consistency between the three stages of the experimental situations: How is the experimental situation set up depending on the farmer’s goals? Are the farmer’s conclusions consistent with what was observed during the experiment? How are results evaluated with respect to the stated goals? Therefore, we think that our framework is more useful to help farmers reflect on and design their experimental situations.

Another originality of our framework is that it makes it possible to deepen the analysis of the short-term dimension of the experimentation. Very few studies focus on this time scale, and we found no similar framework in the literature. Even if our framework is focused on short-term experimentation, it remains compatible with the long-term process thanks to the variable “link to previous experimental situations.” Yet a long-term experimental process was observed in our sample (134 out of the 181 experimental situations studied were linked with previous ones) and as was the case in Mak (2001), Quiroz (1999), Saad (2002), and Kummer (2011). The present study shows that the past is important to understand farmers’ experimentation. It makes it possible to position an experimental situation in terms of motives (what is the main objective of this practice? what is being changed compared to previous practices?) and novelty. Futures studies will be able to take changes in the experimental situations created by the farmers over the years into account using our framework.

While it is widely accepted that agroecology requires being adaptable, it is surprising that in our case study, practices were so rarely adjusted while the experiments were underway. One possible reason is that our framework does not take into account the time farmers needed to conceive their experimental situations: Sometimes, the experiment was designed a relatively short time before the first action was taken. Adaptation could also occur on a multi-year time scale, between experimental situations. For these different reasons, future multi-year analyses of farmers’ experimentation are needed to better understand the dynamics of changes and learning in agroecology.

One limitation of our work is that the method relies on farmers’ recollections, as most experiments were conducted in the past. In experiments that were conducted in the distant past, the farmers surveyed had difficulty remembering some details, particularly because they very rarely keep written records of experiments. This is why it could be difficult to inform variables or modalities as the farmers’ starting goals, or on-going adaptations of the experimental situation. What is more, we only focused on the broad types of information collected during the experiment (4 variables), not on its precise nature and the way farmers collected it, although this would be interesting information to collect. Indeed, Toffolini et al. (2016) pointed to the need to develop new indicators to help farmers engage in the agroecological redesign of their farming systems, including through experimentation. As the use of such indicators is difficult to identify in past experimental situations, they could be discovered more easily with complementary methods, for example, regular monitoring of on-going experimental situations (Baars 2011).

The diversity of experimental situations in the case study

This section describes the diversity of the 181 experimental situations in 10 groups (Table 2). All the descriptive variables of the framework contributed significantly to the grouping. The five most significant variables were “link to previous experimental situations,” “simultaneous comparison” and “novelty” in the Design stage, “progress” in the Management stage, and “discussion of the results” in the Evaluation stage. “Information on work” and “information on the setup” were the least significant descriptive variables. Most groups were described by several variables belonging to each of the three stages of the framework, except for groups 3, 6, 8, and 10, which only contained variables from two stages. The main result of the statistical analysis was the identification of four clusters of experiments that differed in their goals, management, and ways of interpreting the results: experiments to improve previously experimented practices (groups 1 to 5), failed experiments that did not reach the farmers’ goals and were often stopped before the end (groups 6–7), breakthrough experiments where the farmer tried a practice that revealed a new logic for the first time (group 8), and finally experiments based on simultaneous comparisons of several modalities (groups 9–10). After describing the four clusters and the corresponding groups based on the significant descriptive variables, in the last section, we analyze whether some groups are specific to a particular practice, farmer, or type of production.

Table 2 Groups of experimental situations identified in the case study

Improvement experiments (groups 1–5)

The first cluster accounts for half the groups and for 113 out of the 181 experimental situations identified. The five groups correspond to a low or medium level of novelty of the practices being tested. In group 1—low novelty practices to improve yield—farmers had already used the practice being tested and their aim was to improve the yield of the commercial crop; they were thus sufficiently confident to implement it at a large scale. Information on commercial crops was collected but not on the rest of the agroecosystem; no change was made during the experiment, and the results were more or less what the farmers expected. The experimental situations did not lead to discoveries. Groups 2 and 3 are characterized by a medium level of novelty. In group 2—modifying a practice to improve work or economic performance—farmers had already tested the agroecological processes they were targeting. This time, they were adjusting the practices to make them easier or cheaper to implement. As a consequence, information was mainly collected on work. These experimental situations were often applied at a large scale. In group 3—extension of a logic to other farm elements—the experimental situations consisted in transposing an agroecological logic that had already been assessed in the previous experimental situations. When doing so, they started to experiment at a small scale because the practices had to be adapted to another part of the cropping system. Group 4—low novelty experiments with good results—and group 5—experimental situations to improve the mastery and understanding of an agroecological practice—consisted in testing a practice they had already tested, this time with a focus on the agroecosystem and not on the commercial crops. They consisted in two levels of mastery: in group 4, the practice was sufficiently assessed so that farmers mainly repeated the previous experiment without adjusting it during the experiment; they often reached the goals targeted and did not discuss the results. On the contrary, in group 5, the aim was to improve the feasibility of a practice that had already been tested but was not completely satisfactory. Farmers still had to adapt practices during the experiment and discussed the results in order to better reach their goals. These different groups of experimental situations could be part of larger processes that Lyon (1996) called “learning during action” and Millar (1994) “adaptive experiments,” based on small changes without taking large risks. Group 3—extension of a logic to other farm elements—is quite original. It reveals that farmers may experiment in order to adapt and transpose some agroecological practices to other parts of their cropping system.

Failed experiments (groups 6–7)

The second cluster, although a minority (19 experimental situations out of 181) gathers two groups, group 6—discovering technical limits—and group 7—interruption by discovering agroecological limits—, where the practices being tested were judged negatively and the experiments were frequently stopped in the middle. The experimental situations of these groups led to far more discoveries and discussion of the results, either technical and based on how to set up practices (group 6) or for agroecological and agronomical reasons (group 7), where ex-post hypotheses were formulated. These two groups are close to what Chantre (2011) called “trial and error” learning processes. Farmers experiment in a relatively short time on their own, and most of these experiments did not achieve the goals set by the farmers.

Breakthrough experiment

The third cluster corresponds to group 8 alone (29 experimental situations out of 181). This group is made up of experimental situations in which new agroecological logics are being tested for the first time. These experimental situations represent the moment a farmer decides to try a new practice (that involves a new logic) he/she discovered off the farm or imagined and receives evidence of its interest. It is surprising that most of the results of these experiments were not discussed. Two hypotheses can be proposed to explain this phenomenon. Either the expectations from a very new practice are fuzzy, as noted by Toffolini (2016), who showed that farmers trying out practices based on a new agronomic principle first wanted to know if they were consistent rather than aiming at precisely quantifying the processes. Or, a breakthrough experiment involves a practice admittedly unknown to the farmer concerned, but already proven by numerous other farmers. This group is original and could not be related to other types of farmers’ experiments in the literature.

Comparison experiments (groups 9–10)

The last cluster is also a minority (20 out of 181 experimental situations), but group 9 and group 10 are the only groups in which several modalities were compared or compared with a reference in the same plot. In group 9—comparison of several modalities to improve agroecological processes that affect the development of commercial crops—several modalities were implemented in the same plot to simultaneously compare agroecological process of practices targeting a very precise goal: to identify the best modality for developing commercial crops. The practices experimented had not been the subject of previous experimentation but used a logic that resembled one the farmer had used previously. For example, farmer 3 tested no-till maize for the second time. This time, he controlled ray grass weeds before sowing and also sowed faba as a previous cover crop. He compared two modalities: strip till in one plot and direct sowing in the other. Compared to group 9, group 10—small scale experiments with a reference—occurs in a broader range of cases. For example, farmer 1 tested no till on a small strip in a vegetable bed and tried mulching with ramial chipped wood for the first time. He compared changes in the soil structure and in biological life with that in other parts of the vegetable beds that he tilled normally. Most often, the reference is not specifically designed as such but is the farmer’s current practice. These two groups of experimental situations are close to the “accurate experiments” of Kummer (2011), conventional approaches of Lyon (1996), because they both involve simultaneous comparison and small scale.

As a first discussion of the 10 groups, it should be noted that all the variables of the framework were significant for one group or another. Among the five variables that contributed the most to grouping, three belonged to the Design stage, one to Management, and one to Evaluation. In particular, the variables “link to previous experimental situations” and “novelty” were useful to interpret how farmers designed the experiments in several groups, and the variables “progress” and “discussion of the results” were useful to interpret how farmers dealt with surprises or problems in terms of actions and analysis. These variables should therefore be the priority in future attempts to investigate experimental situations in other production systems. Conversely, the variable “objective” frequently did not discriminate among groups. This may be due to the fact the three modalities (improving the yield of a commercial crop, improving agroecosystem functioning, or solving an agronomic problem) are too broad. For the statistical analysis, we chose to distinguish farmers’ positive and negative judgments of the results, which enabled us to identify failed experiments (groups 6 and 7) although their number was low. A bigger sample would perhaps have revealed a wider range of experimental situations that can lead to negative results. Finally, it will now be necessary to assess the robustness of the 10 groups in comparison with other data sets and, in particular, to check if the two original groups in this study (groups 3 and 8) are identified again.

Are some groups linked to the supplementary variables? The “type of practices” and “type of production” were the most significant, and 7 groups out of 10 were significantly correlated with at least one of their modalities. “Farmer’s name” was also significant; six groups were respectively characterized by six different farmers. However, no group was 100% characterized by one farmer and no farmer restricted his/her experimental situations to one particular group. Overall, none of the 10 groups was 100% characterized by one particular modality of the supplementary variables. Our framework is therefore sufficiently generic to cover a large diversity of contexts but specific contexts may lead to a greater proportion of particular groups of experimental situations. For example fertilization was frequently tested in group 1—low novelty practices to improve yield—which can be explained by the fact that it is a classical question for farmers that are closely linked to yields of commercial crops. Reduced tillage practices were common in group 10—small-scale experimental situations with a control—probably because implementing two or more modalities of soil tillage in one plot is feasible for farmers as the modalities will not spill over and interact, contrary to releasing biocontrol agents, for example. Ecological infrastructure practices were the main aspects tested in group 2—practice modification to improve work or economic performance. Concerning the type of production, arable crops were significant in three groups: group 1 (low novelty practices to improve yield) as fertilization is a priority issue in arable crops, group 7 (interruption due to discovering agroecological limits) as testing a cover crop or reduced tillage practices provide clear information on whether it is worth continuing or not, and group 9 (comparison of modalities to improve agroecological process impacting commercial crop development). Market gardening was significant in only one group (group 2—modifying a practice to improve work or economic performance). Because vegetable production is complex and time-consuming, market gardeners frequently experiment how to improve work or save money. Farmer 17 was significant in group 2—practice adjustment to improve work or economic performance—because he frequently tested permanent garden beds and focused on ways to improve work. Farmer 8 was also a market gardener and was significant for group 3—extension of a logic to other farm element. Based on good results with ladybugs and parasitoid wasps, he tested the transfer of different endemic auxiliaries to various crops and pests. Therefore, these elements of the context need to be taken into account in future studies, either for different types of production, for other agroecological practices, or farmers’ contexts. On the contrary, the other supplementary variables “production label” (separating organic and conventional farming) was not significant.

We now discuss the results according to the scale of experiments and the nature of comparisons. First of all, more than 75% of the experimental situations were conducted at a small scale (one or two plots or greenhouses) or less. Our results suggest that the size of the experimental plots is correlated with the risk of failure as perceived by the farmer. Large-scale experimental situations were mostly used in groups 1 and 2, where the practices being tested have a low or medium level of novelty and, probably, a limited risk of failure. On the contrary, failed experimental situations mostly occurred at small scale, probably because farmers were fully aware that the practice they were testing was risky. One questioning point is that breakthrough experiments (group 8) were not necessarily carried out on small- or very small-scale plots. One possible explanation, based on Kummer’s results (2011), is that the farmers implemented such very innovative practices on a large scale when there was no alternative technique to solve an agronomic problem or because they already had sufficient confidence in the practice. The second key dimension deals with comparisons. Very few experimental situations (13%) involved a simultaneous comparison, even with a reference, whereas for scientists, it is embedded in the experimental activity. Although this figure is consistent with what Kummer (2011) and Sumberg and Okali (1997) found (15 and 39%, respectively), it does not mean that farmers do not use a reference, but more likely that their references are based on memories (for instance, what does a field without a cover crop look like in late summer). Yet, farmers who compared different modalities were rare. We explain this by the fact that farmers are usually looking for “best bet” practices or combination of practices. And if one option seems promising, to save time and reduce risks, they will only try that one. For instance, in a participatory experiment, for ease of implementation and evaluation, most farmers chose only one option (associating maize and velvet bean) and tested it on single plots with no replication (Buckles and Perales 1999). To sum up, it seems that farmers, at least those with experience, usually know how to limit risks when they are testing a new practice. They balance the risks as a function of the novelty of a practice, its spatial scale, and, possibly, information seeking. They also limit the time dedicated to reflection and organization. In most cases, they test one “best bet” practice on one plot. Risky situations may still arise when farmers experiment very different practices at a large scale. To avoid such risks, they should either first conduct some experiment at a small scale or make sure they are well informed before starting. One major aid for farmers who wish to experiment would be to look for more simultaneous comparisons. In particular, we believe that farmers who experiment should receive more support in including two or more modalities, which would ensure the outcomes are more efficient. However, one condition first needs to be fulfilled. Several equally worthwhile modalities need to be identified; otherwise, if one modality appears to be much better than the others, the farmers will very likely test only that one. In this context, one way to help the farmers would be to evaluate with them during the Design stage, which modalities could be considered, and, if need be, to encourage them to set them up (choice of the plot, regular observations with the farmer, and visits to other farmers).

Outlook for development and research

First, individual experimental situations should be encouraged because they have the advantage of suiting a particular farmer’s motives and need for change. This kind of approach is close to experiential learning as defined by Baars (2011), which enables the production of situated knowledge in collaboration with farmers. Our work enabled us to access farmers’ ongoing and past experimental situations, but it was sometimes difficult for farmers to remember all of them well, and they were rather tacit. The proposed framework appeared to be efficient in helping them be more explicit. A simple action to help farmers would be to encourage them to record their experimental situations: taking notes on their experimental situations would favor their reflexivity as well as help avoid important information being forgotten. In return, farmers should be helped to conceive new experimental situations. The task would be to reflect on previous experimental situations to clarify the new design stage: explain their motives, discuss possible risks, and think about appropriate modalities to try.

Second, collective experiments on agroecological systems are expanding (Navarrete et al. 2018). By studying farmers’ learning processes, Chantre et al. (2014) underlined the importance of interaction with others (farmer development groups, peer farmers, an advisor) in acquiring new knowledge, including through experimentation. Experimental situations may also help other farmers. As every farmer will explore practices differently, discussing experimental situations can help identify complementarities or summarize the knowledge produced. When the interest of a topic grows in a group of farmers, a collective experiment could be set up. Different modalities should be chosen among members and tested, as proposed by Buckles and Perales (1999). For that purpose, using the framework with a group of farmers could help collect and organize information on experiments, which could then be used to warn new farmers about possible “unpleasant surprises” when certain practices are used or to enable them to benefit from what others already discovered to save time and for cognitive economy. Collecting and capitalizing on experimental situations is an opportunity to empower farmers in their knowledge production. The interest of farmers’ experimental situations is that the emphasis is not on the knowledge content itself but rather on the way it is produced (Briggs 2013). Experimental situations ensure that the context of situated knowledge production is there to be discussed, for instance in meetings between farmers. This helps question the context (the main objective of the experimental situation, links with previously tested practices, and the degree of novelty) and the results (farmers’ judgments and discussion). Our framework could also be used as a support tool for discussion on a field trip during which the farmers present their own experiments. What is he/she referring to? To a breakthrough experimental situation on a single plot or to a practice being tested for the 5th time, aimed at improving agroecological processes?

Finally, are the farmers’ and the scientists’ experimental processes complementary? Toffolini et al. (2017) refer to farmers gaining insight into their actions when they acquire fundamental knowledge, the farmers’ experimental situations could be discussed with agronomists to explain some of the results achieved on the farm in the light of agronomical theory or concepts, thereby sharing scientific and fundamental knowledge with farmers. Following Baars (2011), Hoffman et al (2007), and Sumberg and Okali (1997), farmers’ experimental situations can be considered as a complementary activity to scientific experiments to produce knowledge. Capitalizing on farmers’ experimental situations in agroecology could be a way to integrate farmers’ empirical knowledge in scientific innovation and knowledge, one of the five ways proposed by Doré et al. (2011) to deepen research on ecological intensification.

Conclusion

In this article, we have introduced the concept of an “experimental situation” to better understand farmers’ experiments and to characterize how farmers use experiments when they are engaged in change toward agroecology. The framework is an original way to characterize how farmers design and manage their experiments and how they evaluate the results, in detail. It is sufficiently generic to describe experimental situations for both arable crops and market gardening. The classification analysis enabled the identification of four clusters and 10 groups of experimental situations. These groups illustrate the wide range of experimental activities undertaken by farmers that vary in the goals pursued, the way the experiments are designed, and in their outputs, hence each contributing differently to learning agroecological practices. One key outcome of the study is that even if the farmers’ choices of experiments are much more implicit than those of scientists and differ considerably from one group to another; there is a high degree of consistency between the choices concerning goals, methods, spatial scale, etc. The framework could be used to help farmers increase their reflexivity on their own experiments as well as to become more perceptive when they visit other farmers’ experimental situations. This framework could be adapted for other purposes involving farmers’ expertise (a reflexivity tool for farmers on their own, as a support for groups of farmers engaged in agroecological development, participatory design of farming systems). To pursue research into experimental situations, the framework should be tested so that it can be refined or adapted in other areas and for other productions. The framework could be also completed with other methods to monitor ongoing experimental situations more precisely and, in particular, to identify the indicators used by farmers to make their judgements. Elements that are difficult to retrieve when based on memories, like hypotheses or methods of collecting information, could be identified in this way. Finally, our results show that, although here scrutinized in the short term, experimental activity is a long-term process. Working at multi-year and farmer scales would make it possible to understand how experimental situations evolve over time, how knowledge is produced and refined by farmers, and how the practices that are tested are adapted and finally adopted, which is a key challenge for the extensive development of agroecology.