First stage: a way to devise new concepts for tools and to work on the flexibility required in the tools
Characteristics of the use situations
As explained in the preceding section, we used three complementary points of view to characterize the diversity of situations in which the tools may help to solve a problem, and the uses of existing tools in these situations. Table 1 shows these three points of view within each project. The study of the cognitive point of view shows significant differences between designers' and users' representations of the problem. For instance, in the case of sclerotinia, whereas the agronomists are interested in the frequency of occurrence of a severe attack (justifying a treatment), the farmers (‘users’) are mostly interested in avoiding situations resulting in such an attack. From a functional point of view, there is a diversity of aims and ways of producing information: for instance, in the case of cultivar assessment, the users distinguished different aims (registering, breeding, developing, range designing, indexing and scoring technological suitability), corresponding to different types of trial networks (see Lecomte et al. 2010). Finally, the operational point of view highlights the deep constraints to action experienced by each actor. In the case of sclerotinia, the farmers have to treat before the appearance of the disease, which means that farmers and advisors have little time for gathering information. In the case of cultivar assessment, the actors have little time to collect useful data and to process the data from the trials.
The diagnosis of uses: a way to develop new concepts for tools
The diagnosis of uses can help in devising new concepts for tools, by revealing ways of using existing tools, not envisaged in their design, or by identifying problems peculiar to the users which will modify the original concept of the tool under design. Thus, in the case of sclerotinia, farmers and advisors often combine the available tools to evaluate the risk of infection over the course of time, by means of a rather sophisticated arrangement described by Cerf and Meynard (2006). These authors have developed the concept of Information and Advice Systems (IAS) to account for this arrangement between tools, advisors and farmers. Likewise, the difference in the perception of the risk, as well as the designers' wish to favour low-input agriculture, led some designers involved in the sclerotinia project to consider the possibility of coupling an insurance device to the evaluation of the level of infection by the petal kit, and thus to develop a new concept for a tool to limit the fungicide treatments.
In the second case study, discussions about the functionality of the tool brought to light a major challenge for those involved in the assessment of new cultivars: the cost of the networks and the advantage of reducing the number of trials. This appeared to be possible once more information could be extracted from each of the trials. Thus the tool was to facilitate the optimization of the networks. The diagnosis of limiting factors, by means of the tool, was used to identify the trial sites which were redundant because they had the same limiting factors and thus resulted in the same cultivar rankings.
Integrating the diversity of uses: what flexibility to introduce into the tool?
Acknowledging diversity among the cognitive, functional and operational points of view is important to foster discussions among the participants on the agricultural paradigm and on the use and user models on which the tool is based. This raises questions on the flexibility required in the tool. ‘Flexible’ here means that the tool produces information which will be relevant and reliable for a range of decision contexts and operating methods.
Hence, in the case of sclerotinia, in order to discuss the effect of different agricultural paradigms on the threshold for deciding to spray, we borrowed and adapted some methods from the world of medicine (see Makowski et al. 2005). The idea was to allow users to evaluate more efficiently the quality of different indicators (the one provided by the petal kit, the one which combines the decision grid with the petal kit, etc.) regarding the effects of the disease on the yield, according to the decision rule for fungicide application. The choice of the decision rule also determined how to treat the risks of type 1 and type 2 errors in the interpretation of an indicator of the occurrence of the disease in the field (the incidence of infected flowers). The designers first and foremost wanted the tool to allow for a reduction in the frequency of treatments (accordingly, they tried to give priority to reducing type 2 errors ‘to treat even if in the end it turns out to be unnecessary’), whereas the farmers were particularly anxious not to risk having the disease (thus giving priority to reducing type 1 errors, i.e. ‘not treating even if it would have been necessary to do so’). Being flexible here means getting round this difference while nevertheless keeping the key idea of avoiding unnecessary treatments.
In the case of the assessment of winter wheat cultivars, flexibility first had to address the operational diversity among future users. For example, the future users did not all collect the same data on the trials: some collected weather data and data on developmental stages, while others did not. Some data might moreover be incomplete, or the precision might not always be the same because of the organizational constraints of those doing the assessment. For example, for a growth stage record, some tried to locate the precise day of appearance of the stage whereas others went past on a given day and noted the stage reached for each cultivar. The designers thus envisaged the possibility of making up for the absence of measured data on the trial networks by introducing qualitative data obtained while the assessors were visiting the trials. Flexibility also had to address functional and cognitive diversity among users and designers. The range of objectives identified (Lecomte et al. 2010) questioned the outputs which might be presented to the users, and their precision (functional diversity). In fact, the precision required was not the same if the aim was to exclude the worst genotypes or to rank all the tested genotypes correctly. Relevant and significant information differed among the users, and the tool should have allowed this diversity to be taken into account. It also had to take on board the fact that most users tried to identify cultivars which were stable over a range of environments (which means giving priority to economic considerations), even though the tool was originally designed to identify cultivars that were suited to certain environments with well-known characteristics. This discrepancy on the way to acknowledge GIE should therefore be discussed.
Second stage: a prototype mediating dialogue between designers and users
Dialogue around the prototype took place in different ways. First, as Schön (1983) also found, while using the prototype the potential users dialogued with their working situations. Such dialogue enabled them to formulate claims which they would then address to the designers. Such claims were based on their experiencing the tool as well as its ability to overcome some of the problems they wanted to solve. This experience was then used by the designers to question the model, the tool and the user's decision making and practices during the debriefing sessions. Dialogue was thus established not only amongst the designers and the users (Hatchuel 1996) but also more generally between humans, artefacts and situations. Debriefing sessions create learning environments in which dialogue is supported by the way the prototype responds to the use made of it in work-like situations: what the tool can or cannot do for the users, the capabilities which the users ascribe to it, the difficulties encountered in understanding what the output means, and finally the difficulties in obtaining the input data for the tool. During the debriefings, users related the problems they had encountered and the solutions they had devised to solve them, when possible. They also spoke about their discoveries, and the way they started to change their views on how to go about decision making (the information on which they grounded their decisions, the usefulness of new information). It is therefore important for the designers to accept the fact that the knowledge introduced into the tool, the capabilities which they envisage, and the ways of using the tool will be reviewed, and that a new version of the prototype should be developed.
Debriefings are opportunities to capture discrepancies in the way participants build a representation of the problem (collect and interpret information) and their appraisal of the uncertainty they face in solving this problem. In the case of cultivar assessment, for instance, the users pointed out problems in the choice of indicators for limiting factors, in the precision needed for the input data, in the factors that the tool sorts out as limiting yield in the network, in the intensity they attributed to these factors, etc. In doing so, they discussed the assumptions made in the agronomic model underlying the tool. This was also an opportunity for the designers to point out the limitations of working on a dataset which is too homogeneous (variability is needed in the intensity of the limiting factors on a network scale if they are to be located).
The exploitation of the debriefings then enabled the participants to identify what should, or could, be initiated as regards both the tool and the users' practices. This was done by eliciting the cognitive, functional and operational points of view to which the discussions during the debriefing sessions refer.
For sclerotinia and from a functional point of view, the debriefing showed that the challenges of using tools in a local network, for combining the results in a range of agronomic situations, was not yet clear for the potential users, although they usually knew ‘how it works’ (Cerf and Meynard 2006). But anticipating the way such a network can operate remained a difficult task. From a cognitive point of view, the debriefing also allowed us to point out the lack of reliable knowledge on the protocols for testing the tool on the scale of a small region. Likewise, it was an opportunity for lending support to design concepts which had already emerged during the diagnosis of uses (insurance scheme, Information and Advice System (IAS)), and led to the identification of a new design concept, i.e. ‘an indicator for better timing of treatment in relation to late infections’. It enabled the designers to further elaborate on the design concepts which had been experienced during the prototype use (for example redesign the kit use protocol). From the operational point of view, proposals emerged to facilitate the data collection (how to place the petals on the culture medium for example) or to interpret the results of the kit (photos to create a reference for classifying the colours observed in the Petri dishes). On the other hand, little emerged about what might change as regards the farming practices. In fact, the farmers lacked confidence in the results of the kit in relation to the risk they thought they were taking when skipping a treatment. As a result, they did not commit themselves as to whether they would be willing to change their farming practices, and discussions mainly focused on how risk was assessed differently among the participants.
For the cultivar testing computer program and from a cognitive point of view, the debriefing indicated numerous problems calling for knowledge development (see Table 2) but only two of them were taken into account in a new version of the tool (for details see Prost and Jeuffroy 2007; Prost et al. 2008). Indeed, compromises had to be found here between improving the model underlying the tool and allowing the users to benefit from it while being aware of its possible shortcomings. From a functional point of view, the users confirmed the value of the diagnostic stage as such, as an opportunity to optimize the network and not just a stepping stone to testing the tolerance of cultivars to limiting factors. From an operational point of view, changes were suggested by the potential users, such as modifying their experimental protocols to assess certain data not currently measured, or improving their expertise (through training or by hiring specialists in statistics and modelling). Moreover, they asked for a change in the user interface so that they could go back over the calculations of indicators of the intensity of limiting factors and the thresholds defined, in order to assess the limitations expressed by these indicators.
With hindsight, the collective nature of the debriefing appeared to have several advantages First, the users definitely felt more legitimate in questioning the designers and discussing the knowledge and hypotheses embedded in the tool. Second, in discovering what others have done or intend to do, the users questioned much more critically their own way of using the tool. Finally, the questions asked helped users to explore what others have done. The debriefing was also a way of discussing what should be taken into account for the prototype to evolve or what should be done to change the users' own situations.
Changes in the tools and current state
Table 2 summarizes the changes made to the tools after the analysis of the users' work situations and after the users had used prototypes of the tools. It also shows how this process enabled the researchers to identify the need to extend scientific knowledge. Note that some choices made to include the users' proposals in a new version of the tool or in the users' working environment were to some extent opportunistic taking account of the time scale of the project, the cost of achieving a solution, our ability to mobilize other researchers skilled in the new fields identified, etc. Nevertheless, we kept track of the users' suggestions and of the scientific questions that the debriefings raised.
To end this section, we briefly report on the way these decision support tools are now in use. In 2008, in the case of sclerotinia, an operational kit for use in a network (vigicolza) was developed by CETIOM (Penaud and Duroueix 2009). It is now used extensively throughout France, by advisors mainly. In the case of Diagvar, a new prototype integrating modifications to the input data, new statistical methods and a new interface, was implemented in 2009. In 2010, it was tested on existing databases by the potential users in interaction with the designers, and in 2011 by the potential users alone (the designers only interacted afterwards, to obtain feedback). It is expected to be applied in ‘real-work’ situations on the 2012 harvest. The four plant breeding firms are still involved, as well as the agency in charge of cultivar registration and INRA's breeders. Convinced by the results of the first tests of Diagvar on winter wheat, they have begun its adaptation to other species (sugar beet, potatoes, peas and maize). The software has been developed using an open-source language so that interested parties can take over the formalisms of the tool and adapt them to their own situation and experimental databases.