Contents

1 Introduction

There is an urgent need to improve the sustainability of agrifood systems globally, i.e., considering agricultural and food systems together. Agrifood systems both significantly contribute to and are affected by environmental issues, including climate change, biodiversity loss, and the need to preserve soil and water quality (IPCC 2019). We need to produce more food (almost 795 million people go hungry and about 2 billion people are malnourished around the world), but about 30% of global food production is lost or wasted. Our diets are nutritionally inadequate, with 30% of the global adult population overweight or obese. According to projections, the world food supply will need to increase by 70%, to feed almost 10 billion people by 2050. A profound transformation of agrifood systems towards greater sustainability is required (Tilman and Clark 2015).

However, a first difficulty is the lack of a clear definition of what is (and is not) sustainable and, as a consequence, how to assess what is (and is not) sustainable as the sustainability of a system can change continuously and there is no threshold value at which one can speak of a “sustainable system”. Sustainability is generally acknowledged as the triple-bottom line (Elkington 1997) of balancing the three dimensions of environment, society, and economy, but these three dimensions are often considered insufficient, and finally there are many different sustainability assessment models available that aim at answering completely different questions because they are posed by different actors (upstream agriculture, food processing, distribution, public policies, etc.) and therefore at different scales. For instance, Chaudary et al. (2018) have made a global-scale analysis quantifying the status of national food system performance of 156 countries, defining 7 sustainability domains as follows: nutrition, environment, food affordability and availability, sociocultural well-being, resilience, food safety, and waste. There is not an established standardized methodology to assess sustainability of food systems, but indicator-based approaches are frequently used in the scientific community (Lampridi et al. 2019). SAFA (Sustainability Assessment of Food and Agriculture systems) guidelines provided a framework for sustainability assessment in companies, organizations, and other stakeholders that participate in agrifood value chains (FAO 2014). Environmental integrity, social well-being, economic resilience, and good governance have been described through 21 themes divided into 58 sub-themes. In total, 116 default performance indicators covering all themes were suggested to facilitate measuring progress towards sustainability. Gazan et al. (2018) compiled 279 indicators which allow the simultaneous assessment of the economic, social, environment, and health dimensions of diet sustainability. Moragues-Faus and Marceau (2019) reviewed 422 indicators to assess food systems performance in UK cities, covering governance, health, economic, and environment dimensions. The definition of the relevant criteria and indicators to represent sustainability thus depend on context and objectives of each situation. There is also a real difficulty to articulate global and local visions of sustainability: there is a low applicability of global conceptualizations (e.g., triple-bottom line, UN Goals, FAO recommendations) to local decision-making communities and, at the same time, the diversity of approaches applied at the local level prevents aggregation of results and measurement of global progress (Carlsson et al. 2017). To overcome this difficulty, Carlsson et al. (2017) proposed a framework for assessing food sustainability which accommodates local-level measurement in the context of broader national- and global-scale measures, based on a collective identification of key indicators for tracking progress towards success. Associating stakeholders to sustainability assessment is an increasing trend, because they can help to determine objectives, criteria, and indicators but also to assess modalities and the design of scenarios (De Luca et al. 2017). Stakeholder can be involved in two main ways: (i) by considering their criteria and the associated indicators, which may differ in nature or order of priority between stakeholders, and (ii) by considering their opinions when comparing results or the ranking of scenarios. This participation will ensure the implementation of the method and the acceptability of results (Triste et al. 2014). However, the participatory approaches are not extensively implemented in assessment methods (Gésan-Guiziou et al. 2020). Recently, Chopin et al. (2021) reviewed 117 tools used for sustainability assessment in agriculture. They analyzed that active involvement of stakeholders in the framing of sustainability and design of indicators should be developed to achieve reliable and relevant assessment outcomes. Sustainability assessments are complex and tend to be conducted by experts; however, sustainability transformations involve changes in a range of practices and therefore require a diversity of actors to participate. Hence, it is important to find the right moment to involve stakeholders in the process and to define the importance each of them should have in decision-making.

Concerning the changes necessary for the transition of food systems towards greater sustainability, one of the levers to achieve this goal is innovation (El Bilali 2019; Yannou-Le Bris et al. 2020). Innovations can be technical (such as mixtures of varieties, cropping systems, foods, waste reduction), organizational (on the ways of working between actors), and systemic ones (simultaneous creation of new products, technologies, and services, as well as their markets and uses).

According to the above statements, innovation design must comply with an increasing number of environmental and social criteria, in addition to classic technical and economic ones (Martin et al. 2013; Meynard et al. 2012). Two main difficulties then arise. The first one is related to the evolving nature of the artifact (what is produced during the design process): by definition, the initial concept, the following artifacts, and the final innovation can be significantly different. Characterizing the sustainability assessment framework at an early stage helps stakeholders to define the expected value of the innovation to be designed (Doré et al. 2008; Lesur-Dumoulin et al. 2018). As the identity of the artifact emerges, new criteria are incorporated into the assessment framework, reflecting the knowledge gained by stakeholders. In the final stage, sustainability assessment is used as a basis for decision-making process, for arbitrating between imperfect solutions on the basis of enlightened consideration of the various indicators, which, individually, may not necessarily identify the best solution (Pelzer et al. 2012). Hence, sustainability assessment necessarily takes an evolving shape. This does not mean that the objectives of the evaluation should change, but that the enrichment of knowledge that the life of the project allows (Poudelet et al. 2012) should make it possible to enrich, refine, and specify this evaluation as deeply described in Chebaeva et al. (2021). The second difficulty lies in the measurement of sustainability indicators of an artifact which does not exist yet. The degree of performance of an innovation with regard to sustainability is generally assessed with a high level of uncertainty, making it difficult for designers to choose the best option among the different solutions.

Therefore, because sustainability definition and sustainability assessment are complex and open concepts, and because innovation is by nature “to be defined,” we aim to demonstrate in this review that the design of innovations for sustainable agrifood systems requires the following design process: (1) to formalize the place and mode of sustainability assessment, (2) to make use of relevant sustainability criteria and indicators, (3) to reinforce participatory practices, (4) to adapt the assessment to the context of the designed artifact, to guide choices between imperfect solutions (Fig. 1).

Fig. 1
figure 1

Summary of the main proposal of the article.

Full size image

In the first part of this article, we use design sciences concepts to formalize a process evidencing assessment stages, in an appropriate manner, in the design of agricultural and/or food systems. We will compare current status of design processes in agricultural and food systems, which differ in some ways, and propose a common view in agrifood systems. In the second part, we discuss the consequences of considering sustainability assessment in design processes, and propose key points supporting the integration and management of sustainability assessment in the design process for sustainable agrifood systems. These key points are independent of the assessment method itself; thus, we do not review different sustainability assessment methods, what it is already done elsewhere (e.g., Chopin et al. 2021; Cicciù et al. 2022; Coteur et al. 2020).

2 Methods

A seminar organized by IDEAS (Initiative for DEsign in Agrifood Systems) provided an opportunity to discuss the links between sustainability assessment and the innovation design process. This seminar brought together about twenty scientists from different disciplines (food and design sciences, agronomy, ergonomy, etc.). Discussion after the presentations led to the identification of common questions posed by the authors. Two main questions emerged during a writing workshop facilitated by two ergonomists from IDEAS:

  • When should sustainability be assessed in an innovative design process?

  • What features of a sustainability assessment are essential for it to be efficient and useful?

A literature review was performed on these topics in two parallel stages.

Firstly, the authors selected an illustrative example of the development of an innovation derived from living systems. It is sometimes difficult to build a consensus around the novelty of a product, but the authors thought that biofuels was an emblematic example of innovation in the agricultural world and the energy market. A bibliometric study was performed for a period of 59 years (1960–2019). This period covers the emergence of biofuels, from the very beginning to their current status. We searched for articles on biofuels in the ISI Web of Science and Scopus databases with multiple keywords, due to the variable nature of the vocabulary used in this domain: bio(-)fuel, bio(-)ethanol, bio(-)diesel. We also searched for articles coupling these keywords to (1) life cycle assessment or LCA, (2) second generation or 2G, (3) stakeholder(s), and (4) design process. We coupled the biofuel keywords with Life Cycle Assessment or LCA only for sustainability assessment because we tried to determine when sustainability assessments for biofuels first appeared and sustainability assessment was initially limited to environmental impact when it was first introduced into design processes, and LCA was the preferred method for assessing this impact at the time (Bockstaller et al. 2019). The results were analyzed by counting the number of publications per year. Several publications were analyzed in detail to gain a better understanding of the case studied, the design process, and the place, role, and conditions of assessment. Thanks to this work (which will not be presented here), we have determined how sustainability objectives had been taken into account in the design of biofuels.

Secondly, meetings were held (i) to discuss the literature review and (ii) to share experiences of the world of agricultural production and biomass transformation into food and bioproducts. These exchanges were particularly useful for identifying the key dimensions of sustainability assessment to be taken into account in the design of more sustainable agrifood systems. A consensus design process including assessment steps was thus formalized. Finally, recommendations on the key points of sustainability assessment for agrifood systems were established.

3 Assessment formalization in innovation design processes in agrifood systems

Simon (1969) laid the foundations for the representation of design processes by proposing his IDC (Intelligence - Design Alternatives - Choice) model to illustrate the decision-making processes that take place in design projects. This model highlights the need to test and evaluate solutions before deploying them. Simon’s representation of the design decision assigned a new place to the modeling and testing of the solutions designed within the design process. Cooper (1990) pointed out that representations of the design process can highlight the conditions and decision stages following each other in the design project, whether highly innovative or not, thereby increasing the chances of success of the project. An adequate representation of the design process involves, for each step in the innovation process, (i) controlling the types of activities carried out, (ii) defining the knowledge and skills required, (iii) rendering explicit the information to be collected, and (iv) the performance of validations (Vacek 2006).

3.1 Design processes in agricultural and food systems from design science view

In agricultural systems, agronomists have generally been responsible for the formalization of design approaches (Byerlee et al. 1982; Salembier et al. 2018; Vereijken 1997), with the aim of producing scientific knowledge and/or technical solutions of various degrees of readiness for use. Such approaches tend to be rather linear, with occasional iterative steps (Lançon et al. 2007) or loops back to a previous assessment step (Pelzer et al. 2012). Processes often focus on the main expertise of the agronomists involved, in the field or modeling experimental step (Bergez et al. 2010), or in the participatory prototyping step (Berthet et al. 2016). Innovative system design is implemented within R&D projects to address specific objectives (e.g., decreasing pesticide use or greenhouse gas emissions). This occasional use probably explains why there have been very few attempts to formalize design processes and related know-how.

By contrast, in food companies, or the larger ones at least, design processes are part of usual business activities. Companies allocate resources to innovation, which is an objective for them and, therefore, is managed. In a general way, the models proposed are highly linear, and similar to that proposed by Cooper (1990). Booz et al. (1982) promoted a design process with seven successive steps: new product strategy development, idea generation, screening and assessment, business analysis, development, testing, and commercialization. Siriwongwilaichat (2001) produced a very linear design process separated by go/no go milestones:

  • Product strategy development: initial screening, preliminary market assessment, detailed market research, product concept development, financial feasibility study,

  • Product design and process development: prototype design, in-house testing, consumer testing, scaling-up,

  • Product commercialization: trial production, marketing test,

  • Production launch and post-launch: pre-launch business analysis, production start-up, market launch, post-launch operational and financial analysis.

Assessment is thus clearly established, but only for technical-economic and organoleptic feasibility criteria (consumer tests). It is only since about 2010 that the open innovation defined by Chesbrough (2003) has been integrated into the processes of food innovation companies, to feed the experiences, needs, and ideas of users and customers more rapidly into the innovation. In the last decade, the penetration of Design Thinking methods has led to more agile innovation management methods (Serrault 2015) than the standard linear model deployed in much of business. However, for some authors (Kimbell 2011; Pedersen 2020), Design Thinking privileges the designer as the main agent. In this way, the method fails to highlight the context, the network of actors, and the values they hold, in which the problem to be solved is rooted.

3.2 General framework of the design processes in agricultural and food systems

Despite this heterogeneity of origin and use, design processes in the agriculture and food sectors display similarities, which are included in our generic synthesis (Fig. 2). We do not aim to provide a standardized framework but to highlight activities inherent to any innovation project related to agriculture or food. Understanding the roles of these activities, some of the factors that influence their implementation and the results expected from them, can help designers in the development of their projects. In accordance with published findings, the different stages of the process are project definition, diagnosis, definition of objectives, ideation, technical translation, development, and deployment (Cooper 1990; Earle et al. 1999; Riandita et al. 2013). And above all, we propose to clarify the place and role of assessment in the design process. The linearity and standardization suggested by Fig. 2 are relative. Indeed, it is possible to implement the feedback loops with a very variable frequency depending on the project.

Fig. 2
figure 2

Unified innovation design process for agricultural and food designed artifacts. The main activities are indicated in rectangles, intermediate deliverables in ovals, iterative steps by circular arrows, loops back with black arrows, and assessment steps with blue arrows.

Full size image

Project definition and initial diagnosis

The diagnostic steps are very similar in agriculture and in food companies, although they clearly require adaptation to the context in all cases. In agriculture, most design processes start with a broad diagnosis addressing multiple objectives: identifying the surrounding drivers, constraints, and issues (Lançon et al. 2007; Martin et al. 2013), increasing understanding of the stakeholders’ context (Cerf et al. 2012; Le Gal et al. 2011), and/or establishing a hierarchy of objectives and indicators for assessment (Martin et al. 2013; Pelzer et al. 2012; Vereijken 1997). In the design literature for industry (i.e., in our case, the design approaches used by big food companies), the diagnostic step includes the following (Benner 2005; Siriwongwilaichat 2001): identification of preliminary needs, identification of customer requirements, market segments, competitive position, business opportunity, and compliance of the innovative project with the company strategy. In highly innovative projects, this initial step is decisive, and is called the “Fuzzy Front End” or “Front of Innovation” (Bertoluci et al. 2013).

Ideation

The core of the design process often starts with an ideation step, in which the field of possibilities for identifying or assembling potentially suitable solutions matching the design objectives and constraints is explored (Eckert et al. 2014; Martin et al. 2013). The defined objectives guide ideation, and the knowledge about causal relationships acquired during diagnosis can be used to support solution proposals. These aspects are common to both the agriculture and food sectors.

Technical translation and testing

Small-scale experiments are often performed between ideation and large-scale development. The preferred scale depends on the sector, extending from the bench scale for food laboratories to the field scale on agricultural field stations (Siriwongwilaichat 2001). These experiments generate a first set of values for the estimation of technical, environmental, and economic feasibility.

Development and deployment

In agriculture, the development step is sometimes identified as part of the design process (Lançon et al. 2007; Vereijken 1997), but has been little documented outside of step-by-step design processes (Meynard et al. 2012). The lack of information about this step results from several specific features of the design of agricultural artifacts: (i) the high degree of dependence of these artifacts on the local context (soil, climate), (ii) the time interval between trial implementation and the results, particularly for perennial crops, and (iii) the weak link between designers and final users. For the same reasons, the deployment of the solution once its objectives have been reached is also poorly documented in the literature. By contrast, the development step is at the heart of all innovative food design models. At this stage, the initial food concept is confronted with various technical, social, regulatory, and economic constraints. This step is crucial for the success of the final product and it is important for companies to capture and manage the know-how gained for their future projects. This difference between agriculture and food production can be explained by the artificial nature (equipment) of the systems on which companies act to carry out food processing. In these artificial systems, the aim is to decrease the variability of the production tool, and this requires action at the product development stage. This difference can also be explained by the company’s performance-based approach, for which know-how is an element of competitive advantage, which is not observed in agricultural approaches.

3.3 Place and role of assessment in the design processes

Sustainability assessment is generally used to assess and/or validate the improvement in sustainability of an innovation relative to the current or standard situation. We assume that the form and place of the assessment can affect the designed artifact. Below, we identify the key steps in the assessment and their place and role in the design process.

3.3.1 Diagnosis

For a clearer formalization of the assessment occurring in this broad diagnosis step, we have separated out, in Fig. 2, the project definition step, which aims to characterize the problem (perimeters, stakeholders, needs, design participants), and the diagnosis itself, which aims to characterize the starting point and to identify relevant criteria for the assessment. The objectives of the innovation project can be identified on this basis. The diagnosis mixes the assessment step with a characterization step. The characterization is a descriptive phase to gain insights and knowledge on the functioning of the systems while the assessment aims to identify the problems and the points to improve. This diagnosis can lead to a kind of paradox. On one side, we have an existing system to improve where we can make calculations of indicators or use existing measures. Diagnosis here clarifies what is known about the system studied (strengths and weaknesses) and the causes of the impacts generated. On the other side, at this stage we lack of knowledge on what the future innovation will be and thus the relevant criteria and indicators to assess it. Diagnosis is thus often based on an assessment of criteria defined with various degrees of precision.

3.3.2 Screening

As the identity of the artifact emerges, new criteria are incorporated into the assessment process, reflecting the knowledge gained by stakeholders.

In agriculture, the screening of solutions often goes hand-in-hand with ideation, more or less implicitly, and it is often based on iterations and interactions (Le Gal et al. 2011). The multi-criteria assessment models decompose sustainability into a series of criteria (aggregated or not) that make it possible to characterize the different solutions designed and from there to discriminate between them by being able, if necessary, to take into account the preferences of the different actors (Martin et al. 2013; Pelzer et al. 2012; Sadok et al. 2009). In the food industry, solution screening is a milestone in the process separating ideation from development (Siriwongwilaichat 2001). The specifications resulting from the initial exploration and the choice of concept at the end of the ideation process help in defining the criteria and performance values for screening (Benner 2005). Consequently, the relevance of the criteria selected to evaluate the different concepts is linked to the relevance of the exploration.

3.3.3 Evaluation

In the evaluation stage, sustainability assessment is a key component of the decision-making process that allows for trade-offs between imperfect solutions with various indicators that individually do not necessarily identify the best solution (Pelzer et al. 2012). Complex assessment frameworks are developed for this purpose, addressing diverse objectives and criteria (e.g., Carlsson et al. 2017 or Chopin et al. 2021).

Evaluation, on the basis of experimental data, occurs before the development step. During the translation of the conceptual solution into a solution, a transitory artifact, often referred to as the “prototype,” emerges and evolves through conceptual, model-based, or field experimentation. In the food industry, prototyping is systematically performed either in a pilot plant and then on the production line or directly on the production line. This step is essential, to assess the suitability of the formulation and processes for meeting the specifications. The performance levels achieved may lead to questioning the initial specifications or to modifying the choices of formulation or process settings. In agriculture, it is at this stage that the assessment step is most commonly formalized, with loops back to the ideation or objective definition steps (Lançon et al. 2007; Pelzer et al. 2012; Vereijken 1997). In this case, ex ante assessment involves assumptions regarding changes in scale or the generalizability of solutions. Later in the design process, ex post assessment hinges on fewer uncertainties, as the identity of the artifact crystallizes. However, loops back to previous steps are more expensive. The more innovative the solution, the less likely it is that standardized methods will be suitable for assessment, particularly in the upstream phases of projects. That means that at least a standard method has to be adapted or, in some cases, that a specific one has to be developed (Lairez et al. 2015).

3.4 The importance of clarifying causal relationships to acquire knowledge

The funnel of innovation proposed by Wheelwright and Clark (1992) highlights the importance of the initial exploration phase, followed by successive stages in the synthesis of solutions that gradually fix the final characteristics of the new product or system and produce it. This first stage, exploration, is important because the relevance of the designed artifact depends on the needs of the final user/customer, but also in relation to the needs of the stakeholders of the sociotechnical system in which the innovation emerges (Motte et al. 2011). The stakeholder knowledge can either be derived from their direct participation in the design process or be captured by the designers, who then re-inject it into the project. Similarly, the development of artifacts leads to the creation of knowledge that changes both the designers’ understanding of the targeted objectives and their perception of the limitations in terms of social, technical, and economic feasibility.

One key output of assessment is thus the knowledge it provides, about the artifact itself, and about the causal relationships between its components and the impacts generated. In other words, assessment provides an opportunity to characterize the artifact and its context in detail, and to understand it. Artifact characterization, resulting in knowledge acquisition, is clearly a major element of diagnosis. The quality of artifact characterization, and of the data used, in particular, determines the quality of the assessment and the robustness of the decisions made on this basis. Assessment can therefore be very difficult if the designed artifact is poorly defined and there are no data available, as is generally the case upstream from an innovation design process. Screening and validation also contribute to knowledge acquisition, and can be used to fine-tune artifacts or to trigger a complete reconsideration of the previous steps of the design process. Consequently, assessment can also help to define the identity of the artifact.

Knowledge is an individual property that results from the appropriation of data and information by the individual. The appropriation of knowledge by an individual involves comparisons with personal experiences, perceptions, and previous knowledge (Davenport 1993). It can lead each individual to transform his/her own projection of the situation in which this new knowledge is embedded (Ackoff’s “Knowledge Hierarchy” model 1989). What the individual infers from this information depends on his or her own cognitive abilities and patterns. When it comes to innovation, knowledge is often blamed for limiting creativity, the so-called fixation effect. However, it has been shown that fixation can itself have positive effects on creativity (Crilly and Cardoso 2017), and techniques have been developed for making use of knowledge to counteract the fixation effect, thereby increasing creativity, through the restructuring of knowledge, for example (Brun 2017).

The challenge faced by multidisciplinary design teams is rendering explicit the tacit knowledge of each individual to build a common paradigm (Nonaka 1996). The stakeholders involved in the design process are required to express their view and knowledge about the artifact. These exchanges enable the other team members to discover new dimensions or phenomena and to build a new representation of the design situation. Through the provision and sharing of knowledge, assessment is, thus, an essential element in the design of innovations.

3.5 The importance of feedback loops

Most of the feedback loops in the design process are consecutive to an assessment step (Fig. 2). A loop may simply require a new ideation step, or may have more profound implications, calling objectives or assessment criteria into question (Kline and Rosenberg 1986). Some design approaches, known as “step-by-step” design, initiate a process of continuous improvement based on an iterative process in which aspects that need to be changed are identified, new practices are designed and implemented, and there is then another round of assessment to identify which aspects merit further improvement (Meynard et al. 2012). The positive side effect of a feedback loop is to initiate a learning process supporting designers, as stated above.

In terms of creativity, it has been shown that innovation emerges from iterations between the proposal of solutions and the use of knowledge, leading to a co-expansion of solution proposals and knowledge (Hatchuel and Weil 2009; Le Masson et al. 2006). This suggests that each round of assessment, followed by an ideation step, contributes to the design of more innovative artifacts, by leading stakeholders to explore the unknown. One drawback of iterative processes involving several ideation and ex ante assessment steps is the difficulty in comparing the artifacts resulting from each sequence, as the assessment criteria may change during the course of the process (Lesur-Dumoulin et al. 2018). This suggests the need for adaptive assessment methods, capable of taking into account the expansion of knowledge as the design process unfolds.

4 What are the inherent characteristics of sustainability assessment in design processes for innovations?

As described above, it is beneficial to formalize assessment within the design process. Furthermore, the design of more sustainable agrifood systems involves specific features of the various dimensions of the assessment:

  • The criteria and indicators used;

  • The use or non-use of participatory practices;

  • The adaptive or non-adaptive nature of the process;

  • The high levels of uncertainty to manage.

Obviously, these features can be combined in an assessment. This section aims to present key points, illustrated in Fig. 3 and detailed below, that can be used to guide assessment in design processes for sustainable innovations.

Fig. 3
figure 3

Key points to provide guidance on assessment in design processes for sustainable innovations.

Full size image

4.1 Criteria and indicators

Assessing sustainability is always a multi-criteria issue. Here, criterion is taken in its meaning given in sustainability science, as a concept behind the assessment used to select one or several indicators. For example, in food systems, the criteria may relate to environmental, social, economic, ethical, nutritional or sensory aspects. However, it is not so frequent to deal with all of them equally. New agricultural production systems principally address economic and/or agronomic objectives as a first priority (Le Gal et al. 2011; Martin et al. 2013). With respect to the environment, the most common targets considered are the reduction of nitrogen losses, pesticide pollution, soil erosion, biodiversity losses, or greenhouse gas emissions. However, some tools are emerging, such as SMART-Farm Tool, which integrates four sustainability dimensions, divided into 21 themes and 58 sub-themes of the FAO-SAFA Guidelines, resulting in 327 indicators to measure the degree of sustainability goal achievement of farms (Schader et al. 2019). In food companies, assessment tends to focus on technical-economic criteria derived from market studies, and technical knowledge relating to the different stages of the product life cycle, consumer acceptability of the product, and regulatory constraints governing the product category. In some cases, objectives such as sharing economic value or respecting animal welfare also lead to taking into account the expectations of other stakeholders.

Rational decision-making (Royer 2002) highlights the importance of the definition of criteria for assessing the relevance of the solutions designed throughout the project. The categories of criteria considered during the decision-making processes are diverse and relate to the objectives and constraints of the project. Many studies have been performed on sustainability multi-criteria assessment (Bockstaller et al. 2015; Sadok et al. 2008), and a number of recurrent points have been raised, despite the absence of a preferred methodology relevant to all sustainability assessment issues. Criteria must be relevant, understandable, measurable, non-redundant, and as few in number as possible, and a complete assessment requires all the criteria to be considered together (Baker et al. 2002; Keeney and Raiffa 1976; Maystre et al. 1994; Recchia et al. 2011; Sadok et al. 2009).

Criteria are generally evaluated by qualifying or quantifying indicators (Sadok et al. 2009). Indicators may be intrinsic (i.e., representing the impact of the artifact within its own perimeter and on its stakeholders) or extrinsic (i.e., representing the impact of the artifact outside its own perimeter and stakeholders). For instance, sustainability cannot be evaluated unless extrinsic indicators tackling environmental and social external impacts are included in the assessment.

Thus, expressing the ambition of the sustainability goals has an effect on the design of the innovation and, consequently, on its performance. The starting point of the process, and whether the starting point is an artifact or a challenge, therefore strongly affects both the assessment and the whole design process. Examples of the use of an artifact as a starting point would include, for example, “developing a new plant protein-based food,” whereas an example of starting from a challenge would be “decreasing the environmental impact of protein-based food.” In the first case, indicators relating to the environmental impact of the food would not necessarily be included among the indicators used to assess the product, whereas such indicators would clearly be indispensable in the second case.

Many different methods exist, for the qualification/quantification of indicators (Bockstaller et al. 2015, Chopin et al. 2021; Soulé et al. 2021). Life cycle assessment (LCA) tools have provided scientific standardization for the environmental pillar of sustainability (ISO 140402006). LCA approaches have been extended to economic and social dimensions (e.g., Feschet et al. 2013), but such approaches are less mature in the considered contexts. Nevertheless, even for environmental LCA, not all indicators have the same maturity (e.g., impact on climate change which is considered as robust vs impact on human toxicity which is subject to high uncertainty) or act over the same spatial scale (e.g., ozone depletion, which is a global issue vs eutrophication, which is a local issue) or temporal scale (e.g., resource depletion which is a long-term issue vs particulate emissions, which is a short- to medium-term issue). Moreover, some important sustainability goals for agrifood systems, such as preventing the loss of biodiversity, are currently incompletely addressed in LCA approaches (Van der Werf et al. 2020). In addition, experts generally consider it difficult to compare the results of two LCAs because many aspects may be different between two studies of the same product or system. For the comparison to be effective, it is necessary that the study perimeters, functional units, age and geographical location of the data, and the characterization method be the same. In order to facilitate these comparisons (and thus enable product labeling), the European Commission has proposed a methodology described in the document Product Environmental Footprint, Rules Guidance (European Commission 2017). This methodology proposes 16 mid-point impact categories which could be aggregated in a single score. This methodological framework does not, however, resolve all the difficulties inherent in LCA. Thus, questions remain about the weighting of indicators, the appropriateness of the functional unit, the choice of allocation to co-products (Pedersen 2020), and the non-homogeneity of the maturity of indicator calculations (Hélias et al. 2022). Finally, it is generally accepted that aggregate evaluations facilitate the ranking of solutions among themselves, but that non-aggregate evaluations using mid-point indicators facilitate the analysis of the causes of performance and non-performance of systems with a view to their positive evolution (Hélias et al. 2022).

But above all, LCA is difficult to implement to assess innovation because it requires large amounts of quantitative data and is used in a comparative manner, raising questions about the best choice of reference system for comparisons. Prospective LCA approaches have emerged especially to address innovation situations. Such approaches are progressing but remain challenging to implement because they involve temporal mismatches between foreground systems (based on new researches) and background systems (built with older data) that need to be considered separately to analyze future innovations (Arvidsson et al. 2018). The literature review by Thonemann et al. (2020) reported that three main challenges remain when conducting prospective LCAs of emerging technologies: comparability (variation in perimeters and functional units required for old and new systems assessed), data (availability and variation in the age and scale of the data employed in the models), and uncertainty attached to the assumptions built around the innovative system being modeled. Furthermore, in this type of study, the role of a multidisciplinary team including LCA specialists, engineers, technicians, and experts involved in the design, development, and testing at different scales of innovation development is essential (Calero et al 2022). For these reasons, the use of life cycle assessment tools for ex ante assessment remains difficult, and sometimes even impossible. Among the other methods, energy or exergy assessments can be cited, as well as benefits-costs analysis, input-output analysis, etc. (Sala et al. 2015).

Complementary, MCDM (Multi-Criteria Decision Methods), also called MCA (Multi-Criteria Assessment) or MCDA (Multi-Criteria Decision Analysis), have proved to be widely used for sustainability assessment (Sadok et al. 2008, Pelzer et al. 2012, De Luca et al. 2017, Diaz-Balteiro et al. 2017, Lampridi et al. 2019, Valizadeh and Hayati 2021). “Multi-criteria methods” is an umbrella term for a range of quantitative and qualitative methods that differ across criteria selection, indicator set, aggregation method, and weighing/compensation rules, the two mostly used being those called Analytic Hierarchical Process and Weighted Arithmetic Mean (Diaz-Balteiro et al. 2017). Here again, according to the context and the objectives of the sustainability assessment, the designers will have to select the most appropriate method (Sadok et al. 2008). MCDA objectives are generally to reach rational, justifiable, and explainable decisions as well as to include in an objective manner participative methods in decision processes (De Luca et al. 2017).

4.2 Participation of stakeholders

According to stakeholder theory (Mitchell et al. 1997), the company is considered a sociotechnical system interacting with the stakeholders. As a result, its behavior and activities have an impact on its internal and external stakeholders, who are themselves in a position to influence the company’s future. According to Ackoff (1994), the company’s role is to reconcile the interests of its stakeholders, even when they conflict. Stakeholder theory is used in two ways: instrumentally, to identify the value creation and performance objectives that the company should adopt to satisfy its stakeholders, and normatively, such that the company coordinates the interests of the various stakeholders. This theory was initially designed for companies, but has since been incorporated into the practices of many organizations (Bonnafous-Boucher and Rendtorff 2014), particularly in situations in which it is important to provide populations or associations defending absent or minority values with a direct voice within the constraints of companies.

The instrumentation of this theory within a design project can make it possible to involve stakeholders at any stage of assessment, with different purposes (De Luca et al. 2017). They can help to determine objectives, and to assess modalities and the design of scenarios. This can be achieved in two main ways: by considering the criteria of the stakeholders and their associated intrinsic indicators, which may differ in nature or order of priority between stakeholders, and by considering the opinions of stakeholders when comparing results or the ranking of scenarios. Nevertheless, participatory practices can be laborious and time-consuming to implement, and the choice of stakeholders for inclusion in the process, and the timing and nature of their contributions, must be carefully thought through in terms of the expected outputs of the assessment process. The participatory method (focus groups, workshops, etc.) should therefore be used with caution.

Participatory approaches involving farmers and/or extension services have been shown to be more powerful than non-participatory approaches, for three reasons (Altieri 2004; Cerf et al. 2012). Firstly, local knowledge is essential for an understanding of the adaptations required in new agricultural systems; scientific knowledge alone is not enough. Stakeholders also hold knowledge and data of great potential utility for the assessment process. Secondly, the design process needs to include the perspectives of the agricultural sector, not just those of the researchers. The use of expert knowledge rather than scientific knowledge to define intrinsic indicators can therefore be relevant. Higher intensities of innovation are associated with a lower predictability of criteria upstream from the project and a lower predictability of indicators reporting on the performance of the artifact. As a consequence, in highly innovative systems, for which quantitative data are not available, ex ante assessment is based on approaches, such as DEXi MCDA methodology (Bohanec 2020), in which expert knowledge helps guide the assessment (Pelzer et al. 2012). The mobilization of stakeholders therefore appears to be particularly important at the front end of innovation, when concepts are still shifting and knowledge remains uncertain. Thirdly, involving different stakeholders in the process increases the likelihood of the designed innovation being accepted (Triste et al. 2014).

In the participatory approach, the criteria and indicators are less formalized and often remain in the heads of the operators, who may have different values and preferences. One of the challenges is rendering these criteria and indicators explicit and understandable by all, for their discussion and coherent use. The involvement of stakeholders in defining intrinsic indicators is desirable, because they are, by definition, part of the system under study. However, the consideration of extrinsic indicators, to assess external impacts, is also a key element of sustainability assessment, and it is debatable whether stakeholders can give the best definitions and rankings of all the indicators considered in the assessment. For instance, their opinions may be highly relevant for economic indicators relating to their activity, but ill-informed concerning the impact of the artifact on climate change in a long-term perspective.

The designer—or the facilitator of the design group–then adopts a key role in making it possible and facilitating the emergence of arenas for negotiation between the different stakeholders/actors involved in the emergence of the artifact (Jeuffroy et al. 2022). These arenas are places where the expectations and constraints of the actors are translated (Akrich 2013). By actors, we are referring here to the actor-network theory (Callon 2013), which associates this term with both human and non-human actors. Natural ecosystems can be considered here as one of these non-human actors. It is up to the designer or facilitator of the design to make visible the roles, expectations, values, or constraints of both these non-human actors and the human actors whose social positions make them hard to hear. To this end, the designer or facilitator can rely on intermediary objects (Winck and Jeantet 1995) such as graphic representation and visual or technical object, which are appropriated, translated, and negotiated by the actors.

4.3 Adaptive process

As mentioned above, sustainability is not a static concept. An agricultural or food system considered sustainable today may not remain so indefinitely, and sustainability criteria and indicators can evolve. A general framework for assessing the sustainability of agrifood systems with agreed, unchanging criteria and indicators thus seems neither achievable nor desirable, due to changes in our knowledge of the causal relationships between human activities and environment, social issues, economics, and so on, and to the changing expectations of society. In the same way, innovation, by nature, will evolve along the design process, together with the relevant sustainability criteria and indicators to assess it. Defining a finite list of sustainability indicators is more likely to curb innovation, by promoting standard solutions that only partly address the diversity of sustainability issues. The innovation design process itself is not static either, as shown on Fig. 2, and there are many feedback loops. Conversely, assessment provides a snapshot of the artifact at a given time. Therefore, assessments need to be conducted several times to appropriately inform sustainability and innovative design. Assessment is actually a major component of the dynamics of the process of innovation design, because it feeds the feedback loops, as shown in Fig. 2 and described above. While the design process is underway, the form and content of the assessment must be adapted to the objectives, the state of knowledge, the stakeholders involved, and the expected outputs of the assessment. This is particularly true for sustainability assessment, the definition of which evolves over time and differs between stakeholders (Chebaeva et al. 2021).

In particular, the criteria and indicators used to conduct the assessment, and the information collected, are defined before and during the project. Any changes are likely to be due to evolution in the definition of the innovation. At the beginning of the design process, the range of assessment criteria may be broad, particularly at the diagnosis stage. As it progresses, more arbitration is required, to facilitate decision-making. This can be achieved by reducing the number of indicators over time, or by assigning weights to them according to the importance of the processes taken into account or the preferences of the stakeholders (Greco et al. 2019). Clarification of the arbitration process between criteria/indicators is very important, especially when different stakeholders are involved. An iterative process linking design and assessment is crucial to achieve such adaptation, particularly for sustainability issues.

4.4 The unavoidable levels of uncertainty

Dealing with sustainability in complex systems such as agrifood systems inevitably leads to uncertainty. Different types of uncertainty have been described (Walker et al. 2003).

The first type is intrinsically linked to the definition of sustainability itself, i.e., the framing of the sustainability issue, directly related to the understanding of the artifact under study by the designers and/or the different stakeholders. The framing of the sustainability issue will result in the definition, selection, and prioritization of the indicators. As stated above, this step of the assessment is crucial. Nevertheless, the indicators for the diagnosis, a screening, or an evaluation might not be the same through the design process, due to the possible evolution of the innovation on one side, and due to the increase in the knowledge about the artifact on the other side. Involvement of stakeholder can help to solve such uncertainty, thanks to the different points of view they will brought. This diversity of point of views should adequately encompass all the sustainability issues of the artifact under design. Conversely, stakeholder involvement can increase uncertainty due to potential conflicting points of view (Schader et al. 2019; de Olde 2017).

The second type of uncertainty lies in the data used to quantify/qualify the criteria and indicators. This uncertainty is directly related to their accuracy, precision, representativeness, and natural variability, especially when dealing with biological data. Replication of experimental measurements can generally reduce this uncertainty. However, in the case of an innovation, it can be difficult to perform measurements because the innovation does not exist yet. Using prototypes can help with this purpose, but a good representativeness of a prototype compared to the real final innovation can be difficult to achieve when considering sustainability aspects (e.g., due to scale effects). Stakeholders can help to overcome such a difficulty by providing knowledge and observations (Refsgaard et al. 2007).

The third type is associated to the model used for sustainability assessment. It includes uncertainty due to indicator calculation methods as well as multi-criteria methods and other aggregation methods. The model structure itself embeds uncertainty, related to the algorithms and relationships between model inputs/outputs, especially if they are not linear, together with aggregation rules if any. Beyond the structure of the model, uncertainty is associated to the parameters of the model, their calibration, and validation. Many methodologies and tools have been reported to deal with such uncertainty (Refsgaard et al. 2007; Pastor et al. 2020). Above mathematical and numerical tools, here again stakeholders can be included in the approach. Expert opinions (67 experts from 21 countries) have been considered in a multi-criteria assessment tool at farm level (Schader et al. 2019), asking them to rate the importance of indicators within each of the 58 sub-themes from SAFA guideline methodology. Considerable uncertainty remained in indicator weights even after discussions, showing that expert opinions cannot necessarily overcome all uncertainty. Disagreements were due to regional variation and inherent system complexity, also showing how the different levels of uncertainty are intrinsically linked (representativeness and natural variability of the data as well as framing of the sustainability issue, respectively).

Finally, the output of the model, which supports the decisions, combines all the uncertainties and is unavoidably itself marked by uncertainty. A transparent representation of uncertainty in sustainability assessment would clarify results and support as efficiently as possible the decision-making (Pastor et al. 2020). Merging of fuzzy techniques, mathematical means of representing vagueness and imprecise information, with multi-criteria methods is increasing to improve this (Ardente et al. 2004; Bockstaller et al. 2017; Diaz-Balteiro et al. 2017).

5 Conclusion

The design of innovations to improve the sustainability of agrifood systems is a major goal. We show here that the design process requires (1) to formalize the place and mode of assessment, (2) to use relevant sustainability criteria and indicators, (3) to reinforce participatory practices, and (4) to adapt assessment to the context of the designed artifact, to facilitate choices between imperfect solutions. We formalized a design process that is appropriate for agricultural, food, and agrifood systems and displays these features. In this design process, assessment has three purposes: initial diagnosis, screening to choose between solutions at the ideation stage, and evaluation at the prototyping and development stages. Modes of assessment are discussed for each of these roles and general key points concerning sustainability assessment are highlighted. A first set of key points relate to criteria and indicators, a second set to the involvement of stakeholders, a third to the adaptive nature of the assessment, and a fourth to the uncertainty consideration. Among the different indicators used for multi-criteria assessment, extrinsic indicators targeting sustainability goals, such as indicators relating to the environment, should be mobilized. Expressing the ambition of the sustainability goals should drive the definition of criteria and indicators. Stakeholders should be involved in the definition of intrinsic indicators, and, more generally, in setting objectives, assessing modalities, and/or designing scenarios. Stakeholders provide unique knowledge and data to the design process. Assessment must be adapted to their mindset and abilities. Assessment must also feed the loops of the design process, which cannot be linear. Iterative processes are required to address sustainability goals. For these reasons, criteria are defined both before and during the project, evolving with progress in the design of the innovation. Complementary, a transparent representation of uncertainty in sustainability assessment would clarify results and is strongly recommended. These key points should guide efficient sustainability assessments in innovation design for greater sustainability in agrifood systems. The next step would be to collect case studies and to analyze how those main issues in the assessment during an innovation process are addressed.