Introduction

Context and Motivations

The implementation of circular economy (CE) loops is not systematically beneficial from an environmental standpoint [1]. One can thus question whether and when improving the circularity performance leads to sustainable benefits such as environmental savings and economic advantages. CE measurements should be completed by sustainability measurements through triple bottom line (TBL) indicators [1]. Thus, as circularity does not automatically mean more sustainability, it is key to not only know but also understand when circular means sustainable to help manage the situation (product benchmarking, identification of improvement areas) and to take appropriate actions accordingly (i.e., decision-making, selection between design alternatives). Oliveira et al. (2021) recently confirmed the need for multi-dimensional and multi-criteria approaches for the sustainability evaluation of the transition towards a CE [2]. While Oliveira and colleagues [2] focused on the combination of multiple life cycle assessment (LCA)-based assessment methods (life cycle assessment, social life cycle assessment, life cycle costing), the present study adds the contribution of other assessment tools such as circularity indicators and leading sustainability indicators.

Combining CE and TBL measurements is both an opportunity and a challenge for industrial companies to objectively report the benefits of their CE initiative by linking them to quantitative sustainability performance measures. Additionally, while there is a wealth of methods and tools (such as life cycle assessment (LCA), circularity indicators, and their assessment framework, or stand-alone leading sustainability indicators) to evaluate the performance of a product in a sustainable and circular perspective, the complementary between these approaches remains to be clarified, notably to foster their uptake by industrialists, such as designers, material engineers, or even managers. To Walzberg et al. (2020), additional research is needed to combine existing methods and develop a more holistic approach for assessing the sustainability impacts of CE strategies [3].

If more advanced and systemic correlations can be established between the sustainability performance of c-indicators, such indicators could be practical, as time-efficient heuristic tools, to help design circular and sustainable products, improving the overall environmental and economic performance of products. With this background, the present study addresses the nexus (i.e., the link, correlation, and/or complementarity) between circularity and sustainability indicators to develop and monitor more circular and sustainable systems, using two complementary research approaches: (i) a review and critical analysis of existing works covering this topic (part I) and (ii) new experimentations on more circularity and sustainability to fill the gaps of previous works and give additional and practical insights for practitioners on this matter (part II). Note that part I and part II are complementary yet independent, as they can be read as stand-alone articles with their own contributions to the field. Interestingly, part I performed an extensive literature survey on the correlation between circularity scores and sustainability performance, summarizing and highlighting the contributions and limitations of recently published articles on that topic. With this background, the new experimentations reported in the present part II fill several gaps found in the extant literature.

Research Approach and Objectives

The main research question driving this study is to what extent—i.e., how, which, when, and where in the design and development process of products—circularity and sustainability indicators could be combined to come up with more circular and sustainable solutions? To bring new elements of response to this question, a team of researchers at CentraleSupélec (CS), Université Paris-Saclay, collaborated with researchers from the Technical University of Denmark (DTU), combining their complementary expertise on circularity and sustainability indicators, respectively. CS has developed a classification and an online selection tool (http://circulareconomyindicators.com/) for circularity indicators (c-indicators) [4]. DTU has developed an organized database and an Excel-based selection tool for leading sustainability indicators (s-indicators) [1]. The overarching goal of the present research collaboration is to ensure that the implementation of CE strategies contributes to sustainability. If the current positioning is the left one illustrated in Fig. 1 (where in some cases, circularity does not translate into sustainability), this collaboration aims at “pushing” the circularity sphere inside the sustainability sphere, by guiding designers, engineers, and managers, in the choice of sustainable and circular indicators, to make sure that the selected sets of c-indicators translate in actual sustainability.

Fig. 1
figure 1

Towards fully sustainable circular economy practices

Full size image

Based on a hands-on project with 178 engineering master students (over two consecutive classes: 87 for the class of 2020 and 91 for the class of 2021), new empirical insights are brought to the following sub-questions: Are circularity scores always consistent and well-aligned with sustainability scores when assessing the performance of a product? How LCA, c-indicators, and s-indicators can complement each other and be deployed at different stages of the design and development process of products? What is the practicability (e.g., user-friendliness, reliability) of such approaches to future engineers? This project involves more than 175 future engineers applying and comparing LCA, c-indicators, and s-indicators based approaches to assess and improve the performance of product alternatives. This study aims to further illustrate the potential synergies and conflicts between circularity and sustainability, as well as to provide practical insights on how to combine existing circularity and sustainability indicators. These complementary insights are based on new empirical workshop results, backed by a review of state-of-the-art literature (part I). Practical recommendations and guidance on how to combine existing approaches are given and justified thanks to the new insights generated by this project (including comparative quantitative results and qualitative feedback) and other supplementary case studies published recently in the literature on this topic.

Interestingly, this empirical research work fills some gaps highlighted by Kirchherr and van Santen (2019), stating that [5] (i) scholarly work on CE has yet to translate into practice, (ii) much empirical work on CE is small-N research (i.e., inferior to 10 cases), and (iii) the CE literature lacks tangible advice. Panchal et al. (2021) recently confirmed that [6] (i) empirical studies are required to determine the CE contribution to SDGs and (ii) most of the studies are considering the single product instead of taking the whole product family approach. In the present paper, 18 life cycle assessment midpoint indicators, 9 product circularity indicators, and 50 + sustainability indicators are computed and compared, in all, by 38 groups of 4 to 5 engineering students. To compare the output generated by these indicators, the engineering students had the same dataset to quantify the performance of three different products from the same product family: a conventional gasoline walk-behind lawn mower, an electric walk-behind lawn mower, and an electric autonomous lawn mower. In this line, the remainder of this paper is structured as follows: the “Materials and Methods” Sect. 2 describes in more detail the narrative of the project, the case study, and its associated data, as well as the indicators experimented. Then, the “Results and Interpretations” section reports and analyzes the quantitative results generated by the engineering students, notably on the correlation between LCA, c-indicators, and s-indicators. Next, the “Discussion and Recommendations” section discussed more qualitatively the strengths, limitations, and possible combinations between these measurement instruments. Eventually, the “Conclusion and Perspectives” section concludes on the implication and perspective of such tools and indicators to monitor and advance towards a truly sustainable circular economy.

Materials and Methods

Project Description, Workflow, and Resources

Project Context and Positioning

The results of this project were obtained during the recently developed course entitled “Circular Economy and Industrial Systems” at CentraleSupélec, Université Paris-Saclay, for which 178 engineering students enrolled, taking into account the classes of 2020 and 2021 (87 and 91 master students, respectively). As illustrated in Fig. 2, the first half of the course (from session #1 to #6) covers the different dimensions of CE to provide the students with a global vision of the field, including lectures and workshops on product end-of-life management, ecodesign, extension of product lifespan, and responsible consumption, sustainable procurement, functional economy, and, industrial and territorial ecology. The second half of the course (from session #7 to #11) is focused on the deployment of industrial ecology and CE-related tools. This includes material flow analysis (MFA), to map material and energy flows, and LCA, to calculate the environmental impacts, as well as circularity and sustainability assessment framework to monitor industrial ecology projects. These tools are directly applied by the future engineers on their engineering challenge project (described in the next sub-section). This project put theoretical research on CE indicators into practice through a case study which allows for the experimentation with different kinds of product-level c-indicators and their relationship with LCA results.

Fig. 2
figure 2

Content of the class “Circular Economy & Industrial Systems” on LCA and C-indicators

Full size image

Project Narrative and Data

The project aimed at assessing, benchmarking, and improving the circularity and sustainability performance of three lawn mowing solutions from a life cycle perspective. It has been introduced as it follows to the students: “You just got a new house with a beautiful one thousand square meter yard. To maintain it, you are considering buying a mower to trim and edge your lawn properly. As an environmentally conscious citizen, you wonder what solution is eco-friendlier. Regarding the size of your yard, three lawn mowing solutions appear as potential candidates: (i) a conventional gasoline push mower (product A), (ii) an electric-powered push mower (product B), and (iii) an autonomous electric mower (product C). The questions you set out to answer are: What is the environmental footprint of each solution? Which sustainability indicators are relevant to setting up a sound comparison? How well do these products – sub-components and associate materials – perform in a circular economy? As an engineer, what would you do to improve their circularity and sustainable performances?” This case seemed particularly relevant for experimentation with engineering students for the following reasons: (i) it is both a technical- and engineering-based system without being too complex to perform LCAs and compute c-indicators and s-indicators, (ii) it provides a real-world case study with (simplified) industrial data from the original equipment manufacturer (OEM), and (iii) it allows a comparison between three product variants: conventional, electric, and autonomous version.

Real-world lawn mower models—provided by an OEM and simplified for this project—were used for the three solutions to be analyzed [7]. The same 4-page datasheet (available in Appendix A) was provided to each of the 38 groups of 4 to 5 students. This datasheet contained three main sections with information related to (i) the design and manufacturing, (ii) the usage and maintenance, and (iii) the collection and end-of-life of the three product alternatives. First, regarding the pre-life of the products, a detailed bill of materials was given, including the components, materials, mass, price, recycled feedstock, and destination after use (if collected), as well as recycling and recovery efficiency. Elements were also provided regarding the production process for each of the three lawn mowers, including the electricity consumption, transportation, assembly phase, and packaging. Second, in regard to the life of the products, the average lifespan under proper maintenance for each mower was given for each mowing solution. Information related to maintenance operations was also provided, including engine tune-ups for the gasoline-driven mower and battery replacement frequency for the two electric mowers. Third, realistic assumptions were given related to the collection and end-of-life fate of the lawn mowers, including, e.g., the loss of economic value over time, the average percentage collected after used, and the percentage of equipment refurbished or remanufactured in order to be reused.

Life Cycle Assessment, Circularity, And Sustainability Indicators

Life Cycle Impact Assessment Part

The engineering master students following this course were trained on how to perform an LCA, following the four steps described in ISO 14040–14,044 (2006) [8, 9]. They used the LCA software OpenLCA 1.10.3 [10], the ecoinvent database 3.2 [11], and the ReCiPe 2016 Midpoint (H) life cycle impact assessment (LCIA) methodology [12]. For the impact assessment phase, they have been asked to (i) evaluate the environmental impact of each solution, (ii) compare the environmental impact of the three solutions, and (iii) propose relevant visuals to display and comment on the results. For the interpretation phase, they have been asked to (i) identify and describe the environmental hotspots for each product and (ii) explain if they can decide which lawn mowing solution is better from an environmental standpoint. In terms of implication and critical analysis, the guiding questions were as follows: What are your suggestions to decrease the environmental footprint of the lawn mowing industry? What are the limits of your model? Can you assess the robustness of your study (e.g., by conducting a sensitivity analysis)?

Circularity Assessment Part and Proposition Improvement Solutions

After the LCIA and results interpretation of the three products, each group experienced two of the eight c-indicators [13] selected per year for this study. Note that the present manuscript is the extended version of our initial results for the class of 2020 presented at the 2021 International Conference on Engineering Design [13]. The present manuscript goes more in detail in the comparison and critical analysis of LCA, c-indicators, and s-indicators. It provides additional insights based on (i) the feedback from a new cohort of engineering students (class of 2021) experimenting c-indicators on the same case study; (ii) a new c-indicator being tested, the CCET; and (iii) the integration of the s-indicators results in the analysis and discussion. In all, nine different c-indicators have been computed, considering that the semi-qualitative web-based indicator CET, used in 2020, has been replaced by the quantitative Excel-based CCET indicator in 2021. The distribution of c-indicators by each group is given in Table 1. To ensure a good balance between groups, the c-indicators selection was pre-defined, each group using one computer-based tool and one formula-based indicator (from a journal paper) to compute the c-indicators. A pre-filled one-page response document was provided for each c-indicator, including all the necessary resources (e.g., Excel spreadsheet, website, or formulas) to compute the c-indicators in question. Regarding the assessment phase, they have been asked to report the results (circularity scores) as well as to justify any assumptions made when necessary. Then, based on the results of the circularity assessment, they have been asked to propose at least four solutions (e.g., in terms of circular design, business model, and incentives) to augment the circularity score of lawn mowers and their eco-system. Moving forward, they have been asked to comment on the expected benefits (or impact transfers) in terms of environmental and economic sustainability when increasing the circularity performance of the products.

Table 1 Distribution of circularity indicators by group. CC circularity calculator; CCET circular economy evaluation tool; CEI circular economy index; CEIP circular economy indicator prototype; CET circular economy toolkit; CIRC circularity; CPI circularity potential indicator; LONG longevity; MCI material circularity indicators
Full size table

Sustainability Evaluation Part and Critical Analysis

For the evaluation of the sustainability performance of the three lawn mowing systems, the engineering students have been asked to select and apply up to eight indicators from the database of leading s-indicators [14], according to the improvement areas or solutions proposed by them in the previous part. They also had to explain their thought process in the selection of these indicators. In all, each group had three different assessments of circularity and sustainability: LCA results (lagging environmental impact indicators), c-indicators, and leading s-indicators. On this basis, they have been asked to reflect on these diverse evaluation approaches, e.g., if they are consistent to one another, complementary or contradictory, as well as to elaborate on the insights they provide (e.g., for decision-making) and on their user-friendliness (e.g., the level of expertise required, the time needed, the quantity of data required, and the design of the user interface). Finally, in the response document given to the engineering students, the room was left for open comments, guided by the following question: How—or to what extent—do the indicators provide insights to improve the sustainability performance? Shall one absolutely increase this circularity score to be more sustainable?

Results and Interpretations

In this section, the quantitative results—i.e., the LCA-, circularity-, and sustainability-based indicators—generated by each group are analyzed, compared, and interpreted in the light of assessing and improving the sustainable performance of products in a CE perspective. To do so, first, LCA and c-indicators results are reviewed on an individual basis. Then, following the workflow of this project, the correlation between environmental impact indicators and c-indicators is illustrated and discussed. Next, the inputs and results provided by leading s-indicators to evaluate the relevance of the proposed improvement are commented. Finally, further qualitative findings from this project are discussed, including the feedback and critical analyses made by future engineers on these different approaches.

Stand-Alone Lca Results And Environmental Trade-Offs

Different groups used different ways to represent and communicate the LCA results, including bar charts, radar diagrams, or tables, as illustrated in Fig. 3. Due to environmental trade-offs among the 18 ReCiPe midpoint indicators, it was not straightforward for the engineering students to simply identify or recommend one single solution (i.e., the most commendable from an environmental standpoint). For instance, in the spider diagram of Fig. 3 (bottom left corner), the mowing solution A has the least global warming potential but the highest agricultural land occupation. For instance, group #11 (class of 2020) mentioned: “we cannot immediately say which mower is the best. Indeed, according to the criteria that we favor, the A, B or C can stand out.” As such, to provide a sound and well-justified recommendation on which mower to select based on LCA results, most groups considered the most relevant LCA-based indicators for this specific context of mowing a yard, such as global warming, human toxicity, or land use occupation. Note that the selection of these indicators (to draw realistic recommendations for decision-making) could also be based on the sustainability strategy of the manufacturer (communicated through their ad hoc sustainability report and objectives).

Fig. 3
figure 3

Illustrations of LCA results for different groups

Full size image

Additionally, according to students’ feedback, it was not straightforward to directly propose concrete design improvement based on the pure LCA results, nor to practically assess the impact of possible improvements in design. For example, group #10 (class of 2020) commented that while “LCA and MFA are two ways of evaluating the circular economy strategy of a product, they also have their limits. They are based on products already produced and allow a good assessment of past performance, but make it more difficult for engineers to make decisions about products that are still to be designed.” They added: “we are therefore interested in new circularity indicators which allow us to add a more systemic view to our study, and to take the product into account at different levels.” Nevertheless, one group (#2, class of 2021) mentioned that “for human toxicity in the case of mower B, the main contributor was the production of electricity. To improve this, we could replace the battery materials or increase the lifetime of a battery, which has to be changed once during his life cycle. A strategy to implement to ameliorate the footprint of mower C can be the replacement of plastic parts derived from fossil sources to more ecological alternatives such as BPA-free plastic made up of cellulose and other natural polymers.” Finally, another group (#15, class of 2020) brought out the geographic dependence of LCA results, notably for the use phase impact: “this impact could have been different and distributed differently if another country had been chosen to use the mower.”

Stand-Alone C-Indicators And Robustness Of The Assessment

In this sub-section, the common features and trends provided by product-level c-indicators are quantitatively analyzed and qualitatively discussed, as well as their individual specifics and how they relate to one another. Each of the 38 engineering student groups (20 groups for the class of 2020, 18 groups for the class of 2021) experienced two c-indicators according to the distribution given in Table 1. So, each c-indicator was computed at least by eight groups, over the sessions of 2020 and 2021, except the CET and CCET, which were only used by four groups in 2020 and 2021, respectively. All these 38 groups were working with the same dataset for the three products. On this basis, the variability and robustness of these c-indicators are illustrated through the box and whisker plot of Fig. 4, combining the results from the classes of 2020 and 2021, as they have a similar pattern [13]. While the MCI, CPI, CEIP, CCET, CC, and CEI deliver an overall score between 0 and 1 (or a circularity percentage), the CIRC and LONG scores have been normalized using the min–max feature scaling for comparison purposes. Also, the CET, a qualitative indicator, is not represented in this graph but discussed in the critical analysis hereafter.

Fig. 4
figure 4

Box plot of the circularity scores

Full size image

On the one hand, the results from the CPI, CEIP, and CCET are consistent with one another: (i) they both assess a circularity potential, (ii) tend to underestimate the circularity performance due to many conditions to fulfill to reach a high, or even medium, circularity score, and (iii) present some robustness and a low variability between groups (see box plots of Fig. 4). On the other hand, the MCI and CC, both material flow-based indicators assessing an effective circularity, present similar trends by being highly sensitive to the assumptions set by each group (i.e., in the present case, on the actual end-of-life fate of the products, and their associated components and materials). In this line, the CIRC and LONG, assessing as well an effective and intrinsic circularity of resources, present an important variability between groups. Overall, the c-indicators tend to score product A with a higher circularity than products B and C. One group (#2, class of 2021) mentioned that “the simplicity of mower A makes it more circular whereas the batteries of mower B and C make it more complex to recycle and to manage in the end-of-life.” Note that the CEI appears here to be a stand-alone and complementary indicator by assessing the economic value of material recirculation through CE loops.

Interestingly, the additional feedback from engineering students not only reinforces several advantages and limitations of product-level c-indicators recently mentioned in the literature [15, 16] but also brings out new and practical aspects regarding their usage. For the MCI, the normalizing factor (from material to product level) selected (e.g., product mass, or material mass multiplied by price) can have a significant effect on the final overall aggregated MCI score. For group #1 (class of 2020), “this indicator is therefore useful for a rough comparison of different solutions,” and for group #13 (class of 2020), “results obtained from the MCI indicator must be wisely analyzed before any important conclusions are done.” For the CPI, one group (#6, class of 2020) mentioned that “certain questions would deserve a greater variety of possible answers.” For example, the category “markets for secondary raw materials of the product” allows only one response among “landfill,” “energy recovery,” and “recycling,” whereas a product made up of many materials may require a more complex and accurate breakdown. For the CET, several groups noticed that it has the advantage to be time-efficient (e.g., group #7 (class of 2020) stating that “answering the questions takes about ten minutes and gives a good overview”) as well as to “identify criteria with great potential for improvement such as recycling and selling the product through a service.” Yet, one group mentioned that “the fact that there are only three choices limits the differentiation between the different products.” For the CEIP, most groups mentioned its “ease of use” (i.e., “simple to handle”, “questions are precise and easily understood”). Yet, one group noted that “for this indicator to be effective, a fairly large and precise quantity of data must be collected on each product.” The CCET, aiming at assessing and comparing which product designs are more adapted to CE loops, is perceived as a more subjective indicator by two groups (#3 and #7) of the class of 2021, as it asks the user to rank the most relevant CE strategies, and the scoring system is based on the defined hierarchy. For the circularity score given by the CC, some groups completed this indicator with three other indicators provided by the online platform, namely, the captured value, the recycled content, and the reuse index. For the CEI, one group commented that “it can help decision-making by assessing the economic ‘efficiency’ of recycling” and therefore encourage innovation in the recycling industry or opt for materials that are interesting to recycle for manufacturers. Yet, they acknowledged having “trouble understanding precisely the formula” to actually use and compute it. The LONG indicator disregards the types of materials and mainly focuses on the optimization of the lifetime of a product. As such, it should be used with other indicators such as the CIRC. Note also that the high variability between the CIRC and LONG scores between groups is both due to (i) the complex formula (available only on the associate research paper, with no calculation tool attached) and (ii) the non-negligible number of assumptions required to compute these two complementary c-indicators.

Correlation Between Lca and C-Indicators

After analyzing the LCA and c-indicators results separately, the correlation between these two is now investigated. The color-coding in Table 2 shows the level of correlation from the agreement (green) to disagreement (orange) between the LCA and circularity indicator-based approaches according to the product alternative recommended (i.e., higher circularity for the c-indicators, lower environmental impact for the LCA-based indicators).

Table 2 Correlation between LCA and C-indicators, results for the class of 2021
Full size table

Note that to filter out (in our interpretation of the results) the LCA and/or c-indicators results which have not been computed with sufficient rigor or soundness by some students’ groups, the grades of each group were used. Two reviewers evaluated each report, and the final grades are the average value of these two evaluations. As the c-indicators and LCA interpretation parts were both rated on 3 points (so 6 points in total), we arbitrarily set a threshold at 4 points out of 6, as a proxy of the quality of the results that are commented on in this paper. Also, note that the results for the class of 2021 detailed here are consistent with the ones obtained with the class of 2020, as described in the associated conference paper [13].

Overall, the LCA and circularity indicators tend to correlate and “seem to yield concordant results,” as noted, for example, by one group. Particularly, for most groups, the MCI and CC, both material flow-based indicators, are in adequation with the LCA results. The same remarks applied to the CET, CIRC, and CPI indicators. On the other hand, in the present case, the CEI, assessing the economic value of material recirculation, recommends another product solution that the one favored by LCA results, and therefore, no conclusive correlation with LCA can be done here for this specific c-indicator. Note that the variability of the solutions recommended by each group can be explained both by the LCA indicators selected for decision-making (as previously explained in the “Stand-Alone LCA Results and Environmental Trade-Offs” sub-section), as well as by the assumptions they had to make to compute some c-indicators.

Complementary Leading S-Indicators and Trade-Offs Management

Different leading sustainability-related performance indicators were selected from the database of 290 + s-indicators [17], including 70 economic (EC), 175 environmental (ENV), and 51 social (SOC) indicators. At this point, the students were less guided, and more assumptions had to be made to actually compute the s-indicators, which makes difficult the quantitative comparison between groups, though it becomes interesting to analyze the differences or similarities between future engineers in the choice of s-indicators, how they took potential trade-offs into account, and their thinking process to recommend one particular product alternative or solution. An example of trade-offs between LCA, C-indicators, and S-indicators is given in Table 3. As shared by several groups, two different sets of indicators could lead to different recommendations. In this line, group #4 (class of 2021) mentioned they “can see the advantage of combining two approaches to make a decision that is most respectful of the principles of circular economy and sustainable development.” To deal with such sustainability-related trade-offs, Kravchenko et al. (2021) recently proposed a trade-off navigation framework to be used by management, design, environment, and sustainability teams [18]. It requires the following input data: (i) a list of key environmental, social, and economic indicators, (ii) a set of initiatives for comparison, and (iii) the acceptability ranges and non-negotiability aspects. The trade-off matrix consists of three steps: (i) analysis of the performance on non-negotiable criteria, (ii) analysis of the performance on negotiable criteria, and (iii) decision analysis.

Table 3 Comparison and trade-offs between LCA, C-indicators, and S-indicators (example from group #13, class of 2020)
Full size table

In all, among the 38 groups of 4 to 5 engineering students (each group selecting up to eight leading s-indicators), 61 different s-indicators were selected (21 EC, 32 ENV, and 8 SOC), including 4 identical EC indicators being picked up by three different groups (e.g., the EC4 “Revenues from reused/repurposed products), 9 ENV indicators being used twice (e.g., the ENV73 “Fraction of recyclable materials”), and 2 SOC indicators being used by two different groups as well (e.g., the SOC2 “Take-back offering for product”). For instance, the EC4 has been appreciated by several groups in the present context as it could encourage the manufacturer to develop a new recovery strategy for old mowers.

Note that a couple of groups proposed CE-related improvement solutions for the three products based on the results obtained with the c-indicators. Interestingly, they selected ad hoc leading s-indicators to assess rapidly the economic, environmental, and social relevance of their proposed solutions. For instance, one group (#16, class of 2020) identified four areas of improvement to increase the circularity score of the different product systems throughout their life cycles, namely (i) usage of a higher proportion of recycled products (especially plastics and cardboard packaging); (ii) reduction of energy use during the use phase (this point concerns mowers B and C mowers); (iii) establishment of better customer services (product follow-up, increased guarantee); and (iv) improving end-of-life recycling (e.g., replacing incineration of plastic with recycling, assuming all PVC, PP, ABS, and PE will be bought back for recycling at 0.3 €/kg). As summarized in Table 4, they deployed five leading s-indicators to compare the three reference product systems with their proposed circular improvement scenarios.

Table 4 Deployment of leading S-indicators to validate circular improvement scenarios (example from group #16, class of 2020)
Full size table

Discussion and Recommendations

To open this discussion section, the feedback of future engineers on these indicator-based approaches is first discussed, combining the insights from the classes of 2020 and 2021, and following the guiding questions listed in sub-Sect. 2.2.3 (e.g., on the contribution of these indicators in the decision-making process for sustainable design choices). Then, these critical analyses are completed by findings from the literature to provide further practical recommendation on which indicator-based approaches to use (e.g., LCA, c-indicators, leading s-indicators, or a combination), when (e.g., positioning in the engineering design process), for who (e.g., designers, engineers, managers, LCA experts), and how to combine these sets of indicators to come up with augmented insights to support decision-making?

Reflecting on the differences between both leading and lagging indicators for sustainability measurements and circular economy indicators, one group (#6, class of 2020) stated: “as future engineers, it is crucial for us to understand this complexity by mastering a wide variety of indicators and being able to arbitrate between them to help decision-makers (who are not always trained in very technical indicators like LCA) to make the right choices about circularity and ensuring that the circular solutions they choose are well anchored in sustainability. We think that it is particularly important to master both lagging indicators, to account for the existing and its present impact to correct it if necessary, and leading indicators, to evolve the models.” To another group (#1, class of 2020), “the risk of not taking into account the two types of indicators would be to arrive at a solution that is ultimately counterproductive.” The group #8 of the class of 2021 completed this standpoint by mentioning that “only focusing on a single indicator could lead to undesirable trade-offs.” Accordingly, group #9 (class of 2020) commented that “these three types of indicators seem to be completely complementary” explaining that “LCA results allow analysis of past performance in order to draw conclusions about decisions to be made in the future”; “C-indicators make it possible to have more numerical values and therefore to make it possible to make estimates of future developments”; and “S-indicators further tell us what to do (influence future choices at a company level).” Similarly, group #13 (class of 2020) mentioned that “life cycle analysis in OpenLCA allowed us to evaluate the current impact for all three products, while circularity and sustainability indicators were more useful for identifying potential modifications in the products that would make them eco-friendlier. They could be easily understandable from graphs and could be advertised to the general public, and they provide insights on what are the main issues with these product designs.” In this line, group #18 (class of 2020) stated that “C-indicators make it possible to identify points for improving the circularity of products. However, adding the S-indicators to them is necessary to judge the effectiveness of a measure”; and group #18 (from the class of 2021) saw “the three tools as complementary because they address different issues and audiences.”

Other groups were indeed more nuanced. For instance, group #2 (class of 2020) concluded that “the three indicators used (LCA, C-indicators, S-indicators) do not answer exactly the same questions, so it is difficult to speak of consistency between these indicators,” while mentioning “a real complementarity between these approaches: LCA allows an ‘absolute’ quantitative analysis of the impact of the product, while the leading indicators allow them to assess their performance in a more qualitative and ‘relative’ manner, taking into account other aspects, sometimes more related to socio-economic issues.” To group #4 (class of 2021), the LCA “seemed to be the most robust and complete method.” In addition, reflecting on their experience with these three different indicator-based approaches, group #11 (class of 2020) found that “C-indicators are more useful for engineers and designers, leading S-indicators are more used by decision-makers who have to manage dozens of products in a company.” In this regard, group #15 (class of 2020) added that the “S-indicators appear to be the simplest to use, since they require only a few reasonable hypotheses to be implemented.” Additionally, regarding the usability of each approach, as illustrated through Table 5, most groups agreed that product level c-indicators associated with a computer-based tool, and leading s-indicators, are the quickest and friendliest to deploy, while LCA indicators “require a mastery of the software” to cite the group #4 of the class of 2020. Eventually, group #12 (class of 2021) provided a good summary of the strengths and limitations of each approach: “All these tools have their strengths and weaknesses depending on the working and calculation assumptions made and the choice of data analyzed. LCA is a very comprehensive study, requiring a lot of research and time, but it provides a comprehensive view of the environmental performance of a product over its entire life cycle. Nevertheless, LCA ultimately aggregates a number of very different data, which can make its interpretation complex. On the contrary, an argument could be made for circularity and sustainability indicators which may be easier to calculate and understand for the general public. On the other hand, they generally offer a partial view of things, which implies that to have a global vision, it is necessary to have a multitude of indicators, in the midst of which the user or decision-maker can quickly become lost. Only by being fully aware of this can one use this diversity of indicators to obtain a clear but nuanced picture of a product.”

Table 5 Practicability of LCA, circularity, and sustainability indicator-based approaches
Full size table

Conclusion and Perspectives

The present study experimented and discussed the contributions of LCA, circularity, and sustainability indicator-based approaches to assess and improve the circularity and sustainability performance of products. Also, engineering students reflected on the strengths, limitations, and opportunities offered by each approach, e.g., in terms of their usability or complementarity. In all, the new experimentations and empirical findings reported in this part II contribute in filling several gaps and limitations of the extant literature of this cross-cutting topic on the measurement of the circular economy and sustainability performance, as reviewed in part I: “(i) a lack of diversity in the product-level c-indicators tested (most intrinsic c-indicators measuring the circularity of material flows such as the MCI), (ii) a lack of comparison between the different c-indicators deployed within the same study, (iii) a lack of discussion on potential environmental impact trade-offs between different LCA indicators when comparing multiple CE strategies, and (iv) a lack of critical analysis and feedback on the complementary between LCA and circularity indicator-based tools.” Particularity, combined with the recent state-of-the-art literature, the experimentations on LCA, circularity, and sustainability indicator-based approaches reported in this study contribute in checking and further validating several statements or assumptions made about circularity and sustainability indicators, notably (i) the important time to compute lagging environmental sustainability (LCA-based) indicators, versus the time-efficiency of c-indicators-related tools (CEIP, CET, CPI, MCI); (ii), the complex interpretations of LCA-based indicators to actually improve products versus the practical and explicit improvement orientations in outputs of c-indicators and leading s-indicators; and (iii), the scientific soundness (ISO 14040–44, 2006) [8, 9] and tangible impact of LCA-based indicators (ReCiPe midpoints and endpoints) [12] versus the variability and uncertainty of c-indicators (no ISO standard, and various scoring systems). In a nutshell, the current strengths and limitations of LCA, c-indicators, and s-indicators are synthesized in Table 6, highlighting valuable opportunities in combining them adequately.

Table 6 Strengths and limitations of LCA, circularity, and leading sustainability indicators
Full size table

In fact, the application of lagging and/or leading sustainability indicators in CE assessment can be relevant to further quantify and compare circularity scenarios, as well as to challenge the willingness to loop “at all costs” by adding environmental, economic, and social aspects to decision-making [19, 20]. On the one hand, leading sustainability indicators are particularly suitable to measure performance in the early phase of the design or redesign process (i.e., during requirements definition, concept development, and system-level design phases) to set sustainability-related targets or as sustainability screening tools [17, 21]. Sustainability screening could be performed in the early stages of any business process (for instance, at the conceptual design stage during product development) to ensure the early consideration of sustainability implications, as well as to still have some degree of freedom to act to introduce improvements [22]. On the other hand, lagging sustainability indicators (computed through LCA) are more commendable in the later stages of product design and development, e.g., detailed design, testing, and refinement, or post-implementation review, when a product’s design is more defined [23, 24]. Note that different circularity indicators and their associated computational tools could be deployed at different stages of the design process, as further investigated in a related study [25]. Currently, most products and associated services are still not systematically designed to be integrated into a circular model. As such, product-level c-indicators can be practical tools to (re-) design circular-ready products. Saidani et al. (2020) provided examples of how c-indicators can contribute to designing and developing more circular products [16]. Yet, further experimentations of c-indicators with practitioners (designers, engineers, or managers) are needed to increase their actual uptake by industry [24]. Last but not least, this study brings some hands-on opportunities to students, researchers, teachers, and industrialists in the domain of circular economy and sustainability [26, 27]. These workshops with engineering worked well and can be rerun in various contexts (e.g., academia, industry) to train the workforce of today and tomorrow on making scientifically sound circularity and sustainability measurement and decisions. As such, all the materials and resources used in this study are available in Appendix A (dataset) and on-demand (response documents and computer-based tools) to be reused and disseminated.