1 Introduction

This paper presents a group decision framework for evaluating sustainability of the insulating materials to retrofit traditional rural buildings. The importance of this research derives from the previous studies on both retrofitting solutions tailored to traditional rural buildings as well as judging an overall desirability of coating materials (see, e.g., Krarti 2015; Fabbri et al. 2012; Ma et al. 2012; Yung and Chan 2012; Martínez-Molina et al. 2016). These studies prove that energy efficiency and thermal comfort are crucial for the maintenance of historic buildings.

The context of the study is that of a typical farmhouse in central Italy. The incorporated building model derives from the analysis of over 800 farmhouses surveyed by the census of the scattered rural buildings of the municipality of Perugia (Umbria region). The high landscape values of traditional buildings and the legislation about their preservation prevent external alterations (Mazzarella 2015). Therefore, the most viable solutions are to intervene on the roof of these structures, increasing their thermal inertia with coating materials (Verbeeck and Hens 2005; Kumar and Suman 2013; Taylor et al. 2000).

We comprehensively evaluate the materials for the roof insulation by considering economic, social, and environmental viewpoints. For this purpose, we incorporate a life cycle costing (LCC) approach, a life cycle assessment (LCA), and a dynamic thermal simulation for the evaluation of energy savings and thermal comfort. As such, we aim at identifying the materials that guarantee the indoor thermal comfort, at the same time reducing the consumption of resources in their entire life cycle as well as maintaining cultural and historic significance of the buildings. In this perspective, we differentiate from the vast majority of previous studies concerning coating materials which incorporate a mono-disciplinary approach (Copiello 2017).

To provide an overall sustainability assessment of coating materials, we incorporate Multiple Criteria Decision Analysis (MCDA). MCDA offers a diversity of approaches designed for providing the decision makers (DMs) with a recommendation concerning a set of alternatives evaluated in terms of multiple conflicting points of view. Few applications of MCDA methods for the evaluation of building materials, which are reported in the literature (Ginevicius et al. 2008) deal mainly with the environmental sustainability of materials (Papadopoulos and Giama 2007; Khoshnava et al. 2016). Some combinations of LCA and MCDA were considered by Santos et al. (2017) and Piombo et al. (2016). Applications which included both LCC and LCA for the definition of criteria to be used in MCDA are still rare (Piombo et al. 2016). Decision analysis methods used in the above-mentioned studies involved different variants of AHP (Motuziene et al. 2016; Khoshnava et al. 2016), PROMETHEE II (Kumar et al. 2017), Weighted Sum, TOPSIS (Čuláková et al. 2013), VIKOR, and COPRAS (Ginevicius et al. 2008).

From the viewpoint of MCDA, our study differs from the aforementioned ones in terms of the following major aspects:

  • We formulate the considered problem in terms of multiple criteria sorting, thus aiming at assigning the materials to a set of pre-defined and ordered sustainability classes (categories) rather than at ordering them from the best to the worst;

  • We assess the insulating materials while taking into account preferences of multiple DMs (owners of rural houses), thus incorporating group decision making tools into the evaluation framework;

  • The adopted assignment procedure builds upon outranking-based comparison of the insulating materials with the characteristic profiles composed of the per-class most representative performances on all criteria (Kadziński et al. 2015b);

  • The research results are validated against the outcomes of robustness analysis that takes into account all sets of weights compatible with either the ranking of criteria provided by each DM within the revised Simos (SRF) procedure (Figueira and Roy 2002) or a group compromise ranking of criteria that is constructed with an original procedure proposed in this paper.

The remainder of the paper is organized in the following way. In the next section, we review the existing group decision making methods for multiple criteria sorting. Section 3 describes a three-stage decision aiding method that has been used to evaluate the insulating materials while taking into account preferences of a group of stakeholders. Section 4 exhibits comprehensive results of multiple criteria assessment of the insulating materials. The last section concludes.

2 Review of Multiple Criteria Sorting Group Decision Methods

The objective of the case study presented in this paper is to give an easily interpretable comprehensive assessment of the insulating materials’ sustainability. This is achieved by assigning them to a set of pre-defined and ordered decision classes based on their performances on multiple criteria (Kadziński et al. 2015b). While computing the sorting recommendation, we account for the preferences of a group of experts and stakeholders. This requires implementation of a group decision making framework.

As real-world situations often involve multiple stakeholders, some methods have been proposed to support groups in making collective sorting decisions (Daher and Almeida 2010). These approaches can be distinguished at different levels. In particular, they differ in terms of a preference model employed to represent preferences of the DMs. Furthermore, an underlying classification rule may involve analysis of a single preference model instance or all sets of parameters compatible with the DMs’ preference information. Moreover, sorting methods can be divided with respect to the level on which individual viewpoints are aggregated (Dias and Climaco 2000). Finally, some approaches account for the importance degrees of the involved DMs, while other methods assume that all DMs play the same role in the committee.

Among multiple criteria sorting group decision methods, outranking-based approaches are prevailing. Most decision support systems in this stream incorporate Electre TRI-B (Yu 1992; Roy 1996). For example, Dias and Climaco (2000) proposed an approach that admits each DM to specify imprecise constraints on the parameters of an outranking model, then exploits a set of compatible parameters using robust assignment rule, and finally aggregates individual perspectives in a disjunctive or conjunctive manner (thus, not accounting for the DMs’ powers). The former accepts an assignment if it is justified by at least one DM, whereas the latter confirms some classification only if it is consistent with the preferences of all DMs. In this way, a group may agree on some result even if its members do not share the same model parameters. This idea was extended by Damart et al. (2007) to an interactive preference disaggregation approach that accepts assignment examples provided by different DMs. The method incorporates robustness analysis by deriving for each DM the possible class assignments (confirmed by at least one compatible preference model instance) and guides the group on sorting exemplary alternatives by exhibiting the levels of consensus between the DMs. Analogously, Shen et al. (2016) developed an adaptive approach under intuitionistic fuzzy environment that allows to reach a classification with an acceptable individual and group consensus levels. Moreover, de Morais Bezerra et al. (2017) enriched Electre TRI-B with the tools for visualizing the comparison of individual results and procedures for guiding the changes of model parameters for deriving a better consensus.

Furthermore, Jabeur and Martel (2007) proposed a framework, which derives a collective sorting decision at the output level from the individual non-robust classifications by additionally accounting for the relative importance of group members. Then, Morais et al. (2014) used a stochastic variant of Electre TRI-B, called SMAA-TRI, to consider uncertainty in criteria weights and to derive for each DM the shares of the relevant parameter vectors that assign a given alternative to a certain category. An overview of thus obtained individual results leads to a collective recommendation. Conversely, Cailloux et al. (2012) employed assignment examples provided by multiple DMs for reaching an agreement at the input level. In particular, they proposed some linear programming models for deriving a joint set of boundary class profiles and veto thresholds.

As far as outranking-based sorting approaches incorporating a model typical for PROMETHEE are concerned, Nemery (2008) extended the FlowSort method to group decision making. His proposal derives an assignment for each alternative from its relative comparison (strength and weakness) against the boundary or central class profiles specified by each individual DM. A similar idea was implemented by Lolli et al. (2015) in FlowSort-GDSS. The underlying procedure derives class assignments by comparing comprehensive (global) net flows of alternatives and reference profiles. The proposed sorting rules distinguish between scenarios in which analysis of the individual assignments leads to either univocal or non-unanimous recommendation. Although the viewpoints of different DMs are aggregated at the output level, the method defines some consistency conditions on the preference information (in particular, reference profiles) provided by the individual DMs.

The majority of existing value-based approaches derive a sorting recommendation incorporating robustness analysis and not differentiating between the roles played by the DMs. In particular, the \(\hbox {UTADIS}^\mathrm{GMS}\)-GROUP method (Greco et al. 2012) accounts for the assignment examples provided by each DM and derives collective results that concern two levels of certainty. The first level refers to the necessary and possible consequences of individual preference information, which is typical for Robust Ordinal Regression (ROR) (Greco et al. 2010; Kadziński et al. 2015b). The other level is related to the necessity or possibility of a support that a particular assignment is given in the set of DMs. This method was further adapted by Liu et al. (2015) to account for the uncertain evaluations represented with the evidential reasoning approach, to provide some measures on the agreement between the DMs, and to derive a collective univocal assignment.

Conversely, Kadziński et al. (2013) aimed at a joint representation of assignment examples provided by all DMs by a set of additive value functions and investigating the necessary and possible consequences of applying the latter on the set of alternatives. When there is no value function compatible with preferences of all DMs, some linear programming techniques can be used to remove a minimal subset of inconsistent assignment examples. A similar approach was proposed by Cai et al. (2012), though additionally accounting for the DMs’ priorities. The latter ones intervene in the selection of a representative value function and in resolving inconsistency in the provided assignment examples. These priorities are updated with the progressive preference elicitation process to reflect the preciseness, quantity and consistency of the example decisions supplied by each DM.

Finally, when it comes to using “if ...then ...” decision rules for representing preferences of the DMs, one proposed various extensions of the Dominance-based Rough Set Approach (DRSA) (Greco et al. 2001). These accept preference information in form of individual assignment examples. First, Greco et al. (2006) introduced some concepts (e.g., multi-union and mega-union) related to dominance with respect to minimal profiles of evaluations provided by different DMs. Then, Chen et al. (2012) proposed to aggregate the recommendations suggested by individual linguistic decision rules into an overall assignment be means of a Dempster–Shafer Theory. The crucial concepts incorporated in the DRSA sorting method proposed by Sun and Ma (2015) are a dominance relation on the set of multiple sorting decisions (each provided by an individual DM) and a multi-agent conflict analysis framework. Furthermore, Chakhar and Saad (2012) and Chakhar et al. (2016) illustrated how to combine individual approximations of class unions and derive collective decision rules that permit classification of all alternatives in a way consistent with the judgments of all DMs. These approaches measure the contribution of each expert to the collective assignment in terms of the individual quality of classification. Finally, Kadziński et al. (2016) adapted the principle of ROR to a group decision framework with DRSA, thus considering all sets of rules compatible with the individual assignment examples and combining their indications only at the output level.

In this paper, we propose an outranking-based group decision approach that incorporates Electre TRI-rC. Thus, it derives the assignments by comparing alternatives with the characteristic class profiles rather than with the boundary profiles as in Electre TRI-B. The basic procedure we use takes into account a single preference model instance (incorporating criteria weights derived from the SRF procedure) for each DM and aggregates the individual viewpoints at the output level. While still aggregating the preferences at the output level, we extend the basic framework to offer results of robustness analysis with multiple sets of parameters compatible with the DMs’ value systems. Additionally, we propose a new algorithm for constructing a group compromise ranking of criteria, hence offering aggregation of the individual viewpoints also at the input level. At all stages, we assume that the involved stakeholders have the same importance degrees. Moreover, instead of providing precise assignments, our framework offers acceptability indices indicating the support that is given to the assignment of each alternative to various classes by different DMs and/or preference model instances compatible with their preferences.

3 Multiple Criteria Decision Analysis Method for the Assessment of Insulating Materials

This section describes a three-stage multiple criteria decision analysis method that has been used to evaluate the insulating materials while taking into account preferences of a group of stakeholders. Firstly, we discuss the Electre TRI-rC method (Kadziński et al. 2015b) that has been employed to assign the materials to a set of pre-defined and ordered classes. It incorporates the SRF procedure to compute the criteria weights (Figueira and Roy 2002). The method has been extended to a group decision setting to derive for each material some group class acceptability indices, which indicate the proportion of stakeholders that accept an assignment of the material to a given class. Secondly, we have adapted Stochastic Multi-criteria Acceptability Analysis (SMAA; Lahdelma and Salminen 2001; Tervonen and Figueira 2008; Tervonen et al. 2007) to the context of Electre TRI-rC and SRF procedure. It has been used to conduct robustness analysis (Roy 2010) for the results obtained in the first part, i.e., to validate their certainty while avoiding the arbitrary choice of criteria weights, which is conducted by the SRF procedure. Thirdly, we have proposed an algorithm for constructing a group compromise ranking of criteria based on the orders provided by the individual DMs. This ranking of criteria has been used as an input for SMAA to offer yet another view on the stability of computed results.

Let us use the following notation (Kadziński et al. 2015a):

  • \(A=\left\{ {a_1 ,a_2 ,\ldots ,a_n } \right\} \) is a set of alternatives (insulating materials);

  • \(G=\left\{ {g_1 ,g_2 ,\ldots ,g_m } \right\} \) is a family of evaluation criteria that represent relevant points of view on the quality of assessed alternatives;

  • \(g_j \left( a \right) \) is the performance of alternative a with respect to criterion \(g_j \), \(j=1,\ldots ,m\) (when presenting the method, without loss of generality, we assume that all criteria are of gain type, i.e., the greater the performance, the better);

  • \(C_1 ,C_2 ,\ldots ,C_p \) are the preference ordered classes to which alternatives should be assigned; we assume that \(C_h \) is preferred to \(C_{h-1} \) for \(h=2,\ldots ,p\).

3.1 Assessment of Insulating Materials Within a Group Decision Framework Incorporating Electre TRI-rC and the SRF Procedure

In this section, we present the Electre TRI-rC method (Kadziński et al. 2015b) that is used to assign the materials to a set of pre-defined and ordered classes. The method derives for each material a possibly imprecise assignment by constructing and exploiting an outranking relation S (Figueira et al. 2013). This relation quantifies an outcome of the comparison between the materials and a set of characteristic class profiles (Rezaei et al. 2017). In what follows, we discuss the main steps of the incorporated approach.

Step 1 For each class \(C_h \), provide the most typical (representative) performances on all criteria \(g_j , j=1,\ldots ,m,\) thus specifying the characteristic profiles \(b_h \), \(h=1,\ldots ,p\) (Almeida Dias et al. 2010). Defining such profiles was found intuitive and manageable by the involved experts, which was the main reason for incorporating Electre TRI-rC in the study. The set of all characteristic profiles is denoted by B.

Steps 2–7 are conducted separately for each Decision Maker (\(DM_k \), \(k=1,\ldots ,K)\) in \(\partial ^{\mathrm{K}}=\left\{ {DM_1 ,DM_2 ,\ldots ,DM_K } \right\} \).

Step 2 Determine the weight \(w_j^k \) of each criterion \(g_j \), \(j=1,\ldots ,m,\) using the SRF procedure (Figueira and Roy 2002). This method expects \(DM_k \) to:

  • Assign some importance rank \(l^{k}\left( j \right) \) to each criterion \(g_j \); this is attained by ordering the cards with criteria names from the least to the most important (the greater \(l^{k}\left( j \right) \), the greater \(w_j^k \); some criteria can be assigned the same rank, thus being judged indifferent);

  • Quantify a difference between importance coefficients of the successive groups of criteria judged as indifferent, \(L_s^k \) and \(L_{s+1}^k \), by inserting \(e_s^k \) white (empty) cards between them (the greater \(e_s^k \), the greater the difference between the weights assigned to the criteria contained in \(L_{s+1}^k \) and \(L_s^k )\);

  • Specify ratio \(Z^{k}\) between the importances of the most and the least significant criteria denoted by \(L_{v\left( k \right) }^k \) and \(L_1^k \).

These inputs are used to derive the criteria weights as follows (Figueira and Roy 2002; Corrente et al. 2016):

$$\begin{aligned} w_j^k =1+\frac{\left( {Z^{k}-1} \right) \left[ {l^{k}\left( j \right) -1+\mathop \sum \nolimits _{s=1}^{l\left( j \right) -1} e_s^k } \right] }{v\left( k \right) -1+\mathop \sum \nolimits _{s=1}^{v-1} e_s^k }. \end{aligned}$$

Steps 3–6 are conducted for each pair consisting of alternative a and profile \(b_h \).

Step 3 For each criterion \(g_j \) compute a marginal concordance index \(c_j^k \left( {a,b_h } \right) \) defined as follows:

$$\begin{aligned} c_j^k \left( {a,b_h } \right) =\left\{ {{\begin{array}{l} {1 \quad \hbox { if }g_j \left( a \right) -g_j \left( {b_h } \right) \ge 0,} \\ {0 \quad \hbox { if }g_j \left( a \right) -g_j \left( {b_h } \right) <0.} \\ \end{array} }} \right. \end{aligned}$$

The index quantifies a degree to which a is at least as good as \(b_h \) on \(g_j \). Let us remark that in our study the experts defined the performances of characteristic profiles on all criteria by selecting them from the performances of the considered materials. This facilitated the preference elicitation process when dealing with a set of criteria with heterogeneous performance scales. In this perspective, when comparing the alternatives with the characteristic class profiles, we decided to exploit only the ordinal character of criteria and not use the discrimination (indifference and preference) thresholds, which can be, in general, employed in Electre. That is, in our application, the outranking of alternative a over profile \(b_h \) on \(g_j \) means that \(g_j \left( a \right) \) is at least as good as the most typical (representative) performance for class \(C_h \) on \(g_j \) of some considered material.

Step 4 Compute a comprehensive concordance index \(\sigma ^{k}\left( {a,b_h } \right) \) defined in the following way:

$$\begin{aligned} \sigma ^{k}\left( {a,b_h } \right) =\frac{\mathop \sum \nolimits _{j=1}^m w_j^k c_j^k \left( {a,b_h } \right) }{\mathop \sum \nolimits _{j=1}^m w_j^k }. \end{aligned}$$

The index quantifies a joint strength of a subset of criteria supporting the hypothesis about a  outranking \(b_h \) (\(aS^{k}b_h )\). Note that in our study, no criterion was judged strong enough to be attributed a power to veto against the outranking relation. Thus, no discordance effect has been considered.

Step 5 Specify the cutting level \(\lambda ^{k}\) (also called majority threshold), and compare \(\sigma ^{k}\left( {a,b_h } \right) \) with \(\lambda ^{k}\) to verify the truth of a crisp outranking relation \(aS^{k}b_h \) in the following way:

$$\begin{aligned} \sigma ^{k}\left( {a,b_h } \right) \ge \lambda ^{k} \Rightarrow aS^{k}b_h . \end{aligned}$$

The truth of relation \(b_h S^{k}a\) can be verified analogously.

Step 6 Use information on the truth or falsity of \(aS^{k}b_h \) and \(b_h S^{k}a\) to check the validity of:

  • a being preferred to \(b_h \) (\(aS^{k}b_h \wedge not\left( {b_h S^{k}a} \right) \Rightarrow a\succ _k b_h )\);

  • \(b_h \) being preferred to a (\(b_h S^{k}a\wedge not\left( {aS^{k}b_h } \right) \Rightarrow b_h \succ _k a)\);

  • a being indifferent with \(b_h \) (\(aS^{k}b_h \wedge b_h S^{k}a \Rightarrow a\sim _k b_h )\);

  • a being incomparable with \(b_h \) (\(not\left( {aS^{k}b_h } \right) \wedge not\left( {b_h S^{k}a} \right) \Rightarrow a?_k b_h )\).

Step 7 For alternative a determine its desired class interval \(C^{k}\left( a \right) =\left[ {C_L^k \left( a \right) ,C_R^k \left( a \right) } \right] \) by applying the assignment rules of ELECTRE TRI-rC (Kadziński et al. 2015b). To compute the worst class \(C_L^k \left( a \right) \), compare a successively to \(b_h \), for \(h=p-1,\ldots ,1\), seeking the first (i.e., the best) characteristic profile \(b_h \) such that:

$$\begin{aligned} a\succ _k b_h \wedge \sigma ^{k}\left( {a,b_{h+1} } \right) >\sigma ^{k}\left( {b_h ,a} \right) , \end{aligned}$$

and select \(C_L^k \left( a \right) =C_{h+1} \). When no such a profile is found, \(C_L^k \left( a \right) =C_1 \).

To compute the best class \(C_R^k \left( a \right) \), compare a successively to \(b_h \), for \(h=2,\ldots ,p\), seeking the first (i.e., the worst) characteristic profile \(b_h \) such that:

$$\begin{aligned} b_h \succ _k a\wedge \sigma ^{k}\left( {b_{h-1} ,a} \right) >\sigma ^{k}\left( {a,b_h } \right) , \end{aligned}$$

and select \(C_R^k \left( a \right) =C_{h-1} \). In case no such a profile is found, \(C_R^k \left( a \right) =C_p \).

Step 8 Combine the individual class assignments for all DMs into group class acceptability indices \(E^{\partial }\left( {a,h} \right) \) (Damart et al. 2007; Kadziński et al. 2016). These are defined as the proportion of DMs (stakeholders) that accept an assignment of alternative a to class \(C_h \), i.e.:

$$\begin{aligned} E^{\partial ^{\mathrm{K}}}\left( {a,h} \right) =\frac{\mathop \sum \nolimits _{k=1}^K E^{k}\left( {a,h} \right) }{K}, \end{aligned}$$

where for \(k=1,\ldots ,K\):

$$\begin{aligned} E^{k}\left( {a,h} \right) =\left\{ {{\begin{array}{l} {1\quad \,if\,C_h \in C^{k}\left( a \right) ,} \\ {0\quad \,if\,C_h \notin C^{k}\left( a \right) .} \\ \end{array} }} \right. \end{aligned}$$

This measure indicates a cumulative support given to the assignment of a to \(C_h \) by all group members.

3.2 Stochastic Multi-criteria Acceptability Analysis with Electre TRI-rC

The SRF procedure derives the precise weight values from the ranking of criteria, intensities of preference, and ratio between the most and the least important criteria provided by \(DM_{k}\) applying some arbitrary rule (Figueira and Roy 2002). However, there exist multiple weight vectors compatible with such incomplete preference information. Recently, many researchers have raised the robustness concern in view of the SRF procedure to quantify the impact of uncertainty in the selection of an arbitrary weight vector on the stability of computed recommendation. In particular, Siskos and Tsotsolas (2015) proposed a set of robustness rules for the SRF procedure to obtain tangible and adequately supported results. Then, Govindan et al. (2017) suggested to exploit the whole set of compatible weight vectors to construct the necessary and possible results being confirmed by, respectively, all or at least one compatible vector. Further, Corrente et al. (2017) adapted the stochastic analysis of recommendation with the SRF procedure to the context of Electre III. We follow the latter research direction and integrate Stochastic Multi-criteria Acceptability Analysis (Lahdelma and Salminen 2001; Tervonen et al. 2007) to handle possibly imprecise weight values compatible with the ranking of criteria and to derive robust recommendation with Electre TRI-rC.

SMAA applies the Monte Carlo simulation to provide each DM with the acceptability indices which measure the variety of different preferences (in particular, weight vectors) that confirm the validity of particular elements of the recommendation. In our case, the space \(w^{k}(SRF)\) of weight vectors compatible with preferences of \(DM_{k}\) is defined by the following constraint set \(E^{k}(SRF)\) :

$$\begin{aligned} \left. \begin{array}{c} \left[ {O1} \right] ~w_{i}^{k}> w_{j}^{k} ,~{\text {for all}}~g_{i} \in L_{t}^{k} ,~g_{j} \in L_{s}^{k} ~~{\text {and}}~t> s,\\ \left[ {O2} \right] ~w_{i}^{k} = w_{j}^{k} ,~{\text {for all}}~g_{i} ,~g_{j} \in L_{s}^{k} ,~\\ \left[ {O3} \right] ~w_{i}^{k} = Z^{k} w_{j}^{k} ,~{\text {for all}}~g_{i} \in L_{{v\left( k \right) }}^{k} ,~g_{j} \in L_{1}^{k} ~, \\ \left[ {O4} \right] ~w_{{j + 1}}^{k} - w_{j}^{k}> w_{{p + 1}}^{k} - w_{p}^{k} ,~{\text {if}}~e_{j}^{k} > e_{p}^{k} , \\ \left[ {O5} \right] ~\mathop \sum \nolimits _{{j = 1}}^{m} w_{j}^{k} = 1, \end{array}\right\} \, E^{k} \left( {SRF} \right) \end{aligned}$$

where the interpretation of different constraints is as follows:

  • \(\left[ {O1} \right] \) ensures that criteria ranked better by \(DM_k\) will be assigned greater weight;

  • \(\left[ {O2} \right] \) guarantees that criteria deemed indifferent by \(DM_k \) will be assigned equal weights;

  • \(\left[ {O3} \right] \) sets the ratio Z between weights of the most and the least significant criteria;

  • \(\left[ {O4} \right] \) respects the intensities of preference for different pairs of criteria that have been quantified with the number of inserted empty cards;

  • \(\left[ {O5} \right] \) normalizes the weights.

These constraints also ensure that all weights are positive. For each \(DM_k \), each weight vector \(w\in w^{k}\left( {SRF} \right) \) and each alternative \(a\in A\), we compute the resulting class assignment \(C_w^k \left( a \right) =\left[ {C_{w,L}^k \left( a \right) ,C_{w,R}^k \left( a \right) } \right] \) with Electre TRI-rC.

We define the class range stochastic acceptability index \(CRSAI^{k}\left( {a,\left[ {L,R} \right] } \right) \) (Kadziński et al. 2013) on a range of classes \(\left[ {C_L^k \left( a \right) ,\ldots ,C_R^k \left( a \right) } \right] \) with \(L\le R\) as the proportion of compatible weights \(w\in w^{k}\left( {SRF} \right) \) that assign alternative a precisely to the range of classes \(\left[ {C_L^k \left( a \right) ,\ldots ,C_R^k \left( a \right) } \right] \). Formally, the index is computed as follows:

$$\begin{aligned} CRSAI^{k}\left( {a,\left[ {h_L ,h_R } \right] } \right) =\mathop \smallint \nolimits _{w\in w^{k}\left( {SRF} \right) }m\left( {w,a,\left[ {h_L ,h_R } \right] } \right) dw, \end{aligned}$$

where \(m\left( {w,a,\left[ {h_L ,h_R } \right] } \right) \) is the class range membership function:

$$\begin{aligned} m\left( {w,a,\left[ {h_L ,h_R } \right] } \right) =\left\{ \begin{array}{ll} 1,&{}\quad if\,C_{w,L}^k \left( a \right) =C_{h_L } \hbox { and } C_{w,R}^k \left( a \right) =C_{h_R ,} \\ 0,&{}\quad \hbox { otherwise}. \\ \end{array} \right. \end{aligned}$$

Further, we compute the proportion of \(w\in w^{k}\left( {SRF} \right) \) for which \(C_h \) is within \(\left[ {C_{w,L}^k \left( a \right) ,C_{w,R}^k \left( a \right) } \right] \), i.e., the proportion of weights that either precisely or imprecisely assign a to \(C_h \) (Kadziński and Tervonen 2013; Kadziński et al. 2014). Let us define such a cumulative class stochastic acceptability index \(CuCSAI^{k}\left( {a,h} \right) \) as:

$$\begin{aligned} CuCSAI^{k}\left( {a,h} \right) =\mathop \sum \nolimits _{\left[ {h_L ,h_R } \right] :h\in \left[ {h_L ,h_R } \right] } CRSAI^{k}\left( {a,\left[ {h_L ,h_R } \right] } \right) . \end{aligned}$$

We estimate CRSAIs with acceptable error bounds by sampling the space \(w^{k}\left( {SRF} \right) \) with the Hit-And-Run (HAR) algorithm (Tervonen et al. 2013). Overall, \(CRSAI^{k}( a,[ h_L ,h_R ] )\) and \(CuCSAI^{k}\left( {a,h} \right) \) can be interpreted as a support given by \(DM_k \) to the assignment of a to, respectively, \(\left[ {C_{h_L } ,C_{h_R } } \right] \) or \(C_h \).

To measure a cumulative support given to the assignment of a to \(C_h \) by all DMs in \(\partial ^{\mathrm{K}}\), we consider a cumulative group class stochastic acceptability index \(CuCSAI^{\partial ^{\mathrm{K}}}\left( {a,h} \right) \), defined as follows (Kadziński et al. 2016, 2018):

$$\begin{aligned} CuCSAI^{\partial ^{\mathrm{K}}}\left( {a,h} \right) =\frac{\mathop \sum \nolimits _{k=1}^K CuCSAI^{k}\left( {a,h} \right) }{K}. \end{aligned}$$

3.3 Selection of a Group Compromise Ranking of Criteria

In this section, we introduce a procedure for deriving a compromise complete ranking of criteria based on the rankings provided individually by each \(DM_k\) within the SRF procedure. The procedure builds on the algorithm that was introduced by Govindan et al. (2017) for constructing a utilitarian ranking of alternatives. Hence, we adopt an idea of minimizing a sum of of distances between the compromise ranking and all individual rankings.

When considering a complete ranking of criteria for \(DM_k \), for each pair \((g_j ,g_l )\) one of the three relations holds: \(g_j \) is preferred to \(g_l \quad (g_j \succ _k g_l )\), or \(g_j \) is indifferent with \(g_l \quad (g_j \sim _k g_l )\), or \(g_l \) is preferred to \(g_j \quad (g_j \prec _k g_l )\). Let \(R_{k^{\prime }}^{jl} \) and \(R_{k^{\prime \prime }}^{jl} \) denote the relations holding between \(g_j \) and \(g_l \) in the rankings provided by, respectively, \(DM_{k^{\prime }} \) and \(DM_{k^{\prime \prime }} \) (e.g., \(R_{k^{\prime }}^{jl} \) is \(\succ _{k^{\prime }}^{jl} \) or \(\sim _{k^{\prime }}^{jl} \) or \(\prec _{k^{\prime }}^{jl} )\). The distances \(\delta (R_{k^{\prime }}^{jl} ,R_{k^{\prime \prime }}^{jl} )\) between \(R_{k^{\prime }}^{jl}\) and \(R_{k^{\prime \prime }}^{jl} \) are provided in Table 1 (for a detailed justification of these values, see Roy and Słowiński 1993). A distance between two rankings of criteria provided by \(DM_{k^{\prime }} \) and \(DM_{k^{\prime \prime }} \) involving all ordered pairs of criteria \((g_j ,g_l )\) is defined as follows:

$$\begin{aligned} \mathop \sum \limits _{j,l:j<l} \delta \left( {R_{k^{\prime }}^{jl} ,R_{k^{\prime \prime }}^{jl} } \right) . \end{aligned}$$
Table 1 Definition of distances \(\delta \left( R_{k^{\prime }}^{jl} ,R_{k^{\prime \prime }}^{jl} \right) \) between different pairwise relations
Full size table

In what follows, we present a Binary Linear Program (BLP) for constructing a compromise ranking of criteria for group \(\partial ^{\mathrm{K}}\) involving K DMs. Following Govindan et al. (2017), for each pair of criteria \((g_j ,g_l )\), we introduce two binary variables \(p_\partial ^{jl} \) and \(i_\partial ^{jl} \) (see constraint \(\left[ {R1} \right] \) in \(E^{\partial }\left( {SFR} \right) )\) with the following interpretation:

  • \(p_\partial ^{jl} \) represents a weak preference of \(g_j \) over \(g_l \) in the compromise ranking (i.e., in case \(p_\partial ^{jl} =1\), then \(g_j \succ _\partial g_l \) or \(g_j \sim _\partial g_l )\); note that \(p_\partial ^{jl} \) and \(p_\partial ^{lj} \) can be used to instantiate one of the three relations \(\succ _\partial ^{jl} \), \(\sim _\partial ^{jl} \), or \(\prec _\partial ^{jl} \) for \(g_j \) and \(g_l \); that is, if \(p_\partial ^{jl} \hbox {}=1\) and \(p_\partial ^{lj} =0\), then \(g_j \succ _\partial g_l \); if \(p_\partial ^{jl} =0\) and \(p_\partial ^{lj} =1\), then \(g_j \prec _\partial g_l \); if \(p_\partial ^{jl} =1\) and \(p_\partial ^{lj} =1\), then \(g_j \sim _\partial g_l \);

  • \(i_\partial ^{jl} \) represents an indifference \(\sim _\partial \) between \(g_j \) and \(g_l \) (i.e., in case \(p_\partial ^{jl} =1\) and \(p_\partial ^{lj} =1\), then \(i_\partial ^{jl} =1\) and \(g_j \sim _\partial g_l \); see [R3]).

Since we impose completeness and transitivity on a weak preference relation, we require that \(p_\partial ^{jl} \hbox {}=1\) or \(p_\partial ^{lj} =1\) (see \(\left[ {R2} \right] )\) and that \(p_\partial ^{jr} =1\) and \(p_\partial ^{rl} =1\) imply \(p_\partial ^{jl} =1\) (see \(\left[ {R4} \right] )\). When constructing a utilitarian complete ranking of criteria, we aim at minimizing a comprehensive distance between relations (\(\succ _\partial \), \(\prec _\partial \), or \(\sim _\partial )\) instantiated for all pairs of criteria in the compromise ranking and relations observed for these pairs in the individual DMs’ rankings (for \(DM_k \), the relation between \(g_j \) and \(g_l \) (\(j<l)\) is denoted by \(R_k^{jl} )\):

$$\begin{aligned}&\min \mathop \sum \limits _{j,l:j<l} \mathop \sum \limits _{k=1}^K \left[ p_\partial ^{jl} \delta \left( {R_k^{jl} ,\succ _\partial ^{jl} } \right) +p_\partial ^{lj} \delta \left( {R_k^{jl} ,\prec _\partial ^{jl} } \right) \right. \\&\left. \quad +i_\partial ^{jl} \left[ {\delta \left( {R_k^{jl} ,\sim _\partial ^{jl} } \right) -\delta \left( {R_k^{jl} ,\succ _\partial ^{jl} } \right) -\delta \left( {R_k^{jl} ,\prec _\partial ^{jl} } \right) } \right] \right] \\&\left. {{\begin{array}{l} {{\begin{array}{l} {\left[ {RI} \right] \hbox { for all }j,l=1,2,\ldots ,m:j\ne l} \\ {\left[ {R1} \right] \hbox {}p_\partial ^{jl} ,i_\partial ^{jl} \in \left\{ {0,1} \right\} ,} \\ {\left[ {R2} \right] p_\partial ^{jl} +p_\partial ^{lj} \ge 1,} \\ \end{array} }} \\ {{\begin{array}{l} {\left[ {R3} \right] i_\partial ^{jl} =p_\partial ^{jl} +p_\partial ^{lj} -1,} \\ {\left[ {RII} \right] \hbox { for all }j,l,r=1,2,\ldots ,m:j\ne l\ne r} \\ {\left[ {R4} \right] p_\partial ^{jl} \ge p_\partial ^{jr} +p_\partial ^{rl} -1.5.} \\ \end{array} }} \\ \end{array} }} \right\} E^{\partial }\left( {SFR} \right) \end{aligned}$$

If \(g_j \succ _\partial g_l \) (\(p_\partial ^{jl} =1, \quad p_\partial ^{lj} =0\), and \(i_\partial ^{jl} =0)\), \(g_j \prec _\partial g_l \) (\(p_\partial ^{jl} =0, \quad p_\partial ^{lj} =1\), \(i_\partial ^{jl} =0)\), or \(g_j \sim _\partial g_l \) (\(p_\partial ^{jl} =1, \quad p_\partial ^{lj} =1\), \(i_\partial ^{jl} =1)\) has been instantiated in the compromise ranking, it contributes with, respectively, \(\mathop \sum \nolimits _{k=1}^K \delta \left( {R_k^{jl} ,\succ _\partial ^{jl} } \right) \), \(\mathop \sum \nolimits _{k=1}^K \delta \left( {R_k^{jl} ,\prec _\partial ^{jl} } \right) \) or \(\mathop \sum \nolimits _{k=1}^K \delta \left( {R_k^{jl} ,\sim _\partial ^{jl} } \right) \) to a value of the objective function (for a detailed explanation, see Govindan et al. 2017).

Once a group compromise ranking of criteria is constructed, we conduct robustness analysis with SMAA in the same way as described in the previous section for an individual DM. This leads us to deriving cumulative group compromise class stochastic acceptability indices \(CuCCSAI^{\partial ^{\mathrm{K}}}\left( {a,h} \right) \).

3.4 Decision Aiding with the Proposed Approach

Multiple criteria sorting decisions can be aided with the proposed group decision making framework through the process illustrated in Fig. 1. It starts with specifying the sets of alternatives, criteria, and ordered classes as well as the alternatives’ evaluations (performances) on the criteria.

Then, the preference information is elicited from the involved experts and/or stakeholders. Each stakeholder is required to provide a cutting level as well as a ranking of criteria that incorporates the intensities of preference and the ratio between the importance coefficients of the most and the least significant criteria, as required by the SRF procedure. Moreover, the experts are expected to define a characteristic profile for each class. In our study, the profiles were agreed by multiple experts, but, in general, the methodological framework admits that each stakeholder provides his/her individual set of profiles.

Fig. 1
figure 1

Decision aiding process with the proposed group decision methodological framework

Full size image

Further, the method derives three types of results. These indicate a support that is given to the assignment of considered alternatives to different classes via the application of Electre TRI-rC for different sets of weights and cutting levels compatible with the preferences of the involved experts. In two cases, the preferences of the individual stakeholders are aggregated only at the output level. Depending on whether these individual preferences are processed using the SRF procedure or the Monte Carlo simulation, the method computes, respectively, group class acceptability indices or cumulative group class stochastic acceptability indices. In the third case, the preferences are aggregated at the input level by constructing a group compromise ranking of criteria. Then, the method applies SMAA to derive cumulative group compromise class stochastic acceptability indices.

Finally, these three types of outcomes should be analyzed and combined into the recommended assignments. This is straightforward in case the support given to the assignment of alternatives to decision classes by different results is similar. In case of ambiguous indications by different procedures, the inconsistency needs to be raised by a decision analyst.

Obviously, it is not required to use all three types of procedures and respective results for each study. This may be useful when offering different viewpoints on the robustness of sorting recommendation is desired. Otherwise, one can employ just a single procedure for processing the experts’ preferences depending on whether they should be aggregated at the input or output level and whether the robustness analysis should be incorporated into a particular study.

4 Results of Multiple Criteria Assessment of Insulating Materials with the Outranking Preference Model and Characteristic Class Profiles

The study aims at evaluating overall sustainability of coating materials used in buildings retrofitting. We consider 13 materials listed in Table 2 (they are denoted by \(A=\left\{ {a_1 ,a_2 ,\ldots ,a_{13} } \right\} )\). All materials having a thickness of 15cm were placed internally on the roof of a model building typical for central Italy, and evaluated from the socio-economic and environmental viewpoints. The six relevant criteria which have been used to assess the materials are: hour of discomfort (\(g_1 \); DH),\(\hbox {CO}_{2}\) avoidance (\(g_2\)); Net Present Value (\(g_3\); NPV), human health (\(g_4\)); ecosystem quality (\(g_5\)), and consumed resources (\(g_6\)). In what follows, we explain their meaning.

Discomfort degree Hour (\(g_1 \); the less, the better) evaluates a thermal performance of a building on an annual basis (CEN 2007) in accordance with the EN 15251 standard. Thus defined, it serves as a measure of comfort. The performance on \(g_1 \) is quantified as an overall time during which the temperature falls outside the second comfort category that was considered in the study (Carlucci and Pagliano 2012), and then weighing it by how much the limit has been exceeded. For this purpose, we have used the following equation:

$$\begin{aligned} g_1 \left( a \right) =\mathop \sum \limits _{i=1}^{8760} \frac{10}{60}\left| {CC_2 -OT_i } \right| \end{aligned}$$

where \(CC_2 \) is the lower or upper limit of the assumed comfort category, \(OT_i \) is the operative temperature at hour ì, and the multiplier \(\frac{10}{60}\) refers to an employed time step of 10 minutes.

\(\hbox {CO}_{2}\) avoidance (\(g_2 \); the more, the better) measures the energy saved during the building life by using a particular insulating material when compared to the case of no insulation in the following way:

$$\begin{aligned} g_2 \left( a \right) =\frac{ES*277.78*406.31}{10^{6}} \end{aligned}$$

where ES is the estimated Energy Saved in GJ at time t with a time horizon of 25 years, 277.78 is a conversion factor to GJ in kWH, while 406.31 is the conversion factor for Italy from kWH to kg of \(\hbox {CO}_{2}\) per year (EIA, 2015). Therefore, the \(\hbox {CO}_{2}\) avoided refers only to the use phase, which is not considered in the LCA study.

Net present value (\(g_3\); the more, the better) is the difference between the present values of cash outflows and inflows. On one hand, the outflows involve Primary Energy Input (PEI) cost, installation cost I at time t = 0, and the dismissing cost \(EL_T \) after the lifespan T of the investment (25 years). On the other hand, the inflows refer to the Cost of Energy Saved \(ES_t \) in different time periods t. Overall, we have computed NPV as follows:

$$\begin{aligned} g_3 \left( a \right) =-PEI-I+\mathop \sum \limits _{t=0}^T \frac{ES_t }{\left( {1+i} \right) ^{t}}-\frac{EL_T }{\left( {1+i} \right) ^{T}} \end{aligned}$$

where i is the discount rate. For a detailed justification of this measure, see Menconi and Grohmann (2014). Thus defined, NPV can be seen as an outcome of Life Cycle Costing, which is an economic methodology for assessing the profitability of using different alternatives by taking into account the costs they incur at different stages of a life cycle (e.g., construction, operations, and maintenance).

For the assessment of environmental impacts, we have used the Eco-indicator 99 method (Goedkoop and Spriensma 2001) implemented in the SimaPro software (Product Ecology Consultants 1990). The method aggregates the results of Life Cycle Assessment into a set of parameters that can be interpreted as damage categories. In general, LCA is useful for identifying the environmental implications of a given alternative through the quantification of consumed resources (e.g., energy, raw materials, water) and related emissions (e.g., emissions into the air, water and soil, waste and co-products) (Paolotti et al. 2017). We used the following three environmental Eco-indicators expressed on a dedicated point scale:

  • Human health (\(g_4 \); the less, the better) which is derived from the analysis of the following normalized impact categories: carcinogens, respiratory organics and inorganics, climate change, radiation, and ozone layer;

  • Ecosystem quality (\(g_5 \); the less, the better) which is made up by the following three normalized impact categories: ecotoxicity, acidification/eutrophication, and land use;

  • Resources (\(g_6 \); the less, the better) which aggregates two normalized impact categories: minerals and fossil fuels.

The LCA focused on the production phase, starting from the production of a raw material to the obtaining of its complete version. We omitted the use and disposal phases, hence implementing an LCA “from cradle to gate” (Paolotti et al. 2016). All the impacts were calculated considering a functional unit of 1 m\(^{3}\) of insulating material.

Table 2 Performances of 13 insulating materials with respect to 6 criteria
Full size table

The performances of 13 insulating materials with respect to 6 criteria are provided in Table 2. For all materials but hemp fibres, Ecoinvent Database (Ecoinvent 2010) was used as a source of foreground and background data related to both production and assembly processes as well as to the transport, electricity and fuel consumption. Instead, for the hemp processes the underlying data was derived from Zampori et al. (2013).

The objective of the case study is to give an easily interpretable comprehensive assessment of the materials’ sustainability. This is achieved by assigning them to a set of three pre-defined and ordered classes: \(C_1\) (low sustainability), \(C_2\) (medium sustainability), and \(C_3\) (high sustainability).

The study involved elicitation of preferences from the two groups of stakeholders. On one hand, a characteristic profile \(b_h\) for each class \(C_h\), \(h=1,2,3\), has been collectively specified by the experts from the university-based engineering team specialized in the materials and retrofitting of rural buildings. On the other hand, the preferences on the importance of individual criteria have been elicited individually from multiple stakeholders who were owners of rural buildings interested in a renovation of their houses for improving the energetic performance. Thus, they can be perceived as potential consumers of the insulating materials.

When it comes to the characteristic profiles, the experts decided to define them by indicating one of the performances observed in the set of materials. The consensus between the experts on the most typical performance levels for each class has been reached during an interactive focus group. These levels are summarized in Table 3.

Table 3 Performances of the characteristic profiles for three classes
Full size table

4.1 Results of Multiple Criteria Assessment of the Insulating Materials Within a Group Decision Framework Incorporating Electre TRI-rC and the SRF Procedure

The weights representing the importance of individual criteria have been elicited from the rural buildings’ owner. In what follows, we call them stakeholders. Overall, we approached 63 owners by explaining them the characteristics of different materials, the interpretation of all criteria and their relation to different phases of the materials’ life cycle. Among them, 38 stakeholders (let us denote them by \(\partial ^{\mathrm{K}}=\left\{ {DM_1 ,DM_2 ,\ldots ,DM_{38}}\right\} )\) claimed to understand the meaning and role of different criteria, and expressing their willingness to provide preferences on the criteria importance.

In Table 4, we present the incomplete preference information required by the SRF procedure, which was provided by three selected stakeholders. We also report the computed weights \(w_j^k \) and cutting level \(\lambda ^{k}\). All stakeholders agreed that \(\lambda ^{k}\) should be equal to the sum of weights of the three most important criteria. The complete data for all group members is provided in the supplementary material available as an e-Appendix (the same remark applies to the results discussed in the following sections).

Table 4 The order of cards with criteria names (ranks \(l^{k}\left( j \right) )\), white cards \(e_s^k \), and ratio \(Z^{k}\) provided by the three selected DMs in the SRF procedure, the weights \(w_j^k \) derived from the SRF procedure, and the cutting level \(\lambda ^{k}\)
Full size table

The results of a comprehensive comparison between 13 materials and 3 characteristic profiles are quantified with the comprehensive concordance indices. In Table 5, we present such indices for four exemplary materials for \(DM_1 \). Table 5 exhibits also the justification of delivered assignment for the exemplary materials. For instance, a precise assignment of \(a_{6}\) to \(C_1 \) can be explained with \(b_2 \) being preferred to \(a_{6}\) and there existing sufficiently strong support in favor of \(b_{1}\) outranking \(a_6 \) (\(\sigma ^{1}\left( {a_6 ,b_2 } \right) =0.000<\sigma ^{1}\left( {b_1 ,a_6 } \right) =0.524)\).

Table 5 Credibility indices and class assignments obtained with ELECTRE TRI-rC for four exemplary materials for \(DM_1 \) (cutting level \(\lambda ^{1}=0.714)\)
Full size table

In Table 6, we report the assignments obtained for all materials for different DMs. In particular, for \(DM_1\) there are 6 materials assigned to the best class (\(a_{5}, a_{7}, a_{8}, a_{11}, a_{12}, a_{13})\), 3 materials whose quality is evaluated as medium (\(a_{1}, a_{2}, a_{3})\), and 4 materials judged as bad (\(a_{4},a_{6}, a_{9}, a_{10})\). The assignments for \(DM_5\) are the same except for \(a_{4}\) being imprecisely assigned to [\(C_{1}\), \(C_{2}\)].

Table 6 Class assignments obtained with Electre TRI-rC for all materials and different stakeholders
Full size table

The spaces of consensus and disagreement with respect to the assignments obtained for all DMs are quantified with the group class acceptability indices \(E^{\partial }\left( {a,h} \right) \) (see Table 7). For example, for \(a_{1}\) none stakeholder confirmed its assignment to the worst class \(C_{1}\), 36 out of 38 stakeholders supported its assignment to the medium class \(C_{2}\), and 3 stakeholders suggested the assignment of \(a_{1}\) to the best class \(C_{3}\). These numbers have been translated to the following group acceptability indices: \(E^{\partial }\left( {a_1 ,1} \right) =\frac{0}{38}=0\), \(E^{\partial }\left( {a_1 ,2} \right) =\frac{36}{38}=0.95\), and \(E^{\partial }\left( {a_1 ,3} \right) =\frac{3}{38}=0\).08. On the contrary, for \(a_{2}\) all stakeholders agreed with respect to its assignment to \(C_{2}\) (\(E^{\partial }\left( {a_2 ,2} \right) =\frac{38}{38}=1.0)\), while the results obtained for 6 of them additionally indicated hesitation in terms of its assignment to \(C_{1 }(E^{\partial }\left( {a_2 ,1} \right) =\frac{6}{38}=0.16)\).

Table 7 Group class acceptability indices \(E^{\partial }\left( {a,h} \right) \)
Full size table

The analysis of \(E^{\partial }\left( {a,h} \right) \) leads to indicating the assignments which are necessary (in case \(E^{\partial }\left( {a,h} \right) =1)\), possible (if \(E^{\partial }\left( {a,h} \right) >0)\), and impossible (if \(E^{\partial }\left( {a,h} \right) =0)\) in terms of the support they are provided in the group of stakeholders. Additionally, these results clearly indicate the most and the least probable assignments. In particular, for each material we are able to indicate the class with the greatest support among all stakeholders. It is \(C_{1}\) for \(a_{6},a_{9}\) and \(a_{10}\), \(C_{2}\) for \(a_{1}, a_{2}, a_{3}\) and \(a_{4}\), or \(C_{3}\) for \(a_{5}, a_{7}, a_{8}\), \(a_{11}, a_{12}\), and \(a_{13}\). The support which is given to the assignment of the materials to other classes is significantly smaller. For clarity of presentation, in all tables exhibiting stochastic acceptability indices (Tables 7, 8, 9 and 11), the text in bold indicates the class with the greatest support for a given material.

4.2 Results of Stochastic Multi-criteria Acceptability Analysis with Electre TRI-rC

To validate the recommendation for insulating materials against the arbitrary choice of weights conducted with the SRF procedure, we applied SMAA. For each stakeholder, we considered a sample of 10000 uniformly distributed weight vectors compatible with the ranking of criteria (s)he provided within the SRF procedure.

The analysis of class range stochastic acceptability indices \(CRSAI^{k}\left( {a,\left[ {L,R} \right] } \right) \) and cumulative class stochastic acceptability indices \(CuCSAI^{k}\left( {a,h} \right) \) indicates the potential variability of the recommendation that can be obtained for each DM for different compatible weight vectors. For illustrative purpose, in Table 8 we provide these indices for \(DM_1 \). For some materials, all compatible weight vectors confirm the same assignment. These parts of the recommendation can be deemed as robust (e.g., \(CRSAI^{1}\left( {a_3 ,\left[ {2,2} \right] } \right) =1\) or \(CRSAI^{1}\left( {a_9 ,\left[ {1,1} \right] } \right) =1)\). The same conclusion can be derived from the analysis of the indices which are equal to zero, thus excluding the possibility of the respective assignment. Further, for some other materials the acceptability indices express hesitation with respect to the recommended class though often offering greater support to a particular assignment. For example, although both \(C_{2}\) and \(C_{3}\) are possible for \(a_1 \), the probability of the previous (\(C_{2})\) is significantly greater than of the latter (\(C_{3})\). Finally, the recommendation obtained for various compatible weight vectors can be different, but their intersection can be non-empty. Then, a robust recommendation is confirmed with \(CuCSAI^{1}\left( {a,h} \right) \)=1. It is the case for, e.g., \(a_{13}\) which is assigned imprecisely to [\(C_{2}\),\(C_{3}\)] or precisely to \(C_{3}\), thus always confirming \(C_{3}\) as the possible assignment.

Table 8 Class range stochastic acceptability indices \(CRSAI^{1}\left( {a,\left[ {L,R} \right] } \right) \) and cumulative class stochastic acceptability indices \(CuCSAI^{1}\left( {a,h} \right) \) for all materials for \(DM_1 \)
Full size table

When it comes to a group decision perspective, the cumulative group class stochastic acceptability indices \(CuCSAI^{\partial ^{\mathrm{K}}}\left( {a,h} \right) \) are presented in Table 9. Their values are very similar to the group class acceptability indices \(E^{\partial }\left( {a,h} \right) \) reported in the previous section. The main differences concern a slightly increased support given to the minority class for some alternatives (see, e.g., \(a_{1}\) to \(C_{3}\), or \(a_{2}\) to \(C_{1}\), \(a_{8}\), and \(a_{12}\) to \(C_{2})\).

Overall, the prevailing assignments for all materials are the same as in Sect. 4.1. In this regard, let us emphasize that \(CuCSAI^{\partial ^{\mathrm{K}}}\left( {a,h} \right) =1\) (see, e.g., \(a_{10}\) to \(C_{1}\), \(a_{3}\) to \(C_{2}\), or \(a_{5}\) to \(C_{3})\) confirms an agreement with respect to assignment of a to \(C_{h}\) for all weight vectors compatible with preferences of all stakeholders. Thus, such a recommendation needs to be treated with certainty. Conversely, \(CuCSAI^{\partial ^{\mathrm{K}}}\left( {a,h} \right) =0\) (e.g., \(a_{2}\) to \(C_{3}\), \(a_{3}\) to \(C_{1}\), or \(a_{9}\) to \(C_{3})\) indicates the no classification model of any stakeholder confirmed the respective assignment. This makes it excluded from the potential recommendation.

Table 9 Cumulative group class stochastic acceptability indices \(CuCSAI^{\partial ^{\mathrm{K}}}\left( {a,h} \right) \) for all materials
Full size table

4.3 Results of Stochastic Multi-criteria Acceptability Analysis for a Group Compromise Ranking of Criteria

The results presented in the previous sections were derived by aggregating the outcomes obtained individually for each stakeholder. In this section, we offer another perspective on the stability of results by searching for a compromise between different stakeholders already at the stage of provided preferences. In Table 10, we report the numbers of DMs indicating preference or indifference for all pairs of criteria in the ranking they provided for the purpose of applying the SRF procedure. For example, 14 out of 38 stakeholders preferred \(g_1\) to \(g_2\), 22 stakeholders opted for an inverse preference, and only 2 stakeholders judged this pair indifferent. Conversely, when comparing \(g_5\) to \(g_6\), 31 experts opted for an indifference, and only one claimed that \(g_5\) was more important than \(g_6\).

Table 10 The numbers of stakeholders indicating a preference or indifference (in the round brackets) for all pairs of criteria
Full size table

The information from the DMs’ individual rankings has been used as an input for the algorithm constructing a compromise utilitarian ranking of criteria, i.e., the one which is on average the closest to 38 individual rankings. In this way, the following group compromise order of criteria has been constructed:

$$\begin{aligned} g_5 \sim _\partial g_6 \succ _\partial g_2 \sim _\partial g_4 \succ _\partial g_1 \succ _\partial g_3 . \end{aligned}$$

Thus, the greatest importance has been attributed to ecosystem quality (\(g_5\)) and resources (\(g_6\)), while the least important criteria are NPV (\(g_3\)) and hour of discomfort (\(g_1\)). The relation instantiated for different pairs of criteria is consistent with the opinion expressed by the significant number of stakeholders. For example, 24 stakeholders ranked \(g_5\) and \(g_6\) as the two most important criteria, while 19 of them ranked this pair tied for the first place. Furthermore, 21 stakeholders judged \(g_3\) as the least important criterion.

Obviously, one needs to bear in mind that the compromise ranking of criteria minimizes the sum of distances between relations observed for all pairs of criteria in all individual rankings. In this perspective, it may not be considered representative by all individuals (see, e.g., \(DM_7, DM_9 \), \(DM_{12}, DM_{17}, DM_{19}, DM_{20},\) or \(DM_{36} )\) whose preferences are represented in the compromise ranking to a marginal degree (i.e., an overall distance between their ranking and the compromise one is substantial).

Such a compromise ranking of criteria has been used to simulate DMs’ joint preferences within SMAA. Consistently with the previous sections, the cutting level \(\lambda \) was assumed to be equal to the sum of weights of the three most significant criteria. The results of robustness analysis are materialized with the cumulative group compromise class stochastic acceptability indices \(CuCCSAI^{\partial ^{\mathrm{K}}}\left( {a,h} \right) \) (see Table 11).

For most materials, the variability of results is lesser than in case of deriving the recommendation by aggregating the individual viewpoints. Indeed, for 11 out of 13 materials there is some class which is recommended with certainty (then, \(CuCCSAI^{\partial ^{\mathrm{K}}}\left( {a,h} \right) \) = 1). Also, for all materials but \(a_{4}\) the class assignments with the greatest support have not changed with respect to those reported in the previous sections. The main differences concern a lesser support for the assignment of \(a_{1}\), \(a_{4}\) and \(a_{9}\) to \(C_{2}\) in favor of judging the quality of \(a_{1}\) as high (\(C_{3})\) and the quality of \(a_{4}\) or \(a_{9}\) as low (\(C_{1})\). Finally, although the assignments of \(a_{2}\) and \(a_{8}\) to, respectively, \(C_{2}\) and \(C_{3}\) are robust, the acceptability for their assignment to some worse classes (\(C_{1}\) and \(C_{2}\), respectively) has increased to 0.419.

Table 11 Cumulative group compromise class stochastic acceptability indices \(CuCCSAI^{\partial ^{\mathrm{K}}}\left( {a,h} \right) \) for all materials
Full size table

4.4 Summary

In view of the results derived from an application of a three-stage multiple criteria decision aiding method to our study (see Tables 7, 9, and 11), we recommended the following assignments for the insulating materials:

  • Low (\(C_1 )\): gypsum fibreboard (\(a_{6})\), mineralized wood (\(a_{9})\) and plywood (\(a_{10})\);

  • Low (\(C_1 )\) or medium (\(C_2 )\): fibreboard hard (\(a_{4})\);

  • Medium (\(C_2 )\): autoclave aerated complete (\(a_{1})\), corkslab (\(a_{2})\), and expanded perlite (\(a_{3})\);

  • High (\(C_3 )\): glass wool (\(a_{5})\), hemp fibres (\(a_{7})\), kenaf fibres (\(a_{8})\), polystyrene foam (\(a_{11})\), polyurethane (\(a_{12})\), and rock wool (\(a_{13})\).

The probability of other assignments was often non-negligible though significantly lower than for the above indicated classes. Nevertheless, the results obtained from the stochastic analysis allowed to nullify the risk of a false declaration that some material was assigned to a class which was not confirmed by any compatible set of weights for any expert.

For each insulating material, the recommended decision can be justified by comparing its performances on different criteria with those of the characteristic class profiles. In Table 12, we indicate the subsets of criteria on which the materials outrank (i.e., are at least as good as) the characteristic profiles. In this regard, let us explicitly explain the most likely assignments suggested for some materials:

  • \(a_{10}\) is worse than \(b_{1}\) on all criteria, thus being assigned to the worst class \(C_{1}\); in the same spirit, \(a_{6}\) is worse than \(b_{1}\) on \(g_{2}\), \(g_{4}\), and \(g_{5 }\) (thus, on 3 out of 4 considered environmental criteria), and not better than \(b_{2}\) on any criterion, which makes \(C_{1}\) its most desired class;

  • \(a_{3}\) is better than \(b_{1}\) and worse than \(b_{3}\) on all criteria, which makes its performance vector typical for \(C_{2}\);

  • \(a_{12}\) and \(a_{13}\) are at least as good as \(b_{2}\) on all criteria and better than \(b_{3}\) on four criteria (\(g_1 \), \(g_2 \), \(g_3 \), \(g_5\) or \(g_1 \), \(g_2 \), \(g_3 \), \(g_6\), respectively (note that both scenarios include two accounted socio-economic criteria, \(g_1 \) and \(g_3 ))\), which makes their assignment to \(C_{3}\) the most justified.

Table 12 Subsets of criteria on which the materials attain at least as good performances as these of the characteristic profiles \(b_1 \), \(b_2 \), and \(b_3 \) of three decision classes
Full size table

5 Conclusions

We considered a multiple criteria problem of sustainability assessment of insulating materials. We combined Life Cycle Costing, Life Cycle Assessment, and adaptive comfort evaluation to derive performances of these materials on six socio-economic and environmental criteria. The comprehensive assessment of the materials involved their assignment to three preference-ordered sustainability classes. The classification was performed with a group decision counterpart of the Electre TRI-rC method that compares alternatives with the characteristic class profiles defined by the experts.

To derive a recommendation that would reflect viewpoints of a wide spectrum of potential customers, we accounted for the preference information of a few tens of rural buildings’ owners being interested in the roof’s insulation. The initial recommendation was derived by computing the proportion of stakeholders who accepted an assignment of a particular material to a given class. These results were subsequently validated against the outcomes of a two-fold robustness analysis realized with the Monte Carlo simulation. The latter exploited the space of all criteria weights compatible with either each stakeholder’s preference information provided in the SRF procedure or collective ranking of criteria that was derived with an original algorithm proposed in this paper.

The three-stage analysis revealed that the most sustainable materials were glass wool, hemp fibres, kenaf fibres, polystyrene foam, polyurethane, and rock wool. This was mainly due to their favorable performances quantified with the Net Present Value and Eco-indicators. On the contrary, gypsum fibreboard, mineralized wood and plywood were assessed as the least sustainable materials. This can be justified in terms of their poor performances on thermal comfort, human health, and ecosystem quality. Overall, the proposed method provided greater clarity for decision making and guaranteed credibility in the eyes of the traditional rural houses’ owners. Moreover, all research results—concerning both materials’ performances on the individual criteria and comprehensive sorting recommendation—were well perceived by the experts on insulating materials in Italy.

The proposed framework can be applied to other decision contexts than that of a typical farmhouse in central Italy. This would require, however, accounting for a comfort model as well as warm and cold periods suitable to a particular geographical context, specification of a relevant lifespan for the investment, and adapting life cycle assessment to the reality of a particular study.

From the methodological viewpoint, we envisage the following future developments. Firstly, we plan to extend the SRF procedure to a group decision context so that it tolerates intensities of preference for different pairs of criteria and accepts information on different roles (weights) of the decision makers. Secondly, we aim at extending the proposed group decision framework to methods dealing with choice and ranking problems. This would require elaboration of the algorithms for deriving a compromise recommendation that would appropriately combine results of robustness analysis computed individually for each stakeholder.