Preferential reducts and constructs in robust multiple criteria ranking and sorting

We revisit the multiple criteria ranking and sorting methods based on ordinal regression, which accept preference information in the form of, respectively, pairwise comparisons or assignment examples for some reference alternatives. Robust ordinal regression methods consider the whole set of value functions reproducing these holistic statements provided at the input. Its impact on the recommendation is expressed in terms of the necessary and possible preference relations or assignments. We propose methods for generating explanations of this impact, showing pieces of preference information provided by the decision maker (DM), which led to the observed outcomes. In particular, the minimal set of preference information pieces, called preferential reduct, is identified to justify some result observable for the whole set of compatible value functions (e.g., the truth of the necessary relation for some pair of alternatives). Further, the maximal set of preference information pieces, called preferential construct, is discovered to reveal the conditions under which some result non-observable for the whole set of compatible value functions (e.g., the falsity of the possible relation for some pair of alternatives) is possible. Knowing such explanations, the DM can better understand the impact of each piece of preference information on the result and, in consequence, get conviction about the obtained recommendation.


Introduction
A recommendation that is produced in the course of a decision aiding process results from some cognitive activities aiming at exploring, interpreting, and debating about a particular decision problem.These activities are often supported by a decision support system (DSS), which is a computer-based information system intended to assist the decision maker (DM) in constructive exchange of information between the DM and the model of the decision problem, so as to get a solution being feasible and consistent with the value system of the DM.As noted in Greer et al. (1994), there are three main components of such a system, each playing a unique role in the decision aiding process.These are user interface, simulation model, and explanation module.The role of the user interface consists in querying the user for suitable inputs, acquiring knowledge about the problem, and presenting the output.The simulation model performs the analysis of the user's queries and generates some numerical results.Finally, the explanation module interprets these outputs and generates explanations.
These three components decide upon user acceptance of both a DSS and the recommendation arrived with its use.In particular, the required input information should be as simple as possible, and collected in a way that makes the DM convinced that her/his needs, preferences and value (s)he seeks are taken into account when working out a recommendation.Furthermore, the method itself should guarantee that the DM not only learns about the problem, but also that (s)he is convinced about the psychological convergence of the process and relative advantage of the indicated solution (Belton et al. 2008).Finally, the explanations provided by the system should justify that the recommendation is logical, valid, and correct.As noted by Kass and Finin (1988), such explanations prove to be useful in the development and maintenance of DSSs by making explicit the experts logic and assumptions.
Collecting the input information When it comes to collecting input information for the decision problem (concerning criteria, alternatives, evaluations, and, in particular, the DM's preferences), the effort of making DSSs user friendly has been implemented in different ways.Some of them support the DMs in structuring, decomposition and recomposition of preferences between alternatives by eliciting all required inputs by means of a dialogue conducted in the natural language [see, e.g., multi-attribute utility decomposition (Humphreys and Wisudha 1981)].Some other systems are based on a verbal description of decision-making problems [see, e.g., verbal decision analysis (Larichev 2001)]; the need for this stems from the experimental results revealing that 2/3 of DMs prefer to give the evaluations and preferences in a verbal form.By verbal, we mean qualitative rather than quantitative, and ordinal rather than cardinal.In the recent years, an increasing interest has been also assigned to ordinal regression methods [see, e.g., UTA (Jacquet-Lagreze and Siskos 2001)].They expect the DMs to specify some examples of holistic judgments (e.g., pairwise comparisons or assignments to classes) on a subset of reference alternatives, rather than direct parameter values used in the preference model.Preference information given in terms of decision examples is called indirect.Then, the DM's preference model leading to the same decision as the provided one is inferred.Such a compatible model is applied on the whole set of alternatives to arrive at a recommendation.Decision aiding methods based on the indirect preference information require relatively less cognitive effort from the DMs, being consistent with the "posterior rationality" (March 1978) and with the learning from examples used in artificial intelligence approaches (Furnkranz and Hullermeier 2010).
Arriving at a recommendation As far as the way of translating the DM's preferences to a recommendation is concerned, the decision aiding methods need to be transparent, i.e., they should clearly reveal the correspondence between the elicited information and the final evaluations of alternatives.Let us mention the Even Swaps approach (Hammond et al. 1998) as an exemplary implementation of such a transparency.An even swap is interpreted as a value trade-off, where a change of performance on one criterion is compensated by a comparable change on some other criterion, and thus a new alternative with these revised performances is equally preferred to the initial one.The DM is required to list her/his objectives explicitly by providing a sequence of swaps which are acceptable for her/him.As a result, the method provides a practical means for making trade-offs among any set of objectives across a range of alternatives.By simplifying the mechanical elements of trade-offs, it leads to the final decision through a sequence of choices supposed to be intuitive for the DM.
From another perspective, the decision rule approach (Greco et al. 2004;Slowinski et al. 2009), represents the preferences in terms of "if . . .then" decision rules such as, e.g., "if the maximum speed of car x is at least 175 km/h and its price is at most $12,000, then car x is comprehensively at least medium".Thus, apart from requiring the DM to provide information in the form of examples of decisions, this approach employs a very natural and easily interpretable way of formulating conclusions.Each decision rule is a straightforward scenario relating evaluation of an alternative on selected criteria and its class assignment.
When it comes to ordinal regression methods, there is typically more than a single preference model instance compatible with the holistic statements.To avoid arbitrary selection of a single compatible instance, robust ordinal regression (ROR) has been proposed (Corrente et al. 2013;Greco et al. 2008).It allows taking into account all compatible preference model instances, and provides the DM with two results, the necessary and possible recommendation for the set of considered alternatives.The necessary recommendation is supported by all compatible instances of the preference model, while the possible recommendation is supported by at least one of them.In this way, the DM is forced to confront her/his value system with the results of applying the inferred model on the set of alternatives.
Explaining the recommendation Explanation of the recommended decision requires construction of arguments through which the DM will convince herself/himself and other stakeholders about its logic and validity.The issue of generating explanation of the outcome of a decision model has motivated a great variety of studies.In particular, Papamichail and French (2003) present a general framework for developing explanation facilities in evaluation systems with the use of natural language generation techniques.Ouerdane et al. (2010) aim at adapting argumentation-based approach for providing the reasons underlying the chosen course of action.Apart from explaining the motivations, goals or results, such a mix of argumentation theory and decision aiding allows taking advantage of the expressive nature of the language used as well as handling incomplete or even contradictory pieces of information.Labreuche (2011) introduces a general framework for explaining the results of multi-attribute value theory.It is focused on the selection of arguments to be presented in the case where the decision model is based on the weights assigned to criteria.Some other representative works on providing convincing explanations to accompany recommendation can be found in Amgoud and Prade (2009), Carenini and Moore (2006), Klein (1994), and Labreuche et al. (2011).
When it comes to justifying the outcomes of ROR, Labreuche et al. (2012) investigate whether Even Swaps can be generalized to construct convincing explanations of the necessary preference relation (Hammond et al. 1998).A sequence of such swaps can be seen as the reasoning steps allowing to highlight why alternative a is necessarily preferred to alternative b, or, in general, why alternative a is the best choice.The authors set up a framework for providing such minimal explanations, proving both its practical usefulness in terms of binary domains considered for the value of criteria as well as questionability of the technique in the absence of any restriction on the size of the criteria domain.In the latter case, the sequence of preference-swaps may not be bounded.On the contrary, Greco et al. (2013) applies decision rules to explain the necessary and possible preference relations.These rules aim at explaining preference relations in terms of a minimal conjunction of elementary conditions concerning evaluations on particular criteria that imply their truth.For example, the preference, either necessary or possible, of alternative a over alternative b can be supported by a decision rule indicating the role of a strong preference on criterion g 1 and at least mild preference on criterion g 2 .In this way, the DM is provided with arguments on the role of particular criteria in the decision model, which enables her/him to better understand the conditions for the necessary and possible relations.It is worth noting that due to iterative way of working with ROR methods, the DM can easily see the consequence of an increment of preference information on the necessary and possible recommendations.This observation stimulates elicitation of new preference information or increases the DM's conviction that the current recommendation is the best.
Aim of the paper As discussed above, the use of ROR methods is intuitive for both the required indirect preference information and exhibition of a set of results derived from applying the whole set of compatible preference model instances.In this paper, we reconsider the task of generating explanations of the outcomes of ROR methods designed for supporting multiple criteria ranking and sorting problems.Without loss of generality, we focus on an additive value function preference model which is of particular interest in multiple criteria decision aiding (MCDA) because of an intuitive interpretation of numerical scores of alternatives and straightforward translation of pieces of preference information to the final result (see, e.g., Doumpos 2012;Lahdelma and Salminen 2012).Contrary to Greco et al. (2013) and Labreuche et al. (2012), the constructed explanations do not refer to the evaluations of alternatives but rather directly to preference information provided by the DM.Even if the required preference information is easily definable, like a set of pairwise comparisons or assignment examples, it is processed in a way preventing the DM from seeing the exact relations between the provided preference information and the obtained recommendation.This is particularly true if the set of exemplary judgments is relatively great.Thus, analyzing the ranking recommendation one could ask the following reasonable questions: -in case alternative a is necessarily preferred to alternative b, what are the minimal sets of pairwise comparisons provided by the DM that induce a necessary preference for a pair (a, b) in consequence of using all compatible value functions; -in case alternative a is not even possibly preferred to Answering these questions facilitates understanding of the correspondence between the output of the preference model and the revealed preferences, making these relations more transparent and intelligible for the DM.The arguments are formulated with respect to the previously provided exemplary judgments, being in this way consistent with intuitive reasoning of the DMs, and not forcing them to reason in terms of some abstract parameters of the model or evaluations of the alternatives.From such explanations, the DM learns about the impact of particular pieces of original preference information or their subsets, and gains insights into her/his preferences, which enhance the trust in the obtained recommendation.This confrontation makes the DM more conscious about the conditions for the necessary and possible outputs of ROR, leading her/him also to better understand the employed model.Moreover, the provided explanations constitute a starting point for a more thorough analysis.In case the DM does not accept some implications between the provided input and the obtained recommendation, (s)he can remove or revise some pieces of preference information.
The organization of the paper is the following.In the next section, we introduce basic concepts and notation that will be used in the paper.In Sect.3, we remind existing multiple criteria ranking and sorting methods based on ROR, and we define the models that we refer to in our approach.In Sect.4, we discuss how to compute explanations of the obtained necessary and possible outcomes in terms of the DM's preference information.For the purpose of illustration, we will consider the problem of evaluating democracy regimes of different countries.We will refer to it in the following sections to illustrate different aspects of the proposed framework.In Sect.5, we focus on the computational cost of the proposed procedures.In Sect.6, we discuss how to use the provided explanations within the constructive approach to decision aiding.The last section concludes the paper.

Notation
We shall use the following notation: -A = {a 1 , a 2 , . . ., a i , . . ., a n }-a finite set of n alternatives; -A R = {a * , b * , . ..}-a finite set of reference alternatives, on which the DM accepts to express holistic preferences; we assume that A R ⊆ A; -G = {g 1 , g 2 , . . ., g j , . . ., g m }-a finite set of m evaluation criteria, g j : A → R for all j ∈ J = {1, 2, . . ., m}; although real-coded they may have ordinal or cardinal scales; -C h , h = 1, . . ., p-pre-defined preference ordered classes such that C h+1 is preferred to C h , h = 1, . . ., p − 1; H = {1, 2, . . ., p}; -X j = {x j ∈ R : g j (a i ) = x j , a i ∈ A}-the set of all different evaluations on g j , j ∈ J ; without loss of generality, we assume that all criteria have an increasing direction of preference, i.e., the greater g j (a i ), the more preferred alternative a i on criterion g j , for all j ∈ J and a i ∈ A; -the ordered values of X j , x k j < x k+1 j , k = 1, . . ., n j (A) − 1, where n j (A) = |X j | and n j (A) ≤ n; consequently, X = m j=1 X j is the evaluation space.
To represent DM's preferences, we shall use a model in the form of an additive value function: where the marginal value functions u j , j ∈ J , are monotone, non-decreasing and their sum is normalized, so that the additive value (1) is bounded within the interval [0, 1].Note that for simplicity of notation, one can write u j (a), j ∈ J , instead of u j (g j (a)).Consequently, the basic set of constraints E A R BASE defining conditions on the monotonicity and normalization of general additive value functions has the following form: Defined in this way, the general non-linear monotonic marginal value functions, with all criteria values being their characteristic points, do not involve any arbitrary or restrictive parametrization.However, the selection of the form of marginal value functions involves a trade-off between their flexibility and simplicity.Thus, instead of general marginal value functions, we can use piecewise linear or even linear ones.Then, for each u j , j = 1, . . ., m, we need to define the number of characteristic points γ j .The intervals [x 1 j , x n j (A) j ] are divided into γ j − 1 equal sub-intervals with the endpoints: Then, the monotonicity constraints [B1] should be formulated in the following way: (2)

Robust ordinal regression based on multi-attribute value theory
The majority of MCDA methods require specification of precise numerical values for parameters of the preference model.It is not always realistic to assume that the DM can provide exact values for all these parameters.Thus, disaggregation procedures that enable model inference with exemplary judgments on a set of reference alternatives have been proposed.However, usually there is more than a single model compatible with the given holistic judgments.The traditional disaggregation approaches force selecting a single such model for working out a recommendation.Such a choice is rather arbitrary, requiring either the use of some pre-defined rules or participation of the DM.Instead of defining or selecting the single model, ROR enables analysis of the whole set of compatible preference model instances.Examination of its application on the set of alternatives A leads to identifying the necessary and possible consequences confirmed by all or at least one compatible preference model instance, respectively.

Robust ordinal regression for multiple criteria ranking
As far as methods designed for dealing with multiple criteria ranking problems are concerned, ROR has been implemented for the first time in UTA GMS (Greco et al. 2008).It enables the DM to provide pairwise comparisons for some reference alternatives.The comparison of a pair (a

123
where ε is an arbitrarily small positive value.The pairwise comparisons provided by the DM form the input data for the ordinal regression that finds the whole set of value functions being able to reconstruct these judgments.Such value functions are compatible with the preference information.Precisely, a set of such compatible value functions U A R RANK is defined by a set of constraints E A R RANK given below: Thus, compatible value functions need to be monotonic and normalized (see RANK , is >0 and E A R RANK is feasible, the set of compatible value functions is non-empty.Otherwise, the provided preference information is inconsistent with the assumed preference model.
Necessary and possible preference relations UTA GMS applies all compatible value functions U A R RANK , and defines two binary relations in the set of alternatives A: -Necessary weak preference relation, N , that holds for a pair of alternatives for all compatible value functions; -Possible weak preference relation, P , that holds for a pair of alternatives for at least one compatible value function.
The following linear programs need to be solved to assess whether the relations hold: Maximize : ε (4) 2012) extend UTA GMS with the framework of extreme ranking analysis (ERA) to consider all complete pre-orders compatible with the preference information and to determine the best P * (a) and the worst P * (a) ranks taken by each alternative a ∈ A. Identification of these extreme ranks requires consideration of the following mixed-integer linear programming (MILP) models: and where M and ε are auxiliary variables equal to, respectively, big and small positive values, and v b is a binary variable (v b ∈ {0, 1}) associated with comparison of a to alternative b ∈ A \ {a}.The best rank P * (a) of alternative a is f rank max + 1 and the worst rank P * (a) is n − f rank min .Kadzinski and Tervonen (2013) prove that if we limit U A R RANK to exclude value functions for which two alternatives obtain an equal rank, there are no rank jumps in the interval In any case, combining UTA GMS and ERA, multiple criteria ranking can be approached with a robustness analysis concerning both the outcome of separate n • (n − 1) pairwise comparisons as well as the result of collating each of the n alternatives with all the remaining ones simultaneously.
Case study We reconsider the problem of evaluating democracy of different countries based on the following five criteria proposed by the Economist Intelligence Unit (EIU 2010): electoral process and pluralism (g 1 ), the functioning of government (g 2 ), political participation (g 3 ), political culture (g 4 ), and civil liberties (g 5 ).We rank 24 countries from Australasia as well as Southeast, Central and East Asia.Their evaluations are provided in Table 1.
We assume a DM to have provided preference information in the form of five pairwise comparisons of reference alternatives (see Table 2).These are consistent with respect to the considered additive model.Let us emphasize that all experimental results presented in the paper follow the use of linear marginal value functions for all criteria.Table 2 Pairwise comparisons B R provided by the DM for the problem of ranking countries based on their democracy evaluation The Hasse diagram of the necessary relation N is presented in Fig. 1.This relation is robust with respect to the given preference information meaning that a pair of alternatives is compared in the same way by all compatible value functions.Obviously, the necessary relation reproduces all pairwise comparisons of reference alternatives.Let us also remind that the necessary preference relation is transitive, and thus the arrows that can be obtained by transitivity are not represented in the Hasse diagram.Moreover, if there was no arrow representing the necessary relation between two alternatives, then these alternatives are incomparable in terms of the necessary preference.2).In case the preferential reduct is not listed, the necessary relation corresponds to the dominance relation Table 3 shows the best P * (a) and the worst P * (a) ranks of each alternative in the context of all compatible value functions.New Zealand (NZ) and Australia (AU) should be perceived as countries with the most developed democracy, whereas Myanmar (MM), Turkmenistan (TM), and North Korea (KP) need to be considered as countries with the lowest democracy level being possibly ranked at the very bottom.One can also observe what is concordant with intuition: the alternatives which are good on some criteria while being relatively bad on the others can be attributed a wide range of possible ranks [see, e.g., Singapore (SG) and Timor-Leste (TL)].On the contrary, for the alternatives whose evaluation profiles are more typical (at least within some significant subset of alternatives), the highest and the lowest possible ranks are usually close to each other [see, e.g., Australia (AU), Kazakhstan (KZ), and Taiwan (TW)].

Robust ordinal regression for multiple criteria sorting
When it comes to value-based methods designed for dealing with multiple criteria sorting problems, ROR has been implemented for the first time in UTADIS GMS (Greco et al. 2010).Let us focus on a threshold-based sorting procedure, where the limits between consecutive classes C h , h = 1, . . ., p, are defined by a vector of thresholds These assignment examples are translated to the following constraints E A R SORT : SORT , is >0 and E A R SORT is feasible.Necessary and possible assignments Given set A R of assignment examples and a corresponding set of compatible pairs (U, t) A R SORT , for each alternative a ∈ A: -The possible assignment C P (a) is defined as the set of indices of classes C h for which there exists at least one compatible pair (U, t) assigning a to C h (denoted by a → P C h ); -The necessary assignment C N (a) is defined as the set of indices of classes C h for which all compatible pairs (U, t) assign a to C h (denoted by a → N C h ).
The possible assignment of Let us define L P (a) and R P (a) as indices of the worst and the best classes to which alternative a is possibly assigned.The necessary assignment of alternative a is composed of h ∈ H , such that E(a and Tervonen 2013).
where M is a big positive value (in fact, it is enough if M > 1).
As noted in Kadzinski and Tervonen (2013), when using a threshold-based sorting procedure, C N (a) is either empty or precise.
Case study Let us consider the same data set as in Table 1 with the aim of assigning 24 countries to 4 types of regimes: full democracies (C 4 ), flawed democracies (C 3 ), hybrid regimes (C 2 ), and authoritarian regimes (C 1 ).We assume a DM to have provided preference information in the form of the eight exemplary assignments (see Table 4).
The possible and necessary assignments are presented in Table 5. Obviously, the eight reference countries are necessarily assigned to the classes specified by the DM.Another nine non-reference alternatives are precisely assigned to a single class by all compatible value functions.For the remaining seven alternatives, the necessary assignment is empty and the possible assignments are imprecise.There are six countries possibly assigned to two consecutive classes (i.e., C 1 − C 2 or C 2 − C 3 ) and one country [Malayasia (MY)] with a possible assignment of three classes C 2 − C 4 .The average width of the range of possible classes is 1.33.

Explaining recommendation in terms of decision maker's preference information
Justifying a recommendation to the DM by means of convincing arguments is a central concern for decision aiding tools (Labreuche et al. 2012).Generating a supportive evidence that would validate a certain choice, rank, or assignment is particularly difficult in case of multiple criteria preference models.Understanding the explanation that directly refers to the characteristics of these models may be simply too demanding for the non-experienced users for whom it is difficult to comprehend, e.g., the notions of criteria importance, compensation, or interactions between criteria.In particular, when referring to a single value function, exploiting it directly to justify the recommendation may not be meaningful to the DM (Labreuche 2011).In case of using a set of infinitely many compatible value functions as in ROR, even such direct exploitation of the preference model is not possible, because the DM can neither see a score of each alternative nor (s)he can assess relative importance of the criteria understood as a share of a given criterion in the comprehensive value.) and necessary (C N ) assignments, all assignment preferential reducts (APRs) and the assignment preferential core (ACORE) for all countries (note that we refer to indices of assignment examples provided by the DM from Table 4) Traditionally, explanations that have been constructed to accompany the use of a value function in a decision aiding process referred to performances of some alternatives on different criteria and to importance of these criteria (see, e.g., Greco et al. 2013;Labreuche 2011;Labreuche et al. 2012).In this paper, we propose to emphasize the relation between the original preference information and a resulting recommendation.Explanations in our context will thus have the form of implications associating some pieces of DM's preference information with the necessary, possible, and extreme results.Precisely, we will refer to explanations of two kinds.On the one hand, the minimal set of preference information pieces, called preferential reduct, will be identified to justify some result that is observable for the set of compatible value functions, e.g., the truth of the necessary relation for some pair of alternatives.On the other hand, the maximal set of preference information pieces, called preferential construct, will be discovered to reveal the conditions under which some result non-observable for the set of compatible value functions, e.g., the falsity of the possible relation for some pair of alternatives, is possible.Provided with such explanations, the DM can better understand the impact of each piece of preference information on the result as well as can gain arguments in favor of a recommendation given by ROR in terms of some holistic judgments (s)he previously supplied.Such simple and intuitive explanations should either increase the DM's confidence in the result or stimulate her/his reactions in the subsequent iterations in case (s)he finds some generated explanation (i.e., implication between the input and the output) unacceptable.
Let us remind that computation of the necessary, possible, and extreme results relies on solving a series of special optimization problems.Precisely, the analysis of infeasibility of these problems is of the utmost importance.For example, confirmation of the necessary preference relation or class assignment is based on the proof that constraints corresponding to the opposite preference relation or different assignment, respectively, are in conflict with the constraints defining the set of compatible value functions.Furthermore, when the latter constraints contradict the mathematical conditions corresponding to some preference relation or class assignment, such recommendation is considered as not possible for the set of compatible value functions.As a result, constructing an explanation of some recommendation provided by ROR amounts at identifying the reasons of infeasibility.
Since infeasibility is often encountered during the process of initial model formulation or reformulation, some algorithmic assistance is available to diagnose its cause.As noted in Chinneck (2008), applications of infeasibility analysis can be found, e.g., in backtracking in constraint logic programs, training neural networks, statistical analysis, video broadcasting, protein folding, radiation therapy, and signal processing.In these contexts, two major problems have been raised in the literature.The first one concerns isolating the infeasibility to a subset of constraints, called irreducible infeasible system (IIS) (see Carver 1921;Chinneck 1994;van Loon 1981).
Definition 1 Irreducible infeasible system is a subset of constraints defining the overall initial set of constraints (mathematical program) that itself is infeasible, but for which any proper subset is feasible; the IIS is minimal in the sense that it contains all constraints that contribute to the infeasibility.
Isolating an IIS from the larger model speeds the analysis, diagnosis and repair of the model by focusing the analytic effort to a small portion of the entire model.To emphasize the correspondence between IIS and explanations, we aim to construct, let us note that, e.g., in case of ranking problems, a preferential reduct of the necessary preference relation a N b is a small set of pairwise comparisons from among the many provided by the DM, so that in some way each of them contributes to the truth of U (a) ≥ U (b) for all compatible value functions, i.e., they all have to be there for infeasibility of U (b) > U (a).
When dealing with the infeasibility, another problem is to determine the minimal set of constraints that have to be removed to restore the consistency in the initial system.It is known as solving the minimum IIS set-covering problem, or finding the maximum feasibility set (MFS) (see, e.g., Amaldi et al. 1999;Sankaran 1993).Definition 2 Maximum feasibility set is a subset of constraints defining the overall initial set of constraints (mathematical program) that itself is feasible, but for which any proper superset is infeasible; the MFS is maximal in the sense that it contains all constraints that do not contradict each other.
Let us note that, e.g., in case of ranking problems, a preferential construct of the possible preference relation a P b for a pair (a, b), such that not (a P b), is a set of pairwise comparisons from among the many provided by the DM, so that all of them make U (a) ≥ U (b) possible, i.e., no additional pairwise comparison can be added to this set without contradicting U (a) ≥ U (b).
Algorithms for finding IISs and MFSs of constraints have been proposed, implemented, and tested in the recent years (for a review, see Chinneck 2008).In this section, we adapt these procedures to identify the preferential reducts and constructs.We discuss in detail the approaches for generating explanations of the necessary and possible preference relations.Analogously, we present the procedures for explaining the outcomes of ERA as well as the necessary and possible assignments, however, not discussing them in depth again.We also present preferential reducts and constructs for the case study concerning evaluation of democracy in the 24 countries.

Necessary and possible preference relation
Let us first consider the problem concerning identification of the minimal set of pairwise comparisons provided by the DM that induces a N b in consequence of using all compatible value functions.Let us now discuss a few algorithms for identification of a preferential reduct for a N b.They are based on the procedures for isolating IIS in the set of infeasible constraints.Let us emphasize, however, that a preferential reduct for a N b is not equivalent to IIS in E N (a, b).In fact, we are only interested in the part of IIS that is useful in explaining the necessary relation a N b in terms of pairwise comparisons the DM previously supplied.We intentionally neglect the part that is related to the constraints on monotonicity and normalization.This implies the necessary adaptation of the algorithms for discovering IIS.In particular, the suitably adapted deletion filter (Chinneck and Dravnieks 1991) operates by testing the feasibility of the model when constraints corresponding to the provided pairwise comparisons are dropped one by one (see Algorithm 1).At the end of a single pass through all these constraints, the algorithm identifies a single preferential reduct.Precisely, it indicates a preferential reduct whose first pairwise comparison makes the tested set of constraints feasible once eliminated.
Algorithm 1 Deletion filter for identifying a single preferential reduct for a N b Require: E N (a, b), where ε is an arbitrarily small positive value; it is an infeasible set of constraints Ensure: PR, preferential reduct for a N b.
Algorithm 2 Additive method for identifying a single preferential reduct for a N b Require: As noted by Chinneck (2008), a more efficient procedure consists in combining the additive method with the deletion filter.Precisely, the additive method is applied until first infeasible subset of constraints is found.Then, the deletion filter is used to identify a preferential reduct in this subset.Another idea for speeding up the search consists in grouping the constraints (see Chinneck 2008).Precisely, once deleting (adding) a group of B R k which causes feasibility (infeasibility), the deletion filter (the additive method) should backtrack and delete (add) singly.
Both the deletion filter and the additive method identify a single preferential reduct.However, there may exist several preferential reducts for relation a N b.To enumerate all of them, we can use Algorithm 3. It extends the additive method by progressively adding all possible subsets of constraints corresponding to the provided pairwise comparisons, and eliminating from the test list the proper supersets of these constraint sets that already implied the infeasibility.
Knowing all preferential reducts for a N b, we can identify the preferential core (see Definition 6).
Definition 4 Assuming the use of a particular preference model, the preferential core for a pair (a, b) ∈ A × A, such that a N b, is the intersection of all preferential reducts for this relation.
Moreover, since the following relations hold: the algorithms for identifying the preferential reducts for the necessary relation may also be used for indicating the minimal subset of pairwise comparisons implying that some relation is not even possible.
Let us now consider the problem concerning identification of the maximal set of pairwise comparisons provided by the DM that admits a P b in consequence of using at least one compatible value function, in case not (a P b), i.e., the problem of preferential constructs.
Definition 5 Assuming the use of a particular preference model, a preferential construct for a pair (a, b) ∈ A × A, such that not (a P b), is a maximal set of pairwise comparisons provided by the DM admitting a P b.Preferential construct is maximal in a sense that any of its proper supersets does not allow a P b.Let us denote it by PC(not(a P b)).
Remark 3 Considering a pair (a, b) ∈ A × A, such that not (a P b) and a corresponding infeasible constraint set E P (a, b) with ε > 0, the following holds: -Identifying such a MFS is equivalent to finding a minimal set of pairwise comparisons whose corresponding constraints need to be removed from E P (a, b) so that the remaining constraint set is feasible.
Identification of a preferential construct for not (a P b) can be achieved by means of MILP (see, e.g., Chinneck 2001;Mousseau et al. 2003).Using binary variables v k associated with each pairwise comparison B R k provided by the DM, one rewrites the set of constraints E P (a, b) into E P V (a, b) given below.If v k = 1, then the corresponding constraint is always satisfied, which is equivalent to elimination of this constraint.
Thus, to identify a minimal subset of pairwise comparisons that need to be removed so that U (a) ≥ U (b) is possible, we solve the following problem: where M and ε are arbitrarily large and small positive values, respectively.As a result, the constraints corresponding to v k = 0 indicate pairwise comparisons included in a preferential construct for not (a P b).
Other preferential constructs can be identified by adding constraints that forbid finding again the same solutions which have been already identified in the previously conducted optimizations: where f * i is the optimal value of the objective function in the ith iteration, v * k are values of the binary variables at the optimum found while identifying the ith minimal subset standing behind incompatibility, and Example 4 Let us compute all preferential constructs for a pair (SG, PH) ∈ A × A, such that not(SG P PH): 1. S 1 = {B R 3 }, and thus PC 1 (not 4 } (add v 5 ≤ 0 to the set of constraints); 3. The set of constraints is infeasible, meaning that all preferential constructs have been already identified.
These results should be interpreted in the following way: "two sets of the DM's pairwise comparisons: } make the preference of SG over PH still possible", or from another perspective: "if the DM eliminated from B R either B R 3 or B R 5 , SG would be ranked at least as good as PH for at least one compatible vale function".
Analogously as for the case of reducts, it is interesting to consider the intersection of all preferential constructs.Definition 6 Assuming the use of a particular preference model, the preferential core of constructs for a pair (a, b) ∈ A × A, such that not (a P b), is the intersection of all preferential constructs for this relation.Let us denote it by PCCORE(not(a P b)).
Remark 4 Elimination of all pairwise comparisons contained in the preferential core PCCORE(not(a P b)) would not change the falsity of a P b.From another perspective, these pairwise comparisons are not contained in any preferential reduct for a relation b N U a.
Example 5 The preferential core of constructs for a pair (SG, PH)

Extreme ranking analysis
Definition 7 Assuming the use of a particular preference model, a rank preferential reduct for an alternative a ∈ A, whose best and worst possible ranks are P * (a) and P * (a), respectively, is a minimal set of pairwise comparisons provided by the DM inducing a is neither ranked better than P * (a) nor worse than P * (a) by any compatible preference model instance.Let us denote it by RPR(a).
A rank preferential reduct for a ∈ A, which guarantees a is ranked between P * (a) and P * (a) by all compatible value functions, can be identified using Algorithm 2, where E A R TS provided as the input is equal to E P (a → [P * (a), P * (a)]) given below. if Constraints (C1), (C2), and (C3) ensure that a is ranked not worse than P * (a) − 1.
A lower rank preferential construct for a ∈ A, admitting a is possibly ranked worse than P * (a), is identified analogously with constraints (C1) and (C2) being replaced by (C4) and (C5) given below: Example 6 For an alternative TJ, whose extreme ranks are 17 and 22: -There is a single upper rank preferential construct URPC(TJ) 3 is the only pairwise comparison in B R preventing T J from being ranked better than 17th"; -There are two lower rank preferential constructs LRPC 1 2 ) need to be removed from the set of provided pairwise comparisons B R to allow TJ being ranked worse than 22nd".

Explaining recommendation of robust multiple criteria sorting
Definition 9 Assuming the use of a particular preference model, an assignment preferential reduct for an alternative a ∈ A, whose worst and best possible classes are C L P (a) and C R P (a) , respectively, is a minimal set of assignment examples provided by the DM inducing a is never assigned to a class better than C R P (a) and worse than C L P (a) .Let us denote it by APR(a).
Remark 5 If L = R, a is necessarily assigned to class C L=R (a).Then, an assignment preferential reduct induces a → N C L=R (a).
An assignment preferential reduct for a ∈ A, which guarantees a is assigned to a class between C L P (a) and C R P (a) by all compatible value functions, can be identified using Algorithm 2, where E A R TS provided as the input is equal to E(a → P [C L P (a) , C R P (a) ]) given below.
Constraint (C6) guarantees that a is assigned to a class C L P (a)−1 or worse.Constraint (C7) ensures a is assigned to a class C R P (a)+1 or better.Constraint (v 1 +v 2 = 1) guarantees that either (C6) or (C7) is instantiated with v 1 = 0 or v 2 = 0, respectively, ensuring either U (a) < t L P −1 or U (a) ≥ t R P is satisfied.Thus, finding an assignment preferential reduct is equivalent to identifying a minimal set of assignment examples which contradict both U (a) < t L P −1 and U (a) ≥ t R P .The assignment preferential core (AC O R E) is defined analogously to PC O R E (see Definition 6).
Case study For the case study, all assignments preferential reducts and the assignment preferential cores are provided in Table 5.Let us explicitly provide interpretation of these results for a few alternatives: -APR 1 (AU) = {A R 1 } and APR 2 (AU) = {A R 2 } ⇒ "the necessary assignment of AU to C 4 is a consequence of the DM's single assignment example, either An upper assignment preferential construct for a ∈ A, which admits a is possibly assigned to a class better than C R P (a) , is identified by solving (8), where the constraint set Example 7 There is a single upper assignment preferential construct for an alternative M N , whose best possible class is , which means that "an exemplary assignment A R 3 is the only one in A R that prevents M N from being possibly assigned to C 4 ".
A lower assignment preferential construct for a ∈ A, which admits a is possibly assigned to a class worse than C L P (a) , is identified analogously with constraint U (a) ≥ t R P being replaced by U (a) + ε ≤ t L P −1 .
Example 8 There are two lower assignment preferential constructs for an alternative PH, whose worst possible class is 7 need to be removed from the set of assignment examples A R to allow PH being possibly assigned to C 1 ".
Note that the notion of an upper (lower) assignment preferential construct is closely related to the previous work in the context of robust multiple criteria sorting with ELECTRE methods.In Dias and Mousseau (2003) and Mousseau et al. (2003), infeasibility analysis is applied to indicate assignment examples (or, in general, sets of constraints) which should be removed so as to assign an alternative to a better or worse class than, respectively, its best or worst possible class.

Computational cost
Let us denote the number of the provided preference information pieces by PI.For ranking problems, it is equal to the number of pairwise comparisons of reference alternatives |B R |, while for sorting problems it is equal to the number of assignment examples |A R |.To identify a single preferential reduct with a deletion filter or the additive method, in the worst case we need to solve, respectively, PI or PI • (PI − 1)/2 problems.To discover all preferential reducts with an additive method, up to 2 PI problems need to be solved.
However, each of these problems is relatively small-the range of dimensions in case of using general monotonic marginal value functions is n •m +|B R | constraints for preferential reducts, n • (m + 2) + |B R | for rank preferential reducts, and n • m + 2|A R | for assignment preferential reducts.The number of variables is equal to n • m.In case of identifying rank preferential reducts, additional 2n binary variables are involved in problem formulation.Let us note that when using linear marginal functions the numbers of constraints and variables are significantly less.In particular, we use m variables instead of n • m ones.
Solving all these problems is not a burden for contemporary solvers.It is feasible in reasonably short time, even if all preferential reducts need to be discovered, because sets of reference alternatives in MCDA usually consist of modestly sized collections of choices.In fact, multiple criteria problems considered in operations research and management science (OR/MS) usually involve several dozens of alternatives (Wallenius et al. 2008).Thus, it is not realistic to assume that the DM would be willing to provide tens of preference statements when dealing with such a set of alternatives.
Obviously, in case of ranking problems, the number of pairwise comparisons may be great if the DM provides a complete pre-order for the set of reference alternatives, as in the original UTA method (Jacquet-Lagreze and Siskos 1982).Precisely, a ranking of |A R | alternatives can be decomposed to pairwise comparisons.For large sets A R , indication of preferential reducts composed of individual pairwise comparisons would be rather demanding in terms of required computation time.To reduce the computational complexity, we may treat all pairwise comparisons referring to a single reference alternative as a logical whole.Then, within the deletion filter, the corresponding constraints would be deleted in block, thus removing a single alternative from the reference ranking.Consequently, the solution indicated by a deletion filter would be a minimal complete pre-order of a subset of reference alternatives derived from the reference ranking provided by the DM.This complete pre-order may, however, contain some redundant elementary pairwise comparisons.
Let us note, however, that ROR was designed to deal with problems typically considered within MCDA, i.e., with problems concerning up to few hundred alternatives, and rather not thousands.For larger sets of alternatives, there would be too many optimization problems to compute, even for getting the basic outcomes of ROR, such as the necessary preference relation, extreme ranks, or possible assignment.Thus, ROR cannot be used to solve problems in information retrieval, natural language processing, or bio-informatics.Moreover, when the set of alternatives is very large, ROR is losing its advantage, because one can neither present to the DM all results (e.g., the ranking) and explanations for the alternatives, nor involve her/him in the interactive construction of preference information.

Using explanations in the interactive construction of preferences
Preferential reducts and constructs can be interpreted as transparent and easily understandable arguments that can be used to justify and explain the decision.They enable DM's understanding of the conditions for the necessary, possible, and extreme results by providing useful information on the role of particular preference information pieces in terms of consequences they imply.In the learning perspective of a constructive approach to decision aiding, these explanations can be a starting point for an interactive analysis and construction of DMs preferences.They may serve as a tool for looking more thoroughly into the subject, by reasoning, interpreting, and arguing.We permit the DM to confront her/his value system not only with results of applying the inferred model on the set of alternatives, but also with explanations that make her/his logic and assumptions explicit.This confrontation gives insight into her/his preferences, providing reactions in the subsequent iteration, as well as a better understanding of the employed method.
Preferential reducts and constructs are presented so as to invite the DM to an interaction.On the one hand, (s)he could not accept some explanation in case (s)he does not agree with the arguments it suggests.Let us suppose that the DM disagrees with a preferential reduct revealing the implication between a pair-wise comparison a DM b and the necessary preference relation c N d, for a, b, c, d ∈ A. Then, (s)he can remove or invert a preference relation for a pair (a, b).In the same spirit, in case the DM does not agree with the falsity of the possible preference relation not(e P f ), e, f ∈ A, (s)he may wish to keep preference information pieces contained in its preferential construct only, and remove the remaining ones.On the other hand, the analysis of the provided explanations may also increase the DM's confidence in some pieces of preference information.In case the DM agrees with the necessary or non-possible consequences, (s)he may confirm preference information contained in the corresponding preferential reduct or not contained in the preferential construct, respectively.
Analyzing the obtained results along with the explanations, the DM can judge if the suggested recommendation is logical and valid, and whether (s)he is satisfied with the correspondence between the preferences (s)he has at the moment and the recommendation based on the preference model.In this way, the process of preference construction is either continued (with strengthened preference information) or restarted (with revised preference information).The procedure ends when the DM finds acceptable the possible, necessary and extreme results, as well as the explanations given by preferential reducts and constructs.

Extensions
In this section, we discuss a few extensions of the basic procedures for computation of preferential reducts and constructs.For brevity, we focus on the necessary and possible preference relations, while an adaptation to ERA and multiple criteria sorting is omitted for being obvious.
Accounting for credibility of preference information ROR methods are designed to be used interactively with an incremental specification of preference information.
s be nested sets of DM's pairwise comparisons of reference alternatives associated with decreasing confidence levels.Each of those sets B R t , t = 1, . . ., s, is modeled with a set of constraints E A R RANK,t generating the set of compatible value functions U A R t .Each time we pass from B R t−1 to B R t , t = 2, . . ., s, we add to E A R RANK,t−1 new constraints concerning pairs (a, b) ∈ B R t \ B R t−1 .Thus, the sets of compatible value functions are nested in the inverse order of the related set of pairwise comparisons B R t , t = 1, . . ., s, i.e., We suppose that U A R s = ∅.On the basis of nested sets of pairwise comparisons s , we can build nested necessary N 1 ⊆ N 2 ⊆ • • • ⊆ N s and possible P 1 ⊇ P 2 ⊇ • • • ⊇ P s preference relations.The DM may require that the constructed preferential reducts and constructs contain the most credible preference information which is provided in the initial phases of the interactive process.Computation of such a preferential reduct for a N b requires identification of the first nested necessary relation N t , t = 1, . . ., s, containing a N b.Then, to indicate the preferential reduct consisting of the most credible preference information, the adequate procedures should exploit a set of pairwise comparisons B R t rather than B R s .Analogously, computation of the most credible preferential construct for a P b requires identification of the last nested possible relation P t , t = s, . . ., 1, containing a P b.Then, the adequate algorithm should exploit the set of pairwise comparisons B R s \ B R t to eliminate the least credible statements making a P b impossible.Note that analogous idea has been explored before in case of dealing with inconsistency of preference information.Mousseau et al. (2006) introduce algorithms for providing solutions with the least confident judgments or constraints to be relaxed/deleted with a higher priority than solutions recommending relaxation of high-confidence statements.
Joint reducts and constructs for multiple preference relations Apart from analyzing preferential reducts and constructs for a single preference relation, we can also consider analogous explanations in terms of a set of multiple relations, i.e., NEC = {a 1 N b 1 , a 2 N b 2 , . ..} and POS = {a 1 P b 1 , a 2 P b 2 , . ..}, respectively.To identify a joint preferential reduct for N EC, within a deletion filter we should consider the following set of constraints: for each a i N b i ∈ NEC, instead of considering the set of constraints E N (a, b).Analogously, to compute a joint preferential construct for POS, within the set of constraints E P V (a, b), we should replace a constraint U (a) − U (b) ≥ 0 with a constraint set U (a i ) − U (b i ) ≥ 0, for each (a i P b i ) ∈ POS.Forcing explanations to contain some pairwise comparisons The DM may wish to obtain explanations referring to some background preference information, i.e., to get a reduct which contains some specific pairwise comparisons.This can be achieved by the rearrangement of the list of pairwise comparisons to be dropped one by one in the deletion filter, so that the constraints that should be potentially included in the reduct are tested first.Obviously, it may turn out that the indicated pieces of preference information are redundant with respect to explaining the investigated result, and thus they are not included in the preferential reduct.In this case, the DM should be informed that it is impossible to formulate a minimal explanation using the indicated arguments.
Necessary projection of a set of pairwise comparisons of reference alternatives Let us call the set of necessary preference relations induced by some subset of pairwise comparisons its necessary projection on the set of pairs of alternatives.Analyzing such a projection may be useful for the DM interested in the necessary relations, (s)he needs to accept as a consequence of some specific subset of pairwise comparisons of reference alternatives.In particular, when considering the projection of the preferential reduct for a N b, the DM is provided with all other necessary relations induced by the same subset of preference information pieces and (s)he may judge whether their joint induction is concordant with her/his value system.

Conclusions
In this paper, we presented a new approach to generating explanations of the recommendations offered by multiple criteria ranking or sorting methods based on robust ordinal regression.We have suitably adapted algorithms for identification of irreducible infeasible systems and maximum feasibility sets to indicate preferential reducts and constructs, respectively.The previous associates some result that is valid within the set of compatible value functions with the minimal set of preference information pieces by which it is supported.The latter indicates the maximal set of the holistic judgments which make some currently non-observable result possible for at least one compatible value function.Knowing the exact relations between the provided preference information and the final recommendation, the DM can better understand the impact of each piece of preference information or their subsets on the result, and gain a better insight into her/his preferences.This enhances the interpretability of the outcomes and increases the DM's acceptance of the suggested recommendation.
The presented approach remains valid for different types of holistic preference information.In particular, in case of ranking problems, apart from pairwise comparisons we may use the intensity of preference (e.g., a is preferred to b at least as much as c is preferred to d) (Figueira et al. 2009) or rank-related requirements (e.g., a should be ranked in top 3, b should be among the 10 worst alternatives, or c should be placed in second five) (Kadzinski et al. 2013).Moreover, suitably adapted, the proposed algorithms can be used in the context of other preference models, e.g., outranking-based ranking methods (Greco et al. 2011;Kadzinski et al. 2012) or fuzzy integrals (Angilella et al. 2010), as well as in the case of group decision (Greco et al. 2012;Kadzinski et al. 2013).
alternative b, what are the maximal sets of the provided pairwise comparisons that admit a possible preference relation for a pair (a, b) in consequence of using at least one compatible value function; -in case alternative a is ranked at positions between P * (a) and P * (a) (P * (a) ≤ P * (a)) by all compatible value functions, what are the minimal sets of the provided pairwise comparisons that imply such a ranking interval, and what are the maximal sets of provided pairwise comparisons that admit a being ranked higher than P * (a) or lower than P * (a).Furthermore, in the context of sorting problem, the following analogous questions are relevant: -in case alternative a is necessarily assigned to class C h , what are the minimal sets of assignment examples provided by the DM that induce such a necessary assignment; -in case alternative a is possibly assigned to an interval of classes [C L , C R ], what are the minimal sets of the provided assignment examples that imply such a possible assignment, and what are the maximal sets of assignment examples that admit a being assigned to a class worse than C L or better than C R .
may state the strict preference, weak preference, or indifference denoted by a * DM b * , a * DM b * , and a * ∼ DM b * , respectively.Let each pairwise comparison from B R be denoted by B R k , k = 1, . . ., |B R |.The set of constraints E A R PI−RANK given below translates such a reference pre-order provided by the DM to a value function:

Fig. 1
Fig. 1 Hasse diagram of the necessary relation.All preferential reducts (see Sect. 4.1.1)for the necessary relations visible in the Hasse diagram (note that we refer to indices of pairwise comparisons provided by the DM from Table2).In case the preferential reduct is not listed, the necessary relation corresponds to the dominance relation

Definition 3
Assuming the use of a particular preference model, a preferential reduct for a pair (a, b) ∈ A × A, such that a N b, is a minimal set of pairwise comparisons provided by the DM inducing a N b.Preferential reduct is minimal in a sense that any of its proper subsets does not imply a N b.Let us denote it by PR(a N b).Remark 1 Considering a pair (a, b) ∈ A × A, such that a N b and a corresponding infeasible constraint set E N (a, b) with ε > 0, the following holds: -Since E A R RANK is feasible, IIS in E N (a, b) needs to contain constraint U (b)−U (a) ≥ ε.Thus, IIS contains constraints on monotonicity and normalization of the preference model and/or constraints corresponding to pairwise comparisons provided by the DM that are in conflict with U (b) > U (a), i.e., induce U (a) ≥ U (b) for all compatible value functions.-A preferential reduct for a N b is equivalent to a set of pairwise comparisons whose corresponding constraints are included in an IIS in E N (a, b).
Let us denote it by PC O R E(a N b).Remark 2 If the preferential core for a pair (a, b) ∈ A × A, such that a N b is non-empty, elimination of any pairwise comparison contained in the core would result in the denial of a N b.Example 3 Let us compute the preferential core (PC O R E) for (PG N MY).Since there are two preferential reducts for this pair of alternatives: PR 1 (PG N MY) = PCORE(KR N TL) = {B R 5 } or PCORE(TL N MY) = {B R 3 }], removal of any of its members from B R would result in the falsity of the corresponding necessary relation.The above-described algorithms may be easily adapted to explain the strict necessary preference a N U b ⇔ ∀U ∈ U R , U (a) > U (b) [we introduce notation N U to avoid confusion with the previous papers on ROR, which interpret N as the asymmetric part of N , i.e., a N b ⇔ a N b and not (b N a)].Precisely, in the deletion filter one should use E thus making E P (a, b) infeasible.-Preferential construct for not(a P b) is equivalent to a set of pairwise comparisons whose corresponding constraints are included in a MFS together with constraint set E A R BASE and constraint U (a) − U (b) ≥ 0.
neither assigned to a class better than C 3 nor to a class worse than C 2 by any compatible value function, because the DM provided the following assignment examples:A R 3 = (TW → C 3 ), A R 5 = (SG → C 3 ), and A R 8 = (CN → C 1 )".This allows identification of the precise implications between the provided assignment examples and the obtained possible and necessary assignments.Note that if the range of possible assignments is a proper subset of the set of all classes (as it is the case for countries in the study), then AP R is non-empty.The assignment examples contained inAC O R E [e.g., ACORE(FJ) = {A R 3 , A R 7 } or ACORE(MN) = {A R 3 }]need to be held in A R to guarantee the possible assignment for an alternative is not less precise.Definition 10 Assuming the use of a particular preference model, an upper (lower) assignment preferential construct for an alternative a ∈ A, whose best (worst) possible class is C R P (a) (C L P (a) ), is a maximal set of assignment examples provided by the DM admitting a is possibly assigned to a class better than C R P (a) (worse than C L P (a) ).Let us denote it by U APC(a) (L APC(a)).

Table 1
Performance matrix for the problem of evaluating countries on the type of democracy regime

Table 3
Results of extreme ranking analysis for all compatible value functions (P * (a) − P * (a)) and in case no preference information is available (P * D (a) − P D * (a)) Country P * (a) − P * (a)All rank preferential reducts (RPRs) for each alternative and the rank preferential core (RCORE) (see Sect. 4.1.2;notethat we refer to indices of pairwise comparisons provided by the DM from Table2) and t h−1 and t h are, respectively, the lower and upper threshold of class C h , h = 2, . .., p − 1.Note that t 1 is an upper threshold of class C 1 while the lower threshold is 0, and t p−1 is a lower threshold of class C P while there is no upper threshold t p for C P (or, from another perspective, it is >1, which means that comprehensive values of all alternatives are <t p ).UTADIS GMS represents the DM preferences with a pair (U, t), where U is an additive value function and t is a vector of thresholds delimiting the classes.It enables the DM to provide possibly imprecise assignment examples consisting of a reference alternative a * ∈ A R and its desired assignment [L DM

Table 4
Assignment examples A R provided by the DM for the problem of assigning countries to different types of democracy regimes

Table 5
Possible (C P L − C P R where ε is an arbitrarily small positive value.Require: E A R PI−RANK , a set of constraints corresponding to the provided pairwise comparisons.TS : the current test set of constraints.Ensure: PR, preferential reduct for a N b.