Skip to main content
SpringerLink
Log in

Multiple criteria sorting models and methods. Part II: theoretical results and general issues

  • Invited Survey
  • Published:
4OR Aims and scope Submit manuscript

Abstract

Multiple criteria sorting methods assign objects into ordered categories while objects are characterized by a vector of n attributes values. Categories are ordered, and the assignment of the object is monotonic w.r.t. to some underlying order on the attributes scales (criteria). We drew a landscape of these methods in Part I “Survey of the literature” (published in a previous issue of the present journal) and we aim to provide a theoretical view of the field in this second part. We describe a general framework for MCS models and position some existing models in the picture. Issues related to imperfect or insufficient information are then discussed. We also address questions that arise in the final phase of a decision aiding process as, e.g., explaining a recommendation or suggesting efficient ways of improving an object assignment.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1

Similar content being viewed by others

Notes

  1. See Bouyssou and Marchant (2007a, 2007b) for a general presentation including infinite criteria scales.

  2. The main model of this type is the Choquet integral model (see, e.g., Grabisch 2016, Chapter 6). This model has been characterized by Wakker (1989, Theorem VI.5.1) in a ranking context. However, the latter result applies only in the case all criteria are evaluated on a common scale. This is a convenient setting for decision under uncertainty, but raises difficulties related to criteria scales commensurateness in a MCDM/A context (Grabisch 2016, p. 343).

  3. Different implementations of an “as well as possible” requirement can be found in the literature. Usually, slack variables are introduced in the linear programming formulation. There are several ways of doing so (see, e.g., Jacquet-Lagrèze 1982; Doumpos and Zopounidis 2002). The objective function of the LP minimizes the sum of the slack variables, or the maximal value of slack variables, or a linear combination of such objectives. A positive slack variable means that a constraint is violated. Minimizing the number of wrong assignments requires a MIP formulation.

  4. Note that the latter term is classical in this sense in the field of classification (Aggarwal et al. 2010) but may be ambiguous because some authors use it designate the learning of a sorting model from assignment examples.

References

  • Aggarwal CC, Chen C, Han J (2010) The inverse classification problem. J Comput Sci Technol 25(3):458–468

    Article  Google Scholar 

  • Almeida-Dias J, Figueira JR, Roy B (2010) ELECTRE TRI-C: a multiple criteria sorting method based on characteristic reference actions. Eur J Oper Res 204(3):565–580

    Article  Google Scholar 

  • Bazin A, Couceiro M, Devignes M-D, Napoli A (2020) Explaining multicriteria decision making with formal concept analysis. In: Concept lattices and applications 2020, volume 2668 of CEUR Workshop Proceedings, pp 119–130

  • Belahcène K, Labreuche C, Maudet N, Mousseau V, Ouerdane W (2017) Explaining robust additive utility models by sequences of preference swaps. Theor Decis 82(2):151–183

    Article  Google Scholar 

  • Belahcène K, Labreuche C, Maudet N, Mousseau V, Ouerdane W (2017) A model for accountable ordinal sorting. In: Carles Sierra, editor, proceedings of the twenty-sixth international joint conference on artificial intelligence, IJCAI 2017, Melbourne, Australia, Aug 19-25, 2017, pp 814–820

  • Belahcène K, Labreuche C, Maudet N, Mousseau V, Ouerdane W (2018) Accountable approval sorting. In: Proceedings of the 27th international joint conference on artificial intelligence (IJCAI 2018)

  • Belahcène K, Mousseau V, Ouerdane W, Pirlot M, Sobrie O (2022) Multiple criteria sorting models and methods. Part I: Survey of the literature. 4OR. https://doi.org/10.1007/s10288-022-00530-4 (to appear)

  • Bouyssou D, Marchant T (2007) An axiomatic approach to noncompensatory sorting methods in MCDM, I: the case of two categories. Eur J Oper Res 178(1):217–245

    Article  Google Scholar 

  • Bouyssou D, Marchant T (2007) An axiomatic approach to noncompensatory sorting methods in MCDM, II: more than two categories. Eur J Oper Res 178(1):246–276

    Article  Google Scholar 

  • Bouyssou D, Marchant T (2010) Additive conjoint measurement with ordered categories. Eur J Oper Res 203(1):195–204

    Article  Google Scholar 

  • Bouyssou D, Marchant T (2015) On the relations between ELECTRE TRI-B and ELECTRE TRI-C and on a new variant of ELECTRE TRI-B. Eur J Oper Res 242(1):201–211

    Article  Google Scholar 

  • Bouyssou D, Marchant T, Pirlot M (2009) A conjoint measurement approach to the discrete Sugeno integral. Essays in Honor of Peter C. In: Brams S, Gehrlein WV, Roberts FS (eds) The Mathematics of Preference, Choice and Order. Springer, Fishburn, pp 85–109

    Chapter  Google Scholar 

  • Bouyssou D, Marchant T, Pirlot M (2021) A note on ELECTRE TRI-nB with few limiting profiles. 4OR

  • Bouyssou D, Marchant T, Pirlot M (2022) A theoretical look at Electre Tri-nB and related sorting models. 4OR

  • Doshi-Velez F, Kortz M, Budish R, Bavitz Ch, Gershman S, O’Brien D, Schieber S, Waldo J, Weinberger D, Wood A (2017) Accountability of AI under the law: The role of explanation. CoRR, arXiv:1711.01134

  • Doumpos M, Zopounidis C (2002) Multicriteria decision aid classification methods. Kluwer Academic Publishers, Dordrecht Boston

    Google Scholar 

  • Dubois D, Marichal J-L, Prade H, Roubens M, Sabbadin R (2001) The use of the discrete Sugeno integral in decision making: a survey. Internat J Uncertain Fuzziness Knowledge-Based Systems 9:539–561

    Article  Google Scholar 

  • Fernández E, Figueira JR, Navarro J, Roy B (2017) ELECTRE TRI-nB: a new multiple criteria ordinal classification method. Eur J Oper Res 263(1):214–224

    Article  Google Scholar 

  • Fernández E, Figueira JR, Navarro J (2019) An interval extension of the outranking approach and its application to multiple-criteria ordinal classification. Omega 84:189–198

    Article  Google Scholar 

  • Fernández E, Figueira JR, Navarro J (2020) Interval-based extensions of two outranking methods for multi-criteria ordinal classification. Omega 95:102065

    Article  Google Scholar 

  • Fürnkranz J, Hüllermeier E (2010) Preference learning: an introduction. In: J. Fürnkranz and E. Hüllermeier, (eds), Preference Learning, Springer pp 1–17

  • Ghahraman A, Prior D (2016) A learning ladder toward efficiency: proposing network-based stepwise benchmark selection. Omega 63:83–93

    Article  Google Scholar 

  • Goldstein WM (1991) Decomposable threshold models. J Math Psychol 35(1):64–79

    Article  Google Scholar 

  • Grabisch M (2016) Remarkable polyhedra related to set functions, games and capacities. TOP 24:301–326

    Article  Google Scholar 

  • Greco S, Matarazzo B, Słowiński R (2001) Conjoint measurement and rough set approach for multicriteria sorting problems in presence of ordinal criteria. In: Colorni A, Paruccini M, Roy B (eds) A-MCD-A. aide multicritère à la Décision / multiple criteria decision aid. European Commission, Joint Research Centre, Luxembourg, pp 117–144

    Google Scholar 

  • Greco S, Mousseau V, Słowiński R (2010) Multiple criteria sorting with a set of additive value functions. Eur J Oper Res 207(3):1455–1470

    Article  Google Scholar 

  • Greco S, Mousseau V, Słowiński R (2014) Robust ordinal regression for value functions handling interacting criteria. Eur J Oper Res 239(3):711–730

    Article  Google Scholar 

  • Gunning D, Aha D (2019) DARPA’s explainable artificial intelligence (XAI) program. AI Mag 40(2):44–58

    Google Scholar 

  • Guo M, Liao X, Liu J (2019) A progressive sorting approach for multiple criteria decision aiding in the presence of non-monotonic preferences. Expert Systems and Applications 123:1–17

    Article  Google Scholar 

  • Jacquet-Lagrèze E (1982) Binary preference indices: a new look on multicriteria aggregation procedures. Eur J Oper Res 10(1):25–32

    Article  Google Scholar 

  • Jacquet-Lagrèze E, Siskos Y (1982) Assessing a set of additive utility functions for multicriteria decision making: the UTA method. Eur J Oper Res 10:151–164

    Article  Google Scholar 

  • Janssen P, Nemery P (2013) An extension of the flowsort sorting method to deal with imprecision. 4OR 11:171–193

    Article  Google Scholar 

  • Kadziński M, Ciomek K (2016) Integrated framework for preference modeling and robustness analysis for outranking-based multiple criteria sorting with ELECTRE and PROMETHEE. Inf Sci 352–353:167–187

    Article  Google Scholar 

  • Kadziński M, Ciomek K, Rychly P, Słowiński R (2016) Post factum analysis for robust multiple criteria ranking and sorting. J Global Optim 65(3):531–562

    Article  Google Scholar 

  • Kadziński M, Ghaderi M, Dąbrowski M (2020) Contingent preference disaggregation model for multiple criteria sorting problem. Eur J Oper Res 281(2):369–387

    Article  Google Scholar 

  • Kadziński M, Stamenković M, Uniejewski M (2022) Stepwise benchmarking for multiple criteria sorting. Omega 108:102579

    Article  Google Scholar 

  • Köksalan M, Mousseau V, Özpeynirci Ö, Özpeynirci SB (2009) A new outranking-based approach for assigning alternatives to ordered classes. Nav Res Logistics 56:74–85

    Article  Google Scholar 

  • Labreuche C (2011) A general framework for explaining the results of a multi-attribute preference model. Artif Intell 175(7):1410–1448

    Article  Google Scholar 

  • Labreuche C, Fossier S (2018) Explaining multi-criteria decision aiding models with an extended shapley value. In: IJCAI International joint conference on artificial intelligence, pp 331 - 339

  • Labreuche C, Maudet N, Ouerdane W (2011) Minimal and complete explanations for critical multi-attribute decisions. In: Algorithmic decision theory, pp 121–134

  • Labreuche C, Maudet N, Ouerdane W (2012) Justifying dominating options when preferential information is incomplete. In: Proceedings of the 20th European conference on artificial intelligence, ECAI’12, pp 486-491. IOS Press

  • Liu J, Kadziński M, Liao X, Mao X, Wang Y (2020) A preference learning framework for multiple criteria sorting with diverse additive value models and valued assignment examples. Eur J Oper Res 286(3):963–985

    Article  Google Scholar 

  • Meyer P, Olteanu A-L (2019) Handling imprecise and missing evaluations in multi-criteria majority-rule sorting. Comput Oper Res 110:135–147

    Article  Google Scholar 

  • Minoungou P, Mousseau V, Ouerdane W, Scotton P (2022) A MIP-based approach to learn MR-Sort models with single-peaked preferences. Ann Oper Res

  • Mousseau V, Figueira JR, Dias LC, Gomes da Silva C, Clímaco J (2003) Resolving inconsistencies among constraints on the parameters of an MCDA model. Eur J Oper Res 147(1):72–93

    Article  Google Scholar 

  • Mousseau V, Dias LC, Figueira J (2006) Dealing with inconsistent judgments in multiple criteria sorting models. 4OR 4:145–158

    Article  Google Scholar 

  • Mousseau V, Özpeynirci Ö, Özpeynirci S (2018) Inverse multiple criteria sorting problem. Ann Oper Res 267(1–2):379–412

    Article  Google Scholar 

  • Pelissari R, Oliveira MC, Abackerli AJ, Ben Amor S, Pontes Assumpção MR (2021) Techniques to model uncertain input data of multi-criteria decision-making problems: a literature review. Int Trans Oper Res 28(2):523–559

    Article  Google Scholar 

  • Pereira J, de Oliveira ECB, Gomes LFAM, Araujo RM (2019) Sorting retail locations in a large urban city by using ELECTRE TRI-C and trapezoidal fuzzy numbers. Soft Comput 23(12):4193–4206

    Article  Google Scholar 

  • Pereira J, de Oliveira ECB, Morais DC, Costa APCS, López PA (2019) ELECTRE TRI-C with hesitant outranking functions: application to supplier development. J Intell Fuzzy Syst 37(6):7923–7933

    Article  Google Scholar 

  • Petrović M, Bojković N, Anić I, Stamenković M, Pejčić-Tarle S (2014) An ELECTRE-based decision aid tool for stepwise benchmarking: anapplication over EU Digital Agenda targets. Decis Support Syst 59:230–241

    Article  Google Scholar 

  • Rocha C, Dias LC (2008) An algorithm for ordinal sorting based on ELECTRE with categories defined by examples. J Global Optim 42(2):255–277

    Article  Google Scholar 

  • Roy B, Bouyssou D (1993) Aide multicritère à la décision: méthodes et cas. Economica Paris

  • Słowínski R, Greco S, Matarazzo B (2002) Axiomatization of utility, outranking and decision-rule preference models for multiple-criteria classification problems under partial inconsistency with the dominance principle. Control Cybern 31(4):1005–1035

    Google Scholar 

  • Tehrani AF, Hüllermeier E (2013) Ordinal Choquistic regression. In: Proceedings of the 8th conference of the European society for fuzzy logic and technology. Atlantis Press

  • Tehrani AF, Cheng W, Dembczyński K, Hüllermeier E (2012) Learning monotone nonlinear models using the Choquet integral. Mach Learn 89(1–2):183–211

    Google Scholar 

  • Wakker PP (1989) Additive representations of preferences: a new foundation of decision analysis. Kluwer, Dordrecht

    Book  Google Scholar 

  • Walton D (1996) Argumentation schemes for presumptive reasoning. Erlbaum, Mahwah

    Google Scholar 

  • Wei Y (1992) Aide multicritère à la décision dans le cadre de la problématique du tri : concepts, méthodes et applications. Thèse de doctorat, Université Paris Dauphine, Paris, France (in French)

  • Zhong Q, Fan X, Luo X, Toni F (2019) An explainable multi-attribute decision model based on argumentation. Expert Syst Appl 117:42–61

    Article  Google Scholar 

Download references

Acknowledgements

We are grateful to Denis Bouyssou for reading a previous version of the manuscript and making a number of relevant comments. We also thank the Editors for inviting us to write this survey and for their observations on the final draft. Of course, the responsibility for errors and omissions in this paper as well as the opinions that are expressed remains entirely with the authors.

Author information

Authors and Affiliations

  1. Heudiasyc, Université de Technologie de Compiègne, Compiègne, France

    Khaled Belahcène

  2. MICS, CentraleSupélec, Université Paris-Saclay, Gif-Sur-Yvette, France

    Vincent Mousseau & Wassila Ouerdane

  3. MATHRO, Faculté Polytechnique, Université de Mons, Mons, Belgium

    Marc Pirlot & Olivier Sobrie

Authors
  1. Khaled Belahcène
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Vincent Mousseau
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Wassila Ouerdane
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Marc Pirlot
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Olivier Sobrie
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Marc Pirlot.

Ethics declarations

Conflict of interest

The authors declare that they have no conflict of interest.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Appendix A list of abbreviations

Appendix A list of abbreviations

For the reader’s convenience, we list below, in alphabetic order, the acronyms used in the text, except for acronyms of sorting methods.

  • AVF: Additive Value Function

  • CAI: Class Acceptability Index

  • DM: Decision Maker

  • DRSA: Dominance based Rough Sets Approach

  • LP: Linear Program

  • MCDM/A: Multiple Criteria Decision Making / Aiding

  • MCS: Multiple Criteria Sorting

  • MILP: Mixed Integer Linear Program

  • ML: Machine Learning

  • MOP: Monotone Ordered Partition

  • PL: Preference Learning

  • ROR: Robust Ordinal Regression

  • SMAA: Stochatic Multicriteria Acceptability Analysis

  • VF: Value Function

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Belahcène, K., Mousseau, V., Ouerdane, W. et al. Multiple criteria sorting models and methods. Part II: theoretical results and general issues. 4OR-Q J Oper Res 21, 181–204 (2023). https://doi.org/10.1007/s10288-022-00531-3

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10288-022-00531-3

Keywords

Mathematics Subject Classification

Search

Navigation