Visualising multi-criteria weight elicitation by multiple stakeholders in complex decision systems

Abstract

An efficient and transparent weight elicitation technique is proposed for inclusion into the adaptive, systemic, control and multi-criteria-based methodology, in short ASCM, the purpose of which is piloting in real time complex systems by combining system dynamics (SD) and multi-criteria decision analysis (MCDA). Piloting policies are established and revised on a regular basis and/or constant real-time observation by means of SD simulations; at each revision step groups of stakeholders choose by means of MCDA tools the best policy to be implemented for the ensuing time periods when adaptations are necessary to account for the actual system evolution. An essential but difficult issue at each policy revision step is the weight elicitation process of multiple criteria by the multiple stakeholder groups (SH). The proposed procedure with a strong mathematical background does not require excessive cognitive effort for SH with different priorities and decisional powers. It consists in a two-step approach defining firstly importance classes on ordinal Likert scales, and secondly profiles on those scales for the criteria. It appears to be simple though rigorous; it easily allows fast sensitivity analyses when confronting different opinions. A didactic example and a fishery-management case study illustrate these properties by means of visualisation tools facilitating consensus-seeking among SH.

This is a preview of subscription content, log in to check access.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7

References

  1. Ahn BS (2017) Approximate weighting method for multiattribute decision problems with imprecise parameters. Omega 72:87–95

    Article  Google Scholar 

  2. Arnold BC, Balakrishnan N, Nagaraja N (1992) A first course in order statistics. Wiley, New York

    Google Scholar 

  3. Barron FH (1992) Selecting a best multiattribute alternative with partial information about attribute weights. Acta Physiol (Oxf) 80(1–3):91–103

    Google Scholar 

  4. Barron FH, Barrett BE (1996) Decision quality using ranked attribute weights. Manag Sci 42(21):1515–1523

    Article  Google Scholar 

  5. Behzadian M, Kazemzadeh RB, Albadvi A, Aghdasi M (2010) PROMETHEE: a comprehensive literature review on methodologies and applications. Eur J Oper Res 200(1):198–215. https://doi.org/10.1016/j.ejor.2009.01.021

    Article  Google Scholar 

  6. Brans JP, Kunsch PL (2010) Ethics in operations research and sustainable development. Int Trans Oper Res (ITOR) 17:427–444

    Article  Google Scholar 

  7. Brans JP, Mareschal B (2016) PROMETHEE methods. In: Figueira J, Greco S, Ehrgott M (eds) Multiple criteria decision analysis: state of the art surveys, 2nd edn. Springer, London

    Google Scholar 

  8. Brans JP, Vincke P (1985) A preference ranking organization method. Manag Sci 31(6):647–656

    Article  Google Scholar 

  9. Brans JP, Macharis C, Kunsch PL, Chevalier A, Schwaninger M (1998) Combining multicriteria decision aid and system dynamics for the control of socio-economic processes. An iterative real-time procedure. Eur J Oper Res 109:428–441

    Article  Google Scholar 

  10. Brans JP, Kunsch PL, Mareschal B (2001) Management of the future. A system dynamics and MCDA approach. In: Bouyssou D, Jacquet-Lagrèze E, Perny P, Slowinski R, Vanderpooten D, Vincke P (eds) Special volume dedicated to Professor Bernard Roy. Kluwer Academic Publishers, Dordrecht, pp 483–502

    Google Scholar 

  11. Figueira J, Roy B (2002) Determining the weights of criteria in the ELECTRE type methods with a revised Simos’ procedure. Eur J Oper Res 139:317–326

    Article  Google Scholar 

  12. Figueira J, Greco S, Ehrgott M (2016) Multiple criteria decision analysis: state of the art surveys, 2nd edn. Springer, Boston

    Google Scholar 

  13. Ishizaka A, Nemery Ph (2013) Multi-criteria decision analysis. Methods and software. Wiley, Chichester

    Google Scholar 

  14. Klimberg R, Cohen RM (1999) Experimental evaluation of a graphical display system to visualizing multiple criteria solutions. Eur J Oper Res 119(1):191–208

    Article  Google Scholar 

  15. Kunsch PL, Ishizaka A (2018) Multiple-criteria performance ranking based on profile distributions: an application to university research evaluations. Math Comput Simul 154:48–64. https://doi.org/10.1016/j.matcom.2018.05.021

    Article  Google Scholar 

  16. Kunsch PL, Kavathatzopoulos I, Rauschmayer F (2009) Modelling complex ethical decision problems with operations research. Omega Spec Issue Ethics and Oper Res 37(6):1100–1108

    Google Scholar 

  17. Mareschal B (2014) Visual PROMETHEE 1.4 Academic Edition (freeware) http://www.promethee-gaia.net/software.html

  18. Miettinen K (2014) Survey of methods to visualize alternatives in multiple criteria decision making problems. OR Spectr 36(1):3–37. https://doi.org/10.1007/s00291-012-0297-0

    Article  Google Scholar 

  19. Miller GA (1956) The magic number seven plus or minus two: some limits on our capacity for processing information. Psychol Rev 13:81–97

    Article  Google Scholar 

  20. Simos J (1990) Evaluer l’impact sur l’environnement: Une approche originale par l’analyse multicritère et la négociation. (Evaluate the impact on the environment. A new approach based on multi-criteria analysis and negotiation), Presses Polytechniques et Universitaires Romandes, Lausanne, Switzerland

  21. Sterman JD (2000) Business dynamics—systems thinking and modeling for a complex world. Irwin McGraw-Hill, Boston

    Google Scholar 

  22. Stillwell WG, Seaver DA, Edwards W (1981) A comparison of weight approximation techniques in multiattribute utility decision making. Organ Behav Hum Perform 38:62–77

    Article  Google Scholar 

  23. Ventana Systems, Inc. RightChoice © (2001) www.ventanasystems.co.uk/services/software/rightchoice

  24. Ventana Systems, Inc. Vensim © (2010) User Guide & Software www.vensim.com

  25. Wang J, Zionts S (2015) Using ordinal data to estimate cardinal values. J Multi-Criteria Decis Anal 22:185–196

    Article  Google Scholar 

Download references

Author information

Affiliations

Authors

Corresponding author

Correspondence to Pierre L. Kunsch.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Appendix: Proofs of the property that the centroid weights of ranked weights are the same as Rank-Sum surrogate weights (Sect. 3.1 of the main text)

Appendix: Proofs of the property that the centroid weights of ranked weights are the same as Rank-Sum surrogate weights (Sect. 3.1 of the main text)

We start with the observation that not-normalised weights are defined up to a positive multiplicative factor. In the elicitation process, there is no need to produce normalised weights, so that the Rank Order Centroid (ROC) simplex (2) in Sect. 3.1 should not include a normalising constraint, while the not-normalised weights can be bound by any positive number, using 1 without loss of generality we then consider the weight simplex:

$$W \in \left\{ {0 \le W_{1} \le W_{2} \le \cdots \le W_{K} \le 1} \right\}$$
(A1.1)

First proof

All point values in the simplex (A1.1) give equally valid sets of not-normalised weights, because the sole available information from stakeholders is ranking (A1.1). In the K-D space the simplex is a triangle K2 in 2-D, a tetrahedron K3 in 3-D, and generally a K-simplex. It is well-known that triangle-based simplices (A1.1) are such that their centroid’s coordinates (=centre-of-gravity in the physical sense) are the mean values of the coordinates of their extreme points (Ahn 2017).

The (K + 1) extreme points of any K-simplex are clearly \(\left( {0, \ldots ,0} \right);\left( {0,0, \ldots ,1} \right);\left( {0, \ldots ,1,1} \right); \ldots ;\left( {1, \ldots ,1} \right)\), so that the centroid coordinates, i.e., the centroid weights are given by:

$$W = \frac{{\left( {1,2, \ldots ,K} \right)}}{K + 1}$$
(A1.2)

These values are proportional to the not-normalised rank-sum (RS) weights (1) in Sect. 3.1.□

Second proof

As all point values in the K-simplex (A1.1) bounded by 1 can equally be chosen as not-normalised weight sets, the centroid weights are obtained as the set of mean values of K random ordered-values in [0,1]. This is easily done by generating many times K random numbers uniformly distributed in the [0,1] interval, ranking them in increasing order, and obtaining their mean values. The K ranked random numbers obey order-statistics probability laws (Arnold et al. 1992; pp. 13–14); it is known that their mean values—being the not-normalised weights—are given by (A1.2) as obtained numerically in approximation with simulations.□

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Kunsch, P.L., Brans, J. Visualising multi-criteria weight elicitation by multiple stakeholders in complex decision systems. Oper Res Int J 19, 955–971 (2019). https://doi.org/10.1007/s12351-018-00446-0

Download citation

Keywords

  • Complex systems
  • Multiple criteria
  • Weight elicitation
  • Multiple stakeholders
  • Visualisation tools

Mathematics Subject Classification

  • 90B50