Design decisions: concordance of designers and effects of the Arrow’s theorem on the collective preference ranking

Abstract

The problem of collective decision by design teams has received considerable attention in the scientific literature of engineering design. A much debated problem is that in which multiple designers formulate their individual preference rankings of different design alternatives and these rankings should be aggregated into a collective one. This paper focuses the attention on three basic research questions: (1) “How can the degree of concordance of designer rankings be measured?”, (2) “For a given set of designer rankings, which aggregation model provides the most coherent solution?”, and (3) “To what extent is the collective ranking influenced by the aggregation model in use?”. The aim of this paper is to present a novel approach that addresses the above questions in a relatively simple and agile way. A detailed description of the methodology is supported by a practical application to a real-life case study.

This is a preview of subscription content, log in to check access.

Fig. 1

Notes

  1. 1.

    This concept will be clarified later.

  2. 2.

    I.e., the solution that best reflects designer rankings.

References

  1. Arrow KJ (2012) Social choice and individual values, 3rd edn. Yale University Press, New Haven

    Google Scholar 

  2. Arrow KJ, Rayanaud H (1986) Social choice and multicriterion decision-making. MIT Press, Cambridge

    Google Scholar 

  3. Borda JC (1781) Mémoire sur les élections au scrutin, Comptes Rendus de l’Académie des Sciences. Translated by Alfred de Grazia as Mathematical derivation of an election system. Isis 44:42–51

    Google Scholar 

  4. Bormann NC, Golder M (2013) Democratic electoral systems around the world. Electoral Stud 32(2):1946–2011, 360–369

    Article  Google Scholar 

  5. Cagan J, Vogel CM (2012) Creating breakthrough products: innovation from product planning to program approval, 2nd edn. FT Press, Upper Saddle River

    Google Scholar 

  6. Chen S, Liu J, Wang H, Augusto JC (2012) Ordering based decision making—a survey. Inf Fusion 14(4):521–531

    Article  Google Scholar 

  7. Chiclana F, Herrera F, Herrera-Viedma E (2002) A note on the internal consistency of various preference representations. Fuzzy Sets Syst 131(1):75–78

    MathSciNet  MATH  Article  Google Scholar 

  8. Cook WD (2006) Distance-based and ad hoc consensus models in ordinal preference ranking. Eur J Oper Res 172(2):369–385

    MathSciNet  MATH  Article  Google Scholar 

  9. Dong A, Hill AW, Agogino AM (2004) A document analysis method for characterizing design team performance. J Mech Des 126(3):378–385

    Article  Google Scholar 

  10. Dwarakanath S, Wallace KM (1995) Decision-making in engineering design—observations from design experiments. J Eng Des 6(3):191–206

    Article  Google Scholar 

  11. Dym CL, Wood WH, Scott MJ (2002) Rank ordering engineering designs: pairwise comparison charts and Borda counts. Res Eng Des 13:236–242

    Article  Google Scholar 

  12. Fishburn PC (1973a) Voter concordance, simple majorities, and group decision methods. Behav Sci 18:364–376

    Article  Google Scholar 

  13. Fishburn PC (1973b) The theory of social choice. Princeton University Press, Princeton

    Google Scholar 

  14. Franceschini F, Maisano D (2015a) Checking the consistency of the solution in ordinal semi-democratic decision-making problems. Omega 57(1):188–195

    Article  Google Scholar 

  15. Franceschini F, Maisano D (2017) Consistency analysis in quality classification problems with multiple rank-ordered agents. Qual Eng 29(4):672–689

    Article  Google Scholar 

  16. Franceschini F, Maisano D (2018) Fusion of partial orderings for decision problems in quality management. In: Proceedings of the 3rd international conference on quality engineering and management (ICQEM 2018), July 11–13, 2018, Barcelona (Spain)

  17. Franceschini F, Maisano D, Mastrogiacomo L (2015b) Customer requirements prioritization on QFD: a new proposal based on the Generalized Yager’s Algorithm. Res Eng Des 26(2):171–187

    Article  Google Scholar 

  18. Franceschini F, Galetto M, Maisano D, Mastrogiacomo L (2015c) Prioritization of engineering characteristics in QFD in the case of customer requirements orderings. Int J Prod Res 53(13):3975–3988

    Article  Google Scholar 

  19. Franceschini F, Maisano D, Mastrogiacomo L (2016) A new proposal for fusing individual preference orderings by rank-ordered agents: a generalization of the Yager’s algorithm. Eur J Oper Res 249(1):209–223

    MathSciNet  MATH  Article  Google Scholar 

  20. Franceschini F, Galetto M, Maisano D (2019) Designing performance measurement systems: theory and practice of key performance indicators, Springer, Cham. ISBN: 978-3-030-01191-8

    Google Scholar 

  21. Franssen M (2005) Arrow’s theorem, multi-criteria decision problems and multi-attribute preferences in engineering design. Res Eng Des 16(1–2):42–56

    Article  Google Scholar 

  22. Frey DDet al (2009) The Pugh controlled convergence method: model-based evaluation and implications for design theory. Res Eng Des 20(1):41–58

    Article  Google Scholar 

  23. Frey DD et al (2010) Research in engineering design: the role of mathematical theory and empirical evidence. Res Eng Des 21(3):145–151

    Article  Google Scholar 

  24. Fu K, Cagan J, Kotovsky K (2010) Design team convergence: the influence of example solution quality. J Mech Des 132(11):111005

    Article  Google Scholar 

  25. Hazelrigg GA (1996) The implications of Arrow’s impossiblity theorem on approaches to optimal engineering design. J Mech Des 118(2):161–164

    Article  Google Scholar 

  26. Hazelrigg GA (1999) An axiomatic framework for engineering design. J Mech Des 121(3):342

    Article  Google Scholar 

  27. Hazelrigg GA (2010) The Pugh controlled convergence method: model-based evaluation and implications for design theory. Res Eng Des 21(3):143–144

    Article  Google Scholar 

  28. Herrera-Viedma E, Cabrerizo FJ, Kacprzyk J, Pedrycz W (2014) A review of soft consensus models in a fuzzy environment. Inf Fusion 17:4–13

    Article  Google Scholar 

  29. Hoyle C, Chen W (2011) Understanding and modelling heterogeneity of human preferences for engineering design. J Eng Des 22(8):583–601

    Article  Google Scholar 

  30. Jacobs JF, van de Poel I, Osseweijer P (2014) Clarifying the debate on selection methods for engineering: Arrow’s impossibility theorem, design performances, and information basis. Res Eng Des 25(1):3–10

    Article  Google Scholar 

  31. Kaldate A et al (2006) Engineering parameter selection for design optimization during preliminary design. J Eng Des 17(March 2015):291–310

    Article  Google Scholar 

  32. Katsikopoulos KV (2012) Decision methods for design: insights from psychology. J Mech Des 134(8):084504

    Article  Google Scholar 

  33. Keeney RL (2009) The foundations of collaborative group decisions. Int J Collab Eng 1:4

    Article  Google Scholar 

  34. Kendall (1962) Rank correlation methods. Griffin & C., London

    Google Scholar 

  35. Kendall MG, Smith BB (1939) The problem of m-rankings. Ann Math Stat 10:275–287

    MathSciNet  MATH  Article  Google Scholar 

  36. Ladha K, Miller G, Oppenheimer J (2003 Information aggregation by majority rule: theory and experiment. http://www.gvptsites.umd.edu/oppenheimer/research/jury.pdf

  37. Legendre P (2005) Species associations: the Kendall coefficient of concordance revisited. J Agric Biol Environ Stat 10:226

    Article  Google Scholar 

  38. Legendre P (2010) Coefficient of concordance. In: Salkind NJ (ed) Encyclopedia of research design, vol 1. SAGE Publications, Inc., Los Angeles, pp 164–169

    Google Scholar 

  39. Li H, Bingham JB, Umphress EE (2007) Fairness from the top? Perceived procedural justice and collaborative problem solving in new product development. Organ Sci 18(2):200–216

    Article  Google Scholar 

  40. McComb C, Goucher-Lambert K, Cagan J(2015), Fairness and manipulation: an empirical study of Arrow’s impossibility theorem. In: International conference on engineering design, Milan, Italy, pp 267–276

  41. McComb C, Goucher-Lambert K, Cagan J (2017) Impossible by design? Fairness, strategy and Arrow’s impossibility theorem. Des Sci 3:1–26

    Article  Google Scholar 

  42. Nisan N (2007) Algorithmic game theory. Cambridge University Press, Cambridge

    Google Scholar 

  43. Nurmi H (2012) On the relevance of theoretical results to voting system choice. In: Felsenthal DS, Machover M (eds) Electoral systems: studies in choice and welfare. Springer, Berlin, pp 255–274

    Google Scholar 

  44. Olewnik AT, Lewis K (2008) Limitations of the house of quality to provide quantitative design information. Int J Qual Reliabil Manag 25(2):125–146

    Article  Google Scholar 

  45. Önüt S, Kara SS, Efendigil T (2008) A hybrid fuzzy MCDM approach to machine tool selection. J Intell Manuf 19(4):443–453

    Article  Google Scholar 

  46. Paulus PB, Dzindolet MT, Kohn N (2011) Collaborative creativity, group creativity and team innovation. In: Mumford MD (ed) Handbook of organizational creativity. Elsevier, Oxford, pp 327–357

    Google Scholar 

  47. Reich Y (2010) My method is better! Res Eng Des 21(3):137–142

    Article  Google Scholar 

  48. Saari DG (2011a) Geometry of voting. Elsevier, Oxford

    Google Scholar 

  49. Saari DG (2011b) Decision and elections. Cambridge University Press, Cambridge

    Google Scholar 

  50. Saari DG, Sieberg KK (2004) Are partwise comparisons reliable? Res Eng Des 15(1):62–71

    Article  Google Scholar 

  51. Scott MJ, Antonsson EK (1999) Arrow’s theorem and engineering design decision making. Res Eng Des 11:218–228

    Article  Google Scholar 

  52. See TK, Lewis K (2006) A formal approach to handling conflicts in multiattribute group decision making. J Mech Des 128(4):678

    Article  Google Scholar 

  53. Weingart LR et al. (2005) Functional diversity and conflict in cross-functional product development teams: considering representational gaps and task characteristics. In Neider LL, Schriesheim CA (eds) Understanding teams. Information Age Publishing, Charlotte, pp 89–110

    Google Scholar 

  54. Yager RR (2001) Fusion of multi-agent preference orderings. Fuzzy Sets Syst 117(1):1–12

    MathSciNet  MATH  Article  Google Scholar 

  55. Yeo SH, Mak MW, Balon SAP (2004) Analysis of decision-making methodologies for desirability score of conceptual design. J Eng Des 15(2):195–208

    Article  Google Scholar 

Download references

Acknowledgements

This research was partially supported by the award “TESUN-83486178370409 Finanziamento dipartimenti di eccellenza CAP. 1694 TIT. 232 ART. 6”, which was conferred by “Ministero dell’Istruzione, dell’Università e della Ricerca-ITALY”.

Author information

Affiliations

Authors

Corresponding author

Correspondence to Fiorenzo Franceschini.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Appendix

Appendix

Statistical meaning of W

This subsection analyzes the statistical significance of W. In general, the W distributions are available in tabular terms for small values of m and n (Kendall 1962). For higher values, ​​the Fisher distribution can be used:

$$F=\frac{{(m - 1) \times {W^{(m)}}}}{{[1 - {W^{(m)}}]}}$$
(17)

with parameters \({\nu _1}\) and \({\nu _2}\) defined, respectively, as

$$\left\{ {\begin{array}{ll} {{\nu _1}=n - 1 - \frac{2}{m}} \\ {{\nu _2}=(m - 1)\left( {n - 1 - \frac{2}{m}} \right)} \end{array}} \right..$$
(18)

When n > 7, \({W^{(m)}}\) can be described by a chi-square distribution \(\chi _{r}^{2}=m\left( {n - 1} \right){W^{(m)}}\). \(\chi _{r}^{2}\) is distributed as a \(\chi _{{n - 1}}^{2}\) with \(\nu =n - 1\) degrees of freedom.

For example, considering the data in Table 6, where \(m=10\) designers and \(n=4\) design concepts, \({W^{(m)}}=0.29\). Applying Eq. (8), it can be obtained:

$$F=\frac{{(10 - 1) \times 0.29}}{{(1 - 0.29)}}=3.67.$$
(19)

The degrees of freedom are, respectively:

$$\left\{ {\begin{array}{ll} {{\nu _1}=4 - 1 - \frac{2}{{10}}=2.8 \approx 3} \\ {{\nu _2}=\left( {m - 1} \right)\left( {n - 1 - \frac{2}{m}} \right)=25.2 \approx 25} \end{array}} \right..$$
(20)

From the tables of the Fisher distribution for a significance of 5%, it is obtained \({F_{5\% ;3;25}}=2.99\).

Since \(F>{F_{5\% ;3;25}}\), the significance of the coefficient of concordance for the preference profile in Table 6 is confirmed.

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Franceschini, F., Maisano, D. Design decisions: concordance of designers and effects of the Arrow’s theorem on the collective preference ranking. Res Eng Design 30, 425–434 (2019). https://doi.org/10.1007/s00163-019-00313-9

Download citation

Keywords

  • Engineering design
  • Collective decision-making
  • Design teams
  • Preference ranking
  • Collective ranking
  • Degree of concordance