## Abstract

The problem of collective decision by design teams has received considerable attention in the scientific literature of engineering design. A much debated problem is that in which multiple designers formulate their individual preference rankings of different design alternatives and these rankings should be aggregated into a collective one. This paper focuses the attention on three basic research questions: (1) “How can the degree of concordance of designer rankings be measured?”, (2) “For a given set of designer rankings, which aggregation model provides the most coherent solution?”, and (3) “To what extent is the collective ranking influenced by the aggregation model in use?”. The aim of this paper is to present a novel approach that addresses the above questions in a relatively simple and agile way. A detailed description of the methodology is supported by a practical application to a real-life case study.

This is a preview of subscription content, log in to check access.

## Notes

- 1.
This concept will be clarified later.

- 2.
I.e., the solution that best reflects designer rankings.

## References

Arrow KJ (2012) Social choice and individual values, 3rd edn. Yale University Press, New Haven

Arrow KJ, Rayanaud H (1986) Social choice and multicriterion decision-making. MIT Press, Cambridge

Borda JC (1781) Mémoire sur les élections au scrutin, Comptes Rendus de l’Académie des Sciences. Translated by Alfred de Grazia as Mathematical derivation of an election system. Isis 44:42–51

Bormann NC, Golder M (2013) Democratic electoral systems around the world. Electoral Stud 32(2):1946–2011, 360–369

Cagan J, Vogel CM (2012) Creating breakthrough products: innovation from product planning to program approval, 2nd edn. FT Press, Upper Saddle River

Chen S, Liu J, Wang H, Augusto JC (2012) Ordering based decision making—a survey. Inf Fusion 14(4):521–531

Chiclana F, Herrera F, Herrera-Viedma E (2002) A note on the internal consistency of various preference representations. Fuzzy Sets Syst 131(1):75–78

Cook WD (2006) Distance-based and ad hoc consensus models in ordinal preference ranking. Eur J Oper Res 172(2):369–385

Dong A, Hill AW, Agogino AM (2004) A document analysis method for characterizing design team performance. J Mech Des 126(3):378–385

Dwarakanath S, Wallace KM (1995) Decision-making in engineering design—observations from design experiments. J Eng Des 6(3):191–206

Dym CL, Wood WH, Scott MJ (2002) Rank ordering engineering designs: pairwise comparison charts and Borda counts. Res Eng Des 13:236–242

Fishburn PC (1973a) Voter concordance, simple majorities, and group decision methods. Behav Sci 18:364–376

Fishburn PC (1973b) The theory of social choice. Princeton University Press, Princeton

Franceschini F, Maisano D (2015a) Checking the consistency of the solution in ordinal semi-democratic decision-making problems. Omega 57(1):188–195

Franceschini F, Maisano D (2017) Consistency analysis in quality classification problems with multiple rank-ordered agents. Qual Eng 29(4):672–689

Franceschini F, Maisano D (2018) Fusion of partial orderings for decision problems in quality management. In: Proceedings of the 3rd international conference on quality engineering and management (ICQEM 2018), July 11–13, 2018, Barcelona (Spain)

Franceschini F, Maisano D, Mastrogiacomo L (2015b) Customer requirements prioritization on QFD: a new proposal based on the Generalized Yager’s Algorithm. Res Eng Des 26(2):171–187

Franceschini F, Galetto M, Maisano D, Mastrogiacomo L (2015c) Prioritization of engineering characteristics in QFD in the case of customer requirements orderings. Int J Prod Res 53(13):3975–3988

Franceschini F, Maisano D, Mastrogiacomo L (2016) A new proposal for fusing individual preference orderings by rank-ordered agents: a generalization of the Yager’s algorithm. Eur J Oper Res 249(1):209–223

Franceschini F, Galetto M, Maisano D (2019) Designing performance measurement systems: theory and practice of key performance indicators, Springer, Cham. ISBN: 978-3-030-01191-8

Franssen M (2005) Arrow’s theorem, multi-criteria decision problems and multi-attribute preferences in engineering design. Res Eng Des 16(1–2):42–56

Frey DDet al (2009) The Pugh controlled convergence method: model-based evaluation and implications for design theory. Res Eng Des 20(1):41–58

Frey DD et al (2010) Research in engineering design: the role of mathematical theory and empirical evidence. Res Eng Des 21(3):145–151

Fu K, Cagan J, Kotovsky K (2010) Design team convergence: the influence of example solution quality. J Mech Des 132(11):111005

Hazelrigg GA (1996) The implications of Arrow’s impossiblity theorem on approaches to optimal engineering design. J Mech Des 118(2):161–164

Hazelrigg GA (1999) An axiomatic framework for engineering design. J Mech Des 121(3):342

Hazelrigg GA (2010) The Pugh controlled convergence method: model-based evaluation and implications for design theory. Res Eng Des 21(3):143–144

Herrera-Viedma E, Cabrerizo FJ, Kacprzyk J, Pedrycz W (2014) A review of soft consensus models in a fuzzy environment. Inf Fusion 17:4–13

Hoyle C, Chen W (2011) Understanding and modelling heterogeneity of human preferences for engineering design. J Eng Des 22(8):583–601

Jacobs JF, van de Poel I, Osseweijer P (2014) Clarifying the debate on selection methods for engineering: Arrow’s impossibility theorem, design performances, and information basis. Res Eng Des 25(1):3–10

Kaldate A et al (2006) Engineering parameter selection for design optimization during preliminary design. J Eng Des 17(March 2015):291–310

Katsikopoulos KV (2012) Decision methods for design: insights from psychology. J Mech Des 134(8):084504

Keeney RL (2009) The foundations of collaborative group decisions. Int J Collab Eng 1:4

Kendall (1962) Rank correlation methods. Griffin & C., London

Kendall MG, Smith BB (1939) The problem of m-rankings. Ann Math Stat 10:275–287

Ladha K, Miller G, Oppenheimer J (2003 Information aggregation by majority rule: theory and experiment. http://www.gvptsites.umd.edu/oppenheimer/research/jury.pdf

Legendre P (2005) Species associations: the Kendall coefficient of concordance revisited. J Agric Biol Environ Stat 10:226

Legendre P (2010) Coefficient of concordance. In: Salkind NJ (ed) Encyclopedia of research design, vol 1. SAGE Publications, Inc., Los Angeles, pp 164–169

Li H, Bingham JB, Umphress EE (2007) Fairness from the top? Perceived procedural justice and collaborative problem solving in new product development. Organ Sci 18(2):200–216

McComb C, Goucher-Lambert K, Cagan J(2015), Fairness and manipulation: an empirical study of Arrow’s impossibility theorem. In: International conference on engineering design, Milan, Italy, pp 267–276

McComb C, Goucher-Lambert K, Cagan J (2017) Impossible by design? Fairness, strategy and Arrow’s impossibility theorem. Des Sci 3:1–26

Nisan N (2007) Algorithmic game theory. Cambridge University Press, Cambridge

Nurmi H (2012) On the relevance of theoretical results to voting system choice. In: Felsenthal DS, Machover M (eds) Electoral systems: studies in choice and welfare. Springer, Berlin, pp 255–274

Olewnik AT, Lewis K (2008) Limitations of the house of quality to provide quantitative design information. Int J Qual Reliabil Manag 25(2):125–146

Önüt S, Kara SS, Efendigil T (2008) A hybrid fuzzy MCDM approach to machine tool selection. J Intell Manuf 19(4):443–453

Paulus PB, Dzindolet MT, Kohn N (2011) Collaborative creativity, group creativity and team innovation. In: Mumford MD (ed) Handbook of organizational creativity. Elsevier, Oxford, pp 327–357

Reich Y (2010) My method is better! Res Eng Des 21(3):137–142

Saari DG (2011a) Geometry of voting. Elsevier, Oxford

Saari DG (2011b) Decision and elections. Cambridge University Press, Cambridge

Saari DG, Sieberg KK (2004) Are partwise comparisons reliable? Res Eng Des 15(1):62–71

Scott MJ, Antonsson EK (1999) Arrow’s theorem and engineering design decision making. Res Eng Des 11:218–228

See TK, Lewis K (2006) A formal approach to handling conflicts in multiattribute group decision making. J Mech Des 128(4):678

Weingart LR et al. (2005) Functional diversity and conflict in cross-functional product development teams: considering representational gaps and task characteristics. In Neider LL, Schriesheim CA (eds) Understanding teams. Information Age Publishing, Charlotte, pp 89–110

Yager RR (2001) Fusion of multi-agent preference orderings. Fuzzy Sets Syst 117(1):1–12

Yeo SH, Mak MW, Balon SAP (2004) Analysis of decision-making methodologies for desirability score of conceptual design. J Eng Des 15(2):195–208

## Acknowledgements

This research was partially supported by the award “TESUN-83486178370409 Finanziamento dipartimenti di eccellenza CAP. 1694 TIT. 232 ART. 6”, which was conferred by “Ministero dell’Istruzione, dell’Università e della Ricerca-ITALY”.

## Author information

### Affiliations

### Corresponding author

## Additional information

### Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

## Appendix

### Appendix

### Statistical meaning of *W*

This subsection analyzes the statistical significance of *W*. In general, the *W* distributions are available in tabular terms for small values of *m* and *n* (Kendall 1962). For higher values, the Fisher distribution can be used:

with parameters \({\nu _1}\) and \({\nu _2}\) defined, respectively, as

When *n* > 7, \({W^{(m)}}\) can be described by a chi-square distribution \(\chi _{r}^{2}=m\left( {n - 1} \right){W^{(m)}}\). \(\chi _{r}^{2}\) is distributed as a \(\chi _{{n - 1}}^{2}\) with \(\nu =n - 1\) degrees of freedom.

For example, considering the data in Table 6, where \(m=10\) designers and \(n=4\) design concepts, \({W^{(m)}}=0.29\). Applying Eq. (8), it can be obtained:

The degrees of freedom are, respectively:

From the tables of the Fisher distribution for a significance of 5%, it is obtained \({F_{5\% ;3;25}}=2.99\).

Since \(F>{F_{5\% ;3;25}}\), the significance of the coefficient of concordance for the preference profile in Table 6 is confirmed.

## Rights and permissions

## About this article

### Cite this article

Franceschini, F., Maisano, D. Design decisions: concordance of designers and effects of the Arrow’s theorem on the collective preference ranking.
*Res Eng Design* **30, **425–434 (2019). https://doi.org/10.1007/s00163-019-00313-9

Received:

Accepted:

Published:

Issue Date:

### Keywords

- Engineering design
- Collective decision-making
- Design teams
- Preference ranking
- Collective ranking
- Degree of concordance