A linguistic approach to assess the dynamics of design team preference in concept selection
This paper addresses the problem of describing the decision-making process of a committee of engineers based upon their verbalized linguistic appraisals of alternatives. First, we show a way to model an individual’s evaluation of an alternative through natural language based on the Systemic-Functional Linguistics system of APPRAISAL. The linguistic model accounts for both the degree of intensity and the uncertainty of expressed evaluations. Second, this multi-dimensional linguistic model is converted into a scalar to represent the degree of intensity and a probability distribution function for the stated evaluation. Finally, we present a Markovian model to calculate the time-varying change in preferential probability, the probability that an alternative is the most preferred alternative. We further demonstrate how preferential probability toward attributes of alternatives correspond to preferential probability toward alternatives. We illustrate the method on two case studies to highlight the time-variant dynamics of preferences toward alternatives and attributes. This research contributes to process tracing in descriptive decision science to understand how engineers actually take decisions.
KeywordsDecision-based design Ranking alternatives Social choice
The authors wish to thank the participation of the engineering students in the experiments. The work described in this paper was supported in part by the National Science Foundation under Award CMMI-0900255. This research was also supported in part under Australian Research Council’s Discovery Projects funding scheme (Project Number DP1095601). The opinions, findings, conclusions, and recommendations expressed are those of the authors and do not necessarily reflect the views of the sponsors.
- Dong A (2009) The language of design: theory and computation. Springer, LondonGoogle Scholar
- Dym CL, Wood WH, Scott MJ (2002) Rank ordering engineering designs: pairwise comparison charts and Borda counts. Res Eng Design 13(4):236–242Google Scholar
- Garbuio M, Lovallo D (2011) The under-appreciated role of quality conversations in strategic decision-making. In: 71st annual meeting of the academy of management AoM 2011. Academy of ManagementGoogle Scholar
- Kahneman D, Lovallo D, Sibony O (2011) Before you make that big decision. Harv Bus Rev 89(6):50–60Google Scholar
- Martin JR, White PRR (2005) The language of evaluation: appraisal in English. Palgrave Macmillan, New YorkGoogle Scholar
- Paolacci G, Chandler J, Ipeirotis PG (2010) Running experiments on amazon mechanical turk. Judgm Decis Mak 5(5):411–419Google Scholar
- Pugh S (1991) Total design: integrated methods for successful product engineering. Addison-Wesley, ReadingGoogle Scholar
- Sorokin A, Forsyth D (2008) Utility data annotation with amazon mechanical turk. In: IEEE computer society conference on computer vision and pattern recognition workshops, 2008. CVPRW’08, pp 1–8. doi: 10.1109/CVPRW.2008.4562953
- Ulrich KT, Eppinger SD (2004) Product design and development, 3rd edn. McGraw-Hill/Irwin, New YorkGoogle Scholar
- Weick KE (1995) Sensemaking in organizations. Sage Publications, Thousand OaksGoogle Scholar
- Whitelaw C, Garg N, Argamon S (2005) Using appraisal groups for sentiment analysis. In: Proceedings of the 14th ACM international conference on information and knowledge management, CIKM’05, pp. 625–631. ACM, New York, NY, USA. doi: 10.1145/1099554.1099714