Research in Engineering Design

, Volume 23, Issue 2, pp 85–103

An approach to the extraction of preference-related information from design team language

Original Paper


The process of selecting among design alternatives is an important activity in the early stages of design. A designer is said to express design preferences when assigning priorities to a set of possible design choices. However, the assignment of preferences becomes more challenging on both a practical and theoretical level when performed by a group. This paper presents a probabilistic approach for estimating a team’s overall preference-related information known as preferential probabilities that extracts information from the natural language used in team discussion transcripts without aggregation of individual team member opinions. Assessment of the method is conducted by surveying a design team to obtain quantitative ratings of alternatives. Two different approaches are applied to convert these ratings into values that may be compared to the results of transcript analysis: the application of a modified Logit model and simulation based on the principle of maximum entropy. The probabilistic approach proposed in the paper represents how likely a choice is to be “most preferred” by a design team over a given period of time. A preliminary design selection experiment was conducted as an illustrative case example of the method. Correlations were found between the preferential probabilities estimated from transcripts and those computed from the surveyed preferences. The proposed methods may provide a formal way to understand and represent informal, unstructured design information using a low overhead information extraction method.


Design preferences Design decision-making Concept selection Design process 


  1. Arnold K (2001) Making team decision. In: Biech E (ed) The Pfeiffer book of successful team building tools. Jossey-Bass/Pfeiffer, San FranciscoGoogle Scholar
  2. Arrow KJ (1970) Social choice and individual values. Yale University Press, New HavenGoogle Scholar
  3. Arrow KJ, Raynaud H (1986) Social choice and multicriterion decision-making. MIT, Cambridge, MAMATHGoogle Scholar
  4. Ben-Akiva M, Lerman SR (1985) Discrete choice analysis. MIT, Cambridge, MAGoogle Scholar
  5. Bertrand M, Mullainathan S (2001) Do people mean what they say? Implications for subjective survey data. Am Econ Rev 91(2):67–72CrossRefGoogle Scholar
  6. Bilmes JA (1998) A gentle tutorial of the EM algorithm and its application to parameter estimation for Gaussian mixture and hidden Markov models. Technical report. International Computer Science Institute, Berkeley, CA, USAGoogle Scholar
  7. Bockenholt U (2002) A Thurstonian analysis of preference change. J Math Psychol 46(3):300–314MathSciNetCrossRefGoogle Scholar
  8. Brans JP, Vincke P (1985) A preference ranking organisation method: (The PROMETHEE method for multiple criteria decision-making). Manag Sci 31(6):647–656MathSciNetMATHCrossRefGoogle Scholar
  9. Brockman JB (1996) Evaluation of student design processes. The 26th annual frontiers in education conference, Salt Lake City, UTGoogle Scholar
  10. Busemeyer JR, Diederich A (2002) Survey of decision field theory. Math Soc Sci 43(3):345–370MathSciNetMATHCrossRefGoogle Scholar
  11. Cross N, Christiaans H, Dorst K (1996) Analysing design activity. Wiley, ChichesterGoogle Scholar
  12. Dempster A, Laird N, Rubin D (1977) Maximum likelihood from incomplete data via the EM algorithm. J R Stat Soc Ser B 39(1):1–38MathSciNetMATHGoogle Scholar
  13. Dong A (2005) The latent semantic approach to studying design team communication. Des Stud 26(5):445–461Google Scholar
  14. Dong A (2006a) Concept formation as knowledge accumulation: a computational linguistics study. Artif Intell Eng Des Anal Manuf 20(1):35–53Google Scholar
  15. Dong A (2006b) How am I doing? The language of appraisal in design. Design computing and cognition ‘06 (DCC06). J S Gero. Kluwer, Dordrecht, The Netherlands, pp 385–404Google Scholar
  16. Dym CL, Wood WH, Scott MJ (2002) Rank ordering engineering designs: pairwise comparison charts and Borda counts. Res Eng Des 13(4):236–242Google Scholar
  17. Fishburn PC (1978) Choice probabilities and choice functions. J Math Psychol 18:205–219MathSciNetMATHCrossRefGoogle Scholar
  18. Fisher RA (1922) On the mathematical foundations of theoretical statistics. Philos Trans R Soc 222:309–368CrossRefGoogle Scholar
  19. Geslin MM (2006) An argumentation-based approach to negotiation in collaborative engineering design. Department of Aerospace And Mechanical Engineering, University of Southern California, Los AngelesGoogle Scholar
  20. Giffin M, De Weck O, Bounova G, Keller R, Eckert C, Clarkson PJ (2009) Change propagation analysis in complex technical systems. J Mech Des 131(8):081001CrossRefGoogle Scholar
  21. Gigone D, Hastie R (1997) The impact of information on small group choice. J Pers Soc Psychol 72(1):132–140CrossRefGoogle Scholar
  22. Green PE, Srinivasan V (1990) Conjoint analysis in marketing: new developments with implications for research and practice. J Market 54(4):3–19CrossRefGoogle Scholar
  23. Grefenstette G (1993) Automatic thesaurus generation from raw text using knowledge-poor techniques. The 9th annual conference of the UW centre for the new OED and text research, Oxford, EnglandGoogle Scholar
  24. Hanley N, Mourato S, Wright RE (2001) Choice modelling approaches: A superior alternative for environmental valuation? J Econ Surv 15(3):435–462Google Scholar
  25. Hauser JR, Clausing D (1988) The house of quality. Harv Bus Rev 66(3):63–73Google Scholar
  26. Hazelrigg GA (1998) A framework for decision-based engineering design. J Mech Des 120(4):653–658CrossRefGoogle Scholar
  27. Hensher DA, Johnson LW (1981) Applied discrete choice modeling. Halsted Press, New YorkGoogle Scholar
  28. Hey JD (1998) Do rational people make mistakes? Game theory, experience, rationality. In: Leinfellner W, Kohler E (eds) Kluwer, The Netherlands, pp 55–66Google Scholar
  29. Honda T, Yang MC, Dong A, Ji H (2010) A comparison of formal methods for evaluating the language of preference in engineering design. ASME design engineering technical conferences. Montreal, CanadaGoogle Scholar
  30. Jabeur K, Martel J-M, Khelifa SB (2004) A distance-based collective preorder integrating the relative importance of the group’s members. Group Decis Negot 13(4):327–349CrossRefGoogle Scholar
  31. Jain VK, Sobek DK II (2006) Linking design process to customer satisfaction through virtual design of experiments. Res Eng Des 17(2):59–71CrossRefGoogle Scholar
  32. Jaynes ET (1957) Information theory and statistical mechanics. Phys Rev 106(4):620–630MathSciNetCrossRefGoogle Scholar
  33. Jaynes ET (1968) Prior probabilities. IEEE Trans Syst Sci Cybern 4(3):227–241MATHCrossRefGoogle Scholar
  34. Ji H, Yang MC, Honda T (2007) A probabilistic approach for extracting design preferences from design team discussion. In: Proceedings of ASME 2007 international design engineering technical conferences and computers and information in engineering conferenceGoogle Scholar
  35. Keeney RL, Raiffa H (1976) Decisions with multiple objectives: preferences and value tradeoffs. Wiley, New YorkGoogle Scholar
  36. Kelley CT (2003) Solving nonlinear equations with Newton’s method. Society for Industrial and Applied Mathematics, PhiladelphiaMATHCrossRefGoogle Scholar
  37. Kohrs A, Merialdo B (2000) Using category-based collaborative filtering in the ActiveWebMuseum. The 2000 IEEE international conference on multimedia and expo, vol 1, pp 351–354Google Scholar
  38. Krantz DH, Luce RD, Suppes P, Tversky A (1971) Foundations of measurement volume 1. Academic Press, New York, NYGoogle Scholar
  39. Kulok M, Lewis K (2005) Preference consistency in multiattribute decision making. ASME conference proceedings 2005 (4742Xa), pp 291–300Google Scholar
  40. Li W, Jin Y (2006) Fuzzy preference evaluation for hierarchical co-evolutionary design concept generation. ASME conference proceedings (4255X), pp 31–41Google Scholar
  41. Luce RD (1959) Individual choice behavior. Wiley, New YorkMATHGoogle Scholar
  42. Mabogunje A, Leifer LJ (1996) 210-NP: measuring the mechanical engineering design process. Frontiers in Education Conference. FIE ‘96. In: Proceedings of 26th annual conference, vol 3, pp 1322–1328Google Scholar
  43. Manski CF (1977) The structure of random utility models. Theory Decis 8:229–254MathSciNetMATHCrossRefGoogle Scholar
  44. Miller GA, Beckwith R, Fellbaum C, Gross D, Miller K (1990) WordNet: an on-line lexical database. Int J Lexicogr 3(4):235–244Google Scholar
  45. Otto KN, Antonsson EK (1991) Trade-off strategies in engineering design. Res Eng Des 3(2):87–104CrossRefGoogle Scholar
  46. Otto KN, Antonsson EK (1993) The method of imprecision compared to utility theory for design selection problems. ASME 1993 design theory and methodology conferenceGoogle Scholar
  47. Packard DJ (1979) Preference relations. J Math Psychol 19(3):295–306MathSciNetMATHCrossRefGoogle Scholar
  48. Press WH, Teukolsky SA, Vetterling WT, Flannery BP (2007) Numerical recipes: the art of scientific computing. Cambridge University Press, New YorkMATHGoogle Scholar
  49. Pugh S (1991) Total design: integrated methods for successful product engineering. Addison-Wesley, WokinghamGoogle Scholar
  50. Reich Y (2010) My method is better!. Res Eng Des 21(3):137–142CrossRefGoogle Scholar
  51. Ross SM (2006) Simulation. Academic Press, Burlington, MAMATHGoogle Scholar
  52. Saaty TL (2000) Fundamentals of decision making and priority theory with the analytic hierarchy process. RWS Publications, PittsburghGoogle Scholar
  53. Scott MJ, Antonsson EK (1998) Aggregation functions for engineering design trade-offs. Fuzzy Sets Syst 99(3):253–264CrossRefGoogle Scholar
  54. Scott MJ, Antonsson EK (1999) Arrow’s theorem and engineering design decision making. Res Eng Des 11(4):218–228CrossRefGoogle Scholar
  55. Scott MJ, Antonsson EK (2005) Compensation and weights for trade-offs in engineering design: beyond the weighted sum. J Mech Des 127(6):1045–1055CrossRefGoogle Scholar
  56. See T-K, Lewis K (2006) A formal approach to handling conflicts in multiattribute group decision making. J Mech Des 128(4):678–688CrossRefGoogle Scholar
  57. Shah JJ, Vargas-Hernandez N, Summers JD, Kulkarni S (2001) Collaborative sketching (C-Sketch)—an idea generation technique for engineering design. J Creat Behav 35(3):168–198CrossRefGoogle Scholar
  58. Shannon CE (1948) A mathematical theory of communication. Bell Syst Tech J 27(3):379–423, 623–656Google Scholar
  59. Song S, Dong A, Agogino AM (2003) Time variation of design “story telling” in engineering design teams. In: Proceedings of the 14th international conference on engineering design (ICED 03), Stockholm, SwedenGoogle Scholar
  60. Thompson LL (2003) Making the team: a guide for managers. Prentice Hall, Upper Saddle RiverGoogle Scholar
  61. Thurston D (1991) A formal method for subjective design evaluation with multiple attributes. Res Eng Des 3(2):105–122CrossRefGoogle Scholar
  62. Tribus M (1969) Rational descriptions, decisions, and designs. Pergamon Press, New YorkGoogle Scholar
  63. Ueda N, Nakano R (1998) Deterministic annealing EM algorithm. Neural Netw 11(2):271–282CrossRefGoogle Scholar
  64. Ueda N, Nakano R, Ghahramani Z, Hinton GE (2000) SMEM algorithm for mixture models. Neural Comput 12(9):2109–2128CrossRefGoogle Scholar
  65. von Neumann J, Morgenstern O (1947) Theory of games and economic behaviour. Princeton University Press, PrincetonGoogle Scholar
  66. Wang J (1997) A fuzzy outranking method for conceptual design evaluation. Int J Prod Res 35(4):995–1010MATHCrossRefGoogle Scholar
  67. Wassenaar HJ, Chen W (2003) An approach to decision based design with discrete choice analysis for demand modeling. J Mech Des 125(3):490–497CrossRefGoogle Scholar
  68. Wassenaar HJ, Chen W, Cheng J, Sudjianto A (2005) Enhancing discrete choice demand modeling for decision-based design. J Mech Des 127(4):514–523CrossRefGoogle Scholar
  69. Wood KL, Antonsson EK (1989) Computations with imprecise parameters in engineering design: background and theory. ASME J Mech Transm Autom Des 111(4):616–625CrossRefGoogle Scholar
  70. Yang MC (2003) Concept generation and sketching: correlations with design outcome. ASME Conf Proc 37017b:829–834Google Scholar
  71. Yang MC (2005) A study of prototypes, design activity, and design outcome. Des Stud 26(6):649–669CrossRefGoogle Scholar
  72. Yang MC, Ji H (2007) A text-based analysis approach to representing the design selection process. In: Proceedings of the 16th international conference on engineering design (ICED 07)Google Scholar
  73. Yang MC, Wood WH, Cutkosky MR (2005) Design information retrieval: a thesauri-based approach for reuse of informal design information. Eng Comput 21(2):177–192CrossRefGoogle Scholar

Copyright information

© Springer-Verlag London Limited 2011

Authors and Affiliations

  1. 1.Yahoo! Inc.SunnyvaleUSA
  2. 2.Department of Mechanical Engineering and Engineering Systems DivisionMassachusetts Institute of TechnologyCambridgeUSA
  3. 3.Department of Mechanical EngineeringMassachusetts Institute of TechnologyCambridgeUSA

Personalised recommendations