A comparison of preference elicitation methods for multi-criteria design decisions about resilient and sustainable buildings

Abstract

Designing sustainable and resilient buildings is a complex process involving multiple actors and numerous, often conflicting, criteria to describe the social, environmental, and economic consequences of design decisions. Multi-Criteria Decision Making (MCDM) is a well-established branch of systematic decision science that may be used to narrow the choice set and balance decision criteria and objectives. MCDM methods rely on eliciting decision makers’ (DM) criteria preferences through surveys—a critical, but often time-consuming and expensive process. This study compares two survey methods—Simple Multi-Attribute Rating Technique and Potentially All Pairwise RanKings of all possible Alternatives—used to capture DM’s preferences for subjective criteria relevant to the design of resilient and sustainable buildings. Significant differences were found between the results of the two methods, indicating that building design criteria are evaluated differently in different contexts. However, most DMs reported that both surveys accurately captured their preferences, complicating conclusions about DM’s actual preferences through single surveys. The design professionals surveyed also report lower cognitive load for the direct weighting method. Although this study does not report a preferred survey method, the findings suggest that great care must be taken when eliciting DM preferences, and when attempting to apply MCDM to building design decisions.

This is a preview of subscription content, log in to check access.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6

References

  1. Alexander D (2006) Globalization of disaster: trends, problems and dilemmas. J Int Aff 59(2):1–22

    Google Scholar 

  2. Alexander DE (2013) Resilience and disaster risk reduction: an etymological journey. Nat Hazard Earth Syst Sci 13(11):2707–2716

    Article  Google Scholar 

  3. Aloysius JA, Davis FD, Wilson DD, Taylor A, Ross, Kottemann JE (2006) User acceptance of multi-criteria decision support systems: the impact of preference elicitation techniques. Eur J Oper Res 169(1):273–285

    Article  Google Scholar 

  4. Antonenko PD, Niederhauser DS (2010) The influence of leads on cognitive load and learning in a hypertext environment. Comput Hum Behav 26(2):140–150

    Article  Google Scholar 

  5. Ayres P (2006) Using subjective measures to detect variations of intrinsic cognitive load within problems. Learn Instr 16(5):389–400

    Article  Google Scholar 

  6. Balcomb JD, Curtner A (2000) Multi-criteria decision-making process for buildings. In: Energy conversion engineering conference and exhibit, 2000. (IECEC) 35th Intersociety (Vol 1, pp 528–535). IEEE

  7. Brans J-P (1986) L’élaboration d’instruments d’aide à la décision. Nadeau, Raymond et Maurice Landry, 183–213

  8. Buchanan JT, Daellenbach HG (1987) A comparative evaluation of interactive solution methods for multiple objective decision models. Eur J Oper Res 29(3):353–359

    Article  Google Scholar 

  9. Burroughs S (2017) Development of a tool for assessing commercial building resilience. Proc Eng 180:1034–1043

    Article  Google Scholar 

  10. Carlsson C, Korhonen P (1986) A parametric approach to fuzzy linear programming. Fuzzy Set Syst 20(1):17–30

    Article  Google Scholar 

  11. Cegan JC, Filion AM, Keisler JM, Linkov I (2017) Trends and applications of multi-criteria decision analysis in environmental sciences: literature review. Environ Syst Decis 37(2):123–133

    Article  Google Scholar 

  12. Chang C-C, Yang F-Y (2010) Exploring the cognitive loads of high-school students as they learn concepts in web-based environments. Comput Educ 55(2):673–680

    Article  Google Scholar 

  13. Chen F, Zhou J, Wang Y, Yu K, Arshad SZ, Khawaji A, Conway D (2016) Robust multimodal cognitive load measurement. Berlin: Springer, Ahmad

    Google Scholar 

  14. De Jong J, Den Hartog D (2010) Measuring innovative work behaviour. Creat Innov Manag 19(1):23–36

    Article  Google Scholar 

  15. Deierlein G, Krawinkler H, Ma X, Eatherton M, Hajjar J, Takeuchi T et al (2011) Earthquake resilient steel braced frames with controlled rocking and energy dissipating fuses. Steel Constr 4(3):171–175

    Article  Google Scholar 

  16. Edwards W (1977) How to use multiattribute utility measurement for social decisionmaking. IEEE Trans Syst Man Cybern 7(5):326–340

    Article  Google Scholar 

  17. Edwards W, Barron FH (1994) SMARTS and SMARTER: improved simple methods for multiattribute utility measurement. Organ Behav Hum Decis Process 60(3):306–325

    Article  Google Scholar 

  18. Elkington J (2013) Enter the triple bottom line. In: The triple bottom line. Routledge, London, pp 23–38

    Google Scholar 

  19. Flager F, Welle B, Bansal P, Soremekun G, Haymaker J (2009) Multidisciplinary process integration and design optimization of a classroom building. J Inform Technol Constr 14(38):595–612

    Google Scholar 

  20. Formisano A, Mazzolani FM (2015) On the selection by MCDM methods of the optimal system for seismic retrofitting and vertical addition of existing buildings. Comput Struct 159:1–13

    Article  Google Scholar 

  21. Gandibleux X (ed) (2006) Multiple criteria optimization: state of the art annotated bibliographic surveys, vol 52. Springer, Berlin

    Google Scholar 

  22. Gimino AE (2002) Factors that influence students’ investment of mental effort in academic tasks: a validation and exploratory study

  23. Glavič P, Lukman R (2007) Review of sustainability terms and their definitions. J Clean Prod 15(18):1875–1885

    Article  Google Scholar 

  24. Gopher D, Braune R (1984) On the psychophysics of workload: why bother with subjective measures? Hum Factors 26(5):519–532

    Article  Google Scholar 

  25. Gunderson LH (2000) Ecological resilience: in theory and application. Annu Rev Ecol Syst 31(1):425–439

    Article  Google Scholar 

  26. Hansen P, Ombler F (2008) Decision support system and method: Google Patents

  27. Harkouss F, Biwole PH, Fardoun F (2018) Passive designs and renewable energy systems optimization of a net zero energy building in Embrun/France. Paper presented at the IOP conference series: earth and environmental science

  28. Holling CS (1996) Engineering resilience versus ecological resilience. Eng Ecol Constraints 31(1996):32

    Google Scholar 

  29. Hsieh T-Y, Lu S-T, Tzeng G-H (2004) Fuzzy MCDM approach for planning and design tenders selection in public office buildings. Int J Project Manag 22(7):573–584

    Article  Google Scholar 

  30. Invidiata A, Lavagna M, Ghisi E (2018) Selecting design strategies using multi-criteria decision making to improve the sustainability of buildings. Build Environ 139:58–68

    Article  Google Scholar 

  31. Jin Z, Eikevik TM, Nekså P, Hafner A (2016) Investigation on CO2 hybrid ground-coupled heat pumping system under warm climate. Int J Refrig 62:145–152

    CAS  Article  Google Scholar 

  32. Kalyuga S, Sweller J (2005) Rapid dynamic assessment of expertise to improve the efficiency of adaptive e-learning. Educ Technol Res Dev 53(3):83–93

    Article  Google Scholar 

  33. Kamal MM (2012) Shared services: lessons from private sector for public sector domain. J Enterp Inform Manag 25(5):431–440

    Article  Google Scholar 

  34. Kolokotsa D, Pouliezos A, Stavrakakis G, Lazos C (2009) Predictive control techniques for energy and indoor environmental quality management in buildings. Build Environ 44(9):1850–1863

    Article  Google Scholar 

  35. Laboy M, Fannon D (2016) Resilience theory and praxis: a critical framework for architecture. Enquiry 13(1):39–52

    Article  Google Scholar 

  36. Larichev OI, Nikiforov AD (1987) Analytical survey of procedures for solving multicriteria mathematical programming problems (MMPP) toward interactive and intelligent decision support systems. Springer, Berlin. pp 95–104

    Google Scholar 

  37. Lee CD, Goldman SR, Levine S, Magliano J (2016) Epistemic cognition in literary reasoning. Handbook of epistemic cognition, 165–183

  38. Lin FR, Yaffe K, Xia J, Xue QL, Harris TB, Purchase-Helzner E, Health ABC Study Group F et al (2013) Hearing loss and cognitive decline in older adults. JAMA Intern Med 173(4), 293–299

    Article  Google Scholar 

  39. Linkov I, Bridges T, Creutzig F, Decker J, Fox-Lent C, Kröger W, Nathwani J et al (2014) Changing the resilience paradigm. Nat Clim Chang 4(6):407

    Article  Google Scholar 

  40. Linstone HA, Turoff M (2011) Delphi: a brief look backward and forward. Technol Forecast Soc Chang 78(9):1712–1719

    Article  Google Scholar 

  41. Lützkendorf T, Lorenz DP (2006) Using an integrated performance approach in building assessment tools. Build Res Inform 34(4):334–356

    Article  Google Scholar 

  42. Magelssen M, Supphellen M, Nortvedt P, Materstvedt LJ (2016) Attitudes towards assisted dying are influenced by question wording and order: a survey experiment. BMC Med Ethics 17(1):24

    Article  Google Scholar 

  43. Maggino F, Ruviglioni E (2009) Obtaining weights: from objective to subjective approaches in view of more participative methods in the construction of composite indicators. In: Proceedings NTTS: new techniques and technologies for statistics, pp 37–46

  44. Majumder M (2015) Multi criteria decision making. In: Impact of urbanization on water shortage in face of climatic aberrations. Springer, Berlin, pp 35–47

    Google Scholar 

  45. Martino JP, Harold A, Turoff M (1976) The Delphi method: techniques and applications: Linstone, Harold A., and Murray Turoff. Advanced Book Program. Addison-Wesley, Advanced Book Program, Reading

    Google Scholar 

  46. Mirzaee S, Ruth M, Fannon D (2018) Reconciling diverse perspective of decision makers on resilience and sustainability. In: Advanced introduction to ecological economics. Edward Elgar, Cheltenham

    Google Scholar 

  47. Moser CA, Kalton G (2017) Survey methods in social investigation. Routledge, London

    Google Scholar 

  48. Mota P, Campos AR, Neves-Silva R (2013) First look at MCDM: choosing a decision method. Adv Smart Syst Res 3(2):25–30

    Google Scholar 

  49. Narasimhan R, Vickery SK (1988) An experimental evaluation of articulation of preferences in multiple criterion decision-making (MCDM) methods. Decis Sci 19(4):880–888

    Article  Google Scholar 

  50. Noseworthy T, De Coster C, Naden R (2009) Priority-setting tools for improving access to medical specialists. Poster presentation. Paper presented at the 6th health technology assessment international annual meeting, Singapore

  51. Önüt S, Kara SS, Işik E (2009) Long term supplier selection using a combined fuzzy MCDM approach: a case study for a telecommunication company. Expert Syst Appl 36(2):3887–3895

    Article  Google Scholar 

  52. Paas FGWC, Van Merriënboer JJG (1994) Variability of worked examples and transfer of geometrical problemsolving skills: a cognitive-load approach. J Educ Psychol 86(1):122

    Article  Google Scholar 

  53. Paas F, Renkl A, Sweller J (2003) Cognitive load theory and instructional design: recent developments. Educ Psychol 38(1):1–4

    Article  Google Scholar 

  54. Pater LR, Cristea SL (2016) Systemic definitions of sustainability, durability and longevity. Proc Soc Behav Sci 221:362–371

    Article  Google Scholar 

  55. Phillips R, Troup L, Fannon D, Eckelman MJ (2017) Do resilient and sustainable design strategies conflict in commercial buildings? A critical analysis of existing resilient building frameworks and their sustainability implications. Energy Build 146:295–311

    Article  Google Scholar 

  56. Pill J (1971) The Delphi method: substance, context, a critique and an annotated bibliography. Socioecon Plan Sci 5(1):57–71

    Article  Google Scholar 

  57. Polatidis H, Haralambopoulos D (2002) Normative aspects of renewable energy planning; a participatory multi-criteria approach. The International Society for Ecological Economics. Ecological economics working paper series

  58. Pomerol J-C, Barba-Romero S (2012) Multicriterion decision in management: principles and practice, vol 25. Springer, Berlin

    Google Scholar 

  59. Powell C (2003) The Delphi technique: myths and realities. J Adv Nurs 41(4):376–382

    Article  Google Scholar 

  60. Roy B, Bouyssou D (1986) Comparison of two decision-aid models applied to a nuclear power plant siting example. Eur J Oper Res 25(2):200–215

    Article  Google Scholar 

  61. Ruzgys A, Volvačiovas R, Ignatavičius Č, Turskis Z (2014) Integrated evaluation of external wall insulation in residential buildings using SWARA-TODIM MCDM method. J Civ Eng Manag 20(1):103–110

    Article  Google Scholar 

  62. Saaty TL (1980) The analytic hierarchy process: planning, priority setting, resources allocation. McGraw, New York. p 281

    Google Scholar 

  63. San Cristóbal JR (2011) Multi-criteria decision-making in the selection of a renewable energy project in spain: the Vikor method. Renew Energy 36(2):498–502

    Article  Google Scholar 

  64. Scharfenberg F-J, Bogner FX (2013) Teaching gene technology in an outreach lab: students’ assigned cognitive load clusters and the clusters’ relationships to learner characteristics, laboratory variables, and cognitive achievement. Res Sci Edu 43(1):141–161

    Article  Google Scholar 

  65. Scheffer M, Hosper SH, Meijer ML, Moss B, Jeppesen E (1993) Alternative equilibria in shallow lakes. Trends Ecol Evol 8(8):275–279

    CAS  Article  Google Scholar 

  66. Schön DA (1983) The reflective practicioner: how professionals think in action, vol 1. Basic books, New York

    Google Scholar 

  67. Triantaphyllou E (2000) Multi-criteria decision making methods. In: Multi-criteria decision making methods: a comparative study. Springer, Boston, pp 5–21

    Google Scholar 

  68. Tzeng GH, Chen TY, Wang JC (1998) A weight-assessing method with habitual domains. Eur J Oper Res 110(2):342–367

    Article  Google Scholar 

  69. Walgrave S, Wouters R, Ketelaars P (2016) Response problems in the protest survey design: evidence from fifty-one protest events in seven countries. Mobilization 21(1):83–104

    Article  Google Scholar 

  70. Wallenius J (1975) Comparative evaluation of some interactive approaches to multicriterion optimization. Manag Sci 21(12):1387–1396

    Article  Google Scholar 

  71. Wang T-C, Lee H-D (2009) Developing a fuzzy TOPSIS approach based on subjective weights and objective weights. Expert Syst Appl 36(5):8980–8985

    Article  Google Scholar 

  72. Woudenberg F (1991) An evaluation of Delphi. Technol Forecast Soc Chang 40(2):131–150

    Article  Google Scholar 

  73. Yang AL, Huang GH, Qin XS, Fan YR (2012) Evaluation of remedial options for a benzene-contaminated site through a simulation-based fuzzy-MCDA approach. J Hazard Mater 213:421–433

    Article  CAS  Google Scholar 

  74. Yoon K (1987) A reconciliation among discrete compromise solutions. J Oper Res Soc 38(3):277–286

    Article  Google Scholar 

  75. Zimmermann H-J (1990) Decision making in ill-structured environments and with multiple criteria. In: Readings in multiple criteria decision aid. Springer, Berlin, pp 119–151

    Google Scholar 

  76. Zuo J, Zhao Z-Y (2014) Green building research-current status and future agenda: a review. Renew Sustain Energy Rev 30:271–281

    Article  Google Scholar 

Download references

Acknowledgements

This research is based on work supported by the National Science Foundation grant number CMMI #1455450. The authors wish to thank Dr. Mehrdad Sasani, Dr. Matthew Eckelman, Matthew Joyner, Robert Phillips, and Luke Troup for their iterative and constructive feedback.

Author information

Affiliations

Authors

Corresponding author

Correspondence to David Fannon.

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Mirzaee, S., Fannon, D. & Ruth, M. A comparison of preference elicitation methods for multi-criteria design decisions about resilient and sustainable buildings. Environ Syst Decis 39, 439–453 (2019). https://doi.org/10.1007/s10669-019-09726-2

Download citation

Keywords

  • Multi-Criteria Decision Making (MCDM)
  • SMART
  • PAPRIKA
  • Decision-making cognitive load
  • Criteria preference techniques