Annals of Operations Research

, Volume 197, Issue 1, pp 5–23 | Cite as

Questionnaire design improvement and missing item scores estimation for rapid and efficient decision making

  • Daji Ergu
  • Gang KouEmail author


In unconventional emergency decision making process using the analytic hierarchy process (AHP), it is important to quickly collect and process experts’ opinions to make a rapid decision. Questionnaire survey is a commonly used way to collect opinions and views in the AHP. However, many factors such as tedious design format, redundant content, and long length, may lead to inconsistent comparison matrix for the decision problem. Invalid or bad results of a questionnaire survey may cause the decision makers to make wrong decision. Furthermore, in the AHP, the score items for a comparison matrix in a questionnaire increase drastically if there are more comparisons, which result in longer survey. In this paper, a scale format is used to design the score items for a comparison matrix in questionnaire survey. Besides, an induced bias matrix model (IBMM) is proposed to estimate the missing item scores of the reciprocal pairwise comparison matrix. The survey questionnaire can be improved according to the importance of score items and emergency degree of the surveyed questions. A numerical example is used to illustrate the proposed method in unconventional emergency decision making. In addition, three cases of this example are analyzed and compared to address the effectiveness and feasibility of the proposed estimation model in the survey questionnaire design.


Questionnaire design AHP Missing item scores estimation Emergency decision making Induced bias matrix 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. Altay, N., & Green, W. G. III (2006). OR/MS research in disaster operations management. European Journal of Operational Research, 175(1), 475–493. CrossRefGoogle Scholar
  2. Barzilai, J. (1997). Deriving weights from pairwise comparison matrices. Journal of the Operational Research Society, 48, 1226–1232. Google Scholar
  3. Bryson, N. (1995). A goal programming method for generating priority vectors. Journal of the Operational Research Society, 46, 641–648. Google Scholar
  4. Chiclana, F., Herrera-Viedma, E., & Alonso, S. (2009). A note on two methods for estimating missing pairwise preference values. IEEE Transactions on Systems, Man, and Cybernetics,Part B: Cybernetics, 39, 1628–1633. CrossRefGoogle Scholar
  5. Chu, A., Kalaba, R., & Springam, K. (1979). A comparison of two methods for determining the weights of belonging to fuzzy sets. Journal of Optimization Theory and Applications, 27, 531–541. CrossRefGoogle Scholar
  6. Crawford, G., & Williams, C. (1985). A note on the analysis of subjective judgment matrices. Journal of Mathematical Psychology, 29, 387–405. CrossRefGoogle Scholar
  7. Ergu, D., Kou, G., Peng, Y., & Shi, Y. (2011a). A simple method to improve the consistency ratio of the pair-wise comparison matrix in ANP. European Journal of Operational Research, 213(1), 246–259. doi: 10.1016/j.ejor.2011.03.014. CrossRefGoogle Scholar
  8. Ergu, D., Kou, G., Shi, Y., & Shi, Y. (2011b). Analytic network process in risk assessment and decision analysis. Computers & Operations Research. doi: 10.1016/j.cor.2011.03.005. Google Scholar
  9. Ergu, D., Kou, G., Peng, Y., Shi, Y., & Shi, Y. (2011c). The analytic hierarchy process: task scheduling and resource allocation in cloud computing environment. Journal of Supercomputing. doi: 10.1007/s11227-011-0625-1. Google Scholar
  10. Fedrizzi, M., & Giove, S. (2007). Incomplete pairwise comparison and consistency optimization. European Journal of Operational Research, 183(1), 303–313. CrossRefGoogle Scholar
  11. Hu, Z. H. (2010). Multi-signal cooperative decision for emergency management inspired by immune system. Journal of Computers, 5(9), 1410–1416. Google Scholar
  12. Kou, G., & Lou, C. (2010). Multiple factor hierarchical clustering algorithm for large scale Web page and search engine clickstream data. Annals of Operation Research. doi: 10.1007/s10479-010-0704-3. Google Scholar
  13. Kou, G., Liu, X., Peng, Y., Shi, Y., Wise, M., & Xu, W. (2003). Multiple criteria linear programming to data mining: models, algorithm designs and software developments. Optimization Methods & Software, 18(4), 453–473, Part 2. CrossRefGoogle Scholar
  14. Kou, G., Peng, Y., Chen, Z., & Shi, Y. (2009). Multiple criteria mathematical programming for multi-class classification and application in network intrusion detection. Information Sciences, 179(4), 371–381. CrossRefGoogle Scholar
  15. Li, L. F., & Tang, S. M. (2008). An artificial emergency-logistics-planning system for severe disasters. IEEE Intelligent Systems, 23(4), 86–88. CrossRefGoogle Scholar
  16. Lim, K. H., & Swenseth, S. R. (1993). An iterative procedure for reducing problem size in large scale AHP problems. European Journal of Operational Research, 67, 64–74. CrossRefGoogle Scholar
  17. Mendonça, D., Beroggi, G. E. G., & Wallace, W. A. (2001). Decision support for improvisation during emergency response operations. International Journal of Emergency Management, 1(1), 30–40. CrossRefGoogle Scholar
  18. Millet, I., & Harker, P. T. (1990). Globally effective questioning in the analytic hierarchy process. European Journal of Operational Research, 48, 88–97. CrossRefGoogle Scholar
  19. Özdamar, L., Ekinci, E., & Küçükyazici, B. (2004). Emergency logistics planning in natural disasters. Annals of Operation Research, 129, 217–245. CrossRefGoogle Scholar
  20. Pelaez, J. I., & Lamata, M. T. (2003). A new measure of consistency for positive reciprocal matrices. Computers & Mathematics With Applications, 46(12), 1839–1845. CrossRefGoogle Scholar
  21. Peng, Y., Kou, G., Shi, Y., & Chen, Z. (2008). A descriptive framework for the field of data mining and knowledge discovery. International Journal of Information Technology & Decision Making, 7(4), 639–682. CrossRefGoogle Scholar
  22. Peng, Y., Wang, G., & Wang, H. (2010a). User preferences based software defect detection algorithms selection using MCDM. Information Sciences. doi: 10.1016/j.ins.2010.04.019. Google Scholar
  23. Peng, Y., Zhang, Y., Tang, Y., & Li, S. (2010b). An incident information management framework based on data integration, data mining, and multi-criteria decision making. Decision Support Systems. doi: 10.1016/j.dss.2010.11.025. Google Scholar
  24. Peng, Y., Kou, G., Wang, G., Wu, W., & Shi, Y. (2011a). Ensemble of software defect predictors: an AHP-based evaluation method. International Journal of Information Technology & Decision Making, 10(1), 187–206. CrossRefGoogle Scholar
  25. Peng, Y., Wang, G., Kou, G., & Shi, Y. (2011b). An empirical performance metric for classification algorithm selection in financial risk management. Applied Soft Computing, 11(2), 2906–2915. CrossRefGoogle Scholar
  26. Peng, Y., Kou, G., Wang, G., & Shi, Y. (2011c). FAMCDM: a fusion approach of MCDM methods to rank multiclass classification algorithms. OMEGA. doi: 10.1016/ Google Scholar
  27. Saaty, T. L. (1977). A scaling method for priorities in hierarchical structures. Journal of Mathematical Psychology, 15(3), 234–281. CrossRefGoogle Scholar
  28. Saaty, T. L. (1980). The analytical hierarchy process. New York: McGraw-Hill. Google Scholar
  29. Saaty, T. L. (1994). How to make a decision: the analytic hierarchy process. Interfaces, 24, 19–43. CrossRefGoogle Scholar
  30. Saaty, T. L. (2001). Deriving the AHP 1–9 scale from first principles. In ISAHP 2001 proceedings, Bern, Switzerland. Google Scholar
  31. Saaty, T. L. (2003). Decision-making with the AHP: why is the principal eigenvector necessary. European Journal of Operational Research, 145(1), 85–89. CrossRefGoogle Scholar
  32. Saaty, T. L. (2008). Relative measurement and its generalization in decision making why pairwise comparisons are central in mathematics for the measurement of intangible factors the analytic hierarchy/network process. Revista de la Real Academia de Ciencias Exactas, Fisicas y Naturales. Serie A. Matematicas, 102(2), 251–318. doi: 10.1007/BF03191825. CrossRefGoogle Scholar
  33. Tufekci, S., & Wallace, W. A. (1998). The emerging area of emergency management and engineering. IEEE Transactions on Engineering Management, 45(2), 103–105. CrossRefGoogle Scholar
  34. Yan, S. Y., & Shih, Y. L. (2009). Optimal scheduling of emergency roadway repair and subsequent relief distribution. Computers & Operations Research, 36(9), 2049–2065. CrossRefGoogle Scholar
  35. Yi, W., & Ozdamar, L. (2007). A dynamic logistics coordination model for evacuation and support in disaster response activities. European Journal of Operational Research, 179(3), 1177–1193. CrossRefGoogle Scholar

Copyright information

© Springer Science+Business Media, LLC 2011

Authors and Affiliations

  1. 1.School of Management and EconomicsUniversity of Electronic Science and Technology of ChinaChengduChina
  2. 2.Southwest University for NationalitiesChengduChina

Personalised recommendations