Journal of the Operational Research Society

, Volume 62, Issue 10, pp 1801–1812 | Cite as

Does AHP help us make a choice? An experimental evaluation

  • A IshizakaEmail author
  • D Balkenborg
  • T Kaplan
General Paper


In this paper, we use experimental economics methods to test how well Analytic Hierarchy Process (AHP) fares as a choice support system in a real decision problem. AHP provides a ranking that we statistically compare with three additional rankings given by the subjects in the experiment: one at the beginning, one after providing AHP with the necessary pair-wise comparisons and one after learning the ranking provided by AHP. While the rankings vary widely across subjects, we observe that for each individual all four rankings are similar. Hence, subjects are consistent and AHP is, for the most part, able to replicate their rankings. Furthermore, while the rankings are similar, we do find that the AHP ranking helps the decision makers reformulate their choices by taking into account suggestions made by AHP.


decision analysis multiple criteria decision aid analytic hierarchy process (AHP) validation experimental economics 



We gratefully acknowledge the financial support of the Freiwillige Akademische Gesellschaft Basel and the University of Exeter. We wish to thank Tim Miller for valuable research assistance and comments. We also thank also the two anonymous reviewers for the valuable feedback and constructive criticism.


  1. Ahn B and Choi S (2007). ERP system selection using a simulation-based AHP approach: A case of Korean homeshopping company. J Opl Res Soc 59: 322–330.CrossRefGoogle Scholar
  2. Akarte M, Surendra N, Ravi B and Rangaraj N (2001). Web based casting supplier evaluation using analytical hierarchy process. J Opl Res Soc 52: 511–522.CrossRefGoogle Scholar
  3. Al-Shemmeri T, Al-Kloub B and Pearman A (1997). Model choice in multicriteria decision aid. Eur J Opl Res 97: 550–560.CrossRefGoogle Scholar
  4. Bana e Costa C and Vansnick J (2008). A critical analysis of the eigenvalue method used to derive priorities in AHP. Eur J Opl Res 187: 1422–1428.CrossRefGoogle Scholar
  5. Banuelas R and Antony J (2006). Application of stochastic analytic hierarchy process within a domestic appliance manufacturer. J Opl Res Soc 58: 29–38.CrossRefGoogle Scholar
  6. Barzilai J (2001). Notes on the analytic hierarchy process. Proceedings of the NSF Design and Manufacturing Research Conference,
  7. Becker GM, De Groot MH and Marschak J (1964). Measuring utility by a single-response sequential method. Behav Sci 9: 226–232.CrossRefGoogle Scholar
  8. Beil R (1996). Laboratory experimentation in economic research: An introduction to psychologists and marketers. Psychol & Mark 13: 331–340.CrossRefGoogle Scholar
  9. Belton V and Gear A (1983). On a shortcoming of Saaty’s method of analytical hierarchies. Omega 11: 228–230.CrossRefGoogle Scholar
  10. Brugha C (2000). Relative measurement and the power function. Eur J Opl Res 121: 627–640.CrossRefGoogle Scholar
  11. Brugha C (2004). Phased multicriteria preference finding. Eur J Opl Res 158: 308–316.CrossRefGoogle Scholar
  12. Callaghan C, Gabriel A and Sainty B (2006). Review and classification of experimental economics research in accounting. J Acc Lit 25: 59–126.Google Scholar
  13. Croson R and Donohue K (2002). Experimental economics and supply chain management. Interfaces 32 (5): 74–82.CrossRefGoogle Scholar
  14. Dodd F and Donegan H (1995). Comparison of priotization techniques using interhierarchy mappings. J Opl Res Soc 46: 492–498.CrossRefGoogle Scholar
  15. Donegan H, Dodd F and McMaster T (1992). A new approach to AHP decision-making. Statistician 41: 295–302.CrossRefGoogle Scholar
  16. Dyer J (1990). Remarks on the analytic hierarchy process. Mngt Sci 36: 249–258.CrossRefGoogle Scholar
  17. Forman E and Gass S (2001). The analytic hierarchy process—An exposition. Opns Res 49: 469–486.CrossRefGoogle Scholar
  18. Fukuyama H and Weber W (2002). Evaluating public school district performance via DEA gain functions. J Opl Res Soc 53: 992–1003.CrossRefGoogle Scholar
  19. Golden B, Wasil E and Harker P (1989). The Analytic Hierarchy Process: Applications and Studies. Springer-Verlag: Heidelberg.CrossRefGoogle Scholar
  20. Guitouni A and Martel J-M (1998). Tentative guidelines to help choosing an appropriate MCDA method. Eur J Opl Res 109: 501–521.CrossRefGoogle Scholar
  21. Guitouni A, Bélanger M and Martel J-M (2007). Une typologie des méthodes multicritères: Proposition d'un cadre méthodologique. INFOR 45: 153–174.Google Scholar
  22. Ho W (2008). Integrated analytic hierarchy process and its applications—A literature review. Eur J Opl Res 186: 211–228.CrossRefGoogle Scholar
  23. Hobbs B and Meier P (1994). Multicriteria methods for resource planning: An experimental comparison. IEEE T Power Syst 9: 1811–1817.CrossRefGoogle Scholar
  24. Holder R (1991). Response to holder’s comments on the analytic hierarchy process: Response to the response. J Opl Res Soc 42: 914–918.CrossRefGoogle Scholar
  25. Huizingh E and Vrolijk H (1997). Extending the applicability of the analytic hierarchy process. Socio Econ Plan Sci 31 (1): 29–39.CrossRefGoogle Scholar
  26. Ishizaka A and Labib A (2009). Analytic hierarchy process and expert choice: Benefits and limitations. OR Insight 22: 201–220.CrossRefGoogle Scholar
  27. Johnson C, Beine W and Wang T (1979). Right-left asymmetry in an eigenvector ranking procedure. J Math Psychol 19: 61–64.CrossRefGoogle Scholar
  28. Karlan D (2005). Using experimental economics to measure social capital and predict financial decisions. Am Econ Rev 95: 1688–1699.CrossRefGoogle Scholar
  29. Keeney R, Von Winterfeldt D and Eppel T (1990). Eliciting public values for complex policy decisions. Mngt Sci 36: 1011–1030.CrossRefGoogle Scholar
  30. Korhonen P and Topdagi H (2003). Performance of the AHP in comparison of gains and losses. Math Comput Model 37: 757–766.CrossRefGoogle Scholar
  31. Kornyshova E and Salinesi C (2007). Selecting MCDM techniques: State of the art. In: Bonissone P (ed). Proceedings of the International IEEE Symposium on Computational Intelligence in Multicriteria Decision Making. IEEE digital library, pp 22–29.Google Scholar
  32. Kumar S and Vaidya O (2006). Analytic hierarchy process: An overview of applications. Eur J Opl Res 169: 1–29.CrossRefGoogle Scholar
  33. Lee C and Kwak N (1999). Information resource planning for a health-care system using an AHP-based goal programming method. J Opl Res Soc 50: 1191–1198.CrossRefGoogle Scholar
  34. Leung L, Lam K and Cao D (2005). Implementing the balanced scorecard using the analytic hierarchy process & the analytic network process. J Opl Res Soc 57: 682–691.CrossRefGoogle Scholar
  35. Li X, Beullens P, Jones D and Tamiz M (2008). An integrated queuing and multi-objective bed allocation model with application to a hospital in China. J Opl Res Soc 60: 330–338.CrossRefGoogle Scholar
  36. Liberatore M and Nydick R (2008). The analytic hierarchy process in medical and health care decision making: A literature review. Eur J Opl Res 189: 194–207.CrossRefGoogle Scholar
  37. Linares P (2009). Are inconsistent decisions better? An experiment with pairwise comparisons. Eur J Opl Res 193: 492–498.CrossRefGoogle Scholar
  38. Millet I (1997). The effectiveness of alternative preference elicitation methods in the analytic hierarchy process. J Multi-Criteria Decis Anal 6: 41–51.CrossRefGoogle Scholar
  39. Mingers J, Liu W and Meng W (2007). Using SSM to structure the identification of inputs and outputs in DEA. J Opl Res Soc 60: 168–179.CrossRefGoogle Scholar
  40. Omkarprasad V and Sushil K (2006). Analytic hierarchy process: An overview of applications. Eur J Opl Res 169: 1–29.CrossRefGoogle Scholar
  41. Pöyhönen M, Hämäläinen R and Salo A (1997). An experiment on the numerical modelling of verbal ratio statements. J Multi-Criteria Decis Anal 6: 1–10.CrossRefGoogle Scholar
  42. Saaty T (1977). A scaling method for priorities in hierarchical structures. J Math Psychol 15: 234–281.CrossRefGoogle Scholar
  43. Saaty T (1980). The Analytic Hierarchy Process. McGraw-Hill: New York.Google Scholar
  44. Saaty T (2005). Making and validating complex decisions with the AHP/ANP. J Syst Sci Syst Eng 14: 1–36.CrossRefGoogle Scholar
  45. Saaty T (2006a). Rank from comparisons and from ratings in the analytic hierarchy/network processes. Eur J Opl Res 168: 557–570.CrossRefGoogle Scholar
  46. Saaty T (2006b). There is no mathematical validity for using fuzzy number crunching in the analytic hierarchy process. J Syst Sci Syst Eng 15: 457–464.CrossRefGoogle Scholar
  47. Saaty T and Forman E (1992). The Hierarchon: A Dictionary of Hierarchies. RWS Publications: Pittsburgh.Google Scholar
  48. Salo A and Hämäläinen R (1997). On the measurement of preference in the analytic hierarchy process. J Multi-Criteria Decis Anal 6: 309–319.CrossRefGoogle Scholar
  49. Sha D and Che Z (2005). Supply chain network design: Partner selection and production/distribution planning using a systematic model. J Opl Res Soc 57: 52–62.CrossRefGoogle Scholar
  50. Shim J (1989). Bibliography research on the analytic hierarchy process (AHP). Socio Econ Plan Sci 23 (3): 161–167.CrossRefGoogle Scholar
  51. Simpson L (1996). Do decision makers know what they prefer?: MAVT and ELECTRE II. J Opl Res Soc 47: 919–929.CrossRefGoogle Scholar
  52. Sturm B and Weimann J (2006). Experiments in environmental economics and some close relatives. J Econ Surv 20: 419–457.CrossRefGoogle Scholar
  53. Tavana M (2005). A priority assessment multi-criteria decision model for human spaceflight mission planning at NASA. J Opl Res Soc 57: 1197–1215.CrossRefGoogle Scholar
  54. Vargas L (1990). An overview of the analytic hierarchy process and its applications. Eur J Opl Res 48: 2–8.CrossRefGoogle Scholar
  55. Webber S, Apostolou B and Hassell J (1996). The sensitivity of the analytic hierarchy process to alternative scale and cue presentations. Eur J Opl Res 96: 351–362.CrossRefGoogle Scholar
  56. Wheeler S (2005). An analysis of combined arms teaming for the Australian defence force. J Opl Res Soc 57: 1279–1288.CrossRefGoogle Scholar
  57. Whitaker R (2007). Validation examples of the analytic hierarchy process and analytic network process. Math Comput Model 46: 840–859.CrossRefGoogle Scholar
  58. Yeo G, Song D, Dinwoodie J and Roe M (2010). Weighting the competitiveness factors for container ports under conflicting interests. J Opl Res Soc 61: 1249–1257.CrossRefGoogle Scholar
  59. Yüksel I and Dagdeviren M (2007). Using the analytic network process (ANP) in a SWOT analysis—A case study for a textile firm. Inform Sci 177: 3364–3382.CrossRefGoogle Scholar
  60. Zahedi F (1986). The analytic hierarchy process: A survey of the method and its applications. Interface 16 (4): 96–108.CrossRefGoogle Scholar

Copyright information

© Operational Research Society 2010

Authors and Affiliations

  1. 1.University of PortsmouthPortsmouthUnited Kingdom
  2. 2.University of ExeterExeterUnited Kingdom
  3. 3.University of HaifaHaifaIsrael

Personalised recommendations