A Multicriteria Model for the Evaluation of Intelligent Decision-making Support Systems (i-DMSS)

  • Gloria Phillips-Wren
  • Manuel Mora
  • Guisseppi A. Forgionne
  • Leonardo Garrido
  • Jatinder N. D. Gupta
Part of the Decision Engineering book series (DECENGIN)


Although traditional decision-making support systems (DMSS) have been researched extensively, few, if any, studies have addressed a unifying architecture for the evaluation of intelligent DMSS (i-DMSS). Traditional systems have often been evaluated in the literature on the basis of single-outcome measures, such as decreased cost, increased profit, or improved forecasting, compared to decision making without a DMSS. In cases in which other metrics are used for evaluation, process measures are most often cited, such as increased efficiency, organizational learning, and increased speed. Previous research by the authors has shown that a multicriteria evaluation for DMSS can be provided, combining both outcome and process measures into a single metric using the analytic hierarchy process (AHP). However, the specific categories that should be utilized as evaluation measures have not been defined, and no studies have focused exclusively on categories for the evaluation of i-DMSS. This chapter explores the concept of intelligence in general, and artificial intelligence in particular, as it relates to aiding decision making. It then proposes an architecture for the evaluation of i-DMSS and applies the model to empirical systems. The results are: (1) recognition of the contribution of AI to i-DMSS; (2) identification of the criterion (or criteria) used to evaluate i-DMSS; (3) categorization of the evaluation measures; (4) an architecture for evaluation for i-DMSS; and (5) recommendation of a multicriteria model to assess i-DMSS.


Analytic Hierarchy Process Decision Support System Operational Research Society Intelligent Behavior Analytic Hierarchy Process Model 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. Adelman L (1992) Evaluating Decision Support and Expert Systems, John Wiley & Sons, Inc., New York, NY.Google Scholar
  2. Albus J, Meystel A (2001) Engineering of Mind, Wiley Series on Intelligent Systems, John Wiley and Sons, Inc., New York, NY.Google Scholar
  3. Boering E (1933) The Physical Dimensions of Consciousness. New York: Century.Google Scholar
  4. Borenstein D (1998) IDSSFLEX: an intelligent DSS for the design and evaluation of flexible manufacturing systems. Journal of the Operational Research Society 49(7): 734–744.zbMATHCrossRefGoogle Scholar
  5. Brooks F (1996). The computer scientist as toolsmith II. Communications of the ACM, 19(3): 61–68.CrossRefGoogle Scholar
  6. Brown C, O’Leary D (1995) Introduction to Artificial Intelligence and expert systems. Accessed from, on September 15, 2004.Google Scholar
  7. Cassaigne N, Singh MG (2001) Intelligent decision support for the pricing of products and services in competitive consumer markets. IEEE Transactions on Systems, Man and Cybernetics, Part C (Applications and Reviews) 31(1): 96–106.CrossRefGoogle Scholar
  8. Chan FTS, Jiang B, Tang NKH (2000) The development of intelligent decision support tools to aid the design of flexible manufacturing systems. International Journal of Production Economics 65(1): 73–84.CrossRefGoogle Scholar
  9. Chan W, Naghd F (1997) An intelligent decision support system for body fluid balancing. In: IEEE International Conference on Intelligent Processing Systems, New York, NY, pp 1537–1541.Google Scholar
  10. Dhar V, Stein R (1998) Intelligent Decision Support Methods, Prentice-Hall, Upper Saddle River, pp 7–14.Google Scholar
  11. Elam J, Konsynski B (1987) Using artificial intelligence techniques to enhance the capabilities of model management systems. Decision Sciences, 18: 487–501.Google Scholar
  12. Eom S, Lee S, Kim E, Somarajan C (1998) A survey of decision support systems applications (1998–1994). Journal of Operational Research Society, 49: 109–120.zbMATHCrossRefGoogle Scholar
  13. Eom S (1998) An overview of the contributions to the decision support systems area from artificial intelligence, Proceedings of the ICIS 1998 Conference, 149–151.Google Scholar
  14. Expert Choice (2004) Accessed from on November 1.Google Scholar
  15. Faye RM, Mora-Camino F, Sawadogo S, Niang (1998) An intelligent decision support system for irrigation system management. In: IEEE International Conference on Systems, Man, and Cybernetics; New York, NY, pp 3908–3913.Google Scholar
  16. Fazlollahi B, Parikh MA (1997) Adaptive decision support systems. Decision Support Systems 20(4): 297–315.CrossRefGoogle Scholar
  17. Forgionne G (1999) An AHP model of DSS effectiveness. European Journal of Information Systems 8: 95–106.CrossRefGoogle Scholar
  18. Forgionne G, Kohli R (2001) A multiple criteria assessment of decision technology system journal qualities. Information Management 38: 421–435.CrossRefGoogle Scholar
  19. Forgionne G, Mora M, Cervantes F, Gelman O (2002) I-DMSS: A conceptual architecture for the next generation of decision-making support systems in the internet age. In: Adam F, Brézillon P, Humphreys P, Pomerol J (eds), Decision-making and Decision Support in the Internet Age, Proceedings of the DSIage2002, IFIP WG 8.3, Cork, Ireland, July 4–7, pp 154–165.Google Scholar
  20. Goul M, Henderson J, Tonge F (1992) The emergence of artificial intelligence as a reference discipline for decision support systems research, Decision Sciences, 23, pp. 1263–1276.Google Scholar
  21. Gottinger HW, Weimann P (1992) Intelligent decision support systems. Decision Support Systems, 8(4): 317–332.CrossRefGoogle Scholar
  22. Grabowski M, Sanborn S (2001) Evaluation of embedded intelligent real-time systems. Decision Sciences (Winter) 32(1): 95–123.Google Scholar
  23. Gray P, Watson H (1996) The new DSS: data warehouses, OLAP, MDD and KDD. In: Proceedings of the AMCIS Conference 1996, Phoenix, AZ, USA.Google Scholar
  24. Guerlain S, Brown DE, Mastrangelo C (2000) Intelligent decision support systems. In: IEEE International Conference on Systems, Man and Cybernetics. ‘Cybernetics Evolving to Systems, Humans, Organizations, and their Complex Interactions’, Piscataway, NJ, pp 1934–1938.Google Scholar
  25. Harker P (1988) The Art and Science of Decision-making: The Analytic Hierarchy Process. Working Paper 88-06-03, Decision Science Department, The Wharton School, University of Pennsylvania, Philadelphia, PA.Google Scholar
  26. Holsapple CW, Whinston AB (1996) Decision Support Systems, West Publishing Company, St. Paul, MN.Google Scholar
  27. Ifeachor EC, Garibaldi, JM, Skinner J (1998) Intelligent decision support tools for the management of labour. In: IEE Colloquium on Intelligent Decision Support in Clinical Practice, pp 1–7.Google Scholar
  28. Jensen A (1998) Does IQ matter ? Comentary, pp. 20–21, November. Comments to F. Chabris, “IQ since the Bell Curve”, Commentary, August.Google Scholar
  29. Jensen A (1999) The G Factor: the Science of Mental Ability. Psycoloquy, 10, #23.Google Scholar
  30. Jensen A (2000) Artificial Intelligence and G Theory concern different Phenomena, Psycoloquy, 11, #86.Google Scholar
  31. Kobbacy KAH, Proudlove NC, Harper MA (1995) Towards an intelligent maintenance optimization system. Journal of the Operational Research Society 46(7): 831–853.zbMATHCrossRefGoogle Scholar
  32. Kwok LF, Lau WT, Kong SC (1996) An intelligent decision support system for teaching duty assignments. In: Narasimhan VL, Jain LC (eds), Proceedings of the Australian New Zealand Conference on Intelligent Information Systems, pp 97–100.Google Scholar
  33. Lin R, Wang Q, Hu J (1996) An intelligent decision support system applied to the investment of real estate. In: Proceedings of the IEEE International Conference on Industrial Technology, New York, NY, 801–805.Google Scholar
  34. Macintosh A (2004) The optimization of What?. Proceedings of the DSS2004 Conference, Prato, Italy. Keynote Address.Google Scholar
  35. Maynard S, Burstein F, Arnott, D (2001) A multi-faceted decision support system evaluation approach. Journal of Decision Systems, special issue “DSS in the New Millennium”, 10(3–4): 395–428.Google Scholar
  36. McCarthy J (2003) What is artificial intelligence?. Technical Report. Computer Science Department, Stanford University. Accessed from on September 9.Google Scholar
  37. Mora M, Forgionne G, Gupta J, Cervantes F, Gelman O (2003) A framework to assess intelligent decision-making support systems. In: Proceedings of the 7th International Conference KES 2003, Oxford, UK, September (Lecture Notes on Artificial Intelligence 2774, Springer-Verlag), 59–65.Google Scholar
  38. Nemati HR, Iyer LS (1999) An intelligent decision support system prototype for asset allocation. In: Haseman WD, Nazareth DL (eds), Proceedings of the Fifth Americas Conference on Information Systems; Atlanta, GA, 70–72.Google Scholar
  39. Newell A, Simon H (1976) Computer science as empirical inquiry: Symbols and Search (1975 ACM Turing Award Lecture), Communications of the ACM, 19(3): 113–126CrossRefMathSciNetGoogle Scholar
  40. Palaniappan C, Srinivasan R, Halim I (2002) A material-centric methodology for developing inherently safer environmentally benign processes. Computers & Chemical Engineering 26(4–5): 757–774.CrossRefGoogle Scholar
  41. Parnas D (1985) Software aspects of strategic defense systems, ACM SIGSOFT Software Engineering Notes, 10(5): 15–23.CrossRefGoogle Scholar
  42. Pflughoeft KA, Hutchinson GK, Nazareth DL (1996) Intelligent decision support for flexible manufacturing: Design and implementation of a knowledge-based simulator. Omega 24(3): 347–360.CrossRefGoogle Scholar
  43. Phillips-Wren G, Forgionne G (2002) Evaluating web-based and real-time decision support systems. In: Adam F, Brézillon P, Humphreys P, Pomerol J (eds), Decision-making and Decision Support in the Internet Age, Proceedings of the DSIage2002, IFIP WG 8.3, Cork, Ireland, July 4–7, 166–175.Google Scholar
  44. Phillips-Wren G, Hahn E, Forgionne G (2004) A multiple criteria framework for the evaluation of decision support systems. Omega 32(4): 323–332.CrossRefGoogle Scholar
  45. Pohl J (2005) Intelligent software systems in historical context. In: Phillips-Wren G, Jain LC (eds), Intelligent Decision Support Systems in Agent-Mediated Environments, IOS Press: The Netherlands, 3–34.Google Scholar
  46. Potter W, Ramyaa LJ, Ghent TT (2002) STP: an aerial spray treatment planning system. In: Proceedings of the IEEE Southeast Conference, pp 300–305.Google Scholar
  47. Proudlove NC, Vedera S, Kobbacy KAH (1998), Intelligent management systems in operations: a review. Journal of the Operations Research Society, 49: 682–699.zbMATHGoogle Scholar
  48. Renton MW, Wallace AR (1996) Expert system scheduling of cascade hydro-electric plants. Proceedings of the International Conference on Opportunities and Advances in International Power Generation, IEE, London, UK, pp. 69–72.CrossRefGoogle Scholar
  49. Saaty TL (1977) A scaling method for priorities in hierarchical structures. Journal of Mathematical Psychology 15: 234–281.zbMATHCrossRefMathSciNetGoogle Scholar
  50. Silverman BG (1992) Evaluating and refining expert critiquing systems: A methodology. Decision Sciences 23(1): 86–110.Google Scholar
  51. Simon H (1996) Models of My Life. MIT Press.Google Scholar
  52. Simon H (1997) Administrative Behavior, Fourth edition (Original publication date 1945), The Free Press, New York, NY.Google Scholar
  53. Singh R, Reif HL (1999) Intelligent decision aids for electronic commerce. In: Haseman WD, Nazareth DL (eds), Proceedings of the Fifth Americas Conference on Information Systems (AMCIS), Atlanta, GA, 85–87.Google Scholar
  54. Smith A, Nugent C, McClean S (2001) Intelligent decision support systems for medicine: inherent performance evaluation. In: Conference Proceedings of the 23rd Annual International Conference of the IEEE Engineering in Medicine and Biology Society, Piscataway, NJ, vol. 4, 3746–3749.Google Scholar
  55. Sprague RH (1980) A framework for the development of decision support systems. MIS Quarterly, 4(4): 1–26.CrossRefGoogle Scholar
  56. Strachan SM, West GM, McDonald JR (2001) Knowledge management and intelligent decision support for protection scheme design and application in electrical power systems. In: Seventh International Conference on Developments in Power System Protection, London, UK, 559–562.Google Scholar
  57. Tsumoto S (2003) Web based medical decision support system: application of Internet to telemedicine. In: Proceedings of the Symposium on Applications and the Internet Workshops, Los Alamitos, CA, 288–93.Google Scholar
  58. Turban E, Aronson J (1998) Decision Support Systems and Intelligent Systems. A. Simon and Schuster Company, Upper Saddle River, NJGoogle Scholar
  59. Turing A (1950) Computing machinery and intelligence. Mind 59, 433–460.CrossRefMathSciNetGoogle Scholar
  60. Vraneš S, Stanojević M, Stevanović VL (1996) INVEX: investment advisory expert system. Expert Systems 13(2): 105–120.Google Scholar
  61. Wong AKC, Yang W (2003) Pattern discovery: a data driven approach to decision support. IEEE Transactions on Systems, Man and Cybernetics, Part C 33(1): 114–124.MathSciNetGoogle Scholar
  62. Yang H, Huang Y (1996) Intelligent decision support for diagnosis of incipient transformer faults using self-organizing polynomial networks. In: Proceedings of the 20th International Conference on Power Industry Computer Applications, New York, NY, pp 60–66.Google Scholar
  63. Zeleznikow J, Nolan JR (2001) Using soft computing to build real world intelligent decision support systems in uncertain domains. Decision Support Systems 31(2): 263–285.CrossRefGoogle Scholar

Copyright information

© Springer-Verlag London Limited 2006

Authors and Affiliations

  • Gloria Phillips-Wren
    • 1
  • Manuel Mora
    • 2
  • Guisseppi A. Forgionne
    • 3
  • Leonardo Garrido
    • 4
  • Jatinder N. D. Gupta
    • 5
  1. 1.Department of Information SystemsLoyola College in MarylandBaltimoreUSA
  2. 2.Department of Information SystemsAutonomous University of AguascalientesAguascalientesMexico
  3. 3.Department of Information SystemsUniversity of Maryland Baltimore CountyCatonsvilleUSA
  4. 4.Center for Intelligent SystemsMonterrey TechMonterrey, N.L.Mexico
  5. 5.College of Administrative ScienceThe University of Alabama in HuntsvilleHuntsvilleUSA

Personalised recommendations