Assessing Today: Determining the Decision Value of Decision Support Systems

  • Gloria Phillips-Wren
  • Manuel Mora
  • Guisseppi Forgionne
Part of the Annals of Information Systems book series (AOIS, volume 14)


Decision support systems (DSS) have been assessed on the basis of single criteria such as the improvement in decision outcome, and using multiple criteria such as the perspectives of different stakeholders. This paper uses a systems approach to extend previous studies by linking the type of support provided to the decision maker with the specific DSS design characteristics needed to deliver those services. The proposed framework is implemented with the Analytic Hierarchy Process to measure the overall decision value of a DSS and determine the precise contributions of individual characteristics to the value. An applied problem in intelligent DSS is shown to demonstrate the ability of the extended evaluation approach to provide guidance for further DSS design, development, and implementation.


Analytic Hierarchy Process Decision Support System Improve Decision Making Analytic Hierarchy Process Model Intelligent Decision Support System 



The authors would like to thank the anonymous reviewers whose comments significantly contributed to the quality of the paper. We particularly appreciate the suggested references and the careful reading of the paper.


  1. Adelman, L. (1992). Evaluating Decision Support and Expert Systems. New York, NY: Wiley.Google Scholar
  2. Ahituv, N., and Wand, Y. (1981). Information Systems in Management Science – Information Evaluation and Decision Makers’ Objectives. Interfaces, 11(3), 24–32.CrossRefGoogle Scholar
  3. Akoka, J. (1981). A Framework for Decision Support Systems Evaluation. Information & Management, 4, 133–141.CrossRefGoogle Scholar
  4. Alter, S. (2004). A Work System View of DSS in Its Fourth Decade. Decision Support Systems, 38(3), 319–327.CrossRefGoogle Scholar
  5. Brynjolfsson, E., and Hitt, L. (1998). Beyond the Productivity Paradox: Computers are the Catalyst for Bigger Changes. Communications of the ACM, 41(8), 49–55.CrossRefGoogle Scholar
  6. Chandler, J. (1982). A Multiple Criteria Approach for Evaluating Information Systems. Management Information Systems Quarterly, 6(1), 61–74.Google Scholar
  7. Checkland, P. (2000). Systems Thinking, Systems Practice. Chichester: Wiley.Google Scholar
  8. Clark, T., Jones, M., and Armstrong, C. (2007). The Dynamic Structure of Management Support Systems: Theory Development, Research Focus, and Direction. Management Information Systems Quarterly, 31(3), 579–615.Google Scholar
  9. Devaraj, S., and Kohli, R. (2002). The IT Payoff: Measuring the Business Value Information Technology Investments. Upper Saddle River, NJ: Financial Times Prentice Hall.Google Scholar
  10. Devaraj, S., and Kohli, R. (2003). Performance Impacts of Information Technology: Is Actual Usage the Missing Link? Management Science, 49(3), 273–289.CrossRefGoogle Scholar
  11. Eom, S., and Kim, E. (2006). A Survey of Decision Support System Applications (1995–2001). Journal of the Operational Research Society, 57, 1264–1278.CrossRefGoogle Scholar
  12. Forgionne, G. (1999). An AHP Model of DSS Effectiveness. European Journal of Information Systems, 8(2), 95–106.CrossRefGoogle Scholar
  13. Forgionne, G. (2000). Decision-Making Support Systems Effectiveness: The Process to Outcome Link. Information Knowledge-Systems Management, 2, 169–188.Google Scholar
  14. Forgionne, G., and Kohli, R. (2000). Management Support System Effectiveness: Further Empirical Evidence. Journal of the Association for Information Systems, 1(3), 1–37.Google Scholar
  15. Gorry, G. M., and Scott-Morton, M. S. (1971). A Framework for Management Information Systems. Sloan Management Review, 13(1), 55–70.Google Scholar
  16. Harker, P. (1988) The Art and Science of Decision Making: The Analytic Hierarchy Process. Working Paper 88-06-03, Decision Science Department, The Wharton School, University of Pennsylvania, Philadelphia, PA.Google Scholar
  17. Hitt, L., and Brynjolfsson, E. (1996). Productivity, Business Profitability, and Consumer Surplus: Three Different Measures of Information Technology Value. Management Information Systems Quarterly, 20(2), 121–142.Google Scholar
  18. Holsapple, C. W., and Whinston, A. B. (1996). Decision Support Systems. St. Paul, MN: West Publishing Company.Google Scholar
  19. Jeffers, P., Muhanna, W., and Nault, B. (2008). Information Technology and Process Performance: An Empirical Investigation of the Interaction Between IT and Non-IT Resources. Decision Sciences, 39(4), 703–735.CrossRefGoogle Scholar
  20. Keen, P. (1981). Value Analysis: Justifying Decision Support Systems. Management Information Systems Quarterly, 5(1), 1–15.Google Scholar
  21. Klein G. , Orasanu J. , Calderwood R. , and Zsambok C. , (Eds.) (1993). Decision Making in Action: Models and Methods. Norwood, NJ: Ablex.Google Scholar
  22. Kodogiannis, V. (2007). Decision Support Systems in Wireless Capsule Endoscopy: Revisited. Intelligent Decision Technologies Journal, 1(1–2), 17–31.Google Scholar
  23. Kohli, R., and Devaraj, S. (2003). Measuring Information Technology Payoff: A Meta-Analysis of Structural Variables in Firm-Level Empirical Research. Information Systems Research, 14(2), 127–145.CrossRefGoogle Scholar
  24. Kurikose, A. (1985). Successful Decision Making Starts with DSS Evaluation. Data Management, 23(2), 24–29.Google Scholar
  25. Lee, W.-P. (2004). Applying Domain Knowledge and Social Information to Product Analysis and Recommendations: An Agent-Based Decision Support System. Expert Systems, 21(3), 138–148.CrossRefGoogle Scholar
  26. Manyard, S., Burstein, F., and Arnott, D. (2001). A Multi-Faceted Decision Support System Evaluation Approach. Journal of Decision Systems, 10(3–4), 395–428.CrossRefGoogle Scholar
  27. Mintzberg, H., Raisinghani, D., and Théoret, D. (1976). Structure of ‘Unstructured’ Decision Processes. Administrative Science Quarterly, 21, 246–275.CrossRefGoogle Scholar
  28. Money, A., D. Tromp, and T. Wegner (1988) The Quantification of Decision Support Benefits Within the Context of Value Analysis. Management Information Systems Quarterly, 11(4), 515–527.Google Scholar
  29. Mora, M., Forgionne, G., Cervantes, F., Garrido, L., Gupta, J., and Gelman, O. (2005). Toward a Comprehensive Framework for the Design and Evaluation of Intelligent Decision-Making Support Systems (i-DMSS). Journal of Decision Systems, 14(3), 321–344.CrossRefGoogle Scholar
  30. Newell, A., and Simon, H. A. (1972). Human Problem Solving. Englewood Cliffs, NJ: Prentice-Hall.Google Scholar
  31. Ngai, E., and E. Chan (2005). Evaluation of Knowledge Management Tools Using AHP. Expert Systems with Applications, 29(4), 889–899.CrossRefGoogle Scholar
  32. Phillips-Wren, G., Hahn, E., and Forgionne, G. (2004). A Multiple Criteria Framework for the Evaluation of Decision Support Systems. Omega, 32(4), 323–332.CrossRefGoogle Scholar
  33. Phillips-Wren, G., Mora, M., Forgionne, G., Garrido, L., and Gupta, J. (2006). Multi-Criteria Evaluation of Intelligent Decision Making Support Systems. In J. Gupta, G. Forgionne, and M. Mora (Eds.), Intelligent Decision-Making Support Systems (i-DMSS): Foundations, Applications and Challenges (pp. 3–24). New York, NY: Springer.CrossRefGoogle Scholar
  34. Phillips-Wren, G., Mora, M., Forgionne, G., and Gupta, J. (2009). An Integrative Evaluation Framework for Intelligent Decision Support Systems. European Journal of Operational Research, 195, 642–652.CrossRefGoogle Scholar
  35. Pieptea, D. R., and Anderson, E. (1987). Price and Value of Decision Support Systems. Management Information Systems Quarterly, 11(4), 515–527.Google Scholar
  36. Pohl, J. (2008). Cognitive Elements of Human Decision Making. In G. Phillips-Wren, N. Ichalkaranje, and L. Jain (Eds.), Intelligent Decision Making: An AI-Based Approach (pp. 41–76). Berlin: Springer.CrossRefGoogle Scholar
  37. Pomerol, J.-C., and Adam, F. (2008). Understanding Human Decision Making – A Fundamental Step Towards Effective Intelligent Decision Support. In G. Phillips-Wren, N. Ichalkaranje, and L. Jain (Eds.), Intelligent Decision Making: An AI-Based Approach (pp. 3–40). Berlin: Springer.CrossRefGoogle Scholar
  38. Saaty, T. L. (1997). A Scaling Method for Priorities in Hierarchical Structures. Journal of Mathematical Psychology, 15, 234–281.CrossRefGoogle Scholar
  39. Saaty, T. L. (1986). How to Make a Decision: The Analytic Hierarchy Process. Interfaces, 24(6), 19–43.CrossRefGoogle Scholar
  40. Saaty, T., and Vargas, L. (1994). Decision Making in Economic, Political, Social and Technological Environments with the Analytic Hierarchy Process. Pittsburgh, PA: RWS Publications.Google Scholar
  41. Santhanam, R., and Guimaraes, T. (1995). Assessing the Quality of Institutional DSS. European Journal of Information Systems, 4, 159–170.CrossRefGoogle Scholar
  42. Sharda, R., Barr, S., and McDonnel, J. (1988). Decision Support Systems Effectiveness: A Review and an Empirical Test. Management Science, 34(2), 139–159.CrossRefGoogle Scholar
  43. Shim, J., Warkentin, M., Courtney, J., Power, D., Sharda, R., and Carlsson, C. (2002). Past, Present and Future of Decision Support Technology. Decision Support Systems, 33(2), 111–126.CrossRefGoogle Scholar
  44. Silverman, B., Normoyle, A., Kannan, P., Pater, R., Chandrasekaran, D., and Bharathy, G. (2008). An Embeddable Testbed for Insurgent and Terrorist Agent Theories: InsurgiSim. Intelligent Decision Technologies Journal, 2(4), 193–203.Google Scholar
  45. Simon, H. A. (1960). The New Science of Management Decision. New York, NY: Harper and Row.Google Scholar
  46. Simon, H. A. (1987). Two Heads Are Better Than One: The Collaboration Between AI and OR. Interfaces, 17(4), 8–15.CrossRefGoogle Scholar
  47. Smirnov, A., and Jakobson, G. (2009). Intelligent Decision Making in Dynamic Environments: Methods, Architectures and Applications. Intelligent Decision Technologies Journal, 3(1), 1–2.Google Scholar
  48. Sprague, R. (1980). A Framework for the Development of Decision Support Systems. Management Information Systems Quarterly, 4(4), 1–26.Google Scholar
  49. Sun, Y., and Kantor, P. (2006). Cross-Evaluation: A New Model for Information System Evaluation. Journal of the American Society for Information Science and Technology, 57(5), 614–628.CrossRefGoogle Scholar
  50. Taghezout, N., and Zarate’, P. (2009). Supporting a Multicriterion Decision Making and Multi-Agent Negotiation in Manufacturing Systems. Intelligent Decision Technologies Journal, 3(3), 139–155.Google Scholar
  51. Todd, P., and Benbasat, I. (1999). Evaluating the Impact of DSS, Cognitive Effort, and Incentives on Strategy Selection. Information Systems Research, 10(4), 356–374.CrossRefGoogle Scholar
  52. Turban, E., and Aronson, J. (1998). Decision Support Systems and Intelligent Systems. Upper Saddle River, NJ: Prentice-Hall.Google Scholar
  53. Tweedale, J., Sioutis, C., Phillips-Wren, G., Ichalkaranje, N., Urlings, P., and Jain, L. (2008). Future Directions: Building a Decision Making Framework Using Agent Teams. In G. Phillips-Wren, N. Ichalkaranje, and L. Jain (Eds.), Intelligent Decision Making: An AI-Based Approach (pp. 387–408). Berlin: Springer.CrossRefGoogle Scholar
  54. Wang, Y., and Forgionne, G. (2006). A Decision-Theoretic Approach to the Evaluation of Information Retrieval Systems. Information Processing and Management, 24, 863–874.CrossRefGoogle Scholar

Copyright information

© Springer New York 2011

Authors and Affiliations

  • Gloria Phillips-Wren
    • 1
  • Manuel Mora
    • 2
  • Guisseppi Forgionne
    • 3
  1. 1.Loyola University MarylandBaltimoreUSA
  2. 2.University of AguascalientesAguascalientesMexico
  3. 3.University of MarylandBaltimoreUSA

Personalised recommendations