Abstract
Evaluation of the performance of GDSSs has been dominated by an experimental and laboratory based approach. Other writers have argued for evaluation to be based in the “real-world” of decision making teams. The evaluation criteria have tended to ignore many of the issues that would be paramount for some of the stakeholders in the evaluation process. This article seeks to explore the criteria that might be used by a wide variety of stakeholders, including developers, facilitators, clients, key actors, vendors, as well as academics. By drawing together the criteria associated with all of the stakeholders we discover a broader, and possibly more thorough, framework for evaluation. The evaluation of any particular GDSS in relation to other GDSSs can then be seen in the context of contingent weighting applied to each of the criteria where each GDSS is able to be seen in its best light and in relation to its declared aims.
This article argues for a more eclectic and contingent approach to the evaluation of GDSSs which will encourage their future development to be clearer about purpose and the boundaries of their use.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
AckermannF. (1990). “The Role of Computers in Group Decision Support.” In C.Eden and J.Radford (eds.),Tackling Strategic Problems: The Role of Group Decision Support. London: Sage.
AckermannF. (1992). “Strategic Direction through Burning Issues,”OR Insight 5, 24–28.
AckermannF. (1996). “Participants' Perceptions on the Role of Facilitators Using Group Decision Support Systems.” Group Decision and Negotiation 5, 93–112.
AckermannF., and C.Eden. (1994). “Issues in Computer and Non-computer Supported GDSS's,”Decision Support Systems 12, 381–390.
Ackermann, F., and C. Eden (1995). “Contracting GDSS's and GSS's in the context of Strategic Change-Implications for Facilitation.” Proceedings of AIS Conference, pp. 130–132.
Ackerman, M., and T. Malone. (1990). “Answer Garden: A Tool for Growing Organisational Memory.” InProceedings of the ACM Conference on Office Information Systems, pp. 31–39.
AnsonR., and A.Heminger. (1991). “An Assessment of Process Facilitation in a Group Support System Setting.” Working paper, Boise State University, Boise, ID.
BalesR.F. (1950).Interaction Process Analysis. Reading, MA: Addison-Wesley.
Beauclair, R. (1987). “An Experimental Study of the Effects of Group Decision Support System Process Support Application on Small Group Decision Making.” Unpublished doctoral dissertation, Indiana University, Bloomington.
BeltonV., and S.Vickers. (1993). “Demystifying DEA—A Visual Interactive Approach Based on Multiple Criteria Analysis,”Journal of the Operational Society 44, 883–896.
BostromR., R.Anson, and V.Clawson. (1993). “Group Facilitation and Group Support Systems.” In L.Jessup and J.Valacich (eds.),Group Support Systems—New Perspectives. New York: Macmillan.
CannelC.F., and R.K.Kahn. (1968). “Interviewing.” In G.Lindzey and E.Aronson (eds.),Handbook of Social Psychology, Vol. 2. Reading, MA: Addison-Wesley.
CharnesA., W.W.Cooper, and E.Rhodes. (1978). “Measuring Efficiency of Decision Making Units,”European Journal of Operational Research 6, 429–444.
ClarkP.A. (1972).Action Research and Organizational Change. London: Harper & Row.
CropperS. (1990). “The Complexity of Decision Support Practice.” In C.Eden and J.Radford (eds.),Tackling Strategic Problems: The Role of Group Decision Support. London: Sage.
CropperS., C.Eden, and F.Ackermann. (1993). “Exploring and Negotiating Collective Action through Computer-aided Cognitive Mapping,”The Environmental Professional 15, 1–92.
DennisA., and R.B.Gallupe. (1993). “A History of Group Support Systems Empirical Research: Lessons learnt and Future Directions.” In L.Jessup and J.Valacich (eds.),Group Support Systems—New Perspectives. New York: Macmillan.
DennisA., J.George, L.Jessup, J.Nunamaker, and D.Vogel. (1988). “Information Technology to Support Electronic Meetings,”MIS Quarterly 12, 591–624.
DennisA., A.Heminger, J.Nunamaker, and D.Vogel. (1990). “Bringing Automated Support to Large Groups: The Burr Brown Experience,”Information and Management 18, 111–121.
DennisA., J.Nunamaker, and D.Vogel. (1991). “A Comparison of Laboratory and Field Research in the Study of Electronic Meeting Systems,”Journal of Management Information Systems 7, 107–135.
DennisA., J.Valacich, and J.Nunamaker. (1990). “An Experimental Investigation of the Effects of Group Size in an Electronic Meeting Environment,”IEEE System, Man and Cybernetics 25 1049–1057.
DeSanctisG. (1989). “Small Group Research in Information Systems: Theory and Method.” In I.Benbasat (ed.),The Information Systems Research Challenge: Experimental Research Methods. Boston: Harvard Business Press.
DeSanctisG. (1993). “Shifting Foundations in Group Support System Research.” In L.Jessup and J.Valacich (eds.),Group Support Systems—New Perspectives. New York: Macmillan.
DeSanctis, G., M. Poole, H. Lewis, and G. Desharnaise. (1991). “Using Computing to Improve the Quality Team Process: Some Initial Observations from the IRS-Minnesota Project.” In J. Nunamaker (ed.),Proceedings of the Twenty-fourth Annual Hawaii Internation Conference on System Sciences.
DicksonG., M.Poole, and G.DeSanctis. (1992). “An Overview of the Minnesota GDSS Research Project and the SAMM System.” In B.Bostrom, S.Kinney, and R.Watson (eds.),New Directions in Group Decision Support. New York: Van Nostrand and Reinhold.
DysonR.G., and E.Thanassoulis. (1988). “Reducing Weight Flexibility in Data Envelopment Analysis,”Journal of the Operational Research Society 39, 563–576.
EastonA., J.George, J.Nunamaker, and M.Pendergast. (1990). “Using Two Different Electronic Meeting System Tools for the Same Task: An Experimental Comparison,”Journal of Management Information Systems 7, 85–100.
EdenC. (1992). “A Framework for Thinking About Group Decision Support Systems (GDSS),”Group Decision and Negotiation 1, 199–218.
EdenC. (1995). “On the Evaluation of ‘Wide-Band’ GDSS's,”European Journal of Operational Research 81, 302–311.
EdenC., and F.Ackermann. (1992). “Strategy Development and Implementation: The Role of a Group Decision Support System.” In B.Bostrom, S.Kinney, and R.Watson (eds.),New Directions in Group Decision Support. New York: Van Nostrand and Reinhold.
EdenC., and F.Ackermann. (1993). “Evaluating Strategy—Its Role within the Context of Strategic Control,”Journal of the Operational Research Society 44, 853–865.
EdenC., and C.Huxham. (1996). “Action Research for the Study of Organizations.” In (S. Clegg, C. Hardy and W. Nordleds). Handbook of Organization Studies, Beverly Hills: Sage.
FinlayP.N., and J.M.Wilson. (1987). “The Paucity of Model Validation in Operational Research Projects,”Journal of the Operational Research Society 38, 303–308.
Galer, G. (1990). “A Client's Perspective.” in C. Eden, “Strategic Thinking with Computers,”Long Range Planning 23 42–43.
GallupeR., G.DeSanctis, and G.Dickson. (1988). “Computer-based Support for Group Problem Finding: An Experimental Investigation,”MIS Quarterly 12, 277–296.
GearA.E., and M.J.Read. (1993). “On Line Process Support,”Omega 21, 261–274.
GeorgeJ., F.Easton, J.Nunamaker, and G.Northcraft. (1990). “A Study of Collaborative Group Work with and without Computer Based Support,”Information Systems Research 1, 394–415.
GrayP., D.Vogel, and R.Beauclair. (1990). “Assessing GDSS Empirical Research,”European Journal of Operations Research 49, 162–176.
HuberG. (1984). “Issues in the Design of Group Decision Support Systems,”Management Information Systems Quarterly 8, 195–204.
HuxhamC., S.Cropper, and P.Bennett. (1989). “Decision Aiding Demonstrated,”OR Insight, 2, 15–21.
JarvenpaaS., V.Rao, and G.Huber. (1988). “Computer Support for Meetings of Medium-sized Groups Working on Unstructured Problems: A Field Experiment,”MIS Quarterly 12, 645–666.
Johansen, R. (1991). “Teams for Tomorrow.” In J. Nunamaker (ed.),Proceedings of the Twenty-fourth Annual Hawaii Internation Conference on System Sciences 3, 521–534.
JohansenR., D.Sibbet, S.Benson, A.Martin, R.Mittman, and R.Saffo. (1991).Leading Business Teams: How Teams Can Use Technology and Process to Enhance Group Performance. Reading, MA: Addison-Wesley.
LandryM., J.-L.Malouin, and M.Oral. (1983). “Model Validation in Operations Research,”European Journal of Operational Research 14, 207–220.
LeeA. (1989). “A Scientific Methodology for MIS Case Studies,”MIS Quarterly 13, 33–50.
Lewis, F. (1982). “Facilitator: A Microcomputer Decision Support System for Small Groups.” Unpublished doctoral dissertation, University of Louisville.
LewisF. (1993). “Decision-aiding Software for Group Decisionmaking.” In S.Nagel (ed.),Decision-Aiding Software and Decision Analysis: Theory and Applications. Westport, CT: Quorum Books.
Losada M., P. Sanchez, and E.E. Noble. (1990). “Collaborative Technology and Group Process Feedback: Their Impact on Interactive Sequences in Meetings.” InCSCW 90 Proceedings, October.
McGoffC., A.Hunt, D.Vogel, and J.Nunamaker. (1989). “The Role of the Facilitator in the IBM Decision Support Centre Process.” Working paper, Department of MIS, University of Arizona, Tucson.
MaloneT., and K.Y.Lai. (1992). “Toward Intelligent Tools for Information Sharing and Collaboration.” In B.Bostrom, S.Kinney, and R.Watson (eds.),New Directions in Group Decision Support. New York: Van Nostrand and Reinhold.
NunamakerJ., L.Applegate, and B.Konsynski. (1988). “Computer-aided Deliberation: Model Management and Group Decision Support,”Journal of Operations Research 36, 5–19.
PhillipsL. (1987). “People-centred Group Decision Support.” In G.I.Doukidis, F.Land, and G.Miller (eds.),Knowledge Based Management Support Systems. Chichester: Ellis Horwood, pp. 208–224.
PinsonneaultA., and K.Kraemer. (1989). “The Impact of Technological Support on Groups: An Assessment of the Empirical Research,”Decision Support Systems 5, 197–216.
RohrbaughJ. and C.Eden. (1990). “Using the Competing Values Approach to Explore ‘Ways of Working’.” In C.Eden and J.Radford (eds.),Tackling Strategic Problems: The Role of Group Decision Support. London: Sage.
SchnelleE. (1979).The Metaplan-Method: Communication Tools for Planning and Learning Groups. Hamburg: Quickborn.
SteebR., and S.Johnstone. (1981). “A Computer-based Interactive System for Group Decision Making,”IEEE Transactions on Systems, Man and Cybernetics SMC-11, 544–552.
SuchmanE.A. (1967).Evaluative Research: Principles and Practice in Public Service and Social Action Programs. New York: Russell Sage Foundation.
SusmanG.I., and R.D.Evered. (1978). “An Assessment of the Scientific Merits of Action Research.”Administrative Science Quarterly 23, 582–603.
SykesW. (1990). “Validity and Reliability in Qualitative Market Research: A Review of the Literature,”Journal of the Market Research Society 32, 289–327.
TuroffM., and R.Hiltz. (1982). “Computer Support for Group versus Individual Decisions,”IEEE Transactions on Communications 30, 82–91.
WatsonR., G.DeSanctis, and M.Poole. (1988). “Using a GDSS to Facilitate Group Consensus: Some Intended and Unintended Consequences,”MIS Quarterly 12, 463–478.
WeickK., and D.Meader. (1993). “Sensemaking and Group Support Systems.” In L.Jessup and J.Valacich (eds.),Group Support Systems—New Perspectives. New York: Macmillan.
WilliamsF., R.Rice, and E.Rogers. (1988).Research Methods and the New Media. New York: The Free Press.
YinR. (1984).Case Study Research: Design and Methods. New York: Sage.
ZigursI. (1993). “Methodological and Measurement Issues in Group Support Systems Research.” In L.Jessup and J.Valacich (eds.),Group Support Systems—New Perspectives. New York: Macmillan.
ZmudR., and A.Boyton. (1989). “Survey Measures and Instruments in MIS: Inventory and Appraisal.” Working paper, Florida State University, Tallahassee.
Author information
Authors and Affiliations
Rights and permissions
About this article
Cite this article
Eden, C., Ackermann, F. “Horses for courses”: A stakeholder approach to the evaluation of GDSSs. Group Decis Negot 5, 501–519 (1996). https://doi.org/10.1007/BF00553915
Issue Date:
DOI: https://doi.org/10.1007/BF00553915