Skip to main content
SpringerLink
Log in

On Evaluating the Performance of Problem Structuring Methods: An Attempt at Formulating a Conceptual Model

  • Published:
Group Decision and Negotiation Aims and scope Submit manuscript

Abstract

In the past decade there has been a discussion on the need for and degree of empirical evidence for the effectiveness of problem structuring methods (PSMs). Some authors propose that PSMs are used in unique situations which are difficult to study, both from a methodological and a practical perspective. In another view experimental validation is necessary and, if not obtained, PSMs remain substantially invalidated and thus ‘suspect’ with regard to their claims of effectiveness. Both views agree on one point: the necessity of being clear about the important factors in the context in which a method is used, the method’s aims and its essential elements through which these aims are achieved. A clear formulation of central variables is the core of a theoretical validation, without which empirical testing of effects is impossible. Since the process of PSMs is sometimes referred to as ‘more art than science’, increased clarity on the PSM process also supports the transfer of methods. In this article we consider goals important to most PSMs, such as consensus and commitment. We then focus on outcomes of group model building, and expectations on how context and group modeling process contributes to outcomes. Next we discuss the similarity of these central variables and relations to two sets of theories in social psychology: the theory of planned behavior and dual process theories of persuasion. On the basis of these theories we construct a preliminary conceptual model on group model building effectiveness and address its practical applicability for research on PSM.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  • Ackermann F and Eden C with Brown I (2005). The practice of making strategy. A step-by-step guide. Sage, London

    Google Scholar 

  • Akkermans HA (1995) Modelling with managers: participative business modelling for effective strategic decision-making. Unpublished doctoral dissertation, Eindhoven Technical University, Eindhoven, The Netherlands

  • Andersen DF and Richardson GP (1997). Scripts for group model building. Syst Dyn Rev 13(2): 107–129

    Article  Google Scholar 

  • Andersen DF, Richardson GP and Vennix JAM (1997). Group model building: adding more science to the craft. Syst Dyn Rev 13(2): 187–203

    Article  Google Scholar 

  • Andersen DF, Vennix JAM, Richardson GP and Rouwette EAJA (2007). Group model building: problem structuring, policy simulation and decision support. J Oper Res Soc 58(5): 691–694

    Article  Google Scholar 

  • Ajzen I (1991). The theory of planned behavior. Organ Behav Human Decis Process 50: 179–211

    Article  Google Scholar 

  • Ajzen I (2001). Nature and operation of attitudes. Annu Rev Psychol 52: 27–58

    Article  Google Scholar 

  • Ajzen I and Fishbein M (1980). Understanding attitudes and predicting social behavior. Englewood Cliffs, New Jersey

    Google Scholar 

  • Briggs RO, de Vreede GJ, Nunamaker JFJ (2003) Collaboration engineering with ThinkLets to pursue sustained success with group support systems. J Manage Inform Syst 19: 31–63

    Google Scholar 

  • Brockner J (1992). The escalation of commitment to a failing course of action – toward theoretical progress. Acad Manage Rev 17(1): 39–61

    Article  Google Scholar 

  • Burt G (2000) The role of jolts, managerial recipes and transitional objects in the management of change. Unpublished doctoral dissertation, University of Strathclyde, Strathclyde, Scotland

  • Burt G and Vander Heijden K (2003). First steps: towards purposeful activities in scenario thinking and future studies. Futures 35(10): 1011–1026

    Article  Google Scholar 

  • Campbell D (2001). The long and winding (and frequently bumpy) road to successful client engagement: one team’s journey. Syst Dyn Rev Special Issue Consult Practice 17(3): 195–215

    Google Scholar 

  • Chaiken S, Liberman A and Eagly AH (1989). Heuristic and systematic processing within and beyond the persuasion context. In: Uleman, JS and Bargh, JA (eds) Unintended thought., pp 212–252. Guilford Press, New York

    Google Scholar 

  • Chaiken S, Giner-Sorolla R and Chen S (1996). Beyond accuracy: defense and impression motives in heuristic and systematic information processing. In: Gollwitzer, PM and Bargh, JA (eds) The psychology of action: linking cognition and motivation to action, pp 553–578. Guilford, New York

    Google Scholar 

  • Checkland P (1989). Soft systems methodology. In: Rosenhead, J (eds) Rational analysis for a problematic world, pp 71–100. Chichester, Wiley

    Google Scholar 

  • Checkland P (2000). Soft systems methodology: a thirty year retrospective. Syst Res Behav Sci 17: S11–S58

    Article  Google Scholar 

  • Checkland PB (2006). Reply to Eden and Ackermann: any future for problem structuring methods. J Oper Res Soc 57(7): 769–771

    Article  Google Scholar 

  • Checkland PB and Scholes J (1990). Soft systems methodology in action. Wiley, Chichester

    Google Scholar 

  • Delauzun F and Mollona E (1999). Introducing system dynamics to the BBC World Service: an insider perspective. J Oper Res Soc 50(4): 364–371

    Article  Google Scholar 

  • Dennis AR, Wixom BH and Vandenberg RJ (2001). Understanding fit and appropriation effects in group support systems via meta-analysis. Manage Inform Syst Quart 25: 167–183

    Google Scholar 

  • Doyle JK and Ford DN (1998). Mental models concepts for system dynamics research. Syst Dyn Rev 14(1): 3–29

    Article  Google Scholar 

  • Eagly AH and Chaiken S (1993). The psychology of attitudes. Harcourt Brace Jovanovich, Orlando

    Google Scholar 

  • Eco U (1976) A theory of semiotics. Indiana University Press, Bloomington (Midland Book edn., 1979)

  • Eden C (1989). Using cognitive mapping for strategic options development. In: Rosenhead, J (eds) Rational analysis for a problematic world, pp 21–42. Chichester, Wiley

    Google Scholar 

  • Eden C (1992). A framework for thinking about group decision support systems (GDSS). Group Decis Negot 1: 199–218

    Article  Google Scholar 

  • Eden C (1995). On evaluating the performance of ‘wide-band’ GDSS’s. Eur J Oper Res 81: 302–311

    Article  Google Scholar 

  • Eden C (2000). On evaluating the performance of GSS: furthering the debate, by Paul Finlay [European Journal of Operational Research 107, pp 193–201]. Eur J Oper Res 120: 218–222

    Google Scholar 

  • Eden C and Ackermann F (1996). “Horses for courses”: a stakeholder approach to the evaluation of GDSSs. Group Decis Negot 5: 501–519

    Google Scholar 

  • Eden C and Ackermann F (1998). Making strategy. The journey of strategic management. Sage, London

    Google Scholar 

  • Eden C and Ackermann F (2001a). Group decision and negotiation in strategy making. Group Decis Negot 10: 119–140

    Article  Google Scholar 

  • Eden C and Ackermann J (2001). SODA – The principles. In: Rosenhead, J and Mingers, J (eds) Rational analysis for a problematic world revisited. Problem structuring methods for complexity, uncertainty and conflict, pp 21–41. Chichester, Wiley

    Google Scholar 

  • Eden C and Ackermann F (2006). Where next for problem structuring methods. J Oper Res Soc 57(7): 766–768

    Article  Google Scholar 

  • Eden C and Radford J (1990). Tackling strategic problems: the role of group decision support. Sage, London

    Google Scholar 

  • Eskinasi M, Rouwette E (2004) Simulating the urban transformation process in the Haaglanden region, the Netherlands. In: Proceedings system dynamics conference, Oxford

  • Faber J (1994). Bespreking ‘Participative policy modelling applied to the health care insurance industry’. Sociologische Gids 94/5: 393–394

    Google Scholar 

  • Festinger L (1957). A theory of cognitive dissonance. Row, Peterson, Evanston, IL

    Google Scholar 

  • Finlay PN (1998). On evaluating the performance of GSS: furthering the debate. Eur J Oper Res 107(1): 193–201

    Article  Google Scholar 

  • Fishbein M (1967) Attitude and the prediction of behavior. In: Fishbein M (ed) Readings in attitude theory and measurement. Wiley, New York, pp 477–492

  • Fishbein M and Ajzen I (1975). Belief, attitude, intention and behavior: an introduction to theory and research. Addison-Wesley, Reading, MA

    Google Scholar 

  • Fjermestad J and Hiltz SR (1998). An assessment of group support systems experimental research: methodology and results. J Manage Inform Syst 15(3): 7–149

    Google Scholar 

  • Fjermestad J and Hiltz SR (2000). Group support systems: a descriptive evaluation of case and field studies. J Manage Inform Syst 17(3): 115–159

    Google Scholar 

  • Flood RL and Jackson MC (1991). Creative problem solving: total systems intervention. Wiley, Chichester

    Google Scholar 

  • Forrester JW (1958) Industrial dynamics: a major breakthrough for decision makers. Harvard Bus Rev July–August, 37–66

  • Forrester JW (1961). Industrial dynamics. Pegasus Communications, Williston, VT

    Google Scholar 

  • Forrester JW (1975) Industrial dynamics – after the first decade. Collected papers of J.W. Forrester, Pegasus Communications, Williston, VT (Original paper in Management Science 14(7):398–415, 1968.)

  • Friend J (2001). The strategic choice approach. In: Rosenhead, J and Mingers, J (eds) Rational analysis for a problematic world revisited. Problem structuring methods for complexity, uncertainty and conflict, pp 115–149. Wiley, Chichester

    Google Scholar 

  • Friend J and Hickling A (1987). Planning under pressure. The strategic choice approach. Pergamon Press, Oxford

    Google Scholar 

  • Hickson DJ, Butler RJ, Cray D, Mallory GR and Wilson DC (1986). Top decisions: strategic decision making in organizations. Basil Blackwell, Oxford

    Google Scholar 

  • Huz S (1999). Alignment from group model building for systems thinking: measurement and evaluation from a public policy setting. Unpublished doctoral dissertation. SUNY, Albany, New York

    Google Scholar 

  • Huz S, Andersen DF, Richardson GP and Boothroyd R (1997). A framework for evaluating systems thinking interventions: an experimental approach to mental health system change. Syst Dyn Rev 13(2): 149–169

    Article  Google Scholar 

  • Kelly GA (1955). The psychology of personal constructs: a theory of personality. Norton, New York

    Google Scholar 

  • Korsgaard MA, Schweiger DM and Sapienza HJ (1995). Building commitment, attachment, and trust in strategic decision making teams: the role of procedural justice. Acad Manage J 38: 60–84

    Article  Google Scholar 

  • Lane DC (1992). Modelling as learning: a consultancy methodology for enhancing learning in management teams. Eur J Oper Res 59: 64–84

    Article  Google Scholar 

  • Lane DC (1994). With a little help from our friends – how system dynamics and soft OR can learn from each other. Syst Dyn Rev 10(2–3): 101–134

    Article  Google Scholar 

  • Lane DC (2001). Rerum cognoscere causas: Part I – How do the ideas of system dynamics relate to traditional social theories and the voluntarism/determinism debate?. Syst Dyn Rev 17(2): 97–118

    Article  Google Scholar 

  • Lane DC (2001). Rerum cognoscere causas: Part II – Opportunities generated by the agency/structure debate and suggestions for clarifying the social theoretic position of system dynamics. Syst Dyn Rev 17(4): 293–309

    Article  Google Scholar 

  • Markóczy L (2001). Consensus formation during strategic change. Strategic Manage J 22: 1013–1031

    Article  Google Scholar 

  • McGrath JE and Hollingshead AB (1994). Groups interacting with technology. Sage, London

    Google Scholar 

  • Mingers J (2000). Variety is the spice of life: combining soft and hard OR/MS methods. Int Trans Oper Res 7: 673–691

    Article  Google Scholar 

  • Mingers J and Rosenhead J (2004). Problem structuring methods in action. Eur J Oper Res 152: 530–554

    Article  Google Scholar 

  • Morecroft JDW (1992). Executive knowledge, models and learning. Eur J Oper Res 59: 9–27

    Article  Google Scholar 

  • Morecroft JDW (2004). Mental models and learning in system dynamics practice. In: Pidd, M (eds) Systems modelling: theory and practice, pp 101–126. Chichester, Wiley

    Google Scholar 

  • Morton A, Ackermann F and Belton V (2003). Technology-driven and model-driven approaches to group decision support: focus, research philosophy, and key concepts. Eur J Inform Syst 12: 110–126

    Article  Google Scholar 

  • Near JP (1989). Organizational commitment among Japanese and U.S. workers. Organ Stud 10: 281–300

    Article  Google Scholar 

  • Pawson R and Tilley N (1997). Realistic evaluation. Sage, London

    Google Scholar 

  • Pervan GP (1998). A review of research in Group Support Systems: leaders, approaches and directions. Decis Support Syst 23(2): 149–159

    Article  Google Scholar 

  • Pervan G, Lewis LF and Bajwa DS (2004). Adoption and use of electronic meeting systems in large Australian and New Zealand organizations. Group Decis Negot 13(5): 403–414

    Article  Google Scholar 

  • Petty RE and Cacioppo JT (1986). The elaboration likelihood model of persuasion. Adv Exp Soc Psychol 19: 123–205

    Article  Google Scholar 

  • Petty RE and Wegener DT (1998). Attitude change: multiple roles for persuasion variables. In: Gilbert, DT, Fiske, ST and Lindzey, G (eds) The handbook of social psychology, vols 1, 2 4th edn, pp. McGraw-Hill, Boston

    Google Scholar 

  • Pidd M (2003). Tools for thinking: modeling in management science. Wiley, Chichester

    Google Scholar 

  • Pinsonneault A and Kraemer KL (1989). The impact of technological support on groups: an assessment of the empirical research. Decis Support Syst 5: 197–216

    Article  Google Scholar 

  • Randall DM (1993). Cross-cultural research on organizational commitment: a review and application of Hofstede’s value survey module. J Bus Res 26: 91–110

    Article  Google Scholar 

  • Randers J (1977). The potential in simulation of macro-social processes, or how to be a useful builder of simulation models. Gruppen for Ressurstudier, Oslo, Norway

    Google Scholar 

  • Richardson GP and Andersen DF (1995). Teamwork in group model building. Syst Dyn Rev 11(2): 113–137

    Article  Google Scholar 

  • Richardson GP and Pugh AL (1981). Introduction to system dynamics modelling with DYNAMO. MIT Press, Cambridge, MA

    Google Scholar 

  • Richardson GP, Andersen DF, Maxwell, TA, Stewart TR (1994) Foundations of mental model research. In: Proceedings of the 1994 international system dynamics conference: problem solving methodologies. Stirling, Scotland, pp 181–192

  • Richmond B (1987). The strategic forum: from vision to strategy to operating policies and back again. High Performance Systems, Lyme, NH

    Google Scholar 

  • Richmond B (1997). The strategic forum: aligning objectives, strategy and process. Syst Dyn Rev 13(2): 131–148

    Article  Google Scholar 

  • Roberts EB (1978) Strategies for effective implementation of complex corporate models. In: Roberts EB (ed) Managerial applications of system dynamics. Productivity Press, Cambridge, MA, pp 77–85

  • Rosenhead J (ed) (1989) Rational analysis for a problematic world. Wiley, Chichester

    Google Scholar 

  • Rosenhead J (1996). What’s the problem? An introduction to problem structuring methods. Interfaces 26(6): 117–131

    Article  Google Scholar 

  • Rosenhead J, Mingers J (eds) (2001) Rational analysis for a problematic world revisited. Problem structuring methods for complexity, uncertainty and conflict. Wiley, Chichester

    Google Scholar 

  • Rowe G and Wright G (1999). The Delphi technique as a forecasting tool: issues and analysis. Int J Forecast 15: 353–375

    Article  Google Scholar 

  • (2003). Group model building as mutual persuasion. Doctoral dissertation, Radboud University Nijmegen. Wolf Legal Publishers, Nijmegen

    Google Scholar 

  • Rouwette EAJA and Vennix JAM (2006). System dynamics and organizational interventions. Syst Res Behav Sci 23(4): 451–466

    Article  Google Scholar 

  • Rouwette EAJA, Vennix JAM and Mullekom T (2002). Group model building effectiveness: a review of assessment studies. Syst Dyn Rev 18(1): 5–45

    Article  Google Scholar 

  • Scheper WJ (1991) Group decision support systems: an inquiry into theoretical and philosophical issues. Unpublished doctoral dissertation, University of Utrecht, Utrecht, The Netherlands

    Google Scholar 

  • Scheper WJ and Faber J (1994). Do cognitive maps make sense?  . Adv Manage Cogn Organ Inform Process 5: 165–185

    Google Scholar 

  • Siegrist J (1970) Das Consensus-Modell. Studien zur Interaktionstheorie und zur kognitiven Sozialisation. [The consensus model. Studies on interaction theory and cognitive socialization.] Enke, Stuttgart

  • Stasson MF, Ono K, Zimmerman SK and Davis JH (1988). Group consensus processes on cognitive bias tasks: a social decision scheme approach. Jpn Psychol Res 30(2): 68–77

    Google Scholar 

  • Staw BM and Ross J (1978). Commitment to a policy decision – multi-theoretical perspective. Admin Sci Quart 23(1): 40–64

    Article  Google Scholar 

  • Sterman JD (1994). Learning in and about complex systems. Syst Dyn Rev 10(2–3): 291–330

    Article  Google Scholar 

  • Stevens CA and Finlay PN (1996). A research framework for group support systems. Group Decis Negot 5: 521–543

    Google Scholar 

  • Susskind L, McKearnan S, Thomas-Larmer J (eds) (1999)c The consensus building handbook. Sage, London

    Google Scholar 

  • Thibaut J and Walker J (1975). Procedural justice: a psychological analysis. Erlbaum, Hillsdale, NJ

    Google Scholar 

  • Van den Putte B (1993) On the theory of reasoned action. Unpublished doctoral dissertation, University of Amsterdam

  • Vennix JAM (1995). Building consensus in strategic decision making: insights from the process of group model building. Group Decis Negot 4: 335–355

    Article  Google Scholar 

  • Vennix JAM (1996). Group model building: facilitating team learning using system dynamics. Wiley, Chichester

    Google Scholar 

  • Vennix JAM (1999). Group model-building: tackling messy problems. Syst Dyn Rev 15(4): 379–401

    Article  Google Scholar 

  • Vennix JAM, Akkermans HA and Rouwette EAJA (1996). Group model building to facilitate organisational change: an exploratory study. Syst Dyn Rev 12(1): 39–58

    Article  Google Scholar 

  • Vennix JAM, Andersen DF and Richardson GP (1997). Foreword: group model building, art, and science. Syst Dyn Rev 13(2): 103–106

    Article  Google Scholar 

  • Westcombe M, Franco LA and Shaw D (2006). Where next for PSMs – a grassroots revolution?   . J Oper Res Soc 57(7): 776–778

    Article  Google Scholar 

  • Wicker AW (1969). Attitude versus actions: the relationship of verbal and overt behavioral responses to attitude objects. J Soc Issues 25(4): 41–78

    Article  Google Scholar 

  • Wolstenholme E (1992). The definition and application of a stepwise approach to model conceptualisation and analysis. Eur J Oper Res 59: 123–136

    Article  Google Scholar 

  • Wood W (2000). Attitude change: persuasion and social influence. Annu Rev Psychol 51: 539–570

    Article  Google Scholar 

  • Woolley RN and Pidd M (1981). Problem structuring – a literature review. J Oper Res Soc 32(3): 197–206

    Article  Google Scholar 

  • Zagonel AA (2004). Reflecting on group model building used to support welfare reform in New York state. Unpublished doctoral dissertation, SUNY, Albany, NY

    Google Scholar 

  • Zigurs I (1993). Methodological and measurement issues in group support systems research. In: Jessup, LM and Valacich, JS (eds) Group support systems. New perspectives, pp 112–122. MacMillan, New York

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. PO Box 9108, 6500 HK, Nijmegen, The Netherlands

    Etiënne A. J. A. Rouwette, Jac A. M. Vennix & Albert J. A. Felling

Authors
  1. Etiënne A. J. A. Rouwette
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Jac A. M. Vennix
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Albert J. A. Felling
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Etiënne A. J. A. Rouwette.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Rouwette, E.A.J.A., Vennix, J.A.M. & Felling, A.J.A. On Evaluating the Performance of Problem Structuring Methods: An Attempt at Formulating a Conceptual Model. Group Decis Negot 18, 567–587 (2009). https://doi.org/10.1007/s10726-007-9100-z

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10726-007-9100-z

Keywords

Search

Navigation