Keywords

Making good comparisons and choosing which systems mapping methods to use is one of the most important things you will do in using systems mapping. This plays a big part in determining whether a participatory process is useful to stakeholders or not and whether a map can provide genuinely useful insights on the system of interest. Frequently, the choice also determines whether a good quality mapping process is even possible. Paradoxically, it is often done without much thought, if it is done at all. Often, we chose methods because they are what we know, have used before, or are asked for, based on rather random historical reasons. However, different methods have different strengths, weaknesses, and requirements, answer different sorts of questions, and work well in different situations. This makes it worthwhile broadening the range of methods in your toolkit and selecting them more thoughtfully.

Helping researchers and practitioners to make better comparisons and choices is one of our main motivations for this book. Comparisons need to be made on solid grounds, with a good understanding of the variety of methods, of how maps and models are built but also in how they are analysed and used. We also need to make decisions with an appreciation of the flexibility of many methods, which means they can sometimes accommodate requirements that on face value may seem difficult to meet. This potentially infinite flexibility needs to be tempered by an understanding of where and when methods work best, and what their real strengths are. We want to be able to find the most appropriate method for our, and our stakeholders’, needs, not just those that are adequate given several tweaks and adaptations.

The concept of ‘appropriateness’ is key here. It involves triangulating between the purpose, needs, and constraints of your project; the characteristics of the system and context you are working in; and the nature of the methods you are considering using. Where a method fits with both the project and the system, we can say it is an ‘appropriate’ method. This sounds simple in theory, and sometimes it is quite clear which methods are most appropriate; however, there are powerful forces which might interfere with our decision. We may have bias towards methods we know already, or methods which we are instinctively more comfortable with (e.g. if we are more of a quantitative or qualitative thinker), or methods we think others want us to use (e.g. funders, clients, or colleagues). We may feel stuck using one particular method which we have a track record with, which we get asked to use no matter the issue at hand. These biases and experiences will shape our choices of methods, but importantly, will also more fundamentally change the types of questions, topics, and domains we approach or interact in. It may be that we never put much thought into which methods to use because of the way our preferences and bias shape and narrow our thinking before we even get a chance to think about different methods or approaches.

Combining methods is not always necessary, but it is something we often do informally, as we adapt methods to our needs, taking inspiration from others. Methods can bleed into one another, that is, rather generic systems maps can be built that use some of the construction and analysis modes from multiple methods. For example, you might build a Causal Loop Diagram, but use some network analysis, or build an FCM where you really emphasise the feedbacks in the visualisation. If we had no time or budgetary constraints, combining methods is also something we would often recommend; to approach a question, topic, or issue from a variety of angles, to be more systematic or holistic in our thinking, to illuminate different aspects of a system, ask different questions or generate different perspectives. Exploratory combination of methods is a great source of innovation in methods and often a creative and rewarding endeavour. Thus, our default position is that combining methods in some form will normally be useful or valuable.

So, comparing and choosing systems mapping methods is important and sometimes difficult (i.e. if we feel constrained or don’t have knowledge of multiple methods), but is also rarely done formally. And combining methods is an obvious way to improve our research and can be a creative and rewarding process. The rest of this chapter consider each of these steps in turn: we outline how we compare the methods in this book; consider how we might choose between methods, and outline some of the choices we might make; and consider what combinations, both sequentially and in hybrid form, might be interesting. Finally, we conclude with some tips on resources and getting started yourself.

Comparing Systems Mapping Methods

In essence, the act of comparing methods is a simple task. All we need to do is use information on different methods, such as the information in this book, to compare the characteristics, pros, and cons of each method. In reality, each method chapter in this book is a significant simplification of what the method really is or can do. To make usable comparisons, we also have to summarise a lot of information, and variety, in a relatively small space—in our heads, or on a sheet of paper, or a slide. Table 11.1 attempts to do this ‘usable’ comparison for the methods in this book. We use the following categories to help us compare:

  • Type of map: this captures the nature of the model that is created; whether it is quantitative or qualitative, the type of structure of the network used/allowed, and the way causality is represented (i.e. do connections imply direct causal influence or something more abstract).

  • Level of focus: this draws attention to an often-forgotten characteristic of methods; they are not all focused on a whole system. Some focus on specific subsections of a system such as interventions and outcomes, or dynamical problems within them.

  • Mode of construction: this attempts to make clear the differences in how maps are started and built. This applies whether the method is being used in a participatory or a more data-driven mode. These differences are often missed, underplayed, or underappreciated by experts and beginners alike, but they can have a huge influence on the nature of the maps created.

  • Emphasis on participation: this captures how much imperative is put on the need to use the maps in a participatory mode. Some methods foreground this, others are used in participatory and non-participatory modes.

  • Mode of analysis: this captures the differences in how maps are analysed. These differences tend to be well understood by experts, but beginners don’t always realise the range of ways we might analyse maps and the large differences in the types of insights that can offer.

  • Ease of use: this attempts to describe how easy it is to use the methods with no formal training.

  • Key contributions and constraints: these two categories only mention the most obvious or important contributions and constraints of the methods. There are others for all of the methods, but here we are trying to emphasise the most salient issues.

  • When most appropriate: this is somewhat implicit from the other categories, but we use this category here to hammer-home the situations in which we think each method is at its best.

Table 11.1 Overview comparison of systems mapping methods in this book
Full size table

We hope the table is a useful tool for a relatively quick but detailed comparison of the methods we have focused on. You may want to use the table by adding new columns, extending the comparison into questions and topics specific to your problem, question, or project. Or you could add rows on other methods you are thinking of using. If anything is unclear or you want elaboration on any points, we suggest you start with the chapters on each method.

The history and roots of each method, and their deeper ontological and epistemological standpoints are missing from this table. We omit them here mainly because they are extremely difficult to meaningfully simplify down into table form. Nonetheless, the subtleties around a method’s history, underlying assumptions, and philosophy are important. They are also often easy to misunderstand or underplay. It is common for important differences in methods’ histories, which have affected how methods are thought about, framed, taught, and who has tended to use them, to go unremarked upon or underplayed. In each of the methods chapters in this book, we have attempted to give a sense of the history and philosophy of each method, so we strongly encourage readers to read these when looking at a method in detail, and to attempt to develop a sense of the history of a method.

Choosing Systems Mapping Methods

Choosing which method to use is more than just comparing their characteristics, pros, and cons. We need to also consider the aims, purpose, needs, and context of the project you are hoping to use them in, and the characteristics of the system you are studying. We need to triangulate between these three issues to find the method that is most appropriate. This will be the method that delivers the process, outputs, and insights the project needs in a timely and cost-effective fashion (given your and your users’/clients’ resources), in forms which are usable for users, stakeholders, and clients, but which also captures the salient and important elements of the system we are working in. In practice, there are many interacting and competing elements to this decision, and we will often need to prioritise or weight these, and blend in our own preferences and expertise. Inspired by the design and choice triangles developed in Stern et al. (2012) and Befani (2020), we outline our take on this choice in Fig. 11.1.

Fig. 11.1
A diagram from left to right, characteristics of the system you are studying, dotted arrow, project, aims, needs, and resources, bold arrow, characteristics of available methods, bold arrow connects to the most appropriate method. The bold arrow is the primary influence on the choice, the dotted arrow is for influences on the project.

Influences on the choice of most appropriate method. (Source: Authors’ creation, inspired by Stern et al. (2012) and Befani (2020))

Full size image

It is possible to systematically approach this decision, categorising different requirements and characteristics, weighting and scoring them, to make a choice. Indeed, others have developed tools to help us do this, see Befani (2020) for example. However, in reality, few people make the decision in this way. We do not want to attempt to specify a detailed process for making the decision but rather encourage you to take time to reflect on it, really think about your needs and the characteristics of the system, as well as the methods yourself. Do think about the needs and constraints of the users and stakeholders of your research and how different methods will work for them.

You should also think about what data is available to you, or what mode you are thinking of using a method in (i.e. participatory, qualitative data, quantitative data, evidence review). It is technically true that any of the methods in this book can be used in any of these modes, and with or without any of these types of data. However, there are more and less common ways of using particular methods; for example, PSM is normally used without quantitative data, whereas System Dynamics models are often validated against quantitative data. In practice, people often feel that data availability constrains their modelling choices tightly. However, in reality, this is often only to fit into typical or accepted modes of using methods. We believe we should be more open about the way data availability constrains choice, but also be clearer about how that choice will constrain how model results can be interpreted and used.

Every situation will be different and have its idiosyncrasies, but to flesh this decision out a bit further, and build on the some of the information we summarised in the ‘comparing’ subsection above, in Table 11.2 we have sketched out the methods we think are most appropriate given different characteristics of a project and the system we are working in. It is important to note this table is a simplification and does not include all the aspects that will affect a decision, but it gives a feel for where methods work best.

Table 11.2 Most appropriate methods given different project and system properties
Full size table

Combining Systems Mapping Methods

Combining methods can be a powerful way to approach a question more holistically, generate novel insights, or innovate methodologically. It can also be a helpful way to bring in more people and expertise to a project, and to generate additional energy and enthusiasm around the analysis. Combining methods provides a natural mode of iteration, and way of expanding and increasing the depth of any project. Therefore, we think it worthwhile taking some time here to consider how we might combine the methods in this book, and we encourage you to think about how you could combine multiple methods to improve and extend your work. You may find it helpful to look at the large literatures on mixed methods (using qualitative and quantitative methods together) and multi methods (using two qualitative methods together) approaches (e.g. Anguera et al., 2018; Greene, 2008; Tashakkori & Teddlie, 2010); there is a long history there which is useful.

There are two basic ways to combine methods, either using them sequentially, with one method building on the outputs of the other, or directly together, in a hybrid method, where the aspects of two or more methods are directly brought together in one map. These two modes of combination are easily confused, using them sequentially means producing two mapping outputs, with one method using the other as an input. Hybridising them means producing one outputs that combines elements of two or more methods.

Figure 11.2 outlines some of the most obvious combinations we might make using two methods sequentially. In effect, these are workflows, of methods using the outputs of others. There are too many permutations of how methods might be more deeply hybridised, but Table 11.3 attempts to outline a few we believe are most likely to be valuable or interesting. This table is perhaps one of the best demonstrations in this book of the flexibility of these methods; we can relatively easily add aspects of other methods, either in the emphases we use in building maps, in annotations and additions we make to the full map diagram, and in ways we might analyse them.

Fig. 11.2
A flowchart. From top to bottom, Rich pictures, initial brainstorming, Participatory systems mapping, weight connections, Fuzzy cognitive mapping, apply network analysis, Causal loop diagrams, extract T o C, Theory of Change, convert to stock and flow System dynamics, Constrain, run analysis, and Bayesian belief networks.

Some of the potential sequential combinations of systems mapping methods. (Source: Authors’ creation)

Full size image
Table 11.3 Non-exhaustive list of potential hybridised elements of methods
Full size table

Brainstorming ways in which we might combine methods is quite fun (at least to us), but we need to make sure we are practical and rigours about it too. In any combination, there would be multiple of constraints, primarily because the building blocks of different mapping types, including the types of factors and connections and the connection structure allowed, may be fundamentally different. For example, we have suggested above the possibility of converting (parts of) a PSM into a BBN, which would require constraining the network (i.e. pruning connections so that there is maximum two inputs to a factor and removing any feedbacks). We need to consider how we make the decisions on how to do this. Do we ask stakeholders to do it or do it ourselves? What is the basis for our judgements?

Getting Started with Choosing and Combining Methods

Let’s assume you don’t need to do much more for comparing methods than reading this chapter, and some of the rest of this book (for the methods we cover at least). For choosing methods, there are some useful resources we would recommend:

  • Participatory modelling, ‘selecting the right tool for the job’ paper: Voinov et al. (2018) gives an excellent overview of a range of participatory modelling methods and how we might go about choosing and combining them. It draws on a broader range of methods than just systems mapping, but still goes into a good level of detail and provides some case studies. The paper is long as it is worth taking the time to read it carefully, it is full of important nuggets of information.

  • Choosing appropriate evaluation methods: Befani (2020) is focused on the evaluation of interventions but covers at least five of the methods (or similar methods) in this book. It provides extensive discussion on the factors to include in choosing methods and provides a well-tested spreadsheet tool for you to use too. This allows you to systematically score different aspects to identify and compare the appropriateness of methods.

On combining methods, there are not many resources out there which will directly help you think about how to combine systems mapping methods, it is inherently a creative process with many undefined spaces to explore, but you may find the following inspiring:

  • Mixing Operational Research methods: Howick and Ackermann (2011) review examples of combinations of Operational Research methods, which includes some of the methods in this book (System Dynamics, different types of qualitative system models and influence maps, soft system methodology which includes Rich Pictures). They pull out some emerging themes and lessons from the combinations they consider, which are well worth reading.

  • Combining complexity-framed methods: Barbrook-Johnson and Carrick (2021) review the combination of ‘complexity-framed’ methods, that is, methods which use the ideas and language of complexity science. They draw on a larger set of examples than Howick and Ackermann (2011), but necessarily conduct a lighter review of each example, looking for patterns in how and why combinations are made, before suggesting some potential combinations which are un(der)-explored.

For getting started yourself, most immediately, we would encourage you to take the ‘comparing and choosing’ decision more carefully than you might be tempted to. Don’t rush it, or assume it is a no-brainer. Consider sketching out comparison tables applied to your context, to help you think through, and justify, your decision. Equally importantly, be sure to speak to the users and participants in your work and take their needs into consideration. If the model is not usable in their context, then it will just sit on a shelf gathering dust.

For combining methods, conceptually, the sky is the limit really, but it is hard to find the space and time to innovate methodologically or to approach projects from multiple angles. Thus, advocating, and making the time, for these activities may be a vital first step, before you even start to ‘do it’. When you do, be creative and ambitious, but anchor your explorations in the same concerns as those you include when choosing methods and be sure that all the choices you make are underpinned in appropriate ways.