Theory of Change Diagrams

Abstract

This chapter introduces Theory of Change diagrams, a popular approach to mapping the causal logic between interventions, their impacts, and the assumptions they rely upon. Despite a wide variety in practice, we attempt to outline what Theory of Change diagrams are, how you can use them, and where they sit in the wider Theory of Change approach. We describe their strengths, weaknesses, and a brief history, and point readers to useful resources, as well as offer some tips for getting started.

On a superficial level, we could have included Theory of Change (ToC) in this book simply because it is popular and we often see ToC diagrams that look like the diagrams other systems mapping methods produce. However, ToC is more than ‘just’ an individual system mapping method; it is also an entire approach in its own right to framing, structuring, and implementing the design and evaluation of interventions. ToC has its roots in the evaluation community and literature—that is, the discipline focused on assessing the impact and success of interventions. From these roots, it has spread and is now also used widely in the design of interventions. In some domains, it has also become a core component in the communication of any project, programme, or organisation; in the same way we might ask what our aims and goals are, we now often ask what is our Theory of Change. In these domains, if you can’t quickly explain your ToC, you will often be thought of as ill-prepared or naïve.

The success of ToC in becoming a core component of how we communicate our reasoning with others, and how we frame the design and evaluation of interventions, is founded on the value it has given to people in many different settings. It has, on a basic level, been a very useful tool in many different places. This success is also a reflection of its flexibility and ambiguity. It is used in numerous different ways, which makes it a little difficult to talk about from a methodological perspective. There are many different flavours of ToC, arguably as many as there are people using it, and the term can be used somewhat glibly, or may mean many different things. We won’t be able to describe the full variety here. Nonetheless, we were keen to include it in this book for several reasons. Firstly, because of the immense and undeniable value it has given to people. Secondly, because in its purest forms it maintains a dogged focus on using the logic of cause and effect to understand the impacts of interventions in systems. And thirdly, because it has a practicality and immediate instrumental value, it is a useful connection from and complement to some of the more abstract or exploratory methods in this book, through to the harsh realities of doing systems mapping in pragmatic settings such as government, large businesses, or third-sector organisations.

In the rest of this chapter, we focus mainly on ToC diagrams specifically, though we do touch on the broader ideas and interpretations of the ToC approach. We explain what it is and how it is done. We consider common issues, how these are overcome, and some tricks of the trade for creating and using ToC diagrams. We also make some assertions about what it is good and bad at. We finish by giving a brief history of ToC and highlighting some useful resources for those of you thinking about using it.

What Is Theory of Change Mapping?

In this chapter, we want to focus on ToC diagrams, but it is worth making clear ToCs often also come in the form of text. Textual ToCs can be as short as one sentence, or longer, perhaps a paragraph or two, but don’t tend to be much longer than half a page or so. We have seen examples of textual ToCs structured in tables, breaking down the elements in a more organised way.

ToC diagrams come in many different forms, from simple flow diagram-type images, with maybe only a handful of boxes and one or two connections, through to large and complex diagrams with many boxes, many connections, detailed legends, and lots of annotation and supporting text. What all ToC diagrams have in common is that they are attempting to map out the connections and pathways between an intervention and its outcomes. They all use some form of causal logic to describe what and how impacts might be created by an intervention. They are all intended to explain the ‘logic’ or ‘theory’ of the intervention. All but the simplest examples do this by using the boxes, connections, and any text to describe the elements of the intervention, its immediate outputs, longer-term outcomes, and ultimate impacts. Importantly, any key assumptions about how these will be realised are typically included in the diagram.

Let us look at some examples to flesh this out. Figure 3.1 shows an example of a ToC diagram made to show the ToC of a child support grant programme in South Africa. It uses a top-to-bottom layout; on the right-hand side we can see the key categories of ‘activities’, ‘outputs’, ‘short-term outcomes’, ‘medium-term outcomes’, and ‘long-term outcomes’ (see below for definitions). Within each grouping there are boxes which refer to different types of things. These are connected by different types of arrows. As in many ToC diagrams, shading, additional boxes, and annotations are used to highlight different points. The diagram has many ambiguities about exactly what arrows and boxes can mean or can be, this is normally explained in accompanying text, but you should expect variety and ambiguity in the definitions. We see this is a ToC diagram with a medium level of detail or complexity. There are only a few activities and outputs, but many types of outcomes. In other ToC diagrams, we might see more emphasis on different components of an intervention, that is, more activities, or another common category—‘inputs’. The boxes and annotation at the top left make clear this ToC is being used as part of an evaluation, with the reference to the types of methods which will be used to assess different elements.

Fig. 3.1

Theory of Change diagram for a child support grant programme in South Africa. Source: DSD, SASSA, and UNICEF (2012)

Because there is such variety in ToC diagrams, it is worth looking at another example. Figure 3.2 shows an example for an education improvement programme in Ethiopia. This is a simpler diagram and uses a left-to-right layout. There is only one input (the funding for the programme), no activities listed, and then a list of aspirational outputs which are rather abstractly connected to outcomes and impacts. The diagram is likely useful as a quick communication tool but, because of its simplicity, does not really create any additional framing or analytical value.

Fig. 3.2

Theory of Change diagram for an education improvement programme in Ethiopia. Source: Vogel and Stephenson (2012)

The differences between these two examples are clear, but they do have some things in common. The most obvious thing is that they both use some basic units or categories to structure the diagram (inputs, outputs, etc.), and they both use a direction of flow, from one side of the diagram to the other (we have seen examples which don’t do this, but they are rare). The exact list of the categories differs in many diagrams and are often a source of confusion. We have tended to see the following common categories, which are also discussed in Rogers (2014) (referred to as a ‘results chain’):

• Inputs: the resources (broadly defined) used or required.

• Activities: the actions, events, and undertakings of the intervention.

• Outputs: the immediate tangible products of the intervention. These tend to be easy to define and identify, akin to something like deliverables from a project.

• Outcomes: the potential short and medium-term effects of an intervention. These might be more difficult to measure and will be less tangible than an output.

• Impacts: the long-term effects of an intervention and/or the long-term changes it contributes to.

Both diagrams are also underpinned by a causal logic. Even though they differ in level of detail, they are trying to articulate the cause-and-effect relationships between interventions and outcomes. This is not done at an individual variable, factor, or mechanism level (as in some of the methods in this book) but a more aggregate abstract level. Nonetheless, the causal element is important. Arguably, all the causal assertions in ToC diagrams are based on trying to show what aspects of an intervention are necessary to create the changes they are aiming at. In the first diagram, there is enough detail to see these causal pathways as something like causal mechanisms by which the intervention leads to the long-term goals. In the second diagram however, cause is less clear. One could argue that the outputs are not connected closely enough to inputs and this could mean that potential enablers or barriers to effectiveness of the intervention are not addressed.

Though the clue is in the name, it is important to keep in mind that ToCs are theories, they represent the mental models of the people who constructed them. They are not maps of reality and they may contain gaps and ambiguities, as indeed may the theories they represent. When used in the evaluation of an intervention, one of the core purposes of the evaluation is to test this underlying theory and ask, ‘did this intervention have the effects we hoped it would, i.e. is our ToC correct?’ This is important to remember because it affects the way we might use a ToC diagram. As a theory, we should see it as a rough guide, one that might be incomplete and foggy in places, but one that will become clearer as we evaluate and plan in more detail. A ToC should ideally be iterated multiple times, being refined each time.

As we stated above, there is a lot of variety in the practice of ToC diagrams, but there is also variety in terminology which can be confusing. There are many phrases that are seemingly used interchangeably with ToC, these include: ‘programme theory’, ‘intervention theory’, ‘logic mapping’, ‘logic models’, ‘results chain’, and ‘outcome mapping’. We do not want to attempt to define these here, which is a difficult and thankless task. ToC is often used as an umbrella term for a process within which things like outcome mapping or logic mapping might be done, or an intervention/programme theory might be developed which includes ToC diagrams. It is important to take into consideration the fact that these terms are used in different ways. Some researchers and practitioners have tried to define them precisely, but such is the variety in practice and terminology, that we have found we normally cannot rely on these labels to understand what someone is doing, but rather need to look at the work directly.

How Do You Create Theory of Change Diagrams?

Due to the variety of practice in ToC we cannot outline a definitive or detailed step-by-step guide on how to create ToC diagrams, but we can outline some of the broad steps involved. These can be undertaken in a workshop setting with stakeholders, or in small teams of those directly involved with an evaluation or design process. Occasionally, ToC diagrams are developed by individuals, but we would normally advise against this unless there are very clear reasons why this is appropriate in a given context. As with many of the methods in this book, diagrams can be created with pen and paper, post-its, and so on, or with software, though using software greatly privileges the role of the person who is operating it.

Broadly speaking, the following steps are common in the creation of relatively sophisticated ToC diagrams; if you want to create a simple ToC for communication purposes, these steps are probably overkill:

• Start with the intervention: in many cases it will be clear what exactly the intervention is, and what inputs and activities make it up, so this may be an easy step. The specific elements of the intervention must be agreed upon and turned into the boxes which fall in the ‘inputs’ and ‘activities’ categories of a ToC.

• Define the long-term impacts: next, we jump to the other end of the diagram to the long-term impacts the intervention is trying to affect or contribute to. These tend to be similar to the basic aims or goals of the intervention and are normally quite clear from the start. Again, they should be broken down into distinct elements which can be placed in boxes on the diagram.

• Fill in the gaps: now comes the first difficult part, starting to fill in the gaps between the elements of the intervention and its long-term goals. You will need to choose which set of categories you want to use, we tend to prefer the set we define above. To fill in the gap, we recommend starting at the short and medium-term outcomes which are closely related to the long-term impacts, defining these, and then defining the outputs of the intervention that lead to them (i.e. starting towards the end, and working backwards).

• Make it specific and realistic: as you go, you will likely need to prompt regularly the people involved to be specific and realistic in what they are suggesting. A realistic set of steps which create the pathway or mechanism between intervention and impact should be clearly visible in the diagram. If any step or box feels as if it hides or simplifies away important detail, this should be explored and captured. It can be useful to ask people for specifics by always asking who will be doing something, how much of something will be happening, how that will lead to the next thing, and so on. Another framing which often makes things more realistic is to ask people about what risks they are worried about which might undermine the processes they are describing.

• Surface the assumptions: prompting people to be specific and realistic will likely lead to the surfacing of a lot of assumptions of how interventions will work and how people and organisations will react to them. Where assumptions feel particularly important, contested, or uncertain, we recommend capturing them in the diagram, either with boxes and connections or with annotations.

• Negative or no change: a powerful way of forcing realism in peoples’ thinking is to ask them to also specify ‘theories of no change’ or ‘theories of negative change’ in the diagram. In the former, we are looking for the reason why an intervention might have no effect or why a particular thread may not deliver the change we are hoping for. Taking this further, it can be useful to prompt people on potential negative effects and impacts, or unintended consequences of an intervention, and to capture these too.

• Capture feedbacks and interactions: it is quite normal for ToC diagrams to start with connections which all flow in one direction, or with ‘lanes’ of connections in parallel which don’t interact with each other. It can be a useful exercise to look for interconnections and interaction between the different pathways you have created. Similarly, asking people for feedbacks and connections which flow in the opposite direction can be a useful prompting technique for building a more nuanced and realistic ToC diagram.

• Capture disagreement: areas of disagreement are often key issues which are worth capturing in the diagram and focusing on in an evaluation. Our aim should not be to resolve these immediately but to note them and test them later.

• Use it, then iterate: it can be tempting to aim for a perfect or ‘finished’ diagram straight away. However, this is a very difficult ask. In an evaluation context, it is much more common, and useful, to think of your ToC diagram as a living document, which can and should be refined and updated as an evaluation unfolds. Once you have something usable, get out and start using it, test it, critique it, refine it. We do not discuss in detail here how to use a ToC as part of an evaluation, for guidance on this we recommend starting at https://www.betterevaluation.org/en/resources/theory-change-thinking-practice-stepwise-approach.

The exact details of your ToC diagram process will depend on many factors such as the purpose of the ToC, your preferences, the project needs, your resources, the level of stakeholder engagement, and the size of the role a ToC diagram is to play in a project. There is huge flexibility in the method, you can start with a large stakeholder workshop, or you could create a diagram in a small team and then gather feedback from stakeholders. You may want to use the diagram as more of a communication tool, as opposed to a framing or detailed description of an intervention. In this mode it is common to opt for simpler diagrams which remove some of the more nuanced steps we have described above.

Common Issues and ‘Tricks of the Trade’

There are many potential issues you may bump into with ToC diagrams, but two of the biggest are (i) the multiple pressures to create a narrow or overly simple ToC and (ii) the tendency of people (stakeholders, or clients if you have them) to treat a ToC diagram as a product rather than a process. On the first issue, there is an endless list of pressures which push you towards creating a relatively simple ToC diagram. You may not be able to speak to many stakeholders or get input from people with a variety of views. You may need to put the diagram in a report or a slide deck which means it needs to be readable at A4 size or at low resolutions. You may have teams which are responsible for an intervention that insist on keeping a narrow focus on elements close to their control. These are all powerful forces in their own right, but together can make it impossible for you to stop feeling that you must create a very simple ToC diagram. In some cases, it may be appropriate to have a simple ToC diagram, but we believe in most situations it is useful to resist these pressures if we want an effective ToC to frame, structure, and implement an evaluation or design process. You will likely need to get used to deploying the best arguments for resisting these pressures. It may also be helpful to have two versions of a diagram, the ‘real one’ and a simplified one, which you can use when you cannot resist the pressure to simplify.

On the second issue, it is common to find an over-emphasis on the ‘product’ of a ToC process, normally the diagram itself. This can come from your client or the team using the ToC, but it may be something you naturally tend towards. We are of the view that we should try to maintain a clear understanding of the development of a ToC diagram as an ongoing process, which we iterate through, and may never really ‘finish’, but rather have ‘pause-points’ for. Alongside this, it will likely be useful to cultivate an understanding with clients and stakeholders that the value of ToC diagrams, and the wider approach, is just as much in the process of developing and using it with a wide group of people, as it is with the product of the diagram itself.

To deal with these, and other, issues, the following ‘tricks of the trade’ may be useful:

• Always ask ‘how, who, how many, why?’: peoples’ mental models are almost always vague. Our job becomes helping to translate those vague mental models into specific assumptions and assertions about an intervention. Whenever people want to add a box or a connection to a diagram, we should be asking how that leads to change, who is going to be involved, how many people are involved or how much money is being spent, and why they have confidence that something leads to something else.

• Question the scale of impacts people describe: another common theme in peoples’ thinking, especially those invested in an intervention or dedicated to making change, is that they overestimate the scale of influence something they are attached to may have, and underestimate the influence of everything else going on, and/or the general inertia around persistent challenges. Where people have got specific ideas down in a diagram, we should follow up by trying to make them more realistic. One way to do this is to focus on scale. Will that series of five workshops really change the direction of an industry? Will that outreach programme really change attitudes if only the usual suspects take part?

• Use text and annotation around a diagram: ToC diagrams take numerous forms but many of the best make liberal use of annotations and text around the edges to provide nuance and context. Don’t be afraid to do the same, this will help you communicate complex ideas efficiently.

• Make workshops fun: ToC diagram categories and their very applied nature can make it feel like a bureaucratic, technical, and dry process to build them. Workshops with stakeholders work best when they are energised, relaxed, and having fun. Try to design your workshop to help foster this environment. Keep them as informal as possible, keep people on their feet, avoid setting the ‘rules of the game’ too tightly (i.e. don’t pester people on getting definitions exactly right, use your and their energy to focus on getting specific and realistic diagrams).

What Are Theory of Change Diagrams Good and Bad At?

Hopefully, through this chapter the strengths of ToC diagrams have become clear. They are excellent at representing stakeholders’ mental models of how an intervention leads to outcomes and long-term impacts. They have been very successful and popular in helping people frame and structure the evaluation and design of interventions. They help us surface and make explicit the assumptions which interventions rely on and make their logic more realistic, clear, and shareable. They help us to examine and test this logic and our assumptions by making them visible and facilitating discussion and self-reflection amongst those involved. They help communicate the ideas behind an intervention and can create the appearance, and often the reality, that we have really thought through what we are hoping to do. They can capture important context which may need to be examined as part of an evaluation and by coding or annotating the elements of the ToC can aid in evaluation planning and reveal where gaps in coverage might exist.

However, despite its popularity, the approach does have weaknesses. They can prevent us from taking a whole-system view. The focus on a sole intervention, and not the wider system, can narrow down our perspective quickly, stopping us from seeing wider contexts that are important. They can also reinforce our innate optimism; the inherent focus on the outcomes and impacts we want can blind us to potential unintended consequences, negative impacts, and the underlying inertia in systems we care about. Lastly, the focus on intervention can also exclude people who are not invested or involved in that intervention. Unlike some of the other methods in this book, it might be impossible for them to see their position in a system, or their reality, reflected in a ToC diagram, and so it may become difficult for them to engage.

A Brief History of Theory of Change

Though the ideas and some of the terminology around ToC can be traced back further, the birth of ToC is often attributed to the US-based Aspen Institute and its roundtable event in 1995. The event was organised to discuss the evaluation of community change initiatives. At the time, there was a growing understanding of two fundamental issues: (i) experimental evaluation methods were not appropriate for the large poverty reduction programmes being designed and implemented, and (ii) stakeholders in complex policy programmes were often unclear on how interventions unfolded to create the changes and impacts they sought. The event led to the publishing of a book in which Carol Weiss outlined these ideas and coined the term ‘Theory of Change’ to refer to the description of all the steps that lead to the long-term goals of interventions (Weiss, 1995). This built on the existing idea of theory-based evaluation and has become a core part of this approach to evaluation.

Since then, the approach has been extremely popular and has been adopted in many different domains, most notably in international development, philanthropy, and the third sector. It is used in all sorts of organisations, from small charities and non-governmental organisations, through to government departments and large businesses, right up to the United Nations. In its origins, the motivations to use the method in a whole-systems and complexity-appropriate manner were present (though the language of systems and complexity were not used), but these have often been lost in its application (see the pressures to create narrow ToCs we describe above), likely due to its sheer popularity and use in many different domains. In turn, and somewhat ironically, there have been many criticisms, often around ToC being too linear and not considering wider contexts (Wilkinson et al., 2021). This is now one of the main sites for innovation in the method, with efforts to make ToC more systems or complexity-appropriate coming from many places (full disclosure, this is an effort that we have contributed to). There have also been efforts to build ToC diagrams that emphasise the actions of specific actors, rather than often agency-free mechanisms (see Van Ongevalle et al., 2014), and serious efforts to reflect on the method and its wider set of challenges, both technical and conceptual (see Davies, 2018).

Getting Started with Theory of Change Mapping

Owing to its widespread use, there are many resources on getting started with ToC, and there are likely to be guides or examples relevant to the domains or disciplines you are working in; do look for these. We particularly recommend the following useful resources:

• www.theoryofchange.org: run by the Centre for Theory of Change, this site has a wealth of detailed resources and guidance for ToC. It also is the home of the ‘TOCO’ software for ToC diagrams. We have not used this software ourselves and have had mixed reviews from colleagues that have; it is not free but is worth considering as a software option.

• Logic mapping hints and tips guide (Hills, 2010): this guide was prepared for the UK Department of Transport and provides an invaluable, short, and practical guide to the individual steps within a workshop to build a ToC diagram. Note, the use of different terminology!

• Review of ToC in international development (Vogel, 2012): this review, prepared for the UK Department of International Development, is much longer than the one prepared for the UK Department of Transport, providing similar detailed guidance but also much wider conceptual discussion. International development is one of the fields in which ToC has been most used, so there is a lot of material to be learnt from here.

• UNICEF Methodological brief (Rogers, 2014): this short guide provides another perspective and is a useful complement to the two reviews above.

Hopefully, you now have most of what you need to get started using ToC diagrams yourself. We would strongly recommend looking at some of the resources above, looking for examples in similar domains and contexts to yours, and reaching out to practitioners and researchers using ToC in similar ways or areas to you. Learning from experienced ToC users is perhaps the quickest and most powerful way to learn the near-tacit skills required which is hard for us to get across in a book chapter. It tends to be easy to find examples of ToC diagrams you like, and to draw inspiration from them (you should do this!), but it is much harder to find processes that you like and that you want to emulate. Speaking to experienced ToC users will help you do this. Good luck!