1 Introduction

The impacts of weather and climate-related disasters are increasing (Handmer et al. 2012). Addressing the additional risk from climate change on the impacts of disasters will require collaboration between the governmental stakeholders involved to create policies grounded in scientific knowledge (UNISDR 2015). Effectively integrating science into policy to support such decision-making processes requires synergistic interaction between scientists and policymakers. Both groups must understand what information is needed and can be provided for decisions, along with the barriers to understanding and using that information (Ambani and Percy 2014; Duncan et al. 2014). This requires scientific research to be made more accessible to nonscientists (Ashdown 2011; Duncan et al. 2014).

Common ways of sharing complex information, such as slideshow presentations, are often passive and involve unidirectional learning and engagement (Greenblat 1988; Suarez 2015) and may be boring and dry. In contrast, interactive games can communicate complex concepts in an emotional and engaging, yet rigorous and effective, way. They have the potential to transform passive consumers of information into active players who absorb and retain new understanding, data, and tools more readily (Harteveld and Suarez 2015). This can lead to deeper learning (Suarez and Bachofen 2013) where ideas can be assimilated and applied. Games may therefore be a valuable addition to the ensemble of techniques for sharing scientific information.

This article investigates the relevance of using an interactive game to raise the visibility of probabilistic event attribution (PEA, Allen 2003), an area of climate science that has been developing rapidly over the last decade. PEA can be used to understand the effects of different climate drivers on extreme events. We assess how the game triggers reflection on the potential relevance of PEA for managing the changing risks of events, and consider specifically the United Nations Framework Convention on Climate Change (UNFCCC) loss and damage (L&D) negotiations. In doing this we aim to encourage other scientists to consider the benefits of, and engage with, different ways of communicating and working with stakeholders.

In Sect. 2 we describe the development of serious games for learning. This is followed by an introduction to the L&D policy context this work focuses on, PEA, and a discussion of why this science could be relevant for L&D. Section 3 describes the initial game development process and the game itself, and Sect. 4 discusses what players learned and how their insights informed further development. In Sect. 5 we reflect on using a game to encourage engagement with PEA in a L&D context, and conclusions are elaborated in Sect. 6.

2 Serious Games and Loss and Damage Policy

This section provides a brief introduction to the use of serious games to share information and experiences when dealing with complex problems. The game in this article was developed to share and discuss the science of PEA in the context of the UNFCCC L&D negotiations, so a discussion of the backgrounds of these science and policy contexts, and the potential links between them, is then provided.

2.1 Serious Games

Serious games are designed not just for fun but with an educational purpose (Mendler de Suarez et al. 2012) and are useful tools to simplify and clearly represent complex problems (Greenblat 1988; Juhola et al. 2013). Players make decisions, pay the consequences, and interact (Greenblat 1988), generating meaning and interpretation (Malaby 2007). Players can explore a range of scenarios and outcomes to better understanding processes and decisions (Mendler de Suarez et al. 2012). They share the learning experience with others and have the opportunity to see issues from different perspectives (Mendler de Suarez et al. 2012; Suarez and Bachofen 2013).

Games are successfully being used by nongovernmental organisations such as the Red Cross Red Crescent Climate Centre in communities and with donors, to convey changing risks and discuss disaster preparedness and humanitarian relief, among other topics (Mendler de Suarez et al. 2012; Suarez and Bachofen 2013; Suarez et al. 2014). For example, “Paying for Predictions” was designed for humanitarian workers, to encourage reflection on the value of forecasts of extreme events (Mendler de Suarez et al. 2012). There is also a growing body of academic literature on interactive climate games, including for farmers to learn about changing risks of drought (Patt 2001) and index insurance (Patt et al. 2009), to investigate river management under climate change (Valkering et al. 2012), and to focus on other climate policy contexts (Haug et al. 2011; Juhola et al. 2013).

2.2 Loss and Damage Policy and Science

The UNFCCC Warsaw International Mechanism (WIM) was established to address L&D associated with impacts of climate change (UNFCCC 2013). This includes both extreme events and slow onset events, and is focussed on developing countries that are particularly vulnerable to the adverse effects of climate change. One of its key themes in addressing L&D is “enhancing knowledge and understanding of comprehensive risk management approaches” (UNFCCC 2013, p. 6), which requires dealing with current and future climate risk changes (James et al. 2014).

There are many issues yet to be determined under the WIM. However, one that has received attention from climate scientists is whether establishing L&D from changes in climate risk will require attribution to anthropogenic climate change (Hulme 2014; James et al. 2014; Huggel et al. 2015) or whether L&D should refer to impacts from any climate-related events (as in, for example, Warner and van der Geest 2013). Links between events and climate change may be necessary if it needed demonstrating that impacts were specifically due to climate change to be considered under the WIM (James et al. 2014). Such information could also be relevant for addressing L&D in the future, by considering how events are likely to change in frequency or intensity due to anthropogenic influences in order to inform adaptation (Otto et al. 2015).

While attribution information is potentially relevant for L&D, assessing how an individual extreme event may have been influenced by anthropogenic climate change is challenging (National Academies of Sciences, Engineering, and Medicine 2016). Nonetheless, scientists are developing methodologies to robustly quantify the role of climate change, as is illustrated in the annual Bulletin of the American Meteorological Society (BAMS) supplements, which puts recent extreme events into a climate change context (for example, events of 2014, Herring et al. 2015).

A key methodology used to attribute extreme events is PEA, which estimates the change in the likelihood of an event occurring that is due to specific drivers of climate variability or change, such as anthropogenic emissions. Using large ensembles of climate model simulations, PEA has been used to establish that anthropogenic climate change likely increased the probability of events such as the 2003 European heatwave (Stott et al. 2004) and the 2013 US precipitation extremes (Knutson et al. 2014). Other studies include a demonstration of the decreased probability of snowmelt-induced flooding in Autumn/Winter 2000 in England (Kay et al. 2011), and an analysis that shows there was little evidence of any climate change influence on the 2012 low precipitation in the central United States (Rupp et al. 2013).

Attribution is controversial in the context of L&D, raising questions about blame and liability (James et al. 2014). Science-policy dialogue is therefore important to foster shared understandings and to determine whether and how PEA might be relevant for L&D (Otto et al. 2015). This is not straightforward as there are limited but conflicting perceptions of the ability to attribute extremes among L&D stakeholders (Parker et al. 2016). These perceptions include the beliefs that no link can be made between extreme events and climate change, that there are increased risks of events, and that all extremes are attributable to climate change. Climate scientists on the other hand have reached a consensus that event attribution is possible, most confidently for temperature extremes followed by precipitation extremes, subject to observational and modelling uncertainties (National Academies of Sciences, Engineering, and Medicine 2016). Uses for PEA in L&D and adaptation have also been debated by stakeholders and in the literature (Parker et al. 2016). It has been suggested attribution could help guard against inappropriate adaptation and guide resource allocation (Hoegh-Guldberg et al. 2011; Stott et al. 2013), but that it could lead to prioritizing compensation over capacity building (Hulme et al. 2011) or slow adaptation efforts (Surminski and Lopez 2014).

Given the rapid development of PEA and its controversial nature, but potential relevance, in policy contexts, there is a challenge for scientists to engage with policymakers to ensure that the research is understood and used responsibly. Scientists have a responsibility to share scientific developments so policymakers have the opportunity for informed discussions around potential uses. As multiple stakeholders across sectors are associated with addressing the impacts of extremes, we hypothesized that a game could be a good way to begin to encourage further dialogue and understanding, if facilitated skilfully so all could participate fully (Suarez et al. 2014). This article describes the process of developing that tool.

3 Towards Innovative Learning and Dialogue: The CAULDRON Game

This section describes the creation of the CAULDRON game, from the underlying concepts to the phases of the game itself: farming, science, negotiation, and reflection.

3.1 Creating the CAULDRON Game

The Climate Attribution Under Loss and Damage: Risking, Observing, Negotiating (CAULDRON) gameFootnote 1 (also described in Suarez et al. 2015) was developed to encourage understanding of the basic concept of PEA, that of changing probabilities of extremes under climate change, and consideration of whether this could play a role in international climate policy. The aim was to bridge the gap between climate science researchers and the users of results. It was designed to be played by policymakers, ideally with some understanding of climate science and an interest in attribution. The impact of climate change to alter the probabilities of extreme events can be challenging to communicate and understand, and so the game focuses on this concept, using drought as an example. The game also illustrates the role of climate models in assessing these changes and the difficulties of estimating probabilities with limited data. It should also encourage dialogue between players about whether PEA is relevant to addressing L&D.

Translating these scientific concepts into an easily understandable game was a challenge that took many iterations. It had to balance being:

  • A realistic representation of important scientific concepts.

  • Simple to understand and be played in a short space of time.

  • Able to be led by facilitators who would not need specialist training.

  • An enjoyable and memorable experience.

The game is based on concepts from other climate games designed by the Red Cross Red Crescent Climate Centre, including using rolls of dice to generate rainfall amounts and beans as currency. This simple equipment is readily available across the world, which makes the game easy to replicate in many different countries and contexts.

Players work in pairs, assigned to either a developed or developing country, and groups of pairs make up a region. There are three game phases: farming, science, and negotiation; each is followed by a period of reflection. These are illustrated in Fig. 1, along with the questions players have to consider in each phase. Drought risks change and players have to consider the role of climate change and work together to develop response strategies. While in this case the event is a drought, the game could easily be adapted and tailored to different stakeholder groups by considering other extremes such as floods, or even different intensities of events. Players take on the roles of farmers, scientists, and negotiators, with the goal that they can collect information in the first two phases and take this into their negotiations, which is the most important part for considering the uses of science. The first two phases also give policymakers an insight into the types of decisions farmers have to make and the work that scientists do. The game is intended to be engaging and fast-paced, with time limits enforced throughout to reflect real-life pressures on decision making.

Fig. 1
figure 1

Schematic of the phases of the CAULDRON game. Blue boxes describe the game phases and green boxes highlight the key questions players have to address in each phase

Full size image

3.2 Farming Phase: Planting in a Changing Climate

To begin, players take on the roles of farmers in their countries and have to make planting decisions, with the aim of accumulating as many beans as possible. They play a succession of rounds (symbolizing years) where they must choose to plant beans under either a high-risk high-yield or a low-risk low-yield strategy. The rains are determined by a specially designed “rainmaker” containing a dice (Fig. 2). These are opaque cylinders with one transparent face which allows players to see only the top face of the single contained dice. One face of the dice represents a drought, with all other outcomes representing good rains.

Fig. 2
figure 2

CAULDRON gameplay materials: Country allocation cards (farmer for developing, tractor for developed), farming matrices determining planting gains and losses (blue for developing, red for developed), beans used for planting, and rainmakers

Full size image

Each year one member of each country shakes their rainmaker (dice). Players gain beans when there are good rains and lose beans when a drought occurs, with greater losses in developing countries. They must try to avoid crises, where they do not have enough beans to plant, and can negotiate with other players in their region to be given beans in the form of loans or aid. Facilitators for each region help ensure players shake their rainmakers fairly and lose or gain the correct number of beans each year. For the first few rounds, all rainmakers have a 1 in 6 chance of drought (although players do not know this). Then players are informed that the climate has been changing due to greenhouse gas emissions (chiefly from the developed countries), which may impact the probability of drought occurring in their country. Each country receives a new rainmaker and continues playing, uncertain of whether and to what degree climate change has impacted their drought probability. In reality, some of the new rainmakers now contain dice with two faces that represent drought, doubling the probability. This provides a very simplified illustration of an impact that, in real life, would occur gradually.

The analogy of changing probabilities of events being like loading dice towards or away from the chances of rolling a six is one often used in PEA science. It demonstrates that although the probability of getting a six may have been increased by climate change, a six could have been rolled anyway.

3.3 Science Phase: Assessing Changes in Extremes

For the next phase, players become climate scientists, trying to work out whether climate change increased their country’s probability of drought. They can use their rainfall observations from the farming phase to compare the number of droughts they experienced under normal and climate change conditions. These do not provide a very long time series of observations, reflecting what is often available in reality, especially for developing countries. Players can also use “climate model” data: they are given a third rainmaker, a model of the conditions under climate change, and they can run this a restricted number of times, reflecting real-life available computing power. In practice the “model” rainmaker is identical to the one used in the farming years under climate change, but participants are informed that it may not be a perfect model, a challenge with which climate scientists also have to deal. Players decide in their pairs whether they think the probability of drought in their country has increased or is unchanged and then report their assessment, as well as how confident they are.

3.4 Negotiation Phase: Addressing Loss and Damage

The final phase of the game is the negotiation. Players assume the roles of policymakers for their countries and negotiate in their region how to address the losses that have occurred due to droughts, particularly considering where they think there have been increased drought risks due to climate change. They can use or disregard any of the information from the other phases. They decide whether and how to use their beans to assist countries that have suffered losses and are asked to write a plausible agreement signed by all players in the region before the end of a strict countdown, to reflect L&D negotiations under the UNFCCC.

3.5 Reflection

During CAULDRON there are reflection times after each phase where players consider and discuss with others what they have experienced. There is also a short final debriefing phase, where players can open their rainmakers to see all the faces of the dice, and therefore reveal the probability of drought. They are encouraged to discuss how this probability relates to the conditions they experienced, and consider whether the observational and model data were useful to understand whether their region experienced a change in probability. They can also reflect on whether their understanding of whether their drought risk had changed was useful in their negotiations.

4 Codevelopment of the CAULDRON Game: Dialogue Begins

The development of CAULDRON has been guided by feedback from players, used to see whether it was achieving its aims and for suggestions on how to improve the game. The main sessions from which feedback was collected are summarized in Table 1. An initial prototype version was played as part of the Africa Climate Conference 2013 and the game was then played in Warsaw during COP19. Six additional sessions, with between 14 and 44 participants in each, chiefly with climate scientists and students from a range of backgrounds on courses tailored to enhance learning on complex environmental and development issues, were then carried out (Table 1). While these were not the key intended players of the game, they were able to provide insightful comments to guide its development. CAULDRON aims to encourage learning and reflection for players regardless of their previous knowledge of probabilities, climate change, and PEA itself. The scientists had some knowledge of PEA and were able to provide informed insights on the presentation of the science in the game. Feedback from students was valuable despite their different levels of prior knowledge compared to policymakers. Information and feedback from all these sessions was accumulated and fed into the game. Following this it was played with policymakers in Senegal as part of a workshop focussing on PEA science and addressing loss and damage, which involved 40 government, civil society, and scientist stakeholders.

Table 1 Details of sessions where CAULDRON has been played with the number of debrief sheets and questionnaires collected for analysis
Full size table

Feedback was collected via comments players made on debrief sheets where possible, along with the negotiated texts. In some cases, players completed a questionnaire immediately after the game about their prior understanding of PEA and L&D, their main insights from playing, and whether they thought PEA could be used in L&D. A further short survey was sent to some players in the months following to learn about longer-term influences of playing the game but this had limited response (n = 7). Game facilitators also made systematic observations of sessions. After the game in Senegal, players reflected in groups on what they felt were the key learning points. Comments from the feedback sheets and sessions are used here to illustrate some of the key themes that emerged.

4.1 Learning About PEA

One of the key aims of CAULDRON was that players would learn about the science of attributing extremes to climate change and the probabilistic nature of this. How players learned will have depended on their previous understanding of probabilities, and also their perceptions of climate change and how this is affecting extremes such as droughts. The goal was that players with any level of scientific understanding would be able to benefit from experiencing changing probabilities and reflect on how easy it was to tell if there had been a change.

Players commented that using dice to highlight climate changes was helpful, as one said, “Even though the knowledge was already there, the concept of ‘chance’ and ‘probability’ that is very important became clearer through the experience gained over playing the game.” How players chose to assess whether they had a change in drought probability was up to them. Players with more scientific backgrounds often worked out how many droughts they would have expected to see with a normal dice. Many players noted the difficulty in assessing the changes in probability with such limited data, with players in Senegal commenting that the short timescales made it difficult to see whether there was a change in drought likelihood or it was just variability. This is also a challenge faced by scientists carrying out PEA studies. Another player commented that such analysis could be especially challenging for developing countries. This is particularly the case for those lacking reliable long-term observations and the scientific capacity for analysis of changes. Reflecting on the link to real-life challenges, players in Senegal recognized how the model results could differ from the observations, and that further work was needed to understand how best to use both model and observational data.

The negotiation phase was also useful for seeing how players viewed the science. The views of players with previous knowledge of the science will of course not just be based on the game, but this still provides an opportunity for them to apply their understanding. For example, one region decided not to use attribution information as they felt it was too uncertain, saying “There is still more work to be done on the attribution science to provide evidence for country-specific climate change impacts.” In the game context data are limited and models may be imperfect. Real-world PEA also faces these challenges, but with additional difficulties including framing scientific questions and defining the event to attribute (Otto et al. 2015). These are not incorporated into CAULDRON as the game would become too complex. There are also different methodological approaches being developed to overcome these difficulties (Stott et al. 2016), which are also left out of the game for simplicity. Another player noted that “distinguishing damages from chance, (climate) change, risk taking is tough,” highlighting that L&D can be affected by vulnerability and exposure, which require consideration alongside the meteorological hazard (Huggel et al. 2013).

Most players began the game with no knowledge of PEA (except the Met Office group). After playing, the majority of players reported in questionnaires that their knowledge of PEA had improved (73% slightly improved, 17% greatly improved). We have not systematically analyzed players’ understandings of PEA, but the few responses to the follow-up survey showed very varied understandings. This very small sample included views that the chance of extreme events could be affected by climate change, events could be made more extreme due to climate change, that most extreme events are attributable to climate change, and that it is all random. It would therefore be interesting to analyze more systematically what players understood before and after playing the game, as their own perceptions of how much they learnt may not match whether they have a correct understanding.

4.2 Improving Learning About PEA in CAULDRON in the Future

The Met Office group suggested ways to improve how CAULDRON represents PEA science, including having more consideration of uncertainties in the modeling part. Players often found differences between the observational and model data and were uncertain which to trust. It was suggested that models could also be provided of the unchanged climate to assess the model skill, and that models should not imply that they are perfect representations of the real world.

However there is a careful balance to be drawn between keeping the game relatively simple and easily understandable in many contexts, and incorporating all of the complexities of PEA, which CAULDRON is obviously unable to portray. This requires the game facilitator to have the skill to judge the needs and understandings of the players and lead and tailor the game accordingly. The game documents have been provided so that anyone can learn to run CAULDRON; the skills required for the game to have maximum benefit for participants by encouraging engagement, reflection, and learning between players from different backgrounds (Mender de Suarez et al. 2012) can be more challenging to develop. However by working with more experienced game facilitators and colleagues with skills in different areas, such as experts in PEA science, others can develop these necessary skills through experience and participation.

It is also necessary to consider how the game can be used alongside other activities so players are able to gain a more complete understanding of the science on which the game is based. For example, before a couple of the sessions players were given a more traditional presentation on PEA as this could provide more detailed background.

Work has begun to compare how well players learn during games compared to more traditional methods such as slideshow presentations (Patt et al. 2010), but more long-term monitoring will be needed for the greater impact of games to be assessed (Mendler de Suarez et al. 2012; Harteveld and Suarez 2015). This can be challenging and requires systematic assessment. Haug et al. (2011) suggest that to collect large enough samples of robust data, evaluation could be embedded into games so it is not seen as a time-consuming extra for players. This could be incorporated as CAULDRON develops. We have not included surveys before playing as this is time-consuming and may put players off. We have instead tried to ensure that players with any level of previous understanding of probabilities can build on this during play—how they do this and the insights from the decisions they make can then be discussed in the debrief.

Nevertheless it would also be interesting to collect quantitative data on players’ understandings. Questioning why players made particular decisions, and short, but in-depth, surveys of players’ understandings could improve evaluation. While this area has not been our focus in the development of the game so far, questions that could be investigated, for example focussing on decision making in the farming phase, include: What strategies do players use for planting? What do players do when they experience a drought? Who helps who when drought occurs?

4.3 Promotion of Dialogue About Roles for PEA in L&D

The second key objective of the game was that players would have the opportunity to consider whether PEA has a role in addressing L&D, as this has been debated by academics. An earlier version of CAULDRON had an unstructured negotiation phase where players could discuss however they wished. Players tended mainly to address how they would respond to future losses without considering the science of whether drought risks had changed, and whether losses could be attributed to climate change. While an interesting finding, an aim of CAULDRON was to encourage consideration of uses for PEA, even if it was not then used, so a suggested structure for the negotiations was introduced. Players were encouraged (although they could choose not to do so) to consider how many (if any) of the beans they lost could be attributed to climate change, who was responsible, and then how to address this state of affairs.

Of those who filled in questionnaires following playing, most had no, or very little, knowledge of L&D or UNFCCC negotiations before playing (Development and Climate Days players were likely more knowledgeable). For these players, CAULDRON may have been a useful tool to provide a brief introduction to the L&D negotiations, as well as to provide the opportunity to consider how PEA could be used. This may not be discussed in the real policy world. One player said the negotiation phase helped them improve their “understanding of how the science can be applied” and another “how the understanding of climate risk can be used as a negotiating implement.”

In the negotiation phase some groups did consider what losses could be attributed to climate change and produced deals to compensate these. Others decided against using scientific information, instead focussing on addressing past or future losses regardless of their causes. One player commented that attribution would lead to blame so could be useful for forcing an outcome, but was not important if developed countries chose to support developing countries. Some players did not think their drought probability had changed and therefore did not attribute losses to climate change; in other cases losses were attributed to players’ planting strategies.

Negotiations could be difficult, and from participant observations this was often one of the main messages players carried away from the game experience. They reported that some countries tried to pressure others into agreements, and it was difficult for countries in different circumstances and with different perspectives to agree. Often a deal depended on developed countries taking responsibility, as they generally held the power in negotiations. Players noted that “negotiations are hard because they are not determined only by science but by other factors as well (political etc.)” and participants also commented on the “difficulty of reaching a negotiated deal within a deadline, when working with incomplete and very uncertain information.”

Naturally, many players remarked on the similarities between their own behavior and the patterns that emerge from UNFCCC negotiations. In some circumstances, negotiations could also be unrealistic and often led to simplistic solutions. These exaggerated fair distributions of resources, including common resource pools of beans for the future, redistributions of wealth making countries more equally wealthy, and plans to donate beans in cases of future crises, which are not commonly seen in international negotiations. The negotiated texts from Senegal (Table 2) were much more detailed and less simplistic than others. But players still reflected that countries were willing to help each other in times of crisis more readily than in reality. CAULDRON could perhaps be improved by introducing greater political bias between countries to encourage less “fairness,” as has been suggested by players.

Table 2 Key points from negotiated texts from CAULDRON session in Senegal
Full size table

From the questionnaires, the majority of players thought PEA could be used in addressing L&D in real life, despite many not explicitly using it in the game. Reasons for its use were not explained in detail, but included preparing for future events, demonstrating climate change impacts, distinguishing anthropogenic and natural causes of events, and because the effects of climate change are often caused by different actors than those who are affected. Some climate scientists and other players were concerned that PEA should only be used if robust enough, showing awareness of the uncertainties. Others disputed using PEA in L&D for reasons that included difficulties distinguishing if an event is attributable to climate change, limited data, the time needed to calculate results, and because the use of PEA could encourage a focus on blame rather than on reducing losses in developing countries. Other suggested uses for PEA included in more general policy negotiations, risk analysis, insurance sector policies, investment planning, adaptation, and improved regional projections. Senegal stakeholders were interested in how PEA results could be implemented at national and local levels. These views suggest that players had a chance to consider some of the issues surrounding using PEA in policy.

5 The Interplay Between Science and Policy

Our experiences in developing the CAULDRON game and playing it in a range of contexts have given us the opportunity to reflect on both the key benefits that gameplay can provide for the sharing of information and the challenges that come with such work. These are discussed in this section, followed by a reflection on the policy impact that CAULDRON has had, and could have in the future.

5.1 Benefits of Using a Game

CAULDRON is a tool that promotes experiential learning, as players have to make decisions, such as farming strategies, and address the consequences. One player described how “extreme events had ‘real’ consequences and a political ‘reality’.” Players learnt first-hand “about careful decision making in cases of high unpredictability,” which is vital when considering climate risks. It was observed in a session that players seemed confused when the climate change rainmakers were introduced, but this perplexity was quickly followed by realization that the change had occurred and the discovery sparked animated discussion. One of the benefits of using a game is this rapid learning through experience of how probabilities can be affected by climate change.

Another important part of the game experience is that players have to interact with each other. Discussion is often not possible in a traditional presentation context, or may lack engagement from participants. CAULDRON is designed so discussion between players is vital if they are to succeed, from sharing beans during farming to reaching an agreement to address L&D. Useful parts of the game mentioned by players included “trying to understand other groups’ point of view to come to an agreement” in the negotiation phase.

We have observed that CAULDRON is able to create an engaging learning experience for players. Players become animated and engaged during the game sessions (Fig. 3). A report on the Development and Climate Days session said participants “appeared deeply engaged in the game, as they could be seen jumping to their feet and running to the front of the room to get in their mandates on time” (IISD 2013). This was helped by facilitators encouraging players to stand up and shout “Oh no!” whenever they experienced a drought, and race to get their negotiated texts completed. This, along with the noise from the shaking of the rainmakers, created a stimulating learning environment.

Fig. 3
figure 3

CAULDRON sessions in action. Clockwise from top left: a exchanging beans during farming, b a spokesperson reading out his region’s negotiated text, c discussions during play, d the shock of drought during farming, e shaking a rainmaker, f planting beans during farming, g presenting negotiated texts. Photographs a, c, d, e, g courtesy of IISD/ENB, November 2013 (http://www.iisd.ca/climate/cop19/dcd/); b by Emily Boyd, February 2015; f by Hannah Parker, September 2014

Full size image

Players were also able to learn about many elements of the game context at once. Whereas a traditional presentation tends to focus on one aspect, such as PEA science, by playing CAULDRON people also reported learning about social and political issues. These included differences in resiliency to extreme events between developed and developing countries, and that “climate change reinforces existing inequalities” between rich and poor.

Games have most learning impact when followed by a debriefing and discussion session about insights and how the game relates to reality (Suarez and Bachofen 2013; Macklin 2014). In CAULDRON, the debriefings between phases and at the end give players the opportunity to reflect. Reflection was identified as a vital part of the learning process over a century ago (Dewey 1910), yet traditional presentations can leave little opportunity for this meditative outcome. In the game, players can consider what they have learnt and discuss what they are feeling and thinking. The emotional aspect of a game can also make learning a memorable experience. Players reported a range of emotions (Fig. 4) and it is hoped that they will remember the game and further consider the issues that arose during and after play.

Fig. 4
figure 4

Word cloud of the most common words used when describing emotions and feelings during the playing of CAULDRON. The size of the words correspond to their relative frequency (Created using QSR International’s NVivo 10 software)

Full size image

5.2 Challenges of Using a Game

Despite the benefits, we recognize that a game is not capable of presenting all of the complexities of PEA science that would be possible in a more traditional presentation format. CAULDRON is a tool that can be used in conjunction with other methods to share knowledge about PEA science in complementary ways. It is a greatly simplified version of reality and therefore a challenge to include the right level of complexity. If too simplified, understanding may be limited. Yet the game must be simple enough that players do not get too confused and can pick up the rules and aims quickly. Along with the game design, this balancing act during play relies on skilled facilitation. Facilitators need to know their audience to lead the game accordingly and ensure an atmosphere where players feel sufficiently comfortable to engage with the game.

Figure 4 shows that players reported feeling both positive and negative emotions during play. Some said they felt rushed. Time pressure is an important part of the game as this reflects reality. We have tried in more recent sessions to ensure players have enough time to understand what is going on so they can fully engage and gain maximum benefit from the game environment. Although a game should not be an entirely negative experience for players, some stresses and frustrations can be important (for example, if the farming phase yields poor results or negotiation is difficult). The disparities between players’ experiences are important for discussion in reflection times so players can relate these variations to real-life issues. Providing time for individual and group reflection following play is a key component of CAULDRON. Reflection can provide players with insight into how the game was similar to, yet different from, reality, and how it related to their own experiences. It is at this stage that genuine dialogue between players, particularly those involved in policymaking, may be able to take place.

5.3 Policy Impact

CAULDRON had a key part in the workshop held in Senegal to engage with national policymakers addressing the impacts of extreme events. The informal discussion atmosphere created by the game, encouraging engagement and working together, was able to filter through the rest of the workshop. This led to meaningful and productive discussions about the relevance of PEA for the decisions they have to make in real life.

Some evidence has been found that games can impact policy by strengthening dialogue (Bachofen et al. 2012). However, more work is needed to establish if and how games can lead to changes in policy, and if they are better than other methods. This will require more thorough baseline studies of knowledge and policy contexts, along with improved evaluation and review following game-play (Bachofen et al. 2012). Nonetheless, one of the key outcomes of CAULDRON is that we as game designers and scientists have begun to better understand some of the complexities of the policy situation surrounding L&D through our engagement with stakeholders in this area. We have become much more aware of the controversial nature of L&D and of the limited role that attribution science can play in these negotiations. While we initially designed the game with the ambition of playing with UN delegates involved in L&D negotiations, except for one session with UNFCCC delegates from one country using an early prototype, we have not played CAULDRON with negotiators as it was deemed too controversial for them to discuss. This may be in part due to scientific evidence not being perceived as the highest priority issue in negotiation bottlenecks. This also may be due to a game being perceived as a nonserious way of learning on which delegates cannot justify spending time, or because they believe they already understand enough about the issues the game addresses.

In other contexts, renaming a game a role-playing activity or simulation has resulted in more engagement from target players (Maenzanize and Braman 2012), so this repackaging could help attract target audience attention. Also, the format and aims of CAULDRON may not be ideally suited to supporting L&D negotiations. Amending the game so that it is less controversial may make it more appealing to a wider range of audiences. The negotiation phase is the main controversial element due to its reference to L&D negotiations, which themselves have been controversial over the past few years, particularly due to their association with liability and compensation (James et al. 2014). It might be possible to have a more open discussion in the future by playing only the farming and science phases and then prompting stakeholders to reflect on the relevance of the science for their work, without explicit mention of L&D or UNFCCC negotiations.

6 Conclusions

In this article we have showcased the development of a participatory game in order to encourage other scientists to see the value in engaging with such tools. We have demonstrated that CAULDRON is able to facilitate both the interpretation of probabilistic climate information and the consideration of its relevance to informing policy in both the game and real life. The experiential nature of the game means players must engage with the concepts, make decisions, interact, and discuss and reflect with other players. Different actors have different learning experiences playing CAULDRON depending on their background, previous knowledge, and how they like to engage with this learning format. CAULDRON could therefore be a useful tool for mutually beneficial science-policy dialogue, as scientists may be able to understand more about the types of decisions being made, and policymakers can gain an introductory, although highly idealized, understanding of PEA to consider whether it could inform decision making. This is vital for the effective consideration and integration of PEA into policy, and could even begin to inform the development of this area of science so research outputs are relevant to policy issues.

CAULDRON also has the potential, with skilled facilitation, to be used with a range of stakeholders at different levels of governance, where there are challenges due to the context and stakeholder power and hierarchy issues (Suarez et al. 2014). Developing participatory games such as CAULDRON could be a way forward that engages multistakeholder groups in discussion around complex issues, such as the management of disaster risks. They may be able to help create a level playing field and lead to more genuine dialogue, as players share the game experience and can use this as a basis for discussing real-life issues. There are many problems that can emerge in game-enabled processes, including inadequate simplification of real-world complexity if not considered carefully, unskilled facilitation, ethical dimensions of authority, and cultural diversity. There are also ways to address these risks to ensure the game-playing experience is beneficial for all involved (Mendler de Suarez et al. 2012).

We are continuing to engage with stakeholders to better understand the decision-making contexts to which PEA could be relevant, including L&D as this policy area develops. Developing and using a game has made us step out of our comfort zones, but has prompted many interesting conversations with players about the scientific and political contexts. We hope this will encourage other scientists to consider innovative ways of engaging with stakeholders, in order for effective dialogue to take place and the links between science and policy to be discussed productively.