Bayesian Model-Based Approach: Impact on Science and Policy

Abstract

In this chapter, we summarise the scientific and policy implications of the Bayesian model-based approach, starting from an evaluation of its possible advantages, limitations, and potential to influence further scientific developments, policy and practice. We focus here specifically on the role of limits of knowledge and reducible (epistemic), as well as irreducible (aleatory) uncertainty. To that end, we also reflect on the scientific risk-benefit trade-offs of applying the proposed approaches. We discuss the usefulness of proposed methods for policy, exploring a variety of uses, from scenario analysis, to foresight studies, stress testing and early warnings, as well as contingency planning, illustrated with examples generated by the Risk and Rumours models presented earlier in this book. We conclude the chapter by providing several practical recommendations for the potential users of our approach, including a blueprint for producing and assessing the impact of policy interventions in various parts of the social system being modelled.

In this chapter, we summarise the scientific and policy implications of the Bayesian model-based approach, starting from an evaluation of its possible advantages, limitations, and potential to influence further scientific developments, policy and practice. We focus here specifically on the role of limits of knowledge and reducible (epistemic), as well as irreducible (aleatory) uncertainty. To that end, we also reflect on the scientific risk-benefit trade-offs of applying the proposed approaches. We discuss the usefulness of proposed methods for policy, exploring a variety of uses, from scenario analysis, to foresight studies, stress testing and early warnings, as well as contingency planning, illustrated with examples generated by the Risk and Rumours models presented earlier in this book. We conclude the chapter by providing several practical recommendations for the potential users of our approach, including a blueprint for producing and assessing the impact of policy interventions in various parts of the social system being modelled.

1 Bayesian Model-Based Migration Studies: Evaluation and Perspectives

Following the Bayesian model-based approach in the context of modelling a route network of asylum migration has led to some specific scientific conclusions, reported in Chap. 8, but equally has left several gaps remaining and open for further enquiry. In this section, we look at the contributions in the areas of modelling, data evaluation, psychological experiments, and computing and language development, and the perspectives for enhancing them through more research in specific domains.

In substantive terms, our modelling work suggests that the migrant journey itself – which has received only sparse treatment in migration literature so far – is an important part of migration processes. We were able to show that the dynamics of the uptake and transfer of information by migrants strongly affects the emergence of migration routes. Based on this work, we can also pose specific empirical questions concerning migration itself, but also with respect to human behaviour more generally, that will substantially improve our ability to model and understand social systems. At the same time, we can utilise different types of data (micro and macro, quantitative and qualitative, contextual and process-related) in a way that explicitly recognises their quality and describes uncertainty to be included in the models. This is especially important given the paucity of data on such complex processes as migration: here, a formal audit of data quality, as presented in Chap. 4, is a natural starting point.

Still, large gaps in available empirical knowledge of migration remain, which makes any kind of formal modelling challenging. For one, data on many processes that are known to be important are missing or sparse, especially at individual level. Even with a case study such as the recent Syrian asylum migration, there are parts of the process with little or no data, and the data that exist rarely measure specifically what the modellers may want them to. The challenge is to identify and describe the limitations of the data while also identifying how and where they may be useful in the model, and to make consistent comparisons across a wide range of data sources, with a clearly set out audit framework.

More fundamentally, however, we often do not even know which of the possible underlying processes occur in reality, and even if they do, how they affect migration. Besides, human behaviour is intrinsically hard to model, and not well understood in all the detail. Finally, the combination of a large spatially distributed system with the fact that imperfect spatial knowledge is a key part of the system dynamics leads to some technical challenges, due to the sheer size of the problem being modelled.

One key piece of new knowledge generated from the psychological experiments thus far is that migration decision making deviates from the rationality assumptions often used. We found that people exhibit loss aversion when making migration decisions (they weight losses more heavily than gains of the same magnitude), as well as that people show diminished sensitivity for gains in monthly income (i.e., they are less responsive to potential gains as they get further from their current income level). We have also found that people differentially weight information about the safety of a migration journey depending on the source of the information. Specifically, this information seems to be weighted most strongly when it comes from an official organisation, while the second most influential source of information seems to be other migrants with relevant personal experience.

When conducting cognitive experiments and adding greater psychological realism to agent-based models of migration, several important obstacles remain. One key challenge is how to simulate complex real-world environments within the confines of an online or lab-based experiment. Migration decisions have the potential to change one’s life to a very large extent, be associated with considerable upheaval, and, in the case of asylum migration, occur in life-threatening circumstances. For ethical reasons, no lab-based or online experiment can come close to replicating the real-world stakes or magnitude of these decisions. This is a major challenge for both designing migration decision-making experiments and for applying existing insights from the decision-making literature to migration. Another important challenge is that migration decisions are highly context dependent and influenced by a huge number of factors. Therefore, even if it were possible to gain insight into specific aspects of migration decision making, important challenges would remain: establishing the extent to which these insights are applicable across migration decision-making contexts, and understanding and/or making reasonable assumptions about how various factors interact.

In terms of computation, the languages we developed show that the benefits of domain-specific modelling languages (e.g., separation of model and simulation, easy to implement continuous time), that are already known in other applications domains (such as cell biology), can also apply to agent-based models in the social sciences. The models gradually developed and refined in this project, and other models of social processes intended to give a better understanding of the dynamic resulting from individual behaviour, have a strong emphasis on the agents’ knowledge and decision making.

However, modelling knowledge, valuation of new information, and decision making requires much more flexible and powerful modelling languages than the ones typically used in other areas. For example, we found that the modelling language needs to support complex data structures to represent knowledge. As the resulting language would share many features of general-purpose programming languages, it should be embedded into such a general-purpose language, rather than be implemented as an external domain-specific language.

In addition, our parallel implementation of the core model in two different programming languages demonstrated the value of independent validation of simulation code. To understand and evaluate a simulation model, it is not enough to know how it works; it is also necessary to know why it is designed that way. Provenance models can supplement (or partially replace) existing model documentation standards (such as the ODD or ODD+D protocols, the ‘+D’ in the latter referring to Decisions, Müller et al., 2013; Grimm et al., 2020; see also Chap. 7), showing the history and the foundations of a simulation model. This is especially pertinent for those models, such as ours, which are to be constructed in an iterative manner, by following the inductive model-based approach.

At the same time, the key language design challenge for this kind of models seems to be finding a way to design the language in such a way that it is:

• powerful and flexible enough;

• easy to use, easy to learn and (perhaps most importantly) easy to read; and

• possible to execute efficiently.

For the provenance models, a key challenge is to identify the entities and processes that need to be included, and the relevant meta-information about them. Some of this is common to all simulation studies, independent of the modelling method or the application domain. At the same time, other aspects are application-specific (e.g., certain kinds of data are specific to demography, or to migration studies, and some information specific to these types of data is relevant). This meta-information can be gathered with the help existing documentation standards, such as ODD, which additionally underscores the need for a comprehensive data and data quality audit, as outlined in Chap. 4.

2 Advancing the Model-Based Agenda Across Scientific Disciplines

This section includes additional contributions by participants of the workshop “Modelling Migration and Decisions”, Southampton, 21 January 2020. Many thanks go to André Grow, Katarzyna Jaśko, Elzemiek Kortlever, Eric Silverman, and Sarah Wise for providing the voices in the discussion.

Based on the experience with interdisciplinary model development, and building on the list of outstanding challenges identified in the previous section, we can make some tentative predictions on how model-based approaches and their components may develop in the future.

In terms of migration modelling as such, the further developments are likely to happen in a number of key areas. At this point any modelling effort is necessarily limited by the availability of empirical knowledge in the most general sense – data and other information alike. This means that models have to be either purely conceptual, exploring generic dynamics of the system without specific relation to a concrete real-world scenario, or great effort has to be invested into correctly identifying the uncertainty of model results. However, it is worth noting that statistical models, such as those from the Bayesian uncertainty quantification toolbox, can help shed light even on the behaviour of purely conceptual or theoretical models, without any empirical data, through uncertainty and sensitivity analysis.

The analysis of model results does not at present rely on a standard toolkit of approaches, but on the various methods of uncertainty quantification and emulation, such as those presented in Chap. 5, all of which offer substantial promise. The exploration of the model space can additionally involve tools of artificial intelligence, such as neural networks, especially when the more traditional methods, such as GP emulators, do not work very well, for example in the presence of tipping points or phase transitions between different model regimes. Here, more work needs to be carried out on comparing the results, applicability, and trade-offs of using different meta-models for analysis.

A large part of future progress in modelling migration – or other social systems – depends therefore on improvements in our empirical understanding of the processes under study. Methodologically, it seems promising to try to better understand how the empirical uncertainty in the data and other information leads to uncertainty in modelling results. More fundamentally, we do not have at this point a good understanding of the limits as well as the potential of modelling social phenomena in general. This is an area that will hopefully see increased activity in the future.

When it comes to data, a more tailored application of empirical information to different settings and scenarios is needed, with different uses in mind. Recognition that different data sources are more or less important or useful depends on what is being modelled, and on the research questions or policy objectives of users. Data inventories and formal quality assessments offer a starting point, informing the modellers and users what information is available, but also – perhaps even more importantly – which knowledge gaps remain. At the moment, there is still untapped potential with using digital trace data, for example from mobile phones or social media, to inform modelling. Of course, such data would need to come not only with proper ethical safeguards, but also with knowledge of what they actually represent, and an honest acknowledgement of their limitations.

As the data inventory grows and the quality assessment framework is applied to different settings, the criteria for comparison may be applicable more consistently. For example, it is easier to assess the relative quality of a particular type of source if a similar source has already been assessed. On the whole, the data assessment tools may also be used to identify additional gaps in available data, by helping decide which data would be appropriate for the purpose and of sufficient quality, and therefore can inform targeted future data collection. The quality assessment framework can also encourage the application of rigorous methods of data collection and processing before its publication, in line with the principles of open science.

Besides any statistical analysis, the use of empirical data in modelling can involve face validity tests of the individual model output trajectories, which would confirm the viability of individual-level assumptions. This approach would provide confirmation, rather than validation, of the model workings, and that the process of identifying data gaps and requirements could be iterative. At a more general level, having specific principles and guidelines for using different types of individual data sources in modelling endeavours would be helpful – in particular, it would directly feed into the provenance description of the formal relationships within the model, in a modular fashion. There is a need for introducing minimum reporting requirements for documentation, noting that the provenance models discussed in Chap. 7 are in fact complementary, rather than competing with narrative-based approaches, such as the ODD(+D) protocols (Müller et al., 2013; Grimm et al., 2020).

With cognitive experiments for modelling, one key area for future advancement is the development of experimental setups that reduce the gap between experiments and the real-world situations they are attempting to investigate. The more immersive and interactive experiment suggested in Chap. 6 would attempt to advance experimental work on decision making in this direction, and we expect that future work will continue to develop along these lines. Additionally, it will be crucial for future experimental work to examine the interplay of multiple factors that influence migration decisions simultaneously, rather than focusing on individual factors one at a time.

As also mentioned in Chap. 6, another key challenge is how to map the data from the experimental population to a specific population of interest, such as migrants, including asylum seekers or refugees. The external validity of the experiments, and their capacity for generalisation, is especially important given the cultural and socio-economic differences between experiment participants. One promising possibility, subject to ethical considerations, consists in ‘dual track’ experimentation on different populations at the same time, to try to estimate the biases involved. This could be done, for example, via social media, targeting the groups of interest, and comparing the demographic profiles with the samples collected by using traditional methods.

Furthermore, necessary psychological input on the structures of decision making to be used in the modelling process can be offered by formal description frameworks, such as the belief-desire-intention (BDI) model of Rao and Georgeff (1991), augmented by additional formal models for memory, information exchange, and so on. For migration and similar problems (mobility, relocations, evacuations…), modelling the decision processes for ‘stayers’ can be as important as for ‘movers’, and thus the information on perceived needs and expectations of both groups is key.

In addition, more detailed theoretical work and structured analysis of the already existing literature are also expected to play a key role in improving our knowledge of complex migration decision making. There is a strong need to combine and integrate existing findings from a range of application areas and scientific disciplines, in order to form a more cohesive understanding of the individual and combined impact of various factors on migration decision making (Czaika et al., 2021), and enhance our overall comprehension of the processes involved.

Finally, in computational terms, while we can demonstrate the advantages of the developed domain-specific language, it is hardly possible to create a generic tool that can be readily used by a wider modelling community within a range of large projects, like the one presented throughout this book. Preparing tools, documentation, teaching of the language, and so on, are all very long-term, community-based efforts. One approach to make the developed methods more available for a wider group of users could be to try to include them (or parts of them) into existing tools for agent-based modelling, such as NetLogo, Repast, or Mesa, for example in a form of add-ons for such tools.

As for the practicalities of modelling, one important feature of domain-specific languages is that, despite their being to some extent restricted by construction, they enable the separation of the model logic – the formal description of the model and the underlying processes – from the logic of the programming language. Internal domain-specific languages, embedded as libraries in well-known general-purpose languages, such as Julia, Java or Python, offer a sound compromise solution.

In terms of provenance, future work could lie in automating the provenance modelling in order to aid the modellers in the process. Creating a detailed provenance model, while valuable, can be a demanding and very time-consuming endeavour. To overcome that, provenance information could be, for example, already annotated in the model code, with references to the theory or data sources underling a specific model component, and a provenance model (or at least a part of it) could then be automatically constructed from those annotations.

At a more general level, there are some important implications of our approach for the art and science of modelling. First, while different models can serve different purposes (Epstein, 2008), they are very useful for expanding the imagination of modellers and users alike and for framing the conversation around the processes and systems they are trying to represent. The act of formal modelling forces the assumptions, concepts, and outcome measures to be made and operationalised explicitly, which is already an important step in the direction of fuller transparency and more robust science.

Second, no canonical modelling approaches for social processes exist, or can exist, given the complex and context-dependent nature of many aspects of the social realm. Still, having a catalogue of models, and possibly their individual sub-modules, can offer future modellers a very helpful toolbox for describing and explaining the mechanisms being modelled. At the same time, the modellers need to be clear about the model epistemology and limitations, and it is best when a model serves to describe one, well-defined phenomenon. In this way, models can serve as a way to formalise and embody the “theories of the middle range”, a term originally coined by Merton (1949) to denote “partial explanation of phenomena … through identification of core causal mechanisms” (Hedström & Udehn, 2011), and further codified within the wider Analytical Sociology research programme (Hedström & Swedberg, 1998; Hedström, 2005; Hedström & Ylikoski, 2010).¹ In this way, the modelling gives up on the unrealistic aspiration of offering grand theories of social phenomena. This in turn enables the modellers to focus on answering the research questions at the ‘right’ level of analysis, which choice may well be a pragmatic and empirical one.

Third, the pragmatic considerations around how to carry out model-based migration enquiries in practice are often difficult and idiosyncratic, but this can be partially overcome by identifying examples of existing good practice and greater precision about the type of research questions such models can answer. At the same time, there is acute need for being mindful of the epistemological limitations of various modelling approaches. A related issue of how to make any modelling exercises suitable and attractive for users and policy-makers additionally requires a careful managing of expectations, to highlight the novelty and potential of the proposed modelling approaches, while making sure that what is offered remains realistic and can be actually delivered.

One important remaining research challenge, where we envisage the concentration of more work in the coming years, is how to combine the different constituting elements of the modelling process together. Here again, having agreed guidelines and examples of good practice would be helpful, both for the research community and the users. In terms of the quality of input data and other information sources, there is a need to be explicit about what various sources of information can tell us, as well as about the quality aspects – and here, explicit modelling of the model provenance can help, as argued in Chap. 7 (see, in particular, Fig. 7.3).

In future endeavours, for multi-component modelling to succeed, establishing and retaining open channels for conversation and collaboration across different scientific disciplines is crucial, despite natural constraints in terms of publication and conference ‘silos’. For informed modelling of complex processes such as migration, it is imperative to involve interdisciplinary research teams, with modelling and analytical experts, and diverse, yet complementary expertise of subject matter. Open discussions around good practice, exploring different approaches to modelling and decisions, matter a lot both for the practitioners as well as theorists and methodologists, especially in such a complex and uncertain area as migration. Importantly, this also matters if models are to be used as tools of policy support and advice. We discuss the specific aspects of that challenge next.

3 Policy Impact: Scenario Analysis, Foresight, Stress Testing, and Planning

In the context of practical implications for the users of formal models, it is a truism to say that any decisions to try to manage or influence complex processes, such as migration, are made under conditions of high uncertainty. Broadly speaking, as signalled in Chap. 2, we can distinguish two main types of uncertainty. The epistemic uncertainty is related to imperfect knowledge of the past, present, or future characteristics of the processes we model. The aleatory uncertainty, in turn, is linked to the inherent and irreducible randomness and non-determinism of the world and social realm (for a discussion in the context of migration, see Bijak & Czaika, 2020). The role of these two components changes over time, as conjectured in Fig. 9.1, with diminishing returns from current knowledge in the more distant future, which is dwarfed by the aleatory aspects, driven by ever-increasing complexity. Importantly, the influences of uncertain events and drivers accumulate over time, and there is greater scope for surprises over longer time horizons.

Fig. 9.1

Stylised relationship between the epistemic and aleatory uncertainty in migration modelling and prediction

In the case of migration, the epistemic uncertainty is related to the conceptualisation and measurement of migration and its key drivers and their multi-dimensional environments or ‘driver complexes’, acting across many levels of analysis (Czaika & Reinprecht, 2020). In addition, the methods used for modelling and for assessing human decisions in the migration context also have a largely epistemic character. Conversely, systemic shocks and unpredictable events affecting migration and its drivers are typically aleatory, as are the unpredictable aspects of human behaviour, especially at the individual level (Bijak & Czaika, 2020). At a fundamental level, the future of any social or physical system remains largely open and indeterministic, with social systems additionally influenced by the irreducible uncertainty of human free will – or, in other words, agency (for a full philosophical treatment, see e.g. Popper, 1982).

In this context, an important question with practical and policy bearings is: can following the Bayesian model-based template help manage the different types of migration uncertainty across a range of time horizons? Given that different types of uncertainty dominate in different temporal perspectives, the usefulness of the proposed approach for policy and other practical applications depends on the horizon in question. An important distinction here is that while the epistemic uncertainty can be reduced, the aleatory one cannot, and needs to be managed instead. At the same time, formal modelling and probabilistic description of uncertainty can help address both these challenges.

The areas for possible reduction of the epistemic uncertainty have been highlighted throughout this book. The uncertainty in the data can be controlled, possibly by using formal quality assessment methods and combining information from different sources (Chap. 4); the features of the underpinning social mechanisms, embodied in model parameters, can be identified by formal model calibration (Chap. 5); and the knowledge on human decision making can be enhanced by carefully designed experiments (Chap. 6). Bearing in mind that there are trade-offs between the model precision and feasibility of its construction, an iterative modelling process, advocated in this book, can help identify the knowledge gaps, and thus delineate and possibly reduce epistemic uncertainty.

Given the presence of the aleatory uncertainty, in the strict predictive sense, any models of complex systems can only be valid at most in the short term, and only if uncertainty is properly acknowledged. Nevertheless, models can still be helpful for many other purposes across a range of time horizons, helping to manage policy and operational responses in the face of the aleatory uncertainty. Here, a variety of possibilities exist, from early warnings and stress testing in the short term, to long-range scenario analysis and foresight, all of which can help contingency planning (Bijak & Czaika, 2020).

3.1 Early Warnings and Stress Testing

Early warnings and stress testing are particularly useful for short term, operational purposes, such as humanitarian relief, border operations, or similar. What is required of formal models in such applications is a very detailed description, ideally aligned with empirical data. This description should be linked to the relevant policy or operational outcomes of interest, especially if the models are to be benchmarked to some quantitative features of the real migration system. Here, the models can be additionally augmented by using non-traditional data sources, such as digital traces from mobile phones, internet searches or social media, due to their unparalleled timeliness. In particular, formal simulation models can help calibrate early warning systems, by allowing to set the response thresholds at appropriate levels (see Napierała et al., 2021). At the same time, models can help with stress testing of the existing migration management tools and policies, by indicating with what (and how extreme) events such tools and policies can cope. One stylised example of such applications for the Risk and Rumours version of the migration route formation model is presented in Box 9.1.

Box 9.1: Model as One Element of an Early-Warning System

In the simplest example, corresponding to the operational needs of decision makers in the area of asylum migration, let us focus on the total number of arrivals at the destination, and on how this variable develops over time. There are clear short-term policy and planning needs here, related to the adequate resources for accepting and processing asylum applications, as well as providing basic amenities to asylum seekers: food, clean water, and shelter; possibly also health and psychological care, as well as education for children. All these provisions scale up with the number of new arrivals.

One example of a method for detecting changes in trends is the cumulated sum (‘cusum’) approach originating from statistical quality control (Page, 1954). In its simplest form, the cusum method relies on computing cumulative sums of the control variable, for example of the deviations of the observed migrant arrivals from a baseline level, and triggering a warning when a certain threshold h is reached. After a warning is triggered, the cumulative sum may then be reset to zero, to allow the system to adjust to the new levels of migration flows. Formally, if z_t is the variable being monitored, observed at time t, the cusum can be defined as V_t = max(0, V_t–1 + z_t), where V₀ = 0. The use of the cusum approach to asylum migration has been discussed by Napierała et al. (2021).

Setting the threshold h at which the cusum method would trigger a warning is one of the key challenges of the approach, with visible trade-offs between false alarms (costly overreaction) and unwarranted complacency (costly lack of action). Simulation models, and even theoretical ones, such as the Risk and Rumours introduced in Chap. 8, can help shed light on the consequences of setting the thresholds at different levels. An illustration of this application is shown in Fig. 9.2, which presents a cusum chart based on the numbers of daily arrivals y_t simulated by the model. The variable under monitoring, z_t, measures a standardised number of arrivals, assuming that the average number under normal conditions is 10 persons daily, with a standard deviation of 2, so that z_t = (y_t – 10)/2. In real-life applications, this mean and standard deviation can, for example, correspond to the operational capacity of services that register new arrivals, and provide them with the basic necessities, such as food and shelter. To be able to respond effectively, such services need an early warning signal when the situation begins to depart from the normal conditions.

Fig. 9.2

Cusum early warnings based on the simulated numbers of daily arrivals at the destination in the migrant route model, with different reaction thresholds

In Fig. 9.2, a range of warnings issued at different levels of the threshold h are presented, denoted by black horizontal lines: solid for h = 1, dashed for h = 2 and dotted for h = 4. A warning is generated whenever the cusum line reaches a threshold. This means that for h = 1, the first warning, for the first wave of arrivals, is generated at time (day) t = 90, for h = 2 one day later, and for h = 4 three days later. For the second wave of arrivals, the warnings are generated almost synchronously: at t = 178 for h = 1 and at t = 179 for h = 2 or h = 4. At the same time, the threshold set at h = 1 generates false alarms at t = 145 and 146. Different thresholds have clearly varying implications for the timely operational response: while h = 1 leads to false alarms, and h = 4 may mean unnecessary delays, jeopardising the response, the threshold of h = 2 seems to be generating warnings about the right time. In this way, an agent-based model can be used to calibrate the threshold level of an early warning system for a given type of situation, bearing in mind the different implications of complacency on the one hand, and overreacting to the data signal on the other.

3.2 Forecasting and Scenarios

At the other end of the temporal spectrum, foresight and scenario-based analyses, deductively obtained from the model results (see Chap. 2), are typically geared for higher-level, more strategic applications. Given the length of the time horizons, such approaches can offer mainly qualitative insights, and offer help with carrying out the stimulus-response (‘what-if’) analyses, as discussed later. This also means that these models can be more approximate and broad-brush than those tailored for operational applications, and can have more limited detail of the system description. An illustration of how an agent-based model can be used to generate scenarios of the emergence of various migration route topologies is offered in Box 9.2, in this case with specific focus on how migration responds to unpredictable exogenous shocks, rather than examining the reactions of flows to policy interventions, which is discussed next.

Box 9.2: Model as a Scenario-Generating Tool

To help decision makers with more strategic planning, formal scenarios – coherent model-based descriptions of the possible development of migration flows based on some assumptions on the developments of migration drivers – offer insights into the realm of possible futures, to which policy responses might be required. Ideally, to be useful, such scenarios need to be broad and imaginative enough, while at the same time remaining formal: an important advantage provided by modelling (Chap. 3). Here, scenarios based on agent-based models offer an alternative to other approaches to macro-level scenario setting with micro-foundations, such as, for example, the more analytical dynamic stochastic general equilibrium (DSGE) models used in macroeconomics (see Chap. 2; for a migration-related review and discussion, see also Barker & Bijak, 2020). One important feature of agent-based models in this context is that, being based on simulations, they do not require assumptions ensuring the analytical tractability of the problem, as is the case with DSGE or similar approaches.

As an illustration, we offer a range of scenarios generated by the theoretical version of the Risk and Rumours model presented in Chap. 8, under four sets of assumptions: the baseline one, as discussed before, for the different effects of risk on path choice among the agents (‘risk-taking’ versus ‘cautious’), and varying levels of initial knowledge and communication (‘informed’ versus ‘uninformed’), in each case for ten replicate runs. The scenarios illustrate the reaction of migrant arrivals to two exogenous shocks. The first is an increase in the number of the departures (and arrivals) of migrants seeking asylum from time t = 150, for example as a consequence of a deteriorating security situation caused by armed conflict in the countries of origin. The second shock simulates a situation where it becomes more difficult to cross a geographical barrier, such as the Mediterranean Sea, from time t = 200. In this case, the risk of the loss of life on the way increases, also due to external factors – these may be related to weather conditions, or to a smaller number of rescue efforts undertaken, for example caused by a global pandemic, a political crisis, or as a matter of political choice.

The outcomes of the various scenarios generated by the Risk and Rumours model are illustrated in Fig. 9.3. Unsurprisingly, the increased number of departures translates into an increased number of arrivals (with a time lag), and the number of fatalities reacts instantaneously to the deteriorating chances of a safe crossing. The differences for the number of arrivals obtained under different sets of assumptions are minimal, but for the number of deaths, there is a clear reduction in the fatalities under the higher levels of initial information and communication, although with considerable between-replicate variability, depicted by grey shading. This points to the information about safety of various routes as a possible area for a promising policy intervention, which is explored further in Box 9.3.

Fig. 9.3

Scenarios of the numbers of arrivals (top) and fatalities (bottom), assuming an increased volume of departures at t = 150, and deteriorating chances of safe crossing from t = 200. Results shown for the low and high effects of risk on path choice (‘risk-taking’ and ‘cautious’) and levels of initial knowledge and communication (‘informed’ and ‘uninformed’), including between-replicate variation (grey shade)

3.3 Assessing Policy Interventions

Contingency planning and stress-testing of migration policies and migration management systems can work across different time horizons. Such applications either require numerical input, which restricts the possible applications to shorter-term uses, or not, allowing also qualitative exploration of the space of model outcomes in the long run. In either case, the goal of the associated ‘what-if’ modelling exercise and the ensuing policy analysis is to assess the results of different assumptions and possible policy or operational interventions based on model results. In the migration context, possible examples may include the rerouting or changes of migration flows in response to multilateral changes of migration policies, recognition rates, information campaigns, and deploying other policy levers. Box 9.3 contains an illustrative example related to an information campaign on the safety of crossings.

Box 9.3: Model as a ‘What-If’ Tool for Assessing Interventions

Similar to scenarios driven by external shocks to the migration system, the models can serve as tools for examining ‘what-if’ type responses to changes to the system that can be driven by policies. As signalled in Box 9.2, a relevant example can refer to information campaigns, and to how the different ways of injecting reliable information into the system impacts the outcomes of the modelled migration flows – and of fatalities. Another question here is whether the policy tools work as envisaged by the policy makers, or if they can generate unintended consequences, and if so, what they are.

The example presented in this box is also inspired by a monitoring and evaluation study of information campaigns among prospective migrants carried out in Senegal (Dunsch et al., 2019), as well as by the findings from the Flight 2.0/Flucht 2.0 project (Emmer et al., 2016). Here, we first use the theoretical version of the Risk and Rumours model to examine the impact of a public information campaign carried out by official authorities, introduced in response to the increased number of fatalities during migrant journeys in a range of scenarios introduced in Box 9.2. The resulting trajectories of arrivals and deaths are presented in Fig. 9.4. We use the theoretical model to ascertain the possible direction and magnitude of impact of such an information campaign. The results are subsequently contrasted with those obtained for the empirically grounded model version (Risk and Rumours with Reality), shown in Box 9.4, to check whether they stay robust to additional information included in the model.

Fig. 9.4

Outcomes of different ‘what-if’ scenarios for arrivals (top) and deaths (bottom) based on a public information campaign introduced at t = 210 in response to the increase in fatalities

As can be seen in Fig. 9.4, especially in comparison to the scenarios reported earlier in Fig. 9.3, the information campaign has barely any effect on the two model outcomes, except for minimally increasing death rates in trusting and risk-taking agents. Interestingly, the level of trust in the official information does not seem to play the role in the outcomes (Fig. 9.4). Part of the reason is that, regardless of whether the information campaign is trusted or not, it provides information about topology – possible paths and crossings – which the agents otherwise would not have access to. This effect can counterbalance any gains from the information campaign as such, especially in the situations when the agents trust the information they receive, but choose to ignore the warnings. This is an example of a mechanism possibly leading to unintended consequences of an in principle well-meaning migration policy (see Castles, 2004).

Whether the insights discussed above can be also gained from the model calibrated to the actual data series is another matter. To test it, in Box 9.4 we repeat the ‘what-if’ exercise introduced before, but this time for the Routes and Rumours with Reality version of the model, calibrated by using the Approximate Bayesian Computation (ABC) approach, described in Sect. 8.4.

Box 9.4: Model as a ‘What-If’ Tool for Assessing Interventions (Cont.): Example of the Calibrated Routes and Rumours with Reality Model

In this example, we reproduced the results for the ‘what-if’ assessment of the efficiency of an information campaign, introduced in Box 9.3, for a calibrated version of the empirically grounded model, Routes and Rumours with Reality. A selection of results is shown in Fig. 9.5. The numbers for the original scenario (‘plain’) and for the one assuming an information campaign are very similar. For the latter scenario, 40 runs generated from the posterior distribution obtained by using Approximate Bayesian Computation are shown (solid grey lines) together with their mean (solid black line), while for the plain scenario, just the mean is presented (dashed black line), for the sake of transparency. For comparison, the (appropriately scaled) numbers from the empirical data are also included on the graph, to demonstrate the fit of the emulator to the real data.

Fig. 9.5

Outcomes of the ‘what-if’ scenarios for arrivals (top) and deaths (bottom) based on a public information campaign introduced at t = 210, for the calibrated Risk and Rumours with Reality model

From comparing the results shown in Figs. 9.4 and 9.5 it becomes apparent that the results of the scenario analysis for the calibrated model do not reproduce those for the theoretical version, Risk and Rumours, presented before. The effects that could be seen for the theoretical model disappear once an additional degree of realism is added, with the importance of the decision making mechanism, and the parameters driving it, being dwarfed by the information introduced through the process of model calibration. One tentative interpretation could be that once the model becomes more strongly benchmarked to the reality, the description of the decision processes needs to be more realistic as well. This points to the need for carrying out further enquiries into the nature of the decision processes undertaken by migrants during their journey, enhancing the model by including the possibilities of stopping the journey altogether at intermediate points, returning to the point of departure, travelling via alternative routes or means of transport, and so on.

On the whole, the results of scenarios, such as those presented in Boxes 9.3, and 9.4, can go some way towards answering substantive research and policy questions. This also holds for the questions posed in Chap. 8, as to whether increased risk – as well as information about risk – can bring about a reduction in fatalities among migrants by removing one possible ‘pull factor’ of migration. As can be seen from the results, this is not so simple, and due to the presence of many trade-offs and interactions between risk, peoples’ attitudes, preferences, information, and trust, the effect can even be neutral, or even the opposite to what was intended. This is especially important in situations when different agents may follow different – and sometimes conflicting – objectives (see Banks et al., 2015). These findings – even if interpreted carefully – strengthen the arguments against withdrawing support for migrants crossing the perilous terrain, such as the Central Mediterranean (see Heller & Pezzani, 2016; Cusumano & Pattison, 2018; Cusumano & Villa, 2019; Gabrielsen Jumbert, 2020).

An interesting methodological corollary from the comparison of different scenarios is that it is not necessarily the most sophisticated and realistic version of the model that generates the most valuable policy insights: in our case, the calibration of the migration processes to the arrival and departure data in the Risk and Rumours with Reality model version overshadowed the mechanism of information-driven migration decisions, leading to a better-calibrated model, but with smaller role of the underlying behavioural dynamics of the agents and their interactions. Of course, the process of modelling does not have to end here: in the spirit of inductive model-based enquiries, these results indicate the need to get more detailed information both on the mechanisms and on observable features of the migration reality, so that the journey towards further discoveries can follow in a ‘continuous ascent’ of knowledge, in line with the broad inductive philosophy of the model-based approach.

4 Towards a Blueprint for Model-Based Policy and Decision Support

In practice, the identification of the way in which the models can support policy or practice should always start from the concrete needs of the users and decision makers, in other words, from identifying the questions that need answering. Here, the policy or practical implications of modelling necessitate formulating the model in the language of the problem, and including all the key features of the problem in the model description (see also Tetlock & Gardner, 2015). The type of problem and the length of the decision horizon will then largely determine the type of model. Coupled with the availability of data and other information, this will enable inferring the types of insights from the modelling exercise. This information will also limit the level of detail in modelling, from relatively arbitrary in data-free models, to limited by the availability and quality of data in empirically grounded ones. Hence, unless there is scope (and resources) for ad hoc collection of additional information, the level of reliance on empirical data can be (and often is) outside of the choice of the modeller.

When it comes to the modelling, our recommendation, as argued throughout this book in the spirit of the inductive Bayesian model-based approach, is to start with a simple model and scale it up, adding complexity if needed to answer the question, even in an approximate manner. At this stage, the data should be also brought in, where possible. Once the model produces the results sought, it is then a matter for the decision maker to judge whether the outputs are sufficient for the purpose at hand, and given the data and resource limitations, or if more detail needs adding to the model. The acceptable model version then is used to produce the required outcomes, and – crucially – assess the limitations of the answers offered by the model, as well as residual uncertainty. This broad blueprint for using models to aid policy, operations, interventions, and other types of practical applications is diagrammatically shown in Fig. 9.6.

Fig. 9.6

Blueprint for identifying the right decision support by using formal models

Of course, a key limitation, present in all modelling endeavours, is the fundamental role of model uncertainty – an effect that has been dubbed the Hawkmoth Effect, analogous to the Butterfly Effect known from the chaos theory (Thompson & Smith, 2019). The Hawkmoth Effect means that even with models that are close to the reality they represent, their results and predictions, especially quantitative (in the short run), but also qualitative (in the long run), can be far off. As any model-based prediction is difficult, and long-term quantitative predictions particularly so (Frigg et al., 2014), the expectations of model users need to be carefully managed to avoid false overpromise.

Still, especially in the context of fundamental and irreducible uncertainty, possibly the most important role of models as decision support tools is to illuminate different trade-offs. If the outputs are probabilistic, and the user-specific loss functions are known, indicating possible losses under different scenarios of over- and underprediction, the Bayesian statistical decision analysis can help (for a fuller migration-related argument, see Bijak, 2010). Still, even without these elements, and even with qualitative model outputs alone, different decision or policy options can be traded off according to some key dimensions: benefits versus risk, greater efficiency versus preparedness, liberty versus security. These are some of the key considerations especially for public policy, with its non-profit nature and hedging against the risk preferable to maximising potential benefits or rewards. At the end of the day, policies, and the related modelling questions, are ultimately a matter of values and public choice: modelling can make the options, their price tags and trade-offs more explicit, but is no replacement for the choices themselves, the responsibility for which rests with decision makers.