Keywords

4.1 Introduction

This chapter examines the ways that warning providers connect and collaborate with impact experts to improve and communicate warnings. We first look at the role of the warner, then the impact forecaster and finally the linkages between them. In doing so, we shall see that:

  • A successful warning is used to take action. It is as much a compelling narrative as information.

  • A skilful impact forecast identifies who or what will be impacted, where, when and by how much.

  • Impact data are often confidential, requiring partnership with the data owner and a clear mutual understanding of the objectives of any impact prediction tool.

  • Partnerships between information producers and warning communicators can be facilitated by intermediaries.

4.1.1 Warnings

Warnings are produced by a wide range of actors in the public and private sectors, based on information from weather and hazard forecasts, on science related to weather or hazards and on estimates of the anticipated impact of the hazard, produced using a variety of tools and technologies. The intent is to provide enough lead time to reduce the risk from the hazard and thus prevent a disaster. Expert risk information must be presented in a form that enables the creation of a convincing warning narrative that ultimately supports decision-makers and encourages action. Research in the last decade has demonstrated that strengthening warnings with impact information significantly reduces the disaster (WMO 2015, Casteel 2016, Anderson-Berry et al. 2018, Potter et al. 2018).

In producing a warning, the warner is as much an artist as a scientist, crafting a persuasive story out of a selection of uncertain facts, using their experience of context and precedent and fitting the result into a variety of formats to be delivered through different media, all while under considerable time pressure. Warning information must reach decision-makers in broad, varied disciplines and fields of service and expertise. The available information is constrained by forecasting capabilities, which vary with lead time and location and which may be more or less relevant to the end user. Temporal scales of the potential threat and the time taken to issue warnings add further complexity. Partnerships that help the warner to have confidence in the sources of their information are critical.

The “bridges” connecting the information provider to user may be complex because of their situation within complicated, embedded systems and should therefore be designed prior to hazard occurrence (see Fig. 4.1). Research shows that intermediaries, which may be organisations or individuals within an organisation, can aid in connecting people throughout the warning chain.

Fig. 4.1
figure 1

Interactions from the Warner to Impact Expert embedded in complex systems. (Source: Adapted from Ruti et al. 2020; Beaven et al. 2016; Pielke Jr. 2007)

Full size image

4.2 The Warner and Warning Information

4.2.1 The Warner

The warner of each potential threat will vary with the type of threat and the roles that define positions in organisations and governments, including authority to provide warnings and the systems in which the threat is evaluated. Warnings may be categorised by the type of threat (hazard type and complexity, science and technology that provides analyses), by role (forecaster/nowcaster, modeller, public information officer/risk communicator, emergency manager), by discipline (hydrology, meteorology, physical science, social science), by authority (Meteorological Service, emergency management, government entity) and by geography (local, regional, national or transboundary scale of the threat). The ways in which the warning is issued depend on all of these factors with additional consideration for the means of warning – official press release, television/radio announcements, sirens, SMS/DM mobile device alerts and social media. These researchers, technicians and operators, information systems and technologies are integrated into early warning systems (EWS) which aid in reducing disaster risks (Tan et al. 2020a). Experience in the UK demonstrates the importance of institutional trust in warnings, which is enhanced with impact-based warnings (Taylor et al. 2019).

Warnings may be entirely automated without any human input if a warning system has sufficient information and analytical skill to make warnings reliable and if they can be communicated appropriately. Some situations – such as very short notice warnings – are perhaps better suited to the automated approach, but many of today’s weather and natural hazard warning systems require a mixed approach, with the human adding to the automated system, either “in the loop” or “over the loop” (Pagano et al. 2016). Issuing warnings of this type requires expertise in science and the art of communication in equal measures. It is not a purely mechanistic process , which can be easily automated, and there will always remain an element of subjectivity, but it should at least be a methodical process. The methods adopted will vary depending on circumstances, but all should look to ensure a balance between the scientific, the practical and the useful and should ensure a level of consistency by limiting differences of opinion, biases and risk appetites.

Invariably, no one person has all of the expertise, information or experience across these fields, which is why warning creation needs to be a collaborative and multi-disciplinary process. The resulting diversity of perspectives, experiences and insights is both a strength and a challenge of this collaborative approach. The warner operates within guidelines of EWS design, which may be an automated system managing big datasets and information sources, or a human forecaster analysing hydrometeorological conditions, and/or may further contain interpretation of the nature of the impact. The means of communication and the target audience must also be considered, together with wider aspects of decision-making from the individual to broader governance systems (Tan et al. 2020b).

4.2.2 Warning Content

The warning must focus on what the warner is trying to achieve with the warning. Information is needed both as warning content and for decisions on the importance and timing of the warning. The warner must trust their information sources (official and unofficial), be able to select key aspects of information received and determine ways to interact with information to produce the warning in different situations and contexts.

“An effective warning…specifies the exact nature of the threat” (Casteel 2016). The content should be clear and understandable. The more that the warning includes information about the hazard impact, the more actionable it is and the more effective the warnings become (Lazo 2020; Potter 2018). “The increased specificity provided by the “hazard” portion…should therefore enhance personal relevance and potentially increase the likelihood that the recipient takes protective action” (Casteel 2016).

Even though technological advances have improved the information available, it must be interpreted for sector-specific use. Disaster managers and emergency responders need to know the nature, severity and geographical extent of the disaster, the timing (start and end) and how soon evacuations must occur. The agriculture sector needs to know the likelihood of a threat such as flooding, its geographical extent (how many crops will be affected) and scale (one district or several, cross-jurisdictional boundaries) to effectively enact readiness measures that protect crops, livestock and agricultural livelihoods. The energy sector requires knowledge of the timing and type of event to ensure that energy can be supplied to users, including emergency responders and at-risk populations.

The information required by the warner evolves as the threat approaches. At the early warning stage, there are likely to be few sources, though they may reach the warner through multiple routes. At this stage there is also more time available to refer back to experts for clarification and to carefully craft a convincing narrative for the receiver. As deadlines for specific actions approach, the message needs to be oriented to the impacts that are relevant to those actions, with emphasis on the level of confidence and on alternative outcomes. Once the threat is imminent, there is no longer time for careful analysis, but details of changes in track, intensity, timing and associated impacts may be critical to responders’ actions and safety. At this stage the warner will look for multiple data sources, up and down and outside the warning chain, to maintain situational awareness: of the hazard, of responses to warnings and, once it arrives, of the actual impacts. These will all inform warning updates.

4.2.3 Warning Creation

In preparing a warning, the warner aims to create a compelling narrative that will convince the receiver to take notice and then to take action appropriate to their situation. Selecting the warning level is a critical part of the process. It requires careful interpretation of the available information, especially the predicted impact and the level of confidence, as well as the context of the warning, for instance, if those affected are already dealing with the impacts of another hazard. The content of the warning is selected and presented with all this in mind. Central will be the expected impact on the receiver. In support will be sufficient information on the causal hazard including both the prediction and its confidence, supported, when available, by evidence (e.g. links to CCTV at upwind/upstream locations). Where specific vulnerabilities are relevant, these need also to be included, together with the level of confidence. Where there are significant uncertainties in the impacts, the range of outcomes may be usefully represented by two or more scenarios or storylines, while stressing the need for preparation ahead of the situation becoming clearer.

Taking a typical impact-based weather warning system as an example, it is useful to subdivide the warning process into distinct components.

Weather assessment → Risk assessment → Change assessment → Utility assessment

  • Weather assessment: What the weather will be.

  • Impact/risk assessment: What the weather will do.

  • Change assessment: Does this change my perception of the “story”?

  • Utility assessment: Who needs to know, and how?

Most studies have focused on the weather and risk assessment aspects, but equal emphasis should be given to consideration of the latter components which are related to decision-making and communication.

The range of sources of information for the weather assessment can be very large, typically greater than can realistically be absorbed and processed by even the most experienced meteorologist. For this reason, the information must be filtered, either by limiting the sources used (procedurally or technically) or through intermediary systems which can sift and extract signals from, and summarise, the information.

A range of inputs may be used for the impact/risk assessment:

  • Modelled.

  • Empirical/heuristic based on individual or collective experience.

  • Specific, current knowledge modifiers relating to exposure, vulnerability or the prediction of the hazard itself (e.g. based on assessment of current model performance).

One of the most difficult challenges for the warner is to acknowledge that their assessment has changed sufficiently to change the warning. Once a hazard “story” has been defined, it possesses inertia; it can be difficult to accept that it no longer reflects the best interpretation, even in the face of new, conflicting data. This is the psychological phenomenon known as “anchoring”, which, among other things, is why forecasts and warnings can be most prone to change following handovers between shifts.

Even when the warner feels that other criteria for issuing a warning may have been met, they must consider whether the warning information will be useful to end users. This consideration acts as another filter, with the warner playing an editorial role to determine what user(s) need to know. For example, matrix impact-based warning systems should arguably result in far more long-period, low-probability warnings than they do. This is because warners judge that issuing too many warnings is counterproductive.

Warnings are not issued in isolation. They exist in relation to other warnings, to additional communications and of course to the previous and future versions of warnings for the same event. While a warning to take protective action now for high weather impacts in the next 6 hours may be usable by some recipients, it gives little time for preparation. In this sense, no warning should come as a surprise! So the warner should use a succession of communications: advisories, watches, warnings, press releases, blogs, tweets and other advice to manage uncertainties and expectations well before the event, such that the final “take action now” message is expected when it comes. Best practice is to think strategically over the whole period from initial indications of severe weather up to the event itself and plan the issue of warnings and other communications to best inform users and allow them to prepare while avoiding overcommitting resources should the worst conditions not occur.

The key to preparing high impact weather warnings is the development of expertise: in the needs of the warning’s users, in the reliability of the various sources of guidance (both meteorological and socio-economic) and in the behaviour of the weather system causing the hazard. Expert warning forecasters assimilate the incoming data, generating a conceptual model of the situation and enabling a level of situational awareness sufficient to anticipate events (Klein 1989). Anticipation enables the expert warner to filter the voluminous information and focus on the most relevant aspects for use in decision-making. Decisions are always subject to judgement in the face of uncertainty. Given the uncertainty and the impact of warning decisions, there is never enough information.

Looking forward, research is focusing on using machine learning systems to undertake the process of filtering, so as to identify the key areas of risk uncertainty requiring human judgement. One way of facilitating this is for hazard and impact predictions to be formulated into a first-guess warning, combining the probabilistic and severity elements. In some situations, this might provide the route towards fully automated warnings, but more generally it should be accompanied by tools for the warner to probe the individual components, assess the sensitivity of the outputs and amend the resulting warning.

4.2.4 Tailored Warnings

“Forecasts will occasionally take into account some societal factors (e.g., extending a warning’s timing to cover when schools are releasing students), but often do not directly account for human factors related to decision-making prior to, and during life-threatening extreme events” (Uccellini & Ten Hoeve 2019). Engaging with social scientists to work with specific groups of users in the design of tailored warnings can lead to better warning responses. Research shows that tailoring warnings to the needs of recipients increases their effectiveness. However, this benefit, in better warning response, has to be set against the cost, complexity and potential for inconsistency of doing so. The benefit is not restricted to economics but may include a variety of non-economic benefits and ethical issues of human rights as well. We consider several aspects of tailoring here. The first few options relate to selection of data, while the latter ones relate to presentation.

Selecting the Best Forecast for the User

When working with a decision-maker to improve their access to predictions of hazards, the basis for selecting the source of that information will include several factors, of which accuracy or skill is likely to be an important one. On the other hand, a provider of information, trying to optimise the value to users of the information they provide, has to choose which improvements to their prediction system to implement. In both cases it may appear to be beneficial to focus the evaluation on the conditions of interest to the user, i.e. the hazard itself or perhaps the extreme weather conditions that give rise to the hazard – both things that occur rarely.

If the exercise is being undertaken following a disaster, it may even be felt that the most important consideration should be that the disaster would have been predicted with the new system or data source. This approach can be very dangerous, typically leading to over-prediction and loss of confidence in the warning system. While there are unbiased methods of assessing extreme values of weather variables and of assessing the skill in predicting occurrence of an event, in both cases they are not immediately intuitive and so are not the most widely used evaluation approaches. In addition, the small number of data points available for extreme events means that the error bars in the score are likely to be very broad.

Even if different information sources are reliably identified as giving the best guidance for different phenomena, the risk of inconsistent predictions is considerable. For instance, if one information source gives better hurricane track predictions and another gives better hurricane intensities, it would be foolish to rely entirely on the track from one and the intensity from the other. For the warner this means that decisions on the scenario for the warning should be based on as much information as possible about the current situation, about the evolution predicted by each source and about the performance of that source in similar situations. It is essential that this information includes full probability distributions and that biases have been removed.

Selecting the Information to Communicate

Having obtained a skilful forecast source, there are a myriad of products that could be extracted. Information for inclusion in warnings is unlikely to be the same as that used for routine forecasting. For instance, extracting the probability of key thresholds being passed enables the user to focus immediately on their specific concerns. Having said that, it is advisable to standardise if possible, so that users receiving warnings that contain different thresholds do not perceive an inconsistency. Other means of tailoring the information include recalibration, bias correction, and derivation of user-relevant variables (both physical and socio-economic). Thus a flood forecast may be presented as flow in the river, as water level above the river bed, as a map of flood depth, or more specifically in terms of the depth on roads to the Highways Department, the extent of residential property flooding to the public, and the probability of reaching a critical depth at an electricity substation to a power company. In the pressured environment of an emergency response team, the more precise and actionable statements are likely to produce more effective responses with less room for error. It is recommended that warnings of the highest identified risks should use tailored prediction products incorporating the probability of specific hazard thresholds being crossed and information about exposure and vulnerability of communities in the threatened areas.

Geographical Tailoring

When considering tailoring of information, making allowance for geography is perhaps the most obvious and most necessary. For instance, when forecasting wintry weather in complex terrain, low-lying valleys may have rain, while higher up the slopes, snow is accumulating. The meteorologist may use the height of the snow line, but to communicate this may require particular locations to be identified. Proximity to rivers is a key driver of risk from flooding, but few people know the distance to their nearest river and city dwellers may not realise that a river exists, let alone that they could be flooded by it. Warnings that refer to settlements at risk or escape routes that are safe or threatened may use names that are only current in the immediate vicinity, so that visitors are unaware of their meaning. For instance, reference to numbered highways or to numbered junctions on a highway will be incomprehensible to a portion of the travelling public. Maps can help in avoiding these difficulties, provided they are clearly located relative to major towns, highways and other widely known features.

Tailoring the Communication

This area offers the greatest opportunity for tailoring with the minimum risk of confusion. For instance, warnings should be disseminated in multiple languages, according to the make-up of the population, and through different media (newspaper, TV, radio, Internet, mobile app, social media, etc.) according to accessibility by the population. It is crucial to use geographical names that are generally understood. Maps can be powerful tools for communicating the proximity of a warning area to dispersed communities – but only to those that are able to read them. Colours can provide powerful support to communication - the green-amber-red sequence of traffic lights is understood in many cultures, but not all, and care must be taken to cater for those with colour blindness. The use of cartoon characters to communicate has been very successful in some cultures, but not all. Since users will often seek confirmation from friends and family before responding, it is important that different means of communicating the information are consistent.

Other Areas for Tailoring

Many aspects of warning design affect how particular groups receive, interpret and respond to information. Cultural cues can be important, e.g. the colour red has particular and conflicting cultural meanings. Similarly, the idioms used in the language can be as important as the words. Phrases such as “snowing handkerchiefs” or “raining cats and dogs” are meaningful to some and not to others. Gender is of particular importance in most countries. However, when considering tailoring for women, it is necessary to consider the route by which the warning will reach them. A direct route, e.g. by social media, will require different tailoring from an indirect one, e.g. through a village chief or street warden. Another potential area of tailoring comes from study of the psychological response to challenging situations. Some people typically respond positively, seeking to turn it to their advantage, while others are followers of the crowd, and yet others will fight against change. In the West, marketing companies have learnt to target these groups differently, and it is likely that similarly targeted warning messages may be effective, though research has yet to demonstrate this.

Tailoring for Specialists

Where there is an emergency manager for a large infrastructure facility that will affect thousands of people, the case for tailoring very specifically to that role’s needs is very strong. It is essential that the response is pre-planned and that it is carried out quickly and effectively when the warning threshold is reached. This may involve simplifying the warning down to a simple code word, which is learnt and practised by each provider and user. The same holds for organisers of large public events such as pop festivals. Tailoring for major public facilities such as schools or hospitals is more complex, but equally important. Candidate specialist users for tailored warnings include power generators and suppliers, water suppliers, dam operators, telecommunications operators, road transport, rail transport, air transport, marine transport, food retailers, education, emergency responders, health services, waste disposal, public event organisers, major employers and businesses. Such users should not be using generic public warnings to take decisions. They should have carried out a risk assessment for their business, which identifies the hazards they are exposed to and the level of risk for each. They should also have a risk response plan including trigger points at which action must be taken, together with the information needed, both to identify the trigger and to inform the action. The receipt of a tailored warning should be the primary trigger for preparatory actions ahead of weather-related hazards. Activation of response plans should be tied to the receipt of a warning.

Co-Design in Generic Warnings

Currently a high degree of tailoring cannot be justified for public warnings. The alternative is to co-design a compromise generic warning system that meets most needs and to use education to embed its characteristics in the users’ cultures. Such co-design activities must be very carefully planned to ensure an adequate voice for all sections of the community. Evidence also suggests that a feedback loop is required in which community representatives first identify what they feel are the problems with current capability and then criticise successive sets of upgrade options in an iterative fashion. Not only does this help to produce effective warnings, but it also builds a sense of ownership in the community that helps with the adoption and use of the final product. Looking to the future, social profiling in combination with machine learning techniques, e.g. as used for selecting online advertisements, has the potential to enable individual tailoring of weather information based on individual risk profiles. However, the implications of getting it wrong mean that warnings are likely to adopt such techniques more slowly than other environmental forecasting services.

4.2.5 Evaluating the Warning

While evaluation is important throughout the entire warning value chain, it is particularly critical where risk information is translated into actionable messages for those at risk and those responsible for mitigating and managing hazard-related impacts (e.g. emergency managers). NMHSs have a long history of using statistical methods to verify weather forecasts (Ebert et al. 2015), but relatively less experience in evaluating the use and efficacy of their products and services.

While it would be desirable to demonstrate benefit by observing a decreasing trend in metrics of death, distress and damage as warnings improve, it is rarely possible to do this. Since the objective of warning is to help the recipient make better decisions, surveys of people’s actual receipt and reaction to warnings are probably the best available tool. A baseline is required, so surveys should be designed and established before the introduction or upgrade of a warning system and continued after it is complete, using the same format throughout. If surveys have been part of a co-design process, it may be appropriate to continue these, bearing in mind the difference between anticipated response and actual response.

By definition, evaluations are comparative: over time; between places, jurisdictions and populations; or between other contrasting features or situations. For example, the introduction of a warning service or modification to add impact information should be evaluated using assessments before and after service implementation. Traditionally, one-off evaluations often take place long “after-the-fact” making it difficult to collect and interpret the information. Ideally, evaluation should be undertaken continuously throughout the life cycle of any significant change, permitting course corrections as the service is developed and introduced.

A useful approach is to treat weather warnings as a form of programme intervention, not unlike a campaign to increase vaccination uptake or use of masks in disease prevention. Whether explicitly or implicitly defined, weather warnings are provided to influence awareness, risk perception, behavioural intent, decisions, behaviours and, ultimately, outcomes—all of which can potentially be measured. The theory of change is a methodology for planning and evaluating social change programmes (see, e.g. Taplin & Clark 2012) that is now widely used in international development and is very relevant to the challenge of evaluating warnings. It approaches a social intervention of any kind by first determining the desired outcomes and then associating measurable success indicators with each. It involves documenting the actors and processes through which a service is expected to affect outcomes, together with any intermediary factors (e.g. awareness and comprehension of warning information, trust, beliefs, etc.). The analysis may draw on personal experience, expert opinion or evidence and models obtained from social science research (e.g. Theory of Planned Behaviour, Ajzen 1991; Risk Information Seeking and Processing, Griffin et al. 1999).

The process of confirming a theory of change naturally leads to working hypotheses that may be examined and tested using qualitative and quantitative research. Each approach has strengths and weaknesses, so it is beneficial to adopt multiple lines of inquiry and triangulation over the course of an evaluation. For instance, observational field research, focus group sessions and mental modelling interviews (Morgan et al. 2002) are often helpful in documenting change theories and underlying constructs among those who are developing, providing and utilising warning services. Surveys, however, may be better suited to assess the representativeness of such findings across groups of actors (e.g. emergency managers, Hoss et al. 2018) and the effect of intermediary factors (e.g. trust, perceived threat) on behavioural intent and recalled responses and outcomes, particularly following memorable severe events (e.g. Winter Storm Doris, Taylor et al. 2019).

Experimental research using hypothetical or simulated scenarios allows for selective control of variables that might influence protective decisions and so is particularly useful in comparing multiple formats and content options prior to implementation (e.g. Casteel 2018, Potter et al. 2018). The disadvantage of such flexibility is that hypothetical situations may not adequately capture the context and responses that only fully materialise during actual threat events (Weyrich et al. 2020a). More generally there is a question as to how well stated intent and recalled responses correspond to actual behaviour (Weyrich et al. 2020b). Both limitations can be partially alleviated through the application of experience-based sampling techniques (Hektner et al. 2007) that attempt to measure warning-related variables in near real time. Finally, it is also possible to understand warning efficacy through analyses of behavioural outcomes (e.g. injuries, damage) using cohort or case-control observational study designs (e.g. fall-related and motor vehicle collision injuries, Mills et al. 2020).

4.3 Capabilities of the Impact Forecast

The impact forecast enhances the underlying hazard forecast, incorporating information on vulnerability and exposure to estimate the impact of the hazard. Impact experts provide critical information to “core partners responsible for public safety”. Impact-based decision support services help to better understand and utilise forecasts and warnings when dealing with extreme events (Lazo et al. 2020).

Currently impact information is often generated by the warner based on their experience of previous events and is thus limited by the experience of each warner or their understanding of their relevance to the current situation. Sometimes these analogues may be documented and semi-quantitative (e.g. US water supply forecasts put the current forecast on a scatterplot relative to past years).

“A growing number of experts are suggesting that standard warning information should be augmented with additional information about these factors” (Weyrich et al. 2018). Their expertise is often applied offline to develop tools that either enable the forecaster to convert a hazard forecast into an impact forecast or enable the decision-maker or warner to convert the decision threshold into a hazard threshold. For instance, people are increasingly making real-time forecasts of hurricane damages, particularly in the USA (e.g. this hurricane is expected to cause $750 million in damages if it follows the expected track).

Since they are often not involved in the real-time issue of warnings, the relationship of an impact expert may be more academic and detached than that of some of the other actors in the chain. On the other hand, their studies likely include analysis of events that caused major social and economic loss. By developing warnings within specific hazard early warning systems, the warnings for a single event may link to consequences of actions and decisions and will be better able to deal with potential impacts of cascading events where multiple responses from different sectors will be necessary.

4.3.1 Sources of Impact Information

A fundamental limitation to our ability to estimate impacts comes from the lack of routine collection of data on weather-related socio-economic impacts. Chapter 5 will cover this in more detail, but most available data are highly aggregated – national scale census, production, health, infrastructure performance, etc. More local data are typically not available except to specifically accredited researchers. For health data, this is because of patient confidentiality, while for infrastructure and business performance, it is to preserve commercial confidentiality. The result is that models generated using these data sources cannot be replicated or inter-compared, while those from open sources are mostly too coarse to be useful.

Attempts have been made to overcome this barrier using media reports to catalogue impacts. This approach is used in the International Disasters Database (EM-DAT) (https://www.emdat.be) coordinated by the Centre for Research on the Epidemiology of Disasters (CRED) within the Université Catholique de Louvain in Brussels and the United Nations to categorise and identify disasters globally. While originally dependent on manual interpretation, recent research has demonstrated the use of automated methods for classifying reports. According to a recent survey, 19% of NMHSs in Europe are collecting media reports to an in-house impact database, 9% are storing impact observations from storm spotter organisations and 13% are collecting other types of human impact observations (Kaltenberger et al. 2020). Among other sources, media reports of impacts of severe weather are systematically monitored, quality checked and fed into the European Severe Weather Database (ESWD , e.g. Dotzek et al. 2009). Some NMHSs and DRM organisations in least developed countries are also using media reports to gather impact information for use in establishing impact-based warning services.

Another approach that is likely to grow in the future uses automatic data collection from the Internet of Things. For instance, autonomous vehicles carry sensors for the weather, but also record information about speed, traffic density, etc. Taken together such data could provide a major step forward, both for training impact models and for evaluating forecasts and warnings of highway conditions.

4.3.2 Capabilities of Different Impact Estimation Methods

We can identify some key aspects of impact estimation tools that affect performance. The strongest evidence comes from repeatable laboratory testing and is often used as the basis of impact estimation for engineered structures. Certainly, it is important to know the failure modes and thresholds of the materials of concern. However, reproducing conditions in the real world is very demanding, e.g. ageing of materials, combinations of wind and rain, the complex motions of the sea against a barrier, etc. In designing a structure, the remaining unknowns are often dealt with by adding a safety factor. An appropriate way of dealing with this needs to be included in any failure prediction tool. Examples of this approach are wind impacts on concrete bridges, flood impacts on retaining walls and wind impacts on moving vehicles.

Where there is a clearly identifiable set of processes leading to failure, it may be possible to model these and to calibrate the model parameters using experimental evidence. For instance, the ways in which flood water damage a building are well established for particular construction methods, so a relation between flood depth and cost of recovery can be developed (Penning-Rowsell et al. 2013). In a similar way, the response of networks can be modelled. So, for instance, if a road is closed by a fallen tree or accident, or if a telecommunications link is broken, the resulting impact on road or communication traffic can be modelled as a function of the expected loadings for the time, day and season. This approach can also be used for the spread of vector-borne diseases if the behaviour of the vector (which is typically weather-dependent) can be modelled. These approaches have several limitations. Models are unlikely to be complete, so missing processes may, on occasion, be significant. The models and their parameters are typically validated for a limited range of conditions – which may not include extremes. To minimise the risk of misleading information, models should be run with hazard inputs sampled from across the uncertainty range – preferably from an ensemble prediction system – and using a range of parameter settings consistent with the validation data. The resulting probability distribution should then be interpreted for use in the warning, e.g. by extracting the most likely, the probable worst case or the probability of exceeding a particular damage threshold.

In most cases, however, the processes are hidden or too complex for modelling. In that case, prediction tools must rely on the statistical analysis of historical data to extract relevant relationships. This approach is most developed in the field of epidemiology (Armitage, Berry & Matthews 2002), but similar tools apply in many other impact areas including in the atmospheric sciences (Wilks 2006). Traditional approaches have been based on fitting an appropriate statistical distribution to data by selecting the parameters of the distribution that optimise the fit. Increasingly, these approaches are being replaced by machine learning techniques such as neural networks. In order to extract a useful relationship, data should be pre-processed to remove the influence of extraneous factors, such as time of day, day of the week, holiday periods, policy changes, etc., and to remove trends. It is also essential that all factors that may be expected to influence the impact data are represented. For instance, if a stormy period is being compared with a non-stormy period to study the relationship between weather and accidents, the different mix of people travelling – perhaps less elderly or less women – could bias the results unless allowed for in the analysis. Like the process models, the resulting statistical models should not be used in parameter ranges that are rare or missing in the training data. Standard statistical techniques can be used to assess the uncertainty in the association, and this information should always be incorporated in any predictive model so as to avoid overconfidence.

The normal statistical approach is to look for a repeatable association between the hazard and its impact. We might call this the forward model. However, where there is a unique decision to be made at a specific threshold, it may be more appropriate to predict the probability that this threshold will be exceeded. This involves less complex statistical analysis and provides the probabilistic information directly. However, the influence of confounding factors, trends, etc. can still produce misleading results.

All statistical models must be evaluated using a dataset that is uncorrelated with that used for training the model. It must also be large enough to provide statistical significance in the parameter ranges of importance for hazard impacts.

4.3.3 Sector-Specific Impact Tools

Risks vary across sectors and policy areas, such as health, environment, water/power supply, transport, technology, security, insurance and finance, so this is the first aspect that must be considered before developing warnings (Eiser et al. 2012). In some sectors, advances in data gathering, modelling and computing have increased the ability to provide critical data in their decision-making timeframes (Ruti et al. 2020, Yu et al. 2018). The health sector will be concerned with impacts of death and injury, need and capacity for hospital admissions and services and use impact assessment tools such as epidemiology, transmission, and exposure. The energy sector may be concerned with circuit failures and loss of service to critical users and will rely on detailed engineering modelling. The water sector will have numerous types of threats from lack of supply for drinking and for critical infrastructure support, threats from drainage overflow and contamination and additional health threats; therefore, the water sector must be engaged in detailed modelling of infrastructure. Emergency management is concerned with threats to all critical infrastructure, lifelines and services, such that problems with transport, power, water, energy, agriculture, environment and financial protection must be factored into the types of threats, but also impacts that may result in cascading events and multiple types of emergencies.

4.4 Structures that Facilitate Warning Information

The concept of a bridge between warner and impact expert, across which information flows back-and-forth, reflects a much more complex reality of multiple connections between different types of warners and numerous experts using multiple communication tools for interaction. It is important to have “an integrated warning system that is built on social science research and ensures full communication between all actors throughout the entire emergency management process” (Lazo et al. 2020). In recent research, the development of mitigation actions emerges from inputs of forecasts and warnings through impact-based decision support services, which feed into reducing asset damage, service interruptions and human health.

4.4.1 Relationship Between Information Provider and Warner

The relationship between information provider and warner is critical, and it is important to understand the research structures and methods of working that facilitate the applicability and application of research. Frameworks that link members through early warning systems, disaster management systems and earth system science have established structures and relationships that move from the development of science, forecasts and models to effective communication with emergency managers and decision-makers, including the general public (Beaven et al. 2016). Information needs to be shared across group boundaries, specifically by knowledge brokers (Ali et al. 2019). Boundary organisations and individuals that link research with communication and knowledge application are key facilitators of these relationships (Pielke, Jr. 2007).

Coordinated structures, such as the Natural Hazards Research Platform in New Zealand (Beaven et al. 2016) and the Natural Hazards Partnership in the UK (Hemingway & Gunawan 2018), provide mechanisms to improve tools and models and to evaluate warning systems and improve capabilities. Such structures enable discussion of caveats and uncertainties that may prevent the warner from using the information incautiously or out of context.

Within relationships, tensions between the policy and science domain create a hybrid zone in the “bridge”. Science becomes “applied science” as information turns into action. “Development of…impact-focused information and advice is supported by coordinated access to cutting-edge science and natural hazard impact research” (Hemingway & Gunawan 2018). Policy relevance requires interdisciplinarity and will likely be time-sensitive, driving a move to shorter-term actions. The general public and non-experts require simplified information, but this should not compromise the understanding of uncertain, complex information (Beaven et al. 2016).

4.4.2 Communicating Impacts and their Uncertainty

The objective of the interaction between the warner and the impact specialist is to provide the warner with the means to incorporate relevant impact information in the warning. Typically, this is achieved by providing a model or tool. There are several dangers that must be recognised by those involved if the exercise is to be successful in making the warning more effective. Great care must be taken to identify the impacts that matter to the decision-maker and to avoid simply predicting the impacts for which there are good data or simple models. It is also important to avoid generating complex sets of output that overwhelm the warner with data. Since the warner is aiming to produce a narrative that will help the receiver to act, it may be helpful to consider producing storylines (Shepherd, 2019) of hazard impacts that describe one or more impacts together with their uncertainty. Such an approach could be especially suitable when the impact specialist provides real-time interpretation as part of the warning chain.

One of the challenges faced by intermediaries between warning and impact information is the discussion of uncertainty. An important aspect of building trusted relationships is that there should be discussions of how accurate the forecasts are at different time and space scales. When these conversations are combined with hazard impact and sector impact models, knowledge of the certainty of each of these models and the ways in which the impacts interact will be critical. “The non-communication of these is problematic as interdependencies between them, especially for multi-model approaches and cascading hazards, can result in much larger deep uncertainties” (Doyle et al. 2018). It is important for uncertainty to be communicated effectively to best inform decision-makers and to ensure action is taken that best protects the community.

The full range of uncertainties throughout the warning process must be allowed for (from defining the problem, computational issues, initial conditions, verification and beyond). Scientists must set realistic expectations concerning uncertainties, recognise cultural differences between disciplines, and ensure that engagement develops mutual understandings of the issue and supports decision-makers. “When visualizing uncertainty, the focus must be on the data and uncertainty relevant to the decision” (Doyle et al. 2018).

Communication of uncertainty increases levels of trust (Joslyn & LeClerc 2015). The message should be precise about the sources of uncertainty involved and how to effectively present disagreements between experts in a way that does not minimise the message or credibility. It is also important that the impacts are well-understood by communicators and that they include in their warnings “decision-relevant time frames, including information on when the uncertainty may be reduced” (Doyle et al. 2018). Developing partnerships and communicating uncertainty prior to the need to issue the warning increase trust and confidence.

4.4.3 Exchanging Information About Tools

A general principle across the whole warning chain is that users have greater confidence in warnings if they understand how they were produced. The greatest challenge to achieving that lies in impact prediction, which is often hidden in statistical “black boxes ”. It is therefore an essential part of any partnership between impact modeller and warner to convey the basis of the model, the predictors used and the uncertainty bounds in the predictions. The warner should have access to routine verification and be able to query unexpected results. These requirements place demands on the information produced during tool development and handover and on the availability of ongoing expert support. They also require that users, impact scientists, IT developers and warners are all involved throughout the development process. When a new or upgraded tool is handed over, users should be inducted into its use through presentations and supported hands-on practice. Detailed instructions should be provided, describing its operation, including operating limits and how to deal with any failure modes. It should also be accompanied by a comprehensive test report, together with datasets and any ancillary software required to reproduce the test results. The test report should clearly state the ranges of input data that have been validated and any caveats about tool outputs, including situations where performance will be below the norm, and should identify the sources and magnitudes of uncertainty in the results. In order to guard against overconfident messaging, uncertainty ranges should be provided as standard outputs from the tool. Metrics used in the evaluation should be clearly described, together with the reasons for using them and their limitations.

These technical aspects of handover are important to ensure that the warner does not inadvertently produce misleading information. However, they also contribute to helping the warner to have enough confidence to use and accept the information that is generated. To fully achieve this acceptance, warners should be involved throughout the development process, to ensure that the tool is designed to produce the information that they feel is required, that the developers test the tool in circumstances identified as important by the warners and that performance can be challenged by those who will use it. Ideally the relationship between developer and warner should be personal, but if not, regular contact throughout the development process will help ensure that the tool contributes to better warnings once it is handed over.

4.4.4 Challenges of Evaluating Tools

In order to ensure that warning information is used and useful, it is important to conduct evaluations. The data and models need to provide actionable results, and the results of the models should be validated and verified. Evaluations of each aspect of the system can be complicated, as the warning may be based on integrated, ensemble models with impact scenarios and simulations that are then communicated using various infrastructure and tools throughout the early warning system. Studies have been conducted to determine decisions that are made from warnings, using the determinant that protective action occurred as a measure of success (Gutter et al. 2018). Each of these stages will need evaluation, but finally the decision-making processes must be considered and whether or not action was taken.

4.5 Examples

Box 4.1 A Structure for Warner and Impact Expert Interaction in New Zealand

Cheryl L. Anderson

Research has found that the framework or structure for interactions of the communities issuing and receiving the warnings is critical for ensuring that lives are protected. One example of this type of framework is the New Zealand Natural Hazards Research Platform (NHRP) that was organised to ensure that hazard research and science informed disaster policy. NHRP served as a boundary organisation to facilitate collaboration on disaster risk reduction, with one of the key areas being early warning systems. The interactions of scientists and policy advisors in the boundary organisation aided in developing trusted relationships (Beaven et al. 2016).

The Sendai Framework for Disaster Risk Reduction commits signatory countries to establish coordinating governance arrangements to increase the integration of stakeholders across domains, sectors and levels and to “foster cooperation among scientific and technical communities, other relevant stakeholders and policymakers in order to facilitate a science-policy interface for effective decision-making in disaster risk management (UNDRR 2015, 13). The NHRP facilitated cross-sector collaboration, including the activities of advisory bodies, international climate change and biodiversity initiatives and collaborative approaches to the management of shared resources.

Box 4.2 Research Demonstration Projects at the Olympic Games

Cheryl L. Anderson

The Olympic Games have been used to advance an understanding of the complexities of forecasting and nowcasting since 2000. The WMO World Weather Research Programme (WWRP) organised Forecast Demonstration Projects and Research Development Projects that advanced the development of warning infrastructure, training and use of warning systems and methods for distributing information quickly (WMO 2017). The process involves building relationships with the Olympic committees to understand the end-user needs for the event and developing methods to deliver these needs. The Sydney 2000 Olympics was the first demonstration project, and international teams used the opportunity to install a radar system and learn to provide rapid nowcast warnings, primarily for wind and rain (Wilson et al. 2004, WMO 2017). Knowledge from the Sydney games fed into the Beijing Summer Olympic Games, where improvements in Numerical Weather Prediction (NWP) models, capacity-building in communicating the nowcasts through web interfaces and visualisations and direct weather briefings with Olympic officials established long-term working relationships across the international forecasting community. The Winter Games have provided more challenges. Events such as downhill skiing require wind, precipitation and visibility forecasts at multiple elevations to ensure that events are fair and that competition can proceed. Olympics nowcasting in the 2010 Vancouver Winter Olympics left infrastructure that has benefitted aviation and transportation and advances in forecasting precipitation by improving timing of storms and visibility. It also deepened relationships among forecasters, the Olympics Committee, and events coordinators and managers. Conversations about event needs for information on visibility, snowfall, and wind speeds led to the development of thresholds for making decisions about postponement and delays for each event (Isaac et al. 2014; Joe et al. 2010; Joe et al. 2014; Joe et al. 2004; WMO 2017).

Box 4.3 Linking Fire and Health Impacts to Action in Australia’s Summer of 2019/2020

John Nairn

Australia’s 2019/2020 summer of cascading multi-hazards ceased with flooding rains. Bushfire smoke produced the highest documented human health impact with 417 excess deaths (Borchers-Arriagada et al. 2020) compared to 33 bushfire deaths (Commonwealth of Australia 2020). An extremely active fire season produced unprecedented bushfire intensity, area burnt, significant mortality and property and animals destroyed. Seasonal forecasts set expectations for an extremely intense bushfire season. Fire and emergency services agencies performed rigorous pre-fire season briefings incorporating antecedent climate and seasonal outlook intelligence as the basis for scenario plans, resource allocation and testing of community message systems. Health impact information from the season’s dust, heat waves, fires, persistent smoke and flash floods could benefit from co-design of impact forecast products tailored to public health needs. Public health’s response to the persistent smoke hazard indicated a lack of coordination, with disparate community advice undermining community confidence. An increased focus on pre-season scenario planning would allow the public health sector to achieve the same level of preparedness as is evident with Australia’s fire authorities but extended across multiple hazards.

Box 4.4 Creative Collaborations in Social Media Communication at the UK met Office

Ross Middleham

Creative collaborations and partnerships can help us learn from others, share best practice and accomplish mutual goals. Before entering into a partnership, it’s important to understand your own organisational goals and what activities align with your purpose. At the Met Office, our purpose is to keep people safe and able to thrive, so every decision we make must support this.

Every opportunity starts with a conversation. We actively seek collaborations that can help support our messages, reach new audiences, position us as experts and the authoritative source or bring insight and learning to the organisation. These partnerships can be formal or ad hoc, paid or organic depending on the benefit and impact that will be delivered (Fig. 4.2).

Fig. 4.2
figure 2

Joint Met Office - RAC travel safety video on YouTube. (© Crown Copyright 2021, Met Office)

Full size image

We actively share and support messaging with partners who align with our brand​. The key here is that we have a common aim, so it’s natural for us to share and support each other’s messages. For example, we work with the Royal Automobile Club and Royal National Lifeboat Institute to amplify safety messages.

We actively seek partners who can help position us as a trusted source of information​. For example, we worked with Facebook on their Climate Science Information Centre to become an international partner which sees our climate science content being pulled into their hub. We actively seek creative collaborations that can help our content reach new audiences. We identify people and organisations who share a similar purpose but have their own engaged audience who follow and trust what they say. For example, we approached the Jamie Oliver Group after he mentioned weather and climate on one of his TV programmes and then worked with him to co-create climate and food security content for his 8.4 million followers on Instagram.

Working with others has many advantages, but it’s not an easy thing to do. It takes time: time to identify opportunities, time to build your network and time to develop an idea and make it happen.

The power of LinkedIn and Twitter to approach the organisations you want to speak to should not be ignored. You may need to consider ways to grab their attention, even just to have an initial conversation. That might be doing a mock-up of your idea or sending a demo video. Be prepared for your initial chat by researching the organisation and understanding their objectives. Then act quickly when responding to follow-up emails and idea sharing to maintain momentum.

Partnerships aren’t just about making your own messages go further. For example, we actively seek creative collaborations that inspire and bring insight into the team. Over the years we’ve run lots of workshops in university design studios around the country. We share what we’ve learnt with young people, and in return they give us a different perspective on our problems and give us insight into their worlds, offering us a way to creatively test our ideas. We also actively seek creative collaborations to inspire and drive innovation. For example, we’ve worked with One Minute Briefs to crowd source ideas through mass design participation on Twitter.

In summary, we actively seek creative collaborations to keep us evolving. But why is that important? Because we know that the way we do things now will not stay the same. The digital landscape is becoming noisier and noisier and we continue to fight for attention. Ever-changing algorithms change the way our content is served up on channels, and the way people consume information is continually changing. For example, we’ll soon need to think of ways to reach a whole new user group. The ones growing up gaming, being home-schooled, communicating virtually and who rely on YouTube. We need to work with others to help us understand that audience.

We’ll continue to keep our eyes on the horizon and actively seek opportunities to ensure our information is trusted, listened to and acted upon, helping to keep people safe and able to thrive (Fig. 4.3).

Fig. 4.3
figure 3

Content co-created with Jamie Oliver went out across both organisations’ social media platforms, including Jamie’s 8.4 million Instagram followers. (© Jamie Oliver. Reproduced with permission)

Full size image

4.6 Summary

  • The success of a warning is that people listen, understand the message and use it to take action that protects lives, livelihoods, the environment, property and infrastructure. Impact information is one ingredient in helping this to happen.

  • Expertise in weather-related hazard impacts is widely distributed. Impacts data are often difficult to access and analysis methods can be very specialist. It is therefore important to identify which impacts matter, who has access to relevant data and who has the requisite analysis skills.

  • In order to circumvent issues with proprietary and confidential data, hazard and weather forecasters must be prepared to make their data available to the impact specialist in a form that enables the impact specialist to match it with impact data and develop a model or tool. As the impact data cannot generally be shared, it is essential that the partners have a mutual understanding of what the analysis is trying to achieve, why a tool is needed and how it will be used.

  • Relationships between the “warner” and the “impact expert” can be facilitated within boundary organisations where “honest brokers” serve as intermediaries to effectively translate and convey information.