Near-Real-Time Analysis of Publicly Communicated Disaster Response Information
- First Online:
- Cite this article as:
- Girard, T., Wenzel, F., Khazai, B. et al. Int J Disaster Risk Sci (2014) 5: 165. doi:10.1007/s13753-014-0024-3
- 970 Downloads
Analysis of a disaster event can identify strengths and weaknesses of the response implemented by the disaster management system; however, analysis does not typically occur until after the response phase is over. The result is that knowledge gained can only benefit future responses rather than the response under investigation. This article argues that there is an opportunity to conduct analysis while the response is operational due to the increasing availability of information within hours and days of a disaster event. Hence, this article introduces a methodology for analyzing publicly communicated disaster response information in near-real-time. A classification scheme for the disaster information needs of the public has been developed to facilitate analysis and has led to the establishment of best observed practice standards for content and timeliness. By comparing the information shared with the public within days of a disaster to these standards, information gaps are revealed that can be investigated further. The result is identification of potential deficiencies in communicating critical disaster response information to the public at a time when they can still be corrected.
KeywordsDisaster response information Forensic disaster analysis Near-real-time analysis Post disaster evaluation
Disasters, such as major storm events or earthquakes, trigger an immediate response by the disaster management system of the nation in question. The quality of this response is a large factor in its ability to limit the impacts of the disaster on the local population. Improving the quality of disaster response therefore reduces disaster impacts. Studying past disasters is a valuable exercise to understand what went wrong, identify measures that could have mitigated the issues, and make recommendations to improve future disaster planning and response. The reports that result from this reflective process have a variety of names such as “lessons learned” documents (Birkland 2009), “after action” reviews (Comfort 2005; Donahue and Tuohy 2006), and “ex-post” evaluations (Cosgrave et al. 2009; OECD 2010). Due to the focus on disaster response, this article uses the term “post-response” report (Birkland 2009) to refer to documents that evaluate disaster response activities, make recommendations for improvement, and which are issued after the response phase is over.
A recent effort to study natural disasters that goes beyond the typical post-response reports has been started by the Integrated Research on Disaster Risk (IRDR) program, which was established by the International Council for Science (ICSU) in 2010. These activities, called Forensic Disaster Investigations (FORIN), aim to uncover the root causes of disasters through in-depth investigations. The FORIN working group argues that “thoroughly analyzing cases, including both success stories and failures, will help build an understanding of how natural hazards do—or do not—become disasters” (ICSU 2013). In adopting the IRDR FORIN approach to comprehensive understanding of disasters, the Center for Disaster Management and Risk Reduction Technology (CEDIM) adds a time-critical component to the evaluation process. The goal of the CEDIM Forensic Disaster Analysis (FDA) approach is to understand and assess in near-real-time the evolution of the event where information may be scarce or unclear (Wenzel et al. 2012).
CEDIM recently began a pilot study to include the disaster response as a potentially contributing factor to the overall disaster impact and a methodology was developed for this purpose. Thus, the main contribution of this article is the introduction of a methodology for near-real-time analysis of publicly communicated disaster response information. The term “near-real-time analysis” used throughout this article can be defined as the process of collecting the information available within the first 0–5 days of a disaster, analyzing that information, and producing results within approximately one day of the latest information. For example, if a near-real-time analysis was based on the information available on the fifth day after the disaster, the results of that analysis would be available by the sixth day. The aim of the near-real-time methodology is to base analysis on current information and to produce results while the disaster response is still in operation. This methodology represents a first step in analyzing disaster response within days of a disaster. The application of this methodology following a disaster is intended to enhance disaster response and subsequently reduce disaster impacts. The methodology has already been applied to CEDIM FDA activities following tropical cyclone Phailin in India, as well as the Bohol Earthquake and Typhoon Haiyan in the Philippines.
The next section discusses the importance of post-response reports but also identifies a limitation to their ability to enhance a disaster response. Real-time evaluations and other key concepts that have led to the development of our near-real-time methodology will then be discussed in Sect. 3. The near-real-time methodology is introduced in Sect. 4 along with a classification scheme that was created to standardize the information analysis process. This methodology was then tested in near-real-time following disasters that occurred between July and November 2013, using a process we have termed an “information gap analysis,” the results of which are discussed in Sect. 5. Finally, three factors are identified in Sect. 6 that will need to be addressed to further strengthen the results of analyses.
2 Current Approach
A common method for analyzing disaster response is to carry out post-response evaluations, often referred to as “lessons learned” or “after action” reports. The main purpose of such reports is to identify what changes should be implemented in order to improve future responses. For example, lessons learned from Hurricane Katrina led US Congress to enact a law in 2006 to restructure the Federal Emergency Management Agency (FEMA) (Starks 2012). Subsequent changes led to improvements in FEMA’s response to Hurricane Sandy (Chivers 2012; Starks 2012). Similarly, the post-response report of the 2009 Victorian Bushfires Royal Commission resulted in a AUD 900 million commitment by the Australian Government to implement all but one of the report’s recommendations (Sheales 2010). Post-response reports therefore have major potential to influence change in the disaster management system.
A valuable strength of post-response evaluation is that it is based on actual disaster events. The impacts of the disaster can act as proof for preexisting arguments, such as the often cited need to reinforce school buildings. The events can also bring to light unique issues that were unforeseen during disaster planning, such as the need to alter tsunami warnings in Japan to be more assertive and direct following the Great East Japan Earthquake (Arai 2013).
This article argues that some of the issues raised in post-response reports could be identified within days of a disaster rather than waiting months for a full report to be compiled. The benefit of identifying issues within days of the disaster is that those issues could potentially be corrected during the response phase.
3 Key Concepts
In the field of humanitarian relief, the inability of post-response reports to enhance the response under investigation has led to the development of real-time evaluations. The key aspect of real-time humanitarian evaluations is that they are completed while the operation is still underway. The typical time-frame of such reports is to begin field work 4–6 weeks into a mission and complete a report within a month, while the mission is still operational (Jamal and Crisp 2002). This results in findings that are delivered when they may still make a difference to the on-going humanitarian relief phase. The methodology described in Sect. 4 has taken this key aspect of humanitarian real-time evaluations and applied it to analyzing disaster response. Disaster response has a time-frame of days rather than months, requiring an entirely new approach to produce an analysis so quickly.
Furthermore, real-time evaluations in their current form are aimed at enhancing international humanitarian relief projects. International actors are not a major part of the immediate disaster response phase. It is local governments who are the first institutions to oversee a disaster response. If the local system does not have the capacity to manage the response, then the national disaster management system is typically activated. Only in major events does the international community respond through a response that is subject to national government approval and conditions. This order of response underlines the fact that a nation’s government controls the immediate disaster response, not the international community. Therefore, carrying out an analysis of the disaster response during the first 0–5 days of the disaster also requires a shift in focus from the international actors to the national government and subsequent disaster management system.
Managing a disaster response involves rapid monitoring and evaluation due to the time-critical environment in which emergency relief efforts take place. This results in the disaster management team making quick decisions and taking rapid actions. The details of this process, the decisions made or actions taken, and the reasons for them are often not made public until well after the disaster, if at all. Yet, as an outsider, aspects of the disaster response can be observed within hours, through news media, social media, and, in particular, time-critical information disseminated by the disaster management system.
The role of information in any disaster response is similar regardless of the location, scale, or type of event. The typical response involves gathering information to understand the impact of the disaster and carrying out actions based on that information to reduce human suffering and protect or restore a variety of systems such as transportation, healthcare, and communications. Production of information following a disaster is therefore crucial to any response. A key concept in this research is that communicating a portion of this information to the public should be a core function of any disaster management system. Information is arguably required by the public during any disaster, regardless of where that disaster occurs. This argument is supported by disaster communication literature. For instance, Appleby’s (2013, p. 9) evaluation of the response to the Great East Japan Earthquake stresses the importance of communicating information to the public and concludes that “information saves lives, that communication itself is a form of aid…” Maxwell (2003) explains that better informed citizens are able to make the correct decisions to protect themselves during disaster situations, which has the added benefit of reducing the strain on government resources. Helsloot and Ruitenberg (2004) go further to argue that the flow of information prior to and during disasters should be directed at the average citizen, because they are major actors in rescue and relief.
When the analysis is completed, while the response is still in operation, it can inform the future direction of that immediate response. In fact, analyses already occur within days following disasters. CEDIM FDA activities involve estimates of potential disaster effects within days of a disaster, such as social and economic impacts and building damage (Wenzel et al. 2013). The insurance industry also conducts catastrophe modelling immediately following many disasters to estimate insured losses. The next section describes the methodology developed to carry out a similar near-real-time analysis of publicly communicated disaster response information.
Simulated near-real-time analyses using ReliefWeb
Date of disaster
29 Aug 2005
02 May 2008
Dec 2010/Jan 2011
22 Feb 2011
18 Sept 2011
Tropical Storm Washi
16 Dec 2011
06 Feb 2012
29 Oct 2012
07 Sept 2012
04 Dec 2012
4.1 Development of Classification Scheme
During a disaster situation the public will need to make critical decisions regarding what to do, where to go, how to get there, and so on. While a general assessment of the situation may give the public an indication of the scale of the event, many of the important decisions the public makes during a disaster must be based on specific information. For example, rather than a general description stating that some roads may be flooded, the public needs to know where exactly so they can plan alternative escape routes. In addition to identifying the number of casualties, the public should also know how those casualties occurred so they can better understand the risk to themselves. In order to analyze the extent to which the various information needs are addressed, a classification scheme for the information needs of the public during a disaster was developed.
Three main categories have been derived from the concepts observed: general disaster information, effects to people, and effects to critical systems. These details help the public make critical decisions in times of disaster to maintain or improve their or others’ well-being. The category “General Disaster Information” describes the general characteristics of the event in either past, present, or predicted future state, and identifies the affected areas without getting into the details of the actual impacts. Examples of general disaster information include magnitude and location of an earthquake or the predicted path of a typhoon. The category “Effects to People” covers human life safety and basic human needs. This information addresses the direct disaster impacts on physical well-being. Items include casualties or potential casualties in the near future, and disruptions to those things that if left without could lead to casualties, such as lack of drinking water, food, shelter, and sanitation. Finally, “Effects to Critical Systems” covers those systems which are important to the immediate well-being of the local population that if left without could also lead to further casualties. Transportation, medical, and communication disruptions therefore signify a potential reduction or elimination of the ability of the local population to maintain or improve their physical well-being. For example, transportation disruptions hamper the ability to leave hazardous areas or access aid, medical disruptions limit the ability to receive emergency care, and communication disruptions restrict the ability to request help.
Once sorted, further content analysis was conducted to categorize the information contained under each concept. The resulting categories are referred to as properties. Three types of properties were observed: basic data, analysis, and root causes. “Basic data” makes up the majority of the information and answers the questions of who, what, where, and when or how long. “Analysis” describes results of inquiry or measurement, such as explaining how disruptions occurred, or identifying levels of needs satisfied or outstanding. “Root causes” identify why aspects of the disaster occurred.
4.2 Best Observed Practice Review
Content analysis of all 11 disasters and comparison between them resulted in the establishment of best observed practices for what properties should be associated with each concept and when they can be provided following a disaster. “Best observed practice” refers to the variety of information properties that have been witnessed in practice following the 11 disasters reviewed. This is contrary to best “potential” practice that would need to be based on all disasters as well as systematic research into the information needs of disaster affected communities. Basing the information needs of the public on observed practice confirms that delivery of that information has already been proven feasible. But in order to further develop this classification scheme, future research must consult the expertise of individuals who have been through a disaster situation in order to confirm what information they needed that may not be captured in this classification scheme, which may therefore be extended in the future.
Information needs of public: properties of each concept
Subcategories and concepts
Set of properties/Questions to be answered
General Disaster Info
General disaster info
a. Predictions, description of event & current situation
b. Affected areas
What are the characteristics of the event, where will/did they occur, and when?
So what? (Why are these details important?)
How come details result/ed in effects?
Who/where is confirmed to be affected, and by what?
Effects to People
Human life safety (and potential threats)
c. Life saving response
d. Handling of fatalities
How many casualties, who are they, and where are they located?
How were they injured, killed, or missing?
How come they were vulnerable?
What are the dangers, where are they expected, and when?
How do the details equal a threat?
How come they are vulnerable?
Who is doing what to combat threats (evacuation, SAR), where, and when?
How will this combat the threat?
What is being done to deal with fatalities and when?
How do actions meet needs? (outstanding needs)
Basic human needs
Who/where is affected, by what, and for how long?
How did they occur?
How come the system was vulnerable?
Who is doing what to meet the basic needs of affected, where and when?
Effects to Critical Systems
Transportation, Medical services, Communications, for each:
What are the disruptions, where, and for how long?
How did they occur?
How come the system was vulnerable?
Who is doing what to meet the associated needs of those affected, where and when?
The best observed practice review revealed that basic information can be delivered within 1 day (24 h) of a disaster for all categories. Analyses by the disaster management system regarding how impacts occurred were observed within the first 1–2 days but those regarding outstanding needs were often missing in the first five days of the response. If a disaster management system is unable to identify the outstanding needs then it is very difficult to confirm the extent to which the response is meeting the needs of the affected population. Root causes were extremely rare to find in the first five days of the response, which is understandable considering this information does not typically help the immediate response. Nevertheless, the reasons for why the disaster generated the effects it did are critical pieces of information and in-line with the FORIN approach. For example, the low casualties observed in the aftermath of Cyclone Phailin were identified as being the result of a good warning system and excellent coordination between agencies that successfully evacuated almost one million people prior to landfall (Oxfam 2013). This information is very important to disaster-risk-reduction activities, which attempt to learn from both failures and success by understanding the root causes of each. Although this methodology focuses on the first five days of the disaster, including root causes as properties of the applicable concepts ensures the need for this information in the future is identified.
A set of properties is unique to each concept, but is the same for any disaster. Thus, Table 2 is a template that does not change regardless of the disaster being analyzed. Comparing the properties produced in an ongoing disaster with Table 2 results in the identification of questions that remain unanswered, referred to as “information gaps.” Two types of information gap analyses have been carried out so far, as discussed in the next section.
5 Application and Results
02 July 2013
24 Sept 2013
Pakistan National Disaster Management Agency, ReliefWeb, and other sources: Al Jazeera news/radio, BBC, CBC, CNN, The Guardian, RTnews
12 Oct 2013
CEDIM FDA Report no.2 issued 24 Oct 2013
15 Oct 2013
Earthquake-Report and ReliefWeb
CEDIM FDA Report no.6 issued 02 Nov 2013
08 Nov 2013
National Disaster Risk Reduction and Management Council
CEDIM FDA Report no.2 issued 13 Nov 2013
The publicly communicated disaster response information following each of the five disasters was therefore compared to Table 2, in what we have termed an “information gap analysis.” The following discusses the two types of information gap analyses established, as well as a comparative analysis to be carried out upon completion of the first type to improve or clarify the results.
5.1 Missing Information
Following landfall, reports uploaded to ReliefWeb by various news and relief agencies, such as Act Alliance, Agence France-Presse, European Commission, Reuters, Sphere India, and Times of India were reviewed to identify key disaster messages. The content of each message was then sorted into the properties of the corresponding concept. For example, Banerji (2013) quoted a government official stating that “…17 deaths were due to people being crushed by falling trees, walls, roofs.” This message was sorted under casualties as “basic data” for identifying how many casualties and “analysis” for explaining how the casualties occurred. This process was done for approximately 350 key messages.
Comparing the overall results with the best observed practices identified in Table 2 reveals information gaps that can be further investigated. For example, some concepts, such as “meeting basic human needs” and “meeting transportation needs” are missing analysis of outstanding needs, signifying that the extent to which these actions are meeting the needs of the affected population is unknown. Post disaster needs assessments could therefore be recommended to focus on these concepts. Other fields of inquiry, such as why there was no information concerning medical disruptions or the handling of fatalities, could also be investigated.
The presence of full bars in Fig. 4 does not mean that all information has been provided and that no further details are required. The information in each category will increase with each passing day of the response. The full bars only indicate that each question has been answered; however, the answers may be incomplete or inaccurate leading to the need for further comparative analysis as discussed in Sect. 5.3.
5.2 Value of Information
The major benefit of the second type of analysis is that it allows for comparison between disasters and that such comparisons can be made within days following a disaster. Because time is such a major component, analyses of the second type are better suited for disasters with a finite starting point, such as earthquakes, volcanic eruptions, storm events, and flash-floods. Other events, such as widespread prolonged flooding or droughts could be limited to the first type of information gap analysis.
5.3 Comparative Analysis
The information gaps illustrated in Fig. 4 identify the questions from Table 2 that are clearly unanswered; hence, the focus is on the information that has not been provided. Further understanding of the information needs of the public can be obtained by carrying out a comparative analysis of the provided information. Two important factors that must be examined are the quality of the information provided and the relevance of the information gaps identified.
Information quality can be analyzed in terms of the level of detail, coverage, and accuracy. The level of detail refers to the amount of questions answered from Table 2 and is therefore revealed through the first type of information gap analysis. The coverage refers to the percentage of actual disaster impacts and subsequently required disaster response activities that are identified. For example, information that focuses on a city may leave out rural areas where disaster impacts have occurred and in which response activities are therefore required. Issues regarding coverage and accuracy of the information can potentially be identified by comparing the information contained under different concepts. The basic human needs and transportation, medical, and communication system subcategories have intentionally been split into “disruptions” and “solutions” for this purpose. For instance, the near-real-time analysis completed three days after the 2013 Pakistan Earthquake compared the information provided under basic human needs “disruptions” and “solutions.” Although the information provided under “solutions” discussed provision of relief goods, it appeared they did not match the quantities identified under “disruptions.” In particular, a UNOCHA (2013) report issued two days after the earthquake stated that the government had dispatched 7,600 tents. At the same time, it was reported that 21,000 houses had been destroyed (Saifi 2013) and over 100,000 people made homeless (Agence France-Presse 2013). This represents a “coverage” issue, as the solutions do not appear to cover the full extent of the disruptions. Further information would therefore be required to explain how the shelter needs will be satisfied for those who cannot be accommodated by the tents.
Accuracy issues can also potentially be identified if different information sources have conflicting information. For instance, three days after Typhoon Haiyan made landfall in the Philippines the official confirmed death toll was only 255 with 38 missing (NDRRMC 2013); a day earlier a local official estimated the death toll to be 10,000 (Reuters 2013). Based on this discrepancy, the accuracy of both figures could be called into question.
Comparing between concepts can also help to establish the relevance of the missing information. In some cases, information gaps do not need to be filled. For example, if there are no medical disruptions then there is no need to identify medical solutions. Conversely, if there are large areas that have experienced communication system failure, then identifying solutions to communicate with those potentially affected would be very important.
This information gap analysis represents a starting point. The framework of the classification scheme then allows for more in-depth analysis by comparing within and between subcategories. The result is identification of additional information needs to account for coverage or accuracy issues, and a better understanding of how relevant the information gaps are.
Figure 5 illustrates the variety in results of the analyses carried out so far, the following section discusses three factors to be considered prior to directly comparing results. Addressing these items for future analyses is planned in order to improve the accuracy of results and strengthen comparisons.
6.1 Disaster Type
The potential to compare between disaster types, such as storm events on the left of Fig. 5 with earthquakes on the right, needs to be further investigated. Information for storm events begins flowing prior to landfall, particularly for the categories “monitoring and prediction,” “warnings,” and “life saving response.” For example, the predicted path, timing, and wind speed of Cyclone Phailin was provided two days before landfall (Thomson Reuters Foundation 2013), and warnings to fishermen were issued (Act Alliance 2013) and evacuations underway (Sphere India 2013) one day before landfall. In contrast, earthquakes occur suddenly, resulting in reactionary disaster response messages rather than precautionary. The ability to compare between disaster types will be further investigated as part of future research.
6.2 Information Source Limitations
ReliefWeb was the major source of information for the initial 11 disasters analyzed. Most reports on ReliefWeb are in English, with some in French and Spanish, and a very limited amount in Arabic. Information contained in reports of other languages is therefore unknown unless the reports have already been translated. Furthermore, since a primary purpose of ReliefWeb is to inform humanitarian assistance providers (ReliefWeb 2014), there are fewer reports on disasters that do not require international aid. A combination of both of these issues is potentially why only one update specific to Germany was found in relation to the June 2013 flooding. Another issue with ReliefWeb, but also with disaster data in general, is that currently it appears disasters need to be large in scale in order to generate enough publicly available information to assess. For all of the above reasons, other information sources are continuously being investigated.
Different sources of information were therefore used to analyze the five disasters, as identified in Table 3. Analyses for the Aceh Earthquake and Cyclone Phailin used only information found on ReliefWeb, while the Pakistan Earthquake and Bohol Earthquake used additional sources of information. The analysis for Typhoon Haiyan used only information found in the NDRRMC situation reports available on their website. Although the reports appeared to be comprehensive, it was found that other sources provided some critical information that the NDRRMC situation reports were missing. This was revealed by an additional review, which compared an information gap analysis based on only NDRRMC situation reports with one that also included ReliefWeb sources. The analysis was also extended beyond the typical first five days to include the information produced within 12 days after landfall. During this time no basic data was provided in the NDRRMC reports that described the disruptions to the medical system. This information was provided by other agencies, and is included in reports by International Medical Corps, World Health Organisation, United Nations Population Fund, International Organisation for Migration, Agence France-Presse, Médicines Sans Frontièrs, and UNOCHA. Most detailed were the lists of medical disruptions and needs provided by Humanity Road six days after landfall. This finding highlights the importance of reviewing all information sources to achieve an accurate analysis of publicly available information.
6.3 Local Context
The near-real-time analysis of the 2013 Pakistan Earthquake used data from the Pakistan National Disaster Management Agency website, ReliefWeb, and a variety of news network websites in an attempt to access all available information. These sources, and those used for all of the analyses so far, have been retrieved via the internet, which represents a global information source; however, the focus of the analysis is on the critical information needs of the local population. This raises two questions, the first of which is: what information on the internet is available to or actually reaches the local population? Local populations may not have internet access or they may only have access to a limited amount of online information. For example, affected populations without working computers may be able to access social networking sites such as Twitter and Facebook with cellular phone applications, but cannot download important situation reports. The second question is: is the affected population receiving local disaster messages that cannot be found on the internet? Local sources are very important information sources for an impacted population, and include formal and informal information providers. Formal sources may include local radio, newspapers, posters, or loudspeaker broadcasts. Informal sources may include neighbors, friends, and family with individuals accessing these sources in person, through phone calls or text messaging. Therefore, examining what information is actually reaching the local population from both the internet and local sources will help to improve the accuracy of future analyses.
Near-real-time analysis of disaster response is a new field of research. It is complementary to later in-depth post-response analysis and to real-time evaluation as done by international relief agencies. Near-real-time analysis will help to learn from disasters, to appreciate achievements and understand deficiencies in response by local agencies and institutions. As a first step towards these goals, this article has introduced a methodology for conducting a near-real-time analysis of the publicly communicated disaster response information. The establishment of a disaster data classification scheme has supported this analysis and has led to the development of standards in information production following any disaster. Comparison to those standards has resulted in the identification of information gaps, a process termed an “information gap analysis.” Two types of analyses were discussed. The first type is intended to reveal the missing information by identifying which typical questions from the public remain unanswered. Further comparative analysis can potentially reveal issues regarding quality of the information provided and relevance of the information gaps identified. The second type of analysis calculates a value for the information provided to facilitate comparison of results between disasters. The intent is that such comparisons can act as a first step in measuring the performance of those responsible for gathering and distributing critical disaster information to the public. Due to the potential for incomplete or inaccurate information in the first 0–5 days of the response the near-real-time analysis methodology may be limited to identifying the deficiencies that are clearly evident. The advantage of the methodology is that those deficiencies are identified quickly, at a time when they can still be corrected.
This article is published under license to BioMed Central Ltd.Open AccessThis article is distributed under the terms of the Creative Commons Attribution License which permits any use, distribution, and reproduction in any medium, provided the original author(s) and the source are credited.