1 Introduction

Disaster response training and exercises have the potential to enhance the resilience of a society confronted by natural and man-made calamities because they incorporate all four cornerstones of resilience engineering: (1) monitoring, (2) response, (3) learning, and (4) anticipation (Hollnagel 2009). Disaster response training and exercises not only entail a learning process that is expected to improve disaster response abilities but also provide monitoring opportunities to identify flaws and inadequacies in an emergency response system. The preparation of training and disaster scenarios requires the proactive anticipation of possible disaster situations and speculation about likely risks, as well as developing countermeasures. Moreover, because the periodical implementation of training and exercises is mandatory or highly recommended in many sectors (such as government agencies, medical institutions, commercial companies, and local communities), these activities can serve as a driving force for sustainable and continuous development of the four cornerstones of disaster resilience.

However, the achievements reported to date have not been as significant as expected (Hsu et al. 2001, 2004; Williams et al. 2008). While numerous efforts have been invested in creating and delivering training and exercises, their effectiveness has been limited, even though they include various types of drills and exercises (Michael and Chen 2005; KMC 2016; Kobayashi et al. 2008; Yamagawa et al., 2016; Yamori 2012). One of the reasons for this is that insufficient attention has been given to creating scenarios to describe various assumptions about damage and obstacles under the circumstances of a catastrophe. In fact, the creation of these scenarios depends heavily on developers’ expertise, and designing the scenarios and preparing appropriate exercises entails high costs and extensive person-hours. Many organisations and communities do not have enough know-how and human resources to prepare training and exercise materials; as a result, similar scenarios tend to be repeated, with minor modifications. As several studies suggest, knowledge and skills acquired in similar and standard scenarios are ‘brittle’ rather than adaptive (Casner et al. 2013; Landman et al. 2017), and more variability in training scenarios is important to boost skills needed to cope with unexpected and novel situations (Landman et al. in press). Furthermore, continuous improvements of the exercises are further impeded—at least in Japan—because it is uncommon for organisations and communities to share with each other their experiences and knowledge on the preparation and execution of training and exercises. This is partly because no foundation and easy-to-use technology is available to support and facilitate knowledge sharing and collaboration among different organisations and communities. To address these problems, it is necessary to develop a standard framework and methodology for creating various disaster scenarios. It is also necessary to promote collaborative disaster resilience management by facilitating the exchange of good scenarios and assumptions between different organisations and communities.

This study proposes a disaster context model that provides a framework for describing actual and imaginary disaster situations, as well as for creating and sharing new scenarios. Employing this model, this study also proposes a support system for generating scenarios and materials for disaster exercise in an interactive and semi-automatic manner, using a hospital as a case study.

Section 2 of this paper reviews the current conditions and problems associated with disaster exercises in Japanese hospitals. Section 3 explains the modelling of a disaster context. Section 4 introduces a prototype of the support system for producing disaster scenarios and exercise materials for hospitals. In Section 5, a qualitative evaluation method appropriate for sociotechnical systems is used to evaluate the proposed system. Section 6 encompasses the discussions and conclusion.

2 Problems with disaster exercises in hospitals

Japan’s legal framework for disaster management (Director General for Disaster Management 2015) decrees that each prefectural government designate disaster base hospitals (DBHs) to fulfil the requirement stipulated by the Ministry of Health, Labour and Welfare for ensuring the viability of emergency operations, equipment, and facilities (Ministry of Health, Labour and Welfare 2012). For example, 80 DBHs are currently designated in Tokyo (Tokyo Metropolitan Government 2014). In the event of catastrophe, they are expected to play a central role in providing medical services in the local region, as well as in peripheral regions affected by the disaster. These DBHs stipulate the implementation of various types of emergency training and exercises in their disaster preparedness action and business continuity plans (Bureau of Social Welfare and Public Health 2015). In most cases, other types of hospitals also voluntarily carry out disaster training and exercises, or at the very least perform mandatory fire drills under the Fire Service Act. Therefore, disaster training and exercises are very common and periodically conducted in almost all Japanese hospitals. However, in most hospitals—including DBHs—no personnel or departments specialise in preparing and implementing disaster exercises. Medical staff, including doctors and nurses, usually perform these tasks in between their regular duties, and doing so has proven to be a very big burden for them.

2.1 Disaster scenarios and exercise materials

For almost all kinds of disaster exercises, it is necessary to prepare scenarios that describe the extent of the damage inflicted on the hospital and its neighbouring areas, as well as the assumed responses of related agencies. In addition, materials such as timetables and situation cards are needed to conduct the exercises. Once a disaster scenario has been created, it is not technically difficult to automatically develop related materials (Hada et al. 2004; Sakamoto and Takanashi 2005). Many computer-based systems are available to support, record, and document an efficient and effective disaster exercise once a scenario has been developed (Endo et al. 2009; Zhou et al. 2015). Producing new disaster scenarios is more important and laborious. Several studies (Barbat et al. 1996; FEMA 2018) have generated disaster scenarios; however, the purpose of these tools is to estimate damage and loss, and make a hazard map at the regional level. Thus, they are not directly applicable to the development of exercises for a disaster scenario that is specific to hospitals. Workshop-based disaster risk assessments, such as the Disaster Imagination Game (DIG) (Yanagawa et al. 2016), can be used to devise disaster scenarios that consider local and hospital characteristics, yet it is difficult to draw up a scenario that incorporates changing situations in a single workshop. Moreover, workshop-based methods are very time-consuming. Some guidelines describe a method for composing disaster scenarios for hospitals (Japan Red Cross Society 2006; Metropolitan Chicago Healthcare Council 2006); however, it is based on a manual procedure and does not contribute to reducing workload. Hence, there is a great need to develop tools that can be used to create disaster scenarios for hospitals.

2.2 Feedback and knowledge sharing

In large hospitals like DBHs, small-scale exercises are usually conducted within each department or division, in addition to annual, large-scale exercises organised by the hospital. However, it is uncommon for these different units—including different hospitals—to share their experiences and the knowledge they have gained when preparing and executing the exercises. It has proven practically difficult to accomplish such an exchange without some support system due to the number of departments and divisions in a large hospital.

3 Modelling the disaster context

This paper uses the term disaster context to refer to all the physical and functional elements and events that constitute a disaster situation. The activities undertaken in response to a calamity can be regarded as events and included in the disaster context.

Based on content analysis of past disaster exercise scenarios, reports on past catastrophes (Ojiya General Hospital 2004; Kesen-numa City Hospital 2012), published personal notes on actual experiences (Japan Nursing Association 2011; Japan Red Cross Ishinomaki Hospital 2011), and some models that can be used to summarise scenarios (Kanno and Furuta 2010), we identified three major categories of the constituents of a disaster context: (1) situations, (2) tasks, and (3) problems.

A schematic illustration and formalisation of the disaster context model is shown in Fig. 1 and Table 1, respectively. A scene in the figure refers to the snapshot of a context at any point in time. As represented in Table 1, a context consists of changing scenes that can be formalised by ordered scenes that depict the different statuses of the constituent elements. A scene can be formalised as a triplet consisting a situation, tasks, and problems. We extended these categories further by employing existing models and frameworks, which are explained in the subsequent subsections.

Fig. 1
figure 1

Schematic illustration of the disaster context model

Full size image
Table 1 Definition and formalisation of the disaster context model
Full size table

Using the example of a hospital setting developed for the prototype system, the details of each category are explained, along with instructions on how to build the model for a specific domain.

3.1 Situation

A situation describes the statuses and conditions of the background environments and objects/actors in relation to a disaster scene, which can be formalised as a set of these constituents (Table 1). Each constituent can be formalised and represented according to a frame model with a set of slots and attribute values. In a hospital setting, object elements include hospital rooms, buildings, the premises, hospital facilities, medical devices, and medical and pharmaceutical supplies. Medical staff, inpatients, and outpatients are the major actors in the settings, which are also represented as objects. Background environmental elements are also important when constructing a disaster context; these include the season, weather, and time, as well as information related to the objects and actors outside the main target facilities.

Each object and actor was implemented using an object-oriented modelling approach in the prototype system. Table 2 presents the patient model, which has a frame with slots and values. For example, the combination of values in Table 2 generates various types of patient instances such as an injured person, an elderly person, and a person with a chronic illness. Table 3 presents the model of a sickroom, which consists of architectural elements such as doors and windows, as well as various objects and actors. Other model elements, such as the environment and tools, were implemented using the frame model in the same way. Each object and actor is associated with variables representing the object and/or actor’s conditions and statuses. The collection of various objects/actors and background conditions can represent different situations in and around a hospital.

Table 2 Patient model
Full size table
Table 3 Sickroom model
Full size table

3.2 Tasks

The general meaning of a task is an action or series of actions carried out to achieve a goal. The tasks undertaken in the example domain can be categorised under two headings: (1) regular and (2) emergency. Regular tasks are conducted daily, such as a doctor’s and nurse’s rounds, surgery, and attending to the outpatient clinic. The demand for some of these regular tasks increases during disaster situations, when various difficulties can arise. Emergency tasks are those required only under the circumstances of a calamity to cope with the problems caused by the disaster such as triage, safety confirmation of staff and inpatients, fire-fighting, and evacuation. Several standard models, such as a flow chart and hierarchical tree representations, describe the structure and content of a task (Kirwan and Ainsworth 1992; Shepherd 2001).

A task is defined as a triplet of a goal, a situation, and prerequisites. A prerequisite is a condition required beforehand to do the task—such as the required manpower and resources, as well as the person in charge (PIC) of completing the task. A sequential model that described a task procedure as a simple serial process (an ordered set) was used to develop the prototype system. Altogether, 38 emergency tasks were listed with the help of subject matter experts (SMEs). Table 4 presents several examples of the emergency tasks in the prototype system.

Table 4 Examples of tasks
Full size table

3.3 Problems

In this study, problems refer to situations that prevent someone from accomplishing tasks as planned or desired. These include task constraints such as resource shortages, and obstacles that emerge in specific disaster situations and prevent task execution, for example, walls collapsing, water leakages, and fires. These issues are associated with the prerequisites of tasks and the background situation, and can be described as relationships. In the prototype system, these relationships were described in a relational database. Table 5 displays several examples of problems embedded in the prototype system.

Table 5 Examples of problems
Full size table

4 Support system for creating disaster scenarios

This section introduces an example of an application of the disaster context model: a prototype system to semi-automatically generate a new imaginary disaster scenario, which can serve as a base assumption for disaster training and exercises. A hospital in the situation model was developed based on an actual DBH. This prototype was developed with the Java platform, using the disaster context database built on MS Excel. Figure 2 portrays the system architecture and a flow chart of its usage. The following subsections explain each step and its supporting functions.

Fig. 2
figure 2

System process and architecture

Full size image

4.1 Inputting design requirements for an exercise

The user’s first step is to input design requirements for a disaster scenario, as well as basic information for the hospital and surrounding environments, according to the disaster context model explained in the previous section. In this step, a user is required to edit an XML file of the situation model by specifying a certain set of values for some slots, which will represent detailed information on the hospital and background conditions such as the season, weather, time of the disaster, the structure of the buildings, and the numbers of staff and patients. Next, the user needs to enter information documenting the initial damage to buildings, facilities, and persons that should be considered in the exercise. The user must also select training tasks from the task list stored in the task database, or create new ones using the task model, and specify the starting time for each task within the created context.

4.2 Automatic creation of disaster situations and the search for problems

After the user has entered the required information, the system assigns values to the remaining slots in the situation model, considers the consistency of their combination, and generates a complete initial situation. To exclude the possibility of inconsistent situations, several logical rules—such as ‘no snowfall in summer’—are incorporated into the prototype system. The system then searches for candidate problems in the relational database by referring to the background situation and prerequisites for the selected tasks. Next, the system modifies the initial scene to produce another one. A situation can have four basic modes of change: (1) static, (2) escalation, (3) de-escalation, and (4) cyclic. Unless the user specifies one of these modes, one will be randomly applied to each variable to engender a continuously changing situation comprised of numerous situations. Finally, a disaster context is outputted in an XML format and an MS Excel table, as shown in Figs. 3 and 4.

Fig. 3
figure 3

Example of a disaster context

Full size image
Fig. 4
figure 4

Problems related to a disaster context

Full size image

4.3 Editing and modifying a created scenario

This system was designed as a human-in-the-loop system in which users become proactively involved in the scenario design process by editing output disaster scenarios to fit with their purposes. Since it is technically very difficult to assure perfect consistency in the situations produced premised on a frame-based model, users are required to find and modify obvious inconsistencies in the output context. Further, and more importantly, because this process can foster the ability to design a scenario and anticipate possible events, which is believed to be important for high resilience, it is advisable to involve users when generating scenarios. In addition, users can add and edit details of the contents at any step in the procedure, which enables the creation of disaster assumptions that extend beyond the data stored in the database.

4.4 Output examples

We constructed databases for the situations, tasks, and problems based on information for an actual DBH using inputs from SMEs, and developed a prototype system. Figure 3 shows part of the context created by the system. The first and second columns in the figure represent the hospital’s constituent elements, and the remaining columns indicate the values for each element, with each column corresponding to a single scene. For example, in this context, the broadcasting facility in ‘Ward 1’ is damaged and unavailable from the beginning of the scenario until the third scene, and only becomes available in the fourth scene. Details of other objects and actors are also represented in this way. Figure 4 shows an excerpt of the output problems associated with the created disaster context. For example, the third task is an on-site announcement that requires a broadcasting facility, which is not available until the third scene. Therefore, a problem, namely ‘on-site announcement is not available’, was searched for and shown during the corresponding periods. The bottom part of Fig. 4 displays non-task-related problems that were nonetheless induced by the context. ‘Sickroom 1’, for example, has problems related to ‘difficulty with movement/transfer’ and ‘risk of injury’ owing to the room’s damaged ceiling and window.

4.5 Narrative outputs

A function to automatically engender a narrative/textual summary of a disaster context was developed in the prototype system. The technique used in this function is simple; a framework of a story with ‘fill-in-the-blank’ sentences to summarise a disaster context, as well as several short textual explanations to describe the status of situation elements, is prepared. Once a disaster context is produced, the textual explanation of the model elements corresponding to their values is selected and filled in the blanks so as to create a narrative/textural output. Figure 5 shows an excerpt of a summary generated by the function, where the words with angle brackets represent values of model elements, and the underlined sentences represent brief, prepared explanations for model elements.

Fig. 5
figure 5

Narrative description of a disaster context

Full size image

5 Qualitative evaluation

This section presents qualitative evaluations of the proposed context model and support system. The purpose of this evaluation is not to validate the effectiveness of the prototype system since it is still in the early stages of development and is not in practical use. Rather, this evaluation focusses on its potential use. The first evaluation assesses the comprehensiveness and descriptive power of the context model relative to the disaster assumption model provided in the existing exercise planning manual for hospitals. Based on the assessment method for sociotechnical systems developed by Komatsuzaki et al. (2003), the second evaluation estimates the potential impact of using the prototype system.

5.1 Comprehensiveness and descriptive power of the context model

This evaluation judges whether, and if so how, the context model can describe the elements that should be considered in the disaster assumptions provided in the Japan Red Cross Society’s exercise planning manual (Japan Red Cross Society 2006). Table 6 presents a comparison of the descriptive power of the guidelines in the exercise planning manual and the context model. The first column lists the elements of a disaster assumption to be considered, while the second column provides examples of the details included in the guidelines. The third column describes how the proposed context model would represent the issues listed in the guidelines. As shown in the table, it is possible not only to describe all the items and issues but also to provide further details on the items using the proposed context model. In addition, the proposed model is object-oriented so it is relatively easy to extend its scope and descriptive power.

Table 6 Comparison of the descriptive power
Full size table

5.2 Potential impacts on society

We conducted an interview-based analysis and evaluation of the prototype system by employing an impact analysis method for sociotechnical systems, as proposed by Komatsuzaki et al. (Japan Red Cross Ishinomaki Hospital 2011). This method is to be used in the design and prototyping phase of a new sociotechnical system, and aims to assess the feasibility of the system as a solution for social problems. With this method, a scenario of the intended or expected use of the system, and how it is employed to influence society, is made and described in a network format. Interviews with SMEs are then conducted to validate the forecast scenario. After the interview, analysts rate the comments corresponding to each link using a Likert-type scale, and the results are examined to judge the system’s feasibility and its potential impacts on society.

Table 7 summarises, and Fig. 6 illustrates, the forecast scenario contents and corresponding network used in the current study. The encircled number on each link in Fig. 6 represents a link ID that corresponds to a partial causal scenario in the entire forecast scenario. We had a group interview with four nurses who are SMEs on disaster nursing. During the first 30 min of the interview, we explained its purpose, after which we demonstrated the prototype system and several output disaster scenarios. Following the explanation of the forecast scenario and the causal network, we then asked the participants to evaluate and comment on each part of the forecast scenario that corresponds to each link in Fig. 6, from the angle of feasibility and the obstacles to implementation. The entire interview lasted about 120 min.

Table 7 Scenario contents
Full size table
Fig. 6
figure 6

Causal scenario network

Full size image

Afterwards, we transcribed all comments and two raters assessed the comments that correspond to each link using the seven-level scale shown in Table 8. In the rating process, two people first analysed the results independently. Next, if they disagreed on the ratings, they determined a single rating through further discussion.

Table 8 Rating scale
Full size table

5.2.1 Results

Through the interviews, we obtained 66 comments pertaining to the links of forecasted scenario. The Cohen’s weighted Kappa coefficient (Cohen 1968) that is used to measure inter-rater reliability for ordinal scales by the two raters was 0.60, which indicates good reproducibility and reliability of the ratings.

Almost all of the comments were positive (above 0); only 3 were negative or 0. This outcome suggests that in general, the SMEs supported the validity and feasibility of our proposal. All of the comments and ratings for links No. 1, 2, and 3, which are directly related to the system’s main functions, are shown in Tables 9, 10, and 11, respectively. Figure 7 depicts the average values of all ratings on each link.

Table 9 Comments and ratings for Link No. 1 (Avg. 1.7)
Full size table
Table 10 Comments and ratings for Link No. 2 (Avg. 1.0)
Full size table
Table 11 Comments and ratings for Link No. 3 (Avg. 1.8)
Full size table
Fig. 7
figure 7

Results of the ratings

Full size image

A negative comment on link No. 2 suggested that enabling the formulation of various disaster assumptions is useless if the effort required to modify and edit the disaster contexts created automatically exceeds the current effort, even though thousands of assumptions are readily available. A negative comment on link No. 6 pointed out that although all the functions of the prototype are beneficial, good facilitators are also needed to conduct an exercise efficiently.

One possible solution to the first problem is to provide an appropriate support function and user interface. As mentioned in Section 4.3, it is impossible for the proposed method to eliminate all inconsistencies that arise in the automatic creation of a disaster context. However, it is possible to keep the effort required to edit and modify them to a reasonable level by strengthening the filtering function. The latter issue is beyond the scope of the current study, yet the disaster contexts and related materials created by the system can help facilitators better understand the content of the exercise scenario, which is expected to reduce the burden on facilitators.

We also obtained several suggestions for further development, as follows:

  • A function to add multimedia (such as pictures and video clips) to represent disaster contexts is desired.

  • A function to record variances from the expected responses is desired.

  • The system can also be used in minor and peripheral hospital divisions (such as the security and nutrition divisions).

  • The system is good for visualising and monitoring the state of implementation and the achievements of exercises and training.

6 Conclusions

This paper proposed a disaster context model that provides a standard framework for describing disaster assumptions and situations to be used for disaster exercise scenarios. The model is composed of three basic elements: (1) the situation, (2) tasks, and (3) problems, and a disaster context is represented as a series of successive scenes consisting of various instances of these model elements. By comparing the comprehensiveness and descriptive power of this model to that presented in the Japanese Red Cross Society’s disaster exercise planning manual, we confirmed that the disaster context model not only satisfies all of the recommendations in the planning manual but also can describe disaster contexts in more detail.

This paper also proposes a prototype support system for generating imaginary disaster contexts semi-automatically, based on a disaster context database developed by assuming an actual hospital setting. We evaluated the potential impact of using this system by employing a method for social impact analysis used for sociotechnical systems. The results suggest that the model could reduce the effort needed to prepare disaster exercises, enhance disaster response capabilities, and facilitate the sharing and reuse of the knowledge acquired.

This research is still in the prototyping stage and has only served to illustrate the proposal’s potential. Nonetheless, a support tool for designing and managing disaster exercises is urgently needed to enhance the disaster resilience of a highly developed and complex society such as Japan. Based on the results reported in this paper, the proposed model and the methods used in this research promise to become a technical basis for such support tools.