Introduction

As seen in the Spring 2009 A/H1N1 influenza outbreak, public health practitioners, health care workers and volunteer health providers faced extreme pressures to meet patient surge capacity, slow the spread of influenza, and rapidly organize public health interventions. In general, preparedness plans for pandemic influenza focus on the use of non-pharmaceutical public health interventions to minimize human-to-human transmission, and provide adequate medical services. School closure is one such important non-pharmacological intervention designed to mitigate the effects of pandemic influenza. School closure has been included in federal guidelines [6], and in 47 of 50 US states it has been included in pandemic influenza preparedness plans [11].

In these community preparedness plans, universities and schools have unique place because of the challenges caused by their dense student populations, function with the larger community and frequent international travel. Important components of university preparedness include: 1) establishing a comprehensive planning document, 2) developing and disseminating alternative procedures to assure continuity of instruction and continuity of operations, 3) establishing infection control policies and procedures, and 4) developing a communications plan. Many critical decisions are needed to manage the pandemic successfully, i.e. minimizing the number of cases with loss of lives and mitigating its potential effects on the social and economic life. Because the population that can be affected by a novel virus is very large and diverse, various mitigation strategies including non-pharmaceutical and pharmaceutical interventions will inevitably be needed. These mitigation strategies have to be tested and evaluated before they are implemented.

Health and emergency preparedness officials use several modes of education and training programs to improve their response capability, including tabletop exercises. Tabletop exercises allow participants to role-play during a health emergency in a realistic but risk-free environment and evaluate performance using existing benchmarks. These discussion-based exercises are the first steps of identifying the gaps in the plans, policies, protocols, processes and procedures, which are included in the planning for pandemic mitigation [8, 14, 20]. At a university level, these exercises allow the organization to assess and improve their performance, while demonstrating community resolve to prepare for major incidents. The major limitations of traditional tabletop exercises include difficulty demonstrating the results of policy decisions that are made during the exercise, and the assuming perfect communication among decision makers throughout the catastrophic pandemic event [19].

A novel method for improving traditional tabletop exercises can be incorporating computer simulation models into the discussion-based component of the exercise [3] or constructing the exercise with a formality of computer simulation model [18]. Simulations in exercises may help decision makers analyze the situation better, consider existing resources and tools, explore the effectiveness of various interventions and finally adapt their plans accordingly [12, 13]. There are several papers in the literature about the design of preparedness exercises but to the best of our knowledge none of them have included the integration of recent developments in computer and information technologies. Rutherford [16] presented a framework about using table-top exercises for disaster management. Taylor et al. [19] presented a table-top exercise for the pandemic preparedness of Maryland and outlined the benefits of such exercises for the policy makers’ education and readiness to potential effects of pandemic influenza in the community. More recently, Beaton et al. [3] presented the findings from a table-top exercise designed to improve pandemic preparedness at one of the largest universities in the United States. This exercise focused on several important response capabilities: isolation and quarantine, continuity of operations, disaster mental health services, travel of university students and personnel, communication problems, and meeting the needs of students and faculty during an outbreak. Decker and Holtermann [8] stated that discussion based table top exercises are effective and efficient for identifying policy gaps and fostering pandemic influenza preparedness based on their experience from implementing over 100 exercises.

The important role of decision support systems for public health decision makers have been also discussed in the literature [5]; and [10, 12]. Although designing simulation models for public health emergency preparedness is a difficult task, these studies have concluded that simulation models are useful tools for policy makers to evaluate and test their decisions. In this paper, we present a method of designing a tabletop exercise to test the pandemic influenza preparedness of one of the largest public universities in the United States, Arizona State University (ASU).

We present details of designing a modified tabletop exercise in which several technologies, including computer simulation model, visualizations and multi-media videos, can be used effectively to improve pandemic preparedness. The mathematical and technical details of the simulation model that was developed for ASU pandemic preparedness exercise has been previously published [2] and it is not in the scope of this paper to explain the details of this simulation model and its validation here. We rather aim to present the details of exercise design processes in which advanced technologies can be embedded into an exercise in a way that participants can perceive the public health risks in a more realistic setting while facilitating decision-maker interactions around complex problems. In this paper, we present an overview of the simulation model and the tabletop exercise developed to improve pandemic preparedness at Arizona State University which has been used successfully to identify important gaps in pandemic response plans. We also highlight the benefits of using advanced computer simulation models as a decision-making tool in the exercise and suggest a number of important policy implications that may result from incorporating technological advances into current pandemic planning.

Methods

In this section, we define the purpose of the pandemic preparedness exercise, technical details of the exercise design, participants and setting for the exercise, and the of the disease spread simulation model.

Purpose of the exercise

Arizona State University (ASU) is the largest public research university in the United States under a single administration, with a 2010 student enrollment over 50,000. ASU faculty and staff have been actively working for several years to develop the University’s response and procedures in the event of a health-related emergency. In order to test the University’s preparedness plan, we developed a simulation model and tabletop exercise to designed to stimulate discussion, raise critical public health, social, legal and ethical issues, engage participants, and challenge key personnel to provide prompt and accurate advice and make critical real-time decisions. Key university leadership, federal, state and local health officials, emergency response officials and key community stake holders were involved.

Arizona State University (ASU) conducted this public health emergency exercise—“ASU Pandemic Influenza Tabletop Exercise 2008”—on April 10, 2008. This exercise was designed to engage and prepare university emergency response leadership, executives, management, and operational emergency response infrastructure to collaboratively evaluate the university’s pandemic influenza emergency response plan, evaluate gaps and vulnerabilities and improve the university’s ability to respond and recover from a pandemic influenza. Major areas for discussion involved the policies and procedures for maintaining academic continuity, maintenance of essential campus functions, risk communication, student evacuation, managing human resources and supplies, and isolation and/or quarantine of sick students.

The specific objectives for this exercise were:

  1. 1.

    Identify and discuss criteria which will lead to the activation of campus emergency operations management.

  2. 2.

    Determine strengths and weaknesses in functionality of the incident management structure, coordination and integration of response resources, and communication systems for responding to pandemic influenza at ASU.

  3. 3.

    Identify coordination, collaboration and communication strategies needed between ASU and external agencies that will have to interact with the University during a pandemic (local hospitals, Maricopa County Department of Public Health, Arizona Department of Health Services, Tempe Police and Tempe Fire Departments) for better emergency response.

  4. 4.

    Assess university policies and risk communication strategies for conveying critical information to students, staff, faculty, parents and stakeholders during a large-scale influenza outbreak.

  5. 5.

    Evaluate selected operational aspects of responding to pandemic influenza at Arizona State University including surge capacity, triage of ill students, managing the needs of students living in residence halls, canceling of classes, and maintenance of essential services.

  6. 6.

    Identify problems that could arise in executing social distancing measures, including procedural, logistical, ethical, and/or enforcement issues.’

The exercise was designed using capabilities-based planning which allowed the exercise design team to develop exercise objectives and observe results through a framework of specific action items.

Exercise design

During the exercise, participants were asked to respond to a hypothetical pandemic influenza scenario and make iterative policy decisions in a group setting. Participants were prompted, for example, to decide when to close schools and for how long to close them in response to an emergency outbreak. The simulation gave policy makers the ability to see the real-time impact of their decisions (e.g., percentage of the population infected with influenza, duration of outbreak, costs of school closures). Because the simulation platform that was used through the exercise gave participants the ability to change final outcomes based on their actions during the exercise, we referred to this modified tabletop exercise an “interactive game”. Multiple players participated and the exercise included several scenarios which could generate various outcomes based on participants’ actions.

The exercise included a number of innovative features such as video clips with scenario information, geographical mapping, and the interactive computer simulation model. The exercise scenario incorporated an outbreak of a novel influenza strain outside the U.S., initial cases in the U.S. but not yet affecting cities near ASU and a community outbreak near the ASU campus. The exercise also addressed an ongoing outbreak in the ASU campus with substantial impact on university operations. The scenario was designed to address exercise objectives and engage the Emergency Operations Center (EOC) on campus as well as Incident Command Center and the University President’s Emergency Policy Group. Data was presented in the form of fictional, but realistic video newscasts supported by multimedia injects and discussions included the challenges of “just in time” supply chain and potential medical surge capacity for campus triage and housing of sick students.

Setting for the exercise

Arizona State University (ASU) is dispersed across four several campuses in the Phoenix metropolitan area and the University has a major impact on the states’ economical and social standing. Therefore, the university’s preparedness to this public health crisis is a critical part of the larger State of Arizona’s Pandemic Plan.

The exercise took place on the main ASU campus at the ASU Decision Theater. The ASU Decision Theater is research facility and decision lab for exploring and understanding decision-making and houses state-of-the-art visualization, decision systems science, simulation and solutions tools. The Decision Theater contains a unique room, called the “drum”, which is a 260 degree immersive visual environment surrounded with seven projection screens for displaying video displays, data visualizations and simulation models (see Fig. 1). This open layout of this room helps facilitate improved collaboration among decision-makers, while the cutting edge technology increases risk perception. The literature suggests that visualizations have big impact on the cognation of the presented information about related systems, and therefore has a major role on affecting the decisions about complex policy problems [9].

Fig. 1
figure 1

Arizona State University pandemic influenza preparedness exercise at Decision Theater, April 2008

Full size image

Exercising pandemic preparedness

The exercise was broken down into a series of four scenarios; each had a discussion time of 30 min. Each scenario included a list of events, data injects as appropriate and a list of questions for discussion. The scenarios were initiated by a fictional video newscast and a narrative. Following each scenario, players reviewed the data, discussed the questions listed at the end of the scenario and responded to issues related to their role and responsibility. The facilitators of the exercise guided the discussion around the topics listed under each script in the SITMAN (Situation Manual) through a series of questions. As the scenarios unfolded, the exercise was supplemented with an interactive simulation model for decision support. This model simulated the possible outcomes of decisions as they were being made based on different rates of disease transmission, case fatality, timing of university evacuation decisions and social distancing interventions. After the completion of the exercise, participants were allowed a time to provide some feedback about the exercise through participant evaluation forms.

Participants of the exercise

Exercise participants included members of Arizona State University Pandemic Influenza Planning Steering committee, as well as others identified by the exercise design team on the basis of their university leadership roles or agency’s responsibilities during a pandemic. Participants included administrative representatives such as the university president and vice-president, university provost, finance, communications, risk management, university legal counsel, risk management, communications. Participants also included representatives from key university operations from human resources, facilities, environmental health, university health center, university housing, academics and communications. In addition, exercise observers and evaluators included partners from local hospitals, state and local health departments and local emergency personnel.

Participants were separated into three groups throughout the duration of the exercise: Incident Command (IC), Executive Policy Group (EPG) and Emergency Operations Center (EOC). IC members included an incident commander, backup Incident Commander, and representatives from safety/operations, logistics, communications, finance, planning, human resources, IT, facilities, and ASU police. The EPG consisted of the University President and his team of advisors. The EOC consisted of the Director of Campus Health, Deputy Chief of Police (ASU Police), representatives from media relations, finance, logistics (residential life), legal (OGC), administration and safety. Major areas for discussion involved the policies and procedures for maintaining academic continuity, maintenance of essential campus functions; risk communication; student evacuation, human resources and supplies; and isolation and quarantine of sick students.

The role of the simulation model in the exercise with mitigation strategies

Modeling and simulation can play a key role in pandemic planning to formulate interdependencies and support complex decision making [13]. Building a computer simulation model for a complex system, such as health services in a university for pandemic preparedness, required a holistic approach to understand the dynamics of multiple systems with several policies and to quantify the relations between these systems and policies. The simulation model that we developed and used in the exercise captured the disease dynamics with the social dynamics that can be controlled by several social distancing policies listed in the university’s pandemic influenza mitigation plan. It is built as a system dynamics model with Powersim [15] that is software for solving differential equations and building system dynamics models. The dynamics of the influenza spread was modeled as a basic Susceptible-Exposed-Infected-Removed (SEIR) model, which divides the population into several compartments called susceptible, exposed, infected, recovered and death [1]. The complexity of the simulation model with considered policies that are affecting the public health outcomes during the exercise is simplified and presented in Fig. 2.

Fig. 2
figure 2

Policy-disease model interaction

Full size image

The simulation model gave the university decision makers an opportunity to quantify and visualize the consequences of their decisions (i.e. total deaths and infections on campus). We divided the ASU community into several subpopulations based on their daily activities in the simulation model. These subpopulations included commuting students, residence hall students, faculty, staff and essential personnel who will have to stay on campus for critical operations. These critical operations included surge capacity, triage of sick students, management of students living in residence halls, and maintenance of essential services. The commuting students and the residential students are affected differently by implementation of any non-pharmaceutical interventions. For example, while cancelling school policy forces residential students to stay in dorms for a certain amount of time and have relatively high contacts, commuting students are assumed to stay at home and have fewer contacts.

From a policy perspective, the most important and difficult decisions in the exercise included how to direct people under this emergency situation, which policies should be used for each of the subpopulations and what kind of resources should be allocated to which subpopulation. Therefore, the most critical characteristics of the model that we used was the robustness of moving people from one subpopulation to another as an implementation of the decisions made by the policy makers. For example, if university leadership decides to close the university campus, except the essential operations, then all commuting students would be moved into “evacuated students” subpopulation, except the ones who could not survive the pandemic or could not leave the campus. This is done by stopping the simulation at the critical times given in the scenario and asking decision makers to make decision on closing the campus or not. The user interface of the model with the questions asked to decision makers is presented in the Appendix.

Throughout the exercise, participants interacted with the facilitated simulation model. This model generated possible disease outcomes based on different rates of disease transmission, virulence, timing of university evacuation decisions and social distancing interventions. We made several assumptions regarding to several disease parameters in our simulation model. These assumptions are listed as: 1) the infected mortality rate is 2% as a worst case scenario and it is an estimated value for 1918 pandemic [4]; 2) latent period would be 2 days [4]; 3) infection period would be 3.5 days regardless of any subpopulation [4]; and 4) the contact rate would be approximately 50 people per day for students living off-campus and 75 people per day for students living in residence halls due to the close proximity of living quarters (observed and calibrated parameters). For model validation purposes, contact rates were chosen to be used for calibrating the model to achieve the Cumulative Attack Rate (CAR) estimates of the 1918 pandemic. Although, these parameters were pre-defined before the exercise, the participants had the ability to change any of these parameters and visualize the system state to adopt their decisions interactively. In Appendix, Fig. 4 presents the user interface where the decision makers could see disease specific parameters and change them based on their will. Also they can see the health outcomes (e.g. cumulative number of infections and total deaths as numbers in tables presented in the same figure). The daily campus health resource usage is also reported as shown in this figure. In Fig. 5 we present the way how the simulation model is stopped and how user input is taken. Finally Fig. 6 shows an example of results over time as graphics.

Results

Based upon the identification of the exercise objectives and capabilities, this novel preparedness exercise had allowed us to obtain several results. First, participants were able to identify and discuss the criteria which would lead to the activation of campus emergency operations management. We determined the strengths and weaknesses in functionality of incident management, coordination and integration of response resources, and communication systems for mitigating the effects of pandemic at the university campus. Secondly, from public safety point of view, the exercise help educate the University of the need to identify community stakeholders and coordinate the response plans of these stakeholders with the University’s response plan. Participants also discussed the possibility of isolating sick students and the problems that could arise in executing this social distancing measure (these problems could include procedural, logistical, ethical, and law enforcement issues). In addition, participants were also able to assess university policies and risk communication strategies for conveying critical information to students, staff, faculty, parents and stakeholders during the influenza pandemic. From communications perspective, establishing procedures for communicating academic continuity and emergency response was determined as a necessity.

The final analysis of the exercises included identification of the strengths to be maintained and built upon, identification of potential areas for further improvement, and development of corrective actions. Results from the simulation model demonstrated that in the event of a necessary school closure, the timing of the decision to suspend university operations has a critical role on the severity of the disease among university population.

Discussion

In this section, we first present the advantages of the new exercise methodology and then discuss the results obtained with the survey that was performed after the exercise. Then the shortfalls of this method with possible improvements are listed.

Advantages of the new exercise method

The major strength that we identified during the exercise was the possibility for the participants to effectively find gaps in resources, plans and communication and that might require revising with the help of supporting simulation model. It was also possible to recognize limitations of mass care surge capacity, triage quarantine and isolation activities. Lastly, the participants became aware of the communication problems that may emerge under these kinds of emergency situations. These results are similar to what others have reported in the literature [7]. Many of the critical gaps in the University Pandemic Response Plan would have not been elucidated through discussions in a traditional tabletop exercise.

Survey for exercise evaluation

Following the completion of the ASU exercise, all 32 participants of the exercise were given a survey and all of them returned this survey questionnaire. The survey was designed to assess participants’ impression about the new exercise method for pandemic preparedness. Specifically, we asked whether the cutting edge technologies at the Decision Theater helped them in making informed and ultimately collaborative and better decisions. According to our survey results, the simulation model and visualization tools were found helpful for creating a more realistic decision making environment in the exercise.

Survey results, which are shown in Fig. 3, demonstrate that most of the participants of the exercise found multimedia presentations very helpful in terms understanding the state of the outbreak (65.62%). The majority of the participants (68.75%) also agreed that the displayed scenarios were realistic and plausible. In addition, the simulation model gave the flexibility of displaying various scenarios in terms of disease spread on campus and implementing and testing different mitigation policies. Another important issue was the information flow and communication among the participants who were physically located in different locations and were given different level of information during the exercise. This forced participants to communicate and make better collaborative decisions. The majority (84.37%) of participants agreed that the information exchange during the exercise was in high quality, which means that the decision-making environment facilitated communication.

Fig. 3
figure 3

Survey results on the effectiveness of multimedia presentations and simulation

Full size image

Finally, even though this presented method allowed participants to change several disease parameters and perform sensitivity analyses on them during the exercise, the participants were not supported with real-time statistical information about these parameters and their potential socio-economical impacts. The exercise can be improved if we can support decision makers about the transmissibility and the severity of the disease through a statistical inference tool that uses real-time available data. In addition, to be able to create a discussion environment, our exercise started with the scenario of having the index case far from the US and Arizona. However, as it was the case in the H1N1 outbreak in 2009, the location of the index case could be geographically very close to the jurisdiction of the decision makers and this could yield a very different discussion. Our scenarios in the presented exercise did not include this case, however with minor changes in the simulation model and also in other supporting videos, it could easily be brought to discussion.

Conclusions

Scenario-based exercises, such as classical tabletop exercises, are increasing in number as a means of evaluating preparedness plans of organizations [17]. However, evaluation of exercise design and improving the classical tabletop exercises presents a significant gap in the literature while the rapid developments in the information technology can be helpful for closing these gaps. Furthermore, descriptions of exercises in the literature lack substantial details on innovative features and reportedly no organizations have conducted exercises with the type of multi-media and simulation modeling capabilities as presented in this paper. We have presented a new method of exercising pandemic influenza preparedness plans for one of the biggest universities in the US. Even though the scale of the exercise was at the university level, the method presented in this paper can also be used to design exercises for other universities as well as other local communities.

The presented exercise methodology prompted participants to execute the basic planning elements central to their response plan. Most participants stated that the presented exercising method increased their readiness for a pandemic and the survey results that we presented support these statements. Finally, participants stated that the video scenarios and the simulation model made the exercise appear very compelling and real, which makes this exercising game different than the other table-top exercises presented in the literature. Simulation models and multi-media technologies are useful tools for creating more realistic decision making environments and they are helpful for putting the decision makers in a situation where they are forced to discuss with other responsible individuals. Therefore designing a pandemic preparedness exercise with supporting technologies can help identifying the communication gaps between responsible authorities. Participants have the ability to test various alternative strategies and evaluate them in terms of their effectiveness on several health outcomes e.g. number of lives lost, number of cases in a given time, cumulative attack rate and effectiveness of resource usage which is an invaluable tool.