1 Introduction

Tropical cyclones (TCs) are capable of ravaging coastlines often leading to a significant loss of life. In addition, they also cause billions of dollars in damages (see Gulati et al. 2017). Military bases on the US coastlines are especially vulnerable to TCs as they concentrate expensive assets (e.g., ships or aircraft) in a single area. Consider the catastrophic damage to Camp Lejeune from Hurricane Florence in 2018 and the destruction of Tyndall Air force base in the same year from Hurricane Michael. Rebuilding costs from the damage are estimated to run into billions of dollars (see https://www.military.com/daily-news/2019/04/05/congress-comes-400-million-camp-lejeune-storm-repairs.html and https://insideclimatenews.org/news/18122018/tyndall-military-hurricane-cost-2018-year-review-billion-dollar-disasters-wildfire-extreme-weather-drought-michael-florence.) Advanced preparation and evacuations can mitigate some of these losses; however, these actions are expensive, and so bases try to avoid them when possible.

Whitehead (2003) distilled the economic cost of civilian TC evacuations along the East and Gulf Coasts into a simple rule of thumb—“One million dollars per mile”. That is the estimated cost of imposing mandatory evacuations along these seaboards. The U.S. Military is different in that it has bases with concentrated populations and expensive assets separated by large distances. The Military also has a highly organized and informed population that adheres to strict guidelines for procedures and behaviors in advance of TC winds—tropical cyclone conditions of readiness (TCCOR).

The Department of Defense defines the following set of rules to prepare for an approaching TC. These rules and the ensuing level of preparedness are called TCCOR and are dependent on the probability of 50-knotFootnote 1 winds within a certain time period. The rules and an example of the general levels of preparedness for households in and around Kadena Air Base, Okinawa are summarized below: (https://www.kadena.af.mil/About-Us/Typhoons/).

1.1 TCCOR IV

Destructive sustained winds (defined as winds of 50 kt or greater) are possible within 72 h. Stock up on non-perishable food, bottled water, dry milk, batteries, flashlights, fuel, candles and other emergency supplies. For some bases, TCCOR IV stays in effect through the TC season.

1.2 TCCOR III

Destructive sustained winds are possible within 48 h. Start a general clean-up around your residence and office. Prepare to secure your home, pick up loose items, such as garden tools, porch furniture and toys.

1.3 TCCOR II

Destructive sustained winds are anticipated within 24 h. Secure all outdoor property such as picnic tables, barbecue grills, etc. Board up the windows, sliding glass doors, etc.

1.4 TCCOR I

Destructive sustained winds are expected within 12 h. Fill any available containers with water. Make a final check of food, water and other supplies, disconnect appliance and stay indoors until all clear is issued.

In practice, there are many factors that go into setting TCCOR, including time of day, day of week, terrain and exposure issues, and even scheduled events. The process is by nature subjective, but guidelines such as the objective TCCOR guidance (Sampson et al. 2012) based on National Hurricane Center (NHC) wind probabilities (DeMaria et al. 2009, 2013) can provide input into the decisions. The objective TCCOR guidance was developed principally to maximize hit rate (HR; the ratio of 50-kt wind events at the site to the number TCCOR predicted) at the expense of higher false alarm rate (FAR; the ratio of non-events at the site to the number of TCCOR predicted). The reason for this was that potential loss of life and loss of credibility for a miss were considered to be higher level concerns than monetary savings, so reducing FAR was only a secondary concern. The TCCOR thresholds were developed by mapping 50-kt cumulative wind probability thresholds to operational U.S. Watches and Warnings issued along the Atlantic Seaboard and Gulf of Mexico. The 50-kt cumulative wind probability thresholds for TCCOR IV, III, II, and I were derived to be 5%, 6%, 8%, and 12% within 72, 48, 24, and 12 h, respectively. The objective TCCOR guidance algorithm has two properties that make it suitable for use in testing cost-loss/savings models like the one developed herein: (1) it is entirely objective and thus repeatable, and (2) modification of the thresholds is relatively easy so testing with different thresholds requires little effort.

The intent of this work is to develop an algorithm to measure the economic value of preparing for a skilled forecast over a climatological baseline that we will define in the next section. Specifically, we investigate whether preparing for imminent TC winds based on the objective guidance for TCCOR II generates more savings when compared to preparation based on the climatological baseline. We choose TCCOR II for our study because it is generally the level where preparation costs increase dramatically. Moreover, the bases almost always follow a TCCOR II with TCCOR I at present. We also purposely omit injury, loss of life and forecast credibility costs from the model and are aware that these can dwarf savings/costs in base preparedness. Still, there is merit in using the base preparedness problem to measure the value of skilled meteorological forecasts in this age of increased spending scrutiny as that value would represent the minimum value of our skilled forecasts.

The rest of the paper is organized as follows. In Sect. 2, we construct a cost-loss/savings model and define a no skill climatological baseline. In Sects. 3 and 4, we apply our cost-loss/savings model to the objective TCCOR guidance from Sampson et al. (2012) to investigate if there are savings in a skilled a forecast versus our no skill climatological baseline. Then, we investigate the economic effect of changing the wind probability thresholds for the objective TCCOR guidance. Finally, in Sect. 5, we provide a summary of our findings and make recommendations for future research.

2 The cost-loss/savings model and a climatological baseline

As part of the U.S. National tropical cyclone warning process, the National Hurricane Center (NHC) makes wind speed probability products available to warning coordinators and the general public. The wind speed probabilities are based on the NHC TC forecast and sampled forecast errors to create 1000 forecast realizations that are consistent with the NHC TC forecast and forecast errors. The wind speed probabilities also include landfall effects (Kaplan and DeMaria 1995) and errors based on numerical weather prediction model spread (Goerss 2007). Figure 1 shows a gridded example of 50-kt cumulative (0–120 h) wind speed probabilities for a TC approaching the Gulf Coast. The point value for Tallahassee, Florida is approximately 25%, which can be interpreted as a 25% chance of 50-kt sustained winds within the next 120 h.

Fig. 1
figure 1

Example of 50-kt cumulative wind speed probabilities on a grid over the Gulf of Mexico

Full size image

Although the NHC and other U.S. TC forecast centers make deterministic forecasts (i.e., a single track, intensity, and structure forecast out to 120 h for a given TC), wind speed probabilities are well-suited for use in warning coordination and downstream applications. So, developing TCCOR guidance on these wind speed probabilities, as done in Sampson et al. (2012), is a natural extension of the NHC guidance.

2.1 The cost-loss/savings model

To describe the cost-loss/savings model used herein, we start with the simple scenario where a forecast is either Yes (a TC hit at the base with 50-kt sustained winds) or No (no TC hit). Subsequently, we assume that a Yes initiates actions for base preparedness (which cost money), while No initiates no action (and therefore costs nothing). Let C be the cost of the preparedness and let D be the cost of damage to the base during a TC hit that could have been mitigated if the base had prepared. Then, the savings, assuming there is a hit and the base prepared, are D–C. If the base had not prepared, the base incurs a loss interpreted as 0 savings. Finally, there are the false alarms. If the base prepares for a hit and there is none, then the savings are –C. This can all be expressed in Table 1.

Table 1 Savings based on associated actions in a TC event
Full size table

2.2 The climatological baseline

A climatological baseline is needed to estimate savings from a skilled forecast over an unskilled forecast (our climatological baseline). Let the climatological probability of being hit by a TC (50 kt sustained winds at the base) be represented by P(TC); therefore, the probability of a miss is 1- P(TC). Let S denotes savings and let P denotes the event that the base prepares for the storm. The expected savings (E) from preparing based solely on climatology are:

$$E\left( {S|P} \right) = \left( {\left( {D - C} \right)} \right)*P\left( {\text{TC}} \right) - C*\left( {1 - P\left( {\text{TC}} \right)} \right) = D*P\left( {\text{TC}} \right){-}C$$
(1)

It is clear from the above equation that the expected savings if a base prepares for a storm are positive as long as P(TC) ≥ C/D. In plain English, that means that the savings are positive when the climatological HR is greater than the ratio of preparedness cost to damage costs. So on a base, where the preparation cost is low relative to damage costs (e.g., flying expensive aircraft from Guam or Japan instead of leaving them on the tarmac in Okinawa where they are subsequently damaged) and the climatological likelihood of a TC is high (as it is on Okinawa), preparation decisions based solely on climatological probability can provide economic value. Contingency tables such as Table 1 are often used to estimate the economic value of binary decision-making processes (see Leigh et al. 1998; Lawrence 1987).

P(TC) is generally unknown and so it has to be estimated. A significant effort could be spent on this problem alone, but for demonstration we construct a simple process to estimate it based on a TC climatology database such as that on the Automated Tropical Cyclone Forecast System (ATCF; Sampson and Schrader 2000). From now on, we will use a specific Navy base, Norfolk Naval Station in Virginia, in our discussions to illustrate the methodology.

First, we consider all TCs from 1945 to 2018 that approached within 200, 300, 500, and 700 n mi of the base at 12, 24, 48, and 72 h, respectively. The total for each radius represents the number of TCs that could affect the base in the given time period (see Fig. 2 for the 24-h example). Then, we compute all TCs that approached within 50 n mi of the base, which we as a proxy for TC hits (Fig. 3). This definition is easy to explain and compute.

Fig. 2
figure 2

TCs that came within 300-nm radius of Norfolk between 1945 and 2018

Full size image
Fig. 3
figure 3

TCs that came within a 50-nm radius of Norfolk between 1945 and 2018

Full size image

As shown in Fig. 2, 176 TCs came within a 300-nm radius of Norfolk between 1945 and 2018. Similarly, we see from Fig. 3 that 27 TCs came within a 50-nm radius of Norfolk between 1945 and 2018. So, from our method defined above, the historical probability of a TC hit given a 24-h lead time is estimated to be 27/176 or 0.15. For 200 n mi (representing TCs in the vicinity of Norfolk NS at 12 h), the historical probability (P(TC)) is estimated to be 27/136 (.20). For 500 and 700 n mi (representing TCs in the vicinity at 48 and 72 h), the historical probability estimates drop to 27/275 (.10) and 27/363 (.07), respectively.

Using the above methodology to estimate P(TC) and Eq. (1), we can compute the expected savings associated with preparing for a TC hit whenever there is a storm within the prescribed radius of the base. However, the decision maker (who sets the TCCOR) is usually provided with a skilled TC forecast to help them make the TCCOR decisions. But are these skilled forecasts saving money, and if so, how much?

Computing savings for a skillful forecast against a no skill forecast requires simple Bayesian statistics (a set of methods for statistical estimation). In this paper, we use the methodology detailed in Kite-Powell and Solow (1994) and applied in Solow et al. (1998). If the skilled forecast has known HR and FAR, we can use these to update the probability of a hit to include the skilled forecast. The updated probability of a hit based on the forecast is then used to compute expected savings of the skilled forecast.

Let us denote a TCCOR I setting by the random variable X. X takes on the value 1 if the forecast calls for a 50-kt winds at the base and 0 otherwise. Assume that the hit rate of the forecast is denoted by HR and the false alarm rate is denoted by FAR. Using the methodology from Kite-Powell and Solow (1994), we can show (see “Appendix”) that the expected savings from using the skilled forecast EX(S) are given by:

$$E_{X} \left( S \right) = D*{\text{HR}}*P\left( {\text{TC}} \right) {-}P\left( {\text{TC}} \right)*C*\left( {{\text{HR}} - {\text{FAR}}} \right)) - {\text{FAR}}*C$$
(2)

For example, suppose that the historical probability (P(TC)) of a TC hit is 0.3 and that our skillful method has a hit rate of 0.95 and a FAR rate of 0.05. Let us also specify that the cost of preparation for a TC hit (e.g., setting a TCCOR in preparation for 50-kt winds at the base) is $100,000, while the cost of damage without any preparation is estimated at $1,000,000. If we prepare using the no skill forecast, our expected savings using Eq. (1) are:

$$0.3*\left( {\$ 1,000,000} \right) \, - \, \$ 100,000 \, = \, \$ 200,000.$$

If we use the skillful method, then our savings from (2) are:

$$0.3*\left( {0.95*\$ 1,000,000} \right) \, {-} \, \left( {0.3*\$ 100,000*0.90} \right) - \left( {0.05*100,000} \right) \, = \, \$ 285,000 - \$ 27,000 - \$ 5,000 \, = \, \$ 253,000$$

The interpretation of the above is that our skillful forecast saves $38,000 per event when compared to the no skill baseline.

3 Application to objective TCCOR II guidance

The objective TCCOR guidance thresholds developed were tuned to maximize HR, with FAR being only a secondary consideration (Sampson et al. 2012). The reason for this approach was to ensure safety and minimize destruction at the bases. Recommended thresholds are 12%, 8%, 6% and 5% cumulative probability of 50-kt winds at the base within 12 h, 24 h, 48 h and 72 h for TCCOR I, II, III and IV levels, respectively. To evaluate the objective TCCOR guidance, Sampson et al. (2012) tested the guidance on independent data from 113 potential TCCOR cases (defined as cases where a tropical storm approached close enough to a base to warrant interest) for over 10 years and for many bases (eight in the western North Pacific and four along the Eastern Seaboard and Gulf of Mexico). Hits were determined by whether the base set TCCOR I or not. Table 2 shows the performance on independent data.

Table 2 HRs and FAR’s of the guidance for the different TCCOR levels
Full size table

As mentioned in the introduction, here we compute the economic value using TCCOR II results. As before, we assume that the forecast initiates one of the two actions: going to the TCCOR level or doing nothing. Let C be the cost of the going to TCCOR II and let the damage sustained by the base at the two levels of preparation be represented by D. Thus, if TC conditions are realized at a base that had prepared, the savings are D -C. On the other hand, if the base did not prepare then the base has no savings. Similarly, if the base prepares for the appropriate TCCOR conditions and they are not realized, then the savings are –C. These savings can be expressed in a matrix just like the one in Table 1 (we omit the matrix for brevity.)

As in the previous section, let the probability of a hit at the base be denoted by P(TC). Let S denotes savings and B2 denotes the event that the base prepares for TCCOR level II. We denote the guidance forecast for TCCOR level II by the random variable X2 where X2 takes on the value 1 if the guidance calls for the base to prepare for TCCOR II and 0 if preparation is not called for. Using the same notation as in Sect. 2, let the hit rate of the process X2 (P(X2 = 1|TC)) be denoted by HR2 and the false alarm rate of the process X2 \(\left( {P (X_{2} = 1 | {\text{No TC}}} \right))\) of X2 be denoted by FAR2. Using the same methodology as in the previous section, we update the probabilities to compute the expected savings from the guidance variable X2 in terms of the HR and the FAR of X2 as follows:

$$E_{2} \left( S \right) = P\left( {\text{TC}} \right)*\left( {{\text{HR}}_{2} *D} \right) - C*P\left( {\text{TC}} \right)*\left( {{\text{HR}}_{2} - {\text{FAR}}_{2} } \right) - C*{\text{FAR}}_{2}$$
(3)

Plugging in the values for HR2 and FAR2 in (3) gives us the following expression for the expected savings:

$$E_{2} \left( S \right) = P\left( {\text{TC}} \right)*0.92*D{-}C*0.57*P\left( {\text{TC}} \right){-}0.35*C$$
(4)

To fully illustrate the use of this methodology, we applied it to a hypothetical TCCOR II forecast issued at the Norfolk Naval Base. (For the purpose of this illustration, we assume that TCCOR II conditions are the same as being hit by a storm.) Using the methodology detailed in Section I, the climatological probability of a TC was estimated to be 0.1531. We used information acquired from a web search for the base to estimate the value of the assets (ships, planes, hangars, piers, etc.) at the base. Data from different sites give us the estimated value of these assets to be $210,355,720,000.00. Mitigation costs were assumed to be 0.01% of the total asset value, and damage was estimated as 10 times the mitigation cost giving us C = $21,035,572.00 and D = $210,355,720.00. Plugging the values all this information into the Eq. (4) gives us an expected savings of $20,430,862.41. Note also that from Eq. (1), a no skill forecast gives us an expected savings of D* P(TC) − C = 11,169,888.73. Thus, using the skill forecast results in an expected gain of $9,260,973.68.

To conclude this section, we wanted to examine the effect of improved hit rates and FARs of the forecast on the expected savings. From a mathematical standpoint, we would examine the behavior of (3) as a function of the hit rate and the FAR by taking appropriate derivatives of the equation. To do so, we take the derivative of the expected savings in Eq. (3) with respect to HR2 and FAR2, respectively, and get:

$$\frac{\partial }{{\partial {\text{HR}}_{2} }}E_{{X_{2} }} \left( S \right) = P\left( {\text{TC}} \right)*\left( D \right) - C*P\left( {\text{TC}} \right)$$
(5)

and

$$\frac{\partial }{{\partial {\text{FAR}}_{2} }}E_{{X_{2} }} \left( {S_{2} } \right) = C*P\left( {\text{TC}} \right) - C$$
(6)

It is easily seen that Eq. (5) is positive when D > C, implying that (all other things being equal) the expected savings increase when HR increases as long as the cost of damage to the assets is more than the mitigation costs. Equation (6) is always negative which implies that the expected savings (all other things being equal) decrease as FAR increases. This might lead the reader to conclude that one should work with a process that maximizes HR and minimizes FAR. But doing so simultaneously is not possible. Moreover, the mathematical test to determine the effect of simultaneously changing HR and FAR on (3) is inconclusive. And finally, minimizing FARs come at the expense of a higher miss rate that could risk lives, reduce forecast credibility, and provide less time for base preparation (Sampson et al. 2012). In other words, we are better off maximizing the first term (the Hit Rate) in Eq. (3). This will be examined in more detail in the following sensitivity analysis.

3.1 Sensitivity analysis

Sampson et al. (2012) investigated hit rate and FAR for different cumulative probability thresholds (50% higher, 50% lower, and double thresholds). As expected, lower thresholds generally yield higher hit rates and FARs, while the higher thresholds yield lower hit rates and FARs (Table 3). This sensitivity analysis can also be applied to our cost-loss/savings model to determine the resultant savings of the different thresholds.

Table 3 HRs and FARs for objective TCCOR II guidance at different threshold
Full size table

In Sect. 3, we demonstrated that lowering FARs or raising hit rates alone can increase expected savings. The impact on savings when both hit rate and FAR increase or both decrease is less certain, even in our simple cost-loss/savings model. In Table 3, we see that raising the cumulative probability thresholds by 50% lowers the FAR while leaving hit rate unchanged. Thus, the expected savings using 50% higher thresholds should be mostly positive using our simple cost-loss/savings model. Using 50% lower thresholds have the opposite effect, it does not change the hit rate but increases the FAR. Hence, using 50% lower thresholds decrease expected savings. Doubling the thresholds decrease the hit rates and the FARs coincidentally, so the impact on savings is unclear.

We can clarify the impact on savings of the different thresholds by investigating a specific case like the Norfolk Naval Base case described in Sect. 3. Plugging the HR and FAR values for the thresholds into Eq. (3) give us results shown in Table 4.

Table 4 Savings for objective TCCOR guidance at different thresholds
Full size table

As expected, 50% higher thresholds increase expected savings for TCCOR II, while 50% lower thresholds decrease expected savings. Doubling the thresholds actually increase the expected savings for the example under study. From this table alone and the concluding paragraph in Sect. 3, one might want to increase the thresholds by 50% since the general trend is upward in savings. But the cost-loss/savings model used to create Table 4 does not include injury or death associated with preparedness started too late or never started at all. In practice, preparedness saves lives. If we assume each life to be worth $10,000,000 (Kneiser and Viscusi 2019) and incorporate that into our model, it changes the results dramatically. Assuming the climatological hit rate of 0.15 and preparation costs of $21,000,000 for each event (as we did in the Naval Station Norfolk no skill forecast), then from (1) we need to save fourteen lives to negate the preparation costs (ignoring damage costs for simplicity). However, our skillful forecast with the current thresholds has a hit rate of 0.92 a FAR of 0.35, so saving seven lives recover the total preparation costs for one event (with a savings surplus of $514,500). So, savings in prevention of injury and death can dwarf savings in preventing property loss, and although TCCOR guidance thresholds could be tuned to maximize savings in property damage, the overriding concern is still human life.

4 Summary and conclusions

This study attempts to estimate economic value of setting TCCOR at U.S. Military Bases. We developed a simple cost-loss/savings model based on material assets. We also constructed a “no skill” baseline for setting TCCOR-based solely on climatology. We then constructed a “skilled forecast” using performance of objective guidance (a proxy for operational forecasts) gathered from 113 real cases over 10 years from U.S. Military Bases. Using Norfolk NAS as the base, we developed a cost-loss/savings model that only accounted for assets at the base with prescribed preparedness costs and damage estimates associated with the absence of preparations. The value of assets was prescribed to be approximately $210B with the damage costs assigned to 0.01% of the total asset value and mitigation (preparedness) costs assigned to 10% of damage costs. Using these values, skilled TCCOR I forecasts (yes and no) yield an average of $25 M in savings per TC event at NAS Norfolk. So, savings are on the order of tens of millions of dollars per forecasted event for one large U.S. Navy base, and that is just for physical assets at the base. Death, injury, and human behavior (where operators ignore meteorological recommendations because they are frequently wrong) add to those costs and should probably be condensed into equations to get a more complete cost-loss/savings model.

Another element of the cost-loss/savings model that could use further investigation is how to specify a “no skill” forecast. In this paper, we used a rudimentary estimation of Pc, the climatological probability of a hit, similar to that of Hope and Neumann (1971) and Brettschneider (2008). But do the results change markedly if one uses different climatological baselines? Is there another way to estimate “no skill” that is potentially more skillful than these climatological baselines? These could be important questions to investigate in the future.

We would also like to refine our assumptions regarding the decision to initiate a certain TCCOR level. In this paper, we assumed that if the guidance calls for a TCCOR level(specifically levels I and II), the base will prepare for it. We also assumed that the damage incurred was the same for each TCCOR level of preparation. However, the decision to prepare and the level of preparation are subjective. A base might set TCCOR if it appears that preparation costs Ci ≤ PcD (Regnier and Harr 2006). There is also a tendency to follow through the TCCOR once they have started and probably a reluctance to set lower TCCOR if the higher ones haven’t been set earlier. And finally, TC forecasting has improved dramatically through the years through efforts of the entire TC community. A worthwhile exercise would be to estimate the economic value of those improvements and complexities in our model.