Assessing Safety Behaviors in Fear of Storms: Validation of the Storm-Related Safety Behavior Scale

  • Kirstyn L. Krause
  • Emma M. MacDonald
  • Alasdair M. Goodwill
  • Valerie Vorstenbosch
  • Martin M. Antony


With the exception of one self-report questionnaire assessing storm fear severity (Nelson et al. Journal of Psychopathology and Behavioral Assessment, 36(1), 105–114, 2014), there are few brief published assessment tools to measure the cognitive, behavioral, and physical manifestations of storm fear. A principal feature of phobic disorders is the use of safety behaviors to alleviate distress. Safety behaviors are believed to perpetuate anxiety by preventing the disconfirmation of feared outcomes (Salkovskis Behavioural Psychotherapy, 19(1), 6–19, 1991). To date, no studies have examined the use of safety behaviors in storm fear. The purpose of the current research was to develop and validate the Storm-Related Safety Behavior Scale (SRSBS; Vorstenbosch and Antony 2017), a 24-item self-report scale that measures safety behavior use in adults with a fear of storms. Two studies examined the (1) factor structure, internal consistency, validity, and test-retest reliability of the SRSBS, as well as the frequency with which specific safety behaviors were endorsed; and (2) ability of the SRSBS to differentiate between a group of adults with low and high fear of storms after exposure to a virtual thunderstorm. Factor analysis revealed that the SRSBS is best captured by one factor. Results provided preliminary evidence of convergent and discriminant validity, as well as test-retest reliability. Finally, significant group differences were found between participants with high versus low fear of storms following a virtual thunderstorm. These findings demonstrate the value of the SRSBS for assessing safety behavior use.


Phobia Storm Fear Safety behavior Measurement Factor analysis Virtual reality 

Storm phobia is described as an excessive and persistent fear of severe weather (e.g., thunderstorms), and is classified as a specific phobia of the natural environment type in the fifth edition of the Diagnostic and Statistical Manual of Mental Disorders (DSM-5; American Psychiatric Association 2013). Distressing and impairing anxiety and fear related to storms can occur both in the anticipation or the presence of bad weather, and may be accompanied by physiological reactions (e.g., tension, rapid heartbeat, sweating), maladaptive thoughts (e.g., “If I drive in the rain, I will lose control of the vehicle and crash”), and maladaptive coping behaviors (e.g., distraction, avoidance). The lifetime prevalence of storm phobia in adulthood in the United States is estimated to be 2% (Stinson et al. 2007).

Although storm phobias are common, empirical research on the topic is scarce, with only a small number of studies examining their phenomenology (Munson et al. 2010; Wang 2000; Westefeld 1996) and treatment (Botella et al. 2006; Matthey 1988). Westefeld (1996) recruited participants from the community with a self-reported “intense, debilitating, unreasonable fear of severe thunderstorms and tornadoes” (p. 511). Using a semistructured interview, Westefeld (1996) asked participants to (1) elaborate on the onset, cause, and development of their storm fear, (2) describe the symptom profile of their storm fear, and (3) share impressions of the benefit of any past treatment they may have received for storm fear. Results revealed that storm fear often developed through classical or vicarious conditioning (e.g., experiencing or observing a traumatic storm-related event). Eighty-six percent of participants experienced anticipatory anxiety regarding upcoming storms, which was accompanied by a myriad of other symptoms (e.g., constant monitoring of weather reports, worry about safety, difficulty falling asleep, etc.). Only 10% of the sample had sought treatment for these symptoms. In a two-page report, Wang (2000) described similar phenomenology of storm fear in three cases. All three individuals reported frequent monitoring of weather forecasts and other weather-related cues, as well as anticipatory anxiety and impairment in daily functioning. No information was provided regarding how these data were collected or how storm fear was assessed. Munson et al. (2010) examined the response of female undergraduates with preexisting storm fear to Hurricane Katrina. Those with preexisting storm fear reported greater overall distress and less self-efficacy in coping with this distress following Hurricane Katrina than those without preexisting storm fear. Research on the treatment of storm phobia using exposure-based treatments is limited to single-case studies (Botella et al. 2006; Matthey 1988).

One factor that may contribute to the low number and quality of studies on this topic is a lack of validated measures to assess important features of storm fear, such as maintaining factors, associated behaviors, and symptomatology. To date, there is only one known, validated scale, the Storm Fear Questionnaire (Nelson and Antony 2013). The SFQ is a 15-item self-report scale that measures severity of storm fear by inquiring about behavioral and cognitive features of storm fear. The only study examining the SFQ (Nelson et al. 2014) found that the measure had strong psychometric properties (see description in Measures). Additionally, Nelson et al. (2014) demonstrated that the measure successfully differentiated between individuals reporting high versus low fear during exposure to a thunderstorm in virtual reality.

One potential mechanism contributing to the maintenance of storm fear is reliance on safety behaviors. For example, Westefeld (1996) and Wang (2000) reported that individuals with storm fear frequently checked the weather to be aware of approaching storms. Westefeld (1996) also reported that individuals with storm fear avoided driving or stayed home from work when storms were expected, and ensured they had safe hiding places (e.g., basement) in the event of a storm. Although safety behaviors may successfully relieve anxiety or distress in the short-term, they are theoretically thought to support anxiety in the long term by maintaining biases in beliefs, attention, and memory (e.g., misattribution of safety, hypervigilance to threat, distress intolerance, prevention of new learning; Blakey and Abramowitz 2016; Craske et al. 2008; Salkovskis 1991). For example, the use of safety behaviors may prevent individuals from learning that they are able to stay safe or tolerate an anxiety-provoking situation even without the safety behavior.

Empirical research examining the relationship between safety behavior use and treatment outcome is mixed. On one hand, some studies demonstrate that safety behavior use during exposure-based treatment results in poorer treatment outcome, whereas other studies suggest that safety behaviors may have a positive effect or no effect on treatment (Blakey and Abramowitz 2016; Meulders et al. 2016). Given these mixed findings, thorough assessment of the use and function of safety behaviors prior to and during therapy is an important step in understanding the influence of safety behaviors on treatment.

Although self-report measures of safety behavior use have been developed for certain anxiety-related problems (e.g., Clark et al. 1995, unpublished; Cuming et al. 2009; Pinto-Gouveia et al. 2003), there are no known published measures assessing storm-related safety behaviors. The purpose of the present study was to introduce and validate the Storm-Related Safety Behavior Scale (SRSBS; Vorstenbosch and Antony 2017), a new 24-item self-report questionnaire designed to assess the use of safety behaviors in storm-related fear. Study 1 evaluated the psychometric properties of the SRSBS, as well as the frequency with which specific safety behaviors were endorsed by individuals with high storm fear. Study 2 investigated the association between SRSBS scores and anxiety experienced during exposure to a virtual thunderstorm among individuals with low and high storm fear.

Study 1

The purpose of Study 1 was to evaluate the factor structure, internal consistency, convergent and discriminant validity, and the test-retest reliability of the SRSBS. Additionally, the frequency with which specific safety behaviors were endorsed by individuals with high storm fear was evaluated. The SRSBS is a clinical and research tool created to assess the type, frequency, and severity of safety behaviors in response to storm-related fear.



Factor Analysis

Three-hundred and ninety-seven participants from the United States took part in a study assessing cognitive and behavioral symptoms of several anxiety-related problems, including storm fear. Participants were recruited through Mechanical Turk (MTurk), an online crowdsourcing service operated by Amazon in which anonymous “workers” complete web-based tasks for small amounts of money. MTurk has been shown to provide quality psychometric data (e.g., test-retest, internal consistency) and to be more demographically diverse than other internet-based samples and typical undergraduate samples (Buhrmester et al. 2011; Goodman et al. 2012).

Participants who failed the Instructional Manipulation Check (n = 48, IMC, Oppenheimer et al. 2009; see description in Measures) or completed <80% of the items on the SFQ (n = 6) or the SRSBS (n = 1), were dropped from the sample. The final sample included 342 participants who were primarily female (n = 213, 62.3%) with a mean age of 37.33 years (SD = 12.69). The majority of participants (n = 237, 69.3%) identified as “White or Other”.1 Thirty-five (10.2%) and 29 (8.5%) participants identified as “Black or African American” and “Hispanic or Latino” respectively. The remaining participants identified as American Indian, Alaska Native, Asian, or Bi/Multiracial.

Test-Retest Reliability

Seventy-eight MTurk participants from the United States completed the SRSBS a second time. Participants who completed <80% (n = 2) of the items on the SRSBS at either Time 1 or Time 2 were dropped from the sample. The mean interval between Time 1 and Time 2 was 68 days (SD = 5.70), with all participants completing the Time 2 assessment between 53 and 85 days after Time 1. The final sample included 76 participants who were primarily female (n = 43, 56.6%) with a mean age of 41.75 years (SD = 14.06). The majority of participants (n = 49, 64.5%) identified as “White or Other.”1 Ten (13.2%) and 8 (10.5%) identified as “Black or African American” and “Hispanic or Latino respectively”. The remaining participants identified as American Indian, Alaska Native, Asian, or Bi/Multiracial. The proportions of females or “White or Other” participants were not significantly different across Time 1 and Time 2. Participants who completed the retest were significantly older than those who did not, U = 14,991.50, z = 3.21, p = .001.2



The SRSBS (Vorstenbosch and Antony 2017) was originally a 34-item self-report questionnaire designed to assess the frequency with which individuals used safety behaviors to manage storm fear during a typical thunderstorm (e.g., frequently and repeatedly check weather reports to see if bad weather is expected). Items for the SRSBS were generated by the authors based on safety behaviors observed in clinical practice and reported in the literature. In addition, mental health professionals in the field of clinical psychology with expertise in anxiety-related disorders reviewed these generated items and provided feedback, after which the item list was revised. Items are rated on a 5-point Likert scale ranging from 0 (I never do this to manage a fear of storms) to 4 (I always do this to manage a fear of storms), with higher scores suggesting greater safety behavior use. This questionnaire was revised through the psychometric validation process described in this study. See Appendix for the final 24-item questionnaire.


The SFQ (Nelson and Antony 2013) is a 15-item self-report measure that assesses the cognitive, behavioral, and affective features associated with storm-related fear in adults (e.g., I tend to get so anxious about a storm system approaching that I have a hard time functioning normally), and was included as a measure of convergent validity with the SRSBS. The SFQ has been found to have excellent internal consistency (Cronbach’s α = .95), good convergent and discriminant validity, and good test-retest reliability (r = .78; Nelson et al. 2014). Cronbach’s α for the current study was .95.

Fear of Spiders Questionnaire (FSQ)

The FSQ (Szymanski and O'Donohue 1995) is an 18-item self-report measure that assesses severity of spider phobia, and was included to evaluate discriminant validity of the SRSBS. The FSQ has demonstrated excellent internal consistency (Cronbach’s α ranging from .88 to .92), the ability to differentiate between phobic and nonphobic individuals, and adequate test-retest reliability and convergent validity (Muris and Merckelbach 1996; Szymanski and O'Donohue 1995). Cronbach’s α for the current study was .98.


The IMC (Oppenheimer et al. 2009) was included to measure participants’ use of “satisficing” (i.e., selecting the easiest, quickest answer rather than selecting the most accurate, best answer; Oppenheimer et al. 2009). The IMC begins with a long paragraph about decision-making processes. At the end of this paragraph is a sentence that asks participants to select the sports they play regularly from a list of options (e.g., skiing, soccer, snowboarding, etc.). Embedded within the initial long paragraph is the instruction to ignore the sports items below, and instead, simply click on the title at the top of the screen (i.e., Sports Participation) to proceed to the next screen. Presumably, participants who are paying attention will click the title, whereas participants who are “satisficing” will skip to the end and select sports that they prefer. Goodman et al. (2012) suggest that including an attentional screening question in an MTurk study such as an IMC may result in improved data quality.


All study procedures were approved by the Institutional Review Board at the University. A posting was placed on MTurk, advertising a 75-min survey about thoughts, behaviors, and feelings related to fears of storms and vomit. The data regarding fear of vomit were collected for a separate study. After choosing to participate in the study, individuals were directed to sign the consent form electronically and then complete the study questionnaires. At the conclusion of the study, participants viewed the debriefing form and received $1.10 in their MTurk account for compensation. Participants who took part in Study 1 received an invitation via MTurk to complete the SRSBS approximately 2 months after the first completion of Study 1. Combining of Time 1 and Time 2 data was made possible through nonidentifiable MTurk User IDs provided for Time 1 and Time 2 data. Participants received $0.20 for completing the SRSBS at follow-up. The compensation for the test-retest component of the study was lower than the initial survey given its shorter duration (e.g., 5 to 10 min). Compensation for both tests was comparable to that given for tests of similar length (e.g., Buhrmester et al. 2011; Crump et al. 2013; Horton and Chilton 2010; Paolacci et al. 2010).

Principal Component Analysis (PCA)

PCA was performed on the data to investigate and extract any linear composites of the observed variable scores that maximize the total variance explained between variables. PCA was used because no a priori theoretical model or latent constructs have been identified using the SRSBS items previously. However, if latent components were identifiable, it was assumed these might be correlated to one another; as such, a direct oblimin rotation was also performed during the extraction.



Before completing any statistical analyses, data were screened for missing data. Missing data for participants were replaced using mean substitution. The proportion of individuals with no missing data was 88.6% (n = 305) for the SRSBS and 94.2% (n = 322) for the SFQ. This rate of missing data is comparable to the proportion of individuals completing data in MTurk tests (e.g., 91.6%, Paolacci et al. 2010). The SRSBS had 67 missing items (0.6% of the total possible items) that were replaced by mean substitution including participants with one missing item (n = 25), two missing items (n = 6), four missing items (n = 1), five missing items (n = 4), and six missing items (n = 1). The percentage of missing data for any one item on the SRSBS was below 2.3%. The SFQ had 21 missing items (0.4% of the total possible items) that were replaced by mean substitution including participants with one missing item (n = 19) and two missing items (n = 1). The percentage of missing data for any one item on the SFQ was below 0.7%. While mean substitution has important limitations when dealing with a large proportion of missing data, this method is acceptable when dealing with a very small proportion of missingness as seen in the current study (El-Masri and Fox-Wasylyshyn 2005; Parent 2013). SRSBS item 3 (i.e., stay close to others when bad weather is expected) was removed due to its statistical, r(340) = .95, p < .001, and linguistic similarity to item 4 (i.e., avoid being alone when bad weather is expected). A reliability analysis was conducted on the remaining 32 SRSBS items resulting in a Cronbach’s α of .95.

Before conducting PCA, the frequency with which participants with high storm fear endorsed items of the SRSBS was identified. Participants (n = 59) with high storm fear were defined as those scoring at least 1 SD deviation (14.10) above the mean (16.01) on the SFQ. SRSBS items with a mean score of less than 2 (i.e., I SOMETIMES do this to manage a fear of storms) and a mode of 0 (n = 8) were removed from the scale due to their infrequent endrosement. These included item 6, “Hide under the stairs during a storm” (M = 1.12, SD = 1.31), item 7, “Hide in my bed during a storm” (mean = 1.52, SD = 1.45), item 11, “Close my eyes during a storm” (M = 1.62, SD = 1.37), item 13, “Use medication to reduce my anxiety during storms” (M = 1.14, SD = 1.35), item 14, “Drink alcohol to reduce my anxiety during storms” (M = 0.90, SD = 1.19), item 27, “Leave town when bad weather is expected” (M = 1.18, SD = 1.33), item 29, “Pray when bad weather is expected” (M = 1.88, SD = 1.60), and item 30, “Hold on to a lucky charm when bad weather is expected” (M = 1.16, SD = 1.37).

Based on the entire sample (n = 342), the remaining 24-items of the SRSBS demonstrated excellent internal consistency, as indicated by a Cronbach’s α of .94 (Kline 1999). When examining correlations between all of the 24 SRSBS items, there were no correlations greater than r = .9, indicating that all items were acceptably nonredundant (Field 2013). The highest correlation was between SRSBS items 32 and 33 (r = .76). All 24 items contributed to increasing the scale alpha and none had corrected item-to-total correlations under 0.3, a standard rationale for removing scale items (Kline 1999).

PCA was performed on the data of the entire sample (n = 342). One factor was extracted based on the Scree plot results (Cattell 1966) and relative variance explained. The extracted component accounted for 42.9% of the variance (eigenvalue = 10.30), with all other nonextracted components each accounting for less than one-tenth of the variance. The Kaiser-Meyer-Olkin Measure of sampling adequacy was .93 and Bartlett’s Test of Sphericity was significant, χ2(276) = 4661.16, p < .001, indicating a good fit of the data to the component model. Component loadings ranged from .50 to .77 (M = .65), indicating reasonable correlation between all original variables and the unit-scaled component extracted. PCA loadings ranged from 0.497 (item 31) to 0.768 (item 10).

Convergent and Discriminant Validity

Table 1 reveals the descriptive statistics for the total scores on the SFQ, FSQ, and SRSBS for the 342 participants, indicating that all three scales have significant positive skews (e.g., SFQ = .893, p < .001; FSQ = 1.174, p < .001; SRSBS = .359, p < .05). Cohen (1988) provided the following guidelines for interpreting r: .1 to .3, small effect; .3 to .5, intermediate effect; .5 and higher, strong effect. The correlation between SFQ total score and SRSBS total score was r = .824, p < .001 (N = 342), indicating a strong/large effect (Cohen 1988). The correlation between FSQ total score and SRSBS total score was r = .397, p < .001 (N = 338), indicating an intermediate effect. The correlation between SFQ total score and FSQ total score was r = .383, p < .001 (N = 338), indicating an intermediate effect. The SRSBS and SFQ total score correlation of .824 was significantly different than the correlation of the SRSBS and FSQ total score correlation of .397 (z = 11.05, p < .001).
Table 1

Descriptive statistics for convergent and discriminant validity of the SRSBS

































SRSBS Storm-related safety behavior scale, FSQ Fear of spiders questionnaire, SFQ Storm fear questionnaire, SE Standard error

Test-Retest Reliability

The proportion of individuals who completed the retest with no missing data was 96.1% (n = 73) for both Time 1 and Time 2. Time 1 SRSBS had 4 missing items that were replaced with mean substitution including participants with one missing item (n = 1) and two missing items (n = 2). Time 2 SRSBS had 3 missing items that were replaced by mean substitution including participants with 1 missing item (n = 1) and 2 missing items (n = 1). Pearson correlation of the SRSBS total between Time 1 and Time 2 was significant, r = .67, p < .0001. Means and standard deviations for Time 1 and Time 2 were M = 31.77 (SD = 19.72) and M = 32.83 (SD = 22.28) respectively.

Frequency of Individual Safety Behaviors

Table 2 provides the mean score for each item on the final revised SRSBS, and the percentage of participants with high storm fear who endorsed the item at a 2 or higher (i.e., I SOMETIMES do this to manage a fear of storms). Items that were, on average, endorsed most strongly (i.e., a 3 or higher; I USUALLY do this to manage a fear of storms) included items 4, 7, 8, 12, and 19 on the final revised scale.
Table 2

Mean scores on SRSBS items and frequency of endorsement among individuals with high storm fear

SRSBS item


Mean (SD)

N (%) of participants with score ≥ 2

12. Avoid driving during bad weather.

3.40 (0.84)

56 (94.9%)

5. Stay in a protected room (e.g., a basement) during a storm.

3.02 (0.99)

55 (93.2%)

10. Stay away from windows during a storm.

3.14 (1.12)

53 (90.0%)

25. Avoid outdoor leisure activities (e.g., camping, boating, hiking) for fear that bad weather will occur.

3.07 (1.16)

53 (90.0%)

18. Cancel plans when bad weather is expected.

3.05 (1.01)

53 (90.0%)

2. Stay indoors on days when bad weather is expected.

2.95 (1.07)

53 (90.0%)

4. Avoid being alone when bad weather is expected.

2.86 (1.04)

53 (90.0%)

32. Watch the sky or clouds on days when the weather is bad.

2.83 (1.16)

51 (86.4%)

15. Distract myself (e.g., by listening to music, watching television) during storms.

2.58 (1.21)

51 (86.4%)

8. Restrict myself to rooms without windows during a storm.

2.58 (1.22)

49 (83.1%)

9. Close the curtains during storms.

2.71 (1.33)

48 (81.3%)

16. Avoid talking on the telephone during a storm.

2.76 (1.32)

48 (81.3%)

22. Stock up on supplies (e.g., water, food, batteries) when bad weather is expected.

2.43 (1.11)

47 (79.7%)

33. Frequently and repeatedly look out the window to check the weather conditions.

2.78 (1.21)

46 (78.0%)

28. Avoid leaving home on days when bad weather is expected.

2.71 (1.31)

46 (78.0%)

19. Leave work, school, or appointments early when bad weather is expected.

2.64 (1.22)

46 (78.0%)

17. Avoid using electronic appliances (e.g., television, computer) during a storm.

2.47 (1.44)

43 (72.9%)

26. Keep emergency radio on during storms.

2.24 (1.57)

39 (66.1%)

31. Wear protective gear when I go outside during a storm.

2.05 (1.46)

38 (64.4%)

20. Repeat positive statements (e.g., “I am going to be safe”) during storms.

1.90 (1.28)

38 (64.4%)

21. Ask others for reassurance that the storm is not dangerous.

2.00 (1.40)

37 (62.7%)

23. Avoid using water (e.g., showering, washing dishes) during a storm.

1.91 (1.42)

36 (61.0%)

24. Frequently call friends and family during a storm to determine their safety.

1.69 (1.27)

32 (54.2%)

Items are numbered based on original 34-item scale. SRSBS Storm-related safety behavior scale. High fear of storms = a score of 1 SD above the mean on the Storm Fear Questionnaire (SFQ; Nelson and Antony 2013). A score of 2 on SRSBS items reflects: I SOMETIMES do this to manage a fear of storms. Items are in listed in order, from the item with the greatest number of participants with a score of ≥2 for that item to the smallest number of participants with a score of ≥2 for that item

Study 2

The goal of Study 2 was to examine the association between scores on the SRSBS and subjective levels of anxiety during a virtual thunderstorm. Study 2 also sought to explore the differences in SRSBS scores between participants with high versus low fear of storms. It was hypothesized that (1) scores on the SRSBS would be positively and significantly correlated with anxiety during a virtual thunderstorm, and (2) participants with a high fear of storms would report significantly higher SRSBS scores compared to those with low storm fear.



Participants were recruited from the Greater Toronto Area through the use of flyers and online advertising (e.g., Craig’s list or Kijiji). Upon contacting the experimenter, participants completed a telephone screen to determine eligibility. Using Nelson et al.’s (2014) inclusion criteria, participants were asked to rate their fear of storms on a scale from 0 (no fear of storms) to 4 (extreme fear of storms). Participants were eligible for the high storm fear group if they reported scores of two or higher. Alternatively, only participants who reported scores of zero were eligible for the low storm fear group. In addition to asking for a fear of storms rating, the experimenter administered the specific phobia section of the Structured Clinical Interview of the DSM-5 (SCID-5; First et al. 2015). Eligible participants were invited to complete the study at the Anxiety Research and Treatment Lab at Ryerson University.

Forty individuals with either a high (n = 20) or low (n = 20) fear of storms participated in Study 2. Participants were primarily female (n = 27, 67.5%) and had a mean age of 33.93 years (SD = 14.11). The majority of participants identified either as White/European (n = 18, 45%) or Asian (n = 18, 45%), and had completed at least some college or university education (n = 53, 87.5%). The high and low fear groups did not differ significantly on any demographic variables (Sex: χ2 = 1.03, p = .311, Ethnicity: χ2 = 1.42, p = .701, Education: χ2 = 8.68, p = .122, Age: t(38) = 0.50, p = .620). Two participants (10%) in the high fear group reported symptoms consistent with a diagnosis of specific phobia of storms. Another six participants in the high fear group (30%) would have met full criteria; however, they denied clinically significant distress and impairment. All participants in the high fear group endorsed Criterion A (i.e., marked fear of anxiety of storms) and Criterion E (i.e., the presence of fear for at least 6 months). Ninety percent of participants in the high fear group endorsed Criterion B (i.e., experiencing immediate fear and anxiety when confronted with the stimulus). Fifty percent of participants in the high fear group endorsed Criterion C (i.e., participating in efforts to avoid storms) and another 45% (n = 9) endorsed some avoidance of storms. Diagnostic criteria were not assessed for participants in the low fear group. Participants in the high fear group had significantly higher scores of stress, t(38) = −2.87, p = .006, d = 0.91 and anxiety, t(34.44) = −3.42, p = .002, d = 1.08, than the low fear group, as measured by the Depression Anxiety Stress Scales-21-item version (DASS-21, Lovibond and Lovibond 1995, see description in Measures). However, scores comparing depression were not significantly different, t(38) = −1.44, p = .158.



The SRSBS (Vorstenbosch and Antony 2017) is a 24-item self-report questionnaire created to assess the frequency with which individuals use safety behaviors to manage storm fear during a typical thunderstorm (See Study 1). Cronbach’s α for the current study was .96.

Behavioral Approach Test (BAT)

The BAT was comprised of an eight step virtual thunderstorm using Virtually Better, a company that develops virtual environments for treating a range of psychological disorders (for examples of publications based on Virtually Better environments, see The environment consisted of the interior of a house (i.e., kitchen and living room), as well as a fenced in backyard with a pool and large tree. The neighboring houses and street lamps were visible over the top of the fence, giving the impression of a residential neighborhood. The first step showed a sunny blue sky, and the last step showed a very dark sky with heavy rain, frequent lightning, thunder, and strong winds. Before beginning the BAT, participants were informed that the experimenter was interested in getting an impression of how close they were willing to approach a virtual thunderstorm. At each step, participants were asked to indicate their anxiety using a subjective units of distress scale (SUDS) ranging from 0 (no anxiety) to 100 (the strongest anxiety I have ever felt). Participants were also asked to begin each step immediately following the experimenter’s instructions and were told to inform the experimenter if they were having difficulty performing a particular step.


The DASS-21 (Lovibond and Lovibond 1995 ) is a brief version of the original 42-item DASS (Lovibond and Lovibond 1995). The DASS was created to evaluate depression (e.g., I felt that I had nothing to look forward to), anxiety (e.g., I felt scared without any good reason), and psychological distress/tension (e.g., I felt that I was using a lot of nervous energy) over the previous week. The DASS-21 is a psychometrically sound measure with evidence of strong reliability and validity in both clinical and nonclinical samples (Antony et al. 1998; Clara et al. 2001; Sinclair et al. 2012). In the present study, Cronbach’s α was .92 for depression, .89 for stress, and .87 for anxiety.


The procedure for the current study was adapted from Nelson et al. (2014), and was approved by the University’s Institutional Review Board. Each participant completed the study individually. After participants gave their informed consent, they completed the SRSBS and the SFQ. Following completion of questionnaires they were invited to complete the virtual reality thunderstorm BAT. Testing occurred in a small, dark room, with the experimenter sitting at a computer next to the participant. The computer screen was angled away from the participant to increase immersion in the virtual reality environment. Participants wore a head-mounted display and headphones to view the environment. They remained seated in a chair on top of a platform that vibrated during the “thunder.” Volume settings were consistent across participants. The thunderstorm progressed in a specific pattern, executed by the experimenter, to ensure consistent storm presentation across participants. Before beginning each step, participants were asked if they were willing to progress to the next step. Each step lasted up to 30s, after which participants rated their subjective anxiety. The BAT was discontinued when participants either completed all 8 steps or indicated they did not want to continue. The anxiety rating for the final step completed by each participant was used for data analysis. Participants were then debriefed and received $15 compensation for participation.


Before running analyses, data were screened for missing data. All participants answered >80% of the items on the SRSBS; therefore, no cases were removed. There was only one missing item on any of the questionnaires (Item 12, SRSBS), which was replaced using mean substitution. There were no missing data for the BAT. Seven tests (three dependent t-tests, one independent t-test, and three correlational analyses) were conducted on the same data. Using a Bonferonni adjustment (.05/7), p-values of .007 represented significant findings. A manipulation check was conducted to assess the level of anxiety experienced during the virtual thunderstorm. Thirty-seven (92.5%) out of 40 participants completed all eight steps of the BAT. The other three participants (all in the high fear group) discontinued the BAT after step 4 (dark sky and heavy rain, light wind, and approaching thunder). Three dependent t-tests were conducted to evaluate the extent to which anxiety ratings changed from Step 1 of the BAT to the final step completed in the full sample as well as for each group. All participants in the low fear group reported a SUDS score of 0 during Step 1 of the BAT assessment. For all participants, peak anxiety ratings occurred at the final step completed. Means and standard deviations are reported in Table 3. Dependent t-test results revealed that the BAT was associated with significantly increased anxiety ratings from Step 1 of the BAT to the final step completed for the full sample, t(39) = −6.94, p < .001, d = 2.70, for the low fear group, t(39) = −3.40, p = .003,3 and for the high fear group, t(39) = −8.92, p < .001, d = 3.20.
Table 3

Descriptive statistics of subjective anxiety experienced at step 1 and the final completed step of the BAT


Full sample M (SD)

Low fear M (SD)

High fear M (SD)

Step 1 BAT

4.38 (14.46)

0 (0)

8.75 (19.73)

Final step BAT

43.40 (38.87)

15.15 (19.91)

71.85 (31.82)

BAT Behavioral approach test

A Pearson correlation was computed to examine the association between the SRSBS and the final step BAT anxiety rating in the full sample, revealing a significant positive correlation (r = .71, p < .001).

An independent t-test was conducted to compare scores on the SRSBS in participants with a high fear of storms to those with a low fear of storms. Means and standard deviations of the high and low fear of storms groups were M = 52.30 (SD = 20.32) and M = 14.69 (SD = 12.44) respectively. Results revealed that participants in the high fear group reported significantly higher scores on the SRSBS, t(38) = −7.06, p < .001, d = −3.02 than participants in the low fear group.


The purpose of the present research was to validate the SRSBS, a new self-report measure assessing the use of safety behaviors in individuals with storm fear, and to determine the strength with which specific safety behaviors are endorsed by individuals with high fear of storms. Using a community sample of MTurk participants, Study 1 examined the factor structure, internal consistency, convergent and discriminant validity, and test-retest of the SRSBS. As expected, results revealed that the SRSBS consists of a single, internally consistent factor. Further validity testing supported hypotheses by revealing larger correlations between the SRSBS and SFQ than between the SRSBS and the FSQ, a measure of spider phobia. This finding is consistent with the possibility that the SRSBS demonstrates specificity to storm phobia versus other specific phobias (specifically, spider phobia). The SRSBS also has good test-retest reliability, demonstrating good consistent assessment of the construct over time. In addition, the current study provides preliminary information on the extent to which various safety behaviors are used in individuals with high storm fear. The frequently endorsed safety behaviors included staying in a protected room or away from windows, avoiding driving, canceling plans, and avoiding leisure activities during bad weather. This is the first known study to report on safety behaviors used by individuals with storm fear to manage their distress when confronted with a storm.

The purpose of Study 2 was to further validate the SRSBS by demonstrating the association between self-reported use of safety behaviors during storms and anxiety experienced during a virtual thunderstorm. Results first demonstrated that the virtual thunderstorm was successful at provoking anxiety, as the total sample, high fear group, and low fear group all reported significantly increased anxiety during the final step completed in a virtual thunderstorm BAT compared to step 1. Second, following the final step of the virtual thunderstorm BAT, individuals with high storm fear reported significantly greater subjective anxiety than individuals with low storm fear. Third, SRSBS scores and anxiety ratings for the final completed step of the virtual thunderstorm BAT (where the peak level of anxiety was experienced) were positively correlated in the entire sample, suggesting that greater self-reported use of storm-related safety behaviors is related to the experience of anxiety during a virtual thunderstorm. Finally, consistent with the second hypothesis, individuals with high storm fear reported significantly higher scores on the SRSBS compared to individuals with low storm fear. These results are noteworthy, as they demonstrate that the SRSBS can successfully differentiate between individuals with and without a fear of storms.

The SRSBS may be a useful tool for the treatment of individuals with storm fear. Specifically, the SRSBS can be used to assess the overall extent to which individuals with storm fear use safety behaviors to manage their anxiety. A thorough understanding of safety behavior use in treatment is important as theory suggests that safety behaviors may maintain anxiety (Blakey and Abramowitz 2016; Craske et al. 2008; Salkovskis 1991). Moreover, research demonstrates that the use of safety behaviors may influence clinical outcomes (e.g., Blakey and Abramowitz 2016; Meulders et al. 2016). In a treatment context, scores on specific SRSBS items could be used to identify specific exposure exercises. For example, if an individual endorses the item: “stay indoors on days when bad weather is expected,” exposure or behavioral experiments could promote new learning by preventing this behavior and discovering a discrepancy between predicted and actual outcome. The SRSBS could also be used to track changes in the use of safety behaviors over the course of treatment, though additional research is necessary to identify the scale’s sensitivity to change. While the SRSBS should not be considered a diagnostic tool for storm phobia, a high score on the SRSBS may be an indicator of higher than average storm fear and could warrant further assessment of storm phobia.

The present research has several limitations. First, both studies used nonclinical samples, so it is unclear whether results would generalize to a clinical sample of individuals with a specific phobia of storms. For example, it is possible that some of the items removed in Study 1 due to low endorsement by individuals with nonclinical high storm fear (i.e., items 6, 7, 11, 13, 14, 27, 29, and 30) might have been endorsed more strongly in a clinical sample with a specific phobia of storms. Future research using clinical samples should examine the frequency of additional safety behaviors, including those items that were deleted from the final item set in the SRSBS. Second, Study 2 did not assess comorbidity. It is possible that individuals with high storm fear may have had more additional diagnoses than those with low storm fear, which may have influenced the group differences found. Future research would benefit from recruiting individuals with diagnosed storm phobia, as well as assessing for additional diagnoses. Third, both studies included samples from geographically limited areas (United States in Study 1; Toronto, Canada in Study 2), and the results may not generalize to individuals living in other geographic regions with different weather patterns. Moreover, only participants with a fear of thunderstorms were recruited for Study 2, limiting the generalizability of the results to fear of other forms of severe weather. Finally, the SRSBS assesses the general tendency to use storm-related safety behaviors in the past year rather than safety behaviours in the context of a specific situation, such as the BAT administered in this study. While the use of a VR storm provides an approximation of a storm environment, it is possible that the relationship between SRSBS scores and the responses to a virtual storm may be different than the relationship between SRSBS scores and responses to an actual storm. Greater self-reported use of safety behaviors may not be related to greater anxiety in a real-world thunderstorm.

In summary, the SRSBS is a new tool that can be used in treatment and research settings to identify the extent to which individuals endorse safety behavior use in response to severe weather. This measure can provide insight into behaviors used to avoid, reduce, or prevent feared outcomes in storm fear. This is particularly important given the prevalence of safety behavior use in anxiety disorders, and the potential for these safety behaviors to interfere with treatment (Blakey and Abramowitz 2016). The present research provides preliminary psychometric data for the scale, and demonstrates its ability to differentiate between the extent of safety behavior use in those with high and low storm fear. Presently, the SRSBS has great potential for both clinical and research uses, and continued research will only serve to increase the utility of the measure.


  1. 1.

    The demographic categories of “White” and “Other” were mistakenly combined during data collection. Separate data on these two categories are not available.

  2. 2.

    Nonparametric Mann-Whitney U was used given non-normal distribution of age.

  3. 3.

    Cohen’s d was deemed invalid as Step 1 BAT anxiety rating for the low fear group = 0.



Thank you to Kesla Forsythe for assistance with data collection.

Compliance with Ethical Standards

Conflict of Interest

Kirstyn L. Krause, Emma M. MacDonald, Alasdair Goodwill, Valerie Vorstenbosch and Martin M. Antony declare that they have no conflict of interest.

Ethical Approval

All procedures performed in studies involving human participants were in accordance with the ethical standards of the institutional and/or national research committee and with the 1964 Helsinki declaration and its later amendments or comparable ethical standards and were approved by the University’s Research Ethics Board.

Informed Consent

Informed consent was obtained from all individual participants included in the study.


  1. American Psychiatric Association. (2013). Diagnostic and statistical manual of mental disorders (5th ed.). Arlington: American Psychiatric Press.CrossRefGoogle Scholar
  2. Antony, M. M., Bieling, P. J., Cox, B. J., Enns, M. W., & Swinson, R. P. (1998). Psychometric properties of the 42-item and 21-item versions of the depression anxiety stress scales in clinical groups and a community sample. Psychological Assessment, 10(2), 176–181.CrossRefGoogle Scholar
  3. Blakey, S. M., & Abramowitz, J. S. (2016). The effects of safety behaviors during exposure therapy for anxiety: Critical analysis from an inhibitory learning perspective. Clinical Psychology Review, 49(1), 1–15.CrossRefPubMedGoogle Scholar
  4. Botella, C., Baños, R. M., Guerrero, B., García-Palacios, A., Quero, S., & Alcañíz, M. (2006). Using a flexible virtual environment for treating a storm phobia. PsychNology Journal, 4(2), 129–144.Google Scholar
  5. Buhrmester, M., Kwang, T., & Gosling, S. D. (2011). Amazon’s mechanical Turk: A new source of inexpensive, yet high-quality data? Perspectives on Psychological Science, 6(1), 3–5.CrossRefPubMedGoogle Scholar
  6. Cattell, R. B. (1966). The scree test for the number of factors. Multivariate Behavioral Research, 1(2), 245–276.CrossRefPubMedGoogle Scholar
  7. Clara, I. P., Cox, B. J., & Enns, M. W. (2001). Confirmatory factor analysis of the depression-anxiety-stress scales in depressed and anxious patients. Journal of Psychopathology and Behavioral Assessment, 23(1), 61–67.CrossRefGoogle Scholar
  8. Cohen, J. (1988). Statistical power analysis for the behavioral sciences (2. Auflage). Hillsdale: Lawrence Erlbaum Associates.Google Scholar
  9. Craske, M. G., Kircanski, K., Zelikowsky, M., Mystkowski, J., Chowdhury, N., & Baker, A. (2008). Optimizing inhibitory learning during exposure therapy. Behaviour Research and Therapy, 46(1), 5–27.CrossRefPubMedGoogle Scholar
  10. Crump, M. J. C., McDonnell, J. V., & Gureckis, T. M. (2013). Evaluating Amazon’s mechanical Turk as a tool for experimental behavioral research. PloS One, 8(3), 1–18.CrossRefGoogle Scholar
  11. Cuming, S., Rapee, R. M., Kemp, N., Abott, M. J., Peters, L., & Gaston, J. E. (2009). A self-report measure of subtle avoidance and safety behaviors relevant to social anxiety: Development and psychometric properties. Journal of Anxiety Disorders, 23(7), 879–883.CrossRefPubMedGoogle Scholar
  12. El-Masri, M. M., & Fox-Wasylyshyn, S. M. (2005). Missing data: An introductory conceptual overview for the novice researcher. Canadian Journal of Nursing Research, 37(4), 156–171.PubMedGoogle Scholar
  13. Field, A. (2013). Discovering statistics using IBM SPSS statistics (4th ed.). London: SAGE Publications.Google Scholar
  14. First, M. B., Williams, J. B. W., Karg, R. S., & Spitzer, R. L. (2015). Structured clinical interview for DSM-5, research version 1.0.0 (SCID-5-RV). Arlington: American Psychiatric Association.Google Scholar
  15. Goodman, J. K., Cryer, C. E., & Cheema, A. (2012). Data collection in a flat world: The strengths and weaknesses of mechanical Turk samples. Journal of Behavioral Decision Making, 26(3), 213–224.CrossRefGoogle Scholar
  16. Horton, J. J., & Chilton, L. B. (2010). The labor economics of paid crowdsourcing. (2010). Proceedings of the 11th ACM conference on Electronic commerce. 978-1-60558-822-3. doi: 10.1145/1807342.1807376.
  17. Kline, P. (1999). The handbook of psychological testing (2nd ed.). London: Routledge.Google Scholar
  18. Lovibond, P. F., & Lovibond, S. H. (1995). The structure of negative emotional states: Comparison of the depression anxiety stress scales (DASS) with the Beck depression and anxiety inventories. Behaviour Research and Therapy, 33(3), 335–343.CrossRefPubMedGoogle Scholar
  19. Matthey, S. (1988). Cognitive-behavioural treatment of a thunder-phobic child. Behaviour Change, 5(2), 80–84.CrossRefGoogle Scholar
  20. Meulders, A., Van Daele, T., Volders, S., & Vlaeyen, J. W. S. (2016). The use of safety-seeking behavior in exposure-based treatments for fear and anxiety: Benefit or burden? A meta-analytic review. Clinical Psychology Review, 45(1), 144–156.CrossRefPubMedGoogle Scholar
  21. Munson, M. S., Davis, T. E., Grills-Taquechel, A., & Zlomke, K. R. (2010). The effects of hurricane Katrina on females with a pre-existing fear of storms. Current Psychology, 29(4), 307–319.CrossRefGoogle Scholar
  22. Muris, P., & Merckelbach, H. (1996). A comparison of two spider fear questionnaires. Journal of Behavior Therapy and Experimental Psychiatry, 27(3), 241–244.CrossRefPubMedGoogle Scholar
  23. Nelson, A. L., & Antony, M. M. (2013). Storm fear questionnaire. Toronto: Ryerson University.Google Scholar
  24. Nelson, A. L., Vorstenbosch, V., & Antony, M. M. (2014). Assessing fear of storms and severe weather: Validation of the storm fear questionnaire (SFQ). Journal of Psychopathology and Behavioral Assessment, 36(1), 105–114.CrossRefGoogle Scholar
  25. Oppenheimer, D., Meyvis, T., & Davidenko, N. (2009). Instructional manipulation checks: Detecting satisficing to increase statistical power. Journal of Experimental Social Psychology, 45(4), 867–872.CrossRefGoogle Scholar
  26. Paolacci, G., Chandler, J., & Ipeirotis, P. G. (2010). Running experiments on mechanical Turk. Judgment and Decision making, 5(5), 411–419.Google Scholar
  27. Parent, M. C. (2013). Handling item-level missing data: Simpler is just as good. The Counseling Psychologist, 41(4), 568–600.CrossRefGoogle Scholar
  28. Pinto-Gouveia, J., Cunha, M. I., & do Céu Salvador, M. (2003). Assessment of social phobia by self-report questionnaires: The social interaction scale and the social phobia safety Behaviours scale. Behavioural and Cognitive Psychotherapy, 31(3), 291–311.CrossRefGoogle Scholar
  29. Salkovskis, P. M. (1991). The importance of behaviour in the maintenance of anxiety and panic: A cognitive account. Behavioural Psychotherapy, 19(1), 6–19.CrossRefGoogle Scholar
  30. Sinclair, S. J., Siefert, C. J., Slavin-Mulford, J., Stein, M. B., Renna, M., & Blais, M. A. (2012). Psychometric evaluation and normative data for the depression, anxiety, and stress scales-21 (DASS-21) in a nonclinical sample of U.S. adults. Evaluation & the Health Professions, 35(3), 259–279.CrossRefGoogle Scholar
  31. Stinson, F. S., Dawson, D. A., Chou, S. P., Smith, S., Goldstein, R. B., Ruan, W. J., & Grant, B. F. (2007). The epidemiology of DSM-IV specific phobia in the USA: Results from the National Epidemiologic Survey on alcohol and related conditions. Psychological Medicine, 37, 1046–1059.CrossRefGoogle Scholar
  32. Szymanski, J., & O'Donohue, W. (1995). Fear of spiders questionnaire. Journal of Behavior Therapy and Experimental Psychiatry, 26(1), 31–34.CrossRefPubMedGoogle Scholar
  33. Vorstenbosch, V., & Antony, M. M. (2017). Storm-related safety behavior scale. Toronto: Ryerson University.Google Scholar
  34. Wang, A. G. (2000). Storm phobia: A North Atlantic phenomenon. Nordic Journal of Psychiatry, 54(1), 67–68.CrossRefGoogle Scholar
  35. Westefeld, J. S. (1996). Severe weather phobia: An exploratory study. Journal of Clinical Psychology, 52(5), 509–515.CrossRefPubMedGoogle Scholar

Copyright information

© Springer Science+Business Media, LLC 2017

Authors and Affiliations

  • Kirstyn L. Krause
    • 1
  • Emma M. MacDonald
    • 2
  • Alasdair M. Goodwill
    • 1
  • Valerie Vorstenbosch
    • 3
  • Martin M. Antony
    • 1
  1. 1.Department of PsychologyRyerson UniversityTorontoCanada
  2. 2.Community Mental Health and AddictionsIWK Health CentreHalifaxCanada
  3. 3.Eating Disorders ProgramHomewood Health CentreGuelphCanada

Personalised recommendations