Background

Epidemiologic studies after disasters have shown elevated levels of health problems among survivors such as post-traumatic stress disorder (PTSD), depression and physical symptoms [1–3]. The majority of the disaster studies have been cross-sectional, and although cross-sectional studies are useful for assessing the public health burden of the disaster, they do not give insight into the course of the health consequences and the health needs of survivors at different times post-disaster.

Since relatively little is known about the course of health problems among survivors of disasters, more longitudinal studies are needed [1, 2, 4]. However, attrition is a main methodological problem in longitudinal studies. A common approach for handling attrition is to delete observations with missing values, but this complete case analysis can result in a substantial loss of power. In addition, if respondents systematically differ from non-respondents, deleting incomplete observations might introduce bias in the prevalence estimates of the health problems [5, 6]. A good way to deal with missing data and to overcome possible selection bias in the prevalence estimates is to conduct multiple imputation [6, 7]. This technique fills in various values for each missing data point based on a statistical model. Because the missing values are drawn from a distribution, there will be a range of values imputed for each missing value, with variation appropriately reflecting the uncertainty about that value. Using this technique, it can be estimated what the prevalence of the outcomes of interest would have been if there had been no (systematic) attrition in the longitudinal study.

Evidence concerning selective response among survivors of disasters is conflicting. Some studies have shown that non-respondents are more likely to be male, single and to have a low socioeconomic status [8–10], while other studies did not observe such an association [11–15]. Little is known about the association between the level of disaster exposure and non-response at follow-up. One may speculate that survivors who were highly affected by the disaster or who had high levels of post-disaster distress would be more motivated to participate at the follow-up of a health survey than survivors who were less affected. On the other hand, it can be hypothesized that highly exposed or distressed survivors would be less likely to respond because they do not want to be reminded of the stressful event. Several studies have found that depression, distress and symptoms of PTSD at baseline were associated with non-response at follow-up [8, 16, 17] while other studies found no association between baseline distress and non-response at follow-up [12–15]. In addition, the determinants of response at follow-up might also differ between groups of survivors. We recently observed that baseline health problems were associated with response among immigrant survivors and with non-response among native Dutch survivors at wave 2 of a study after a fireworks disaster in the Netherlands [18]. The information from these disaster studies is, however, not sufficient to understand the response mechanisms of survivors of disasters. Furthermore, none of the previous studies after disasters have examined whether selective response biased the prevalence estimates of the health problems among survivors.

Since attrition will most likely occur in future longitudinal studies after disasters, more insight into the response mechanisms among survivors and possible bias resulting from selective response is desirable. In the present longitudinal study after a fireworks disaster in the Netherlands, we examine the selective response among survivors at the follow-up surveys. In addition, we study whether possible selective response had biased the prevalence estimates of health problems among survivors at wave 3 by comparing the estimates resulting from multiple imputation with estimates resulting from complete case analysis.

Methods

Background

On May 13 2000, a fireworks depot exploded in a residential area in the city of Enschede, the Netherlands. As a result of the explosion and subsequent fire, 23 persons were killed, more than 900 people were injured, and about 1,200 people were forced to relocate because their houses were destroyed or severely damaged. The Dutch government declared this a national disaster and started a longitudinal study into the health consequences of the disaster.

Study design

The first survey was performed 2.5 to 3.5 weeks post-disaster (wave 1). In total, 4,456 adult residents were living in the area that was designated by the municipality as the official disaster area. All residents of this area were invited to participate in the health survey by means of announcements in the local media and letters.

Approximately 18 months after the disaster, from November 2001 through January 2002, a second survey was conducted (wave 2). All participants at wave 1 who had given informed consent for future contact received an announcement letter. To stimulate participation, survivors were telephoned. If the survivor agreed to participate, a questionnaire in the preferred language (Dutch, English, German or Turkish) was sent, together with a gift voucher, to their home address. Survivors who did not return the questionnaire within three weeks were reminded by phone or by letter when the person could not be reached by phone.

In January through March 2004, nearly four years post-disaster, wave 3 of the longitudinal study was performed. All participants at wave 1 of the health survey who had given informed consent for future contact, and were not lost to follow-up, received an announcement letter. Participation was stimulated by means of telephone calls and home visits. If the questionnaire was not returned within three weeks, the respondents were reminded by phone or by letter. Details of the study population and the health problems of survivors at the different waves of the study have been described elsewhere [18–22].

Measures

We selected the following demographic variables to examine possible selective response: sex; age; educational level; employment status (having a paid job), and marital status (single).

To examine whether respondents and non-respondents at waves 2 and 3 had different disaster-related experiences, the following disaster-related factors were selected: injury (requiring medical treatment) sustained as a result of the disaster; the loss of loved ones (family, colleagues, friends); relocation due to severely damaged or destroyed house; whether survivors had experienced intense anxiety during the disaster and whether survivors had seen frightening things during the disaster.

We compared health problems at wave 1 between respondents and non-respondents at the follow-up surveys to study possible selective response. We used the Dutch versions of various validated instruments to measure health problems. Feelings of depression and anxiety were measured by the symptom check list (SCL-90) [23, 24]. We dichotomized the scales into 'very high' and 'high' versus 'above average', 'average' and below 'average', according to established references for the healthy Dutch population [24]. The impact of event scale (IES) [25–27] was used to measure intrusions and avoidance reactions which serve as an indication for a clinical level of PTSD. Consistent with Carr et al., Basoglu et al. and others, survivors with an overall score above 25 were considered as having symptoms of PTSD [28, 29]. We used a questionnaire into subjective health complaints (VOEG) [30] to measure 13 physical symptoms such headache and fatigue. In this study, we used a cut-off of six or more symptoms which is one standard deviation above the reference mean. Sleeping difficulties were measured by the Groninger Sleep Quality Scale [31]; survivors with a score above 4 were defined as having severe sleeping difficulties. The RAND-36 was used to measure different aspects of functional status [32]. To examine selective response, four sub-scales were used; social functioning, physical role limitations, emotional role limitations and general health. Scores on the sub-scales were dichotomized using the cut-off scores resulting from a national study in the Netherlands; in this study cut-off scores were based on the mean score minus one standard deviation [33]. We used the scales described above as the outcomes of interest to examine whether possible selective response had biased the prevalence estimates of the health problems at wave 3.

Statistical analyses

Because the response mechanisms between waves 1 and 2 were found to be different for native Dutch survivors and non-western immigrant survivors [18], we stratified the analysis according to ethnic background (native Dutch and immigrant status) in order to study selective response and possible biased prevalence estimates. We defined a non-western immigrant as either a respondent who was born in a non-western country of whom at least one parent was also born in a non-western country, or a respondent whose parents were both born in a non-western country. Most of the non-western immigrants were of Turkish origin (44.7%), followed by immigrants of Moroccan origin (14.0%).

We compared demographic characteristics, disaster-related experiences and wave 1 health problems for respondents and non-respondents at waves 2 and 3 in order to study possible selective response by performing Chi-square tests for categorical variables and ANOVA for continuous variables.

We used multiple imputation (MI) to study the effect of possible selective response on the prevalence estimates of the wave 3 health problems [5]. MI assumes that the missing data are "missing at random"; in other words, the missingness is not related to factors that were not measured in this study. Multiple imputations were performed with the Markov chain Monte Carlo (MCMC) simulation in SAS version 9.1 using the 'MI' procedure. The method imputes plausible values for the missing data using correlations between observed variables [5]. For that reason we included the health problems of interest (described above) from all three waves in the imputation model. In addition, other relevant predictor variables of these health problems were selected; sex; age; educational level; immigrant status; employment status; language; cigarette smoking; alcohol use; sustained injury due to the disaster; relocation; intense anxiety and having seen frightening things during the disaster. Since the power of the model increases when additional data other than the variables of interest are used [6], other important health-related variables were selected: the somatization, hostility and interpersonal sensitivity sub-scales of the SCL-90 [23, 24], the use of sedatives and the presence of chronic diseases among survivors. These variables were measured at all three waves. Peritraumatic dissociation [34] (measured only at wave 1), and optimism [35], the distrust sub-scale of the search for meaning scale [36], and the pain, vitality and mental health sub-scales of the RAND-36 [33] (all measured at waves 2 and 3) were also included in the imputation model. In addition, we included a variable for the approach of participants at wave 3 of the health survey (no contact, telephone contact or face to face).

We used one MI model for both the native Dutch and immigrant survivors because separate analysis results in a substantial loss of power and such an analysis assumes that there is a difference between native Dutch and immigrant for each variable in the MI model. Because some determinants of response differed between native Dutch and immigrant survivors, interaction terms for immigrant status and gender, educational level, marital status, relocation, injury self, lost a loved one, intense anxiety, saw frightening things, feelings of depression, feelings of anxiety, physical symptoms, sleeping problems, social functioning, physical and emotional role limitations, and general health (measured at the three waves) were entered into the model. We did not dichotomize the variables entered in the model, instead linear effects were used in the MI model.

We generated five datasets that were analyzed separately. The results were combined using the 'MIANALYZE' procedure in SAS, in order to produce valid confidence intervals. Finally, we compared the imputed prevalence estimates of wave 3 health problems with the prevalence estimates resulting from complete case analysis for native Dutch and immigrant survivors separately.

Results

Selective response at waves 2 and 3 among native Dutch survivors

In total, 1,083 native Dutch survivors completed the questionnaire at wave 1 (figure 1). Of these survivors, 663 (61.2%) participated at all three waves of the longitudinal study. Three other response groups can be distinguished: 128 survivors (11.8%) who responded to wave 1 only; 198 survivors who responded to waves 1 and 2 (18.3%); and 94 survivors who responded to waves 1 and 3 (8.7%).

Figure 1
figure 1

Flow chart response native Dutch survivors.

Full size image

Demographic characteristics of native Dutch respondents and non-respondents at the three waves are shown in table 1. There were some demographic differences between respondents and non-respondents at the follow-up surveys; men were more likely to respond to wave 1 only, while native Dutch women were more likely to respond to all three waves of the study. Those survivors who responded to waves 1 and 3 but not to wave 2 were younger than the other response groups. Native Dutch survivors who responded only to wave 1 or to waves 1 and 2 had a somewhat lower educational level than survivors who responded to wave 3, however this difference was not statistically significant. Finally, respondents who responded to all three waves were less likely to live alone than those who responded to waves 1 and 2.

Table 1 Demographic characteristics of native Dutch respondents and non-respondents at the three waves of the longitudinal study*
Full size table

Table 2 shows the associations between response and disaster-related experiences and wave 1 health problems for the different native Dutch response groups. There were no clear differences in disaster exposure among the different groups, except for a lower percentage of injured survivors among those who responded to waves 1 and 2 but not to wave 3 compared to survivors who responded to all three waves. Native Dutch survivors who participated only at wave 1 were more likely to report a high level of feelings of depression (59.7%; 95% CI 51.2–68.2) compared to those who responded in all three waves (45.4%; 95% CI 41.6–49.2). Also, wave 1 only respondents tended to have somewhat more problems with social functioning at wave 1 compared to the other response groups, though this difference was not statistically significant. Overall, survivors who responded to waves 1 and 2 but not to wave 3 seemed to have lower levels of health problems three weeks post-disaster compared to the other response groups. Finally, the wave 1 health problems among respondents at waves 1 and 3 tended to be similar to all-wave respondents.

Table 2 Disaster exposure and health problems three weeks post-disaster among native Dutch respondents and non-respondents at the three waves of the longitudinal study*
Full size table

Selective response at waves 2 and 3 among survivors of non-western origin

At wave 1 of the health survey, 352 survivors of non-western origin participated (figure 2). Of this group, only 86 (24.4%) responded to all three waves; 118 immigrant survivors (33.5%) responded to wave 1 only, 75 immigrants (21.3%) responded to waves 1 and 2, and 73 immigrants (20.8%) responded to waves 1 and 3.

Figure 2
figure 2

Flow chart response survivors of non-western origin.

Full size image

Male survivors of non-western origin tended to respond to wave 1 only, while female immigrant survivors were somewhat more likely to respond also to wave 3, but this difference was not statistically significant. In addition, immigrants who responded to waves 1 and 2 were somewhat older than respondents at wave 1 only and respondents at waves 1 and 3 (table 3).

Table 3 Demographic characteristics of respondents and non-respondents of non-western origin at the three waves of the longitudinal study*
Full size table

Non-western immigrants who did not respond to all three waves tended to have a somewhat lower level of exposure to the disaster than survivors who responded to all three waves of the health survey, although this was not true for personal injury (table 4). In addition, immigrant survivors who responded to wave 1 only as well as those who responded to waves 1 and 3 had a lower level of health problems three weeks post-disaster compared to survivors who responded to all three waves. For example, those who responded to wave 1 only and those who responded to waves 1 and 3 had a lower level of intrusions and avoidance reactions, physical symptoms, and sleeping problems.

Table 4 Disaster exposure and health problems three weeks post-disaster among respondents and non-respondents of non-western origin at the three waves of the longitudinal study*
Full size table

Comparison between imputed and complete case prevalence estimates of wave 3 health problems among native Dutch survivors

Figure 3 shows the imputed and complete case prevalence estimates of wave 3 health problems among native Dutch survivors. The imputed prevalence estimates were systematically higher than the prevalence estimates resulting from the complete case analyses. The most notable difference between the imputed and complete case estimates was found for feelings of anxiety (prevalence 25.3%; 95% CI, 22.4–28.5 and prevalence 20.1%; 95% CI, 17.2–23.0 respectively).

Figure 3
figure 3

Imputed and complete case prevalence estimates of health problems at wave 3 among native Dutch survivors.

Full size image

Comparison between imputed and complete case prevalence estimates of wave 3 health problems among survivors of non-western origin

The prevalence estimates of the health problems at wave 3 among non-western survivors resulting from multiple imputation and complete case analysis are shown in figure 4. The imputed estimates for depressive feelings, feelings of anxiety, sleeping problems and social functioning hardly differed from the estimates resulting from complete case analysis. The imputed prevalence estimates of intrusions and avoidance reactions, physical symptoms, physical and emotional role limitations and general health tended to be lower than the complete case estimates.

Figure 4
figure 4

Imputed and complete case prevalence estimates of health problems at wave 3 among survivors of non-western origin.

Full size image

Discussion

In this study among survivors of a fireworks disaster, selective response occurred at the two follow-up surveys. We examined whether selective response had biased the prevalence estimates of the wave 3 health problems by comparing prevalence estimates resulting from complete case analysis and estimates resulting from multiple imputation. The complete case prevalence estimates of the wave 3 health problems were only somewhat biased, and the direction differed between the native Dutch survivors and immigrant survivors.

Similar to other studies after disasters, non-respondents at follow-up were more likely to be male [8–10]. In agreement with two longitudinal studies after disasters, we did not find an association between damaged or destroyed house or property loss and non-response at follow-up [10, 14]. In addition, we did not find clear associations between response at follow-up and other disaster-related experiences, such as the loss of a loved one and intense anxiety.

In this study, we found an association between health problems at wave 1 and response at waves 2 and 3. Some previous studies also showed an association between health problems at baseline and non-response at follow-up [8, 16, 17]. However, the response mechanisms in this study differed between native Dutch survivors and survivors of non-western origin. Among native Dutch survivors, health problems at wave 1 tended to be associated with non-response at follow-up (waves 2 and 3). In contrast, among immigrant survivors, health problems at wave 1 were associated with response at follow-up. These different response mechanisms between native Dutch and immigrant survivors were also found in a recent study among the survivors of the fireworks disaster in which determinants for response at wave 2 of the health survey were examined [18]. It can be speculated that immigrant survivors of the fireworks disaster were not accustomed to participation in a health survey and believed that completion of the questionnaire was not meaningful in the absence of health problems. Although the underlying reasons remain unclear, different response mechanisms among ethnic groups have also been found in a longitudinal general population study by Psaty et al [37]. In their study among whites and non-whites in the USA, poor health status was associated with non-response among whites and with response at follow-up among non-whites.

In this study we allowed survivors who participated at wave 1 but not at wave 2 to re-enter the study at wave 3. At wave 3, all eligible survivors were strongly motivated to participate; all survivors were stimulated to participate by means of telephone calls. In addition, eligible survivors for whom the telephone number was unknown and all immigrant survivors were visited at home. Survivors who re-entered the study at wave 3 differed from survivors who did not (wave 1 only). For example, among both native Dutch and immigrant survivors, those who re-entered the study at wave 3 were somewhat less likely to be male, were somewhat more likely to have a paid job and were somewhat less likely to be single. This group of respondents indirectly provides insight into non-response at wave 2 as well as additional information that is useful when performing MI.

The different response mechanisms among native Dutch and immigrant survivors were also confirmed after multiple imputation of the missing values. Among native Dutch, the imputed estimates of the wave 3 health problems tended to be higher than the complete case estimates. In contrast, the imputed prevalence estimates among immigrant survivors tended to be somewhat lower than the estimates of health problems at wave 3 resulting from complete case analysis. Additional analyses showed that the differences between imputed and complete case estimates of wave 2 health problems were similar to the differences between the complete case and imputed prevalence estimates of wave 3 health problems (data not shown). This confirms the robustness of our findings, since both native Dutch and immigrant survivors have similar response mechanisms from wave 1 to wave 2 and from wave 2 to 3.

We could not demonstrate very large differences between the imputed and complete case estimates of the health problems at wave 3. This result was unexpected since the selective response at the follow-up surveys would suggest prevalence estimates that were more strongly biased. While these results are reassuring, we can not exclude that some of the prevalence estimates were more biased than our results indicate. First, the lack of biased prevalence estimates at wave 3 might be due to weak associations between the predictor and outcome variables in the imputation model. Second, it is possible that other variables, not included in the model, were more important predictors of response and that missing data were not missing at random. However, we included all variables in the imputation model that were likely to be related to response in a health survey. Despite this, it is possible that the mechanism for missing data was non-ignorable; in other words, the missing data depended on variables not measured in this study or on the health status of non-respondents at follow-up. We believe that the method of multiple imputation was adequate given the strong correlations between the variables that were used in the multiple imputation model. In addition, a necessary condition for this method was fulfilled [5]; the existing correlations between all factors used in the imputation model were systematically in the same direction. In this study, multiple imputation gives insight into the magnitude of selection bias on the prevalence estimates. Multiple imputation has some additional advantages above other methods to handle missing data; with multiple imputation all available information is used, therefore avoiding the loss of power associated with complete case analysis [5, 6]. Furthermore, the fact that standard errors and confidence intervals resulting from multiple imputations are more appropriate than those resulting from other techniques such as single imputation is another important advantage of multiple imputation [5–7].

Besides non-response at follow-up, selective participation occurred at wave 1 of the longitudinal study in which 35.2% of all affected residents participated. Affected residents who participated at wave 1 were more likely to be women, to be between 25–44 or 45–64 years of age, to live with a partner, to be a single parent and to be of immigrant background. Analyses of multiple imputations showed that the selective participation did not affect the prevalence estimates of health problems at wave 1 [38]. In the present study we were, however, primarily interested in bias resulting from selective attrition at the follow-up surveys. Despite selective response, the prevalence estimates of health problems at wave 3 were not completely different when we corrected for the selective response by means of multiple imputation. This is important information, emphasizing the fact that selective response is only problematic when it biases the prevalence estimates of health problems.

Conclusion

To date, most studies that have examined response in longitudinal studies have focused on whether respondents were systematically different from non-respondents. Our results indicate that considerable attrition and selective response only somewhat biased that prevalence estimates of health problems among survivors. Therefore, future studies, both among survivors of disasters and among the general population, should not only examine selective response, but should also investigate whether selective response has biased the prevalence estimates of health problems by using statistical techniques such as multiple imputation. Although the present study focused on potential bias in the prevalence estimates of health problems, investigations into risk factors for health problems should also take into account possible bias due to selective response. This is especially important in longitudinal studies after disasters since these studies examine the health needs of survivors and provide information on which post-disaster health interventions are based.