Introduction

The number of individuals killed or injured at work each year is staggering. In the U.S. in 2011 alone, over 4,600 workers were killed, and nearly three million workers sustained serious injuries while working (U.S. Bureau of Labor Statistics 2013). Safety climate, or the relative priority of safety in an organization as perceived by employees (Zohar 1980, 2003, 2011), is an important contributing factor to safety-related events in the workplace, including both personal injuries and organizational and process damages (e.g., fires, chemical releases, and property damage; Baker 2007; Beus et al. 2010; Christian et al. 2009; Zohar 2011). Accordingly, a deeper understanding of the role that safety climate plays as a contributing factor to these incidents is critical to our ability to reduce them.

One understudied challenge in organizational climate research is the fact that climate changes over time (Neal and Griffin 2006; Ostroff et al. 2012), and it is likely to be constantly changing. An assessment of safety climate is only a snapshot of climate at that particular moment in organizational history. It is unclear how long that assessment provides meaningful information about the organization. Although this is true of nearly any workplace construct, this is especially problematic for safety climate, as it is an indicator of risk to the health and well-being of personnel and the organization. If organizations rely on out-of-date assessments of safety climate, they might become complacent about the climate, deploy remedial resources to the wrong units, or otherwise misread the status of safety in the organization.

The purpose of this paper is to examine the “shelf life” of a safety climate assessment by determining the optimal period over which safety–critical incidents should be aggregated. The term “shelf life” refers to the length of time a perishable item (e.g., food, drug, etc.) has before it is considered unsuitable for sale, use, or consumption. Analogously, we seek to determine the length of time that a safety climate assessment is suitable for meaningful prediction within an organization, both as the predictor and as a criterion, before a “fresh” assessment is needed. Essentially, we seek to optimally define the appropriate time period for studying incidents relative to a safety climate assessment. We examine two related but conceptually distinct relationships: safety climate → incidents and incidents → safety climate. Thus, we reveal the period of time into the future that safety climate predicts work-related injuries and the period of time from the past that work-related injuries predict safety climate. Both of these relationships are important to study because of their interdependence, as the former (climate → incidents) projects the likelihood of future incidents, while the latter (incidents → climate) provides information about safety climate and identifies one of the critical levers in its development.

To that end, we first briefly review the state-of-the-science on safety climate and its order-dependent relationship with safety incidents. Second, we discuss some issues to consider when using incident data and how we deal with these issues in the current study. Then, we describe our method and present results that examine the issue of shelf life in a sample of over 7,700 workers at 42 sites in 12 countries of a large, multinational chemical processing and manufacturing company. In the discussion, we begin a conversation about the shelf life of safety climate assessments in particular and how these results might be applicable to other psychological constructs in organizational assessments.

Safety Climate

Safety climate is the shared perception of the policies, procedures, and practices related to workplace safety (Zohar 1980, 2011), indicating the extent to which safety is a workplace priority (Zohar and Luria 2005). In this study, we focus on safety climate at the site level; thus, we are examining organizational climate, rather than psychological climate (James and Jones 1974; Ostroff et al. 2012). However, like all climates, safety climate is rooted in the perceptions of individuals; so, climate perceptions are measured at the individual level and then aggregated to the site level (Ostroff et al. 2012).

Safety Climate–Incident Relationship: Temporal Precedence Matters

Zohar (2003, 2011) advanced the compelling notion that safety climate should both predict and be predicted by safety-related incidents in the organization. The former—that safety climate should predict incidents—is consistent with the broader literature and theories about organizational climates (Kuenzi and Schminke 2009; O’Reilly 1989; Schneider and Reichers 1983). However, the latter—that safety climate should be predicted by safety-related incidents—has only relatively recently been acknowledged in the literature (Beus et al. 2010; Zohar 2003, 2011). Essentially, safety climate should be affected by safety-related incidents, because they provide information about the status of safety in the organization (Zohar 2003, 2011).

For simplicity, and consistent with the engineering and economics literatures (e.g., Stock and Watson 1989; Vinnem et al. 2006), we use safety climate as the referent and refer to its different time-sequenced relationships with safety incidents as leading (safety climate → incidents) and lagging (incidents → safety climate). Leading indicators signal events ahead of their occurrence (Hopkins 2009; Mearns 2009). For example, in traffic, amber lights are leading indicators of a red light. In contrast, lagging indicators reflect prior conditions (Hopkins 2009; Mearns 2009). For example, the unemployment rate is a lagging indicator of the state of the economy, as increasingly negative economic conditions precede rises in unemployment rates. Safety climate is a leading indicator when it is used to predict work-related incidents that occur in the future. In contrast, safety climate is a lagging indicator when predicted by previous safety events in the workplace (e.g., arrival of new personal protective gear, cancelation of a safety training program, injuries, fires, and explosions).

Conceptually, safety climate is both a leading and lagging indicator of safety, because it should influence and be influenced by the safety–critical events in the organization (Payne et al. 2009; Zohar 2003). Both leading and lagging relationships are essential to study from a shelf life perspective, because the leading and lagging relationships are interdependent (Payne et al. 2009; Zohar 2003). Safety climate and safety incidents are on going. Neither is static. Instead, they are each in constant, often incremental, adjustment relative to the other.Footnote 1

Neal and Griffin’s (2006) study is one of few studies that assessed both safety climate and incidents more than once. Although not the primary focus of their work, Neal and Griffin (2006) reported correlations over a 5-year period between yearly counts of workgroup accidents and workgroup-level safety climate in years 2 and 4 of the same time period. Their results showed no direct relationship between accidents and subsequent safety climate. Further, patterns between safety climate and subsequent accidents were difficult to explain (e.g., concurrent relationship between accidents and climate in year 4 but not year 2 and predictive relationship between year 2 safety climate and year 5 accidents—but not years 3 or 4 accidents).

In a compelling meta-analysis, Beus et al. (2010) examined safety climate → injury relationships and injury → safety climate relationships separately, at both the individual and the group level. Of particular interest to this study was Beus et al.’s (2010) examination of the length of time over which safety incidents were aggregated as a moderator of safety climate-incident relationships. Although the incident time period did not moderate injuries → organizational safety climate associations, Beus et al. found that organizational safety climate → injuries associations were meaningfully attenuated as this period lengthened. Thus, safety climate had the strongest negative associations with subsequent workplace injuries that occurred in closer temporal proximity to safety climate assessment. This lends support for the notion that a given safety climate assessment is predictive of future injuries for a limited period of time, which might not be well-represented by a 1-year time period (e.g., Neal and Griffin 2006), and that time period effects might not be symmetric across leading and lagging relationships.

The current study explicitly addresses this issue by examining the leading and lagging relationships between safety climate and incidents over 2-year time periods before and after a single, organization-wide safety climate assessment. Specifically, we construct a series of accumulating incident rates, adding 1-month periods to each successive variable, so that we can determine the optimal period for constructing incident rates when (a) predicting safety climate (lagging relationship) and (b) being predicted by safety climate (leading relationship). We examine these relationships considering incidents at four levels of severity. This allows us to begin a discussion regarding the shelf life of safety climate assessments and thus how often organizations should be conducting these surveys.

Modeling Incident Data: Constructing Incident Rates

The leading and lagging relationships between safety climate and incidents can be conceptualized as occurring across incidents themselves or across incidents over time (i.e., incident rateFootnote 2). At first glance, using incidents themselves seems like the appropriate approach, as unsafe events inform workers about the quality of safety in the workplace. Under this paradigm, the question for the lagging relationship is, “how many incidents have to happen (or not happen) for safety climate to be noticeably different?” The problem with using incidents themselves is that there is no straightforward way of conceptualizing “non-events” without referencing another variable. That is, a “non-event” can only be understood in a metric of events.

One solution is to measure incident rate relative to time (e.g., days passed or hours worked) or number of people (e.g., number of employees) or both (Bonita et al. 2006), because rates account for a lack of incidents by describing the density of incidents relative to some baseline. When there is a period of time or a population of people in which no incidents occur, the incident rate is zero, and thus, non-events can be understood. Thus, incident rates are superior to incident counts, because incident rates better define risk by standardizing the unit of comparison and clarifying non-events (i.e., a rate of zero).

Incident rates constructed over time describe how frequently incidents occur within a given time period. These incident rates can be compared so long as the time period is equivalent (e.g., is the incident rate in Year 2 higher or lower than in Year 1?). However, there is little theoretical or empirical guidance as to what makes an appropriate time period. Our study directly addresses the issue of appropriate time periods for constructing incident rates.

Incident rates can also be constructed across people, calculated as the number of incidents relative to the number of people at the worksite. This idea is not new, as epidemiological studies of disease typically account for the incident rates relative to the number of people potentially at risk (Bonita et al. 2006). This is important to model as well, because of opportunity bias; all else being equal, larger worksites staffed with more people have higher probability of having incidents occur than smaller worksites staffed with fewer people. Analogously, state-level car accident rates are determined by assessing the number of accidents relative to drivers,Footnote 3 rather than the sheer number of accidents. Controlling for the rate of drivers is essential, because otherwise California and Texas would—just by the sheer size of the population—have worse accident records than Montana and Mississippi; yet when the number of drivers is accounted for, it is clear that the riskiness of driving is higher in Montana and Mississippi than in California and Texas (Sauter 2012; US Census Bureau 2012).

However, it is not a choice between constructing incident rates with time or with people as the denominator. When constructing incident rates with people in the denominator, time must also be considered,Footnote 4 even if only implicitly. Returning to the issue of car accident rates in different states, the rate of car accidents cannot be determined over the number of people without also indicating the time period in which car accidents are counted. Such statistics are constructed over some time period, such as the number of motor vehicle deaths in a year’s time. Even when such questions focus on the prevalence of events, such statistics consider the number of car accidents in a time frame (e.g., a lifetime). As noted above, incidents need to be accounted for over time, because the lack of incidents cannot be accounted for without considering time. Thus, the question becomes, what is the appropriate time period for constructing incident rates?

The Current Study

The current study addresses this question directly, examining time periods for incident rates as both the predictor of safety climate (lagging relationship) and as the criterion of safety climate (leading relationship). All research on the safety climate-incident relationship examines a set of events over an extended period of time, often 6 months or 1 year (e.g., Katz-Navon et al. 2005; Neal and Griffin 2006; Siu et al. 2004; Vinodkumar and Bhasi 2009; Williamson et al. 1997; Zohar and Luria 2004). Beus et al.’s (2010) meta-analysis shows that the mean time frame over which the lagging injury → climate relationship was estimated in their included studies was 11.24 months (k = 25), whereas the mean time period over which the leading climate → injury relationship was examined was 9.45 months (k = 11). We suspect that researchers decide to aggregate over these time periods, because (a) it is the window of access to organizational records the researchers have been granted and (b) incident data often have low base rates. The latter, in particular, likely encourages researchers to aggregate across large time periods in order to achieve some reliability and variability in the measurement of the events, as the prediction of low base rate events is a well-documented challenge in the organizational sciences (Blau 1998; Hanisch et al. 1998; Harrison and Hulin 1989; Jacobs 1970; Johns 1998). However, as noted above, this assumes that the safety climate assessment at one point in time is still relevant to events occurring over some time period (e.g., 1 year later). That is, the assumption is that the safety climate assessment has not “expired” over that time period. We explicitly test that assumption here.

We chose month-long blocks of incidents as our smallest time unit. Our aggregation choices depart from previous research here, as we divide our incident data into a series of cumulative periods of incidents rates that allow us to test the shelf life of the leading and lagging relationships. As we will describe further in the Method section, we analyze progressively larger incident data windows (i.e., 1, 2, 3 months, etc.) to determine when the safety climate-incident relationship waxes and wanes. Within our series of analyses is the more common 6-month and annual incident rate time frames; so, we can see whether this time frame is too long for optimizing prediction.

Choosing the “correct” period of time to aggregate across is difficult, as there is little theory to guide such a choice and no coherent theory of time in psychology. We chose 1-month intervals as the aggregation periods for our models for several reasons. First, 1 month is sufficiently long to allow for some accumulation of events but still permitted variation both within and between sites in incident rate. Second, from a practical standpoint, it is difficult to envision a scenario in the workplace whereby more frequent assessment of safety climate across the worksite population could be reasonably accomplished, connected to organizational data, and reported back to organizational stakeholders. Third, research on source memory (memory for when and/or where an event occurred; Johnson et al. 1993) suggests that workers are more likely to attribute organizational events to a specific month (e.g., June) than to a specific week (e.g., the 24th week of the year), because months represent more salient and cognitively meaningful time periods relative to weeks which are less easily distinguished from each other. Although employees in this study were not asked to indicate in which months events occurred that informed their psychological safety climate, we felt that it was important to calculate incident rates in a way that was more likely to mirror the cognitive processes used by individual respondents.

Consistent with our above arguments, we also included site size in the incident rate. Because the sites varied considerably in size, this was essential in order to model the risk across sites. Thus, the ultimate incident rate was calculated as incidents per month(s) per employee.

Four Levels of Incident Severity

In addition to aggregating in terms of time and site population, we also aggregated our incident data into four levels of incident severity, following the participating organization’s global incident recording standards. We differentiated incidents based on severity because of the possibility that safety climate-incidents relationships differ in magnitude across incidents types. For example, it is plausible that more severe incidents will be more strongly associated with subsequent safety climate assessments than minor incidents, because the occurrence of more severe incidents could more prominently indicate to employees that safety has a lower organizational priority (Beus et al. 2010). Further, because the organization determined that it was important to categorize these levels of severity separately, we expected that organizational response to these different events might also differ, creating different times to expiry for the safety climate assessment for each incident severity level.

The four levels of incident severity include higher level of actual damage (Level 2), lower level of actual damage (Level 1), near misses, and learning events. The specific operationalizations of these severity levels are described further in the “Methods” section. We anticipate safety climate will be negatively related to Level 2 incident rates, Level 1 incident rates, and near miss incident rates, because greater attention to safety processes by the organization should result in both a better safety climate and a lower number of incidents. In contrast, reporting of learning events was not required in the participating organization; instead, the organization encouraged reporting of events, when employees perceived that information about an event could provide a learning opportunity within or between sites. Thus, although it is likely that learning events are more common at sites with worse safety climate, it is also possible that more learning events are reported by sites that have a better safety climate because of their attentiveness to improving safety.

Summary

This study examines the shelf life of a safety climate assessment relative to both (a) safety incidents prior to the safety climate assessment and (b) safety incidents following the safety climate assessment at the site level of a large, multinational chemical processing company. Incident rates are constructed at the site level as the number of incidents per a particular time period per person, indicating a unitized level of risk across sites, with the time period changing cumulatively across the series of analyses in order to examine changes in the predictive leading and lagging relationships. Because safety climate serves as a lagging and a leading indicator, it should capture to some extent the safety that the organization has experienced in the past and predict the safety that the organization will experience in the future, respectively. Due to the reciprocal adjustments that occur between safety climate and safety incidents, it is essential to understand safety climate as both a lagging and a leading indicator of safety, as it is important to know what has been going on, as well as what is likely to occur in the future. Our goal is to identify when the safety climate-incident relationship expires, or at least fails to gain additional predictive power, indicating that it is time for the organization to deploy another safety climate assessment. Thus, we address these two research questions.

Research Question 1: How far into the future does a safety climate assessment predict a safety incident rate? That is, what is the shelf life of a safety climate assessment as a leading indicator?

Research Question 2: How far into the past from a safety climate assessment does the incident rate predict that safety climate assessment? That is, what is the shelf life of a safety climate assessment as a lagging indicator?

Methods

Participants and Procedures

For a 1-month period in 2007, a health and safety survey was administered to personnel of a large international manufacturing organization. Approximately 14,000 employees were invited to participate. Of those, 8,198 employees responded to the survey (58 % response rate). The data examined in this study are limited to sites in which we had survey responses, organizational incident data, and a site population count, resulting in 7,467 employees at 42 sites in 12 countries.

Employees were sent a survey link embedded in a message about the health and safety survey by the global director charged with safety and health issues. Messages from the global director were also sent to site leadership requesting leaders to encourage employee participation. Banners with information about the survey were also placed on the organization’s electronic employee portal. Each week, site leaders were provided with the number of employees at their site who had completed the survey, with the goal of creating managerial awareness and some competition across sites. Reminders were sent to the employees about the survey approximately once a week. Surveys were administered in nine languages.

Measures

Safety Climate

Safety climate was assessed with eight items adapted from Zohar and Luria (2005). All items were administered on a 5-point agreement scale (1 = strongly disagree, 5 = strongly agree). Example items read “My supervisor insists we wear our protective equipment even if it is uncomfortable” and “Site management is strict about working safely at all times even when work falls behind schedule.” At the individual level of analysis, coefficient alpha was 0.82. Safety climate scale scores were calculated as the mean of all nonmissing items; the site level mean was used in our analyses.

Before conducting further analyses, we tested for sufficient within-site agreement and between-site variability in safety climate perceptions to determine whether safety climate could meaningfully be considered to represent a site-level construct (Bliese 2000). Employees within each site shared a high level of agreement about safety climate (median r*wg(j) = 0.93; Lindell et al. 1999). Further, intraclass correlations demonstrated that meaningful proportions of item variability were explained by group membership and indicated that site means are fairly stable (ICC[1] = 0.02; ICC[2] = 0.89). Taken together, these indices provide sufficient evidence to suggest that safety climate exists at the site level.

Incident Rates

The organization maintains a database of all reported safety-related incidents that occur at every plant and classifies them according to organization-wide standards, based on severity and nature (e.g., injury, fire, etc.). Incidents for 2 years prior to the survey and 2 years after the survey were gathered from the organization’s archives. Over 14,000 incidents were included in the database. All incidents are recorded at the site level.

Five levels of incident severity are indicated in the participating company’s standards. A Learning Event is a situation that warrants information sharing for its potential to mitigate future risk and/or improve controls (e.g., procedure performed without permit, leaking pipe, and lockout/tagout procedures not followed). A Near Miss is defined as a situation or event where given a slight shift in time or distance or other factors, an incident could have easily occurred (e.g., lanyard not secured and pump trips). Level 1 incidents include injuries below Occupational Safety and Health Administration (OSHA) recordable guidelines and generally include first aid (e.g., small cuts, bruises, etc.), which have been previously referred to as microaccidents in the safety literature (Zohar 2000). Level 1 incidents also involve property damage of less than $10,000. Level 2 incidents include injuries that meet OSHA recordability guidelines but are not permanently disabling or deadly. They include, for example, broken bones, strained back resulting in missed days of work, or cuts requiring stitches. Other examples of Level 2 incidents involve damage between $10,000 and $150,000. Finally, Level 3 incidents include injuries that are fatal or life threatening, or cause permanent or long-term disability or functional impairment. Other examples of Level 3 incidents involve extensive financial damage (greater than $150,000). Fortunately, there were too few Level 3 incidents in the 4-year period to model in this study. These various levels of incidents included six general incident types, including: (a) personal injury/illness; (b) fire/explosion; (c) property damage; (d) transportation (i.e., incidents involving transit vehicles such as railcars, trucks, barges, or trailers); (e) security breach; (f) environmental impact (e.g., chemical spill or release).

Incident rates were calculated by counting the number of incidents in the time period and dividing by the site population for each of the four included incident levels of severity. Two series of incident rates were calculated, one for lagging and one for leading relationships, for each of the four incident levels. Essentially, month-long blocks were successively added to create a series of incident rates that included incidents further and further in time from the survey assessment period (Fig. 1). In the lagging indicator analyses, the incident rate is the predictor variable. The first incident rate calculated included only the incidents in the single month prior to the safety climate assessment; the 2-month lagging predictor calculated the incident rate for the 2 months prior to the safety climate assessment, and so forth until all 23 months prior to the survey period were included.Footnote 5 In the leading indicator analyses, the incident rate is the criterion variable. The first incident rate included the single month following the survey assessment period, the second included the 2 months after the assessment period, and so forth until all 24 months of incident data were accumulated into a criterion variable. Thus, there are 23 incident rates for each severity level as predictors and 24 incident rates for each severity level as criteria of safety climate. Table 1 summarizes the incident data for each month individually.

Fig. 1
figure 1

Accumulation of safety incidents to create incident rate variables

Full size image
Table 1 Monthly means and standard deviations of incident rates, per employee (across sites)
Full size table

Control Variables

Several control variables from multiple sources were included as covariates, because they represented regulatory- and engineering-related risks to safety; safety climate should contribute to incidents beyond these variables, just as incidents should contribute to safety climate beyond these variables. First, the risk associated with the chemical processes used at each site was controlled; there are four distinct chemical process businesses at the participating organization, as well as some office sites. The categorization of sites into this variable was determined by our participating organization and is part of their corporate structure; a five-point ordinal scale (0 = office to 4 = most hazardous chemical process) was used to account for this risk as more hazardous processes are likely to be associated with lower perceptions of organizational safety by the employees at the site. Additionally, dummy variables were created to represent five corporate regions (Europe, Asia, Mexico, North America, and South America) due to differences in regulatory statutes and oversight as well as reporting lines within the participating organization. Further, survey respondents indicated their typical working environment (0 = office, 1 = operations), because every site has office workers, regardless of the chemical processes occurring on the site; thus, we controlled for the proportion of operations workers among survey respondents within site. We also included mean site tenure of the respondents to account for opportunity for sharedness in safety climate to develop (Beus et al. 2010). Finally, we controlled for survey response rate, in case responding was influenced by safety climate.

Data Analysis

In many ways, our research resembles previous research on changes in validity coefficients (Henry and Hulin 1987; Humphreys 1968; Keil and Cortina 2001; Mitchel 1975). Like this previous research, we are explicitly examining the role of time in the predictor-criterion relationship. Further, like this previous research, there are several possible levers of change that could cause changes in the relationship. In the predictor–performance criterion relationship, changes could arise through changes in person’s skills or changes in tasks, or both (Henry and Hulin 1987; Keil and Cortina 2001). In our research, changes in safety climate or changes in incident rate at the site level, or both, could cause changes in predictiveness. However, our research departs from the previous line of inquiry in an important way. In the dynamic predictor-criterion literature, time is a moderator of the predictor-criterion relationship (Keil and Cortina 2001). In the study here, time is not a moderator per se; instead, we examine time as part of the incident rate variable, changing the actual calculation period of incident rates (whether predictor or criterion). Thus, although the changes in the time period are expected to change the relationship between safety climate and incident rates, it is not because we have pinpointed a different time for incident measurement but rather that we have changed the nature of the incident measurement.

Two sets of analyses were performed at the site level, the first using safety climate as a leading indicator of incidents and the second using safety climate as a lagging indicator of incidents. All analyses were performed using OLS regressions. In the leading indicator analysis, safety climate was used to predict the four different incidents rates, one for each level of incident severity (Learning Events, Near Misses, Level 1, and Level 2) using each of the 24 different accumulations of incidents as the incident rate dependent variable, with the first model including incident rates for only the month following the survey period and the 24th model including the incident rate for the 2 years following the survey period.

In the lagging indicator analysis, the four incident rates in the 23 months preceding the survey were used to predict safety climate. As in the leading indicator analysis, each incident rate was entered in a series of 23 regression models, but this time as a predictor. The first model used incident rates from the month preceding the survey, whereas the 23rd model included the nearly 2-year period prior to the survey. The lagging indicator analysis was also repeated with all four incident rates entered together as a block of predictors in a series of 23 regression models, using the graduated cumulative incident rates as the predictor variables.

Although the number of incidents and the number of surveys were both quite large, the analysis was performed at the site level because the incident data were at the site level. Thus, the sample size for all regression models was the number of sites: 42. Given this small sample size and consequent low power, results were interpreted in terms of effect size rather than in terms of statistical significance. The effect sizes examined were semipartial r 2 (sr2) values, the proportion of variance that is uniquely attributable to the independent variable of interest: safety climate assessment (leading indicator analyses) or the various incident rates (lagging indicator analyses). Following Cohen’s (1988) rule of thumb for unique variance accounted for (η 2 in the ANOVA framework), a sr2 of at least 0.01 was considered to be large enough to be interpreted. Compared to the effects expected for the set of control variables (including the inherent differences in risk across sites based on the chemical process in use at the site), sr2 of 0.01 is likely to be relatively small. However, 1 % of the variance in incidents that totaled greater than 14,000 over 2 years is likely to be practically important to the participating organization and the people in it.

Results

Table 2 contains the means, standard deviations, and correlations among the study variables at the site level. Incident rates are included in the table as aggregations over the first year and over the second year both before and after the survey period. Thus, for each incident level, there are four incident rates calculated.

Table 2 Descriptive statistics and correlations among the study variables at the site level
Full size table

Safety Climate as a Leading Indicator

The first research question asked how far into the future a safety climate assessment can predict safety climate incident rates. A series of 24 regressions were undertaken for each of the four incident rates (Learning Events, Near Misses, Level 1, and Level 2). Each set of 24 regressions is distinct, with a different dependent variable.

In all regressions, control variables were entered first (see Table 3 for a summary across models within each series of regressions), followed by the safety climate assessment. Unsurprisingly, a substantial amount of variance is explained by the set of control variables given that they reflect engineering-based risk (i.e., different chemical processes) as well as different regulatory rules and oversight. For the leading relationship, the site process risk was the strongest predictor of incidents for all levels of severity, except for Level 1 incidents in which process risk and percentage of respondents who were in operations versus office locations within site were approximately equally predictive.

Table 3 Summary of variance accounted for (sr2) by control variables
Full size table

Table 4 contains the sr2 and unstandardized regression coefficients; Fig. 2 displays the sr2 graphically. We focus on sr2, rather than regression coefficients, because our interest is in the variance accounted for by safety climate as a predictor of incident rates. As seen in Table 4, very few of the sr2 were 0.01 or higher in magnitude for Learning Events or Near Misses, with prediction of only the 5- and 6-month Learning Event variables exhibiting a sr2 greater than 0.01. Although these values are in the range of small effects (Cohen 1988), it is difficult to discern why these two dependent variable time periods, and no others, were predictable. Thus, our conclusion is that site-level safety climate assessment is unable to predict the rate of Learning Events or Near Misses over a 2-year period.

Table 4 Semipartial r 2 and unstandardized regression coefficients for safety climate assessment as a predictor of later safety incidents (i.e., safety climate as a leading indicator)
Full size table
Fig. 2
figure 2

Graphical representation of the semipartial r 2 for safety climate assessment as a predictor of later safety incidents (i.e., safety climate as a leading indicator)

Full size image

However, all of the regressions predicting Level 1 incident rates had sr2 for the safety climate assessment that were greater than 0.01. In the month immediately following the safety climate assessment, the sr2 was 0.018 and then quickly increased to 0.052 in the next regression (i.e., the 2 months following the safety climate assessment). As seen in Table 4 and further illustrated in Fig. 2, the sr2 stays between 0.032 and 0.048 for the regressions modeling Level 1 incidents up through 9-months post-survey, and then changes in magnitude to around 0.055 (range 0.051–0.061) for the remainder of the 2-year window. Thus, a single safety climate assessment contributes to the prediction of Level 1 incidents—the least damaging of actual incidents (rather than Learning Events or Near Misses)—for at least 2 years following the survey period. Thus, the typical year-long accumulation of incidents in safety climate research appears to be an appropriate time period for obtaining optimal prediction of incidents.

Finally, an interesting pattern emerged in the prediction of Level 2 incident rates. For the month immediately following the survey period, the safety climate assessment had a sr2 of 0.108. However, as additional months were accumulated into the dependent variable, the sr2 quickly decreased; by the 4-month incident rate, the sr2 was just above 0.01, and by the 7-month incident rate, the sr2 was more often below 0.01 than above. This is not to discount that there were many cumulative periods of safety incidents post-assessment that were predicted by the safety climate assessment, but rather to highlight that incidents in the period immediately following safety climate assessment were highly predictable by the assessment compared to the incident rates that included late months in the 2-year window. In sum, it appears that Level 2 incidents—more serious than Level 1, but falling short of catastrophic events—are more predictable by a safety climate assessment in the very near term following safety climate assessment than at later dates. Thus, the typical aggregation of incidents over a year-long period—or even a 6-month period—would suggest that safety climate can do little to predict these more severe incidents, yet if incidents were aggregated over a shorter time period (e.g., 1–2 months after the survey period), then a much different picture would emerge.

Safety Climate as a Lagging Indicator

The second research question asked how far from past incidents that a safety climate assessment acts as a lagging indicator of safety. Here, safety climate is the dependent variable. As in the models in which safety climate is a leading indicator, control variables were entered in the model first (Table 3). Again, it is unsurprising that control variables accounted for a substantial amount of variance in the models. Two different series of analyses were conducted. In the first series, site-level safety climate was predicted by each of the four incident rates separately, using the accumulated months incident rate variables for the 23 months prior to the safety climate assessment; in the second series, the same variables were used but were entered in one step as a block to determine the effect of the total set of incident rates. For Learning Events, Near Misses, and Level 2 incidents examined separately, the proportion of respondents in operations positions was the control variable that had the largest effect on safety climate assessment. For Level 1 incidents as well as the analysis in which all four incidents were entered as a block, the site process risk had a slightly stronger average effect on safety climate than did the proportion of respondents in operations positions.

Series 1: Separate Regressions for Each Incident Rate

In contrast to the regressions using safety climate as a leading indicator, the analyses of safety climate as a lagging indicator show that each of the four incident rates predict later safety climate. Results for these analyses appear in Table 5 and Fig. 3. Looking first at the analyses for Learning Events, for the month immediately preceding the safety climate assessment, the sr2 is 0.035. In the following regressions for the 2-month through 6-month period prior to the survey assessment, the semi-partial r 2 varies between 0.036 and 0.064. However, beginning with the 7-month period prior to the survey, the sr2 is 0.097, and for the remaining period—nearly 2 years—prior to the assessment, the sr2 for each regression is 0.110 (the 23-month variable) or above, with a peak of 0.133 in the 13-month variable regression. Thus, with Learning Events as the predictor, with inclusion of more time ahead of the safety climate assessment—at least up through approximately 1-year—prediction of safety climate is improved. Thus, for Learning Events, the typical 1-year aggregation of incidents would evidence a relationship with safety climate, but even longer periods (e.g., 2 years) further optimizes this predictive relationship. Note also that Learning Events are negatively related to safety climate, indicating that more Learning Events are related to worse safety climate. Earlier, we suggested we could not anticipate the direction of the Learning Events-safety climate relationship, because the recording of Learning Events was encouraged but optional within the participating organization; thus, recording Learning Events could indicate a healthy safety climate of a site that focused on learning from safety-related incidents or could be related to worse climate, because worse climates would have more incidents to learn from. The negative relationship between Learning Events and safety climate supports this latter view, with more events associated with worse climate.

Table 5 Semipartial r 2 for safety climate assessment as predicted by prior safety incidents (i.e., safety climate as a lagging indicator)
Full size table
Fig. 3
figure 3

Graphical representation of the semipartial r 2 for safety climate assessment as predicted by prior safety incidents (i.e., safety climate as a lagging indicator)

Full size image

Turning next to the analyses for Near Misses, nearly all of the regressions have sr2 above 0.01. The time periods accumulated just prior to the safety climate assessment are associated with the smallest sr2 (e.g., 0.014 for the 4- and 5-month predictor variables); as more time is accumulated in the predictor incident rate, the sr2 increased to 0.036 (the 23-month variable). The exception to this trend is the 1 month prior to the safety climate assessment, which had a sr2 of 0.029. Thus, there might be greater predictiveness in the very short term than in the near term, but generally greater predictiveness in the longer-term for Near Misses as the predictor and safety climate assessment as the lagging indicator. Interestingly, this suggests that the typical annual aggregation of incidents is no better, and possibly slightly worse, than using just the month prior to the safety climate assessment; further, using a 6-month period might understate the effect of incidents on safety climate, while using a considerably longer period (e.g., 2 years) accounts for greater variance in safety climate.

Level 1 incidents exhibited a pattern similar to Learning Events. All regressions exhibited sr2 that exceeded 0.01. For the regressions using the months closest to the survey period, the sr2 were the smallest for Level 1 incidents, whereas those regressions using longer time periods before the survey were associated with higher levels of sr2. The effects were always smaller than those for Learning Events, but in the closest month periods and in the longest time periods the effects were nearly equal. The conclusion from this series of analyses is that Level 1 incidents are important predictors of a safety climate assessment, and the ability to predict the safety climate assessment improves as more time accumulates in the prediction of safety climate (at least, over a nearly 2-year period). Further, it appears that the common 1-year period for accumulation of incidents would demonstrate prediction of safety climate, but even longer periods (e.g., 2 years) further optimize the validity coefficient.

Finally, Level 2 incidents exhibited a U-shaped trend. The regression for the month immediately preceding the survey period had a sr2 of 0.058. As months were added to the predictor variable, the sr2 dropped steadily, falling below 0.010 at the 5-month period and going as low as 0.000–0.002 for the 7-, 8-, 11-, and 12-month variables. However, when the 15th month before the survey was accumulated into the predictor variable, the sr2 rose to 0.017 and continued to rise for each month added through the remainder of the 2-year window, with a sr2 of 0.080 for the 23-month predictor variable. Thus, for safety climate as a lagging indicator, very recent Level 2 incidents loom large, as do events that occurred at least 15 months prior. These results suggest that the typical accumulation of incidents into a 1-year period would give the impression that incidents do not predict safety climate, yet other time periods—especially when considering only the nearest term—show that, in fact, incidents do matter.

Series 2: Regressions Including All Four Incident Rates Entered as a Predictor Block

In the second series of analyses, site-level safety climate was predicted by the block of four incident rates simultaneously, with a different regression for each of the 23 accumulated month incident rate variables from the period prior to the safety climate assessment. Results for these analyses appear in Table 6 and Fig. 4.

Table 6 Semipartial r 2 for safety climate assessment as predicted by prior safety incidents (i.e., safety climate as a lagging indicator) entered as a block
Full size table
Fig. 4
figure 4

Graphical representation of the semipartial r 2 for safety climate assessment as predicted by prior safety incidents (i.e., safety climate as a lagging indicator) entered as a block

Full size image

As a block (Table 6, last column), the four incident rates considered simultaneously together were each negatively related to safety climate. As a block, they gained predictive power as the amount of time included in the incident rates increased, until the 8-month variable; then, the effect size remained approximately equal across the accumulated time periods for the next seven regressions (all but two sr2 in the range of 0.157–0.164) before slowly declining to 0.137 in the 23-month variable regression. Looking individual contributions of the incident rates as predictors when entered as a block, Learning Events have the greatest average contribution to predictive power. In the nearer terms, Level 1 incidents have nearly equivalent predictive power to Learning Events; in the middle range of this window, Level 2 events outperform Level 1 incidents and have nearly the predictive power of Learning Events. Near Misses, on the other hand, never rise to the 0.01 standard and therefore do not appear to contribute to the prediction of safety climate when the other incident rates are also accounted for. Thus, it appears that the block of incidents together are important to the prediction of safety climate across the entire 2-year window before the safety climate assessment, with Learning Events as the most important of the predictors, Near Misses as an inconsequential predictor, and the usefulness of Level 1 and Level 2 incidents depending on the time window that is under consideration.

Discussion

The goal of this paper was to determine the shelf life of a safety climate assessment. Our results indicate that it depends on whether we are examining leading or lagging effects as well as the kind of incidents. However, when considering the most critical relationship where safety climate predicts the more severe Level 2 incidents, it is clear that the shelf life of the assessment is extremely short. The ability to predict such incidents is optimized in the month following the safety climate assessment and expires after 3 months. These results indicate that the assumption (often unstated in organizations) that a safety climate survey can be conducted annually—like a typical job satisfaction survey or a health insurance satisfaction survey—and be assumed to retain its explanatory power for 12 months is incorrect, at least for more threatening events. If organizations want to be able to predict which sites (or other sub-groups) are most likely to experience the more threatening events, then these results suggest that organizations should conduct safety climate assessments quarterly at the least and possibly as often as monthly.

As for the other relationships examined, our results show that there is fairly consistent predictive power of a single safety climate assessment for less severe incidents over a 2-year period. In some cases, the consistency is the lack of predictive ability (i.e., leading effects in predicting Near Misses and Learning Events), whereas, in others, there are some steady gains over time as the incident accumulation period expands (i.e., leading effects in predicting Level 1 incidents; lagging effect in prediction by Near Misses, Learning Events, and Level 1 incidents). Thus, there is no single shelf life of a safety climate survey; instead, shelf life depends—like validity—on use. Our conclusions and recommendations about the frequency of safety climate assessment draw upon the criticality of predicting more severe events.

Next, we turn to further discussion of the findings of the current study. Then, we acknowledge some of the limitations of our work. Finally, we turn to broader issues in understanding shelf life and the factors—beyond the scope of this work—that might contribute to the expiration of a climate assessment in its ability to predict, or be predicted by, workplace events.

The Current Study

There were several interesting findings in our results. First, the incidents that are predicted by safety climate are not entirely the same incidents that predict safety climate. Safety climate is a substantial lagging indicator of Learning Events, but not a leading indicator. Considering incidents where actual damage occurred (Level 1, Level 2), the safety climate assessment was an important leading and lagging indicator over the 2-year period for the less severe Level 1 incidents; in contrast, safety climate was usually an important leading or lagging indicator only for more severe Level 2 incidents that occurred in months closest to the assessment period. The exception to this was that Level 2 incident rate periods including months more than 15 months prior to the safety climate assessment also predicted the subsequent safety climate assessment. It is important to recognize, however, that although the general effect is the same whether a leading or lagging relationship is considered for these incidents, the leading and lagging relationships with these incident rates are not symmetric; that is, the total variance accounted for is not the same for the leading and lagging relationships. Finally, safety climate was never a good leading or lagging indicator of Near Misses. In brief, our results show that the relationship between safety climate and incidents depends on several factors: (a) the kind of incident; (b) the time period over which incident rates are accumulated; (c) whether safety climate is a leading or a lagging indicator of incidents.

Although it is important that a safety climate assessment serve as both a leading and a lagging indicator of safety in the organization, it is especially important from a practical standpoint to have safety climate predict incidents. Thus, one of the most interesting issues in our work is why safety climate is a leading indicator of some incidents and not others. In particular, the inability to predict Learning Events and, especially, Near Misses in our data is of interest. Near Misses are just that—very near to disaster, but just missed. That is, they are events that but for slight differences in circumstances, actual damage would have occurred; it seems unlikely that the root causes of Near Misses differ from the Level 1 and Level 2 incidents that caused actual personal or organizational damage. From a practical standpoint, it is particularly interesting that root cause analysis—an iterative investigative process intended to discover the true causes underlying an event and not just the most proximal contributors—is encouraged for near misses (Berry and Krizek 2000; Phimister et al. 2003) in addition to incidents that cause actual harm, yet near misses were not predicted by safety climate, nor were they a good predictor of safety climate, especially when the other categories of incidents were included (Table 6; Fig. 4). The lack of relationship between Near Misses and safety climate is disconcerting. Although beyond the scope of this paper, it is worth asking whether near misses are either or both leading or lagging indicator of incidents that cause actual harm, and if not, why not.

In contrast, safety climate was a lagging (but not leading) indicator of Learning Events; as the window of observation increased for previous incidents, safety climate was better predicted. This suggests that sites might not have learned from these events, as the relationship between Learning Events and subsequent safety climate assessment should be disrupted if (a) the Learning Event is safety critical and (b) the site resolved the situation. This, of course, assumes that sites do not “trade off” problems, such that once a particular Learning Event is learned from, new and different problems (that were not happening or were not being recorded during the previous period) arise.

Further, the guidelines at the participating organization for Learning Events and Near Misses might not be as clear as desired (unlike Level 1 and Level 2 incidents, which have clear guidelines and standards, such as cost of property damage or whether an injury is required to be reported back to OSHA). It may be the case that Near Misses can be categorized as Learning Events, and this is why Learning Events, but not Near Misses, predict safety climate. There could be political forces at work inside the participating organization whereby it is less problematic for site management to have high rates of Learning Events rather than Near Misses. Regardless, the results suggest that the organization does not learn from Learning Events, whether they were Near Misses or something else, because there is no disruption in the prediction of safety climate from much earlier events. Further, it is clear that organizations must be as concerned about their incident recording processes as they are about their survey data; incident data are not as “objective” as they seem given that people make judgment calls about how to categorize and describe events.

Further, safety climate was a useful leading and lagging indicator of Level 1 incidents over the entirety of the pre-survey and post-survey 2-year incident windows. This might be an indication of what the safety climate items particularly tap in the perceptions of workers, even though the items were not meant to cue respondents to low-level injuries (e.g., small abrasions or cuts requiring first aid) and incidents (e.g., fender-benders on chemical plant grounds, broken alarms, and minor loss of chemical materials). Further, it might be the case that because the severity of these events is rather low (e.g., injuries are not reported to OSHA and monetary damage is relatively small), the organization might not attempt to change the conditions surrounding these events; so, a single safety climate assessment serves as a long-lasting lagging indicator and continues to serve as a leading indicator well into the future. It may also be the case that the causes of less severe incidents and more severe incidents differ (Wallace and Vodanovich 2003); for example, it might be that mental lapses and cognitive failures cause low-level injuries (e.g., tripping on a set of stairs) but organizational priorities cause severe injuries (e.g., broken bones when falling from height while conducting maintenance).

Finally, regarding the more severe Level 2 incidents, safety climate was a substantial leading and lagging indicator, but mostly in the months particularly close to the survey period. As we will discuss further below, it could be that these incidents garnered significantly more attention and responsiveness from the organization relative to the other categories of incidents. Thus, it could be that the safety climate assessment quickly expired as a leading or lagging indicator, because the occurrence of these events led to organizational interventions that prevented their recurrence at the particular sites that experienced them. Further, when incidents prior to the survey period by 15 months or more were included in the incident window, safety climate again served as a lagging indicator. It is difficult to speculate why this might be, but it is possible that investigations of events take considerable time and become more commonly thought of as more information is released following investigations. Alternatively, a site might invest in resources following an incident to mitigate future similar events, and the pressures to follow the new protocols—or the financial investments in those new resources—might run out after a year of implementation (and implementation does not usually occur the day after an event but rather in the weeks or months following it), causing those events from 15 months or prior to become predictive of safety climate. Future research should investigate this intriguing finding. Regardless, the results for Level 2 events show that—unlike other incidents—the common 6-month or annual accumulation of incidents would suggest that safety climate cannot predict, or be predicted by, these relatively severe events, yet this result is driven entirely by the accumulation period. If very short time periods are used instead, it is clear that the sites at greater risk for Level 2 events can be identified using a brief safety climate assessment.

Limitations

Before discussing our results further and linking them to broader issues in examining the shelf life of a safety climate assessment, we need to acknowledge some of the limitations of our research. First, we acknowledge that although we had a large number of useable survey responses as well as a vast database of incidents, all analyses were conducted at the site level, which limited our sample size to 42. Although this is not an inconsequential size, it does limit the power available as well as the degrees of freedom available in regression models. We acknowledge that it is possible that we are overinterpreting somewhat small fluctuations in predictability based on a rule of thumb from Cohen (1988) rather than being able to test for significance.

Second, although these analyses were conducted in a large multinational organization, it represented only one industry (i.e., chemical processing). There are likely to be differences in inherent risk, operational tempo, and government or industry regulation that influence the rate of safety–critical events across industries. The recommendations we made above regarding the frequency of safety climate assessments come from an organization in the chemical processing and manufacturing industry, which is likely to have a rate of safety–critical events that is higher than many other industries (e.g., academia, retail sales, and hospitality). However, because the chemical processing industry is a high reliability industry where safety incidents have catastrophic potential (Roberts 1990), it tends to have stricter safety regulations than other industries where safety is likewise a critical concern (e.g., construction, agriculture; see www.csb.gov). Thus, this sample could have had fewer safety incidents over the examined time periods than a sample taken from a different industry that has a high level of inherent risk. Our discussion of our results should be interpreted with this in mind. A possible future research avenue would be to conduct this kind of research in multiple industries, using inherent risk factors and previous industry-level incident rates (e.g., OSHA recordables by industry, Bureau of Labor Statistics data, or New Zealand’s Accident Compensation Corporation [see www.acc.co.nz]) as moderators or controls in the analysis (Smith et al. 2006).

Third, we were unable to model the most severe (Level 3) incidents—those that are the most damaging to person and property—because their rate was too low. This is a fortunate result for our partner organization, but consequently we are unable to provide any information about safety climate as a leading or lagging indicator of these events. Because these are the most safety critical events, knowing more in particular about how to predict them is important to the health and well-being of organizations and their personnel.

Further, we conducted these analyses with only eight safety climate items. On one hand, this is a limitation, because we might have missed some important factors in safety climate that might have provided greater levels of predictive validity than the few that were included. On the other hand, this is a strength of our work, because it shows that a safety climate assessment does not have to be particularly long to take the pulse of the organization and predict which sites are more likely to have serious incidents in the next few months, or even less severe incidents over the next few years.

Additionally, we chose to use 1-month periods as the smallest unit to increase in our graduated incident rates. We certainly could have chosen smaller periods (e.g., hourly, daily, weekly, and biweekly) to further refine our understanding of the optimal time period for incident rates. We chose 1-month periods in part for theoretical reasons (Johnson et al. 1993), but some of this choice was due to practical reasons, such as the number of regressions that would have to be presented with smaller time periods. Certainly, we could have used smaller time periods between months to further pinpoint optimization in prediction.

Finally, although we had a rich dataset, there were numerous additional pieces of information that could have shed further light on the expiration of the safety climate measure as a leading or a lagging indicator. As we address further in the remainder of the discussion, it is not just incidents that influence safety climate. Information about these other occurrences could help explain why the safety climate-incident relationship waxes or wanes over time.

What Factors Influence the Shelf Life of a Safety Climate Assessment?

In the following, we address several factors that are likely to influence the shelf life of a safety climate assessment. This is not meant to be exhaustive but rather a selection of factors that might be the most promising to consider in research on the shelf life of safety climate assessments. Further, this list is not meant to reflect any specific events that happened in our participating organization; unfortunately, we do not have information about turnover rates, interventions, or post-incident responses. Instead, we offer these ideas to open the discussion about the shelf life of safety climate assessments in the hopes that more critical examination of the assumptions regarding the aggregation of incident rates will occur. We anticipate that many of these issues will also be relevant to assessments of other types of climate.

Organizational Responses to Incidents

Although incidents should directly influence safety climate (Zohar 2003, 2011), the organization’s responsiveness to incidents should also inform safety climate. By responsiveness, we mean the post-incident investigation and intervention efforts, rather than the in-the-moment actions that attempt to mitigate harm to person and property. Such responses to incidents could change both the objective risk in the organization and the safety climate. Regarding objective risk, organizations could change work processes or complete maintenance following an incident; these behaviors would change the objective organizational environment and work processes, which should have an effect on the totality of risk to the workers and the organization. As for safety climate, events that are not treated seriously at the managerial level but are deemed serious at the individual level could cause safety climate to worsen even further than the negative effects of the event itself because of the mismatch in the priority of safety for management relative to employees. In contrast, organizational responses to incidents could mitigate the incidents’ negative effects on safety climate, because the organization’s prioritization of safety could be demonstrated and reinforced. Thus, safety climate should be a lagging indicator of responsiveness to incidents.

Interestingly, safety climate should be a leading indicator of organizational responses to incidents and not just incidents themselves; sites that are less likely to have incidents should also be better at responding to incidents because of their better safety climate. This suggests that there might be a spiral of gain or loss in the safety climate-incident relationship, where the good get better and better while the bad get worse and worse. Unfortunately, we do not have multiple assessments of safety climate here to examine this potential relationship.

Other Organizational Interventions

Organizational responses to incidents are not the only interventions that occur. Whereas responses to incidents are reactive and can only affect the shelf life of safety climate as a leading indicator (i.e., future responses cannot affect past incidents and past climate assessments), other organizational interventions can influence both the leading and the lagging relationship. Interventions are designed to change the organization. If done well, interventions should not only change the organization, but also the perceptions of the organization. That is, a well-designed and well-implemented intervention should change the level of risk, the actual safety, and the safety climate. Proactive interventions (such as the launch of a new safety program, the arrival of new personal protective gear, or the deployment of additional training, audits, or oversight at organizational sites that have poor safety climate assessment scores; Zohar and Luria 2005) should interrupt the trends in the safety climate-incident relationship that were ongoing, effectively making the safety climate assessment expire. For example, if a large-scale intervention were to be successfully implemented 6 months prior to the assessment period, then incidents that occurred 7 months prior to the assessment period (or earlier) are less likely to predict the safety climate assessment. The same would be true for a successful intervention occurring 6 months post-assessment and the prediction of subsequent incident rates.

The administration of a safety climate survey in and of itself could be conceptualized and interpreted as a relatively simple safety intervention. It certainly signals to employees that management is potentially interested in their perceptions of safety rules and the enforcement of them. Changing to a predictable cyclical administration of a safety climate measure (e.g., monthly or quarterly) could also be conceived as a safety intervention as it signals a desire to gather this information more frequently and to monitor it more closely.

Importantly, the expiration of a safety climate assessment under such conditions might be a useful indicator of the success of the intervention. Of course, direct evaluation of the intervention should also occur. But, the inability of a particular assessment period to link to incident rates beyond the intervention window could be additional evidence of the intervention’s success (assuming that mean levels of the safety climate assessment are higher).

Personnel Changes

Further, personnel change could create changes in safety climate. Because safety climate is shared among employees, it should remain relatively stable across small numbers of personnel changes but should change as greater numbers of personnel changes occur, especially as the number of key stakeholders (such as management) who turnover increases (Beus et al. 2010; Zohar 2011). Extensive turnover within the organization, a merger, or an acquisition could render the assessment expired, because it no longer represents the perceptions of enough of the current population (cf. Beus et al. 2012).

Other Factors to Consider When Assessing Shelf Life

Beyond the factors that directly influence the shelf life of a safety climate assessment, we must also consider the factors that might obscure the shelf life.

Memory Biases

When considering a safety climate assessment, it is important to recognize that memory biases play a role in individual perceptions of climate. For example, negative events tend to be weighted more strongly than positive events (Kahneman and Tversky 1979), such that incidents that threatened safety are likely to count more heavily in perceptions of safety climate than would a safety training program or the provision of personal protective gear. It is also important to recognize that memory biases do not only occur at the individual level. Organizations develop reputational histories, with legends that are passed down across generations of workers (Chao et al. 1994). New workers become socialized to know about “old times” and catastrophic events—or even more minor events—that happened prior to the worker’s arrival. Thus, it is possible for events prior to a worker’s organizational tenure to influence their perceptions of safety climate.

Although memory biases can only directly intervene in the lagging relationship, they still influence the leading relationship because of their influence on assessments of safety climate and on the practices that arise through socialization (Chao et al. 1994). To the extent that perceptions of safety climate and its subsequent assessment are contaminated by memory biases and other misperceptions of the safety of the organization, a safety climate assessment will have impoverished prediction of future events. However, to the extent that particular historical events are highlighted, memorialized and mythologized, and passed down through workers over time, then both the safety climate assessment and the actual safety practices will be affected. Thus, although memory biases in general should result in less than optimal prediction of incidents by safety climate, some of the group-level memory practices should affect both safety climate and safety practices in similar ways.

Underreporting

Another issue to consider is the extent to which incidents are reported. Underreporting of safety-related incidents is a well-documented concern and reality (e.g., Leigh et al. 2004; Probst et al. 2008; Probst and Estrada 2010). As we noted above, there was more leeway in reporting Near Misses and Learning Events—overall, as well as how they were categorized when reported—compared to Level 1 and Level 2 events, and this might account for the fact that Learning Events, but not Near Misses, was a predictor of safety climate.

In any organization, there could be disincentives for reporting (e.g., time, effort, and peer pressure) or misunderstandings over the definitions or minimal criteria for incidents. In order for organizations to learn from their safety records, employees need to know what should be reported, encouraged to do so, and not be punished (formally or informally) for doing so. The shelf life of a safety climate assessment as a leading or lagging indicator of incidents can only be determined relative to the incident data available. Poor incident records will distort the shelf life of an assessment.

Speed of Change

The speed at which safety climate changes is likely to be influenced by both the rate of incidents over time as well as the size/severity of incidents. These are linked to the operational tempo and inherent risks of an organization, respectively. By operational tempo, we mean the density of daily work activity and production at a site (Britt et al. 2005; Castro and Adler 2000). By inherent risk, we mean that organizations and organizational processes differ in their possibility of harm to people, the organization, and the environment (e.g., chemical processing, construction, transportation vs. academia, hospitality, and retail sales). We contend that organizations with a higher operational tempo are likely to see more rapid change in safety climate than organizations with a lower operational tempo. Likewise, events that are bigger should have a greater effect on safety climate than those that are smaller in scope; regardless of the rate of incidents, organizations with greater inherent risk are more likely to experience severe incidents than those with a lower inherent risk.

Higher operational tempo should lead to faster changes in safety climate merely because more happens at that organization over time. From a safety–critical incident standpoint, the incident rates should be higher in organizations with higher operational tempos. Further, higher operational tempo does not have to occur only at the production level; some organizations institute training, safety programs, and other organizational changes at a higher rate than others. The implication is that organizations with a higher operational tempo will need to conduct safety climate assessments more frequently, because they will expire faster in their organizations, because conditions within the organization will change more rapidly than at organizations with lower operational tempos.

Bigger events—events that affect more people or that inflict greater damage (or create greater improvement, such as a training program), or both—should also have a greater effect on safety climate than would smaller events. If catastrophic events do occur, safety climate is likely to rapidly change—just as massive (successful) undertakings to improve safety are also likely to change safety climate relatively quickly, compared to smaller events. Further, bigger events grab the attention more than smaller events do (Kahneman and Tversky 1979). They also affect or are witnessed by a greater number of people, causing more individuals to shift their psychological climates, resulting in a shift in the organizational climate as well. Relatedly, shocks (Thompson 1967) as opposed to expected events are likely to result in more rapid change for safety climate. Shocks are just that—unexpected jolts to the system to which the organization and its people must rapidly adjust and make sense of (Weick 1995; Weick et al. 2005). Change in safety climate is one of those adjustments. In contrast, expected changes allow people time to adjust and engage in sensemaking over time (see also Prochaska et al. 1994).

Can We Use Multiple Safety Climate Assessments to Predict Incidents?

Another interesting issue to consider in shelf life research is whether multiple assessment points can increase the predictiveness of future events. Further, could the change in climate over time be predictive of future incidents? That is, might there be an effect of the improvement or disintegration of safety climate over time, in addition to the direct effect of the level of safety climate at each of those time periods?

Future research should examine these questions. Further, it is possible that technology could create near real-time assessment of safety climate, finding “hot zones” in organizations that might best pinpoint critical time periods and better prevent disasters. Undoubtedly, technology already exists that allows for people to provide their opinions or perceptions in real time (e.g., reactions to candidates during debates). Similarly, organizations could assess safety climate among employees by asking employees to set their perception of climate upon arrival and to change their setting as events occur throughout the day. Of course, there would be numerous logistical concerns, such as whether these types of measures could distract workers (putting them at risk) and how to maintain confidentiality of employee responses.

Conclusion

Current safety climate theory and research do not address the optimal aggregation period for incident rates, making it impossible to know when a safety climate assessment expires as a meaningful leading or lagging indicator of safety incidents. From a practical standpoint, it is essential to recognize when safety climate needs to be reassessed—even if no major incidents have occurred—because of its ability to improve the predictability of harmful events beyond objective organizational factors such as size or inherent risk. Our study begins to identify when a safety climate assessment expires in order to determine how often an organization should assess safety climate, a critical component of its defense against potentially life-threatening events. Our results show that for the most critical relationship—predicting more severe incidents by safety climate—organizations should deploy safety climate surveys at least quarterly if not monthly. Further, we have outlined numerous ways that a safety climate assessment can expire, because conditions in the organization have changed. We hope that this paper spurs safety climate researchers and safety practitioners to consider investigating the shelf life of safety climate assessments to determine when they expire and to subsequently further refine recommendations for the frequency of safety climate assessment and the aggregation period for incident rates, in the service of greater health and well-being for organizations and workers.