Introduction

The US Bureau of Labor Statistics has reported an upward trend in total occupational fatalities since 2013. In 2019, occupational fatalities exceeded 5,000, the largest annual number in over a decade (U.S. Bureau of Labor Statistics, 2020). Construction is among the deadliest US professions. In 2019, construction trades had the second highest number of total fatalities and a fatality rate that was over 10 times the national average for all occupations (40 vs. 3.5 per 100,000 full-time equivalent workers; U.S. Bureau of Labor Statistics, 2020). Research indicates that workers employed by residential construction firms have an elevated risk of fatality—particularly from experiencing falls—even within their already high-risk industry (Dong et al., 2014; Sa et al., 2009). For example, Dong and colleagues (2014) analyzed two large national datasets (Census of Fatal Occupational Injuries and the Current Employment Statistics), revealing that between 2003 and 2010 nearly 80% of fatalities in residential construction were due to falls. Similarly, Sa et al. (2009) found that residential roofers in particular were 57% more likely to experience a fall than commercial roofers. Moreover, the proportion of residential roofers rating their worksites as “very unsafe,” was about five times greater than commercial roofers (11% vs. 2%). Kaskutas et al., (2010) reported similar findings with apprentices; those working in residential construction were more three times more likely to report a fall from a ladder (odds ratio = 3.13, 95% CI = 1.13, 8.83) and twice as likely for any type of fall (odds ratio = 2.10, 95% CI = 1.27, 3.48) as compared to apprentices who only worked on commercial construction sites.

Barriers to addressing the problem include difficulties in identifying and reaching residential construction companies to assess their safety practices and develop interventions (Hurtado et al., 2020). Such barriers include the smaller size of residential construction companies compared to their commercial counterparts, as well as more frequent movement of workers across sites. Some intervention work targeting fall/injury prevention and safety communication has been implemented with residential construction workers (Darragh et al., 2004; Evanoff et al., 2012, 2016; Kaskutas, Buckner-Petty et al., 2016). However, residential companies are quite underrepresented in the construction safety literature precisely because they are hard to reach for research purposes. For example, of 49 studies published between 1997 and 2018 that included either a traditional health and safety training intervention (e.g., toolbox talks) or a new technology application (e.g., serious game, virtual reality), only five (~ 10%) explicitly studied workers at residential construction firms (two of which used data from the same project) (Gao et al., 2019). Additionally, there is little evidence of intervention effectiveness in improving construction workers’ safety behaviors generally (Mullan et al., 2015), let alone in the residential construction sector specifically. Thus, residential construction supervisors and workers are in particular need of prevention research and intervention.

In the construction industry, having project-specific training and safety meetings has been identified as one of the three most commonly adopted safety “innovations” in the construction industry over the past four decades (Esmaeili & Hallowell, 2012). Furthermore, participation of all contractors in safety meetings has been identified as one of 14 differentiating safety practices of construction projects with lower experienced recordable injury rates (Hinze et al., 2013). Holding regular safety meetings may be so integral to successful outcomes in the construction because they contribute to fostering a positive safety climate (i.e., supervisors’ exhibited prioritization of safety affects their workers’ attitudes about safety behavior) (Zohar, 1980; Zohar & Tenne-Gazit, 2008). Additionally, engaging workers during these safety meetings with structured guides may further improve the quantity and quality of leader-member exchanges (i.e., social exchange theory), further increasing safety behaviors (Hofmann & Morgeson, 1999). As such, it is important to both implement safety meetings as a regular construction practice and to facilitate participation within those meetings (e.g., having as many employees attend and promoting discussion and engagement). Still, most interventions in the construction industry focus on reducing injuries or improving safety knowledge as the primary outcome, with mixed results (Gao et al., 2019; Li et al., 2020; Mullan et al., 2015; van der Molen et al., 2018). In contrast, little research has been conducted evaluating intervention effects on construction safety communication outcomes (Aburumman et al., 2019; Kines et al., 2010; Kaskutas, Buckner-Petty et al., 2016), particularly in the residential construction industry. Therefore intervention research is needed to target safety meeting improvements, as these may be a driving force behind injury rates. Moreover, qualitative evidence indicates residential construction workers in particular may benefit from high quality, informal safety training while on-the-job (Hung et al., 2013; Lipscomb et al., 2008).

The Oregon Occupational Safety and Health Administration’s (OR-OSHA) standard for safety meetings requires that construction supervisors/companies hold a safety meeting at least once per month, as well as at the beginning of every project lasting more than one week (OR-OSHA, 2019). Several states such as Connecticut, California, Minnesota, Montana, Nebraska, Nevada, New Hampshire, North Carolina, and Washington have similar standards (OSHA, 2016). Little research, however, has evaluated the normative frequency of safety meetings in residential construction firms, and available results indicate mixed levels of compliance. For example, within a sample of Latino residential construction workers in North Carolina (roofers, framers, and general construction), only small proportions within each trade (8.6-34.5%) reported that workers attended regular safety meetings (Olbina et al., 2011). In most cases, meeting frequency has been measured generally (e.g., “regular meetings”; Gillen et al., 2002) rather than specifically (e.g., one meeting per month), so the compliance of residential construction companies with state or federal safety meeting frequency standards is unknown.

To support effective safety communication in the construction industry, the Oregon Fatality Assessment and Control Evaluation (OR-FACE) program (one of eight state-based FACE programs funded by the National Institute for Occupational Safety and Health) developed and evaluated scripted safety toolbox talks to help supervisors share fatality stories and discuss prevention recommendations in safety meetings (Olson et al., 2016). Talks are approximately 350 words in length and based on construction fatalities that occurred in Oregon between 2003 and 2008. Each two-page talk includes a page with a narrative script and discussion questions to be presented by supervisors to workers, and a second page to be shown to workers with a simplified line drawing of the incident with bullet pointed prevention recommendations in large text (Olson et al., 2016). Construction supervisors’ use of OR-FACE toolbox talks reduced safety meeting preparation and presentation time by 15 min compared to using longer-form technical fatality investigation reports, and much of that time saved was spent in safety discussions with crew members (Olson et al., 2010). Other research suggests that presenting fatality case studies may increase discussion of contributing factors and encourage a dialogue about injury prevention methods (Bajpayee et al., 2004). These findings taken together are particularly important because the time discussing safety with crews likely functions as an improvement in leader-member exchange, which consequently may improve overall safety communication and reduce safety incidents and injuries (Hofmann & Morgeson, 1999).

Documented attempts to improve the effectiveness of toolbox talks include training supervisors on how best to develop talks (e.g., Kaskutas, Jaegers et al., 2016). Training supervisors to craft talks themselves is a worthy cause, however, providing scripted guides and other similar content may substantially reduce supervisors’ preparation effort and increase the chances they engage in this effective workplace injury prevention tactic. Research consistently shows that reducing response effort for new or complex tasks can increase compliance with commitments or standards (Friman et al., 1987). A mobile phone-based distribution of safety toolbox talks may reduce the response effort required of supervisors to obtain safety talk information, and prepare and deliver information to crew members. According to one report, mobile phones are a key medium for communication among construction workers with 97.6% reporting using a smartphone and 81.1% using a mobile phone provided by their company (Welsh, 2015). Thus, mobile phone-based delivery of safety toolbox talks—utilizing both text and email functions—is a promising modality for reducing preparation efforts and increasing compliance with safety meeting frequency standards.

The goals of this study were to evaluate the feasibility of delivering OR-FACE’s toolbox talks to residential construction supervisors via mobile phone. We also aimed to estimate preliminary effects on supervisors’ compliance with safety meeting frequency standards, and measure their perceptions of safety communication quality and crew safety performance. We hypothesized that this method of toolbox talk delivery would increase the proportion of supervisors that met or surpassed OR-OSHA’s safety meeting standard. We also expected that residential construction supervisors’ perceptions of safety communication quality and crew safety behaviors would improve.

Materials and methods

Prior to participant enrollment, all study protocols were reviewed and approved by the Oregon Health & Science University human subjects Institutional Review Board.

Design and Planned Sample

A within-subjects pre/post design was planned to evaluate the feasibility of the intervention and estimate preliminary effects on supervisor safety meeting frequency compliance. Our target sample size after attrition was 60 supervisors based on an a priori power and sample size analysis. This sample size was calculated to possess 0.82 power to detect a pre/post intervention shift of 20% in the proportion of supervisors who met or exceeded OR-OSHA’s safety meeting standard. More sophisticated and larger cluster-based designs and analyses (with organizations as clusters) were not considered feasible given our project’s funding and staffing level and the known challenges of recruiting study participants in the residential construction industry (Hurtado et al., 2020).

Recruitment methods

Study staff recruited participants between August, 2016, and September, 2018 (see Table 1 for the sequence and description of study procedures). Residential construction companies were identified for cold-call contacting using the Angie’s List (now Angi) “Roofing” and "Handymen [now Handyperson]" search term selections for the local metropolitan area. These search terms were selected because of the heightened risk of fatal injuries among roofers and small companies within the construction industry (Dong et al., 2013).

Primary contacts at each partner organization helped the researchers identify construction supervisors within their respective companies that might consider participating in the study. Additionally, partners scheduled appointments for researchers to meet with prospective participating supervisors. At each meeting, the research team gave supervisors an overview of the study, determined eligibility status, obtained informed consent, administered the pre-intervention survey, and guided participants through the text-based enrollment steps. Eligibility criteria included owning a smart phone or tablet and an expectation to be managing a crew for the majority of the 14-week study.

Table 1 Sequence of Study Procedures
Full size table

Participants

A total of 60 supervisors were recruited into the study at baseline. Due to limits in funding and project duration, we were not able to continue recruiting to enroll an excess of supervisors that would assure we maintained our estimated statistical power after attrition. Due to survey question revisions1 after the first 4 participants completed the experience, our analyses focus on 56 participants. At baseline the sample was predominantly non-Hispanic white (n = 44; 78.57%), males (n = 49; 87.50%), with a mean age of 45.70 years (SD = 10.49). Most participants (n = 31; 55.38%) worked in a business with over 50 employees. The remaining participants worked in businesses of various sizes, including less than 5 (n = 2; 3.57%), 5–10 (n = 2; 3.57%), 10–25 (n = 7; 12.50%), and 25–50 (n = 13; 23.21%). Supervisors reported working an average of 20.50 years (SD = 12.87) in the construction industry and 7.52 years (SD = 9.39) at their current company. All companies represented in the study performed residential construction; however, a minority of firms did a mixture of residential and commercial work. Companies provided a range of building construction and specialty trade services, including new construction of both single and multi-family housing, as well as interior and exterior remodeling and maintenance.   

Toolbox Talks

Seven toolbox talks were used in the study that employed the evidence-based structure previously developed by the OR-FACE program and described in the introduction (Olson et al., 2016). Each was based on an in-depth OR-FACE fatality investigation report (see Table 2). The scripted two-page talks were sent to supervisors as links to pdf documents via text message and were available in English and Spanish. Links to YouTube video versions of toolbox talks were also provided in text messages as an alternative method for supervisors to learn about the case or present the talk to crews by playing the video (see Fig. 1). In the videos, a narrator presented each talk in English and inserted pauses in places where discussion was called for. Figure 2 provides an example toolbox talk screen in each of the two formats as they appeared on a mobile device. Additional toolbox talks were also made available to participants from the Center for Construction Research and Training’s (CPWR) online library of toolbox talks.

Table 2 Bi-Weekly Toolbox Talk Topics
Full size table

Once a participant consented and completed a pre-intervention survey, researchers assisted them with text-based self-enrollment in a Trumpia® (Anaheim, CA) “auto-campaign,” a mass, short-message service system primarily used for marketing purposes. To facilitate toolbox talk utilization during the following work week, participants were sent a text message and email every other Sunday at 8:00PM including links to a featured toolbox talk. At the same time, participants were sent an additional resources message including a link to access both OR-FACE’s and CPWR’s online libraries of toolbox talks. Participants were asked to use the featured talk when it was appropriate for their jobsite’s safety concerns and phase of work, or to follow the additional resources link to find a toolbox talk better aligned with their needs. On the interim weeks when participants did not receive a toolbox talk, they were sent a short poll asking if they gave a safety talk, and if so, how many people attended (see Fig. 1).

Fig. 1
figure 1

Featured toolbox talk message and bi-weekly poll

Full size image
Fig. 2
figure 2

Toolbox Talks in Video and PDF Format

Full size image

Measures

The primary evaluation outcome was compliance with OR-OSHA’s administrative rule on safety meeting frequency. This administrative rule dictates that construction supervisors should have safety talks with their employees at least once a month, as well as at the beginning of any shorter projects lasting at least a week. We measured adherence to this standard by asking respondents (both pre- and post-intervention) how many safety talks they had with their employees in the last three months and what proportion (0-100% in increments of 10%) of their short projects (1–3 weeks) included a safety talk prior to beginning the project(s). Supervisors were considered to have met or exceeded the OR-OSHA standard if they reported at least three safety meetings in the last three months (i.e., once per month on average) and had 100% of their short projects begin with a safety meeting.

Secondary outcomes were supervisors’ self-rated safety communication quality and ratings of their workers’ safety performance—including both safety compliance and participation subscales—and were derived from Griffin and Neal’s (2000) Safety Climate Survey. The safety communication quality measure included three items (e.g., “There is open communication about safety issues within the workplace”), and internal consistency was acceptable at baseline (Cronbach’s α = 0.75) and excellent at post-intervention (α = 0.92). The employee safety compliance measure included three items (e.g., “Employees use all the necessary safety equipment to do their job”), and internal consistency was good at baseline (α = 0.88) and post-intervention (α = 0.95). The employee safety participation measure included three items (e.g., “Employees promote the safety program within the organization”), and internal consistency was good at baseline (α = 0.85) and post-intervention (α = 0.93). Our adaptation of the items to reflect supervisors’ ratings of employees is in line with previous similar adaptations of the same scale to focus on supervisor ratings of their employees (e.g., Ali et al., 2020; He et al., 2020).

Feedback/acceptability items were created for the study (see Table 3) and modeled on an augmented version of Kirkpatrick’s levels of training evaluation criteria (Alliger et al., 1997) . These questions assessed whether participants liked the intervention or found it useful (reaction) and what behaviors they changed (transfer). Responses were both quantitatively assessed on a 1 (“Strongly disagree”) to 5 (“Strongly agree”) scale (e.g., “I enjoyed the mobile toolbox talks”), and qualitatively assessed with open-ended questions (e.g., “Please describe what you liked and what you did not like about the digital delivery system and the toolbox talks that you received?”). Quantitative feedback items were included as part of the post-intervention survey. Exit interviews were conducted with a subset of supervisors (n = 6) whom we were given permission to speak with by their respective company. Participants provided verbal responses during the structured interview, and a research assistant took notes summarizing their responses (Table 4).

Data Analysis

For primary outcomes analyses, we used McNemar’s test for paired-sample data to assess supervisors’ compliance with OR-OSHA safety talk frequency standards pre/post intervention. Paired-samples t-tests were conducted to assess changes in safety communication quality and crew members’ safety performance. Cohen’s d effect sizes were computed (0.20 = small, 0.50 = moderate, 0.80 = large; Cohen 1992). Means, standard deviations, and proportions were calculated for acceptability and feedback quantitative items. Notes taken during exit interviews for selected questions are provided in full, and these notes are reviewed and summarized. No formal analysis was applied to the qualitative data.

Results

Thirty-one supervisors (56.36%) completed both pre-and post-intervention surveys. One additional participant completed both surveys, but due to a technical error, they never received the toolbox talks. As such, they were excluded from further analyses. Fisher’s exact test revealed that participants who dropped out were more likely to be supervisors at subcontracting organizations rather than supervisors at the prime company (p < 0.01). They did not significantly differ on any other demographic variable or work characteristic.

Primary outcomes

At baseline, 34.55% (n = 19 out of 55) of supervisors met or exceeded the OR-OSHA standard for safety meeting frequency (one per month and at the beginning of every project lasting at least a week). At follow-up, 48.39% (n = 15 out of 31) of supervisors met or exceeded the OR-OSHA safety meeting standard. Although the within-subjects rate of compliance increased 19.39% (+ n = 6), the McNemar’s test result was non-significant (Odds Ratio [OR] = 4.00, p = 0.11). The increase in overall estimated compliance was possibly due to an increase in short project talk frequency (p = 0.07) relative to baseline, not monthly safety talks (p = 1.00).

Secondary outcomes

Secondary outcomes included changes in safety communication quality and crew members’ safety performance. Perceived communication quality did not significantly change from baseline (M = 4.39, SD = 0.61) to follow-up (M = 4.48, SD = 0.78; d = 0.13). Supervisor-rated employee safety compliance did not significantly change from baseline (M = 3.91, SD = 0.57) to follow up (M = 4.08, SD = 0.88; d = 0.23). However, the effect size indicated a small increase in expected direction. Safety participation was stable between baseline (M = 3.89, SD = 0.76) and follow up (M = 3.90, SD = 0.86; d = 0.01).

Drop out analyses

We explored the role that attrition bias may have played in the study trends observed. We compared compliance with the OR-OSHA safety meeting frequency standard, safety communication quality, and employee safety compliance and participation at baseline between individuals that dropped out and those that remained. Overall, there was a non-significant trend for individuals who dropped out to have reported higher levels in these safety measures at baseline, including a higher likelihood to have been OR-OSHA compliant, OR = 1.75, 95% CI [0.57, 5.37] and have higher safety communication quality (MDiff=0.29, p = 0.10, d = 0.45). No substantial differences between employee safety compliance (MDiff=0.07, p = 0.68, d = 0.12) and employee safety participation (MDiff=0.04, p = 0.85, d = 0.05) were found. In general, these results indicate that those that remained in the study had slightly lower levels of safety at baseline. Given this finding, those who remained in the study had more “room to improve” in two of our primary safety outcomes.

Acceptability of intervention and feedback

Table 3 shows quantitative results for ratings of the acceptability of the intervention (n = 29). For the Reaction criterion, supervisors generally indicated agreement (somewhat or strongly) with each of the items (58.62-65.52%) with mean ratings approaching 4.00 (scale 1–5) for all four questions. Transfer items had levels of agreement that varied between 37.93% and 58.62%, with mean ratings at or just above neutral. Other feedback items had very strong levels of agreement, with most participants reporting that they would like to receive toolbox talks in the future (75.86%) and that they would recommend these talks to other supervisors (72.41%).

Table 3 Ratings and Feedback about Toolbox Talks
Full size table

Table 4 provides a summary of comments from participants’ exit interviews (n = 6) about what they liked and disliked about the intervention, as well as how the system could be improved in the future. Generally, supervisors reported liking the consistent reminders and the content delivery. However, they tended to dislike the timing of reminders (i.e., Sunday) and felt that topics could be more specific to the current job tasks. Similarly, participants thought that more content covering a wider range of topics would be beneficial, because it is highly desirable to have meeting content relate directly to the tasks and tools relevant in the current active phase of construction.

Discussion

The present study evaluated the feasibility of implementing a mobile phone delivery of toolbox talks to residential construction supervisors. We also aimed to estimate preliminary effects on supervisors’ compliance with safety meeting frequency standards, and measure their perceptions of safety communication quality and crew safety performance. We planned the study with a target sample size that would provide sufficient power to detect an estimated effect of a 20% increase in the proportion of supervisors meeting the OR-OSHA standard for safety meeting frequency. Although we did not attain our target sample size of 60 supervisors after attrition, we did see a 19.39% increase in the proportion of supervisors meeting this standard post-intervention. While this increase was not statistically significant, it is an encouraging trend given the injury burden in residential construction and scarcity of safety communication intervention research with the same population. We found a small but non-significant trend in supervisor-rated employee safety compliance, in line with safety meeting compliance findings. However, safety communication quality and supervisor-rated crew safety participation stayed stable. Supervisor feedback on the program was generally positive, with most supervisors reporting that they liked the toolbox talks, found them useful, and that receiving them via text made holding a safety meeting easier.

Table 4 Exit Interview Notes Summarizing Feedback from Supervisors
Full size table

In addition to failing to achieve our target sample size after attrition, thereby experiencing lower than desired statistical power, there are a number of other factors that may explain some of the non-significant findings observed. First, a ceiling effect may have been present for supervisors’ self-reported safety communication quality (i.e., scores for both self-reported communication quality averaged above 4 of a possible 5, leaving little room for improvement). With regard to safety performance, compliance refers to following established safety rules, while participation refers to voluntarily going above and beyond to promote safety in one’s workplace (Griffin & Neal, 2000). Research generally supports differential effects between each type of safety performance (Christian et al., 2009), and previous research has shown lower levels of participation compared to compliance in the construction industry (Guo et al., 2016). As such, the routine occurrence of toolbox talks may influence individuals to comply with safety requirements but may not provide motivation to engage in more discretionary safety behaviors. In addition, qualitative comments suggested that many talks were somewhat irrelevant for their current job tasks, indicating a desire for more tailored talks to deliver to their crews during specific construction stages. Although we provided links to additional toolbox talk topics, accessing and choosing an alternative talk required more effort. This lack of a tailored approach may also explain some of the observed results.

Strengths, Limitations, and future directions

A major strength of the present study was involving supervisors from residential construction firms in an occupational safety intervention study. This population is at particular elevated risk for serious injuries, yet is rarely engaged in safety research and interventions (Gao et al., 2019). It should be noted that we made contact with 238 firms in order to yield our target sample size. We learned that with this difficult-to-reach population, direct outreach, networking with local construction organizations, and assistance from partners had a marked impact on participant recruitment and overall feasibility of intervening with residential construction supervisors. However, an important area of future research is finding more effective ways to engage small residential construction companies in research and prevention efforts. For example, recent research suggests that small residential construction companies that recently had an employee experience a serious injury may be more prepared or ready to engage (Hurtado et al., 2020). In that same study, smaller residential construction companies were efficiently engaged through an intermediary organization—a workers’ compensation insurance company. Ongoing partnerships between researchers and key intermediary organizations, as well as the timing of invitations, may enhance future success and efficiency of engagement.

Other strengths included the innovative design of the study and our focused outcome of safety meeting compliance. Toolbox talks have historically been administered in a wide variety of ways (Kaskutas, Jaegers et al., 2016); however, using cell-phones to deliver scripted toolbox talks has not yet been studied. The addition of mobile-based text delivery in the present study addressed a notable gap in intervention mediums. With a large proportion of construction workers owning a cell phone (Welsh, 2015), toolbox talks would greatly benefit from utilizing this ubiquitous technology. We also focused our intervention on safety meetings, which is a common practice of companies with effective safety programs (Hinze et al., 2013) and likely serve as a social exchange catalyst to improve safety communication and behaviors (Hofmann & Morgesen, 1999). An additional strength was using compliance with the OR-OSHA-based safety meeting administrative rule as the primary outcome measure. Research has commonly used more general criteria to assess safety meeting frequency (Gillen et al., 2002). Using the OR-OSHA administrative rule criterion, we were able to show improvement in explicit adherence to state regulations, instead of simply identifying a change in safety meeting frequency.

A limitation of the present study was a lack of success at collecting data from crewmembers. We attempted to recruit crewmembers indirectly through their supervisors, but so few completed pre- and post-surveys that data were insufficient for analysis. Thus, we do not know crewmembers’ characteristics, perspectives, and responses to the intervention. Additionally, supervisors may be more susceptible to social desirability bias, because they may want to be perceived as having a strong safety climate. Therefore, supervisors may be more likely than line workers to report more safety meetings and higher levels of safety communication quality and employee safety behavior. As such, it is important for future research to incorporate crews’ ratings of safety in order to be able to compare perceptions across these two organizational levels. Crewmembers may have been reluctant to participate due to their supervisor’s involvement in the recruitment, even though names were not included on crewmember surveys. Partnering with unions (e.g., Marín et al., 2015) or workers compensation insurers (e.g., Hurtado et al., 2020) to recruit crewmembers instead of through their direct supervisors may increase willingness to participate. Additionally, taking a community-based participatory research approach whereby researchers collaborate with research participants to design recruitment and intervention strategies, including culture and language-relevant materials, may be fruitful to engage more residential construction workers in projects (Marín & Roelofs, 2018, p. S73). Working with crews directly and through community-based and participatory methods may help to identify specific safety interests for targeted residential construction workers, which may be leveraged to tailor materials for crews to address hazards at their specific worksite. Additional helpful tactics may include procedures to enhance confidence in confidentiality protections, as well as tailored and increased incentives to keep workers engaged throughout the study.

Another limitation was the supervisor retention rate. Although we recruited enough supervisors to meet our sample size requirement at baseline, just slightly more than half remained in the study post-intervention (many of the dropouts were subcontractors). As such, effects of the intervention on safety outcomes should be considered preliminary and in need of replication and extension. Also, the small sample may not be representative and therefore is at-risk of not generalizing to the residential construction industry as a whole.

Previous research has shown that subcontractors’ perceived responsibility for providing safety training is often ambiguous (Hung et al., 2013), so participants who dropped out may not have been explicitly tasked with holding safety meetings during the intervention period. Our comparison analyses between those that dropped out and those that remained indicated that supervisors who dropped out tended to report higher baseline levels of safety compliance, participation, and communication, which may have made them feel a lower need to continue participating in a safety intervention. However, these results also indicate that our intervention was experienced by supervisors with more room for improvement. As such, perhaps assessing safety compliance prior to intervention implementation, and targeting only those for whom such an intervention may be needed, may improve feasibility of implementation and retention. Additionally, we intended to focus enrollment on very small organizations (with 10 employees or fewer) to reflect the common characteristic of the residential construction industry at greatest risk for serious injuries (Sa et al., 2009). However, we ultimately enrolled supervisors from some larger companies (between 10 and 50 + employees). A different incentive structure, possibly based on supervisor weekly presentation of the toolbox talks themselves, may increase participant retention post-intervention. Increasing the number of toolbox talks available and widening the scope of content covered in talks may also increase retention. CPWR provides a number of toolbox talks on a wide range of topics (CPWR, n.d.), which may be more useful to supervisors—particularly subcontractors—on a week-to-week basis. Although we provided this website to supervisors to use in the event a particular week’s talk was irrelevant, a tailored approach (e.g., sending links to individual work-relevant talks from CPWR’s website) would have further reduced the effort needed to select a relevant talk. Future interventions may benefit from communicating with supervisors about their current level of safety compliance and communication quality, the phase of construction, their current work tasks, and trade to inform the specific, weekly toolbox talks provided to them. More sophisticated mobile applications with larger libraries of talks may assist in such efforts to make talks maximally useful, relevant, and timely.

Finally, we also experienced issues throughout the study with the use of the web-based platform used to collect data from supervisors, which impacted the quality and completeness of longitudinal data collection during the intervention. The data validation features of the platform were unexpectedly strict so as to not allow any typographical errors. If typos were present, responses became unmatchable to the participant, because they were transmitted to a separate database where identifying information was not present. Additionally, an unplanned 2-hour time limit on text responses significantly reduced response rates throughout the study. Text-based data collection is a promising avenue for construction research, but improvements in web-based delivery systems are needed to simplify the user experience and enhance the the quality of longitudinal data.

Conclusions

Residential construction workers are a high-risk population for workplace injuries and fatalities yet are a rare focus for intervention work. Increasing the frequency and quality of safety communications can reduce these outcomes though pathways such as increasing salience of safety need, improving safety climate, and ultimately increasing safety performance and compliance with safety standards. Our findings indicate that mobile phone-based delivery of toolbox talks may be an acceptable and feasible way to increase compliance with safety meeting frequency standards at residential construction firms. This approach resulted in an approximately 19% increase in supervisors’ reported compliance with a state-specific safety meeting frequency standard. This criterion and these results are relevant to multiple states that require frequent safety meetings for construction crews. Non-significant trends, as well as logistic lessons learned, suggest directions for future research. These include improving recruitment and retention of supervisors, direct recruitment and deeper involvement of construction crew members, increasing involvement of smaller construction firms, tailoring toolbox talks to the needs of supervisors in specific trades and specific phases of construction, and increasing the reliability of text-based implementation and data capture methods. Most importantly, trends observed in our primary and secondary outcomes encourage future research with the intervention practice of providing toolbox talks to supervisors via mobile phone. Such future research should include replications and variations/extensions of toolbox talk delivery methods in order to improve safety practices and reduce injury rates in the residential construction industry.

FOOTNOTES

1 Critical edits to the survey items were made after the initial 4 supervisors completed participating and before the next 56 were recruited, such that data could not be reliably merged. First, for the primary outcome of OR-OSHA safety meeting compliance, the first participant wave received the question, “How often do you conduct safety meetings/toolbox talks?” with a Likert frequency-based response (e.g., “Daily,” “2–4 times a week,” etc.). The second wave, in contrast received the question, “During the past 3 months, how many safety meetings/safety toolbox talks did you have?” with numeric response options between 0 and 12. Additionally, safety compliance and participation items in the first wave were directed at the supervisors (e.g., “I use all the necessary safety equipment to do my job”), whereas in the second wave, items were directed toward employees (e.g., “Employees use all the necessary safety equipment to do their job”).