Participants and procedures
This study was conducted in a Dutch health service of 4500 employees. A total of 30 teams of health-care workers (mainly nurses) were recruited for participation and split into 15 teams (with 252 nurses) for the experimental condition where Stress-Prevention@Work was offered and matched with another 15 teams (with 221 nurses) as waitlisted controls. An independent researcher matched teams on working conditions and size and allocated the teams to the intervention or control group. This researcher did not have information about the perceived stress levels in the teams. Since the intervention focused on the organisation and not on individual workers, individual randomisation was not feasible. The main outcome of the study was productivity losses, assessed using the Trimbos and iMTA Cost questionnaire in Psychiatry (TiC-P), which is a self-completed questionnaire identifying days absent from work (absenteeism) and days working while sick (presenteeism). As we were specifically interested in productivity losses, only these items of the TiC-P were used (i.e. number of working hours per week, number of working days per week, number of days absent past months, number of days working while not fully fit). As the paper compromised an investment appraisal, we looked at both costs and (financial) benefits, which are described below. Measurements were conducted in May 2016 (baseline, t0) and then at 6 and 12 months post-baseline (t1 and t2). The control group was given access to the intervention after 12 months. The study design is therefore best described as a matched cohort study in two parallel groups with clustering at the team level. More details are provided in Hoek et al. (2018).
Implementation strategy
Stress-Prevention@Work targeted all employees within the intervention groups and contained a search engine for interventions for stress prevention. The portal, and concomitant training in its usage, takes a stepwise approach directed at (1) raising awareness that proper management of work-related stress is important; (2) screening for determinants of work stress; (3) setting intervention goals and selecting appropriate preventive interventions; (4) implementing the selected stress management intervention in the workplace [e.g., construct a personal action plan to implement the intervention(s)]; and (5) evaluating the strategy’s impact on work-related stress. Types of interventions ranged from guidelines for communicating about work stress to more extensive, tailor-made interventions, possibly involving intermediaries/consultants. The interventions were either at organisational or employee level. For example, organisational interventions included a guideline to start a dialogue between employees and their manager(s) about the presence of work-related stress within the organisation or team. Individual interventions could consist of online self-help modules to reduce work-related psychosocial risk factors (Hoek et al. 2018).
Computation of costs
This investment appraisal looked at the costs associated with the Stress-Prevention@Work implementation strategy and compared these costs with the economic benefits derived from greater work productivity through lesser absenteeism and presenteeism. On the costs side, the following items are included:
- 1.
The per-team costs of using Stress-Prevention@Work is €100. This per-team tariff helps to pay for hosting, maintaining and periodically upgrading the Stress-Prevention@Work portal. In an average team of 16 health-care workers this amounts to €100/16 = €6.25 per employee.
- 2.
In each team, one employee receives training in the Stress-Prevention@Work strategy at €250. In an average team of 16 nurses, this translates in an additional per-nurse cost of €250/16 = €15.63.
- 3.
Each employee spends 30 min (at most) during office hours on working through all the steps of the Stress-Prevention@Work implementation strategy. This is equivalent to €17.40 per employee, when valuing 1 h of work of an employee at €34.75 (for the year 2014) in line with the Dutch guideline for costing in health-economic evaluation (Zorginstituut Nederland 2015).
- 4.
However, there is one employee in each team who operates the portal and this takes 3 h at €34.75 per hour, which adds (3 × €34.75)/16 = €6.50 per employee.
- 5.
Finally, the Human Resource Management department at the health service invests 150 h of work annually to promote the Stress-Prevention@Work strategy across all 4500 employees. This represents costs of (150 h × €34.75)/4500 = €1.60 per employee.
The sum total of costs of Stress-Prevention@Work is therefore 6.25 + 15.63 + 17.40 + 6.50 + 1.60 = €47.38 per employee per annum. This is rounded to €50 per employee. It is assumed that these costs are incurred by the employer. Moreover, it is assumed that more extensive interventions take place outside office hours.
Computation of benefits
The economic benefits from Stress-Prevention@Work are generated by greater work productivity via lesser absenteeism and lesser presenteeism. As before, we value 1 h of work at €34.75. Thus, 1 day of sick leave costs 8 h × €34.75 = €278. The number of days of sick leave was estimated using the TiC-P at each of the three measurements points (t0, t1 and t2) and multiplied by €278.
In a similar vein, the costs of presenteeism were estimated as the cost of workdays lost due to presenteeism. The TiC-P helps to assess the number of lost workdays owing to presenteeism by first asking how many days the employee went to work not feeling well and then by asking to rate the efficiency at work while not feeling well. Thus, 2 days worked at half one’s usual efficiency translates into one workday lost, which is again valued as €278.
Here, it should be noted that there are three measurements at t0, t1 and t2, each 6 months apart. At each time point, the costs of sick leave and work cutback days (C0, C1, C2) were assessed over the last 4 weeks. The area under the curve (AUC) method was used to compute the cumulative costs (CC) over the full follow-up period of 12 months, while interpolating the costs in the months between the measurement points. This was done with the following equation: \({\text{CC}}\, = \, 2C_{0} \, + \, 3\left( {C_{0} \, + \,C_{ 1} } \right)/ 2\, + \, 2C_{ 1} \, + \, 3\left( {C_{ 1} \, + \,C_{ 2} } \right)/ 2\, + \, 2C_{ 2}\).
Finally, the difference between the experimental and control condition in the cumulative costs represents the economic benefits when one condition has lower cumulative costs than the other condition. It was hypothesised that the experimental condition would be associated with lower cumulative costs and hence with monetary benefits.
Statistical analysis
The analyses were conducted in several steps.
First, we checked for baseline imbalances using descriptive statistics that are presented in Table 1.
Table 1 Sample characteristics at baseline Second, we adhered to the intention-to-treat principle as per the CONSORT and CHEERS guidelines (Schulz et al. 2010; Husereau et al. 2013). To this end, missing observations at t1 and t2 were imputed for which we used regression imputation with predictors of the outcome for precision and with predictors of missingness to account for possible selective dropout. This way, a regression model was estimated to predict the observed values of a variable based on other variables, which was then used to impute values in cases where the value of that variable was missing.
Third, using the imputed costs of absenteeism and presenteeism at t0, t1 and t2, we computed the total annual cumulative costs using the area under the curve methodology as outlined before. Next, we added the intervention costs of €50 per employee for those who were the recipients of the Stress-Prevention@Work intervention.
Fourth, we evaluated the between-group difference in the total cumulative cost (as the outcome of interest) in a regression model with the group variable (0 = control; 1 = experimental) as predictor. The b-coefficient belonging to the group variable captures the net benefits, NB, when the cumulative costs of the intervention group are lower than those of the waitlisted control group. The corresponding t test belonging to the b-coefficient provides a test if the net benefit is statistically significant. It is worth mentioning that the employees are ‘clustered’ in teams and we carried out a design-based regression analysis that took this clustering into account using robust standard errors that were obtained under the first-order Taylor-series linearisation method. In addition, our regression model took into account the non-normality of costs and therefore we employed a non-parametric bootstrapped regression model which was bootstrapped 2500 times.
Finally, with the NBs in hand, we completed the investment appraisal by computing the cost–benefit ratio as C/NB, where C is the intervention cost of €50 per employee. We also computed the return-on-investment, ROI, as ROI = NB/C. These metrics, especially NB and ROI, are key to an investment appraisal and serve to decide if investing in the Stress-Prevention@Work implementation strategy is worthwhile as seen from the employer’s business perspective. All analyses were carried out in Stata 14.1 and Microsoft Excel 2010.
Sensitivity analyses
While our main analysis is based on robust techniques, we also carried out several sensitivity analyses.
First, the study was subject to substantial dropout and this may have affected the cost estimates at t1 and t2. In the main analysis, the intention-to-treat analysis was performed using regression imputation (RI) of missing observations at follow-up. In the sensitivity analysis, the intention-to-treat analysis was repeated using linear mixed modelling (LMM), to see if the costs followed the same trajectory over time as estimated under the main analysis.
Second, the main analysis was conducted without adjusting for baseline costs of presenteeism and absenteeism, but there was a small (and statistically insignificant) difference between both conditions. Hence, the main analysis was repeated including costs of presenteeism and absenteeism at baseline as a covariate to adjust for this slight baseline imbalance.
Third, given the relatively high dropout, a sensitivity analysis was conducted in which missing observations were imputed using multiple imputations (5 times) by chained equations with predictive mean matching in which ‘‘real’’ observed values from similar cases are imputed instead of imputing regression estimates. This technique is often used to account for non-normality of data which is often the case for costs.
Last, cumulative costs in the base case were calculated on a relatively conservative basis, where the costs at baseline were assumed to be stable for the first 8 weeks (in favour of waitlisted control condition). Hence, in a sensitivity analysis, cumulative costs were calculated using the following formula: \({\text{CC}}\, = \, 5\left( {C_{0} \, + \,C_{ 1} } \right)/ 2\, + \,C_{ 1} \, + \, 5\left( {C_{ 1} \, + \,C_{ 2} } \right)/ 2\, + \,C_{ 2} .\).