Purpose Despite the importance and cost of workers’ compensation (WC)-based vocational rehabilitation (VR) programs, outcome evaluations are rare, in part due to the scarcity of suitable comparison groups. The aims of this study were to assess (1) the adequacy of a commonly recommended internal comparison group, i.e., workers who were eligible for but did not receive services, and (2) return-to-work (RTW) expectations as a potential source of bias. Methods In this prospective cohort study, we used WC claims data and worker-reported RTW expectations to compare workers who received vocational retraining services to eligible workers who did not receive such services. Workers were surveyed after retraining eligibility determination, prior to the initiation of retraining activities. VR progress and RTW wage outcomes were followed for 3 years. The magnitude of confounding contributed by RTW expectations and other covariates was quantified. Results Workers who were somewhat or very certain they would RTW had significantly better outcomes. RTW expectations played a strong confounding role, reducing the retraining plan effect estimate by about 23 %, while education and physical capacity each changed the effect estimate by <5 %. Conclusions RTW expectations predicted long-term RTW outcomes and can play a strong confounding role if unmeasured. We found that the internal comparison group approach, commonly recommended for VR program evaluation, is inappropriate for WC-based VR evaluations. Ultimately, there is no simple solution to the challenge of identifying a comparison group; however, measurement of RTW expectations, an easily-measured multi-dimensional construct, may be a useful addition to the VR evaluation toolbox.
This is a preview of subscription content, log in to check access.
Buy single article
Instant access to the full article PDF.
Price includes VAT for USA
Subscribe to journal
Immediate online access to all issues from 2019. Subscription will auto renew annually.
This is the net price. Taxes to be calculated in checkout.
Data for this methodological study were drawn from the Abt Associates’ evaluation of the Aid to Families with Dependent Children (AFDC) Homemaker-Home Health Aide Demonstrations.
MacEachen E, Kosny A, Ferrier S, Chambers L. The, “toxic dose” of system problems: why some injured workers don’t return to work as expected. J Occup Rehabil. 2010;20(3):349–66.
Washington State Department of Labor and Industries. Vocational Improvement project report to the legislature: as required by RCW 51.32.099; 2012 December, 2012.
Barth P, Grob H, Harder H, Hunt A, Silverstein M. Washington pension system review, Upjohn Institute technical report No. 08-025. Prepared for State of Washington, Department of Labor & Industries, Contract No. K1018. 2008 [cited March 10, 2014]. http://www.upjohninst.org/publications/tr/tr08-025.pdf.
MacEachen E, Kosny A, Ferrier S, Lippel K, Neilson C, Franche RL, et al. The ‘ability’ paradigm in vocational rehabilitation: challenges in an Ontario injured worker retraining program. J Occup Rehabil. 2012;22(1):105–17.
MacEachen E, Kosny A, Ferrier S, Lippel K, Neilson C, Franche RL, et al. The ideal of consumer choice: challenges with implementation in an Ontario injured worker vocational retraining program. Disabil Rehabil. Online first: 25 April 2013; doi:10.3109/09638288.2013.771704.
Sears JM, Wickizer TM. Evaluation of the vocational rehabilitation pilot program. Report to the Washington State Legislature as required by ESSB 5920 (Chapter 72, Laws of 2007). 2012 [cited March 10, 2014]. http://www.lni.wa.gov/ClaimsIns/Files/Vocational/VocPilotProgEval.pdf.
KPMG LLP. WSIB Labour Market Re-entry (LMR) Program value for money audit report. 2009 [cited March 10, 2014]; http://www.wsib.on.ca/files/Content/VFMAVFMALMR2009/LMRvfma.pdf.
McPherson K. Evaluation of vocational rehabilitation under the IPRC Act 2001. 2007 [cited March 10, 2014]. http://hazelarmstronglaw.co.nz/beta/wp-content/uploads/2012/07/Evaluation-of-Vocational-Rehabilitation-under-the-IPRC-Act-2001-Kathryn-McPherson.pdf.
Bell SH, Orr LL, Blomquist JD, Cain GG. Program applicants as a comparison group in evaluating training programs: theory and a test. Kalamazoo, MI: W.E. Upjohn Institute for Employment Research; 1995.
Dean D, Dolan R, Schmidt R, Wehman P, Kregel J, Revell G. A paradigm for evaluation of the federal-state vocational rehabilitation program (Chap. 1). In: Kregel J, Dean D, Wehman P, editors. Achievements and challenges in employment services for people with disabilities: the longitudinal impact of workplace supports. Richmond: Virginia Commonwealth University—Rehabilitation Research and Training Center; 2002.
Dean DH. Comparing employment-related outcomes of the vocational rehabilitation program using longitudinal earnings. Am Rehabil. 1991;17(1):4–9.
Dean DH, Dolan RC. Assessing the role of vocational rehabilitation in disability policy. J Policy Anal Manage. 1991;10(4):568–87.
Dean DH, Dolan RC. Fixed-effects estimation of earnings impacts for the vocational rehabilitation program. J Hum Resour. 1991;26(2):380–91.
Dean DH, Dolan RC, Schmidt RM. Evaluating the vocational rehabilitation program using longitudinal data. Evidence for a quasiexperimental research design. Eval Rev. 1999;23(2):162–89.
Nowak L. A cost effectiveness evaluation of the federal/state vocational rehabilitation program–using a comparison group. Am Econ. 1983;27(1):23–9.
Iles RA, Taylor NF, Davidson M, O’Halloran PD. Patient recovery expectations in non-chronic non-specific low back pain: a qualitative investigation. J Rehabil Med. 2012;44(9):781–7.
Stewart AM, Polak E, Young R, Schultz IZ. Injured workers’ construction of expectations of return to work with sub-acute back pain: the role of perceived uncertainty. J Occup Rehabil. 2012;22(1):1–14.
Kapoor S, Shaw WS, Pransky G, Patterson W. Initial patient and clinician expectations of return to work after acute onset of work-related low back pain. J Occup Environ Med. 2006;48(11):1173–80.
Lovvik C, Overland S, Hysing M, Broadbent E, Reme SE. Association between illness perceptions and return-to-work expectations in workers with common mental health symptoms. J Occup Rehabil. 2013.
Bandura A. Self-efficacy: toward a unifying theory of behavioral change. Psychol Rev. 1977;84(2):191–215.
Cole DC, Mondloch MV, Hogg-Johnson S. Listening to injured workers: how recovery expectations predict outcomes—a prospective study. CMAJ. 2002;166(6):749–54.
Gross DP, Battie MC. Work-related recovery expectations and the prognosis of chronic low back pain within a workers’ compensation setting. J Occup Environ Med. 2005;47(4):428–33.
Gross DP, Battie MC. Recovery expectations predict recovery in workers with back pain but not other musculoskeletal conditions. J Spinal Disord Tech. 2010;23(7):451–6.
Heijbel B, Josephson M, Jensen I, Stark S, Vingard E. Return to work expectation predicts work in chronic musculoskeletal and behavioral health disorders: prospective study with clinical implications. J Occup Rehabil. 2006;16(2):173–84.
Hogg-Johnson S, Cole DC. Early prognostic factors for duration on temporary total benefits in the first year among workers with compensated occupational soft tissue injuries. Occup Environ Med. 2003;60(4):244–53.
Laisne F, Lecomte C, Corbiere M. Biopsychosocial determinants of work outcomes of workers with occupational injuries receiving compensation: a prospective study. Work. 2013;44(2):117–32.
Turner JA, Franklin G, Fulton-Kehoe D, Sheppard L, Wickizer TM, Wu R, et al. Worker recovery expectations and fear-avoidance predict work disability in a population-based workers’ compensation back pain sample. Spine (Phila Pa 1976). 2006;31(6):682–9.
Fadyl J, McPherson K. Return to work after injury: a review of evidence regarding expectations and injury perceptions, and their influence on outcome. J Occup Rehabil. 2008;18(4):362–74.
Gray H, Adefolarin AT, Howe TE. A systematic review of instruments for the assessment of work-related psychosocial factors (Blue Flags) in individuals with non-specific low back pain. Man Ther. 2011;16(6):531–43.
Hallegraeff JM, Krijnen WP, van der Schans CP, de Greef MH. Expectations about recovery from acute non-specific low back pain predict absence from usual work due to chronic low back pain: a systematic review. J Physiother. 2012;58(3):165–72.
Iles RA, Davidson M, Taylor NF. Psychosocial predictors of failure to return to work in non-chronic non-specific low back pain: a systematic review. Occup Environ Med. 2008;65(8):507–17.
Iles RA, Davidson M, Taylor NF, O’Halloran P. Systematic review of the ability of recovery expectations to predict outcomes in non-chronic non-specific low back pain. J Occup Rehabil. 2009;19(1):25–40.
State of Washington. RCW Title 51: Chapter 51.12. Employments and occupations covered. [cited March 10, 2014]. http://apps.leg.wa.gov/rcw/default.aspx?Cite=51.12.
Tanabe RP. Workers’ compensation laws as of January 2012. WC-12-18. Cambridge, Mass.: Workers Compensation Research Institute; 2012 March 2012.
Franklin GM, Fulton-Kehoe D. Outcomes research in Washington state workers’ compensation. Am J Ind Med. 1996;29(6):642–8.
Franklin GM, Wickizer TM, Fulton-Kehoe D, Turner JA. Policy-relevant research: when does it matter? NeuroRx. 2004;1(3):356–62.
Washington State Department of Labor and Industries. Workers’ compensation benefits: a guide for injured workers. Publication F242-104-000 [03-2012]. 2012 [cited March 10, 2014]. http://www.lni.wa.gov/IPUB/242-104-000.pdf.
The American Association for Public Opinion Research. Standard definitions: final dispositions of case codes and outcome rates for surveys. 5th ed. Lenexa, KS: AAPOR; 2008.
Sears JM, Wickizer TM, Schulman BA. Injured workers’ assessment of vocational rehabilitation services before and after retraining. J Occup Rehabil. Online first: 25 Sept 2013. doi:10.1007/s10926-013-9479-0.
Washington State Employment Security Department. Quarterly census of employment and wages, fourth quarter 2011 preliminary. LM-12-0263.; July 2012.
Sears JM, Wickizer TM, Schulman BA. Improving vocational rehabilitation services for injured workers in Washington State. Eval Program Plann. 2014;44:26–35.
Wang Z. Two postestimation commands for assessing confounding effects in epidemiological studies. Stata J. 2007;7(2):183–96.
Simon JG, De Boer JB, Joung IM, Bosma H, Mackenbach JP. How is your health in general? A qualitative study on self-assessed health. Eur J Public Health. 2005;15(2):200–8.
This work was funded by a contract (#K1009) from the Washington State Department of Labor and Industries (L&I). The content of this publication does not necessarily reflect the view or policies of L&I. The authors declare that they have no conflicts of interest. The authors wish to acknowledge Wayne Shatto, Rich Wilson, Kendra Hanson, Sidse Nielsen and many other L&I staff members for their crucial assistance with developing the survey sample, data delivery, and explanations of the very complex data generating processes. Most importantly, we gratefully acknowledge the injured workers who elected to participate in our survey.
Conflict of interest
The authors declare that they have no conflicts of interest.
About this article
Cite this article
Sears, J.M., Rolle, L.R., Schulman, B.A. et al. Vocational Rehabilitation Program Evaluation: Comparison Group Challenges and the Role of Unmeasured Return-to-Work Expectations. J Occup Rehabil 24, 777–789 (2014). https://doi.org/10.1007/s10926-014-9509-6
- Vocational rehabilitation
- Workers’ compensation
- Return to work
- Control groups
- Confounding factors
- Program evaluation