Skip to main content
SpringerLink
Log in

Methods for evaluating a mature substance abuse prevention/early intervention program

  • Research-In-Progress
  • Published:
The Journal of Behavioral Health Services & Research Aims and scope Submit manuscript

Abstract

The authors describe methods for work in progress to evaluate four workplace prevention and/or early intervention programs designed to change occupational norms and reduce substance abuse at a major U.S. transportation company. The four programs are an employee assistance program, random drug testing, managed behavioral health care, and a peer-led intervention program. An elaborate mixed-methods evaluation combines data collection and analysis techniques from several traditions. A process-improvement evaluation focuses on the peer-led component to describe its evolution, document the implementation process for those interested in replicating it, and provide information for program improvement. An outcome-assessment evaluation examines impacts of the four programs on job performance measures (e.g., absenteeism, turnover, injury, and disability rates) and includes a cost-offset and employer cost-savings analysis. Issues related to using archival data, combining qualitative and quantitative designs, and working in a corporate environment are discussed.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. Cothen G, Jr.: Stopping substance abuse in its tracks: Alcohol and drugs on the railroad.Alcohol Health and Research World 1985; 9:46–49.

    Google Scholar 

  2. Mannello TA, Seaman FJ:Prevalence, Costs and Handling of Drinking Problems on Seven Railroads. Report of Project REAP DOT-TSC-1375. Chevy Chase, MD: University Research Corporation, 1979.

    Google Scholar 

  3. Ames G: Research strategies for the primary prevention of workplace alcohol problems.Alcohol Health and Research World 1993; 17, 19, 27.

    Google Scholar 

  4. Roman, PH: Some sources of EA knowledge.Employee Assistance, May 1993, pp. 16–17.

  5. Bacharach, SB, Bamberger, PA, WJ Sonnenstuhl: Labor-based peer assistance in the workplace.Industrial Relations 1996; 35(2):261–275.

    Google Scholar 

  6. Eichler, S., Goldberg, CN, Kier, LE, et al.:Case Study of a Peer Prevention Substance Abuse Program for Railroad Industry Personnel. Rockville, MD: Institute for Human Resources, 1988.

    Google Scholar 

  7. Wisman, EW: Peer pressure curbs drug use.Personnel Journal 1990; 69(11):29–30.

    Google Scholar 

  8. Holley, SJ: Second look at substance abuse: Peer prevention and education.Alcohol, Drugs and Driving 1992; 8(3–4):265–276.

    Google Scholar 

  9. Johnson, RA; Management by walking around: An entrepreneurial approach for implementing addiction education through wellness programs. In: Wahlberg, RB (Ed.):Proceedings of the 35th International Congress on Alcoholism and Drug Dependence, Vol. II. Ronkonoma, NY: Federal Aviation Administration.

  10. Feuer, B: Innovative behavioral health care model.EAPA Exchange, October 1992, pp. 28–33.

  11. Bamberger, P, Sonnenstuhl, WJ: Peer referral networks and utilization of a union-based EAP.Journal of Drug Issues 1995; 25(2):291–312.

    Google Scholar 

  12. Nicholson, L: Putting EAP in the driver's seat.Business & Health 1991; 9(3):59–62.

    Google Scholar 

  13. Naegle, MA: Professional resources and impaired nursing practice.Journal of Addictions Nursing 1998; 10(2):91–95.

    Google Scholar 

  14. Scriven M: Beyond formative and summative evaluation. In: McLaughlin GW, Philips DC (Eds.):Evaluation and Education: At Quarter Century. Chicago: University of Chicago Press, 1991, pp. 19–64.

    Google Scholar 

  15. Chen HA: Comprehensive topology for program evaluation.Evaluation Practice 1996; 17:121–130.

    Google Scholar 

  16. Patton MQ:Qualitative Evaluation and Research Methods. Second ed. Newbury Park, CA: Sage, 1990.

    Google Scholar 

  17. Patton MQ:Utilization-Focused Evaluation. Third ed. Thousand Oaks, CA: Sage, 1997.

    Google Scholar 

  18. Scolari:QSR NUD•IST Revision 4. Qualitative Solutions and Research Pty Ltd. London: Sage Publications Software, 1997.

    Google Scholar 

  19. Weitzman ES, Miles MB:Computer Programs for Qualitative Data Analysis. Thousand Oaks, CA: Sage, 1995.

    Google Scholar 

  20. Knodel J: The design and analysis of focus group studies: A practical approach. In: Morgan DP (Ed.):Successful Focus Groups: Advancing the State of the Art. Newbury Park, CA: Sage, 1993.

    Google Scholar 

  21. Miller TR, Pindus N, Douglass J, et al.:Databook on Nonfatal Injury: Incidence, Costs, and Consequences. Washington, DC: Urban Institute, 1995.

    Google Scholar 

  22. Statistical Analysis System. Cary, NC: SAS Institute.

  23. McCleary, R, Hay, RA.Applied Time Series Analysis for the Social Sciences. Beverly Hills, CA: Sage, 1980.

    Google Scholar 

  24. Box G, Jenkins G:Time Series Analysis: Forecasting and Control. Oakland, CA: Holden-Day, 1976.

    Google Scholar 

  25. Box G, Tiao G: Intervention analysis with applications to economic and environmental problems.Journal of the American Statistical Association 1975; 70:70–79.

    Google Scholar 

  26. Holder HD, Blose JO: The reduction of health care costs associated with alcoholism treatment: A 14-year longitudinal study.Journal of Studies on Alcohol 1992; 53:4, 293–302.

    Google Scholar 

  27. Elder GH, Pavalko EK, Clipp EC:Working with Archival Data. Newbury Park, CA: Sage, 1993.

    Google Scholar 

  28. Smith NL: Citizen involvement in evaluation: Empirical studies.Studies in Educational Evaluation 1983; 9:105–117.

    Google Scholar 

  29. Patton MQ: A context and boundaries for a theory-driven approach to validity.Evaluation and Program Planning 1989; 12:375–377.

    Google Scholar 

  30. Marshall C, Rossman GB:Designing Qualitative Research. Newbury Park, CA: Sage, 1989.

    Google Scholar 

  31. Creswell JW:Research Design: Qualitative and Quantitative Approaches. Thousand Oaks, CA: Sage, 1994.

    Google Scholar 

  32. Greene JC, Caracelli VJ (Eds.):Advances in Mixed-Method Evaluation: The Challenges and Benefits of Integrating Diverse Paradigms. San Francisco: Jossey-Bass, 1997.

    Google Scholar 

  33. Rossman GB, Wilson, BL: Numbers and words: Combining quantitative and qualitative methods in a single large-scale evaluation study.Evaluation Review 1985; 9:627–643.

    Google Scholar 

  34. Lancy DF:Qualitative Research in Education: An Introduction to the Major Traditions. New York: Longman, 1993.

    Google Scholar 

  35. Datta L: A pragmatic basis for mixed-method designs. In:New Directions for Evaluation, No 66. San Francisco: Jossey-Bass, 1997.

    Google Scholar 

  36. Weiss CH:Evaluation Research: Methods for Assessing Program Effectiveness. Englewood Cliffs, NJ: Prentice Hall, 1972.

    Google Scholar 

  37. Weiss CH: Nothing as practical as good theory: Exploring theory-based evaluation for comprehensive community initiatives for children and families. In: Connell JP, Kubisch AC, Schorr LB, et al. (Eds.):New Approaches to Evaluating Community Initiatives: Concepts, Methods, and Contexts. Washington, DC: Aspen Institute, 1995, pp. 65–92.

    Google Scholar 

  38. Cronbach LJ, Ambron SR, Dornbusch SM:Toward Reform of Program Evaluation. San Francisco: Jossey-Bass, 1980.

    Google Scholar 

  39. Lipsey MW, Pollard JA: Driving toward theory in program evaluation: More models to choose from.Evaluation and Program Planning 1989; 12:317–328.

    Google Scholar 

  40. Scheirer MA: Program theory and implementation theory: Implications for evaluators.Using Program Theory in Evaluation: New Directions for Program Evaluation 1987; 33:59–76.

    Google Scholar 

  41. Chen H:Theory-Driven Evaluations. Newbury Park, CA: Sage, 1990.

    Google Scholar 

  42. Bickman L (Ed.): Using program theory in evaluation.New Directions for Program Evaluation 1987; 33.

  43. Bickman L (Ed.): Advances in program theory.New Directions for Program Evaluation 1990; 47.

  44. Mark M: From program theory to tests of program theory.Advances in Program Theory: New Directions for Program Evaluation 1990; 47:37–51.

    Google Scholar 

  45. Smith NL: Using path analysis to develop and evaluate program theory.Advances in Program Theory: New Directions for Program Evaluation 1990; 47:53–57.

    Google Scholar 

  46. Weiss CH: How can theory-based evaluation make greater headway?Evaluation Review 1997; 21:501–524.

    Google Scholar 

  47. Treno AJ, Holder HD: Evaluating efforts to reduce community-level problems through structural rather than individual change.Evaluation Review 1997; 21:133–139.

    Google Scholar 

  48. Gruenewald PJ: Analysis approaches to community evaluation.Evaluation Review 1997; 21:209–230.

    Google Scholar 

  49. Holder, HD: Prevention aimed at the environment. In: McCrady BS, Epstein EE (Eds.):Addictions: A Comprehensive Guidebook for Practitioners. Oxford, UK: Oxford University Press, in press.

  50. Schwarz DF, Grisso JA, Miles C, et al.: An injury prevention program in an urban African-American community.American Journal of Public Health 1993; 83:675–680.

    Google Scholar 

  51. Steele PD: A history of job-based alcoholism programs: 1955–1972.Journal of Drug Issues 1989; 19:511–532.

    Google Scholar 

  52. De Leon, G.: The therapeutic community: Toward a general model and theory. In: Tims, FM, De Leon, G, Jainchill, N (Eds):Therapeutic Community: Advances in Research and Application. NIDA Research Monograph. Rockville, MD: National Institute of Drug Abuse, 1994.

    Google Scholar 

  53. Harwood HJ, Fountain D, Livermore G:The Economic Cost of Alcohol and Drug Abuse in the United States, 1992. Rockville, MD: National Institute on Drug Abuse and National Institute on Alcohol Abuse and Alcoholism, 1998.

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Safety and Health Policy Program, Pacific Institute for Research and Evaluation, 8201 Corporate Drive, Ste. 220, 20785, Landover, MD

    Les R. Becker Ph.D. (NREMT-P, Deputy Director), Margruetta Hall Ph.D. (senior program director), Deborah A. Fisher Ph.D. (associate research scientist) & Ted R. Miller Ph.D. (principal research scientist)

Authors
  1. Les R. Becker Ph.D.
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Margruetta Hall Ph.D.
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Deborah A. Fisher Ph.D.
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Ted R. Miller Ph.D.
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Les R. Becker Ph.D..

Rights and permissions

Reprints and permissions

About this article

Cite this article

Becker, L.R., Hall, M., Fisher, D.A. et al. Methods for evaluating a mature substance abuse prevention/early intervention program. The Journal of Behavioral Health Services & Research 27, 166–177 (2000). https://doi.org/10.1007/BF02287311

Download citation

  • Issue Date:

  • DOI: https://doi.org/10.1007/BF02287311

Keywords

Search

Navigation