Administration and Policy in Mental Health and Mental Health Services Research

, Volume 39, Issue 3, pp 180–186

Implementation of an Evidence-Based Depression Treatment Into Social Service Settings: The Relative Importance of Acceptability and Contextual Factors

Authors

    • College of NursingUniversity of Iowa
  • Jennifer E. McCabe
    • Department of PsychologyUniversity of Iowa
  • Sara M. Stasik
    • Department of PsychologyUniversity of Iowa
  • Michael W. O’Hara
    • Department of PsychologyUniversity of Iowa
  • Stephan Arndt
    • Consortium for Substance Abuse, College of Medicine: Departments of Psychiatry, College of Public Health: Department of BiostatisticsUniversity of Iowa
Original Paper

DOI: 10.1007/s10488-011-0345-0

Cite this article as:
Segre, L.S., McCabe, J.E., Stasik, S.M. et al. Adm Policy Ment Health (2012) 39: 180. doi:10.1007/s10488-011-0345-0

Abstract

Listening Visits (LV), an empirically supported depression treatment delivered by non mental health specialists, were implemented into two distinctly structured programs. The relative importance of providers’ views and organizational context on implementation were examined. Thirty-seven home visitors completed pre- and post-LV training surveys assessing their views toward implementing LV. Implementation rates markedly differed in the two organizations (73.9% vs. 35.7%). Logistic regression results showed that when predicting the implementation rate, the impact of the organizational setting outweighed home visitors’ personal views. These results underscore the importance of organizational context in the implementation of empirically supported treatments.

Keywords

Depression treatmentImplementationViewsOrganizational context

The “creation of a new order” has long been a goal in the arena of mental health services. As early as 1963, Senator Hubert Humphrey urged psychologists to build bridges from research to community programs (Schoenwald et al. 2008). This call to action has been more recently echoed in the NIMH strategic aim to enhance the public health impact of its research (Insel 2008). However, the successful implementation of empirically supported treatments into community-based social service settings is not determined solely by their availability (Glaser 1973; Gotham 2004; Kelly et al. 2000; Kerner et al. 2005). An extensive services research literature describes the multitude of factors influencing implementation, including characteristics of the intervention, adopters/providers, the organization, as well the surrounding macro systems (Durlak and DuPre 2008; Glisson 2002; Mendel et al. 2008; Zazzali et al. 2008). As the relatively young field of implementation science advances, researchers have more recently applied this knowledge to facilitate the successful implementation of mental health treatment for children (Glisson and Schoenwald 2005; Schoenwald 2008; Schoenwald et al. 2008) and families (Zazzali et al. 2008).

Despite these advances, bridging the gap between university-based research and implementation of treatments in community-based settings remains an ongoing challenge in the area of perinatal mental health. Clinically significant depression is prevalent in women during the childbearing years, affecting approximately 18% of pregnant women and 19% of women in the first three postpartum months (Gavin et al. 2005). There is an increased prevalence of postpartum depression among women with few financial resource: 37.7% of those with annual incomes of less than $20,000 reported clinically significant levels of depressive symptoms compared to only 7.5% of those with annual incomes above $70,000 (Segre et al. 2007). Depression in pregnant and postpartum women is associated with pervasive negative effects. In infancy these negative effects include preterm birth, low birth weight, impaired mother-infant interactions, negative parenting behaviors, and disrupted mother-infant attachment (Field 2010; Grote et al. 2010; Martins and Gaffan 2000). The negative effects extend to childhood and include compromised child development (Beck 1998; Murray et al. 1999), as well as an increased risk for behavior problems, including externalizing disorders (Beck 1999; S. H. Goodman and Gotlib 1999).

Empirically supported depression treatments for pregnant and postpartum women are available (Chabrol et al. 2002; O’Hara et al. 2000), including those focused on increasing the accessibility and acceptability of depression treatment for at-risk, low-income and ethnic-minority women (Levy and O’Hara 2010). Nevertheless, the majority of depressed mothers do not receive care (Watt et al. 2002), particularly ethnic-minority and low-income women (Song et al. 2004). Treatment barriers for these women are numerous and include the stigma associated with depression, mistrust of mental health providers, a lack of access to treatment services, cost, and logistical barriers (e.g., lack of time, difficulty reaching the office, or inconvenient hours) (Dennis and Chung-Lee 2006; J. Goodman 2009; McCarthy and McMahon 2008; Nadeem et al. 2007; Sword et al. 2008). Many of these barriers are inextricably linked to reliance on mental health professionals to deliver treatment. To circumvent these treatment barriers, the healthcare system in the UK developed Listening Visits (LV), an effective depression intervention that can be delivered by public health nurses (Cooper et al. 2003; Holden et al. 1989; Morrell et al. 2009; Wickberg and Hwang 1996). Because LV can be effectively delivered by individuals who are not mental health specialists, this intervention also has significant potential to overcome some depression treatment barriers in the US among pregnant and postpartum women. Indeed, in the US, home-visiting programs for low-income perinatal women are already well-established in every state, e.g., Nurse Family Partnership, Healthy Families and Healthy Start (Leis et al. 2009). These home-visiting programs thus provide an accessible venue to deliver mental health treatments. Listening Visits delivered by US home visitors thus hold significant promise to “create a new order” and bridge the gap in mental health treatment.

Yet “good ideas are not sufficient for change to occur” (Bickman 2008, p. 229). Additionally, the availability of an empirically supported treatment—like LV—in one country does not guarantee its implementation in another. Nevertheless, in 2005, LV were introduced into two organizationally distinct Visiting Nurse Services (VNS) home-visiting programs for low-income pregnant and postpartum women as a feasible way to address depression treatment gaps. Two parameters of this treatment implementation provided a unique opportunity to examine factors affecting the home visitors’ decision to implement this intervention. First, because LV were not yet validated in a US setting, implementation was strongly supported by VNS executive staff but could not be mandated. Thus the decision to implement was under the control of individual home visitors. Second, because the intervention (LV) was the same in both settings, there was no variation in the difficulty of the intervention—one factor associated with implementation success (Durlak and DuPre 2008). With intervention difficulty held constant, we were therefore able to assess the relative importance of two distinct forces: the home visitors’ views about the treatment vs. the organizational support for putting the treatment into practice.

The current study of the implementation of a mental health treatment into two organizations extends implementation research in two ways. First, to our knowledge, this study is the first to explicitly assess factors influencing the implementation of depression treatments for pregnant and postpartum women. Second, this study is unique in its assessment of factors affecting the implementation of mental health treatment by non mental health specialists. Because of the numerous barriers preventing low-income women from receiving treatment for depression, and the fact that many of these barriers are intertwined with the use of mental health professionals, treatment by nonspecialists will be critical to the creation of a new order to effectively reach this population of women.

Methods

Setting and Rationale for Implementing LV

Visiting Nurse Services (VNS) is a Midwestern, umbrella organization that provides medical and social services through approximately 30 different social service programs. Two of these programs, Healthy Start and Maternal Child Health, provide home-visiting services to low-income women who are either pregnant or have young children. Although both of these VNS programs provide similar home-visiting services (i.e., family support, parent education, and basic health care such as infant immunization), they are distinct programs with unique organizational characteristics, including different funding sources, directors, and home visitors. Specifically, Healthy Start is a federally funded program through which home-visiting services are provided by case managers. In contrast, Maternal Child Health is funded through the Title V Maternal and Child Health Services Block Grant, with matching funds from the State of Iowa, through which home-visiting services are provided by nurses.

Through VNS, both of these home-visiting programs established similar maternal depression-screening protocols, which determined that the majority of women identified as depressed were not receiving treatment. As an evidenced-based means of providing accessible and effective depression treatment, VNS executive staff recommended that home visitors in both programs use LV. In 2005, a British trainer provided a two-day LV workshop in the US to home visitors in both Healthy Start and Maternal Child Health.

Participants

A total of 37 home visitors participated in this study, 23 from Healthy Start and 14 from Maternal Child Health. The home visitors in both programs were predominately female, non-Hispanic whites, in their early thirties, and with bachelor’s level of education (Table 1). Comparisons between the two groups revealed no significant demographic differences. Furthermore, the work experience was similar in the two groups of home visitors, with an average of 46 and 40 months of experience in Healthy Start and Maternal Child Health, respectively. Compared to those in Healthy Start, however, home visitors in Maternal Child Health reported a significantly larger caseload and less previous experience with providing counseling (Table 1).
Table 1

Demographic comparison of healthy start and maternal child health home visitors

Variable

Healthy start

Maternal child health

Comparison

(n = 23)

(n = 14)

Two-sample t-test

M (sd)

M (sd)

Age

32.44 (7.04)

34.80 (10.61)

P = 0.42 (t = 0.81, df = 35)

Home-visiting

Experience (months)

46.05 (44.96)a

39.77 (68.51)b

P = 0.75 (t = −0.32, df = 32)

Counseling traininge

0.42 (0.68)

−0.70 (0.45)

P < 0.001 (t = −5.45, df = 35)

   

Mann–Whitney testd

Caseload size

16.00 (4.55)

21.32 (9.15)c

P = 0.01 (z = 2.50)

 

N (%)

N (%)

Fisher’s exact test

Race

  

P = 0.65

 White

20 (86.96)

10 (71.43)b

 

 Black

3 (13.04)

3 (21.43)

 

Ethnicity

  

P = 1.00

 Hispanic

2 (8.70)

1 (7.14)

 

 Not Hispanic

21 (91.30)

13 (92.86)

 

Gender

   

 Female

22 (95.65)

14 (100)

P = 1.00

 Male

1 (4.35)

0 (0)

 

Education

 Post high-school

0 (0.00)

1 (7.14)

P = 0.29

 Bachelor’s degree

21 (91.30)

13 (92.90)

 

 Master’s degree

2 (8.70)

0 (0.00)

 

aTwo participants were missing data on this item

bOne participant was missing data on this item

cThree participants were missing data on this item

dMann–Whitney test was used due to inequality of variances on this variable

eThe training variable represents a standardized composite of participants’ experience and training in counseling

Measures

Pre-LV Training Survey

This 30-item survey, developed specifically for this study, collected information on the home visitors’ demographics (6 items), occupation, mental health-related training, and home-visiting experience (11 items), as well as their views toward implementing LV (13 items). Because no published surveys have assessed views of implementing LV, the first author rationally generated the 13 “views” items. The response format of the 13 “views” items varied, using both Likert scale and semantic differential scale formats (examples of each response format are provided in Table 2). To establish content validity, these items were reviewed by the fourth author, as well as the director of the Healthy Start program.
Table 2

Views items: exemplars of response formats and views’ categories

Response format/views category

Example item

Likert-scale/Knowledge

Home visitors do not have the knowledge to provide good psychological care to depressed women

1. Strongly agree

2. Moderately agree

3. Slightly agree

4. Slightly disagree

5. Moderately disagree

6. Strongly disagree

Semantic differential/good idea

Having home visitors nurses and counsel depressed clients is: (circle one for each adjective pair):

Good 1 2 3 4 5 6 Bad

Likert-scale/comfort

I am comfortable with the idea of counseling depressed clients

1. Strongly agree

2. Moderately agree

3. Slightly agree

4. Slightly disagree

5. Moderately disagree

6. Strongly disagree

Post-LV Training Survey

This 14-item survey included all of the “views” items from the Pre-LV survey, as well as one additional Likert scale item assessing the degree to which home visitors wanted to implement LV with their clients.

Visiting Nurse Service’s Database

The organizational setting for each home visitor (Healthy Start vs. Maternal Child Health), as well as LV implementation status (implemented or did not implement LV), was determined from VNS records.

Procedures

All research procedures were approved by the university IRB. Since May 2005, all VNS home visitors (in both the Healthy Start and Maternal Child Health programs) were required to complete LV training as part of their orientation. The first LV workshop, in 2005, was provided by the British trainer, while subsequent sessions were conducted by the first author. Listening Visits training workshops were held jointly for new home-visiting staff in both Healthy Start and Maternal Child Health, approximately twice per year. Before each LV training workshop, home visitors were invited to participate in a study of their views of LV. They were asked to complete two surveys: one before and another immediately after the LV training workshop. Survey data for this study were collected from 2005 to 2009 and the LV implementation status for each home visitor was determined from the VNS database in 2009.

Data Analyses

Assessing LV-Implementation Rate

Descriptive statistics were used to calculate the percentage of home visitors that implemented LV. Fisher’s exact test was used to compare percentages of implementers in the two organizational settings.

Creation of Composite Indices of Pre- and Post-LV-Training Views

Each item that assessed views on both the pre- and post-LV training surveys was placed by the first three authors into one of the three rationally generated categories: (1) perceptions of skills/knowledge to deliver LV, (2) personal views of whether counseling by home visitors is a good idea, and (3) degree of comfort with providing counseling to depressed clients (examples of items in each category are listed in Table 2.) Within each category, items that correlated significantly were then standardized and aggregated into composite indices. Any items that had been placed in a category on a rational basis, but did not significantly correlate with the other items in that category, were excluded.

Comparing Views

Utilizing repeated measures ANOVA in SPSS, differences between the two groups of home visitors’ pre- and post-LV training views were examined.

Assessing the Significance of Views and Organizational Context

Logistic regression was conducted using SPSS to examine whether organizational context and views of home visitors predicted LV implementation. Variables associated with post-training views were not included in the model, to address the issue of low power (i.e., N = 37) and because these variables correlated with variables corresponding to pre-training views. In this analysis, the organizational context variable was forced into step one, and the three pre-training-views variables were forced into step two.

Results

Implementation Rate

While almost three-quarters (73.90%) of the Healthy Start home visitors implemented LV with a depressed client, significantly fewer (35.70%) in Maternal Child Health employed this intervention (P = .04, Fisher’s exact test).

Comparison of Views

Repeated measures ANOVA revealed that, prior to LV-training, the Healthy Start home visitors were more comfortable with the idea of providing counseling to depressed clients than were the Maternal Child Health home visitors (F [1,35] = 5.71, P = .02). The time by group interaction was not significant, thus changes in level of comfort from pre- to post-LV-training were not different between groups. From pre- to post-training, no other significant differences in views were found, between or within groups. At post-training, the groups did not differ significantly with regard to their desire to implement LV.

Relative Significance of Views and Organizational Context

Logistic regression results show that the odds that home visitors would implement LV were over five times higher in Healthy Start than in Maternal Child Health (OR = 5.10, Wald χ2 = 4.95, df = 1, P = .03). A second hierarchical logistic regression model was tested to examine whether home visitors’ pre-LV training views added significant prediction of LV implementation beyond organizational setting. Results showed that model improvement at step two was not significant, indicating that the home visitors’ views did not significantly predict implementation of LV beyond organizational setting.

Discussion

Moving evidenced-based practice into real world settings is an NIMH priority (Insel 2008). This study documents the experiences of a team of depression treatment researchers attempting to achieve this goal by implementing an empirically supported depression treatment delivered by non mental health specialists in two home-visiting programs. As treatment researchers, they naturally focused on issues pertaining to treatment training, supervision, and evaluation. This focus is conceptualized as the first phase of dissemination and implementation studies (Landsverk et al. 2010). However, because the implementation of LV was not mandated, the treatment research team also anticipated that some of the home visitors would not feel comfortable implementing a depression treatment. As a consequence, they broadened their traditional treatment focus to include an assessment of the home visitors’ pre- and post-LV training views as a potential factor that might influence implementation. Although the assessment of organizational factors related to the “dissemination and implementation phases of research” (Landsverk et al. 2010) was not initially planned, the implementation of LV into two different home-visiting programs also allowed for the broad assessment of the importance of organizational context.

Consistent with prior studies of the effectiveness of LV (Cooper et al. 2003; Holden et al. 1989; Morrell et al. 2009; Wickberg and Hwang 1996), there is preliminary evidence for the effectiveness of LV as delivered by US home visitors. The results of a recent open trial with 19 low-income and ethnic-minority women revealed significant reduction of depressive symptoms, improved quality of life and, most importantly, client satisfaction with LV (Segre et al. 2010). Despite evidence of effectiveness, acceptability and demonstrated feasibility of this intervention in a US context, the results from the current study found the implementation rate in the two home-visiting programs to differ markedly: three-quarters of Healthy Start home visitors implemented LV compared to one-third of those working for Maternal Child Health. Through pre- and post-LV training survey assessment, we examined the home visitors’ comfort with implementing LV, perceived skills/knowledge about counseling, and belief that counseling by home visitors is a good idea. The only difference in their pre- and post-training views was that, before LV training, the Healthy Start home visitors were significantly more comfortable with the idea of providing counseling than the home visitors in Maternal Child Health. Moreover, LV training did not significantly change the views of the home visitors. It is also notable that the majority of home visitors in both home-visiting programs held positive views about the idea of implementing LV.

The logistic regression results revealed that implementation of LV was determined not by the home visitors’ views, but rather by some aspect of their organizational setting. First, the odds that home visitors would implement LV were over five times greater in Healthy Start than in Maternal Child Health. Second, and perhaps surprisingly, the home visitors’ views about LV did not significantly improve the odds of implementation beyond organizational context. Collectively, these results suggest that even providers with favorable views may not be able to implement a new practice because some aspect of their organizational setting makes it difficult to do so.

This study is an important first step in understanding factors that drive implementation of an empirically supported depression treatment for perinatal depression in a community-based setting. Nevertheless, the clinical researchers were limited to implementation in two programs, hence only a small number of home visitors participated. Additionally, although the results of this study highlight the importance of the organizational setting, specific aspects of organizational setting were not assessed, making it impossible to pinpoint which specific variables most influenced the implementation of LV. Several possibilities, however, warrant discussion. First, it is possible that the smaller caseloads of Healthy Start home visitors gave them more flexible schedules, which allowed them to implement LV with their clients. Second, the two programs are funded differently and therefore have different billing requirements; this may have, in some way, encouraged or discouraged spending extra time to conduct LV. Finally, home visitors in the two programs differed in terms of their occupation; case managers work in Healthy Start, while nurses work in Maternal Child Health. Although nurses consider communication with patients a core part of their job, a nurses’ basic educational curriculum more heavily emphasizes the sciences, and their clinical training emphasizes physical assessment, basic teaching, and safety. In contrast, case managers often have an educational background in social work or psychology, and so report having more training in counseling. This prior training may account for the case managers’ significantly greater comfort with providing counseling (even before the LV training workshop) and their more frequent use of LV.

Despite some limitations, this is the first study to examine factors influencing implementation of perinatal depression treatment in real-world, clinical service settings by non mental health specialists. The findings of this study suggest that merely having an organization of receptive individuals does not guarantee that an empirically validated intervention will be widely adopted. Consistent with findings from other implementation studies (Durlak and DuPre 2008; Glisson 2002; Mendel et al. 2008; Zazzali et al. 2008), the successful implementation of depression interventions by non mental health specialists may depend even more on organizational factors. Nevertheless such variables are typically overlooked by treatment researchers. Thus, these results have important clinical and research implications. For treatment researchers trying to implement evidence-based practice in real-world settings, our results suggest that the traditional focus on training providers and evaluating treatment outcome is too narrow. To be effective, the scope must be broadened to include assessing whether the organization can support a new practice and further, whether there are factors unique to a particular organization that require alternative methods of implementation. In addition, these results highlight the current gap in our knowledge of how specific organizational factors influence whether an innovative perinatal depression treatment is implemented. This lack of understanding undermines progress towards maximizing the impact of interventions such as LV and demands additional research to discover ways to overcome the organizational barriers blocking effective dissemination of empirically validated treatments.

Acknowledgments

The authors would like to acknowledge the contribution of the staff of Visiting Nurse Services in the conduct of this research, including the VNS Chief Executive Officer, Dr. Jennifer Van Liew, the directors of Healthy Start and Maternal Child Health, Ms. Darby Taylor and Ms. Cari Spear, as well as the home visitors from both programs.

Disclosure

No competing financial interests exist.

Copyright information

© Springer Science+Business Media, LLC 2011