Background

Because each organizational environment represents a complex human activity system that directly impacts technology/innovation transfer, studies of innovation transfer within drug treatment programs can benefit from understanding its organizational factors [1]. Organizational [2, 3], systemic (e.g., workplace stress perceptions), and idiographic variables (e.g., longer service tenure [4]) have been shown to predict implementation and adoption attitudes.

Many frameworks and models have been proposed for understanding this issue [5–10]. One literature review of 81 studies on quantitative and qualitative factors affecting implementation [11] identified five essential areas (innovations, providers, communities, organizational capacity, training/technical assistance) with conclusions summarized in a hierarchical framework consisting of three nested ovals. Adaptability of the innovation in the proposed environment is central among innovation characteristics addressed in the innermost oval [7, 11] where implementation is hypothesized to be a function of organizational capacity (e.g., leadership, a program champion/internal advocate, managerial/supervisory support) and training/technical assistance. The middle oval encompassed provider characteristics (e.g., perceived need and perceived benefits of the innovation, self-efficacy, and skill proficiency for the innovation). The outer oval consists of community factors (e.g., funding, community readiness for change). The emphasis on training and technical assistance as central to improvement efforts is one of the review’s important contributions.

Factors affecting organizational capacity include a positive work climate, organizational norms regarding change, new programming integration, and shared vision, while social context/work environment perceptions affect not only technical assistance usage [5, 8, 12, 13] but also adoption, implementation, sustainment and effectiveness [8]. Another relevant organizational process factor is absorptive capacity [14]. This addresses access and ability to use information effectively by the organization [15, 16]. It is enhanced by a professional workforce, engagement in environmental scanning (i.e., identification of training and professional development opportunities), and satisfaction data from the organization’s buyers and suppliers. Moreover, identification of individuals who serve as “information brokers” can enhance absorptive capacity.

While physical and perceived program resources have been shown to affect training exposure and utilization [17], it is program leadership that is considered the primary factor, as it affects the entire organizational environment and is instrumental in adoption decisions [18]. Strong leadership is particularly important to organizational climate in times of system and organizational change [19] by making the climate more open to change and by empowering staff to be participants in change [20]. Delegation of authority to competent people is important [21], for they can serve as effective change agents through working/bonding with treatment teams, administrators, key opinion leaders, and community stakeholders [6, 8, 22]. Valued characteristics for internal change agents include self-efficacy [23–25] and adaptability [9, 26].

Similarly, interest in innovations and influence with other staff are likely important attributes [27, 28], as are growth, efficacy, influence, and adaptability. Being employed in organizational climates conducive to change (i.e., higher mission, cohesiveness, autonomy, communication, and openness to change, but lower stress) have been found to have more favorable attitudes about innovation training and its adoption [28]. Professional growth and influence predicts openness to the use of medications [27] while staff adaptability and influence significantly predicts opinions toward treatment manuals and integrated mental health services [27]. Thus, implementation of new practices is influenced directly by program leadership’s adoption decisions and indirectly through leadership’s shaping of the organizational climate, task delegation, and selection of change agents [1, 20].

Current study model

Of the many factors that have been proposed for better understanding innovation transfer, increased insight into the interrelationships among program leadership, organizational climate, and change agents would seem to be critical. Furthering knowledge about these change agents to whom transfer tasks are delegated with regard to what might be potential effective characteristics and their relationship to program leadership and climate is an area that requires additional study. For the current study, the proposed interrelationships of selected factors related to this concept are examined simultaneously in an implementation model for drug treatment innovations. The interrelated roles among program leadership, organizational environment, counselor leadership, and the clinical staff attributes are conceptualized in the path model presented in Fig. 1 and summarized by the hypotheses below. Principal are the effects of two leadership paths (director and staff).

Fig. 1
figure 1

Conceptual model of counseling innovation interest on training participation and endorsements

Full size image

Hypotheses

  1. (1)

    Director Leadership will have direct effects on Organizational Climate (OC).

  2. (2)

    Director Leadership will have an indirect positive relationship with Counseling Innovation Interest (CII) through its relationships with OC and Counselor Efficacy, which has a direct positive relationship with CII.

  3. (3)

    Counselor Adaptability will have both direct and indirect (through Counselor Efficacy) positive relationships with CII.

  4. (4)

    CII will be positively related to Counselor Influence and participation in recent trainings.

  5. (5)

    Counselor Influence will moderate the relationship between CII and subsequent endorsements of innovation adoption following training.

Methods

Procedure

Data were collected in 2013 as part of the Treatment Readiness and Induction Program (TRIP) project funded by the National Institute on Drug Abuse (NIDA), National Institutes of Health (NIH), and Department of Health and Human Services. The project was structured as 2 phases. Effectiveness was examined in Phase 1 [29, 30]. Phase 2 examined implementation of TRIP in multiple juvenile-justice and community-based sites in the United States.

The current study is based on the phase 2 data. Regional Addiction Technology Transfer Centers (ATTCs), which are responsible for providing training and technical assistance to agencies and their staffs on best practices for substance use, assisted with program recruitment. With the help of four ATTCs (Great Lakes, South Southwest, Pacific Southwest, and Northeast and Caribbean), the implementation sample included 312 counselors from 52 adolescent treatment facilities located in 12 US states. Institutional Review Boards (IRBs) for the research center and treatment programs reviewed and approved study protocols.

Data collection procedures consisted of (a) pre-training survey of the organization (completed by director or director’s designee), (b) pre-training survey of counseling staff, (c) survey at the conclusion of the training by “trainees,” and (d) follow-up counseling staff survey at 4 months after the training.

Pre-training data were collected from February to July 2013. About 1–2 months prior to the training workshop, all counseling staff at the participating facilities with direct client contact (counselors, social workers, case managers, clinical supervisors, and therapists) were provided with information describing the study and informed consent was obtained. Participants were asked to complete the TCU Survey of Organizational Functioning and Leadership (SOFL; [31]), which as noted by Broome and his colleagues is the TCU Survey of Organizational Functioning (SOF) and 4 subscales (Encourages Innovation, Inspirational Motivation, Develops Others, Task Delegation) from the Survey of Transformational Leadership instrument [32]. The SOFL scales covered organizational climate, staff attributes, motivation for change, resources, job attitudes, workplace practices, transformational leadership, and training exposure and utilization. Additionally, one month prior to the training, the program director (or program designee) completed the TCU Survey of Structure and Operations (SSO) [33], which included organizational level information.

Approximately 1 month after staff surveys were administered, 1–2 clinical staff at each participating program were chosen by its program director to attend a 2-day developer-led workshop on use of the TRIP curriculum and serve as trainers. At the conclusion of the 2-day workshop (approximately 13 h), participants received a survey consent form and were asked to complete a 30-min TCU Workshop Evaluation Form (WEVAL) [34] addressing satisfaction with the workshop/materials, perceived adequacy of resources, training, and support for implementing TRIP at their treatment program. These data were collected anonymously onsite by research staff.

Participant samples

Two samples were examined in the current study: (a) counseling staff who completed the SOFL at Time 1 (used to examine the structural model), and (b) counseling staff trainees (used to examine the interaction of counselor innovation interest with attitudes on subsequent endorsement of TRIP). There were 312 counselors from 52 programs who completed the SOFL. Because of missing data on analysis variables, 11 staff representing 3 facilities were omitted. From the SSO, the counselors were classified into 6 treatment modality groups: secure juvenile justice settings (SJJ), residential or modified therapeutic community with 60% or more juvenile justice clients (R/TC + 60), other residential or modified therapeutic community (R/TC-60), outpatient with 60% or more juvenile justice clients (OP + 60), other outpatient (OP-60), or both outpatient and residential/modified therapeutic community (OP/TC). This was done to adjust for possible modality differences that might affect the results. The final sample for examining the structural model (hypotheses 1–4) included 301 clinical staff, representing 49 facilities (7 SJJ, 7 R/TC + 60, 13 R/TC-60, 5 OP + 60, 12 OP-60, and 5 OP/TC).

There were 75 counselors from 47 treatment programs who attended TRIP training and completed the WEVAL [34]. Of these clinical trainees, 68 had data on the WEVAL scales and the SOFL measure on counselor innovation interest that were used for examining post-training attitudes in hypothesis 5. They represented 43 treatment facilities [7 SJJ: 12 staff; 6 R/TC + 60: 10 staff; 9 R/TC-60: 12 staff; 11 OP-60: 19 staff; 5 OP + 60: 9 staff; 5 OP/TC: 6 staff]. Table 1 presents background information on both the sample used for the structural model and the subsample of clinical trainees. The table suggests there are similarities between the two samples. In both samples, the average age was approximately 41, the percentage of females was nearly two-thirds, Whites comprised a majority (although a smaller percentage in the TRIP training sample), and as expected, a majority had a bachelor’s degree or higher.

Table 1 Background characteristics of structural equation model and TRIP training samples
Full size table

Measures

Organizational climate, staff attributes, and director leadership

The SOFL [31, 32] is a 165-item instrument. The SOFL includes the TCU Organizational Readiness for Change (ORC) instrument, which measures 18 dimensions covering four major areas: Program Needs/Pressures, Program Resources, Staff Attributes, and Organizational Climate [35]. The background and development of these scales have been discussed previously, emphasizing their intended link to organizational change [35]. In addition to the ORC, the SOFL includes 8 additional scales covering workplace practices and job attitudes as well as 4 scales from the TCU Survey of Transformational Leadership (STL) instrument: encourages innovation, inspirational motivation, develops others, and task delegation [32]. Each item was rated on a 5-point Likert scale [1 = disagree strongly, 2 = disagree, 3 = uncertain, 4 = agree, 5 = agree strongly; 32, 31]. In computing the scale scores, the responses on the items for a scale are averaged and then multiplied by 10. The present study uses items from the staff attributes, organizational climate, and transformational leadership scales to address the hypotheses. These scales are listed in Table 2.

Table 2 Scales and sample items included in estimated structural model
Full size table

Recent training activity

Three items from the SOFL Training Exposure index addressed “active participation in training in last 6 months.” These included “learned new skills or techniques at a professional conference,” “how often attended training workshop within 50 miles,” and “how often attended training workshop over 50 miles.” The first item using the 5-point response format (disagree strongly/agree strongly) was dichotomized (disagree strongly, disagree, or uncertain = 0, agree or agree strongly = 1). The latter two items had a 5-point response format (1 = none, 2 = 1, 3 = 2, 4 = 3, and 5 = 4 or more) and they were also dichotomized (“none” = 0 or “at least 1” =1). This 3-item index had a coefficient alpha of .59.

Counseling innovation interest (CII)

This scale, counseling innovation interest, addressed both seeking professional growth and “openness to using new counseling technology” [36]. As an expression of innovation interest, it includes previous staff attendance at workshops [36] and growth-related personal activities, including willingness to try new therapeutic approaches [20]. This measure taps “leadership initiative” among counseling staff, paralleling the organizational absorptive capacity (identification of training and professional development ideas) at the individual level measure and may be indicative of an individual’s potential as an internal agent of change within the organization. It is hypothesized to serve as a mediator between counselor attributes (efficacy and adaptability) and indicators of implementation (e.g., positive training endorsement, adoption, use of new practices). The CII measure was created using 9 items (see Table 2) from the SOFL including reading about new treatment techniques, updating skills, and willingness to try new therapeutic approaches.

Training endorsements

The TCU Workshop Evaluation Form for TRIP Training (WEVAL) [34] consists of 42 items with a 5-point Likert response scale (1 = Disagree Strongly, 2 = Disagree, 3 = Uncertain, 4 = Agree, and 5 = Agree Strongly). Five scales (Table 3) were used and they addressed staff attitudes toward using TRIP within their treatment setting: Acceptability, Appropriateness, Adoption Expectation, Preparation Adequacy, and Leadership Engagement. Scores for each scale were calculated by averaging responses to its set of items and multiplying by 10.

Table 3 Scales and items from the TCU Workshop Evaluation Form (WEVAL)
Full size table

Analytic plan

In order to account for treatment modality differences, a within-modality covariance matrix was used for the analysis of the structural model. This estimated matrix represents the covariance matrix common to the treatment modalities. It was obtained from SAS PROC DISCRIM [37], and the structural equation model was tested using SAS PROC CALIS [37]. Goodness-of-fit χ 2, normed χ 2, root mean square error of approximation (RMSEA), standardized root mean square residual (SRMSR), comparative fit index (CFI), and Normed fit index (NFI) were used to assess the fit of the model to the data. For goodness-of-fit χ 2, RMSEA, and SRMSR, smaller values indicate better fits. Because the goodness of fit χ 2 may be sensitive to sample sizes larger than 200, the normed χ 2 (χ 2 divided by its degrees of freedom) is considered a better χ 2 indicator of fit; that is, when less than 3.84 [38]. For RMSEA, values of .01, .05, and .08 indicate excellent, good, and mediocre fit, respectively [39]. For SRMSR, a value less than .08 is considered a good fit [40]. The CFI ranges from 0 to 1, with values above 0.9 indicating reasonable fit. For NFI, values of .90 to .95 reflect a good model fit. For hypothesis 5, SAS PROC GLM was used to address endorsement in terms of CII and Influence.

Results

Hypotheses 1–4: model for counseling innovation interest (CII)

Initial model

The initial structural model examined the first 4 hypotheses of the conceptual model outlined in Fig. 1 and was based on the sample of 301 counselors. While all estimated paths were significant, with the exception of the path from CII to Influence, this “basic model” (Time 1 measures only; excluding Training Endorsements and Future Implementation) had relatively “poor fit-statistics” [χ 2(11) = 44.6993, p < .0001; normed χ 2 = 4.06; RMSEA = .102; SRMSR = .072; CFI = .94; NFI = .92].

Revised model

An examination of the Lagrange Multiplier test indices suggested two modifications: a path from Counselor Adaptability to Counselor Influence and a path from Director Leadership to Recent Training Activity. These modifications were made, as well as the deletion of the path from CII to Influence, and this revised model was re-estimated. Results indicated a “good fit” since its RMSEA was under .05. All other fit criteria were also improved [χ 2(10) = 16.2064, p < .0939; normed χ 2 = 1.62; RMSEA = .046 (.000, .085); SRMSR = .044; CFI = .99; NFI = .97], and all hypothesized paths were significant at p < .01 or less. This final model is displayed in Fig. 2.

Fig. 2
figure 2

Estimated structural model for leadership, climate, staff attributes, counseling innovation interest, and training activity relationships

Full size image

The significant paths (standardized) were from Director Leadership to Organizational Climate (b = .67, se = .03, t = 20.84, p < .0001) and to Recent Training Activity (b = .17, se = .05, t = 3.21, p < .0013). From Counselor Adaptability there were significant paths to Counselor Efficacy (b = .53, se = .04, t = 13.01, p < .0001), CII (b = .18, se = .06, t = 3.13, p < .002), and Counselor Influence (b = .27, se = .06, t = 4.78, p < .0001). Organizational Climate had a significant path to Counselor Efficacy (b = .19, se = .05, t = 3.99, p < .0001). Counselor Efficacy had significant paths to CII (b = .42, se = .05, t = 7.68, p < .0001) and Counselor Influence (b = .32, se = .06, t = 5.71, p < .0001). CII had a significant path to Recent Training Activity (b = .30, se = .05, t = 5.86, p < .0001). Counselor Influence was significantly correlated with Training Activity (r = .13, se = .05 t = 2.90, p < .004).

Results suggested that both Counselor Efficacy and CII are pivotal endogenous variables. Counselor Efficacy carried an indirect path to CII and Counselor Influence through its relationship with Organizational Climate and Counselor Adaptability. CII was a key predictor of Recent Training Activity in the model but not Counselor Influence. The R2 values (which represent the proportions of variance that the predictors account for in each endogenous variable) ranged from .13 (Recent Training Activity) to .45 (Organizational Climate). The R2 for the two key endogenous variables (Counselor Efficacy and CII) were .34 and .30, respectively, indicating these two variables had fairly large percentages of variance being accounted for.

Hypothesis 5: effects of CII and counselor influence on training endorsement

While a fairly large sample was available to test the structural model, only the subset of 68 counselors selected to attend training on implementing TRIP were available to address endorsement. This subsample therefore does not represent a random sample of the total as they were selected by their treatment sites to attend the training. It might be expected that these counselors would be higher on some of the characteristics that are considered important in technology transfer compared to the remaining sample.

Indeed, it was found that the subsample was significantly higher on Counselor Influence [TRIP Training: mean = 40.0 (SD = 5.8); non-TRIP Training: mean = 36.8 (SD = 6.9), t = 3.36, p < .0009] and on Counselor Adaptability [TRIP Training: mean = 41.3 (SD = 4.4); non-TRIP Training: mean = 39.8 (SD = 5.3), t = 2.00, p < .046]. However, the two samples were not significantly different on Counselor Efficacy, Organizational Climate, Director Leadership, CII, or Recent Training Activity.

Even though the trainees were found to be higher on some of the characteristics that are considered important in technology transfer (Counselor Influence, Counselor Adaptability), it was expected that attribute variations existed within this trainee subsample and that level of counselor influence would moderate the relationship between CII and endorsement of innovation adoption. To display the relationships between training endorsement measures with Influence and CII (hypothesis 5), both Influence and CII were dichotomized at 35 or higher (agreement range) versus lower than 35 (disagreement range).

Of the four quadrants produced by these two dichotomous variables (high CII and high Influence; high CII but low Influence; low CII but high Influence; low CII and low Influence), two quadrants contained very few individuals [4 in the “both low” quadrant (5.9% of total); 6 in the high CII but low Counselor Influence quadrant (8.8% of total)]. Therefore, to test the “interaction hypothesis,” the individuals who possessed both high CII and Counselor Influence (n = 32) were compared with all other individuals (n = 36). It was hypothesized that the group who had high scores on both measures would show significantly greater workshop endorsements relating to Acceptability of workshop training, Appropriateness of TRIP for their setting, Adoption Expectations, and Preparation Adequacy. As shown in Fig. 3, the high CII-Influence group had significantly higher means than the remainder group for Acceptability {(43.1 vs. 41.1); [F(1, 66) = 7.26, p < .01, η2 = .10, ES = .33]}, Adoption Expectation {(47.8 vs. 43.1); [F(1, 66) = 10.95, p < .0009, η2 = .14, ES = .40]}, and Preparation Adequacy {(43.4 vs. 41.3); [F(1, 67) = 4.33, p < .041, η2 = .06, ES = .25]}. The two groups were not significantly different on Leadership Engagement {(38.9 vs. 38.4); [F(1, 66) = .21, p < .65, η2 = .00, ES = .00]} or Appropriateness {(43.1 vs. 41.5); [F(1, 67) = 2.25, p < .14, η2 = .03, ES = .18]}. The significant findings were associated with medium to large effect sizes [41].

Fig. 3
figure 3

Future training (WEVAL) outcomes by CII and influence groups

Full size image

Discussion

The present work is an exploratory study examining how “leadership” is important to the facilitation of implementation. It extends the work of Becan and colleagues [20] by conceptualizing change-oriented staff attributes (efficacy, adaptability, influence and professional growth) as “opinion leadership” and modeling the interrelationships of program leadership, organizational climate, opinion leadership and participation in recent trainings.

Two avenues of leadership and their potential effects on technology transfer were explored. The first was that overall leadership (represented by the director of the drug treatment organization) can affect implementation by instilling an organizational climate favorable to innovation adoption. The second was “opinion leadership” [6] among some counselors, as represented by an interest in innovation adoption and who have influence among co-workers within the organization. These counselors might serve as change agents, especially if they are delegated with some responsibility for orchestrating the new service implementation, and they might be candidates to become trainers in the train-the-trainer model for innovation implementation [42–45].

In summary, there were two key exogenous variables (variables not predicted by any other variable in the model) and five key endogenous variables (variables predicted by a variable in the model). The exogenous variables – Director Leadership and Counselor Adaptability – had significant paths to CII, either indirectly (Director Leadership) or directly (Counselor Adaptability). Director leadership had a pervasive relationship on all of the endogenous variables through its path to the organizational climate of the treatment program. Counselor Adaptability was related to multiple variables in the model directly, including perceptions of Counselor Efficacy, CII, and Counselor Influence. The relationship of Counselor Adaptability to Counselor Efficacy was stronger than Organizational Climate to Counselor Efficacy.

These findings corroborate previous work documenting the cascading effect that director leadership has on the organizational environment and subsequently on counseling staff attributes [20]. The findings also support the importance of both program leadership and organizational environment in implementing change [2, 14, 46].

While the present results were promising, they are not definitive, as the model data were limited and collected in a single instrument. Therefore, the results only confirm that the paths for the constructs are consistent with their hypothesized relationships but not evidence for causality. That would require additional data collection and testing. Nevertheless, the study does provide evidence that leadership both at the management level and at the staff level deserve further consideration in implementation efforts. The delegation of implementation activities to key staff who have an “aptitude” (e.g., innovation interest, efficacy, adaptability, and influence) for assisting in implementation, and who can serve as a “champion of the innovation” among staff, is believed to be critical to success. The results are consistent with previous research that show unique connections between director leadership, climate, and staff level leadership attributes, and contribute to previous work showing organizational climate as a mediator of the relationship between leadership and staff turnover intentions and turnover [19].

The second analysis addressed counselor-level leadership (high counseling innovation interest and individual influence) effects on endorsement attitudes toward treatment innovations. While this subsample reported higher Counselor Influence and Counselor Adaptability compared to the total sample, reported differences in training endorsement were detected. The results support the idea that counselors who pursue clinical skills development and who have influence among other staff are likely to have a skill set that positions them to champion change in their organization [2]. Future studies are needed to examine the process by which the combined effect of strong program leadership and the presence of change agents (staff that avidly pursue clinical skill development and influence coworkers in the use of new innovations) can together promote the uptake of innovations among staff within their agency.

While there are several strengths of this study, including the potential to generalize these findings to a wide variety of adolescent treatment settings, limitations should be acknowledged. First, data for the estimated model are cross-sectional, thereby eliminating the possibility to examine causality. Second, the sample to evaluate the effect of counselor innovation interest and influence among coworkers on innovation endorsement was small. Third, the examination of this combined effect was conducted on a specific innovation, which limits generalizability to other innovations.

Conclusions

Leadership at the helm as well as within the ranks are important factors in technology transfer. Directors that involve staff in decision making and who delegate implementation tasks could bolster staff adaptability and self-identified influential staff. Key people leading change, supportive organizational culture, a positive pattern of managerial and clinical relations, and match between the core mechanisms of the innovation with the change agenda are important to successful change [47]. Absence of leadership at either level is a barrier to innovation adoption.

As noted by Flynn and Brown [48, 49], the knowledge base for guiding adoption and implementation of evidence-based practices (EBP) in the drug use treatment field is still modest. Treatment program resources are often limited, and programs are not likely to undertake change by themselves or allocate resources for an external change agent to guide implementation. Therefore, using internal personnel with “innovation leadership attributes” can help minimize costs associated with system change by serving as a resource in assisting implementation of certain manual-driven EBPs.