In the current healthcare climate, there are ever-present calls for quality improvement and increasing efficiency of operations (Allen 2013). Although Pfadt and Wheeler (2006) recommended use of data in improving quality of services to individuals with intellectual disabilities almost a decade ago, there continues to be little use of such data to drive system improvements. As such, there has been a renewed push for use of evidence-based practices and data-driven quality improvement for individuals with intellectual disabilities (Turk 2013).

Complicating this call for data-driven improvements are the complex presentation, etiology, and treatment needs confronting healthcare professionals. Technical assistance, provision of specific knowledge or services to agencies by individuals with expertise in that area, has emerged as a primary assistive process to help treatment teams deal with the complexity of health and behavior challenges some individuals present. Technical assistance exists for a wide range of issues and concerns, such as substance use (Lundgren etal. 2013), perinatal depression (Segre etal. 2013), and HIV/AIDS (Alexander etal. 2012). Unfortunately, there is minimal information about technical assistance for individuals with intellectual disabilities, and that which does exist focuses primarily on a need for technical assistance without measures of effectiveness; hence, the need for implementation of evidence-based procedures, including technical assistance, within the intellectual disability arena (Bacon etal. 2011).

Much of the work related to technical assistance with individuals with intellectual disabilities focuses on school-aged individuals. Some authors, however, have effectively argued a need for such assistance at the postsecondary level (Hart etal. 2010) and as part of early intervention strategies (DeVore and Russell 2007) as well. Still others (Kozleski and Smith 2009) applaud the concept of technical assistance without providing any relevant data to support effectiveness. In a recent study, Jenkins etal. (2011) described the use of technical assistance from the World Health Organization to establish a sustainable mental health system that also addressed providing services to individuals with intellectual disability. This model was effective at creating a nationwide system with low resources and improving access to services, but much work remains in assessing quality and outcome services. This is true in the field of intellectual and developmental disabilities (I/DD) in general. Few studies have effectively evaluated the impact (either process or outcome) of technical assistance, which was the purpose of this study.

Organization

The Centers for Medicare and Medicaid Services (CMS) and the Commonwealth of Pennsylvania’s Office of Developmental Programs (ODP) share an interest in promoting quality services for individuals with I/DD. The Home and Community-Based Services (HCBS) Quality Framework (Medicaid Home and Community-Based Services Measure Scan: Project Methodology: Contract Report, December 2012), part of the National Quality Inventory Project, serves as a reference for providing those quality services. Eight Health Care Quality Units (HCQUs), established across the Commonwealth of Pennsylvania, support ODP and County Administrative Entities (AEs) in the domains of participant-centered service planning and delivery, provider capacity, participant safeguards, participant rights and responsibilities, and system performance. HCQU activities build capacity and competency to address physical and behavioral healthcare needs of individuals with I/DD through training, technical assistance, and outreach to community resources. The HCQU organization that conducted this study provides expertise and assistance to residential and day program providers and others who support individuals with I/DD residing in eight southwestern Pennsylvania counties.

Procedure

One method developed to improve quality of life of individuals with I/DD was a process called complex technical assistance (CTA), a holistic approach to complex physical and behavioral health issues of individuals with I/DD. Collaboration with stakeholders produced a detailed process encompassing all levels of support for an individual, including management, direct support caregivers, family members, and ancillary support systems, such as supports coordinators and behavior specialists. Through this process, the HCQU supports interprofessional teams by providing facilitation, education, resources, and alternative approaches to issues specific to individuals. The HCQU implemented the process with training sessions for stakeholders throughout the eight-county region (Fig. 1).

Fig.1
figure 1

Comparison of previous and revised CTA processes. Bolded processes indicate part of the quality management revision. Outside of specified timeframes there are no required time limits

Full size image

We developed a process for addressing all CTAs referred to the HCQU, from original receipt of referral through identification of appropriate members of the interprofessional team to closure of the CTA. Upon receipt of the referral, HCQU team members met with interprofessional team members at their place of convenience until team agreed they had completed the process (for details, see Fig. 1). Following implementation of our original CTA process, the HCQU identified concerns regarding omission of key team members from CTA activities, insufficient communication among the team, and a long average length of time required to complete CTAs. To address these concerns, we revised the process and recognized the need for a Complex Technical Assistance Quality Measurement (CTA QM) Committee to explore methods for measuring quality of the CTA process and its impact on lives of individuals involved. This committee consists of an interprofessional team who worked together to oversee development of a standard operating procedure (SOP) to effect consistency in execution of CTA activities, with a focus on development/refinement of two process quality measurement tools: Team Process Assessment (TPA) survey and CTA Evaluation survey.

We developed the TPA survey to measure CTA team dynamics and functionality. The TPA consisted of nine items that HCQU team members rated after each CTA meeting regarding their perceptions of team functioning. Using a six-point Likert scale, HCQU team members rated the interprofessional teams along various dimensions (see Table 1). To establish a level of inter-rater reliability, the assigned HCQU nurse and behavioral health coordinator completed TPA surveys independently following each CTA meeting and then completed a joint TPA survey upon debriefing (see Table 1). We piloted the TPA process for 1 month prior to implementation for education of HCQU teams and refinement of the survey and process. This process works well for other types of surveys and provides a measure of inter-rater reliability (Goreczny etal. 2005). We computed inter-rater reliability for 144 process assessments involving 59 CTAs completed by both the nurse and behavioral health coordinator. Results revealed a high rate of inter-rater reliability (see Table 1). Analysis revealed that all inter-rater reliabilities were above 85 % when comparing responses within one rating of each other. On statement 4 (i.e., team members were able to develop goals/outcomes that were measurable and relevant), which is not on a Likert scale and therefore not amenable to such comparisons, we evaluated only exact agreement, which was 89.6 %, also indicating high inter-rater reliability (Table 1).

Table1 Inter-rater reliability of process assessment
Full size table

We also modified the CTA process to include development of a person-centered goal statement by the CTA team at the first or second meeting. We incorporated that goal into the CTA Evaluation survey used to evaluate satisfaction regarding HCQU services. Interprofessional team members rated survey statements along a five-point Likert scale (see Table 3). The purpose of including an outcome goal was twofold: (1) to focus CTA activities on issues specified on the original referral to facilitate timely closure and (2) to increase rate of return for CTA evaluations because involvement in the process increases feedback. Additional efforts toward increasing this rate of return included revision of evaluation cover letter to specify return methods available and addition of follow-up contacts to participants of evaluations not received within 2 weeks from the date distributed. We offered multiple methods of return for evaluations, including phone, mail, email, and fax because data indicates this increases the rate of return (Black etal. 2009; McCluskey and Topping 2011).

Results

During the 2 years following implementation of the revised process, we received 75 new CTAs. In order to compare responses on process assessment surveys for meetings throughout the entire course of each CTA to determine if responses improved as each CTA progressed, we analyzed data only on those that had process assessments completed throughout the entire duration of the CTA process, from initial meeting through closure. Fifty-one CTAs met this criterion. The vast majority of CTAs closed after two meetings; one CTA required seven meetings before closure. Results indicated that HCQU team members rated the CTA process very highly throughout the entire process (Table 2).

Table2 Results of process assessment
Full size table

We received satisfaction surveys pertaining to 50 CTAs. There were 27 cases from which we received one survey, 16 cases from which we received two surveys, and 7 cases from which we received three surveys, resulting in a total of 80 surveys returned. Thirty-five surveys were from supports coordinators; the remaining surveys were from a wide range of participants, including executive administrators, family members/guardians, and residential administrators/supervisors. Results from these data indicate a very high level of satisfaction on all items; the lowest rating was 4.32 on the five-point scale (see Table 3).

Table3 Results from satisfaction questionnaire
Full size table

In order to evaluate impact of the new procedure on efficiency of operations, we compared the length of time required for cases open prior to implementation of the revised process versus those open subsequent to implementation. We collected data on all 290 cases during the past 7 years. Of these 290 cases, 18 remained open at the time of analysis. Thus, there was complete data regarding length of time the case remained open on 272 cases. Of these 272 cases, 217 closed after completion of HCQU CTA functions. Other reasons for closure included: referral to the Positive Practice Resource Team (a cooperative effort and initiative between the Office of Mental Health and Substance Abuse Services and the Office of Developmental Programs; 13 cases), change of individual’s residence (13 cases), or death of individual (29 cases).

We analyzed data related to the 217 cases closed following completion of HCQU CTA functions. Eighty cases opened and closed prior to implementation of the new process; 12 cases opened prior to that implementation but closed after its implementation; 125 cases opened and closed after that implementation. We eliminated those cases opened prior to implementation of the new process and closed subsequent to that implementation in order to avoid transition effects related to process changes. Using a t test with equal variances not assumed, analysis of average number of days opened was statistically significantly different between the old and new process, t (118) = 3.94, p = 0.000. The average number of days cases remained opened decreased from 211.2 (standard deviation = 138.39) to 143.3 (standard deviation = 85.24), a decrease of over 30 %. Range of days opened for the old process was between 5 and 576 days; the range for the new process was between 15 and 386 days. In addition, utilization of the CTA process as evidenced by the number of CTAs requested increased by over 100 %, with an average of 20 CTAs per year during the first 4 years of operations and 41 per year over the last 3 years of operations following implementation of the new process.

Discussion

The process assessment yielding such high results indicates that the HCQU team members evaluated the interprofessional team functioning very high. This is important because effective interprofessional team functioning is critical to overall health outcomes. In addition, such evaluations are one component of a quality improvement process (Lim etal. 1999). The statement, “Team members actively participated in generating ideas for ways to solve the problems,” averaged the lowest initial response of the nine statements. Together with high ratings on the other items, these responses seem to indicate that the teams function well but are in need of additional perspectives to inspire new ideas for managing complex issues. The HCQU provides such direction through the CTA process and facilitates teams in a manner that cultivates the fundamental skills the teams need to generate ideas independently.

Responses indicating that the teams participate more actively in the generation of ideas at subsequent meetings than at initial meetings supports the validity of the CTA process. However, the propensity toward increasing participation declines with the fourth meeting, then fluctuates for subsequent meetings. This calls into question the benefit of continuing CTAs beyond the third meeting. It might appear that the most progress occurs within the first three meetings, but there are other factors to consider. Future research might involve a large sample of cases that extend past three meetings to help understand factors involved (e.g., complexity of cases) and reach a conclusion about an optimal number of meetings.

Use of teams consisting of professionals from several different disciplines is becoming increasingly common. This is the case for professionals working with a wide range of developmental disabilities. Much, if not most, of this work, however, has taken place in educational settings addressing educational needs of individuals with developmental disabilities. A recent study (Molteni etal. 2013) utilized a variety of assessment methods (including focus groups and direct observation) to assess the teamwork process of a group of professionals aimed at providing support to increase social communication and emotional regulation among a group of individuals with autism. Results from that study indicated that the team approach helped promote a good process of daily exchange of information that assisted in collaborative educational planning. Another group of investigators (Garcia-Biggs etal. 2010) described the importance of using a team approach to address the issue of bullying in schools, but, like many such articles, there was no assessment or evaluation data.

As mentioned, most studies evaluating the use of teams for helping individuals with disabilities are in educational settings, but there is a real need for research on individuals with autism as they transition to adulthood (Friedman etal. 2013). One reason for this is that they move from a coordinated entitlement system to a system of underfunded and uncoordinated programs. In a recent study, Kripke etal. (2010) described a program using technical assistance as one piece of an overall program designed to ensure that youth with developmental disabilities have access to needed healthcare services during their transition into adulthood. This program (called CART, which stands for Clinical Services, Advocacy, Research, and Technical Assistance) represents a unique system of care that can serve as one possible comprehensive model. Programs such as the CART model described by Krikpe and the complex technical assistance model described in the current paper embody the team approach that is necessary to provide quality care to individuals with intellectual and developmental disabilities, especially with their complex needs and within the changing healthcare environment.

That these two programs (i.e., the Kripke study and the current study) focused on technical assistance is important because technical assistance has become one of the primary methods used to support the states’ efforts to promote community integration of individuals with disabilities (Substance Abuse and Mental Health Services Administration 2012). This has especially been the case for individuals with intellectual and developmental disabilities, particularly because of the closure of so many large residential facilities during the past few decades to facilitate community integration (Cocks etal. 2014). Thus, it becomes increasingly important to assess impact of programs like technical assistance. Unfortunately, few studies have done so. In fact, a review of the literature as well as calls to other states yielded no data on effectiveness of technical assistance. The current study, however, does provide support for the positive impact of technical assistance using interprofessional teams.

Although there is a definite need for interprofessional teams, there still also remains a role for specialized behavior therapy teams when dealing with challenging behaviors among individuals with intellectual and developmental disabilities, a common referral to technical assistance teams. What is essential, however, especially given the current data-driven climate, is also for research to evaluate effectiveness of these teams, something that has been sparse. Recognizing this need, Hassiotis etal. (2009) evaluated effectiveness of a specialized behavior therapy team compared to standard treatment. Results of this randomized, single-blind study revealed that the specialized behavior therapy team evidenced superior treatment outcomes over standard treatment, and there was a trend toward significantly lower costs for the behavior therapy team compared to standard treatment. Given that some challenging behaviors may also have their roots in areas outside of behavior therapy specialists, it seems likely that interprofessional teams might have even greater impact than specialized behavior therapy teams on amelioration of challenging behaviors among individuals with intellectual and developmental disabilities. Future research needs to evaluate whether any such advantage is present for interprofessional teams.

The high level of satisfaction with CTA involvement from among stakeholders is an important measure of quality of services. The stakeholder satisfaction survey statement rating the impact of the CTA on the individual’s life was the lowest-scoring statement (mean of 4.32), as shown in Table 3. Participants expressed in the “comments” section of the evaluations that they could not determine impact within 30 days; a longer timeframe is necessary to practice the skills learned during the CTA, fully implement suggestions, and accurately assess the impact. Therefore, to accurately measure effectiveness of CTAs in terms of their impact on individuals involved, we added a 6- to 9-month follow-up to the process, completed by the County Administrative Entity (AE) gatekeeper who will review desired and actual outcomes and assessment of the status of the individual related to those outcomes. Completion of a significant number of follow-up assessments will allow the HCQU to examine the long-term effectiveness of the CTA process and possibly glean ideas for future refinement of the process. That the current study included input and evaluations from stakeholders is a strength. Very few studies have evaluated stakeholders’ views regarding perceptions of improvement, but this is a critically important perspective to understand and appreciate when working with individuals with developmental disabilities (Cocks and Boaden 2011).

Response data for the statement, “Team members were able to develop goals/outcomes that were measurable and relevant” produced interesting results. Although responses from the HCQU nurses and behavioral health coordinators suggest that almost all teams were able to develop measurable and relevant goals, review of those goals found many to be relevant but not necessarily measurable. This is an area for future improvement of our processes as well as for others to review. This is especially important because healthcare is becoming increasingly data-driven. Electronic medical records combining a person’s records from all involved healthcare providers and allowing each of those providers access to the combined data are becoming the new standard. Each section of that electronic record will likely include a field for clinicians to indicate an outcome measure. Refining processes to improve quality and measure outcomes represents an important step in this current changing healthcare environment.

The significant decrease in length of time required to complete CTAs is an important indicator of increased efficiency. Still, this increase in efficiency did not result in low ratings from stakeholder. In fact, it was quite the opposite—the number of CTA referrals doubled, one clear sign of satisfaction with services. Several process improvements may have helped result in the decrease in length of time. We coached AE gatekeepers to encourage submission of CTA referrals prior to individuals reaching crisis level. In addition, our participation in AE quality improvement and risk management committees affords us the opportunity to identify cases for referral to avert crises, thereby decreasing intensity of cases referred and length of time required to complete them.

Improvements to the CTA process resulted in a targeted, efficient approach. Initially, each CTA started with one or more meetings dedicated to educating participants about the guiding principles of the process and HCQU teams about the case. The amended process includes, prior to the initial meeting, distribution of an introductory informational packet to participants and a thorough chart review by the HCQU nurse to identify current and past diagnoses, medications, incidents, laboratory results, etc. and enable the HCQU team to formulate questions and suggestions to present at the initial meeting. Certain activities, such as trainings for caregivers and updates on the individual’s condition, now occur between meetings, which preserves meetings for activities requiring participant interaction, discussion, and decision. The addition of goals/outcomes to each section of the referral form and to the satisfaction survey helps teams maintain focus on the specific issues presented by the CTA referral. These process changes, along with increasing familiarity with and understanding of the process across the HCQU region over time, reduced the number of meetings required to complete a CTA and appear to have improved cost-effectiveness.

Although results from this study suggest that a planned, programmatic approach toward addressing difficult health and behavior issues among individuals with I/DD can have positive effects, development and measurement of actual health outcomes will be critically important in the coming years. Indeed, state-financed programs designed to improve quality of life of individuals with I/DD remain essential providers of support services, but impact of the changing healthcare environment and healthcare reform on individuals and the programs is unclear (Bachman etal. 2012). These state-funded programs, however, do provide a continuing opportunity to evaluate best practices for achieving desired results. Utilization of data from these programs is essential to continue to advance the health, safety, and quality of life of individuals with I/DD.