The Development and Evaluation of an Online Healthcare Toolkit for Autistic Adults and their Primary Care Providers

ABSTRACT

Background

The healthcare system is ill-equipped to meet the needs of adults on the autism spectrum.

Objective

Our goal was to use a community-based participatory research (CBPR) approach to develop and evaluate tools to facilitate the primary healthcare of autistic adults.

Design

Toolkit development included cognitive interviewing and test–retest reliability studies. Evaluation consisted of a mixed-methods, single-arm pre/post-intervention comparison.

Participants

A total of 259 autistic adults and 51 primary care providers (PCPs) residing in the United States.

Interventions

The AASPIRE Healthcare toolkit includes the Autism Healthcare Accommodations Tool (AHAT)—a tool that allows patients to create a personalized accommodations report for their PCP—and general healthcare- and autism-related information, worksheets, checklists, and resources for patients and healthcare providers.

Main Measures

Satisfaction with patient–provider communication, healthcare self-efficacy, barriers to healthcare, and satisfaction with the toolkit’s usability and utility; responses to open-ended questions.

Key Results

Preliminary testing of the AHAT demonstrated strong content validity and adequate test–retest stability. Almost all patient participants (>94 %) felt that the AHAT and the toolkit were easy to use, important, and useful. In pre/post-intervention comparisons, the mean number of barriers decreased (from 4.07 to 2.82, p < 0.0001), healthcare self-efficacy increased (from 37.9 to 39.4, p = 0.02), and satisfaction with PCP communication improved (from 30.9 to 32.6, p = 0.03). Patients stated that the toolkit helped clarify their needs, enabled them to self-advocate and prepare for visits more effectively, and positively influenced provider behavior. Most of the PCPs surveyed read the AHAT (97 %), rated it as moderately or very useful (82 %), and would recommend it to other patients (87 %).

Conclusions

The CBPR process resulted in a reliable healthcare accommodation tool and a highly accessible healthcare toolkit. Patients and providers indicated that the tools positively impacted healthcare interactions. The toolkit has the potential to reduce barriers to healthcare and improve healthcare self-efficacy and patient–provider communication.

BACKGROUND

Autism is increasingly being recognized and diagnosed, with current estimates that 1 in 68 children are autistic.1 Though often considered a childhood condition, autism persists throughout the lifespan, with no difference in prevalence by age and likely no change in the true prevalence of autism over time.2,3 Due to changes in diagnostic criteria and their application, many autistic individuals are receiving a first diagnosis as adults.4 Furthermore, the large cohort of children diagnosed in the last two decades is now approaching, or has entered, adulthood. Yet resources and services for autistic adults remain extremely scarce.5

This scarcity is particularly apparent in healthcare. Although most autistic adults may not be seeking healthcare to address or “treat” autism, they still require care for non-autism-related health issues and co-occurring conditions, such as seizures, anxiety, sleep disturbances, and gastrointestinal disorders.6–8 Autistic adults face many barriers to receiving healthcare. A majority of primary care providers (PCPs) lack the training necessary to care for autistic adults.9 Moreover, autism is characterized by atypical communication and interpersonal relationships—factors that are critically important for effective healthcare interactions. Our prior survey found that autistic adults had a greater number of unmet health needs, lower use of preventive services, higher use of the emergency department, and lower ratings of patient–provider communication and healthcare self-efficacy than did non-autistic adults.10 Autistic adults also reported a greater overall number of barriers to healthcare and numerous autism-specific barriers to care.11 In our prior qualitative study, autistic patients and their supporters described both positive and problematic healthcare interactions, illuminating a complex interplay between an individual’s autistic characteristics, the healthcare provider’s knowledge and attitudes about autism, and the healthcare system.12 For example, autistic patients described healthcare providers making incorrect assumptions about their abilities and needs, being unwilling to accommodate written communication, or using inaccessible language. Interactions were improved when supporters were appropriately incorporated, healthcare facilities were accessible, and clinicians showed a willingness to provide other accommodations.

Our objective was to use a community-based participatory research (CBPR) approach to create and evaluate an online healthcare toolkit for autistic adults and their PCPs. This paper describes a series of three studies used to develop and evaluate the toolkit: a cognitive interview study to assess the content validity of the Autism Healthcare Accommodations Tool (AHAT) survey and reports; a 2-week test–retest reliability study to assess the AHAT’s stability over time; and a pre/post-intervention study evaluating use of the full healthcare toolkit among autistic patients and their primary care providers.

METHODS

Community–Academic Partnership

The Academic Autism Spectrum Partnership in Research and Education (AASPIRE; http://aaspire.org/) is an ongoing academic–community partnership comprising academic researchers, autistic adults, family members, and healthcare and disability services providers. We use a CBPR approach, whereby academic and community partners serve as equal partners throughout each project. In this study, partners were involved with the development of the research question, the design of the study, the creation of protocols and research materials, the development of the toolkit, the analysis and interpretation of data, and the writing of this manuscript.13,14 The project was approved by the institutional review boards at each affiliated university.

Participants and Recruitment

For all three studies, participants needed to be 18 years old or older, reside in the United States, and communicate in English. Additional eligibility criteria for autistic adults were a medical diagnosis on the autism spectrum (autistic disorder, Asperger’s, pervasive developmental disorder not otherwise specified, or autism spectrum disorder) and, in the toolkit evaluation study, a designated PCP. We encouraged autistic adults to participate directly, with or without help from a supporter. In cases where the autistic adult could not participate directly, even with accommodations and supports, we asked a supporter who had experience supporting the autistic adult in healthcare settings to participate on their behalf. For the cognitive interview study, PCPs' eligibility included a current primary care practice with adult patients. In the toolkit evaluation study, PCPs were included only if their patient participated in the study. We recruited participants via fliers, postings, and announcements to autism and disability-related organizations, email distribution lists, recruitment databases, and forums. Potential participants completed a brief screening questionnaire online or via telephone to assess eligibility.

Toolkit Content Development

The content of the toolkit was informed by a prior series of AASPIRE studies, including 1) a survey comparing the healthcare experiences10 and barriers to care11 of 209 autistic and 228 non-autistic adults, 2) a qualitative study of the healthcare experiences of 39 autistic adults and 16 supporters,12 and 3) a brief survey about the autism-related practices and training needs of 129 PCPs for adults, and qualitative interviews with 9 PCPs about their experiences in providing care to autistic adults (unpublished data). These data helped us identify potential leverage points that could be targeted with our intervention and informed the types of tools and resources to include. Table 1 depicts some of the concrete ways those findings informed the toolkit.

Table 1 Examples of How Findings from Prior Studies Informed Toolkit

Our team of academic and community partners jointly created and edited materials to ensure their relevance, utility, and accessibility. In general, we found that our autistic participants and team members desired a high degree of detail and examples, especially on topics related to navigating the healthcare system. The resulting AASPIRE Healthcare Toolkit has a section for patients and supporters and another for healthcare providers. It includes general healthcare and autism-related information, checklists, worksheets, and other resources. Figure 1 shows sample toolkit contents; the full toolkit is available at http://autismandhealth.org .

Figure 1

Examples of toolkit contents.

Recognizing the substantial heterogeneity of autistic individuals, our research participants and team members felt that communicating personalized information about each individual patient was essential.12 The toolkit’s centerpiece is thus the Autism Healthcare Accommodations Tool (AHAT), which allows patients to create a personalized accommodations report for their healthcare provider. A patient or his/her supporter completes the AHAT survey to automatically generate a customized cover letter and AHAT report for the provider.

We used the recommendations elicited in our qualitative studies and the lived and professional experience of the community and academic partners on our team to create the AHAT survey items. We created two versions of the survey, one for autistic adults and one for supporters. The AHAT survey includes five multiple-choice items about the patient’s spoken and written communication abilities and use of alternatives to speech, and 12 items about areas where autistic adults may need strategies and accommodations to facilitate care, or where providers may need to be aware of autism-related characteristics. Each of these items uses a check-all response format and lists 5 to 14 potential accommodations, strategies, or characteristics, an option indicating that no accommodations are needed, and one indicating that accommodations are needed but not listed. There are also open-ended items related to patients’ strengths, special interests, and strategies for recognizing and addressing anxiety in healthcare settings and a section eliciting information about supporters. Table 2 further describes the AHAT. The final version of the full AHAT survey and samples of AHAT reports are provided in Appendix A.

Table 2 Autism Healthcare Survey and Report Contents

Preliminary Testing and Refinement of the AHAT

We assessed content validity of the AHAT survey and reports by conducting cognitive interviews with autistic adults, supporters, and PCPs. We also informally reviewed the AHAT reports with several local groups of PCPs and autism experts. Participants could choose to take part in the cognitive interviews in person or via telephone, email, or text-based chat. We used purposeful sampling to ensure that we included autistic participants with a broad range of disability characteristics. The interviewer showed autistic participants or supporters the AHAT survey items, and showed providers a sample cover letter and sample reports from three hypothetical patients. The interviewer used a variety of qualitative techniques to assess understanding, relevance, and utility, including asking participants to paraphrase items or response options, asking them to describe what potential accommodations might look like in their own situation, and eliciting ideas for additional strategies not already included in the AHAT. The interviewer took notes on the responses and entered them into a matrix. We used an iterative process, periodically reviewing participant responses to the cognitive interview questions, revising the surveys, reports, and letters based upon feedback, and showing later participants the most current versions. We also went back to some of the PCP participants to assess whether our changes adequately addressed their concerns.

We then conducted a 2-week test–retest reliability study with autistic participants and supporters to test the stability of the tool over time. Taking each response option for the check-all items as a separate dichotomous variable, the original AHAT included 132 variables. For each of the 132 variables, we calculated the percentage of time given by participants to the same response on the two versions of the survey, as well as a kappa and a phi statistic for each.

Toolkit Evaluation

Study Design

We evaluated the full toolkit in a real-life setting using a mixed-methods, single arm pre/post-intervention study design. After completing a baseline survey, autistic participants (or their supporters) used the AHAT tool to create a personalized report and decided whether to have it sent to their PCP. They then gained access to the remainder of the online toolkit. One month after using the toolkit, participants completed a post-intervention survey. In cases where participants asked us to send the AHAT report to their provider, we surveyed PCPs approximately 1 month after the intervention to assess whether they found the report useful.

Data Collection

We conducted surveys with autistic participants using an online audio computer-assisted survey interview (ACASI) system that has been found to be highly accessible to people with developmental disabilities.15,16 Surveys had two versions, one for autistic adults participating directly and one for supporters. The supporter version asked questions about the autistic person in the third person and instructed the supporter, where applicable, to either answer on behalf of the autistic person or offer their own opinion. The surveys are provided in Appendix B.

We assessed healthcare self-efficacy using a 21-item scale created de novo by our community and academic partners, with special attention to the self-efficacy issues previously identified in our qualitative work.12 Items addressed aspects related to healthcare navigation, successful interactions with providers, and disease self-management. Response options used a four-point Likert scale with anchors of “0 – Not at all confident” to “3 – Totally confident.” We scored self-efficacy by adding responses from the 21 items, resulting in a possible range of 0 to 63, with higher scores corresponding to higher self-efficacy.

We assessed 16 barriers to healthcare using a checklist that we had previously developed11 for use with autistic individuals. The supporter version of the survey included a few slightly modified items to differentiate between barriers faced by the autistic individuals and those faced by the supporters. We compared the total number of barriers endorsed by participants in the pre- and post-intervention surveys.

We collected data about patient–provider communication using an eight-item scale we had previously adapted10 from the 2007 Health Information National Trends Survey (HINTS).17–21 We did not assess patient–provider communication for those who were participating via a supporter, because we did not feel that a supporter could adequately rate the patient's satisfaction with communication. Autistic participants were asked these questions again in the post-intervention survey only if they had seen their PCP since using the healthcare toolkit. Responses used a five-point Likert scale with anchors of “1 – Strongly disagree” to “5 – Strongly agree.” We analyzed items by summing the responses to yield a composite score from 8 to 40, with higher scores indicating higher satisfaction.

The pre-intervention survey also included information about demographic and disability characteristics. The post-intervention survey included seven multiple-choice items and nine open-ended items assessing the autistic patient’s or supporter’s impression of the toolkit’s usability and utility.

In cases where participants asked us to send their AHAT report to their PCPs and provided enough information for us to locate the PCP, we mailed the PCP an information sheet about the study, a cover letter, and the AHAT report. A month later, we attempted to contact the PCP to recruit them to the study. Due to an initial low response rate, we shortened the PCP survey to fit on a single page. Likert-scale and open-ended items assessed the PCP's impression of the AHAT report and the toolkit. PCPs could participate in the survey via telephone, fax, email, or Internet.

Data Analysis

We calculated summary statistics for all measures. We calculated Cronbach’s alphas for scored scales (patient–provider communication and healthcare self-efficacy). We could not do so for the barriers checklist or the AHAT survey because responses were dichotomous or in check-all format. We compared pre- and post-intervention outcomes using paired t tests. We conducted all quantitative analyses using Stata software (version 13.0; StataCorp LP, College Station, TX).

We conducted a thematic analysis22 of responses to open-ended survey items using an inductive approach, on a semantic level, and a constructivist paradigm. The two co-principal investigators (CN, DR) reviewed all responses using an electronic spreadsheet, collaboratively devised a coding schema, categorized responses, collapsed codes into common themes, and chose representative quotes.

RESULTS

Participant characteristics for all three studies are presented in Table 3. Approximately half of the participants were men, most were non-Hispanic white, and 59–70 % participated independently. Approximately half lived in their own home and half with family or in group homes. Approximately one-third of participants almost always required assistance in healthcare settings, a third required it frequently, and a third required it rarely or never. Most PCPs practiced internal medicine or family medicine.

Table 3 Participant Characteristics

Preliminary Testing and Refinement of the AHAT

Autistic participants consistently indicated that the AHAT was important and that they were able to paraphrase the items being tested and elaborate on how or why potential accommodations would or would not be useful to them. Most felt that all their necessary accommodations were included in the available options. Based on participant feedback, we made minor changes to the introduction and to the wording of a few items on the survey. All PCPs indicated that the content of the report was very helpful, but some PCPs looking at earlier versions thought it would be difficult to access the information. We revised the AHAT report format multiple times until we found one that maximized ease of use for PCPs.

In 2-week test–retest comparisons, participants answered AHAT items similarly 80 % of the time (autistic mean 0.803, SD 0.08; supporter mean 0.799, SD 0.08). We reviewed 21 items (16 %) where responses matched less than 70 % of the time or where the kappa or phi statistics were below 0.4. We decided to make minor modifications to simplify three items/response options, remove 11 response options, and keep the other seven response options.

Toolkit Evaluation Study

Almost all autistic participants and supporters felt that the AHAT and the toolkit were easy to use, important, and useful. Over 90 % said they would recommend the healthcare toolkit to a friend or their healthcare provider (see Table 4).

Table 4 Patient and Provider Impressions of the Toolkit 1 Month After Use

In pre/post-intervention comparisons, the total number of barriers encountered by patients decreased significantly (from a mean of 4.07 at baseline to 2.82 post-intervention; p < 0.001). Participants’ self-efficacy in navigating the healthcare system also increased (37.92 to 39.39, p = 0.02; see Table 5).

Table 5 Change in Outcomes Between Pre- and Post-Intervention Surveys

We identified several themes from patients’ open-ended answers that may help explain how the toolkit affected outcomes. First, participants described the toolkit as giving them a means to clarify and communicate their needs. For example, one participant wrote, “Filling out the survey helped me clarify some things of which I was only vaguely aware. It also helped put into words things I am unable to communicate because I cannot think of the right words.”

Similarly, many participants felt that the toolkit validated their experience and empowered them to self-advocate more effectively. For example, one wrote, “It was validating. Previously, I felt that some of the things I was doing, like bringing support with me, was a sign of weakness. Now, I view it as part of accommodation. It also gave me some ideas of things to try that I hadn't thought of.” Another explained, “I used my iPad to communicate with [provider] at the beginning of the visit…. It is a fairly new provider for me so it was helpful to establish the accommodations I need. It gave me the confidence to make the changes I need to communicate my issues effectively.”

Participants also often gave examples of how the toolkit improved their self-efficacy, especially by helping them prepare for visits. One participant wrote, “It takes away a lot of my uncertainty about the appointments. Whether I'll bring up everything I want to bring up, whether I asked the right questions about follow-up care, and being prepared for talking to new doctors. It's a game changer for me.”

Most participants were enthusiastic about how the AHAT report might affect their PCPs' behavior. For example, one wrote, “Y'all sent things to the doctor so MAYBE THIS TIME he'll listen to me.” However, some were concerned that providers would be unlikely to pay attention to the AHAT report (“Not confident that healthcare provider(s) will take it seriously.”). A minority of participants voiced concerns that their PCP would have a negative response. For example, one wrote, “When I asked my psychiatrist to not use air fresheners she said she had to or else her office smelled, and she acted like it was a really big burden on her. I think that if I handed your nice letter to my physicians … they would think I am asking too much of them. I already stand out; I don't want to stand out more.”

Sixty-five percent of participants gave permission for us to mail their AHAT report to their PCP. Thirty-five participants answered an open-ended item about why they chose not to have their report sent, indicating that they planned on taking the report to the PCP themselves (n = 8), were in the process of changing or did not have a PCP (n = 8), did not need it (n = 6), did not wish to disclose their ASD to their PCP (n = 4), had privacy concerns (n = 2), had technical problems seeing the report (due to a temporary problem with our server; n = 2), or didn’t like the report or worried that their PCP would not react well to it (n = 3). Comments on the post-intervention surveys indicated that several additional patients had taken their AHAT reports to their PCPs.

Among the 43 patients who saw their PCP within the 1-month follow-up period, satisfaction with PCP communication improved significantly (from 30.9 to 32.6, p = 0.03; Table 5). Comments from participants who had seen their PCPs in the intervening month described many concrete positive changes in providers or their staff:

“I brought a copy of the accommodation letter in case he had not received it. He had and it was already scanned into his computer. He went over it with me and did what had been recommended.… I was reassured by the doctor taking the accommodation letter seriously…. I felt like some of the difficulties I experience were addressed and that they wouldn't have been had I not made use of the Healthcare Toolkit.”

And a supporter described the effect the AHAT report had on her daughter’s care:

I think the Toolkit validated that my concerns and my daughter's "issues" (sensory, behavioral, etc.) which often presented during medical appointments were typical for ASD patients and should be accommodated.

Several participants voiced frustration that their providers did not read the AHAT report or did not make any changes based on the report (e.g. “It was disregarded :-("). However, patients gave no examples of PCPs responding negatively to the report. In addition, a few participants indicated that they already had a good relationship with their PCPs and thus did not need the AHAT. “The visit went fine but it seemed to me that it only brought to attention things that were already known as I have a very close relationship with my healthcare provider.” Participants offered numerous suggestions for minor changes to the toolkit or additional content that could be included. Many also requested that we create a version for other types of providers such as dentists, psychiatrists, and medical subspecialists.

We were able to send AHAT reports to 88 PCPs, 41 (47 %) of whom completed the survey. Most PCPs rated it as moderately or very useful and indicated that they would recommend it to their patients (Table 4). Most answers from PCPs to open-ended questions were positive and provided examples of the tool's utility (e.g. “Extremely helpful. What I needed were specific, but concise suggestions regarding how to make my patient more comfortable. The report will be in her chart and I will use it at each visit.”) However, several PCPs noted that they already were doing what was recommended in the report (e.g. “Reinforces what I had learned to be effective”), and two PCPs felt that they did not have time to implement accommodations (e.g. “In a busy primary care office the time demands are immense. This creates further time demands.”).

DISCUSSION

Autistic adults face many challenges when trying to access healthcare services, including locating disability-competent providers, understanding and navigating the health system, and obtaining the necessary accommodations for their communication, cognitive, emotional regulation, and sensory needs. We developed tools to help autistic people and their healthcare providers address these challenges. Our CBPR process resulted in a healthcare accommodation tool with good content validity and test–retest reliability, and a highly accessible and usable healthcare toolkit. Most patients and PCPs indicated that the tools were useful in facilitating care. Quantitative results indicate that the toolkit has the potential to reduce barriers to healthcare and improve healthcare self-efficacy and patient–provider communication. Qualitative results suggest that these improvements may have been driven by heightened self-awareness and enhanced self-advocacy, as well as changes in patient and provider behaviors.

To the best of our knowledge, this is the first intervention specifically developed to improve primary care services for autistic adults. Our findings suggest that the toolkit may act on several potential leverage points critical to improving healthcare. Many of the qualitative responses focused on the effect the toolkit had in helping participants name the challenges they experience with healthcare and understand them as issues commonly experienced by others on the spectrum. Importantly, for some participants, it also seems to have helped reframe these issues, to view them not as personal weaknesses, but as disability-related needs that can and should be accommodated. This type of patient empowerment appeared to be particularly helpful when combined with practical resources and tools that participants could use to prepare for visits or for following recommendations. A strong body of literature documents the importance of patient activation and empowerment.23,24 Our study points to a potential means of increasing empowerment and activation in autistic patients.

But patient empowerment must be coupled with changes in provider attitudes, skills, and behaviors. We designed the toolkit using a model in which PCPs seek information about a specific autistic patient on a “need-to-know" basis. Evaluation data support the notion that many PCPs found the AHAT reports useful. Though it is likely that an accommodation report may not be not sufficient to eliminate all constraints affecting PCPs’ ability to care for their autistic patients, our hope is that the AASPIRE Healthcare Toolkit will be a first step toward the larger training and system changes necessary to adequately address the healthcare needs of autistic adults.

Our project has several limitations. The focus was primarily on developing the toolkit and conducting a preliminary evaluation of its feasibility, acceptability, potential effects, and mechanisms of action. The evaluation study was not designed to rigorously test the effectiveness of the toolkit in changing health or healthcare outcomes. Our use of non-systematic recruitment methods could have biased the sample toward participants who were more open to this type of intervention. We did not seek to independently confirm ASD diagnoses, but felt that the potential inclusion of people who were misdiagnosed with ASD was consistent with the real-world primary care setting in which the toolkit will be used. As with many studies of PCPs, we had a modest response rate; it is possible that PCPs who did not find the toolkit useful were less likely to respond.

Despite these limitations, our project has several important implications: 1) Individual patients and clinicians may benefit from using the AASPIRE Healthcare Toolkit. Though targeted to autistic adults in primary care settings, in our clinical experience, several of the resources and tools have also been helpful in facilitating care in hospital and specialty care settings and with patients with other developmental disabilities or low healthcare literacy. Similarly, though tested exclusively with adults, such tools may aid in transitioning autistic youth to the adult healthcare system. 2) Providers should make every effort to fulfill their legal obligation to provide accommodations to autistic patients, and should support patients’ efforts at self-advocacy, self-efficacy, and self-management. 3) Healthcare systems should try to find ways to incorporate such tools into their clinical protocols, training initiatives, and electronic medical records. 4) The CBPR process was a key element in developing relevant, useful, and accessible tools. Other groups attempting to develop and test interventions for marginalized or difficult-to-reach populations should consider using a CBPR approach. 5) Further research is needed to rigorously test the effectiveness of such interventions in changing health or healthcare outcomes.