Introduction

Augmentative and alternative communication (AAC) refers to strategies used to support people who have complex communication needs characterised by difficulties with speech and/or language production in spoken or written modes [1]. They range from simple, paper-based systems consisting of single pages or books of words, phrases or pictures (known as ‘low-tech AAC’) to more complex electronic or computer-based systems (known as ‘high-tech AAC’). High-tech AAC can be used to produce synthesised speech of messages stored within them or entered into them by the person who uses AAC or their family, carer or AAC professional [1].

The population who use AAC is diverse and consists of people with multiple and complex physical and cognitive difficulties [2]. People who use AAC may have communication difficulties from birth associated with conditions such as cerebral palsy or acquire difficulties as an adult following a stroke, head injury or from a degenerative condition such as Parkinson’s [1]. Approximately one in 150 people in England (0.5% of the population) could benefit from using AAC [3].

Children and young people use AAC to support them to access education and enable them to build peer relationships [4]. Adults and older people use AAC to maintain relationships, occupation and to avoid social isolation [5]. However, AAC is a type of assistive technology (AT) and obsolescence and non-use of AT has been identified as a concern for a long time [6]. There is evidence to suggest that AAC is also at risk of being under-utilised or abandoned if, for example, people have limited access to support or training, devices are not maintained, or if there is poor fit between the AAC device and the individual using it [7]. Some researchers have identified a connection between the level of engagement of the AT end-user and the overall use of AT solutions [8, 9]. Understanding that an individual’s needs and priorities are at the heart of clinical assessment, can foster improved engagement in healthcare [10] and can lead to improved health outcomes [11].

Patient-reported outcome measures (PROMs) refer to tools that have been designed to provide information on the status of a patient’s health condition [12]. A PROM should measure a specific concept (or set of concepts), known as a conceptual framework, which has been developed with relevance to an intended population [12]. The use of PROMs is not considered credible unless there is evidence that it has been validated with the population of interest [12]. The purpose of PROMs is to get patients’ own assessment of their health or their health-related quality of life (typically concepts relating to emotional health and physical functioning) and therefore patients usually complete these directly [13]. PROMs are usually in the form of questionnaires and are typically used as evaluation tools. They can also be of value as part of the clinical interview and assessment [14]. For example, the information can be used by clinicians as a mechanism for engaging patients in decision-making about their healthcare at assessment but also during goal-setting, treatment planning and evaluation. Completion of a PROM might also contribute to helping patients to feel cared for, providing a framework for structuring patients’ discussions with their clinician [15].

People who have communication difficulties are inherently difficult to engage in traditional mechanisms for collaborative decision-making [16]. Difficulties in understanding spoken or written words as well as physical limitations often co-occur in conditions associated with the speech impairments necessitating AAC. These additional difficulties may make the completion of paper questionnaires or engaging in interviews challenging. People who require AAC may also have additional cognitive limitations or learning difficulties which require adapted materials and information methods [17]. Communication interactions may need to be navigated via multi-modal approaches, involving visual and pictorial support, facilitated by experienced and skilled communication partners. The nature of how information is attained (i.e. authorship) is critical to understanding the extent to which the person who uses AAC has truly been involved in providing it.

Improving collaboration, engagement and person-centredness in AAC service provision has the potential to improve AAC use and reduce the risk that use of the assistive technology is discontinued by better matching technology to the needs and expectations of the individual [18]. Yet people who have communication difficulties who may benefit from AAC and their families are rarely involved in decision-making relating to AAC [19]. There are no consistently used patient-reported outcome measures specifically for AAC [20]. Neither is there any consensus about what constitutes a successful outcome from AAC from the perspective of the person who uses on it [21]. The lack of appropriate support for, or engagement with, people who use AAC can cause frustration, disillusionment and finally abandonment of equipment [22]. Effective use of suitable PROMs by professionals working with people who use AAC has the potential to enable inclusivity by capturing important outcomes, providing targeted training and support, and evaluating success from the perspective of the people who use AAC.

This systematic review aims to identify:

  1. 1.

    What tools have been used to collect patient-reported outcomes in people who use AAC?

  2. 2.

    What are the conceptual frameworks, domains of interest and validity of the available tools?

  3. 3.

    What methods are employed to enable authorship (i.e. completion) of PROMs by people who use AAC?

Methods

A systematic review protocol to address the review question was developed and registered on PROSPERO. A list of search terms related to (a) AAC/AT, (b) PROM and (c) communication disorders was generated based on search terms used in reviews on similar populations [21]. The search strategy was deliberately broad initially, including terms relating to communication and AT, to ensure that all measures were captured (for a full copy of the search strategy, see PROSPERO: https://www.crd.york.ac.uk/prospero/display_record.php?RecordID=80567).

Searches

Databases searches were carried out using CINAHL (EBSCO), ERIC (ProQuest), MEDLINE (EBSCO), PsycINFO (ProQuest), and Scopus (Elsevier) from inception to January 2018.

Inclusion/exclusion

Studies were included based on meeting the following six criteria: (i) people who had communication difficulties; (ii) people aged 12 years old and above (and where data for this population can be disambiguated); (iii) people who had used an external aid to facilitate communication; (iv) the tools identified had been used to record outcomes from the perspective of the person using the aid (including but not exclusively: published scales and measures, questionnaires, software, descriptive outcomes, and author developed tools), (v) all study types (i.e., qualitative, quantitative, and mixed methods studies); and finally (vi) all contexts outside of acute and fixed-term rehabilitation hospitals. Papers not written in English, where English translations were not available were excluded for pragmatic reasons. Papers reporting participants as having severe intellectual disability [23] and participants who have significant cognitive impairment affecting reasoning and judgement were excluded as it was judged that they would be unable to complete a PROM. Participants with autistic spectrum disorders (ASD) or social communication difficulties were not included as clinicians within the review team decided that outcomes from AAC may be significantly different within this sub-population. Papers concerning participants who exclusively use gesture (including sign language), facial expression, or postures were not included as they did not include the use of an external aid to facilitate communication. Papers concerning assistive devices not commonly issued by AAC services were excluded such as: brain–computer interface; speech recognition technology; assistive devices for hearing or visual impairment. Papers reporting reviews, editorials, and opinion paper were not included as they were not reporting on primary data.

Screening

Titles and abstracts of all papers were screened for relevance by one author (KB), and 10% of the papers were independently checked by a second author (DH). There was a 5% inclusion/exclusion disagreement between the screeners which was resolved by discussion. Full-text papers were screened by the first author and 10% of these were checked by a second author (DH) with no disagreement. The PRISMA flow chart (Fig. 1) summarises the screening process.

Fig. 1
figure 1

PRISMA 2009 flow diagram

Full size image

Quality appraisal

Papers included in the review were appraised for quality using the Mixed Methods Appraisal Tool (MMAT) [24]. This quality appraisal tool was selected due to the range of study types included in this review. The MMAT enables reviewers to assess the quality of a paper on a 5-point scale from 0 to 4, depending on what information is available in the report. Quality appraisal was carried out by one author (KB) and checked for consistency by a second author (KS). These two authors compared and discussed independent scoring of the papers until consensus was agreed.

Data extraction

Data were extracted from the included papers and documented in a table designed for the purpose of this review. The table was piloted by two authors (KB, KS) to ensure that it captured the necessary information and to ensure a consistent approach was being used. Data were gathered pertaining to the study details, methods, participant demographics and characteristics, intervention, outcomes, the PROM used (including domains, conceptual framework and content validity as reported in the paper), and author reported strengths and limitations of the tools and study.

Data synthesis

A complementary approach to data analysis was adopted for this mixed-methods review [25]. A range of data pertaining to population characteristics, study type and PROM used, was extracted (Table 1) as well as descriptive data about PROMs; validity, authorship (Table 2) and concepts measured (Table 3). These data enabled analysis on the depth and breadth of information available as opposed to specifically the type or source of the data [25]. The use of complementary data sources (i.e. about the study and the PROM) enabled the authors to construct a narrative synthesis [26] of the range of PROMs used with people who use AAC. Once the PROMs had been identified and descriptive data about those PROMs had been extracted, the specific concepts (items measured by the PROMs) were tabulated (Table 3). One author (KB) reviewed the concepts and carried out a preliminary synthesis by grouping them into over-arching domains. These domains were then presented to a second author (KS), the strengths and limitations of the groupings were discussed, and a secondary synthesis was agreed and is presented as a final set of domains (see Table 4).

Table 1 Study characteristics
Full size table
Table 2 PROMs and significant descriptive data
Full size table
Table 3 Items/concepts measured by PROMs (as reported in the review papers)
Full size table
Table 4 Synthesis of PROM domains
Full size table

Results

A total of 5453 titles were identified by the search strategy after duplicates were removed. Of these 5453, 84 full text journals were read and screened (see PRISMA flow in Fig. 1 for details). The search resulted in 15 papers that met the inclusion/exclusion criteria for this review. These 15 papers were quality appraised using the MMAT tool. Of the 15 papers: five reports scored 0, four reports scored 1, three reports scored 2, and three reports scored 3. See Table 1 for a list of study characteristics and the results of the MMAT appraisal process.

Tools

Twenty-five instances of PROM use with people who use AAC were reported across these 15 studies, comprising 15 different published tools. Author-developed tools were used in four studies [27,28,29,30]. A summary of the characteristics of the PROMs identified by the review can be found in Table 2.

Ten of the 15 PROMs were used with participants who relied on either or both low-tech and high-tech communication aids. The McGill Quality of Life Questionnaire (MQoL), Communication Effectiveness Index—modified (CETI-m) [31, 32], Rosser classification (modified), Nottingham Health Profile (NHP) and the measure of goal fulfilment [33] were only used with people who rely exclusively on high-tech communication.

Content

Of the 15 PROMs identified, reports documented the conceptual frameworks for 12 and consequently, 62 items were identified that the tools measured. For example, the SF-36 consists of items pertaining to eight health concepts: physical functioning, bodily pain, role limitations due to physical health problems, role limitations due to personal or emotional problems, emotional well-being, social functioning, energy/fatigue, and general health perceptions [34]. Following synthesis of all the items measured by each tool, nine conceptual domains were generated: communication, cognition, role, health, quality of life, physical, social, spirituality, and AAC equipment. Within each of these domains, several sub-domains were created which represent areas of nuance or distinction within the main domains. For example, within ‘health’, there are four sub-domains: general, mental, emotional well-being, and sleep. In total, 20 individual domains or sub-domains were synthesised.

Validity

Authors were rarely explicit about their rationale for selecting particular PROMs. For articles that reported on their rationale, PROMs were used to capture different aspects of AAC technical function and the purposes for which AAC was used. Some papers were evaluating a specific intervention [28, 29, 35], whereas others were reporting longer-term outcomes for AAC [21, 27, 36].

The content validity, as reported in the review papers, was extracted but there were no reported instances of tools that have been psychometrically evaluated specifically in relation to people who use AAC. Some authors did however report on their rationale for amending certain PROMs. Londral el al. [31] reported that the MQoL has been used in various studies with people who have Amyotrophic Lateral Sclerosis (ALS) and that the CETI, a measure for evaluating functional communication in people with aphasia, has been demonstrated to be sensitive to changes over time (in a population of people who have aphasia). However, a modified version of the CETI was used in this particular study (to account for its use with a different population) but there was no information about whether the modification was also sensitive to change. The SF-36 has been used widely in health research and with a range of different conditions but was, again, modified in the study that reported on the validity of the tool [34], with no comment about the nature nor the validity of the modification. The Quality of Life Profile- Physical Disabilities (QOLP-PD) and the communication questionnaire were both used in the study by Hamm and Mirenda [36] and the report cites a previous study which had evaluated efficacy of both tools with a similar population, but the sample size of this study was small (nine) so validity cannot be ascertained.

Authorship

Authorship (i.e. who completed the PROM) was extracted, capturing how data gathering was adapted to accommodate any physical, cognitive or communication difficulties. There were five examples where the PROMs used were reportedly completed by the participant directly (Quebec User Evaluation of Satisfaction (QUEST) and Psychosocial Impact of Assistive Devices (PIADs) [37]; SF-36 [36, 38]; QOLP-PD and International Classification of Functioning (ICP) levels [21]); six examples where tools were completed by the researcher (author developed tools [28,29,30]; CETI-M [32]; ALS-Qol-r and Taiwanese Depression Questionnaire (TDQ) [39]) and five examples of tools being completed either in collaboration with or by a proxy (QOLP-PD, communication questionnaire [36]; Rosser classification, NHP, measure of goal fulfilment [33]). Six of the studies did not explicitly state who completed the tools used to collect PROMs (QUEST [35]; QUEST and PIADs [40]; author developed tool [27]; MQoL and CETI-M [31]).

Discussion

This review confirmed that there are no PROMs specifically developed and evaluated for capturing outcomes from people who use AAC. Of the PROMs that have been used with people who use AAC, a range of concepts were measured but there was little or no evidence that any of the tools used had been validated for people who use AAC. PROMs were not consistently completed by the study participants and there was scant explanation about any adaptations that had been made to enable participants to engage directly with the tools. The strength of the findings of the research studies in this review is limited by the often-poor quality of reporting. Nevertheless, some insights about the tools used, the content of these tools and the authorship of PROMs in people who use AAC have been identified.

Tools

The adaptations to existing, validated PROMs and the use of author-developed tools in studies identified by this review indicate that, despite the lack of suitable tools for use with people who use AAC, attempts are being made to capture patient-reported outcome data. A review by Enderby [20] also found that there was no single clinician-reported outcome measure that was consistently used by AAC services within the UK, but that a range of tools had been adopted and adapted for use with this population. The adaptation of tools in both clinical settings and research studies could indicate that, despite existing tools being insufficient or unavailable, clinicians and researchers are motivated to capture patient-reported outcome data. The current strategic drivers in healthcare in England are focused on empowering patients to become more involved in healthcare decision making [11]. These strategies are likely to lead to an increased demand on health services to capture the impact of greater patient involvement, using tools such as PROMs. Increased interest from clinicians, researchers and policy makers in patient involvement will drive the demand for psychometrically robust PROMs. Greater use of PROMs would increase the involvement of the person who uses AAC in decision making during assessment and also improve the viability of PROMs in the metrics for evaluation.

Content

One of the critical measures of adequacy of PROMs is the conceptual framework and the description of the relationships between the items (or domains) and concepts being measured [12]. The papers included in this review reported a range of tools that captured a large number of different concepts. The heterogeneity of the PROMs used to capture various aspects of AAC function and use within this review exemplifies one of the challenges of PROMs and AAC—identifying the nature of preferred outcomes. There is a lack of clarity about what outcomes are important to people who use AAC [21] and a range of factors that can affect outcomes. Are positive outcomes in people who use AAC concerned with the function or use of the AAC devices specifically? Or are outcomes concerned with the impact that using AAC has, e.g. on communication, relationships, taking part in activities or independence? Several papers used more than one PROM. One possible advantage of using a range of measures to record PROMs is the opportunity to capture the range of outcomes. This may help researchers to see the “big picture” and illuminate reports about people who use AAC by highlighting ‘that AAC in and of itself is not an end goal… [but] can be used as a tool to achieve other goals’ [21, p. 295]. It is also important to acknowledge that a range of factors can affect outcomes in people who rely on AAC including the patient’s milieu, personality, cognitive skills and the technology of the device itself [38]. A clearer understanding about what constitutes important outcomes from the people who use AAC and in what context, is needed in order to evaluate whether the tools used in research reflect the priorities of the end users.

Authorship

The poor reporting of authorship (i.e., who completed the PROM) in this review disguises some of the challenges experienced by people who use AAC and therefore may limit validity of the results. Where studies reported that the researcher completed PROM questionnaires, there was no reflection on how this may have influenced the results. In studies where participants had completed PROMs, there was little detail about how this was made possible. It is important to note here that the studies that paid attention to reporting, in detail, the collaborative nature of the PROM authorship were rated as higher quality papers overall on the MMAT tool. Hamm and Mirenda [36], for example, provide details about the extent to which the participants were able to complete the questionnaires independently and discussed the limitations inherent in asking questions of people who have limited functional speech. With a research population that cannot engage easily in traditional methods of data collection, the trustworthiness of the study is strongly correlated with how well adjustments and modifications to methods are reported.

A note about Talking Mats™

It is of note that there are no papers reporting the use of Talking Mats™ as a patient-reported outcome measure included in this review. Talking Mats™ is a collaboratively produced, picture or text-based tool that has been used to gather opinions and feedback from people who have communication difficulties in both research and service settings [42]. A report using Talking Mats™ was identified by the initial search terms [41] but was excluded from the review during full-text screening process. This was because data relevant to the review population could not be disambiguated. Talking Mats™ has the unusual quality of being used as an AAC device, a research tool and an outcome measure in a range of different studies. It is not a PROM per se as the content is not fixed and therefore not based on a conceptual framework [12], nor has it been psychometrically evaluated with a specific population. It does, however, meet the requirements of a large proportion of people who use AAC in ways that the PROMs reported in this review do not. It is a flexible tool that allows for the use of text or pictures. It is available in paper and digital forms and can be completed by the person who uses AAC, in collaboration with a communication partner or with a trained facilitator who can work with an individual who is employing multiple-modalities to communicate [42]. In the absence of a suitable PROM for people who use AAC, a tool that incorporated some of the flexibility and accessibility of Talking Mats™ could be a useful resource for professionals and services supporting people who use AAC.

Limitations

The overall quality of the reports identified by this review was relatively low which had an impact on the amount of data that could be extracted. Information about authorship and the conceptual frameworks for PROMs was variably reported, so the discussion of these areas is based on the small amount of data that it was possible to extract. In several of the papers excluded during screening, the research team were unable to disambiguate data about populations or age groups of interest from the reports available. As a result, there was insufficient data available to carry out a sub-group analysis of people 12 to 18 years old as was planned in the original review protocol. The search strategy excluded people who had severe intellectual disability, autism or significant cognitive impairment as there is an additional layer of complexity in using PROMs with these populations which is beyond the scope of this review. These populations do constitute a significant number of people who use AAC however and there would be value in exploring the PROMs employed with these groups in a separate review. The search strategy for this review was large, including terms relating to communication impairments and assistive technology, in order to capture PROMs in populations with similar difficulties that may be of interest or value in the field of AAC. Following title and abstract screening, too many of these reports were identified to be included in this review (see Prisma diagram). The review team decided that this data may instead be useful in a complementary review to be analysed at another point in time.

Conclusion

This review has identified that there is no single patient-reported outcome measure suitable for use with people who use AAC. No tool was identified by this review that has a conceptual framework specifically for AAC, and of the tools that have been used, there was limited evidence that the necessary adaptations were made to accommodate the multi-modal nature of communication in people who use AAC. Clinical services that provide AAC and support people to use AAC cannot therefore consistently capture outcome data from the patient-perspective. Future investigation into whether or not the domains generated during this review, from the tools that have been used with people who use AAC, reflect outcomes that are important to this population will be necessary.

PROMs can be valuable tools to aid understanding of the impact of a condition, treatment or intervention from the patient’s perspective. They can also improve patient-provider communication by facilitating discussion, supporting decision-making and clarifying shared aims for treatment. When selecting a PROM, the conceptual frameworks of the tool need to adequately reflect the priorities of the population of interest. There is currently no consensus about the important outcomes of AAC from the perspective of the people who use it. The population of people who use AAC have a particular set of needs and requirements to enable their inclusion in clinical decision making in healthcare and in research. The multi-modal and collaborative nature of communication by and with people who use AAC should be accommodated by developing PROMs that adequately reflect the needs and priorities of this population. Acknowledging the adaptations that have been made during research studies to accommodate people with additional needs, such as accurately describing authorship in people with communication difficulties, is essential if the results are to be considered authentic and trustworthy.