Background

In 2019, it was estimated that one in every fourteen people aged 65 years and over were living with dementia in the United Kingdom (UK). In the UK, around 61% of older people with dementia live in their own homes irrespective of dementia severity [1]. Notably, at the early stages of dementia, most people prefer to continue to live at home [2]. Hence, it is important to understand how best to support people to stay at home and maintain their independence, links with local community, and wellbeing. As dementia progresses, many people find it increasingly difficult to look after themselves and their homes and may need help with their daily activities [3, 4]. The majority of this help is provided by family members, close friends or neighbours (also referred to as unpaid [5], informal [6] or family carers [7]). However, community-based social care services, including home care and day activities, are also important sources of support [8].

The Adult Social Care Outcomes Toolkit (ASCOT) is a suite of self-report [9,10,11], interview [12], easy read [13] or mixed methods [14] measures designed to measure social-care related quality of life (SCRQoL) of service users and their carers [15]. The measures are preference-based measures [10, 16] that may be used in economic evaluation of social care services (also known as long-term care) [17, 18], interventions or policy. However, it is difficult to collect information using self-report questionnaires from people who have memory or communication difficulties, including people with moderate to severe dementia [14, 19]. To work around this, an adapted version of the ASCOT self-completion version (SCT4), the ASCOT-Proxy has been developed for completion by someone who knows the person well, such as, a close friend or relative [20,21,22]. The ASCOT-Proxy uses the same eight domains (items) of SCRQoL identified in the original development work and the final version of the ASCOT-SCT4 [10]. However, the item wording, format and response options were adapted to improve the acceptability of the measure to proxy respondents (care workers or unpaid carers) [20, 21]. Based on the development studies of the ASCOT-Proxy [20,21,22] and informed by Pickard and Knight’s [23] conceptual framework of proxy-response, the ASCOT-Proxy includes two proxy perspectives: proxy-proxy perspective (‘My opinion: What I think’) and proxy-person perspective (‘What I think the person I represent thinks’) in the response options.

There is also another version of the questionnaire called the ASCOT-Carer, which looks at aspects of life that are important to friends and relatives who look after someone with social care support needs. Previous studies have established that the ASCOT-Carer is a valid and reliable measure of SCRQoL among carers in England [15]. However, the study in England only included a relatively small number of carers of people with dementia, so separate subgroup analysis was not possible [15]. As such, this study will address the evidence gap with regard to the measurement properties of the ASCOT-Carer with this subgroup of carers [24, 25].

The aim of the present study was to establish the feasibility, reliability and construct validity of the ASCOT-Proxy and ASCOT-Carer. In addition, we aimed to compare structural characteristics of the ASCOT-Proxy against the original ASCOT-SCT4 (self-completion form) [10]. The structural characteristics of the ASCOT-Carer using the same data as this study is reported elsewhere [26]).

Methods

Participants and setting

Data analysed in this paper were collected as part of the ‘Measuring Outcomes of People with Dementia and their carers’ (MOPED) study. Data were collected using a self-administered questionnaire from unpaid carers in England between 27th January 2020 and 30th April 2021. Participants could choose between a postal or online version (using Qualtrics XM Platform™). The online version asked participants if they wished to leave the question blank, but completing items was not mandatory.

Carers were eligible to take part in the MOPED study if they provided unpaid support to a relative, partner/spouse or friend with dementia, living at home (not in residential or nursing care), who was unable to self-complete a structured questionnaire, even with help. The person living with dementia or their unpaid carer had to use at least one community-based social care service at the time of recruitment to the MOPED study. This was adapted from ‘at the time of recruitment’ to ‘before the start of the COVID-19 pandemic’, since some people temporarily stopped receiving social care support or had reduced or adapted support as a result of the legal restrictions on social interaction due to the pandemic [27]. These services could be funded via the local authority, voluntary (third) sector organisations, or paid for by the person with dementia or their carer. Social care was defined as any type of community-based social care support, including equipment and homes adaptations, information and advice, home care, day activities, or support from carers’ organisations. We included people irrespective of type and frequency of the social care community support received, as in previous research [10, 11]. Carers had to be able to provide informed consent and respond to a self-completion format questionnaire in English.

We recruited participants via Join Dementia Research (JDR), an online panel of volunteers interested in dementia research including people living with dementia, unpaid carers and healthy participants. In addition to JDR, we recruited participants via 25 National Health Service (NHS) research sites.

Measures

The ASCOT-Proxy [21] is an adapted version of the ASCOT-SCT4 [10], designed to be completed by proxy-report rather than self-report. The eight SCRQoL domains (items) of the ASCOT-SCT4 are also included in the ASCOT-Proxy, with similar wording for the domains (items): food and drink, accommodation cleanliness and comfort, personal cleanliness and comfort, personal safety, social participation and involvement, occupation (doing things a person values and enjoys), control over daily life, and dignity. Each item has four response options that correspond to four different outcome states (ideal state (coded as 3 in the analysis), no unmet needs (coded as 2), some unmet needs (coded as 1), and high-level of unmet needs (coded as 0)). For example for food and drink, the response options are: ‘As clean and comfortable as s/he wants’; ‘Adequately clean and comfortable’; ‘Not quite clean or comfortable enough’, ‘Not at all clean or comfortable’. For occupation, the response options are: ‘Is able to spend his/her time as s/he wants, doing the things s/he values or enjoys’; ‘Is able to do enough of the things s/he values or enjoys with his/her time’; ‘Does some of the things s/he values or enjoys with his/her time, but not enough’; ‘Doesn’t do anything s/he values or enjoys with his/her time’.

The key difference between the ASCOT-Proxy and ASCOT-SCT4 is that the ASCOT-Proxy is designed for completion on behalf of services users by someone who knows the person well, typically a family member or care staff [20, 21], rather than self-report [10]. In the MOPED study, we collected the ASCOT-Proxy data from unpaid carers only, not care staff. The questionnaire is available from the ASCOT website (www.pssru.ac.uk/ascot) [28].

The full details on how we calculated the scores for the ASCOT-Proxy (both perspectives) is provided in the ASCOT-SCT4 Proxy guidance (pages 6-9, [29]). As the ASCOT is a preference-weighted measure of SCRQoL [10, 16] for use in social care research and economic evaluation [17, 18], we converted the raw scores per each item into the preference-weighted values, reflecting their relative importance/value to the general population [10]. We then added together the preference-weighted values and entered them into a formula (overall score = (0.203 x weighted score) – 0.466) to give the overall score [29]. The overall score ranges from -.17 to 1.00. Higher scores indicate better SCRQoL. The formula is based on a Time Trade Off exercise with members of the public [10]. Thus, while a score of 1.00 would mean that the person has reported the ideal state in all domains (items), a score of 0.00 is, in the view of the general population, valued the same as being dead. Scores, and the states that they represent, between -0.01 and -0.17 are seen as being ‘worse than death’. A negative score will be calculated, for example, if a person has high unmet needs in all domains (items) (e.g. no control over daily life, not at all clean or comfortable, socially isolated, not enough food and drink etc.). During the preference study, people said death would be preferable to living in this state.

The ASCOT-Carer-SCT4 is a measure of SCRQoL for unpaid carers, aged 18 years or over [15]. It includes seven domains (items), one each for the following SCRQoL attributes: feeling encouraged and supported, space and time to be yourself, self-care, personal safety, social participation and involvement, occupation (doing things a person values and enjoys) and control over daily life. Each domain (item) has four response options, like the other ASCOT measures, reflecting four different outcome states (ideal state (coded as 3 in analysis), no unmet needs (coded as 2), some level of unmet needs (coded as 1), and high-level of unmet needs (coded as 0)). For example, the response options for occupation are: ‘I’m able to spend my time as I want, doing things I value or enjoy’; ‘I’m able to do enough of the things I value or enjoy with my time’; ‘I do some of the things I value or enjoy with my time, but not enough’; ‘I don’t do anything I value or enjoy with my time’. For self-care the response options are: ‘I look after myself as well as I want’; ‘I look after myself well enough’; ‘Sometimes I can’t look after myself well enough’; ‘I feel I am neglecting myself’.

The questionnaire is also available from the ASCOT website (www.pssru.ac.uk/ascot) [30].

The full details on how we calculated the scores for the ASCOT-Carer is provided in the ASCOT-Carer-SCT4 guidance (pages 5-7 [31]). The ASCOT-Carer is a preference-weighted measure of carers’ SCRQoL. We converted the raw scores for each item into the preference-weighted value, based on best-worst scaling [16]. These reflect their relative importance/value to the general population. The overall SCRQoL score for carers is calculated by summing the preference-weighted values for each domain. The overall score ranges between 0 (high-level of unmet needs) and 1 (ideal state).

The full detail of measures used for establishing construct validity is reported in Additional File 1.

Statistical analysis

Analyses were conducted in Stata 16 [32] and in WINSTEPS software (version 3.92.1.). The full detail of the descriptive analysis, feasibility, structural characteristics, Rasch analysis, reliability/internal consistency, construct validity, and sample size calculation is reported in Additional File 2.

Results

Descriptive analysis

The flow of participants through the MOPED study is shown in Fig 1. Characteristics of participants (n=313 carers) are shown in Table 1.

Fig. 1
figure 1

Flow of the participants through the study.

Abbreviations: JDR (Join Dementia Research); NHS (the National Health Service). Notes: n/a (information was not available)

Full size image
Table 1 Sample Characteristics
Full size table

The distribution of responses as well as overall scores for the ASCOT-Proxy (both perspectives) and ASCOT-Carer are shown in Tables 2 and 3.

Table 2 Distribution of responses for the ASCOT Proxy: both perspectives (n=313)
Full size table
Table 3 Distribution of responses for the ASCOT Carer (n=313)
Full size table

Feasibility

Overall, the percentage of missing values was low for the ASCOT-Proxy (both perspectives) ranging from 0% to 1.92% (Table 2). The dignity item had a higher percentage of missing data (Proxy-proxy: ‘My opinion: What I think’: 6.71%; proxy-person: ‘What I think the person I represent thinks’: 7.03% of missing data). The overall ASCOT-Proxy-proxy score could be calculated for 90.7% of our sample while the overall ASCOT-Proxy-person score could be calculated for 88.8% of our sample. Immediate form of one-sample test of proportion confirmed that there was no significant difference (p= 0.258) in the proportion of missing data between the ASCOT-Proxy-proxy (29/313) and ASCOT-Proxy-person (35/313) overall scores. This indicates that both ASCOT-Proxy perspectives have similar feasibility for unpaid carers of people living with dementia.

We explored whether the feasibility of the ASCOT-Proxy perspectives differ based on mode of administration (paper version versus online) using the immediate form of two-sample test of proportions to compare the proportion of missing data for the ASCOT-Proxy scores by mode of administration. For the ASCOT-Proxy-proxy perspective, the proportion of missing data was significantly (p<0.001) higher for participants who used paper questionnaires (missing: 13/63) when compared to those using online version (missing: 16/250). For the ASCOT-Proxy-person perspective, the proportion of missing data was also significantly (p<0.001) higher for participants who used paper questionnaires (missing: 17/63) when compared to those using online version (missing: 18/250).

Regarding the ASCOT-Carer, all but one item (personal safety; 0.32% of missing data) had no missing data indicating a good feasibility (Table 3). We were able to calculate the ASCOT-Carer overall score for 99.7% of our sample. As only one observation was missing for the ASCOT-Carer overall score, we did not explore the role of administration mode in acceptability of the ASCOT-Carer.

Factor structure of ASCOT-Proxy perspectives

As the ASCOT-Proxy has Likert-type items which yield ordinal data, we undertook several steps when examining structural characteristics of the ASCOT-Proxy (both perspectives) as recommended by Gugiu et al. [33]. First, based on the results from Horn’s parallel analysis (principal component analysis as a factor estimation type, 5000 iterations, using 95th percentile for randomly generated eigenvalues, similarly as in a previous study [34]), we retained two factors for the ASCOT-Proxy-proxy (Fig. 2). To inform our decision regarding the number of factors to retain, we compared the observed principal component eigenvalues with the 95th percentile eigenvalues (random) from the simulated datasets. We retained those factors where observed eigenvalues exceeded the eigenvalues generated by random.

Fig. 2
figure 2

Parallel analysis of the ASCOT-Proxy-proxy.

Note: Plot of actual principal component eigenvalues versus randomly generated 95th percentile eigenvalues

Full size image

Then, we performed maximum likelihood exploratory factor analysis on the polychoric correlation matrix (ordinal exploratory factor analysis). In case of the ASCOT-Proxy-proxy we specified to retain two factors and applied oblique rotation (promax) to allow for the factors to be correlated. Next, we verified that the suggested solution from the parallel analysis, supported the stability and interpretability of the factors structure. We checked whether any items had low factor loadings (<.4) on all the factors, or salient loadings (>.5) on two factors [33]. Based on this (see Table 4) we retained two factor solution in case of the ASCOT-Proxy-proxy. These two factors can represent (1) basic domains (items) that relate to basic care needs/support to sustain life and health (food and drink, accommodation cleanliness and comfort, personal cleanliness and comfort, personal safety) and 2) higher order domains (items) that relate to aspects beyond basic care needs and/or relate to a person’s sense of self and identity (social participation and involvement, occupation, control over daily life) [10]. While we consider our findings from parallel analysis robust (the real eigenvalue line for two factors is above the 95th percentile eigenvalue for the simulated data), it is the first time when the results indicate that one of the ASCOT tools has different structure than other ASCOT tools. For example, the original ASCOT-SCT4 [10] has a weak unidimensional one-factor structure which was also confirmed for other adapted version, e.g. easy-read ASCOT [13] and the translation of ASCOT into Finnish [35].

Table 4 Rotated factor loadings and unique variances for the ASCOT-Proxy-proxy
Full size table

In case of the ASCOT-Proxy-person, the suggested solution from the parallel analysis was to retain one factor (Fig. 3). However, given the close proximity between the random eigenvalue line and the observed eigenvalues for two factors, the result from the parallel analysis suggests that in a different sample, one may potentially find a two factor solution. Therefore, we ran maximum likelihood exploratory factor analysis on the polychoric correlation matrix (ordinal exploratory factor analysis) and specified to retain one factor and then two factors (oblique rotation - promax). Next, we checked whether any items had low factor loadings (<.4) on all the factors, or salient loadings (>.5) on two factors [33]. As one factor solution better supported the stability and interpretability of the factor structure, we conclude that the ASCOT-Proxy-person has a weak unidimensional (one-factor) structure (Table 5).

Fig. 3
figure 3

Parallel analysis of the ASCOT-Proxy-person.

Note: Plot of actual principal component eigenvalues versus randomly generated 95th percentile eigenvalues

Full size image
Table 5 Factor loadings and unique variances for the ASCOT-Proxy-person: comparison of one factor solution vs two-factor solution
Full size table

Rasch analysis (the category probability curves; see below) suggested that the lowest two (high-level of unmet needs, some needs) and highest two categories (no needs, ideal state) should be combined for food and drink and for control over daily life in case of the ASCOT-Proxy-person. As a result of the Rasch findings, we reran the parallel analysis and exploratory factor analysis using the recoded data. The suggested solution from the parallel analysis was to retain two factors (Fig. 4). Therefore, we ran maximum likelihood exploratory factor analysis on the polychoric correlation matrix (ordinal exploratory factor analysis) and specified to retain two factors (oblique rotation - promax). Next, we checked whether any items had low factor loadings (<.4) on all the factors, or salient loadings (>.5) on two factors [33]. Based on the results (Table 5) two factor solution seemed not to be appropriate. Therefore, we reran the exploratory factor analysis and specified to retain one factor and explored the factor loadings of each domain (item). As one factor solution better supported the stability and interpretability of the factor structure, we conclude that the ASCOT-Proxy-person has a weak unidimensional (one-factor) structure (Table 5).

Fig. 4
figure 4

Parallel analysis of the ASCOT-Proxy-person with recoded values for food and drink and control over daily life.

Note: Plot of actual principal component eigenvalues versus randomly generated 95th percentile eigenvalues

Full size image

In addition, it is important to highlight that the dignity domain (item) had low factor loadings (<.4) in the ASCOT-Proxy (both perspectives) (Tables 4 and 5). This may indicate that the dignity domain (item) should be dropped from further analysis. However, as highlighted by Gugiu et al. [33], this may be not always the case, e.g. researchers may consider to keep the item if dropping it compromises the content validity of the instrument. The strongest evidence to support the decision whether dropping the item would compromise the content validity comes from qualitative concept elicitations studies. In the ASCOT-SCT4 [10] and therefore ASCOT-Proxy, dignity is defined as ‘the negative and positive psychological impact of support and care on the service user’s personal sense of significance’. It has been identified as an important aspect of SCRQoL based on reviewing previous literature about measures of SCRQoL, as well as a series of qualitative studies with service users [10]. As the ASCOT-Proxy is an adapted version of the ASCOT-SCT4, the dignity domain (item) was considered for inclusion in the development studies of the ASCOT-Proxy [20, 21]. In the early development study of the ASCOT-Proxy, the participants (n=35 unpaid carers, n=8 care workers) described having greater difficulty answering those ASCOT questions by proxy that related to abstract concepts (e.g. dignity, occupation, control over daily life) than concrete concepts (e.g. food and drink) [20]. This issue was then further explored with 25 unpaid carers and care workers in three iterative rounds of cognitive interviews [21]. Overall, 22/25 participants correctly interpreted the dignity item and 18/25 found the item (in the draft version presented) to be acceptable. The following concerns over acceptability were raised: care staff (as proxies) may not respond accurately, as it may reflect badly on them (n=1), it was difficult for the respondent to imagine the person’s perspective on dignity due to condition-specific considerations (e.g., perceived lack of self-awareness) (n=3), the person did not currently receive paid help (i.e. it was not applicable) (n=2) or another unspecified reason (n=1). These may be mitigated by focussing on unpaid carers as proxies and also limiting the sample to those receiving care, as in this study. Given the fact that dignity is an important aspect of SCRQoL as established in the ASCOT-SCT4 [10] and was found acceptable in the development study for the ASCOT-Proxy [21], in this study, we retained it in further analysis. However, it is important that researchers report structural characteristics of the ASCOT-Proxy in the future studies. If the present finding is replicated, it may be necessary to revisit whether dignity should be included as an important aspect of SCRQoL of social care users when the answers are provided by proxy (e.g. through qualitative concept studies with unpaid carers as proxy respondents).

As the results from factor structure analysis inform Rasch analysis, reliability and construct validity analysis, these were only conducted with the ASCOT-Proxy-person in this study. We decided not to proceed with these analysis for the ASCOT-Proxy-proxy as whether and how the ASCOT-Proxy-proxy tool should be used when having two factor solution have to be considered carefully.

The one factor structure of the ASCOT-Carer using the same data as this study has been established elsewhere and is not included in this paper [26].

Rasch analysis: Overall model fit

For the full description of Rasch analysis please see Additional File 2. We assessed model fit using the information-weighted mean square (INFIT MNSQ), outlier-sensitive mean square (OUTFIT MNSQ) statistics and point-measure correlation. Similarly as in another study [36], we considered values of INFIT and OUTFIT MNSQ statistics in the range of .5 to 1.5 as satisfactory [37]. The INFIT MNSQ and OUTFIT MNSQ statistics for the ASCOT-Proxy-person and ASCOT-Carer are shown in Table 6 and Table 7. All items were in the acceptable INFIT MNSQ and OUTFIT MNSQ range of .5 to 1.5 and consisted of positive item-Rasch measure correlations, except for the ASCOT-Carer personal safety (OUTFIT MNSQ=2.27). This indicates there may be an issue with the item, indicating that the item should be dropped. As this is the first study to use Rasch analysis when exploring psychometric properties of the ASCOT-Carer in unpaid carers, it is important that future studies using the ASCOT-Carer explore this further in their analysis.

Table 6 Item statistics including difficulty (in logits), infit and outfit mean square and point-measure correlations for the ASCOT-Proxy-person
Full size table
Table 7 Item statistics including difficulty (in logits), infit and outfit mean square and point-measure correlations for the ASCOT-Carer
Full size table

In this study, we kept the item in further analysis, as the item relates to an aspect of SCRQoL that has been identified as important to unpaid carers in previous research and is part of the ASCOT-Carer measure that has been widely used.

Rasch analysis: Rating scale

As in another study [36], we evaluated the functionality of the ASCOT-Proxy-person and ASCOT-Carer 4-point rating scales using criteria proposed by Linacre: 1. the occurrence of more than 10 endorsements per response category, 2. the observation that both average measures and category thresholds increase across each response category, and 3. an observed OUTFIT MNSQ value of less than 2 for each response category [38]. If we observed disordered thresholds or group means or a high OUTFIT MNSQ for a category that had less than 10 respondents, we dismissed the finding (as we did not have enough evidence that subjects were unable to distinguish between the adjacent response categories).

Following domains (items) had observed average disordered in the ASCOT-Proxy-person (Table 8): food and drink; person cleanliness and comfort; and personal safety. In case of the ASCOT-Carer (Table 9), following items had OUTFIT MNSQ higher than two: personal safety; and feeling supported and encouraged.

Table 8 Response category (rating scale) diagnostics based on Linacre’s criteria: the ASCOT-Proxy-person
Full size table
Table 9 Response category (rating scale) diagnostics based on Linacre’s criteria: the ASCOT-Carer
Full size table

Rasch analysis: Functionality of the response categories

We conducted Rasch analysis (the category probability curves) to examine whether unpaid carers are able to distinguish between the four responses options for each domain (item) (both the ASCOT-Proxy-person and the ASCOT-Carer). The category response curves indicated disordered thresholds for food and drink and control over daily life for the ASCOT-Proxy-person (Fig. 5). The lowest two (high-level of unmet needs, some needs) and highest two categories (no unmet needs, ideal state) were combined for food and drink. Similarly, the lowest two (high-level of unmet needs, some needs) categories and highest two categories (no unmet needs, ideal state) were combined for control over daily life (Fig. 6).

Fig. 5
figure 5

Category response curve for the ASCOT-Proxy-person.

Note: Red: high-level of unmet needs; Blue: some unmet needs; Purple: no unmet needs; Black: ideal state

Full size image
Fig. 6
figure 6

Category response curve for the ASCOT-Proxy-person: collapsed categories.

Note: Red: high-level of unmet needs; Blue: some unmet needs; Purple: no unmet needs; Black: ideal state. In case of food and drink and control over daily life: Red: high-level of unmet needs and some unmet needs; Blue: no unmet needs and ideal state

Full size image

There were no disordered category thresholds for the ASCOT-Carer (Fig. 7).

Fig. 7
figure 7

Category response curve for the ASCOT-Carer.

Note: Red: high-level of unmet needs; Blue: some unmet needs; Purple: no unmet needs; Black: ideal state

Full size image

Wright-Andrich maps: examination of floor and ceiling effects

Figures 8 and 9 present the Wright-Andrich Maps for the ASCOT-Proxy-person (collapsed categories for food and drink and control over daily life) and the ASCOT-Carer respectively. The Wright-Andrich Map plots the person measures along the left side and the item measures for each of the domains (items) along the right side. The M to the left of the centre line stands for the mean of the person logits. The M on the right represents the mean of the domain (item) logits. The S’s and T’s stand for 1 and 2 standard deviations from the means, respectively. From Fig. 8 (ASCOT-Proxy-person) we observed two significant gaps (approximately 1.5 logit) on the latent continuum. We see a lack of alignment between the person and item distributions, which is best characterised as the 1-point logit difference between the means of the items and persons. Similarly as in another study [36] floor and ceiling effects were defined as the existence of persons with logit scores at the bottom or top of the persons distribution (left side of the Wright-Andrich Map) that were at least one logit from the nearest item measure (right side of the Wright-Andrich Map). Specifically, we considered effects to be mild if less than 10% of respondents met this definition, moderate if 10% to 20% met the definition, and severe if more than 20% of respondents met the definition [36]. In case of the ASCOT-Proxy-person (Fig. 8), we detected a mild floor effect, which can be seen by the lack of overlap between persons and items at the top of the figure. Based on the Wright-Andrich Map, the ASCOT-Proxy-person would benefit from either adding new ‘easier’ items (within an existing cognitive framework) measuring slightly different aspects of the same domain or by moderating the intensity of the question using different wording.

Fig. 8
figure 8

Results of Rasch Analysis, Wright-Andrich Map of the ASCOT-Proxy-person (collapsed categories).

Note: The Measure Scale (-4 to +4) is the logit scale resulting from the Rasch Analysis. Top of the figure represents the floor (high-level of unmet needs coded as 0) while the bottom represents the ceiling (ideal state coded as 3)

food and drink (collapsed; food_2R); accommodation cleanliness and comfort (home_2); personal cleanliness and comfort (pers_2); personal safety (safety_2); social participation and involvement (social_2); control over daily life (collapsed; control_2R); occupation (occ_2); dignity (dignity_2)

Full size image
Fig. 9
figure 9

Results of Rasch Analysis, Wright-Andrich Map of the ASCOT-Carer.

Note: The Measure Scale (-6 to +6) is the logit scale resulting from the Rasch Analysis. Top of the figure represents the floor (high-level of unmet needs coded as 0) while the bottom represents the ceiling (ideal state coded as 3)

personal safety (safety_c); social participation and involvement (social_c); control over daily life (control_c); occupation (occ_c); feeling encouraged and supported (support_c); space and time to be yourself (time_c); self-care (selfcare_c)

Full size image

From Fig. 9 (the ASCOT-Carer) we observed one significant gap (approximately more than 1 logit) on the latent continuum. We see that the two distributions are roughly aligned, (approximately less than .5 logit difference), as demonstrated by the proximity of the average person measure to the average item measure. In case of the ASCOT-Carer (Fig. 9) we detected a mild floor effect, which can be seen by the lack of overlap between persons and items at the top of the figure. Based on the Wright-Andrich Map, the ASCOT-Carer would benefit from either adding new ‘easier’ items (within an existing cognitive framework) measuring slightly different aspects of the same domain or by moderating the intensity of the question using different wording.’

Rasch analysis: Differential Item Functioning

Lastly, we performed a Rasch differential item functioning (DIF) test to see if respondents conceptualised items differently based on the mode of administration (postal versus online). In the analysis of uniform DIF by survey administration, the Mantel-Haenszel statistic, adjusted for multiple comparisons (p<0.007), indicated DIF for two items from the ASCOT-Carer: occupation (χ2=12.33, p=0.0004) and feeling encouraged and supported (χ2=14.33, p=0.0002). This indicates that these two items respond differently, by survey administration group. For occupation, the DIF measure was 1.71 (DIF S.E. =0.30) for those who completed the survey via postal return and 0.87 (DIF S.E. = 0.14) for those who completed the survey online (DIF contrast=0.84, joint S.E. =0.33). For feeling encouraged and supported, the DIF measure was -0.47 (DIF S.E. =0.22) for those who completed the survey via postal return and 0.61 (DIF S.E. = 0.11) for those who completed the survey online (DIF contrast=-1.08, joint S.E. =0.25).

To better understand why respondents conceptualised items differently based on the mode of administration (postal versus online), we have explored whether the two samples (postal and online survey) differed in sociodemographic characteristics: age, gender and ethnicity. We identified that those who filled in postal version were older than those who filled in online version (mean age: 70.63 versus 60.38 years). Therefore, we performed a Rasch DIF test to see if respondents conceptualised items differently based on their age (under 65 years old versus 65 years or over). In the analysis of uniform DIF by age, the Mantel-Haenszel statistic, adjusted for multiple comparisons (p<0.007), indicated DIF for two items from the ASCOT-Carer: occupation (χ2=13.53, p=0.0002) and feeling supported and encouraged (χ2=12.73, p=0.0004). For the occupation, the DIF measure was 0.72 (DIF S.E. =0.16) for those who were under 65 years old and 1.44 (DIF S.E. =0.20) for those 65 years or over (DIF contrast=-0.72, joint S.E.=0.25). For feeling encouraged and supported, the DIF measure was 0.77 (DIF S.E. =0.13) for those who were under 65 years old and -0.06 (DIF S.E. =0.15) for those 65 years or over (DIF contrast=0.83, joint S.E.=0.20).

There was no evidence of DIF by survey administration or age for the ASCOT-Proxy-person (p>.006).

Reliability/Internal consistency

The Cronbach’s alpha was 0.69 for the ASCOT-Proxy-person; and 0.83 for the ASCOT-Carer. The ordinal alpha was 0.78 for the ASCOT-Proxy-person; and 0.87 for the ASCOT-Carer, which meets the .70 reliability standard [39].

Construct validity

Table 1 in Additional File 2 provides an overview of the hypotheses used in this study for assessing construct validity. Table 10 shows Spearman correlation coefficients between the ASCOT-Proxy-person, ASCOT-Carer overall scores (continuous variables) and related constructs (continuous variables). The majority of the Spearman’s rank correlations (>75%) between the ASCOT-Proxy-person, ASCOT-Carer and other variables were same as hypothesised. There were few exceptions to this. The correlations between ASCOT-Proxy-person overall scores and DEMQOL-Proxy-proxy were weaker than expected but stronger with I/ADLs (Proxy-proxy) than expected.

Table 10 Convergent validity of the ASCOT-Proxy-person and ASCOT-Carer
Full size table

Table 11 displays the associations between the ASCOT-Proxy-person, ASCOT-Carer overall scores and various subgroups of the sample. Both measures detected differences between their scores and various subgroups as hypothesised.

Table 11 Known-groups validity of the ASCOT-Proxy perspectives and ASCOT-Carer
Full size table

Discussion

The aim of this cross-sectional study where data were collected using self-administered questionnaire (online or paper form) was to establish the feasibility, construct validity and reliability of the ASCOT-Proxy and ASCOT-Carer among unpaid carers of people with dementia who are unable to self-report. In addition, we aimed to compare the structural characteristics of the ASCOT-Proxy (an adapted version of the ASCOT-SCT4), against the original ASCOT-SCT4 [10].

Overall, the percentage of missing values was low for the ASCOT-Proxy (both perspectives) and ASCOT-Carer indicating a good acceptance of both measures among unpaid carers of people with dementia who are unable to self-report. It is important to mention though, that the dignity item had a higher percentage of missing data in the ASCOT-Proxy (Proxy-proxy: 6.71%; Proxy-person: 7.03%). The overall scores for SCRQoL are only calculated when all items have non-missing values [10]. As such missing data imputation techniques should be considered when calculating overall SCRQoL score.

Next, we found that the proportion of missing data was significantly higher for participants who used paper questionnaires when compared to those using online version. This should not be interpreted that online version of the ASCOT-Proxy was more acceptable than the paper one. For example, unlike the paper version, where participants could accidentally skip over pages, the online version asked participants if they wished to leave the question blank. In addition, when participants could choose which form they wanted to fill in those who were older preferred the paper version (70.6 vs 60.4 years old). Without being able to offer the paper version, it is possible participants would not be willing to take part. This was highlighted to us by research staff across NHS sites, who recruited approximately half of the participants (140/313) on our behalf. Of these, 55 people recruited through the NHS chose to fill in a paper questionnaire, as opposed to 88 people who selected online questionnaire. Only eight participants out of 159 recruited through JDR selected a paper form. It is important to know that those recruited through this platform for the majority of time (between April and October 2020) did not have an option to fill in paper form of the questionnaire due to lack of access to printing facilities by research team as a result of lockdown measures [27]. This is important, as it highlights that there are issues with exclusion of some participants if offering only an online option as a mode for data collection.

In this study, we tested the structural characteristics of the ASCOT-Proxy using exploratory factor analysis against the ASCOT-SCT4 [10]. We confirmed a weak unidimensional (one-factor) solution for the ASCOT-Proxy-person. Similar to this study, one-factor structure has been found for the ASCOT-SCT4 in samples of older adults [10] as well as for other adapted version, e.g. easy-read ASCOT [13] and the translation of the ASCOT into Finnish [35]. On the other hand, we found two factor solution for the ASCOT-Proxy-proxy representing basic domains (items) (Factor 1) and higher order domains (items) Factor 2 [10]. As highlighted in the Results section, the two factor solution for the ASCOT-Proxy-proxy was, while robust, a novel finding. Therefore, future studies should report the factor structure for ASCOT-Proxy-proxy and if our results are replicated, it has to be considered carefully whether and how the ASCOT-Proxy-proxy tool should be used (e.g. in terms of calculating overall scores, establishing construct validity etc.). It is important to highlight though, that the ASCOT-Proxy has been developed and tested at all stages of its development with both perspectives, including this study, so we do not know how e.g. the ASCOT-Proxy-person would perform on its own. More importantly, inclusion of both perspectives improved the acceptability and face validity of the measure during earlier stages of its development [21].

Next, the dignity domain (item) had a low factor loading in case of both ASCOT-Proxy perspectives. In this study, as explained in the Results section (Factor structure of ASCOT-Proxy perspectives) we retained dignity domain (item) as ASCOT-Proxy was developed to conceptually align closely to the ASCOT-SCT4 self-completion tool where each domain (item) represents one dimension of SCRQoL. Dignity is an important aspect of SCRQoL as established in ASCOT-SCT4 development studies [10] and the item was acceptable when developing the ASCOT-Proxy [21]. However, as this was first study to explore structural characteristics of the ASCOT-Proxy, it is important to test structural characteristics of the ASCOT-Proxy (both perspectives) as part of future studies to see whether there are any issues with the dignity item. If our findings are replicated it may be necessary to revisit whether dignity should be included as an important aspect of SCRQoL of social care users when the answers are provided by proxy (e.g. through qualitative conceptual studies with unpaid carers).

In addition to evaluating psychometric properties of the ASCOT-Proxy and ASCOT-Carer by Classical Test Theory methods, we have conducted Rasch analysis [40, 41] to further evaluate the measurement properties of the ASCOT-Proxy-person and ASCOT-Carer. To summarise, the following issues were highlighted when conducting Rasch analysis. First, the ASCOT-Carer personal safety was outside of OUTFIT MNSQ range of .5 to 1.5 when evaluating overall model fit. This is the first study to suggest there may be an issue with the personal safety item in the ASCOT-Carer. It is important to keep in mind though that this was the first time when psychometric characteristics of the ASCOT-Carer were tested in a sample of unpaid carers of someone living with dementia in England. This was also the first time when the Rasch analysis was used as an approach when investigating psychometric characteristics of the ASCOT-Carer. It is also important to highlight that we collected data during COVID-19 pandemic which could impact how people responded when asked about their personal safety. Therefore, we suggest when the ASCOT-Carer is going to be used in the future as a measure, researchers will explore whether there are any issues with personal safety item as identified in this study. Next, this study indicates that respondents may not distinguish between all four response categories for food and drink and control over daily life in case of the ASCOT-Proxy-person. There is an indication that two-response categories option (by combining the two lowest categories and the two highest categories) may be a better solution than four response categories for both food and drink and control over daily life. Next, based on Wright-Andrich maps we detected a mild floor effect in case of both ASCOT-Proxy-person and ASCOT-Carer. Based on the Wright-Andrich Map, both measures would benefit from either adding new ‘easier’ items (within an existing cognitive framework) measuring slightly different aspects of the same domain or by moderating the intensity of the question using different wording. Lastly, Rasch differential item functioning test indicated that the differential item functioning in case of two items in the ASCOT-Carer (occupation; feeling supported and encouraged) when compared postal vs online form of survey is not a bias of the tool but it is likely due to differences between the two samples, i.e. those who filled in postal version were older (70.63 years old) than those who filled in online version (60.38 years old). This implies that the samples cannot be readily compared, unless the items are removed or different item measures are used to score subjects based on their age. The latter would require using Rasch person measures for calculating the preference-weighted values. This is something what should be explored in the future.

Looking across the results when testing convergent and known-group validity, we conclude that both ASCOT-Proxy-person and ASCOT-Carer are valid measures of SCRQoL. Both ASCOT-Proxy-person and ASCOT-Carer met the .70 reliability standard [39] when we calculated ordinal alpha (introduced by Zumbo et al. [42]). The .70 reliability standard was not met when we calculated Cronbach’s alpha for ASCOT-Proxy-person, however it is known that Cronbach’s alpha underestimates the reliability of the ordinal response scales [43]. It is important to highlight that in the present study, we only investigated one aspect of reliability (internal consistency) while test-retest reliability has not been studied. This was due to the impact of COVID-19 on this study, as we only managed to collect data at one time point. Therefore, test-retest reliability should be established in the future studies.

The most important potential limitation of our study is the lack of access to information about the characteristics of participants who started but did not finish the questionnaire (n=33) as we did not have permission to use information from incomplete online survey. Having the access to this information, would allow us to understand whether the questionnaires are more feasible for some participants. We were also unable to recruit participants through the providers of adult social care including local authorities, independent sector care providers and voluntary sector organisations potentially introducing selection bias in our sample. Instead, all our participants were recruited through the NHS and JDR. In this study, we used the same preference-weighted values for ASCOT-Proxy, as for ASCOT-SCT4. A subsequent step should be the development of preference weights for ASCOT-Proxy, which represent the relative importance of the response levels of each domain for quality of life. For example, there is suggestion, that self-reported measures used by people with dementia and proxy versions need different preference-weight values [44]. Next, we did not compare self- and proxy-report. This was not possible in this study, as we focussed on those unable to self-report (our main target group).

Conclusions

This was a first study to explore psychometric characteristics of the ASCOT-Proxy and ASCOT-Carer, with unpaid carers of people with dementia living at home unable to self-report. There are some aspects of the psychometric characteristics of the ASCOT-Proxy and ASCOT-Carer that warrant further investigation in future work.