FormalPara Key Points for Decision Makers

There is a lack of established generic pediatric measures of health-related quality of life (HRQoL) appropriate for use in economic evaluation for young children despite young children being relatively high health system users.

The CHU9D proxy version for children under 5 years of age is a potential instrument for measuring HRQoL in economic evaluations. However, no psychometric evidence on it is available. This is the first study assessing the psychometric properties of the CHU9D proxy version completed by parents or caregivers of children aged 2–4 years old.

This study provides evidence that CHU9D is valid and reliable overall for use by parents of 2–4-year-olds compared with PedsQL, although with relatively low test–retest reliability in some dimensions. This evidence will be useful for those wishing to measure HRQoL for children aged 2–4 years including for incorporation in economic evaluation.

1 Introduction

Children under 5 years of age are important users of health care services and have greater health service use than older children [1]. Many new healthcare technologies target early childhood diseases [2,3,4]. It is thus important to make wise health resource allocation decisions for this age group. The use of economic evaluation for childhood interventions to aid resource allocation decisions has increased in recent years, especially cost-utility analysis [5, 6]. However, there are few instruments appropriate and validated for utility measurement for young children [7,8,9]. A recent systematic review of 372 studies assessing the psychometric performance of pediatric utility instruments reported a prominent research gap in the validation of instruments for preschool-aged children [10].

Many economic evaluations for younger ages used utilities obtained from generic pediatric preference-accompanied measures developed for older age groups or adults [11, 12]. This is problematic [13] as there is evidence that children under 5 years old have different developmental stages and may have different quality of life dimensions or constructs compared with older populations [14]. It is questionable whether instruments having common health dimensions with versions for older children or adults are suitable for use in younger children directly; they usually have adapted wording (or added guidance notes) and different report types (e.g., proxy report or self-report), which often requires further validation evidence [15]. Health technology assessment authorities in Australia and the UK have also noted the lack of utilities used in pediatric economic evaluations and promote the use of concise, generic measures of pediatric HRQoL accompanied by relevant value sets [16, 17]. There is potential to unfairly penalize young children in the health technology assessment process due to poor quality, missing, or uncertain utility evidence [18]. It is therefore important to explore appropriate HRQoL measurement in young children.

The evaluation of the performance of HRQoL measures is important before their wide application. There are four important considerations in these assessments: feasibility, reliability, construct validity, and responsiveness [19]. Feasibility refers to the practicality and acceptability of the instrument to participants, such as the time required to complete the survey and whether the questions are difficult to understand. Reliability concerns the consistency of responses when health status remains unchanged. Psychologists usually examine three forms of consistency: over time (test–retest reliability or intra-rater reliability), across items (internal consistency), and between different assessors (inter-rater reliability). Validity refers to whether the instrument accurately measures the intended concepts. As no “gold standard” exists for HRQoL measures in young children, the typical approach employed is hypotheses testing for construct validity, including convergent validity (testing expected relationships with other measurement instruments, also referred to as concurrent validity) and known-group validity (testing expected differences between relevant groups). Responsiveness assesses the instrument’s ability to detect important changes in HRQoL over time. Reliability is a necessary but not sufficient criteria for validity [20]. Validation is context specific. In other words, one instrument may perform very well in discriminating some diseases but not others.

Recently, five HRQoL measures have become available with the potential for cost-utility analysis for children aged 2–4 years old, all with limited validation evidence. They include: the EuroQol Toddler and Infant Populations (for children aged 0–3 years) instrument [21], the Health Status Classification System for Pre-School Children (for children aged 2.5–5 years) [22], Health Utilities Preschool [23] [for children aged 2–4 years, which has been developed from the Health Utilities Index Mark 3 (HUI3)] [9], EQ-5D-Y adapted version, and Child Health Utility 9D (CHU9D) with guidance notes. The EuroQol Toddler and Infant Populations HRQoL instrument were assessed for convergent validity, known-group validity, and test–retest reliability [21, 24]. The Health Status Classification System for Pre-School Children was assessed for feasibility, known-group validity, convergent validity, test–retest reliability, and inter-rater reliability between parents and clinicians [22, 25]. The Health Utilities Preschool instrument was evaluated for inter-rater reliability, construct validity through hypothesis testing, interpretability, and acceptability [23]. The EQ-5D-Y [26] and CHU9D [27] are two established instruments originally for older children. They now have versions with either adapted wording or guidance notes, providing the potential for measurement of HRQoL in young children for cost-utility analysis. While measuring HRQoL for children aged 2–4 years old using instruments with the same constructs of HRQoL as older children would enable consistent HRQoL measurement throughout childhood, there is currently no validation evidence for the two adapted instruments. This current paper focuses on CHU9D.

CHU9D is a concise, generic measure of HRQoL, accompanied by utilities, which was developed specifically for children [28]. It has been well validated for use for children between 5 and 17 years of age, with good feasibility and validity, although relatively poor test–retest reliability [29,30,31,32]. CHU9D developers also offered a proxy version with guidance notes for measuring HRQoL for children aged 2–4 years old [33]. However, its psychometric performance remains unclear. The available research on the measurement of HRQoL for young children aged 2–4 years old is rather limited. There is no gold standard instrument for measuring HRQoL for children aged 2–4 years old. There are some nonpreference-based HRQoL measures for children under 5 years old including Infant Toddler Quality of Life Questionnaire [34] and the Pediatric quality of life inventory (PedsQL) 4.0 [35]. Reviews are available on their performance [36, 37]. Although being nonpreference based, they could be a useful comparison in validation studies for health utility measures. More specifically, the PedsQL is widely used and well established, with the toddler version for 2–4 years olds shown to be valid and acceptable for pediatric health research [38,39,40]. There is no validation evidence for PedsQL toddler version in Australia; however, no HRQoL tool for this age group has been validated in Australia.

The primary objective of this study is to assess the psychometric properties of CHU9D proxy version administered to parents or caregivers of Australian children aged 2–4 years compared with the PedsQL. Specifically, we aim to assess the CHU9D’s feasibility, ceiling/floor effects, test–retest reliability, convergent and divergent validity, known-group validity, and responsiveness, compared with the PedsQL. We hypothesize that the CHU9D would show good convergent validity with PedsQL due to their similar constructs. Other tests are exploratory due to little previous evidence for measurement of HRQoL for children aged 2–4 years.

2 Method

2.1 Sample

Survey data were from a large Australian pediatric multi-instrument comparison study (P-MIC) conducted during June 2021 to September 2022; data cut 2 dated 10 August 2022 was used in this study, which includes approximately 94% of the total planned P-MIC participants [41, 42]. Any parent, caregiver, or guardian of a child aged 2–18 years (inclusive) at the time of study enrolment was eligible to take part. We included data from those parents/caregivers of children aged 2–4 years old in the current study. The sample was roughly divided as: (1) generally healthy sample and (2) sample with health condition(s). The generally healthy sample included the online general population sample and those recruited through the hospital who were not receiving care (e.g., small number of siblings of patients or children of staff). The sample with health condition(s) included online disease group samples and those recruited through the hospital who were receiving health care. We compared the characteristics of the generally healthy sample with a similar nationally representative sample, i.e., the Longitudinal Study of Australian Children, to check the general representativeness of our sample.

2.2 Survey

Detailed data collection methods were published elsewhere [41, 42]. Data were collected at two time points: the initial survey and a follow-up. There were two follow-up intervals; the first at 2 days (for a subset of the online general population sample) to assess test–retest reliability and the other at 4 weeks (for the remaining whole sample), mainly to assess responsiveness. Data were collected and stored on REDcap, an online survey system [43].

At the beginning of the initial survey, screening questions were presented to establish the eligibility of participants [42]. Respondents who consented would proceed with the survey. The survey then asked participants about their sociodemographic characteristics including age, gender, language, income, education, and general health status of their child. The survey asked if the child had any chronic conditions that have lasted or are likely to last for 6 months or more. If yes, then the caregivers would be prompted to select listed conditions. Only conditions with sample sizes equal to or larger than 30 were included in the analysis. The next survey section presented multiple HRQoL instruments including CHU9D and PedsQL, with the order of these instruments randomized to minimize order and survey fatigue effects [44]. The order of the instruments was the same for the initial and follow-up survey for each participant. Questions about changes in the child’s health status since the first survey were included in the follow-up survey. Time to complete sections of the survey was also recorded on the online REDcap system.

2.3 HRQoL Instruments

The CHU9D has a proxy version with guidance notes for children under 5 years on which parents/caregivers report their child’s HRQoL “today” [33]. The CHU9D consists of nine dimensions (worried, sad, pain, tired, annoyed, schoolwork/homework, sleep, daily routine, and able to join in activities), with five levels of responses for each dimension. The developer of CHU9D developed the guidance notes, with input from other health outcome researchers. The guidance notes provide additional instructions and adaptations on how to interpret schoolwork/homework, daily routine, and ability to join in activities questions for children aged under 5 years (Appendix Table S1). In this study, the CHU9D scoring algorithms, developed based on preferences obtained from Australian adolescents, were applied to calculate and report CHU9D utilities, with the UK adult weights used for sensitivity analysis [45, 46]; no specific value set was available for CHU9D proxy version with guidance notes for children under 5 years.

PedsQLTM version 4.0 is an established, standardized, generic profile instrument for nonpreference-based HRQoL measurement for children aged 2–18 years old [39]. The toddler (ages 2–4 years) version contains 21 items and measures four health dimensions: physical, emotional, social, and school functioning (questions related to school or daycare if attended) [39]. The PedsQL toddler version asks, “please tell us how much of a problem each one has been for your child during the past one month.” This was completed by the study child’s parent/caregiver, who rated the frequency of each item in the past month on a 5-point Likert scale from 0 (never) to 4 (almost always). Items were reversed scored and linearly transformed to a 0–100 scale (0 = 100, 1 = 75, 2 = 50, 3 = 25, 4 = 0), with higher scores indicating better HRQoL [39].

2.4 Psychometric Analyses

Several subgroups were defined to facilitate analysis: subgroups defined by variables including general health status (excellent, very good, good, fair, poor), having special health care needs (yes, no), having a chronic health condition (yes, no), or general health status change (much better, somewhat better, about the same, somewhat worse, much worse). More details of classifications are available in the relevant sections below.

2.4.1 Acceptability and Feasibility

Acceptability and feasibility were measured by examining the time taken to complete the survey and respondents’ reported level of difficulty completing the instrument [47]. There were no established criteria for good feasibility. We assumed that it would be acceptable if completion time was less than 5 min, with more than 90% respondents reporting that the survey was “not difficult” to complete for the general population.

2.4.2 Ceiling/Floor Effects

The presence of ceiling and floor effects is often measured by the distribution of responses. The percentages of respondents choosing the highest/lowest levels in all items were calculated, with above 15% commonly considered high ceiling/floor effects [48]. The percentage of respondents choosing the highest level of each item was also calculated, with percentages > 70% considered potentially problematic [49]. The ceiling effect is often more of a concern when it appears in a patient or unwell sample, while less of a concern if present in a healthy sample where good health is expected.

2.4.3 Test–Retest Reliability

Participants who completed the 2-day follow-up survey and reported “about the same” (i.e., no change) on the general health status change indicator question were included when assessing the test–retest reliability. Intraclass correlation coefficients (ICCs), a widely used index for test–retest reliability, were calculated (using an absolute agreement, two-way mixed effects model) for overall scores of instruments [50]. It is suggested that ICC values < 0.5, 0.50–0.74, 0.75–0.90, > 0.90 are indicative of poor, moderate, good, and excellent reliability, respectively [50]. Weighted kappa coefficients were used to evaluate the test–retest reliability of ordinal responses for individual instrument items. These coefficients took into account differences in reported levels within items to provide a more accurate measure of agreement [51]. They were interpreted as follows: ≤ 0.2 for poor agreement, 0.21–0.40 for fair agreement, 0.41–0.60 for moderate agreement, 0.61–0.80 for substantial agreement, and ≥ 0.81 for almost perfect agreement [52]. Additionally, a larger sample (the 4-week follow-up with unchanged health) was used to calculate the weighted kappa and ICCs as a second measure of test–retest reliability.

2.4.4 Convergent and Divergent Validity

As the CHU9D and the PedsQL measure broadly the same concept (i.e., generic health-related quality of life), we hypothesized that their similar prespecified items (e.g., sad versus feeling sad; pain versus having hurts or aches) and overall scores should demonstrate moderate to high correlation (≥ 0.3) [53]. We hypothesized that their unrelated prespecified items (i.e., worried versus lift something; sad versus lift something) should demonstrate weak correlations (< 0.3). Using an a priori consensus method, the study team collaboratively examined various combinations of instrument items to determine whether they anticipated a moderate correlation between an item from CHU9D and a corresponding PedsQL item (to evaluate convergence) or no correlation at all (to evaluate divergence) [42]. These hypotheses were based on the likeness (convergence) or dissimilarity (divergence) of item wording [42]. Spearman’s rank correlation was applied to assess the correlation [54]. We adopted thresholds whereby 0.1–0.29 indicates low, 0.3–0.49 indicates moderate and 0.5 or above indicates high correlation [55].

2.4.5 Known-Group Validity

Known-group validity refers to the extent to which an instrument discriminates between groups with expected health differences. Groups were defined as: (1) children with any chronic health condition (yes/no); (2) children with special health care needs [56] (yes/no); (3) children with relatively poor health defined by general health status of being good, fair or poor (yes/no); and (4) children with a specific chronic condition (yes/no condition, for example, children with autism compared with children without any health condition). The difference between groups was tested using nonparametric Mann–Whitney U test as the overall indexes and responses for individual dimensions are not normally distributed [57]. Cohen’s d between-subject design (mean difference divided by pooled standard deviation) [49] was estimated to assess effect sizes based on standard thresholds, with 0.2 to < 0.5, 0.5 to < 0.8, and 0.8 or more indicating small, medium, and large effect sizes, respectively [58].

2.4.6 Responsiveness

Responsiveness is used to demonstrate the extent to which an instrument’s response reflects changes in underlying health status [19]. Caregivers were asked to report their child’s general health status change, general health status change related specifically to the initially reported main condition, and health change related to new events occurring during follow up (e.g., new illness or treatment) at the follow-up survey. We identified two subgroups for the analysis of responsiveness: “improved” (answer of “much better”) and “worsened” (answers of “somewhat worse” or “much worse” combined because of small sample size). Mean changes in scores between baseline and follow-up were tested by paired t-test in each group [59]. One sided P values were used as we had specific hypothesis for the direction of the changes [60]. Standard response mean (SRM) or Cohen’s d within groups is another type of effect size and is widely used to assess responsiveness [61, 62]. The SRM was computed by dividing the mean score change by the standard deviation of the change. The magnitude of responsiveness was evaluated using conventional threshold according to Cohen, with < 0.2 deemed as trivial, 0.2 to < 0.5 as small, 0.5 to < 0.8 as medium, and ≥ 0.8 as large [55]. Both the SRM and Cohen’s d are methods to calculate effect sizes, but they are typically used in different contexts. SRM is most used for within-group comparisons over time to assess instrument responsiveness, while Cohen’s d is more versatile and used for between-group as well as within-group comparisons. Cohen’s d within groups (or paired samples Cohen’s d) shares the same formula as the SRM, and the two terms are sometimes used interchangeably.

Statistical analyses were performed using Stata version 16 (Statacorp, Texas). Significance levels were set at P = 0.05.

3 Results

3.1 Basic Characteristics

The total sample had a generally even distribution of gender and age, with slightly more males (54%) and children aged 4 years (39%). The characteristics of the generally healthy sample were comparable with the estimates from population representative Australian data (Longitudinal Study of Australian Children), except that the study sample had higher parental education and income (Table 1).

Table 1 Baseline characteristics
Full size table

3.2 Acceptability and Feasibility

Parents/carers took on average 1.1 and 1.4 min to complete CHU9D and PedsQL, respectively, for the total sample (Appendix Table S2). Most respondents found CHU9D and PedsQL easy to complete, with only 5.5% and 4.8% of the total sample reporting difficulty completing the two instruments, respectively (Appendix Fig. S1).

3.3 Ceiling/Floor Effects

Ceiling effects were not present for CHU9D in the total sample or the sample with special health care needs, with only 12.4% and 6.1% of respondents reporting best levels for all nine dimensions; 15.5% of respondents reported best levels for all nine dimensions in the sample with no special health care needs, just exceeding the ceiling effects threshold. PedsQL did not demonstrate ceiling effects in any sample, with only 3%, 4%, and 1% of respondents reporting best levels for all 21 items in the total sample, the sample with no health care needs, and the sample with special health care needs, respectively. No floor effects were found for any sample. In terms of CHU9D dimensions, pain dimension had over 70% of respondents reporting best level in the total sample (82.30%) and the sample with special health care needs (70.25%). In general, CHU9D had a distribution of different levels of response in the sample with special health care needs and the unwell sample (Fig. 1), which was similarly observed for the PedsQL (Appendix Fig. S2).

Fig. 1
figure 1

Distribution of CHU9D response in different samples. (1) In all dimensions, level 1 always indicates the best state of health, while level 5 always indicates the worst state. (2) CHU9D proxy version for under 5 years has same wording as older version, only with added guidance notes for dimensions “school,” “daily routine,” and “able to join activities” as appropriate for their age. For example, Dimension “school” asks parents to think about activities such as coloring, looking at books/reading, and concentrating, as appropriate for their child’s age if their children did not go to any preschool/nursery/kindergarten. (3) The groups are defined by a variable asking about special health care needs (yes, no) and a variable asking about the general health status of the child, with responses of excellent, very good, good, fair, and poor

Full size image

3.4 Test–Retest Reliability

The median days between initial and the follow-up survey completion for participants for the 2-day and 4-week follow-up were 3 days and 35 days, respectively. The CHU9D had moderate test–retest reliability overall, with estimated ICCs of 0.52 [95% confidence interval (CI) 0.21, 0.72] and 0.60 (95% CI 0.52, 0.67) for CHU9D Australian utilities in the 2-day and 4-week follow-ups, respectively. PedsQL also had moderate test–retest reliability, with ICCs of 0.63 (95% CI 0.34, 0.80) and 0.80 (95% CI 0.75, 0.84) for PedsQL total score in the 2-day and 4-week follow-ups, respectively. The 95% confidence intervals for ICCs at 2-day follow-up were wide due to a small sample size of 53 (Appendix Table S3).

The test–retest reliability for individual dimensions were diverse for CHU9D, with four dimensions (worried, pain, annoyed, and schoolwork) having moderate agreement (weighted kappa ranging 0.44–0.48) and the remaining five dimensions (sad, tired, sleep, daily routine, and joining activities) having fair agreement (weighted kappa ranging 0.19–0.29) (Table 2). Results using the 4-week follow-up without health change sample had generally similar or larger agreement except the “worried,” “sad,” and “pain” dimensions. PedsQL generally had better test–retest reliability for individual items than CHU9D, with 13 (out of total 21) items demonstrating moderate agreement (kappa above 0.4). PedsQL generally showed similar results using the two follow-ups.

Table 2 Weighted-kappa of CHU9D dimensions compared with PedsQL for children reporting no health changes at different follow-ups
Full size table

3.5 Convergent and Divergent Validity

As hypothesized, CHU9D utilities strongly correlated with PedsQL total scores (r = 0.63). In addition, CHU9D and PedsQL displayed moderate correlations (r = 0.3–0.5) across all hypothesized correlated items, except for “sleep” and “trouble sleeping,” which had a high correlation (r = 0.7). Weak correlations were found in items hypothesized not to be correlated (r < 0.3) (Table 3).

Table 3 Convergence between CHU9D and PedsQL in total sample
Full size table

3.6 Known Group Validity

The CHU9D and PedsQL were both able to discriminate between groups with health difference defined as presence versus not of any chronic health conditions, or with special health care needs versus without, or having versus not having good/fair/poor general health status (Table 4). The group mean differences of CHU9D utilities and PedsQL total scores were all significant, with medium-to-large Cohen’s d effect sizes. Known-group validity was also tested in 15 specific health conditions identified in this study compared with those with no health conditions. CHU9D performed well in discriminating individual chronic conditions compared with those with no health conditions, with significant utility differences (0.16–0.36) and large effect sizes (0.86–2.03). The top five conditions with largest effect size were behavioral/cognitive/emotional problems, autism, genetic condition, soiling, and developmental delay. CHU9D had similar or better known-group validity compared with the PedsQL using all different definitions of health differences.

Table 4 Known group validity (Cohe’s d effect size) of CHU9D and PedsQL for different health difference groups
Full size table

The effect sizes varied across CHU9D and PedsQL dimensions (Appendix Table S4.2). For example, children with anxiety compared with healthy children had a large effect size for “worried” but smaller effect size for “pain.” In addition, for both CHU9D and PedsQL, the effect sizes for parents of children aged 2 years old were generally larger than parents of children aged 3 and 4 years (Appendix Table S4.3).

3.7 Responsiveness

In the sample with health condition(s), CHU9D had small effect sizes of responsiveness to general health change and health change to initially reported condition, with a SRM of 0.25–0.30 in the “improved” group and a SRM of 0.21–0.44 in the “worsened” group (Table 5). The results in the “worsened” group need to be treated with caution considering the small sample sizes (n = 14 and 16). PedsQL had small effect sizes (SRM 0.26–0.41) in the “improved” group and trivial effect size (SRM 0.15–0.18) in the “worsened” group. The supplementary results demonstrated that the CHU9D was able to reflect health changes in those who reported worsened health when developing new illness, with medium-to-large effect sizes (SRM 0.72-0.82). PedsQL was able to reflect this health change with medium effect size (SRM = 0.50) (Appendix Table S5.2).

Table 5 Responsiveness of CHU9D and PedsQL in sample with health condition(s)
Full size table

The test–retest reliability, known-group validity, and responsiveness results were similar using CHU9D UK weights (Appendix Table S3, Table S4.1, Table S5.1). There were relatively large differences in the mean utilities when using the Australian- and UK-derived CHU9D utility weights for the same groups, which was expected (Appendix Table S4.1).

4 Discussion

4.1 Overview

Our study showed that the CHU9D with guidance notes proxy reported for 2–4 year old Australian children was easy to complete, had no ceiling effects in a sample with special health care needs, had moderate to high correlation with PedsQL prespecified similar items, medium-to-large effect sizes of known-group validity, overall moderate test–retest reliability (with diverse results for individual dimensions), and showed some responsiveness to meaningful health changes over time (with small to large effect sizes using different definitions of health change). Compared with the PedsQL, CHU9D had similar feasibility, known-group validity, responsiveness, and slightly poorer test–retest reliability.

4.2 Distribution of Responses

CHU9D did not exhibit ceiling effects except in the sample with no special health care needs. However, the ceiling effects issue was minor as the percentage of those reporting best levels in all dimensions (15.5%) just exceeded the criteria (15%). In addition, it may be less of a concern as good health was expected in the generally healthy sample with no special health care needs. Most CHU9D dimensions had a good distribution across different levels in the sample of children with impaired health. However, 70.3% of respondents reported the best level for dimension “pain” even in the sample with special health care needs, which indicates that this item may not distinguish children well. This is consistent with the results from the EuroQoL Toddler and Infant Populations, with 73% and 88% respondents reporting the best level for “pain” in acute and chronic condition samples [21]. This may be because pain is infrequent for young children or is difficult for parent/caregivers to observe. Recommended observable pain related behavior in young children includes grimacing, restless movement, and inconsolable crying [21]. These, or other behaviors, could be added as guidance notes for the “pain” dimension in CHU9D to help improve the sensitivity of this item.

4.3 Test–Retest Reliability

In our study, CHU9D shows overall moderate test–retest reliability, with ICCs of 0.52 and 0.60 for 2-day and 4-week follow-ups, although the reliability for individual dimensions were more diverse (kappa 0.19–0.47). The test–retest reliability is similar or slightly poorer compared with previous studies of CHU9D or other similar pediatric HRQoL measures in older children [21, 26, 30, 53]. For example, Yang et al. found an ICC of 0.653 for the CHU9D utility score, and kappa estimates ranging 0.20–0.53 for different CHU9D dimensions in 232 school children aged 8–17 years old who completed a retest survey 2 weeks post the initial survey [53]. Ravens et al. found satisfactory ICC (0.82–0.83) and fair-to-moderate kappa estimates up to 0.67 in children aged 8-19 years old who completed the retest for EQ-5D-Y 7–10 days after the first examination [26].

The kappa should be interpreted with caution as it is also impacted by other factors such as the distribution of different levels for each dimension [63]. ICC results also relate to the variation in participant characteristics and study sample sizes [50, 64]. Similarly, Ravens et al. reported concerns that high ceiling effects in EQ-5D-Y impacted the test–retest reliability results and that the kappa coefficient was of limited value (kappa = −0.003) as nearly all retest responses were in the “no problems” category [26]. As the 2-day follow-up retest sample in our study was only from the online general population [41], the lack of variance of responses and high ceiling effects might contribute to the low kappa and ICC estimates.

4.4 Convergent and Divergent Validity

The CHU9D displayed convergent validity with PedsQL, confirming that the same latent construct of HRQoL was being measured by these two instruments. Our correlation coefficients (0.34–0.70 for similar items and 0.62–0.65 for overall scores) were generally similar with previous studies, with some slight differences. Petersen et al. found that correlations between CHU9D and PedsQL for related dimensions and overall scores were 0.40–0.50 and 0.69, respectively, for a Danish high school student sample and 0.28–0.46 and 0.63, respectively, for an Australian adolescent sample [65, 66]. Our stronger correlation coefficients compared with previous studies may be because previous studies calculated correlations between CHU9D items with PedsQL summary functions instead of with PedsQL individual items. Only a small number of potentially divergent items were prespecified and divergence was identified for each (PedsQL lifting something and CHU9D sad, worried). More item pairs could have been selected for divergence (such as bathing/picking up toys and sad/worried) however it was felt that in children 2–4 years of age both bathing and chores could be accompanied by an emotional response, especially with a “today” recall period for the CHU9D. It is also worth noting that CHU9D and PedsQL have different recall periods; CHU9D asks about today while PedsQL asks about the past month, which may reduce the correlation between similar constructs.

4.5 Known-Group Validity

The CHU9D was able to discriminate between groups with known health differences, with medium-to-large effect sizes, regardless of which scoring algorithm was applied. The utility difference between those “with and without chronic conditions or disabilities” was 0.13 using an Australian adolescent algorithm and 0.07 using the UK adult scoring algorithm, with differences similar to previous studies [65, 66]. Peterson et al. found that the utility differences between “with and without chronic conditions or disabilities” in a Danish high school student sample were 0.11 and 0.06 for Australian adolescent and UK adult scoring algorithms, respectively [65]. In a similar study conducted with an Australian adolescent sample using Australian adolescent weights, the utility difference between “with and without chronic conditions or disabilities” is 0.15 [66]. Neither of the prior studies reported Cohen’s d effect sizes; however, the utility differences are similar to our findings. This suggests that CHU9D may have comparable known-group validity in children 2–4 years old compared with older age groups.

CHU9D utilities showed large effect sizes (range 0.86–2.13) for 15 health conditions (identified in this study with sample sizes larger than 30) compared with those reported no conditions, indicating that CHU9D can be applied in a variety of disease groups with good known-group validity in children aged 2–4 years old. There was a large difference in mean utilities when different value sets are applied. Nevertheless, the effect sizes for known-group validity remained very similar between the two value sets, emphasizing that the conclusion was not influenced by the choice of the value set.

4.6 Responsiveness

In our study, CHU9D demonstrated responsiveness to health changes over time, with mainly small effect sizes (SRM 0.25–0.44) according to different definitions of health change in 2–4-year-olds, except in those who developed new illness where large effect size was found (SRM 0.82). To our best knowledge, only one study has investigated the responsiveness of CHU9D, with no studies investigating young children. Wolf et al. (2021) examined the responsiveness of the proxy-reported CHU9D in 396 Danish children aged 6–15 years with mental health problems and found a SRM of 0.634–0.654 for children who experienced clinically significant improvements [67]. Our study had smaller magnitude of responsiveness in terms of SRM (0.25–0.55) for those self-reporting changes in general health status, although it was difficult clearly understanding why there was a change in health. The magnitude of responsiveness for those developing new illnesses was instead much larger (SRM 0.72–0.82); the results needed to be treated with caution considering the small sample size. This suggests that the context of health change may matter in assessing responsiveness and caution should be paid to the comparability of responsiveness between different studies or instruments.

4.7 Implications and Limitations

Our study provides consistent measurement of child health using the CHU9D across child age which could be important for measurement within pediatric clinical trials or in routine clinical care. Further development and validation work is warranted given the limitations and discussion as below.

Several limitations have been identified. First, missing data were not permitted for CHU9D based on a structural decision to not allow skipping items. This had its advantage such as reducing the percentage of missing data but might have forced people to randomly select an answer even when they thought the answer was not rational or suitable. We thus lacked the ability to assess the content validity of CHU9D for this age group through observation of missing data. Canaway et al found no missing values in CHU9D responses with interviewer-administered data collection (questions being read to the child) in slightly older children aged 6–7 years, which reduces our concerns [30]. Despite having good psychometric information on the CHU9D with guidance notes we are unable to determine the impact that the guidance notes themselves had on respondent’s cognitive processing. This could usefully be explored in a follow-up study. Another limitation is that the sample size for the 2-day follow-up test–retest reliability was only 53 respondents from the general population. Despite being small this is still deemed adequate according to consensus-based standards for the selection of health measurement instruments (COSMIN) study design checklist [19]. In the responsiveness analysis, the sample sizes of the groups reporting health changes were also small, especially for the “worsened” group. However, this evidence is difficult to obtain given the low probability of serious health states and worsening health in children and in those populations the low tolerance for survey burden. Further studies in clinical studies with populations having severe health states or studies targeting children aged 2–4 years old with larger sample sizes might be beneficial. The P value of the paired difference may be of limited value considering the small sample sizes in some subgroups and therefore the SRM results were mainly reported. The SRM provided useful indication of potential effect sizes of responsiveness of CHU9D for future users. However, it is acknowledged that it might not be appropriate to report effect sizes if the differences were not significant and the effect sizes for nonsignificant differences were shown for illustrative purpose only. There are potential methodological limitations in applying scoring algorithms developed for older children to calculate CHU9D utilities in children aged 2–4 years old. For example, the preferences for health states of different age groups may differ. However, there is no alternative until a value set for this young age group is developed or the validity of the existing value set is confirmed for this purpose. There is a need to understand and test appropriate preference-weighted scoring for this instrument in this age group, which will further allow utility values to be accurately and consistently produced by the CHU9D in children as young as 2 years old for economic evaluation. Obtaining preference-weighted scores for CHU9D proxy version with guidance notes or developing mapping algorithms to other existing scoring systems could be important next steps to facilitate use of the CHU9D in economic evaluation and resultant policy decisions for this age group.

There is an ongoing debate on the validity of proxy-reported HRQoL, particularly due to poor agreement between self-report by older children and proxy-report by adults [68]. While proxy reports are discouraged when children can self-report, they remain the only option for very young or cognitively challenged individuals. Parents of young children under 5 years old, who usually spend more time caring for their children, may serve as better proxies due to their close observations and connections. There is evidence that agreement is stronger in the youngest age group (5.5–6.5 years) than older age groups (6.5–8.5 years) [68]. Concerns regarding proxy-report, especially for more subjective dimensions such as “worried”, “sad,” and “pain” may be addressed by including externally observable indicators. Evaluating the validity of proxy HRQoL measures is controversial. Nevertheless, the pressure to include young children and their QALYs for cost-utility analysis and the existence of valid preference-based measures for older children continues to underscore the practical value of these investigations for younger children.

It is acknowledged that children under 5 years of age may have different health dimensions of HRQoL and it may not be suitable to directly apply HRQoL measures designed for older children to this younger age group. This study was unable to evaluate the fundamental construct validity of CHU9D to measure HRQOL for this 2–4-year-old age group, i.e., to explore whether the included dimensions were appropriate and/or whether dimensions were missing. Developing a new instrument that incorporates literature reviews and qualitative research would be the ideal way to guarantee the appropriate construct of HRQoL for a new age group [28]. However, the time and expenses associated with this development task mean that it is worthwhile to better understand the performance of existing options and smaller modifications.

Adding guidance notes is assumed to enhance the applicability of CHU9D for children under 5 years old. However, uncertainty remains regarding its suitability. While it is ideal to conduct qualitative research to assess the content validity of these guidance notes first, in this case, we proceeded with testing as the CHU9D with guidance notes are already widely in use. Our study serves a crucial role in evaluating these guidance notes relative to the validated but nonpreference-based PedsQL, with findings offering valuable insights to further refine CHU9D to better suit this age group. Future qualitative research aimed at testing and improving the CHU9D would be highly beneficial.

5 Conclusion

CHU9D proxy version with guidance notes demonstrated good psychometric performance overall for measuring HRQoL for 2–4-year-old Australian children and shows potential as a valid and reliable instrument for assessing the HRQoL for this population.