FormalPara Key Points for Decision Makers

Pictorial patient-reported outcome measures (PROMs) improve paediatric health-related quality of life (HRQoL) assessment.

Pictorial HRQoL PROMs are well-suited for children facing difficulties in understanding textual content.

Incorporating children in the development of pictorial HRQoL PROMs improves content validity.

When choosing pictorial HRQoL PROMs, it is essential to factor in their cross-cultural applicability.

1 Introduction

Patient-reported outcome measures (PROMs) are increasingly being incorporated into clinical practice and health research to inform economic evaluation and quality assessment [1,2,3]. PROMs report an individual’s subjective assessment of their health-related quality of life (HRQoL) at a particular time point or repeated time intervals over an extended period. PROMs may vary from simple assessments to complex multi-dimensional instruments [4]. PROMs can be categorised into those that are preference weighted and those with scoring systems that are not based on preferences; the former are suitable for application in economic evaluation.

Preference-weighted PROMs incorporate scoring algorithms that are typically based on the preferences assigned by general population samples, generated using valuation methods such as discrete choice modelling, standard gamble, or time trade-off, and are typically anchored on the 0 (dead) to 1 (full health) utility scale [4].

PROMs can be further differentiated into condition-specific (e.g. Paediatric Asthma Health Outcome Measure [PAHOM]) [5] and generic (e.g. EQ-5D-Y) [6] PROMs. Generic PROMs are designed for application in all subgroups and allow comparisons between them. Condition-specific PROMs are intended to apply to a single group (e.g. individuals with a specific health condition) and, compared with generic PROMs, generally have a higher sensitivity for detecting changes resulting from the condition or associated factors [7, 8].

Most instrument developers for PROMs for measuring HRQoL in children require for self-completion that children are 8 years old or older [9,10,11]; for younger children and those with developmental delays and/or intellectual impairments, proxy-reported (by a parent/guardian and/or health care professional) or interviewer-administered modes of administration are recommended [4]. The Professional Society for Health Economics and Outcomes Research (ISPOR) Patient Reported Outcomes (PRO) Good Research Practices for assessing children and adolescents Task Force Report found insufficient evidence to establish whether self-reporting of HRQoL by children younger than 8 years of age is reliable and valid [12]. Self-reporting of HRQoL is generally considered preferable to proxy assessment wherever possible [4].

Prior research in HRQoL reporting in children has reported discrepancies between child self-assessments and proxy assessments [13,14,15,16], and the need for psychometrically sound measures for young children [17]. Three previous systematic reviews on the accuracy of proxy-reported versus child-self-reported assessments in deriving childhood utility values for economic evaluation found disparities between the two perspectives [13,14,15].

To support decision-making in research and clinical practice, the ISPOR PRO Good Research Practices report makes five recommendations for assessing childhood PROMs: (1) consider developmental differences and determine age-based criteria for PROM administration; (2) establish content validity of paediatric PROM instruments; (3) determine whether an informant-reported outcome instrument is necessary; (4) ensure that the instrument is designed and formatted appropriately for the target age group; and (5) consider cross-cultural issues [12].

It is well documented that younger children [18, 19] and children with severe chronic conditions [20] face challenges with self-reporting health-related outcomes using PROMs presented in traditional text-based formats (written words). Though proxy reporting is an option, finding ways to support and extend self-report in the aforementioned groups is essential to ensure that HRQoL data represents the child’s views. Measures that use pictorial representations such as pictures, cartoons, and smiley faces (happy–sad faces) to convey information may facilitate reliable self-assessment of HRQoL and ensure the inclusion of children who find reading text challenging.

Previous literature reviews have shown that incorporating pictorial formats alongside text-based information may facilitate a better understanding of PROMs to assess HRQoL among children [21, 22]. Images can also facilitate word memory, increased attention, and conceptual processing, and they demand less cognitive effort to understand than words [21, 22]. Other advantages of using pictures to support written text include engaging young children’s interest and sustaining attention, which may contribute to more meaningful responses and improve completion rates [23].

Four systematic reviews have assessed HRQoL measures used in child populations, which included several pictorial PROMs [19, 24,25,26]. However, these reviews have highlighted certain limitations that are relevant to the objectives of the present systematic literature review. The previous reviews by Cremeens et al. [25] (limited to ages 3–8 years), Arsiwala et al. [26] (primarily focused on generic PROMs with Likert scales and with pictorial elements), and Coombes et al. [24] (focused on 'faces' scales for pain reporting, limited diversity, age group specificity, incomplete coverage of recent developments) all had certain limitations that are relevant to the objectives of this systematic literature review. These limitations include narrow scope, age group specificity, limited diversity in pictorial scales, and potential gaps in coverage of recent developments. Additionally, the study by Solans et al. [19] was undertaken some time ago (1980–2000 timeframe) and primarily focused on generic and disease-specific PROMs without specific attention to pictorial approaches. These limitations indicate the need for a more comprehensive and up-to-date review, encompassing a wider age range, diverse pictorial approaches, and recent developments in paediatric PROMs that incorporate pictorial elements. Therefore, the aims of this review were twofold, to (1) identify PROMs designed for children that use pictorial approaches (where pictorial representations are used either alongside or replace text) to assess HRQoL and (2) provide a comprehensive description of the development and key characteristics of the instruments.

The assessment of these PROMs focussed on ISPOR PRO Good Research Practices report recommendations 1, 2, 4, and 5. The scope of the review did not include proxy-reported outcomes, which typically involve reports from caregivers or parents. Therefore, we did not address ISPOR PRO Good Research Practices recommendation 3, which pertains to informant-reported outcome instruments, as it did not align with the objectives and scope of our study.

In our review, the ISPOR recommendations for the evaluation of pictorial paediatric PROMs were incorporated as follows:

  1. 1.

    Developmental differences were considered by assessing the appropriateness of pictorial elements for various age groups and addressing age-related factors.

  2. 2.

    Content validity was assessed by examining children's involvement in creating pictorial elements and their alignment with specific content domains.

  3. 3.

    While not explicitly addressed, our review centred on self-reported aspects of pictorial paediatric PROMs, which are typically child centred.

  4. 4.

    Suitable design and formatting were emphasised by evaluating how graphical content aligned with cognitive abilities and readability, highlighting features like large fonts and accessibility options.

  5. 5.

    Cultural considerations in the development of pictorial PROMs were discussed, acknowledging the need for further exploration of cross-cultural applicability.

Although no exclusive frameworks exist for pictorial PROMs, we adapted ISPOR guidelines originally designed for PRO instruments, enabling us to effectively assess the development and content validity of these instruments. This adaptation clarified our rationale for including relevant information and enhanced the alignment of our review with a focus on the presentation of graphical techniques in pictorial paediatric PROMs.

2 Methods

2.1 Protocol and Registration

The protocol for this review was registered with the International Prospective Register of Systematic Reviews (PROSPERO), registration number CRD42021222771. This review was conducted following the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) 2020 guidelines for systematic reviews [27, 28]. The PRISMA checklist is presented in Table B in the electronic supplementary material (ESM).

2.2 Data Sources and Search Strategy

The systematic search covered seven electronic academic research databases: Medline (via Ovid), Scopus, Web of Science Core Collection (via ISI Web of Science), CINAHL (via EBSCOHost), Emcare, and Embase and PsycINFO (both via Ovid). A combination of Medical Subject Heading (MeSH) terms and keyword searches were used and adapted to enhance search sensitivity and specificity across databases. The initial search encompassed content from the inception of each database up to 1 March 2022. This process was then updated for each individual database, spanning from its inception to 31 January 2023.

Four themes were used to categorise the search terms. These were (1) ‘health-related quality of life’; (2) communication methods, such as ‘pictogram’, ‘pictorial’, and ‘animation’; (3) the population (e.g. ‘child’ and ‘adolescent’); and (4) the instrument (e.g. ‘tool’, ‘measure’, and ‘questionnaire’). The full search strategy is presented in Table A in the ESM. The search strategy was first developed by the first author (CMK) and refined with feedback from a research librarian and the authorship team. The research librarian translated the search strategy for each database.

2.3 Selection Criteria

2.3.1 Inclusion Criteria

This systematic review included peer-reviewed studies of children < 18 years of age that focus on the development of HRQoL PROMs that use pictorial representations, such as animations, pictures, pictograms, and easy-to-read techniques all designed to enhance children's comprehension of information. Our criteria encompassed both generic and condition-specific HRQoL PROMs intended for children that were either self-report or interviewer assisted. We also considered both preference-weighted and non-preference-weighted HRQoL PROMs. We also extended our criteria beyond published articles to include discussion papers, reports, and published theses for a comprehensive evaluation. This comprehensive approach allowed us to thoroughly assess studies within the scope of our systematic review.

Our search criteria did not impose any country restrictions; we considered all English-language studies for inclusion in the review. For studies identified in languages other than English, we used Google Translate [29] to ensure their inclusion in the review.

2.3.2 Exclusion Criteria

Studies that included participants 18 years of age or over were excluded. Studies examining participants across both child and adult age ranges were only included if the majority of the sample were children under 18 years and if results for the children were reported separately. Reviews, reports, conference papers, book chapters, and opinion pieces were also excluded.

2.4 Article Screening

Identified articles were imported into Endnote X9.3 (2020) reference management software and transferred to Covidence (www.covidence.org), online screening and data extraction software for systematic reviews. The title and abstract of each identified paper were reviewed independently by two reviewers from a team of three (CMK, LL, NB) to determine eligibility. Disagreements were resolved via discussion among all three reviewers. Full texts of publications were further screened independently by each of the three reviewers against the inclusion criteria. All full texts were forward and backward citation checked.

Overall agreement among the reviewers was assessed by Cohen’s kappa, with categories defined as follows: values ≤ 0 indicated no agreement, 0.01–0.20 none to slight (poor), 0.21–0.40 fair, 0.41–0.60 moderate, 0.61–0.80 substantial (good), 0.81–0.99 almost perfect (very good), and 1.00 perfect agreement [30].

2.5 Data Extraction

A standardised data extraction form was developed to ensure consistency and completeness when extracting data from the included articles. The unit of analysis was a PROM and not a research article. Data for each PROM were extracted independently by two reviewers. Discrepancies throughout this process were resolved through discussion. The data extraction form included the following:

  1. 1.

    Characteristics of the PROM: name and acronym of the PROM, name of the author(s)/developer(s), year developed, country/countries where the measure was developed, target age range, type of respondent (e.g. child, proxy), mode of administration (e.g. self-report, interviewer administered), recall period, number of dimensions and items, response options, scoring methods, preference-based or non-preference-based measures, length of PROM, pictorial representations, and electronic data collection (ePRO).

  2. 2.

    Content and item generation process or methodology (e.g. literature review, focus groups, cognitive interviews) and whether children were involved in item generation, pilot testing, and content validation.

2.6 Data Synthesis

An overview of the PROMs identified was presented, with the rest of the data synthesised according to ISPOR PRO Good Research Practices report recommendations 1, 2, 4, and 5 (each described below). A description of measurement properties of the PROMs, if reported, was also provided. Note that a full appraisal of measurement properties for the identified PROMs was beyond the scope of the review. Given the nature of our research question and the focus on the description of these instruments, we did not conduct a risk of bias assessment for the included articles.

In the overview of PROMs identified, the type of PROM (generic and condition- and domain-specific PROMs), health conditions focussed on for condition-specific PROMs, the scoring method used (preference or non-preference based), year of development, country of development, target population, setting in which the PROM was developed, and versions of PROMs available were reported.

To align with ISPOR PRO Good Research Practices report recommendation 1 (determine developmental differences and age-based criteria for PROM administration), the following age groups were reported: < 5 years or early childhood (including infants, toddlers, or pre-schoolers); 5–7 years (younger children); 8–11 years (pre-adolescents); and 12–18 years (adolescents) [12]. PROMs covering more than one age group were classified into a fifth group, ‘multi-age’.

To address ISPOR PRO Good Research Practices report recommendation 2 (establish the content validity of paediatric PROMs), we assessed the methodology and processes used to develop these PROMs. Each PROM was summarised according to (1) the methods used for concept elicitation and development and domain and item generation and (2) whether individuals from the target population were involved in the development, pilot testing, and/or validation of the measure.

To investigate ISPOR PRO Good Research Practices report recommendation 4 (ensure that the instrument is designed and formatted appropriately for the target age group), the response scales were categorised according to the scale format used (Likert or visual analogue) and presentation style, i.e. written, pictorial, verbal/audio, animation, or props. Likert scaling typically measures a positive or negative response to a statement with two or more response options. The visual analogue scale (VAS) is a rating scale in a continuous graphical format, usually presented as a horizontal or vertical straight line extending from one end of the scale (e.g. ‘strongly agree’) to the other (e.g. ‘strongly disagree’) and generally ranges between 0 and 100. We also examined other key characteristics of childhood PROMs as recommended by ISPOR, including the recall period, length of the PROM (number of items), type of pictorial representations applied (e.g. pictorial, verbal/audio, animation, or props) administration approaches, and availability of electronic PROMs.

For ISPOR PRO Good Research Practices report recommendation 5 (consider cross-cultural issues), countries or jurisdictions of PROM origins, cross-cultural influences, and language translations were considered.

3 Results

3.1 Search Results

Figure 1, the PRISMA flowchart, presents the systematic review process and search results.

Fig. 1
figure 1

Literature search flow diagram using the PRISMA checklist. From: Page MJ, McKenzie JE, Bossuyt PM, et al., The PRISMA 2020 statement: an updated guideline for reporting systematic reviews. BMJ 2021;372: n71. PRISMA Preferred Reporting Items for Systematic Reviews and Meta-Analyses, QoL quality of life

Full size image

A total of 26,434 studies were identified. Of these, 16,010 duplicate articles were removed, resulting in 10,424 studies. A further 12 were identified through web searching via Google Scholar. Titles and abstracts were screened, and 10,374 studies did not meet the inclusion criteria and were removed. The reasons for exclusion were (1) adult population and (2) childhood HRQoL PROMs that do not use pictorial response scales or items. Full texts for 44 studies were assessed independently by the three reviewers. A total of 22 PROMs, including 28 unique versions, were identified in the review. The reviewers’ overall agreement (inter-rater reliability) was calculated using Cohen’s kappa and yielded a good agreement (k = 0.72).

3.2 Overview of Identified PROMs

3.2.1 Origin, Population, and Setting

Tables 1 and 2 provide a detailed summary of the identified generic and condition-specific PROMs, respectively. More measures were developed in the USA (n = 9, [32%]) [31,32,33,34,35,36,37,38] and the UK (n = 8, [29%]) [39,40,41,42,43,44] than elsewhere. Four PROMs were from the Netherlands (n = 4, [14%]), three from France (n = 3, [11%]) [45,46,47], and one each from Finland [48] and Australia [49]. Two PROMs, the DISABKIDS Smileys TAKE 6 [50] and the Hvidoere Smiley Faces International diabetes quality of life assessment tool for young children (Hvidoere Smiley Faces) questionnaire [51], were developed as part of multi-country collaborative projects.

Table 1 Summary of characteristics of included generic preference-based and non-preference-based childhood patient-reported outcome measures
Full size table
Table 2 Summary of characteristics of included condition-specific non-preference-based childhood patient-reported outcome measures
Full size table

All the PROMs were developed for children < 18 years, except for one (Dartmouth Primary Care Cooperative Information Project Charts [COOP Charts]) [33], designed for children 12–18. Study participants were recruited from schools, clinics, hospitals, or the general public.

3.3 Criteria for Pictorial PROMs Administration

3.3.1 Target Age Groups

The youngest age for self-reported childhood PROMs with pictorial illustrations available is 3 years (TedQL), while the oldest is 18 years (COOP Charts). Of the 28 PROM versions examined, one measure, AUQUEI Nursery, targets children younger than 5. The PedsQL GCS—Young Child, PedsQL SF15—Young Child, and the Pediatric Asthma Quality of Life Questionnaire (PAQLQ)—Pictorial version target children aged 5–7 years, while three measures (17D, Paediatric Measure Yourself Medical Outcomes Profile [P-MYMOP], and CAQ-B) are intended for children 8–11 years. Measures explicitly designed for adolescents (12- to 18-year-olds) include the COOP Charts and the CAQ-C. Seventeen PROMs covered multiple age groups: the AUQUEI Primary, TedQL, Child Health and Illness Profile—Child Edition Child Report Form (CHIP-CE CRF), CHRIS, DUX-25, Exeter Quality of Life (ExqoL), KidIQoL, Quality of Life Scale for Children (QoL-C), the animated computer program for children with IBD, Bt-DUX, CAQ-A, Children’s Dermatology Life Quality Index (CDLQI), CHRIS—preoperative anxiety, CHRIS—postoperative pain management, DISABKIDS Smileys TAKE 6, and the Hvidoere Smiley Faces measure.

3.4 Development and Content Validity of Paediatric Pictorial PROMs

Table 3 reports on the development process of paediatric pictorial PROMs, highlighting the involvement of children in the process, the graphic approaches employed, and the pivotal role of pilot testing.

Table 3. Methodological approaches used for developing pictorial measures of quality of life
Full size table

3.4.1 Content Generation Methods

In the development of the identified paediatric pictorial PROMs, various methods were used, including literature reviews, expert consultation, focus groups, and interviews. The breakdown of these approaches among the PROMs is as follows: almost half (43%) of the identified PROMs (12 out of 28) utilised literature reviews in their development process. Similarly, 43% of the PROMs (12 out of 28) incorporated focus groups during development. Expert consultation played a role in developing approximately one-third (32%) of the PROMs (9 out of 28). Interviews were the most commonly employed method, with 17 out of 28 PROMs (32%) using them as part of their content and item generation process. These approaches highlight the diverse methodologies employed in the development of pictorial paediatric PROMs, contributing to their depth and comprehensiveness.

3.4.2 Children’s Involvement in the Development of Pictorial Elements

Thirteen studies emphasised the active participation of children in crafting the pictorial elements for HRQoL measures. PROMs in this category include CHU9D animation/CHILDSPLA, KidIQoL, P-MYMOP, QoL-C, TedQL, PAQLQ—Pictorial version, Cochlear Implant Quality of Life (Cochlear Implants—QOL), CHRIS—postoperative pain management, CAQ-A, CAQ-B, CAQ-C, and DISABKIDS Smileys TAKE 6. This collaborative approach aimed to create instruments that are more child-friendly, engaging, and relevant to the unique experiences of children.

In contrast, 15 PROMs, including 17D, AUQUEI Nursery, AUQUEI Primary, CHIP-CE CRF, COOP Charts, DUX-25, ExqoL, PedsQL GCS—Young Child, PedsQL SF15—Young Child, CDLQI Cartoon version, CDDUX, Hvidoere Smiley Faces, Bt-DUX, CDDUX, and the animated computer program for children with IBD, did not explicitly mention the participation of children in the design of pictorial versions.

3.4.3 Diverse Graphic Methods for Varied Content Domains

Various graphic methods were employed depending on the content of different domains in the development of paediatric pictorial PROMs. This approach was taken to address the potential challenges in visual representation and ensure that the instruments could effectively capture children's experiences in diverse aspects of HRQoL.

For example, in domains related to emotional well-being and mood, some PROMs, such as CHU9D animation/CHILDSPLA, CAQ-A, CAQ-B, and CAQ-C, used ‘smiley’ faces or emoticons with different expressions (such as happy, sad, or neutral) to help children express their feelings. This graphic method made it easier for children to convey their emotional states.

In contrast, for domains related to physical health or daily activities, different graphic approaches were used. Some instruments employed interactive characters, animations, or cartoons to represent various aspects of a child's life, making it more engaging and relatable for them. For instance, the CHU9D animation/CHILDSPLA PROM utilised interactive group sessions with children and an animation filmmaker to create engaging characters that children could relate to.

Additionally, in domains where the assessment required quantifying bother levels or responses to specific items, thermometers or visual scales were used. These visual tools allowed children to indicate the extent to which they were bothered by certain issues, helping in a more precise assessment. For instance, the PAQLQ—Pictorial version PROM used thermometers to gauge bother levels, providing children with a clear and visual way to express their discomfort.

By adapting graphic methods to suit the specific content domains, instrument developers aimed to create measures that were child-friendly, comprehensible, and relevant. This approach recognised the importance of considering the unique challenges associated with visual representation when assessing HRQoL in children.

3.4.4 Pilot Testing

Pilot testing was found to be an important component of the development phase of 11 of the identified PROMs, playing a vital role in refining the measures. For example, in the case of KidIQoL, children actively took part in the testing process and provided feedback on the instrument, leading to refinements in item representation. Similarly, in the case of P-MYMOP, a pilot feasibility study involving children aged 7–11 was conducted to identify and address challenges, ensuring alignment with children's comprehension and preferences. This thorough pilot testing process ensured that the measures effectively captured children's experiences.

3.4.5 Adaptation from Existing Measures

Six PROMs were adapted from existing ones: the CHU9D animation/CHILDSPLA (adapted from the CHU9D); the CDLQI Cartoon version (adapted from the CDLQI); the CHIP-CE CRF (adapted from the CHIP CE Parent Report Form [CHIP-CE/PRF]); the QoL-C (adapted from the EQ-5D); the PAQLQ—Pictorial version (adapted from the PAQLQ—Established version); and the P-MYMOP (adapted from the adult MYMOP). Adjustments were made to item and response options and the inclusion of pictures, cartoons, photographs, happy–sad faces, and illustrations to enhance children's understanding. Preferences of children for certain images were observed, emphasising the importance of visual elements in paediatric HRQoL assessment.

3.5 Design and Format of the PROM

3.5.1 Response Scales and the Use of Pictorial Representations

Tables 1 and 2 show that PROMs’ response options and presentation formats were divided into Likert and VAS response scales or a combination of the two. Both scale types were supported by pictorial representations and are thus referred to as pictorial Likert or pictorial VAS hereafter. Almost all the identified PROMs (n = 26, [93%]) used a pictorial Likert scale to present response options. The pictorial Likert scales ranged from a 3-point scale (PedsQL GCS—Young Child, PedsQL SF15—Young Child, QoL-C, PAQLQ) to a 7-point scale (P-MYMOP). Most PROMs (n = 17, [61.0%]) used 4- or 5-point scales. Two PROMs employed the VAS to present response choices [55, 56], i.e. the ExqoL, whose VAS ranged from ‘not like me’ to ‘exactly like me’ [55]. Another PROM, the QoL-C, used a combination of the pictorial Likert scale and the VAS (a horizontal child-friendly health meter ranging from 0 to 10 anchored with happy–sad faces at the extreme ends and in the middle) [41].

Five pictorial presentation styles were identified, with the majority using the happy–sad faces: happy–sad faces (15), cartoons/pictures (9), graduated circles (1), lines (1), thermometers (1), and props (1). For example, the CHU9D animation/CHILDSPLA used a cartoon character, which was then animated and presented in an interactive application on a touchscreen device (Fig. 2). The CHIP-CE used two illustrations depicting each question's two end extreme health states. Each question had five possible response circles, which gradually increased to show the increasing severity as response options advance from ‘never’ to ‘always’ (Fig. 3). In another example, the TedQL used a response scale with circles and lines along with props (two teddy bears) presented by the interviewer to the child (Fig. 4) [44]. Finally, the PAQLQ—Pictorial version employed a response scale using thermometers (Fig. 5).

Fig. 2
figure 2

Example of a response scale using cartoons (CHU9D animation/CHILDSPLA). The example depicts 5 levels of the sad dimension CHILDSPLA computer animation program, from left to right, 1—‘I don’t feel sad’, 2—‘I feel a little bit sad’, 3—‘I feel a bit sad’, 4—‘I feel quite sad’, and 5—‘I feel very sad’. Reprinted from Abrines Jaume, N., et al., CHILDSPLA: a collaboration between children and researchers to design and animate health states. Child: Care, Health & Development, 2015. 41(6): p. 1140–51, with permission from John Wiley &Sons. CHU9D Child Health Utility 9 Dimension, CHILDSPLA Children’s Health States Preferences Learnt from Animation

Full size image
Fig. 3
figure 3

Example of a response scale using illustrations and graduated circles (CHIP-CE). Reprinted from Riley, A.W., et al., The child report form of the CHIP-child edition: reliability and validity. Medical care, 2004: p. 221–231, with permission from Lippincott Williams & Wilkins. CHIP-CE Child Health and Illness Profile-Child Edition

Full size image
Fig. 4
figure 4

Reprinted from Lawford, J., Volavka N., and Eiser C., A generic measure of Quality of Life for children aged 3–8 years: results of two preliminary studies. Pediatric Rehabilitation, 2001. 4(4): p. 197–207, with permission from Taylor & Francis Ltd.

Example of a response scale using circles and lines (TedQL).

Full size image
Fig. 5
figure 5

Example of a response scale using thermometers (PAQLQ—Pictorial version). Reprinted from Everhart, R.S. and B.H. Fiese, Development and initial validation of a pictorial quality of life measure for young children with asthma. J Pediatric Psychology, 2009. 34(9): p. 966–76, with permission from Oxford University Press. PAQLQ Pediatric Asthma Quality of Life Questionnaire

Full size image

3.5.2 Recall Period

The ISPOR task force report (2013) [12] suggests that shorter recall periods of 24 h or less are favourable for paediatric PROMs used within a regulatory context. A recent systematic review on the design of childhood PROMs in preliminary studies recommends a recall period of 48 h or less for children under 8 years old and 14 days and under for children above 8 [24]. Eight [29%] of the PROMs in this review had a recall period of today or current health: the CAQ-A, CAQ-B, CAQ-C, CHU9D animation/CHILDSPLA, DUX-25, TedQL, 17D, and QoL-C. Three [11%] measures, ExqoL, KidIQoL, and CDDUX, used a general recall period (did not specify a recall period). Another eight [29%] PROMs had recall periods ranging from the past week (CDLQI, P-MYMOP, PAQLQ—Pictorial version, Cochlear Implants—QOL) to the past month (CHIP-CE CRF, COOP Charts, PedsQL GCS, PedsQL SF15). The remaining nine PROMs [32%] did not state the recall period.

The available information for the paediatric pictorial PROMs does not offer specific details about how recall periods were dealt with graphically in these studies.

3.5.3 Mode of Administration

In terms of mode of administration, 71% of paediatric pictorial PROMs were designed for self-administration, while 29% were intended for interviewer-assisted administration using electronic and non-electronic data collection methods. Among these, seven PROMs (25%) used electronic data collection methods.

In terms of incorporating pictorial representations in the identified PROMs, there is considerable variability among the studies. The CHU9D animation/CHILDSPLA study notably utilised pictorial versions in multiple modes, including animated presentations, touch-screen interfaces with still images, and hard-copy questionnaires. Conversely, the KidIQoL study did not explicitly specify the use of pictorial versions in either administration method.

Other studies adopted unique approaches. The P-MYMOP study integrated faces scales within interviews, while the QoL-C study employed diverse administration modes, such as one-on-one interviews, classroom activities, and distribution through parents. The TedQL study conducted individual interviews, while the animated computer program for children with IBD employed computer-based animations.

Some studies employed VAS, as demonstrated in the Bt-DUX study. However, others like CAQ-A, CAQ-B, and CAQ-C did not provide explicit information regarding the use of pictorial versions in administration. The CDLQI Cartoon version study lacked specific details on administration methods, and the CDDUX study did not address this aspect at all.

Furthermore, the PAQLQ—Pictorial version study did not specify the mode of administration for pictorial versions, and the Cochlear Implants—QOL study utilised computer-administered methods without providing detailed specifics. Finally, the DISABKIDS Smileys TAKE 6 study did not mention the mode of administration for pictorial versions.

In summary, the incorporation of pictorial versions into the mode of administration for PROMs varied significantly across studies, with some employing diverse methods, while others either did not specify or lacked detailed information on this aspect.

3.5.4 Length of PROM

All the measures contained multiple items, ranging from four (P-MYMOP) to 45 for the CHIP-CE CRF. The estimated time to complete the PROMs ranged from 3 to 5 min (CHU9D animation/CHILDSPLA) to 20–30 min (17D)Footnote 1. Comparing the number of items across versions of the measures showed slightly more items in the versions of the PROMs for older children than those for younger children. For instance, the AUQUEI Nursery version for ages 3–5 years has 26 items, whilst the AUQUEI Primary version for ages 6–11 has 31 items (a difference of 16%). Similarly, the CAQ-A for ages 4–7 years has 14 items, the CAQ-B for ages 8–11 years has 23 items, and the CAQ-C for 12- to 16-year-olds has 41 items.

3.5.5 Scoring of PROMs

The PROMs were scored in one of three main ways: (1) multi-item PROMs only yielding a summary score; (2) multi-item, multidomain scales producing both domain-specific and summary score; and (3) preference-weighted measures that use a value set based on stated preferences for the health states. The scores for the former two are typically determined directly through summary scoring of responses to individual items in the measure.

3.5.6 Domains

Table 4 outlines the domains covered by the PROMs that employ pictorial representations. We combined similar constructs of PROMs under common domain definitions developed for this review. The final ten PROM domains were physical health, social functioning, mental health, emotional health, school, environment, autonomy, pain, disease-specific concepts, and overall QoL.

Table 4. Pictorial patient-reported outcome measures (PROMs) domains
Full size table

Physical health refers to PROM items that examine physical functioning status, activity impairment, physical conditions, and related symptoms. Social functioning refers to items that address the ability to engage in social interactions. Items assessing acceptance, friendship, family life and relationships, and social support were also classified under this domain. Mental health refers to items that capture mental and cognitive health conditions. Emotional health refers to items describing one’s experience, perception, and expression of emotions. It includes items such as sadness, worry, and distress. School has facets that address school attendance and participation in schoolwork and activities. Environment covers different aspects of the immediate environmental living conditions. Autonomy refers to items that address a child’s ability to manage their everyday tasks, become more independent, and make their own decisions, e.g. daily routine and looking after myself. Pain relates to items about bodily pain and discomfort. Disease-specific concepts address disease-specific issues, e.g. disease severity, diet, and device management. This domain was limited to condition-specific PROMs only. Overall QoL refers to a single overview item asking respondents to rate their QoL.

The most examined domain was physical health (measured by 21 PROMs), followed by emotional health and social functioning (assessed by 16 PROMs each). The least measured domain was environment, captured by only two PROMs.

3.6 Cross-Cultural Issues

PROMs are generally designed for widespread usage. As such, it is important to address differences due to culture and language during the development of the PROM, as they may impact use and acceptance.

Three measures addressed cultural issues in the PROM development stage. The animated computer program to assess QoL in children with IBD was designed to be culturally acceptable to all Dutch children [57], while the DISABKIDS Smileys TAKE 6 was developed across seven European countries (Austria, France, Germany, Greece, the Netherlands, Scotland, and Sweden) [50, 58]. The Hvidoere Smiley Faces questionnaire was developed in 17 countries as part of a multi-collaborative study [51].

4 Discussion

This systematic literature review has identified PROMs designed for children that incorporate pictorial approaches to assess HRQoL. Furthermore, this systematic review has provided a comprehensive description of the development and key characteristics of these instruments, focusing on aspects highlighted in the ISPOR PRO Good Research Practices report recommendations, specifically considering developmental differences, content validity, appropriate design and formatting, and cross-cultural issues.

The systematic review identified 22 condition‐specific, generic preference-weighted, and non-preference-weighted childhood PROMs that use pictorial formats to assess HRQoL. Twenty-eight versions of the PROMs were included. These PROMs were developed in several countries, with the USA and the UK contributing significantly. The targeted age groups for these instruments varied, encompassing children as young as 3 years old up to young people aged 12–18 years. Many PROMs were designed with several age-specific formats to accommodate the diverse developmental stages of children.

To our knowledge, this study is the first to systematically identify and describe existing generic and condition-specific, preference-based and non-preference-based PROMs that use pictorial formats to assess HRQoL in children using good research guidance recommended by the ISPOR task force report. This review considered measures where the child is the respondent to self-reported or interviewer-administered HRQoL PROMs for children with or without health conditions 18 years and below. This review is a valuable resource for establishing current knowledge of pictorial paediatric PROMs. It underscores the need for future research on pictorial versions of preference-weighted paediatric PROM development, broadening the array of condition-specific measures and addressing cross-cultural relevance.

4.1 Developmental Differences and Age-Based Criteria for PROM Administration

4.1.1 Target Age Groups

The review shows a diverse range of target age groups, spanning from children under 5 to adolescents aged 12–18. Specific PROMs are designed for each age group, reflecting their developmental needs. Notably, several PROMs exhibit versatility by accommodating multiple age groups, highlighting their adaptability. This diversity underscores the significance of selecting age-appropriate instruments to ensure meaningful assessments of HRQoL for children and adolescents.

4.2 Establish Content Validity of Paediatric PROMs

Establishing the content validity of a PROM is an important stage in its development [59, 60]. Thirteen studies highlighted the active participation of children in crafting the pictorial elements, aiming to create instruments that were more child-friendly, engaging, and relevant to the unique experiences of children. Various graphic methods were employed based on the content domain, including the use of smiley faces, cartoons, and other illustrations, e.g. thermometers. These approaches were tailored to facilitate children's expression of their feelings and experiences effectively. Despite growing evidence of children and adolescents being effective content experts [61, 62], only half of the PROMs identified involved children in the item or domain generation. However, over 70% of the PROMs were pilot-tested in child populations. And while some PROMs used a wide age range in content validation, most PROMs used narrow age ranges, as recommended by ISPOR and others [12]. Thus, it is necessary to evaluate the extent to which domains of standardised pictorial PROMs for children accurately capture the most relevant concepts.

4.3 Ensure that the Instrument is Designed and Formatted Appropriately for the Target Age Group

The design and formatting of child PROMs will usually reflect the age and cognitive abilities of the respondents [8, 55, 63]. The length of the PROMs ranged from four to 45 items, with the estimated completion time varying from 3 to 5 min to 20–30 min. The scoring methods differed, with some PROMs yielding only summary scores, while others provided both domain-specific and summary scores. Additionally, preference-based measures used value sets based on the valuation of health states.

4.3.1 Domains

Implicitly, HRQoL, by definition, is multi-dimensional. Some authors suggest that, at a minimum, paediatric HRQoL measures should address psychological, biological, and social concepts [64]. This study found that while some core domains are included in most pictorial PROMs for children (i.e. physical, emotional, mental, and social well-being, and school functioning), there is no universal consensus on which domains to include. Individual PROM developers included dimensions they consider relevant for a given population. This finding is expected since individual measures are developed for different purposes.

4.3.2 Recall Period

The recall period is another important consideration in paediatric PROMs. The appropriate duration for the recall period depends on the child's memory and understanding of past time. A short recall period of 24 h is recommended for PROMs for regulatory decision-making [12]. Eight of the PROMs evaluated in this study had a recall period of today or current health, aligning with the ISPOR task force recommendations for shorter recall periods in paediatric PROMs. However, there was variation in the specified recall periods across the remaining PROMs, with some not explicitly stating the recall period. It is worth noting that the available information did not provide specific details about how recall periods were graphically represented in these studies.

4.4 Consideration of Cross-Cultural Issues

PROMs are generally designed for widespread usage. As such, it is important to address differences due to culture and language during the development of the PROM, as they may impact use and acceptance. Only three PROMs addressed cultural issues in their development, which suggests the need for this aspect to be considered more in the development of pictorial PROMs as they can be limited if not validated in a particular cultural setting. Two of the three PROMs (DISABKIDS Smileys TAKE 6, and the Hvidoere Smiley Faces questionnaire) addressed cross-cultural effects and multiple translations through multi-country collaborations, which should be viewed in a positive light.

4.4.1 Practical Recommendations

Drawing from the insights gained in this review, we present a set of practical recommendations to guide the selection of appropriate instruments for children and adolescents with specific characteristics. First, it is imperative to take into account the age and cognitive abilities of the intended respondents, choosing instruments that are developmentally suitable. Some PROMs have demonstrated reliable psychometric properties even in children as young as 5 years, underlining the feasibility of utilising pictorial PROMs for younger age groups. Thus, matching the PROM to the target age group is a fundamental consideration. Second, active involvement of children in the development process is encouraged, as it results in child-friendly, engaging, and relevant instruments. It is particularly beneficial when children contribute to the creation of items or domains within the instrument. Third, the design and scoring of child PROMs should align with the cognitive abilities and age of the respondents, emphasising the importance of appropriate length, scoring methods, and recall periods. Ensuring that the design is tailored to the target age group while providing meaningful scoring insights is critical. Finally, for PROMs intended for widespread use, addressing cross-cultural factors and language differences is vital. In this regard, PROMs that have undergone cross-cultural validation, multi-country collaborations, and translations to ensure broader applicability and acceptance are preferable.

4.5 Strengths and Limitations

4.5.1 Strengths

The review follows a systematic approach in identifying, describing, and evaluating a wide range of paediatric PROMs that use pictorial formats to assess HRQoL. It provides a thorough overview of the existing literature in this area. This systematic review provides comprehensive evidence of existing self-report and interviewer-administered PROMs that use pictorial illustrations to enhance child self-reports of HRQoL. The review distinguishes between generic and domain- and condition-specific PROMs and between preference- and non-preference-based PROMs. The PROMs were assessed relative to ISPOR guidelines to inform the choice of PROMs that will accurately measure children’s QoL and be considered in regulatory decisions. The review provides comprehensive information about the identified PROMs, including their origin, target populations, development methodologies, response scales, administration formats, domains, and measurement properties. Furthermore, it advocates child-centred practice in line with best practice recommendations safeguarding the right of children to have a voice in things that affect them.

4.5.2 Limitations

This review did not comprehensively critique the psychometric properties of identified PROMs, which is recommended for PROM selection in research and decision-making [12], as this is the subject of a planned and separate systematic search and synthesis.

The review is based on the literature available up to its last search date, 31 January 2023. Newer studies or developments after that date are not included, potentially affecting the comprehensiveness and accuracy of the review's findings at publication.

4.5.3 Future Research Directions

Our research suggests the need to focus on a thorough evaluation of the psychometric properties of pictorial paediatric PROMs, which should involve rigorous assessments of their reliability, validity, responsiveness, and sensitivity to change. Conducting such evaluations is vital to solidify the credibility and robustness of these tools in both research and clinical applications. A subsequent systematic review is already in the planning stages, and it will be conducted by our team to further explore these aspects.

5 Conclusion

In conclusion, this review serves as a valuable resource for understanding the current landscape of paediatric PROMs using pictorial formats. It highlights the need for further research in developing pictorial preference-weighted PROMs, expanding the scope of condition-specific measures, and considering cross-cultural applicability. Additionally, ensuring age-appropriate design and rigorous content validation are crucial for the development of effective and reliable paediatric pictorial PROMs. Further investigation is needed to validate the potential benefits of pictorial PROMs in assessing HRQoL and facilitating self-report, particularly in younger children and children with developmental delays or severe chronic illness. An important observation is that only half of the existing pictorial PROMs for children actively involved children in their development, indicating room for improvement in research practices.