Abstract
Introduction
Retinitis Pigmentosa (RP) and Leber Congenital Amaurosis (LCA) are rare inherited retinal degenerative disorders. The Visual Symptom and Impact Outcomes patient-reported outcome (ViSIO-PRO) and observer-reported outcome (ViSIO-ObsRO) instruments were developed in this population to assess visual function symptoms and impacts on vision-dependent activities of daily living (ADL) and distal health-related quality of life (HRQoL). This study aimed to explore the psychometric properties of the ViSIO-PRO and ViSIO-ObsRO in RP/LCA.
Methods
The 49-item ViSIO-PRO and 27-item ViSIO-ObsRO instruments were completed by 83 adult and adolescent patients and 22 caregivers of child patients aged 3–11 years with RP/LCA, respectively, at baseline and 12–16-day follow-up. Concurrent measures were also administered at baseline. Psychometric analyses assessed item (question) properties, dimensionality, scoring, reliability, validity, and score interpretation.
Results
Item responses were mainly evenly distributed across the response scale, and inter-item correlations were mostly moderate to strong (> 0.30) at baseline within hypothesized domains. Item deletion was informed by item properties, qualitative data, and clinical input and supported retention of 35 ViSIO-PRO items and 25 ViSIO-ObsRO items. Confirmatory factor analysis in line with pre-hypothesized domains supported a four-factor model assessing visual function symptoms, mobility, vision-dependent ADL, and distal HRQoL. A bifactor model supported calculation of total scores and four domain scores. Internal consistency was high for domain and total scores (Cronbach’s alpha > 0.70) and test–retest reliability for total scores was strong between baseline and 12–16-day follow-up (intraclass correlation coefficients 0.66–0.98). Convergent validity was supported by strong correlations in a logical pattern with concurrent measures. Mean baseline scores differed significantly between severity groups. Distribution-based methods provided initial insights to guide interpretation of scores.
Conclusions
Findings supported item reduction and established scoring of the instruments. Evidence of reliability and validity as outcome measures in RP/LCA was also reported. Further research is ongoing to explore responsiveness of the ViSIO-PRO and ViSIO-ObsRO instruments and interpretation of change scores.
Similar content being viewed by others
Avoid common mistakes on your manuscript.
Why carry out this study? |
Previous research evaluating the appropriateness of clinical outcome assessments (COAs) identified that there are no disease-specific instruments to assess the visual function symptoms and impacts of Retinitis Pigmentosa (RP)/Leber Congenital Amaurosis (LCA). The Visual Symptom and Impact Outcomes patient-reported outcome (ViSIO-PRO) and observer-reported outcome (ViSIO-ObsRO) instruments were developed through qualitative research with RP/LCA patients and caregivers/parents of children. The purpose of this study was to evaluate the psychometric properties of the instruments for use in RP/LCA. |
What was learned from the study? |
This study provides evidence that the ViSIO-PRO and ViSIO-ObsRO are valid and reliable instruments that capture the most relevant and important concepts of visual functioning and associated impacts on vision-dependent activities of daily living (ADL) and distal health-related quality of life (HRQoL) in an RP/LCA population, across a range of genotypes. |
The findings strongly support the ViSIO-PRO and ViSIO-ObsRO to be appropriate instruments for inclusion as outcome assessments in future RP/LCA clinical trials and research studies. Further research will explore whether the instruments are able to detect changes over time in an RP/LCA population. |
Introduction
Retinitis Pigmentosa (RP) and Leber Congenital Amaurosis (LCA) are rare inherited retinal degenerative disorders (IRD) caused by a variety of autosomal dominant, autosomal recessive, or X-linked-associated gene mutations that lead to impairments in the rod and cone photoreceptors and subsequent progressive visual function symptoms [1, 2]. Gene mutations associated with RP/LCA are commonly seen in the RHO, USH2A, RPGR, and RP2 genes, with rarer mutations seen in genes such as RPE65 and RLBP1, among others [3]. Patients typically lose night vision in adolescence, peripheral vision in young adulthood, and central vision in later life, in addition to a variety of other visual impairments [4]. These visual function symptoms experienced as part of RP/LCA can have a significant impact on patients’ vision-dependent activities of daily living (ADL) and health-related quality of life (HRQoL) [5,6,7,8].
Previous research has assessed the appropriateness of existing patient-reported outcome (PRO) instruments used to assess functional vision in other ophthalmologic conditions, for use in RLBP1 RP/LCA [9]. The findings demonstrated that existing PRO instruments do not provide a comprehensive assessment of the patient experience of RLBP1 RP (and likely RP/LCA more broadly) and contain items that lack relevance or are difficult for patients to interpret due to being insufficiently specific (i.e., items do not account for the range of lighting conditions and familiarity of environment conditions that act as moderating factors for visual function symptoms). As a result, the Visual Symptom and Impact Outcomes patient-reported outcome (ViSIO-PRO) and observer-reported outcome (ViSIO-ObsRO) instruments have been developed to assess the visual function symptoms and impacts on vision-dependent ADL and distal HRQoL concepts in patients with RP/LCA, in line with best-practice guidelines for clinical outcome assessment (COA) development and regulatory guidance [10,11,12]. Specifically, development of the instruments has been informed by the prior research and two rounds of qualitative concept elicitation (CE) and cognitive debriefing (CD) interviews that explored the patient experience of RP/LCA and qualitatively evaluated the ViSIO-PRO and ViSIO-ObsRO instruments, to provide evidence of their content validity in this population [5, 7, 8].
In addition to content validity, PRO and ObsRO instruments should provide evidence of scoring and psychometric properties including reliability, construct validity, ability to detect change, and score interpretation, within the target population [13]. A non-interventional observational study was conducted with adult and adolescent patients and caregivers of child patients with RP/LCA, to evaluate the psychometric properties of the ViSIO-PRO and ViSIO-ObsRO instruments.
Methods
Study Design
This was a non-interventional observational study to evaluate the psychometric properties of the newly developed ViSIO-PRO and ViSIO-ObsRO instruments. Participants completed the ViSIO-PRO or ViSIO-ObsRO, along with Patient Global Impression of Severity (PGI-S) or Caregiver Global Impression of Severity (CGI-S) items and the EQ-5D-5L or EQ-5D-5L (parent/caregiver proxy version) over the telephone at two separate timepoints, 12–16 days apart [14]. All participants also completed the Visual Functioning Questionnaire-25 (VFQ-25) at baseline only (Table 1).
Participant Sample
One-hundred-and-twenty participants were targeted for this study. The target sample size was determined by best-practice guidelines for psychometric validation studies (i.e., minimum sample size of 100 patients overall for factor analysis, or at least 5 patients per item in the instrument under assessment), while considering that RP/LCA is a rare disease. Participants were recruited from clinical sites in France, Denmark, Germany, Canada, and the USA. Centralized ethical approval and oversight for this study was provided by Copernicus Group Independent Review Board (CG-IRB) to conduct the study in the USA and Denmark. Local ethical approval was provided at the specialist clinical sites in France, Germany, and Canada. Documentation of informed consent was obtained prior to any research activities being conducted. The study was performed in accordance with the Helsinki Declaration of 1964, and its later amendments [20].
Eligibility Criteria
All participants were required to meet the eligibility criteria outlined in Table 2 to be enrolled into the study.
Overview of Instruments
ViSIO-PRO
The 49-item ViSIO-PRO instrument is designed to assess impacts on vision-dependent ADL, mobility, and distal HRQoL by measuring the level of difficulty experienced by patients with RP/LCA when in specific situations or performing a variety of ADLs that significantly rely on visual function. There are two versions of the ViSIO-PRO: a self-administered version and an interviewer-administered version for those who are unable to self-complete due to their vision. Both are identical other than differences in the instructions provided. During the observational study, all participants completed the interviewer-administered version of the ViSIO-PRO to ensure standardization across the sample. All items require patients to answer on the basis of the “past 7 days” and each ViSIO-PRO item is assessed on a 4- or 5-point verbal descriptor frequency or severity response scale. An additional response option is also provided for most items, for patients to select if they have not been in a specific situation or performed the activity within the past 7 days for reasons other than their vision, rendering the item not applicable.
Items within the ViSIO-PRO are grouped under four hypothesized domains assessing visual function symptoms (items 1–13), impacts on vision-dependent ADL (items 14–27), impacts on mobility (items 28–35), and impacts on distal HRQoL (items 36–49).
ViSIO-ObsRO
The 27-item ViSIO-ObsRO instrument is designed to assess impacts on vision-dependent ADL, mobility, and distal HRQoL experienced by children with RP/LCA (aged 3–11 years), on the basis of informant report (i.e., a caregiver of a child aged 3–11 years with RP/LCA). The ViSIO-ObsRO instrument requires parents/caregivers of children with RP/LCA to report the observed level of difficulty or frequency that a problem is experienced when their child is in specific situations or performing vision activities, over the “past 7 days.” Items are assessed on a 2- and 3-point verbal descriptor response scale. Three different response scales are used. An additional response option is provided for all items, for the parent/caregiver to select if they have not observed their child do that activity in the past 7 days, rendering the item not applicable.
Items within the ViSIO-ObsRO are grouped under the same four hypothesized domains as the ViSIO-PRO, assessing visual function symptoms (items 1–2), impacts on vision-dependent ADL (items 3–10), impacts on mobility (items 11–18) and impacts on distal HRQoL (items 19–27).
Translations of the ViSIO-PRO and ViSIO-ObsRO were informed by best-practice guidelines for translation, cultural adaptation, and linguistic validation, as outlined in the International Society for Pharmacoeconomics and Outcomes Research (ISPOR) task force report, before their inclusion in this study [21].
Validation Instruments
Concurrent measures were administered alongside the ViSIO-PRO and ViSIO-ObsRO instruments during the observational study, to support psychometric validation analysis. An overview of the additional study instruments used for validation are presented in Supplementary File 1. All instruments are scored so that higher scores indicate worse functioning or well-being, with the exception of the VFQ-25 and EQ-5D-5L VAS.
Statistical Methods
A series of tests and analyses were performed throughout the study, each designed to evaluate different aspects of an item or score’s performance. All planned statistical analyses were detailed in a psychometric analysis plan. Table 3 details the main statistical methods used in this study. A description of the analysis populations is provided in Supplementary File 2.
Results
Demographic Characteristics
Eighty-three patients with RP/LCA (including 65 adults and 18 adolescents) and 22 caregivers of children aged 3–11 years with RP/LCA participated in the observational study (Table 4).
Clinical Characteristics
Overall, there was a good range of different RP/LCA genotypes within the sample across adults and adolescents including a range of VA severity levels (Table 5).
Stage 1: Item Properties
Quality of Completion
Quality of completion for the ViSIO-PRO was excellent at both timepoints, with no items missed at baseline and only one item missed by one patient at 12–16-day follow-up. Four patients did not participate in the 12–16-day follow-up, due to starting treatment within the 12–16-day timeframe.
For the ViSIO-ObsRO, quality of completion was excellent across all timepoints, with no items missed at baseline or 12–16-day follow-up.
Item Response Distributions
For the ViSIO-PRO, baseline scores were relatively evenly distributed across the full response scale (Fig. 1). However, item 21 (driving) was heavily skewed to the higher end of the scale, i.e., great difficulty with driving. Some floor and ceiling effects were observed for a number of items and flagged for discussion (Supplementary File 3).
For the ViSIO-ObsRO, although responses were well distributed at baseline across the full response scale for most items (Fig. 2), there were several items with responses skewed towards the bottom of the scale, indicating that patients did not have difficulty or had minimal difficulty with the concept being assessed. ViSIO-ObsRO items showing floor or ceiling effects were flagged for discussion (Supplementary File 3). Full item response distribution tables for the ViSIO-PRO and ViSIO-ObsRO instruments are provided in Supplementary File 4.
Stage 2: Dimensionality and Scoring
Inter-Item Correlations
Inter-item correlations between item pairs within the four specified subdomains were mostly moderate to strong (> 0.30) [31]. High item-pair correlations (≥ 0.90) were observed for some items in the visual function symptoms, vision-dependent ADL, and distal HRQoL domains. Weak correlations were found for four item pairs in the vision-dependent ADL domain (range 0.03–0.29), two item pairs in the mobility domain (range 0.16–0.22), and several item pairs within the distal HRQoL domain (range 0.00–0.28). Inter-item correlation matrices for each ViSIO-PRO domain are presented in Supplementary File 5.
Twenty-four ViSIO-ObsRO item pairs demonstrated high inter-item correlations (≥ 0.90; Supplementary File 5). However, these results should be interpreted with caution due to highly skewed item response distributions (Fig. 2).
Factor Analysis
Exploratory factor analysis (EFA) for the ViSIO-PRO identified two-factor or three-factor solutions, but item-domain groupings did not align conceptually with item content. Several items were removed, on the basis of the item response distributions and floor and ceiling effect results, while considering earlier qualitative findings and clinical relevance/importance (for further details on item deletion, see Supplementary File 6).
A confirmatory factor analysis (CFA) was then conducted to assess the factor structure with these selected items removed (Supplementary File 6). CFA results broadly supported a four-factor model reflecting the a-priori hypothesized domain structure with adequate model-fit (CFI 0.914, RMSEA 0.088, SRMR 0.112). Items excluded in the CFA were subsequently excluded from the finalized ViSIO-PRO except from item 21 (driving), which is scored separately from the remaining items.
Rasch Analysis
Rasch analysis was conducted within each of the four ViSIO-PRO domains. The presence of local dependencies informed further deletion of six items (Supplementary File 6).
Bifactor Model
A bifactor model with one general factor (representing a total score) and four specific factors was fit to investigate essential unidimensionality of all ViSIO-PRO items (Fig. 3). The general factor explained common variance (ECV) equal to 0.73—thus falling slightly below the recommended threshold to meet essential unidimensionality of 0.80 or higher [32]. However, as this result is likely due to the small sample size, it was still deemed acceptable to calculate a total score of all items.
The sample size for the ViSIO-ObsRO (n = 22) was not large enough to perform factor analysis, Rasch analysis, or bifactor modeling.
Item Deletion
Following all stage 1 and 2 psychometric analyses, the study team and expert clinicians discussed possible item deletion. Decisions were made on the basis of stage 1 and 2 results, findings from the previous qualitative interviews [5, 7, 8], and input from the expert clinicians. Given the small ViSIO-ObsRO sample size and the limited analyses performed, a cautious approach towards item deletion was taken. Fourteen items were removed from the ViSIO-PRO and two items from the ViSIO-ObsRO. Rationale for item deletion is provided in Supplementary File 6.
Conceptual Framework
Thirty-five items were retained in the ViSIO-PRO and 25 items in the ViSIO-ObsRO for further psychometric validation (final conceptual framework provided in Tables 6 and 7, respectively).
Scoring Algorithm
A scoring algorithm for the ViSIO-PRO was established on the basis of the EFA, CFA, Rasch analyses, bifactor modeling, and findings from the previous qualitative research supporting the conceptual relevance of the domains and items. For the ViSIO-ObsRO, the scoring algorithm was based on findings for the ViSIO-PRO and conceptual relevance informed by previous qualitative research.
All ViSIO-PRO items are rated using either a 4-point or 5-point verbal descriptor scale. An additional “not applicable” response option is included for items 1–28 and 35 (Table 6), but does not contribute to the scoring of the ViSIO-PRO and instead can be reported descriptively (i.e., treated as equivalent to missing data). Items responded to using a 4-point verbal descriptor scale (items 29–34) are rescaled to a 5-point scale prior to calculating a domain or total score, so that every item is scored using the same 0–5 scale (Table 8).
Similarly, all ViSIO-ObsRO items are rated using a 2-point or 3-point verbal descriptor response scale and include an additional “not applicable” response option that does not contribute to the scoring of the ViSIO-ObsRO. Items assessed using a 2-point verbal descriptor response scale (items 1, 2, and 22–24 in Table 7 are rescaled to a 3-point so that every item is scored using the same 0–3 scale; Table 9).
For the ViSIO-PRO and ViSIO-ObsRO, an overall total score and domain scores for the four domains (Tables 6, 7) in each instrument can be calculated.
Domain scores: Domain scores are calculated by summing scores for all items within a domain and dividing by the number of items completed within that domain. Thus, each domain score is an average or mean of the item scores for the items included in that domain.
Total score: All items included in these four domains can be combined into a single total score, calculated by averaging the mean domain scores. Each domain and total score can range from 0–5 to 0–3 for the ViSIO-PRO and ViSIO-ObsRO, respectively, where higher total scores reflect worse visual functioning or HRQoL.
Item 17 (driving in dim lighting) does not contribute to any ViSIO-PRO domain score or the total score due to a high proportion of patients with RP/LCA not being able to drive. However, this ViSIO-PRO item was retained on the basis of evidence from previous research, to assess and monitor change in driving ability over time [7, 8].
Stage 3: Reliability and Validity of Scores
Reliability
ViSIO-PRO
All Cronbach’s alpha coefficients for the ViSIO-PRO total and domain scores were ≥ 0.80, indicating high internal consistency reliability (Table 10) [33]. The impact of ViSIO-PRO item/domain removal provided further support for internal consistency reliability, with no domains and only few items showing marginal increases using the alpha-if-item deleted method (Supplementary File 7 for all internal consistency reliability findings). The ViSIO-PRO items also had strong item–total correlations. Only item 40 (recognizing facial expressions) showed a correlation < 0.40 (with the distal HRQoL domain; Supplementary File 8 for all item–total correlations).
Intraclass correlation coefficients (ICCs) for the ViSIO-PRO total score and domain scores all surpassed 0.75 across each TRTAP (Table 10), indicating excellent test–retest reliability (TRT; Supplementary File 9).
ViSIO-ObsRO
Cronbach’s alpha coefficients for the ViSIO-ObsRO total score and domain scores were ≥ 0.70, indicating good internal consistency reliability (Table 11; Supplementary File 7). Using the alpha-if-item deleted method, all ViSIO-ObsRO Cronbach’s alpha coefficients were high and only marginally increased with the removal of select items/domains (Supplementary File 7). For item–total correlations, ViSIO-ObsRO items performed well, with most correlating highly with their corresponding domains (Supplementary File 8). However, four items in the distal HRQoL domain (items 21, 23, 26, and 27) had item–total correlation coefficients < 0.40 (0.08–0.36).
ICCs for the ViSIO-ObsRO total and domain scores all surpassed 0.50 indicating moderate to excellent TRT reliability across each TRTAP (Table 11; Supplementary File 9).
Construct Validity
Convergent and Discriminant Validity
Convergent validity was supported by a logical pattern of correlations between the ViSIO-PRO total score and scores from concurrent measures expected to be closely related, including the PGI-S (r = 0.77), VA scores (r = 0.54–0.60), and VFQ-25 composite score (r = −0.93). Moderate to strong correlations (r = 0.23 to −0.89) were also reported between the ViSIO-PRO domain scores and scores assessing similar concepts. These patterns of results were also observed for the ViSIO-ObsRO total score in respect to the CGI-S (r = 0.76), VA score (r = 0.54), and VFQ-25 composite score (r = −0.89).
Weak correlations between the ViSIO-PRO and ViSIO-ObsRO domain scores and scores from measures assessing dissimilar concepts indicated discriminant validity, for example, low correlations between the vision-dependent ADL domain in both the ViSIO-PRO and ViSIO-ObsRO and the EQ-5D-5L anxiety/depression score (0.10 and −0.05, respectively). Weak correlations were observed between the ViSIO-PRO and ViSIO-ObsRO scores and pain domains of concurrent measures, consistent with evidence that pain is not considered a symptom of RP/LCA [4,5,6,7,8], and none of the ViSIO-PRO or ViSIO-ObsRO scores assesses pain. For all convergent and discriminant validity outputs, see Supplementary File 10.
Known-Groups Validity
There was a statistically significant difference in mean ViSIO-PRO scores when groups were defined by PGI-S response (p < 0.001), with monotonically increasing mean scores with greater severity (Table 12) and large between-group ES. Statistically significant differences in mean ViSIO-PRO scores by age group (p < 0.001) were also observed with higher scores among patients in the older age groups as expected, consistent with the degenerative nature of RP/LCA. Known-groups comparisons for the ViSIO-PRO domains followed a similar pattern with monotonically increasing mean scores by PGI-S and age (Supplementary File 11).
Due to the small ObsRO sample size, a small number of parents/caregivers of child patients were in each of the CGI-S response categories and age groups, limiting interpretation. As expected, higher mean scores were observed in the CGI-S “Severe/Very severe” group compared with the “Mild/Moderate” group, with statistically significant differences between severity groups for the ViSIO-ObsRO total (Table 13) and most domain scores (Supplementary File 11). Mean scores did not differ significantly by age.
Stage 4: Preliminary Exploration of Interpretation of Scores
Distribution-Based Methods
Distribution-based estimates of meaningful change ranged between 0.30 and 0.65 for all ViSIO-PRO scores (Table 14) and 0.15–0.49 for all ViSIO-ObsRO scores (Table 15), indicating that changes above those magnitudes would be indicative of a true score change beyond measurement error.
Exploratory Analyses
Mean ViSIO-PRO domain and total scores at baseline were examined by genotype-defined subgroups with sufficient sample sizes (RLBP1 RP, RPE65 RP/LCA, and X-linked/RPGR RP). All other genotypes were grouped together due to smaller sample sizes (n < 10).
There were no clear differences between RP/LCA genotypes based on ViSIO-PRO scores, with the exception of statistically significant differences across subgroups for the mobility domain score. The RLBP1 RP subgroup showed the highest mean scores across all domains and the total score, indicating worse visual functioning or HRQoL compared with other genotypes, although a higher mean age could be observed in the RLBP1 RP subgroup compared with the other genotypes (Supplementary File 12).
Exploratory analyses were not performed for the ViSIO-ObsRO due to small sample size.
Discussion
The ViSIO-PRO and ViSIO-ObsRO are the first known RP/LCA-specific instruments developed for use with adult/adolescent patients with RP and LCA and caregivers of children with RP/LCA. Qualitative evidence supporting the content validity of the instruments has been demonstrated previously, across a range of RP/LCA genotypes [5, 7, 8]. However, according to regulatory guidance, to be considered fit-for-purpose, there must also be adequate evidence supporting the psychometric properties of the ViSIO-PRO and ViSIO-ObsRO in their intended context of use [10,11,12].
To address this, the current study aimed to evaluate the psychometric properties of the ViSIO-PRO and ViSIO-ObsRO instruments using data collected from a non-interventional, observational study in France, Germany, Canada, Denmark, and the USA.
ViSIO-PRO
Results provide strong evidence for the psychometric validity and scoring of the ViSIO-PRO instrument, supporting its use in broader RP/LCA and as an appropriate outcome assessment in future clinical trials, other large research studies, or clinical practice.
Some floor and ceiling effects were observed; however, this was anticipated given differing severities across genotypes and the progressive nature of RP/LCA. Moderate to strong inter-item correlations were reported between items within each hypothesized domain. Strong correlations were not considered problematic, as item pairs with very strong correlations typically assessed similar concepts in different environmental conditions, an important feature of the ViSIO instruments. Strong correlations also likely reflect the nature of the condition, with visual functioning underpinning the majority of concepts assessed across ViSIO-PRO domains.
There was good representation of different RP/LCA genotypes in the sample, including coverage of the topmost prevalent disease-causing genes (e.g., RHO, PRPF31, PRPH2, EYS, USH2A, RPGR, and RP2) [3, 34] and those treated with voretigene neparvovec (Luxturna). The findings are therefore considered adequate to support conclusions regarding the psychometric properties, dimensionality, and scoring of the ViSIO-PRO across multiple genotypes of RP/LCA. Item response distributions, dimensionality analyses, and previous qualitative interviews, alongside input from clinical experts informed the removal of several items, resulting in a 35-item instrument. Bifactor analysis showed adequate fit for a mean total score, in addition to four domain scores (visual function symptoms, vision-dependent ADL, mobility, and distal HRQoL). Domain and total scores performed well in tests of reliability, demonstrating good internal consistency reliability and TRT reliability. Construct validity was also supported, with significant differences in mean scores by severity and age group reported, with higher scores observed for older patients, in line with the degenerative nature of the disease. Exploratory analyses indicated that there were no clear differences in scores between RP and LCA genotypes, supporting the use of the ViSIO-PRO with patients with a broad range of RP/LCA genotypes; an important consideration for a condition with such an extensive and expanding number of gene mutations involved. This finding was consistent with the qualitative research that highlighted that similar visual function symptoms, impacts on vision-dependent ADL and mobility, and impacts on distal HRQoL were reported among patients with different gene mutations [5, 7, 8].
ViSIO-ObsRO
Analysis of the ViSIO-ObsRO was limited by the sample size due to challenges associated with recruiting children aged 3–11 years with a rare and progressive condition such as RP/LCA. However, initial evidence generated supports the psychometric validity of the ViSIO-ObsRO and established dimensionality and scoring, supporting its use in future clinical practice to demonstrate maintenance or rate of deterioration of visual functioning in pediatric populations.
Item responses at baseline were well distributed for most items; however, some demonstrated very high ceiling effects. Given the degenerative nature of RP/LCA and that severity of disease is often milder in pediatric populations, ceiling effects were expected and did not necessarily indicate problematic items [1, 35].
Due to the limited sample size, a more cautious approach was taken towards item deletion and a scoring algorithm consistent with the ViSIO-PRO was applied. The resulting total score and four domain scores demonstrated good internal consistency and TRT reliability as well as good convergent, discriminant, and known-groups validity.
Limitations
For both the ViSIO-PRO and ViSIO-ObsRO instruments, analyses were limited by the small sample size for psychometric validation. Given the rarity of the disease, the study achieved a reasonable sample size and did include good representation of different RP/LCA genotypes; however, representation of certain genotypes in the sample was limited. Collection of additional data in future studies are recommended to further confirm validity across genotypes and cross-culturally in a larger sample.
Nevertheless, in this sample, a large variety of genotypes including RPE65 RP, RLBP1 RP, and RPGR X-linked RP were well represented. The study was conducted exclusively within a non-syndromic RP/LCA population, supporting use of the ViSIO instruments within this specific population across RP/LCA genotypes. This is unlike other PRO instruments developed, such as the Michigan Retinal Degeneration Questionnaire (MRDQ), which have not been specifically validated in RP/LCA but rather developed to capture the visual function symptoms occurring across a range of IRDs and, thus, may assess concepts that lack relevance or fail to comprehensively assess all concepts of importance to the RP/LCA population [36, 37].
The analyses were also limited by the lack of objective, clinical measures available for use as anchors in the psychometric analyses, specifically for the construct validity analyses. However, the anchors used were selected as they were expected to be sufficiently correlated with the ViSIO-PRO and ViSIO-ObsRO instruments and better reflected aspects of visual functioning and HRQoL compared with objective, clinical measures that differed in use across clinical sites or were reported as missing.
Finally, within the current dataset it was not possible to examine the ability of the instruments to detect changes over time, particularly in response to treatment. Future analyses using pre- and post-treatment data are planned with RPE65 patients receiving voretigene neparvovec gene therapy. Data will be analyzed to provide initial insights to the sensitivity of the ViSIO-PRO and ViSIO-ObsRO to changes over time.
Conclusion
The findings support the 35-item ViSIO-PRO and 25-item ViSIO-ObsRO as valid and reliable measures across RP/LCA genotypes, to serve as outcome assessments in future clinical trials or track disease severity in clinical practice. Further evidence is required to establish the ability of the ViSIO-PRO and ViSIO-ObsRO to detect changes over time and to inform meaningful change thresholds.
References
Rivolta C, Sharon D, DeAngelis MM, Dryja TP. Retinitis pigmentosa and allied diseases: numerous diseases, genes, and inheritance patterns. Hum Mol Genet. 2002;11(10):1219–27.
Ferrari S, Di Iorio E, Barbaro V, Ponzin D, Sorrentino FS, Parmeggiani F. Retinitis pigmentosa: genes and disease mechanisms. Curr Genomics. 2011;12(4):238–49.
Fahim AT DS, Weleber RG. Nonsyndromic retinitis pigmentosa overview. In: Adam MP, Ardinger HH, Pagon RA, et al. (eds) GeneReviews. University of Washington, Seattle; 1993–2020. https://www.ncbi.nlm.nih.gov/books/NBK1417/ed2017.
Hartong DT, Berson EL, Dryja TP. Retinitis pigmentosa. Lancet. 2006;368(9549):1795–809.
Kay C, Williamson N, Bradley H, Barclay M, Sims J, Arbuckle R, et al. Qualitative interviews with patients and caregivers regarding visual function impairments and impacts on vision-dependent activities of daily living and health-related quality of life in RPE65-related Retinitis Pigmentosa and Leber Congenital Amaurosis. ARVO 20212021.
Green J, Tolley C, Bentley S, Arbuckle R, Burstedt M, Whelan J, et al. Qualitative interviews to better understand the patient experience and evaluate patient-reported outcomes (PRO) in RLBP1 retinitis pigmentosa (RLBP1 RP). Adv Ther. 2020;37(6):2884–901.
Green J, Williamson N, Bradley H, Barclay M, Sims J, Arbuckle R, et al. Qualitative exploration of patient experiences of visual function impairments and health-related quality of life impacts associated with RLBP1 retinitis pigmentosa in a sample of Canadian patients. ISPOR 2021.
Audo I, Williamson N, Bradley H, Barclay M, Sims J, Arbuckle R, et al. Qualitative exploration of patient and caregiver experiences of visual function impairments and impacts on vision-dependent activities of daily living and health-related quality of life associated with retinitis pigmentosa and leber congenital amaurosis in Germany and France. ARVO 20212021.
Tolley C, Mullins A, Kilgariff S, Arbuckle R, Green J, Burstedt M, et al. Qualitative interviews to inform development of a patient reported outcome (PRO) strategy in RLBP1 retinitis pigmentosa (RLBP1 RP). Value Health. 2017;20(9):A761.
Food and Drug Administration. FDA Patient-Focused Drug Development Guidance Series for Enhancing the Incorporation of the Patient’s Voice in Medical Product Development and Regulatory Decision Making 2020 [Available from: https://www.fda.gov/drugs/development-approval-process-drugs/fda-patient-focused-drug-development-guidance-series-enhancing-incorporation-patients-voice-medical.
Food and Drug Administration. Guidance for industry: patient-reported outcome measures: use in medical product development to support labeling claims. December 2009.
European Medicines Agency (EMA). Reflection paper on the regulatory guidance for the use of health-related quality of life (HRQL) measures in the evaluation of medicinal products 2005. Available from: http://www.ema.europa.eu/docs/en_GB/document_library/Scientific_guideline/2009/09/WC500003637.pdf.
Food and Drug Administration. Patient-Reported Outcome Measures: Use in Medical Product Development to Support Labeling Claims. Guidance for Industry [Internet]. 2009. Available from: https://www.fda.gov/downloads/drugs/guidances/ucm193282.pdf.
Streiner DL, Norman GR, Cairney J. Health measurement scales: a practical guide to their development and use. Oxford University Press; 2015.
Mangione CM, Lee PP, Gutierrez PR, Spritzer K, Berry S, Hays RD, et al. Development of the 25-item national eye institute visual function questionnaire. Arch Ophthalmol. 2001;119(7):1050–8.
Mangione C. NEI VFQ-25 scoring algorithm. Rockville: National Eye Institute; 2000.
Pickard AS, Law EH, Jiang R, Pullenayegum E, Shaw JW, Xie F, et al. United States valuation of EQ-5D-5L health states using an international protocol. Value Health. 2019;22(8):931–41.
Devlin NJ, Shah KK, Feng Y, Mulhern B, van Hout B. Valuing health-related quality of life: an EQ-5D-5L value set for England. Health Econ. 2018;27(1):7–22.
Kline RB. Principles and practice of structural equation modeling. 4th ed. Guilford Publications; 2015.
Goodyear MD, Krleza-Jeric K, Lemmens T. The Declaration of Helsinki. BMJ. 2007;335(7621):624–5.
Wild D, Grove A, Martin M, Eremenco S, McElroy S, Verjee-Lorenz A, et al. Principles of good practice for the translation and cultural adaptation process for patient-reported outcomes (PRO) measures: report of the ISPOR Task Force for Translation and Cultural Adaptation. Value Health. 2005;8(2):94–104.
Hu LT, Bentler PM. Cutoff criteria for fit indexes in covariance structure analysis: conventional criteria versus new alternatives. Struct Eq Model. 1999;6(1):1–55.
Samejima F. Graded response model. In: Hambleton RK, editor. van der Linden WJ Handbook of modern item response theory. Springer; 1997. p. 85–100.
Van der Linden WJ, Hambleton R. Handbook of item response theory. Taylor & Francis Group Citado Na Pág. 1997;1(7):8.
Rodriguez A, Reise SP, Haviland MG. Evaluating bifactor models: calculating and interpreting statistical indices. Psychol Methods. 2016;21(2):137.
Reise SP. The rediscovery of bifactor measurement models. Multivar Behav Res. 2012;47(5):667–96.
Bernstein I. Psychometric theory, 3rd. New York: McGraw Hill; 1994.
Nunnally JC. Psychometric theory 3E: Tata. McGraw-Hill Education; 1994.
Cohen J. Statistical power analysis for the behavioral sciences. Taylor & Francis; 2013.
McLeod LD, Coon CD, Martin SA, Fehnel SE, Hays RD. Interpreting patient-reported outcome results: US FDA guidance and emerging methods. Expert Rev Pharmacoecon Outcomes Res. 2011;11(2):163–9.
Fayers PM MD. Quality of life: the assessment, analysis, and interpretation of patient-reported outcomes.
Reise SP, Moore TM, Haviland MG. Bifactor models and rotations: exploring the extent to which multidimensional data yield univocal scale scores. J Pers Assess. 2010;92(6):544–59.
Nunnally J. Psychometric theory. 3rd ed. Tata McGraw-Hill Education; 2010.
Mansfield BC, Yerxa BR, Branham KH. Implementation of a registry and open access genetic testing program for inherited retinal diseases within a non-profit foundation. Am J Med Genet Part C Semin Med Genet. 2020;184:838–45.
Foxman SG, Heckenlively JR, Bateman JB, Wirtschafter JD. Classification of congenital and early onset retinitis pigmentosa. Arch Ophthalmol. 1985;103(10):1502–6.
Lacy GD, Abalem MF, Andrews CA, Popova LT, Santos EP, Yu G, et al. The Michigan retinal degeneration questionnaire: a patient reported outcomes instrument for inherited retinal degenerations. Am J Ophthalmol. 2020;222:60–8.
Lacy G, Abalem MF, Baig N, Rakine H, Andrews CA, Ehrlich JR, et al. Development of a validated patient-reported outcomes measure for retinal dystrophy therapeutics. Investig Ophthalmol Vis Sci. 2020;61(7):1564.
Acknowledgements
Funding
Novartis provided funding for the design of this study, collection, analysis and interpretation of data and for the writing of this manuscript. Novartis also provided funding for the journal’s Rapid Service Fee.
Medical Writing, Editorial, and Other Assistance
Rob Arbuckle (Adelphi Values), Kieran Boparai (Adelphi Values), Jan Stochl (Adelphi Values) and Claudio Spera (Novartis Pharma AG) contributed to the study design, study conduct, the acquisition and interpretation of data.
Author Contributions
M. Dominik Fischer, Francesco Patalano, Christel Naujoks, Judit Banhazi, Paul O’Brien, Christine Kay, Jane Green, Todd Durham, Christine Bouchet and Isabelle Audo contributed to the conception and design of the work, the acquisition and interpretation of data; and have drafted the work or substantively revised it. Helena Bradley, Nicola Williamson, Melissa Barclay and Joel Sims contributed to the conception and design of the work, the acquisition, analysis and interpretation of data; and have drafted the work or substantively revised it. The assistance was funded by Novartis. All authors read and approved the final manuscript.
Disclosures
Christel Naujoks, Christine Bouchet and Francesco Patalano are employees of Novartis Pharma AG, Basel, Switzerland. Judit Banhazi was an employee of Novartis Pharma AG, Basel, Switzerland at the time the research was conducted. Paul O’Brien was an employee of Novartis Ireland Ltd, Dublin, Ireland at the time the research was conducted. Helena Bradley, Nicola Williamson, Melissa Barclay and Joel Sims are employees of Adelphi Values, Cheshire, UK, a health outcomes agency commissioned to conduct research by companies in the pharmaceuticals industry. Adelphi Values received funding from Novartis to conduct the research summarized in this article. Todd Durham was employed by Novartis as an advisor and received a fee for their involvement. M. Dominik Fischer, Isabelle Audo and Jane Green were paid consultants of Adelphi Values on behalf of Novartis as scientific advisors and received a fee for their involvement. M. Dominik Fischer further reports consulting fees from Advent France Biotechnology, Alphasights, Atheneum, Axiom Healthcare Strategies, Biogen, Decision Resources, Dialectica, Frontera Therapeutics, Janssen Research & Development, Navigant, Novartis, Roche, Sirion, STZ eyetrial. Isabelle Audo is also a consultant for Novartis, Sparing Vision, Roche and Biogen. Christine Kay reports consultancy/investigator fees from AGTC, Foundation Fighting Blindness, Alkeus, Gyroscope, REGENXBIO, Nightstar Therapeutics/Biogen, Spark therapeutics, Novartis, Iveric Bio, ProQR Therapeutics, MeiraGTx, Janssen, Atsena Therapeutics, 4D Molecular Therapeutics and Kodiak. The authors declare that there are no other competing interests.
Compliance with Ethics Guidelines
Centralized ethical approval and oversight for this study was provided by Copernicus Group Independent Review Board (CG-IRB) to conduct the study in the US and Denmark (approval number: 20190129). Local ethical approval was provided by the Comité de protection des personnes (CPP) and Recherches et Collections Biologiques (RCB) in France (RCB ID: 2019-A01883-54, Réf. CPP: 19078-43420) and by specialist clinical sites in Germany (Universitätsklinikum Tübingen: 343/2019B01) and Canada (Newfoundland and Labrador Health Research Ethics Authority (HREA): #2019.186). See Supplementary File 13 for ethical approval information. Documentation of informed consent was obtained prior to any research activities being conducted. The study was performed in accordance with the Helsinki Declaration of 1964, and its later amendments.
Data Availability
The datasets generated during and/or analyzed during the current study are available from the corresponding author on reasonable request. The ViSIO-PRO and ViSIO-ObsRO are available for use under a formal licensing agreement. Please contact amanda.rosett@rws.com to request permission for use or for additional information.
Author information
Authors and Affiliations
Corresponding author
Supplementary Information
Below is the link to the electronic supplementary material.
Rights and permissions
Open Access This article is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License, which permits any non-commercial use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by-nc/4.0/.
About this article
Cite this article
Fischer, M.D., Patalano, F., Naujoks, C. et al. Psychometric Validation of the ViSIO-PRO and ViSIO-ObsRO in Retinitis Pigmentosa and Leber Congenital Amaurosis. Ophthalmol Ther 12, 1359–1386 (2023). https://doi.org/10.1007/s40123-023-00670-8
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s40123-023-00670-8