RLBP1 RP is an autosomal recessive form of retinitis pigmentosa (RP), characterized by night blindness, prolonged dark adaptation, constricted visual fields and impaired macular function. This study aimed to better understand the patient experience of RLBP1 RP and evaluate the content validity of existing patient reported outcome (PRO) instruments in this condition.
Semi-structured concept elicitation and cognitive debriefing interviews were conducted with RLBP1 RP patients in Canada and Sweden. Interviews started with open-ended concept elicitation questioning, and then patients were cognitively debriefed on The National Eye Institute Visual Functioning Questionnaire (NEI VFQ-25), the Low Luminance Questionnaire (LLQ) and four light/dark adaptation items of the Visual Activities Questionnaire (VAQ). Qualitative interviews were also conducted with three expert clinicians. Anonymized, verbatim transcripts were analyzed using thematic analysis.
Twenty-one patients were interviewed (Canada n = 10; Sweden n = 11). Symptoms reported included night blindness (n = 21), difficulty adapting to changes in lighting (n = 21) and difficulties seeing in bright lighting (n = 18). Patients experienced substantial impacts on daily activities (n = 21) and physical functioning (n = 17). Patients had difficulty interpreting and selecting a response for some items in the NEI VFQ-25 and LLQ. Some items were not relevant to patients’ disease experience. There were both gaps and overlaps in the conceptual coverage of the instruments.
Visual impairment due to RLBP1 RP has a substantial impact on physical functioning and daily activities. To adequately assess all important symptoms and associated functional impacts in RLBP1 RP, it is recommended to either modify one or more existing instruments or to develop a new non-syndromic RP specific instrument.
|Why carry out this study?|
|The symptoms of RLBP1 RP have a substantial impact on patients’ daily lives and physical functioning.|
|Semi-structured concept elicitation and cognitive debriefing interviews were conducted with RLBP1 RP patients (n = 21) in Canada and Sweden to better understand the patient experience of RLBP1 RP and evaluate the content validity of three existing patient reported outcome (PRO) instruments in this condition.|
|What was learned from the study?|
|Symptoms of RLBP1 RP reported included night blindness, difficulty adapting to changes in lighting and difficulties seeing in bright lighting, and patients experienced substantial impacts on daily activities and physical functioning.|
|When asked to complete three different questionnaires, patients with RLBP1 RP highlighted problems interpreting some questions and response options from all three PRO instruments, and no one instrument provided a comprehensive assessment of concepts relevant to patients with RLBP1 RP.|
|To adequately assess all important symptoms and associated functional impacts in this population, either modifying one or more existing PRO instruments or developing a new disease specific PRO instrument is recommended.|
RLBP1 retinitis pigmentosa (RLBP1 RP) is an autosomal recessive form of retinitis pigmentosa (RP), caused by biallelic mutations of the RLPB1 gene on chromosome 15. RLBP1 RP is associated with progressive loss of vision due to retinal degeneration and is a rare disease , with only 152–160 cases worldwide reported in the literature [2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18]. Sweden and Canada have the highest number of diagnosed cases, and in each country the cases are clustered in a specific geographic area: Västerbotten County in Sweden and Newfoundland in Canada.
Symptoms of RLBP1 RP are a result of progressive loss of rod and cone photoreceptor cells . The phenotype caused by mutations identified in RLBP1 RP is most often referred to as Bothnian dystrophy (BD) , Newfoundland rod-cone dystrophy (NFRCD)  or retinitis punctata albescens (RPA) . RLBP1 RP is characterized by severe night blindness from childhood due to prolonged dark adaptation, constricted visual fields (VF) and reduced macular function with decreasing visual acuity (VA) in early adulthood . The impact of RLBP1 RP on quality of life (QoL) can largely be attributed to visual impairment. Though RLBP1 RP is limited only to retinal degeneration, it affects patients’ ability to perform vision-dependent functions of everyday life and has a profound effect on physical, social and psychological wellbeing.
A patient-reported outcome (PRO) instrument measures information that comes directly from the patient about the status of their health condition, without amendment or interpretation of the patient’s report by a clinician or anyone else [4, 23]. PRO instruments can assess symptoms, impacts, functioning, and treatment satisfaction and adherence, providing insight into concepts that cannot be measured objectively through biological or functional assessments.
The Food and Drug Administration (FDA) PRO guidance  outlines best practices for developing PRO instruments for use to support labeling claims. As a first step in evaluating the appropriateness of an instrument, it is crucial to ensure, through qualitative patient research, that the instrument assesses concepts relevant to the patient population and to ascertain patients’ comprehension of the questionnaire [5, 23].
Although it is understood that RLBP1 RP affects patients’ visual functioning and therefore activities of daily living (ADL), qualitative evidence of this in the literature is minimal. Currently, no PRO instruments have been validated in an RP population. Several non-disease specific PRO instruments that assess the impact of visual impairment have been used in RP and other visual conditions such as glaucoma, including the National Eye Institute Visual Functioning Questionnaire (NEI VFQ-25) [25, 26], the Low Luminance Questionnaire (LLQ)  and the Visual Activities Questionnaire (VAQ) .
Previous studies in RLBP1 RP provide some evidence that NEI VFQ-25 scores are associated with VA, VF and LLQ scores in this population [22, 29, 30]. However, other than these preliminary insights into construct validity, there is limited evidence regarding the content and psychometric validity of these instruments in this specific population. Moreover, neither the NEI VFQ-25 nor the LLQ assesses the process of light-dark adaptation, a key characteristic of RLBP1 RP. The VAQ contains a four-item dark adaptation subscale.
This study aimed to conduct qualitative research with participants diagnosed with RLBP1 RP and expert clinicians to: (1) further understand participants’ experiences with visual symptoms and the associated impacts on domains of functioning, (2) evaluate the content validity of the NEI VFQ-25, LLQ and four items from the VAQ (assessing light/dark adaptation) in this population; (3) obtain clinical insight into the experiences of participants with RLBP1 RP and the clinical relevance of the concepts assessed by the selected instruments.
This was a cross-sectional, qualitative study involving individual, semi-structured, concept elicitation (CE) and cognitive debriefing (CD) interviews with patients with RLBP1 RP. In addition, two pilot interviews were conducted with adult patients who had RP not associated with a mutation in the RLBP1 gene to test the interview process and allow for comparisons between the two patient groups. Semi-structured interviews were also conducted with expert clinicians with experience of managing and treating patients with RLBP1 RP.
Twenty adults and one child (aged 11 years) with RLBP1 RP were recruited in Canada (n = 10) and Sweden (n = 11) through expert clinicians specialized in RLBP1 RP. All participants had a clinical diagnosis of RP with documentation of the RLBP1 gene mutations. No quotas for representation of subgroups were employed given the rare nature of the condition. The two RP patients participating in the pilot interviews were recruited from Switzerland and Ireland and had been identified via Retina International (an umbrella group of 43 patient-led charities, foundations and voluntary groups funding research into retinal dystrophies). Expert clinicians (n = 3) from the recruiting clinical sites in Canada and Sweden were also interviewed.
The adequacy of qualitative sample sizes is commonly evaluated based on the principal of conceptual saturation, defined as the point when no new concept-relevant information emerges with the analysis of additional interviews [31, 32]. Typically, saturation can be achieved with 12 participants; 10 participants per country were targeted because of possible heterogeneity of the populations and to allow for between country comparisons.
Patient Interview Procedure
The majority of interviews were conducted face-to-face; however, the two participants in the pilot interviews and two participants in Canada were interviewed via Skype. The patient interviews lasted approximately 90-min (45-min concept elicitation and 45-min cognitive debriefing) using a semi-structured interview guide. Concept elicitation was completed prior to review of the PRO instruments to avoid bias.
Concept elicitation questioning explored participants’ experiences of RLBP1 RP. Questioning started with broad open-ended questions designed to elicit spontaneous comments, followed by focused questions to probe on topics of interest.
The NEI VFQ-25, LLQ and four items from the VAQ were then cognitively debriefed. These instruments were selected from a review of existing PRO instruments that have previously been used in RP and similar conditions. The review considered prior use of the instruments and evidence of content and psychometric validity. The instruments selected had the greatest potential to assess the concepts most important and relevant to RP patients. Cognitive debriefing involved use of a ‘think aloud’ approach, where participants spoke their thoughts aloud as they completed each item. Participants were also asked debriefing questions to explore their interpretation of the items and response options, to evaluate the relevance of concepts assessed by the instruments and to assess conceptual comprehensiveness of the instruments.
Clinician Interview Procedure
The 30-min clinician interviews were conducted via telephone. One was conducted prior to patient interviews to help inform the content of the patient interview guide, and two were conducted following patient interviews to assess the clinical relevance of reported concepts. Key symptoms and functional impacts of RLBP1 RP were discussed, as were clinicians’ perceptions of the appropriateness of the NEI VFQ-25, LLQ and four items from the VAQ for use in the RLBP1 RP population.
Interviews were anonymized and transcribed verbatim. Swedish transcripts were translated to English. Qualitative analysis was performed using Atlas.Ti software  and thematic analysis methods [34,35,36]. For concept elicitation, this focused on identifying concepts important to participants and the language participants used to describe those concepts. Saturation (the point at which no new concepts are emerging from the data) was evaluated by comparing spontaneously reported concepts from successive sets of interviews. A conceptual model was generated to summarize the key concepts relevant for assessment in RLBP1 RP. A conceptual model is useful for presenting a concise overview of the patient experience of a condition and for prioritizing, identifying and evaluating measurement strategies .
Ethical approval was obtained from the Newfoundland and Labrador Health Research Ethics Board in Canada (reference no. 2016.224) and Umea University in Sweden (reference no. 2016/357-31). Written informed consent from each participant was obtained prior to data collection.
Patient Sample Characteristics
Demographic and clinical characteristics of the RLBP1 RP sample are summarized in Table 1. Mean age was 45 years (range 11–67 years), and there were marginally more males (male 12/21, female 9/21). According to clinical data provided by clinicians, participants were diagnosed with RLBP1 RP on average 14 years after experiencing initial symptoms of RP. Participants were assigned severity ratings (mild, moderate, severe or very severe) based on their VA score and Visual Field Index (VFI). The VA scores were converted to a severity rating based on the adjusted World Health Organization (WHO) categories of visual impairment . The VFI categories were informed by published data regarding VFI severity categories in glaucoma . Visual function scores for right and left eyes were summarized into a single weighted average where the better eye is weighted 0.75 and the worse eye 0.25. The sample included a wide range of VA severities, but most had severe or very severe VFI ratings (Table 2).
Patient Concept Elicitation Results
Figure 1 presents the conceptual model developed based on the interview findings to illustrate the key concepts and impact on functioning relevant to patients with RLBP1 RP. Fourteen visual symptoms were reported during concept elicitation. The symptoms most frequently discussed were:
Night blindness (n = 21; all spontaneously reported). The majority of participants described being unable to see anything in dark conditions (n = 15/21). The remaining six participants reported that they had limited vision in darkness. Those that described being unable to see at all used phrases such as “can’t see anything” and “can’t see at all” (n = 10/15), “night blindness” (n = 4/15) and “see absolutely nothing” (n = 1/15).
“For me when it gets darker, I see less and less. And all of a sudden, I can’t see anything at all” (Male, 20).
Difficulties adapting to changes in lighting (n = 21; all spontaneously reported). All of the participants reported that they had difficulties adapting to changes in light when moving from brightly lit areas to dimly lit areas, and/or vice versa. Eleven participants reported difficulty adapting in either direction; eight participants described difficulty moving from light to dark areas only, and two participants described difficulty moving from dark to light areas only.
“Sometimes if I leave a dark room and enter say a lit office, that can be sometimes a little bit excruciating” (Male, 40).
Difficulties seeing in bright lighting (n = 18; spontaneous n = 15, probed n = 3). Participants reported that in bright lighting conditions they were no longer able to see (n = 12/18) or found it more difficult to see (n = 2/18). Five participants also reported that they were ‘sensitive’ to bright light (n = 5/18). Terminology used included ‘blinded’ (n = 8/18), ‘can’t see’ or ‘difficulty seeing’ (n = 3/18). Five participants also referred to this symptom as difficulty with ‘glare.’
“Sunlight is the worst, it’s the most intensive. It blinds me the most” (Male, 25).
Poor visual acuity (n = 15; all spontaneously reported). Participants used a variety of terms to describe low visual acuity. Approximately half of participants (n = 8/15) directly used the term “low visual acuity.” Other terms used included describing their vision becoming ‘blurry’ (n = 2/15), ‘farsighted’ (n = 1/15) and ‘myopic, near-sighted’ (n = 1/15).
“I have sight loss, my visual acuity has decreased. Especially in the middle of the eye, it’s becoming increasingly difficult to read” (Female, 28).
Impaired color vision (n = 13; spontaneous n = 11, probed n = 2). Thirteen participants described difficulty in seeing and distinguishing between colors. Six participants discussed the specific colors that they have a problem distinguishing. The majority (n = 5/6) reported that they are unable to distinguish between dark colors, e.g., between a navy blue and black.
“I’ve lost the part you see colors with, so I don’t see colors. I see TV in black and white” (Female, 58).
Loss of peripheral vision (n = 13; spontaneous n = 9, probed n = 4). Thirteen participants reported experiencing difficulties with their peripheral vision, eight of whom used or were familiar with the term ‘peripheral vision’ (n = 8/13). Four participants (n = 4/13) described instances where they were unable to see objects when they were presented to the side of their visual field. Two participants (n = 2/13) additionally used the terms ‘tunneling’ (n = 1/13) or ‘tunnel vision’ (n = 1/13) to describe their visual field.
“It seems like just tunnel vision is all I can see. I can’t really see in the side of me much anymore, it’s just kind of straight, central” (Female, 35).
Participants across the sample appeared to experience a similar core set of symptoms of RLBP1 RP that worsened in severity over time. Participants also described environmental factors that affected the symptoms reported; these included weather and lighting conditions (n = 16/21) and unfamiliar environments (n = 16/21).
All participants discussed the impact of RLBP1 RP on ADL. Impacts were separated into proximal impacts and distal impacts. Proximal impacts were considered to be any functional impacts directly related to vision impairments associated with RLBP1 RP (e.g., reading). Distal impacts were considered to be any impacts that occurred as a result of the proximal impacts that had wider implications for participants’ daily life (e.g., work, reading menus, etc.).
The most frequently reported proximal impacts on ADL included:
Difficulty reading (n = 17; spontaneous n = 12, probed n = 5). Impact on ability to read was affected by font style (n = 6/17), size of the text (n = 4/17) and the color contrast of the text and background (n = 3/17). Reduced reading ability had a subsequent impact on various aspects of life such as work, education, social life, shopping and cooking. Three participants also expressed difficulty in navigating because of their reduced ability to read signs. Methods to manage this impact on reading were discussed by seven participants and included using electronic devices such as computers, use of magnifiers and use of audio books.
“I read a lot using a tablet where you can zoom in the text” (Female, 28).
Being unable to drive or having difficulties driving (n = 15; all spontaneously reported, only one participant was still able to drive). The remaining six participants did not discuss the impact of RLBP1 RP on their ability to drive and were not asked. Reasons for being unable to drive described by 14 participants included poor nighttime vision (n = 5/14), the eyes not being able to adjust (n = 1/14), experiencing blind spots (n = 1/14) and defects in peripheral vision (n = 2/14). Four participants reported the impact of not being able to drive on their daily life; these included the need for advanced planning and the impact on family life. Five participants talked about requiring assistance from family and friends to drive them around. Four participants reported feelings of frustration, sadness, depression and devastation due to their inability to drive.
“Something that is very bothersome is that I don’t have a driver’s license. I can’t drive” (Male, 54).
Problems with chores and cleaning (n = 13; spontaneous n = 12, probed n = 1): Participants expressed the need for assistance in carrying out chores and cleaning (n = 6/13). Participants also made clear that their ability has deteriorated overtime (n = 3/13), with chores taking considerably longer to complete (n = 3/13). Participants also mentioned the importance of adaptations and routines when completing chores (n = 4/13).
“Cleaning too is another one. I miss a lot of things because I can’t see a lot of the things” (Female, 35).
Problems navigating in unfamiliar environments (n = 11; all spontaneously reported). Participants indicated that difficulty to navigate was often caused by darkness (n = 5/11) and the inability to see obstacles (n = 5/11). Five participants talked about specific strategies they often use when navigating. Such strategies included following familiar streetlights or house lights, use of technology, being organized and memorizing routes. Difficulties navigating affected participants ability to travel to appointments (n = 1/11) and their social life (n = 1/11).
“At the house it’s not too bad because I know where everything is, but especially if I were in a strange place, I’d need some form of lighting or know the layout of the room. Otherwise, I could fall down or trip over something” (Male, 47).
The most frequently reported proximal physical impacts included:
Reduced ability to take part in sports (n = 11; spontaneous n = 3, probed n = 8). Eight of the 11 participants reported that they were unable to take part in certain sports and activities, and three participants reported that they could still take part but were limited in their ability. Examples of sports and activities reported by more than one participant included: hockey (n = 5/11), football (n = 3/11), riding a bike (n = 3/11), baseball (n = 2/11) and swimming (n = 2/11).
“I do some sport, if it’s adapted, but, you know, regular sports are not really an option” (Male, 40).
Walking into objects (n = 10; all spontaneously reported): More frequently reported examples of objects that participants had walked into because of their impaired vision included: obstacles associated with road works (n = 3/10), other people (n = 3/10) and objects on the floor at home (e.g., furniture and bins; n = 2/10).
“We would do things together, and I could not see on this side, and we would bump into each other all the time” (Female, 58).
Having difficulties with their mobility due to poor vision (n = 8; all spontaneously reported): Three of the eight participants reported that they had difficulty generally getting around going to work or the supermarket, three participants reported difficulties going up or down stairs, one participant reported difficulties with outdoor activities, and one participant reported having to ‘walk more carefully.’
“I guess steps, the depth, the difference between the steps… I can’t see going down steps, but I can see sometimes going up steps” (Female, 63).
Wider impacts on quality of life were also identified, including:
Impacts on social functioning (n = 21; all spontaneously reported). Social impacts included having to rely on others for help (n = 14/21), difficulties going out at night time to social events (n = 13/21), missing out on social events (n = 12/21), problems recognizing people (n = 6/21), problems attending the cinema (n = 6/21) or restaurants (n = 5/21), being unable to help family members (n = 5/21), missing out on activities with children/grandchildren (n = 4/21) and impacts on current/future relationships (n = 4/21).
“Uh, well, uh, I guess, I used to, uh, go out with friends and to parties and such and I usually depended on some of them to, uh, guide me around” (Male, 29).
Emotional wellbeing (n = 19; all spontaneously reported). Emotions experienced by participants because of their condition in order of frequency included frustration (n = 11/19), sadness (n = 10/19), concerns about the future (n = 8/19), depression (n = 8/21), fear (n = 5/19), anger (n = 5/19), stress (n = 5/19), embarrassment (n = 4/19) and loneliness (n = 2/19).
“So, I just feel like I’m a big burden on people a lot of the times, and it’s very frustrating. It gets very emotional for me” (Female, 35).
Work and education (n = 18; spontaneous n = 13, probed n = 5). Participants described adapting their work role because of their condition (n = 10/18), having a limited choice of career or job (n = 10/18), being unable to continue working in a particular job (n = 7/18) and having difficulties at work related to their ability to read (n = 6/18).
“And then my workday, like I said, I’ve switched my duties so that I can be, um, able to do my job without having need perfect vision. So, I’m in my office most of the day. I do have, um, a setting on my computer that reverses the contrast of the lighting so it’s not so bright” (Female, 32).
An important aspect of the qualitative analysis was to evaluate the degree of consistency in the findings to determine whether conceptual saturation had been achieved among the concept elicitation interviews and that all concepts of importance had been identified. It was concluded that saturation was achieved for all core symptoms and impacts of RLBP1 RP.
As would be expected, participants with more severe RLBP1 RP (with regards to visual acuity and visual fields) appeared to have a more severe experience of certain symptoms. For example, participants categorized as severe and very severe took longer to adapt to changes in lighting than mild or moderate participants. In general, participants categorized as severe and very severe reported a greater impact of RLBP1 RP compared with mild and moderate participants. Minor differences in impacts experienced by the Swedish and Canadian populations were observed. These are likely a result of differences in the sample severities or lifestyles of the two populations rather than differences in disease manifestation.
Pilot Interview Findings
The findings from the interviews with RP patients did not identify any notable differences between the experiences of patients with RLBP1 RP and those with ‘general RP’ interviewed in pilot interviews. However, as only two pilot interviews were conducted, conclusions regarding these comparisons should be interpreted with caution.
Cognitive Debriefing Results
Challenges interpreting items identified for the NEI VFQ-25 and LLQ are summarized in Table 3. Detailed cognitive debriefing results for the instruments are provided as supplementary files (NEI VFQ-25: Supplementary file 1; LLQ: Supplementary file 2; VAQ: Supplementary file 3).
Interpretation of the NEI VFQ-25
For 19 of the 25 items of the NEI VFQ-25, at least some of the participants had difficulty interpreting the item or choosing a response, most often because of a lack of specification of the lighting conditions or whether the environment was familiar (Table 3). Additionally, some items included multiple examples of activities that represented different levels of functional impairment, making it difficult to choose a response. Finally, some response options were inappropriate (e.g., a response option of ‘stopped doing this’ for items assessing difficulty noticing objects off to the side’), and some items described examples of tasks that were outdated. Note that items related to driving were not debriefed with all participants given the skip patterns built into the NEI VFQ-25.
Feedback for item 13 “Because of your eyesight, how much difficulty do you have visiting with people in their homes, at parties or in restaurants?”:
“Restaurants I find bad… But in homes, friends, like I said, they know I’m coming, they’ve got an extra few lights on or things like that, and a strange home, yes, I’d, I’d have a lot of trouble… If it’s in my friends’ house, a moderate difficulty, but someone else’s would be extreme difficulty” (Male, 50).
Relevance of the NEI VFQ-25
In total, 22/25 of the NEI VFQ-25 items were relevant to over half of the 21 participants. Items were classed as relevant if a participant had ever experienced the impairment or functional limitation, even if they did not currently experience it. The items that were least relevant were item 4 (n = 16) and item 19 (n = 17), which both assess pain and discomfort around the eyes. Item 15, “Are you currently driving at least once in a while?,” was only relevant to one participant, as only one participant could still drive.
Feedback for item 7 “Because of your eyesight, how much difficulty do you have finding something on a crowded shelf?”:
“If the question had mentioned light, if it’s difficult to find things on a crowded shelf when you come from outside. Or in a shop with poor lighting, is it difficult to find things, that would have been relevant” (Male, 25).
Interpretation of the LLQ
For 24 of 32 items of the LLQ, at least some of the participants had difficulty interpreting the item or choosing a response. Similar to the NEI VFQ-25, participants had issues interpreting and choosing a response to items because lighting conditions and familiarity of environment were not specified or because the examples of activities listed in the questions correspond to different levels of functional impairment (Table 3).
Feedback for item 19 “Do you have difficulty seeing when you visit other people’s homes because there is not enough light?”:
“But where I go most people know and the lights are on, so, you know, I would say a lot of difficulty based on if I were to go somewhere and it was dimly light, somebody else’s house, unfamiliar surroundings it would be a lot of, a lot of difficulty” (Female, 32).
Relevance of the LLQ
All of the LLQ items were relevant for over half of the 21 participants, with the exception of seven items related to driving. Apart from the driving items, the two least relevant items were item 2, “Do you have difficulty seeing in fluorescent lighting, like that found in stores and offices?,” and item 12, “Do you worry or are you concerned that you might fall at night because of your vision?” Of note, several participants reported that they were more likely to fall but were just not concerned or worried about it; hence, item 12 was considered less relevant:
“That’s not saying that I might not fall, it’s just that I’m not worried about it” (Male, 47).
Interpretation and Relevance of the Four VAQ Items
Participants had no difficulty interpreting the four VAQ items assessing light/dark adaptation. The two items assessing adaptation from bright to dim/dark-light (item 12 “It takes me a long time to adjust to darkness after being in bright light” and item 28 “I have trouble adjusting from bright to dim lighting, such as when going from daylight into a dark movie theater”) were relevant to all 21 participants. The two items assessing adaptation from dim to bright light (item 1 “I have problems adjusting to bright room lighting, after the room lighting has been rather dim” and item 23 “It takes me a long time to adjust to bright sunshine after I have been inside a building for a lengthy period of time”) were relevant for 17 of the 21 participants.
Pilot Interview Findings
The findings from the two pilot interviews with RP patients did not identify any notable differences between the experiences of patients with RLBP1 RP and those with ‘general RP.’ Furthermore, there were no substantial differences in item interpretation and relevance of concepts between participants with RLBP1 RP and RP. However, as only two pilot interviews were conducted, conclusions regarding these comparisons should be interpreted with caution.
Table 4 summarizes the extent to which each of the instruments being evaluated assesses the concepts that were identified as important through concept elicitation interviewing. No instrument alone assesses all concepts relevant to RLBP1 RP patients. Notably, the NEI VFQ-25 and LLQ do not assess dark/light adaptation, a key defining symptom of RLBP1 RP; the NEI VFQ-25 does not assess vision in bright light or depth perception, which are important concepts in RLBP1 RP. There are also gaps in the conceptual coverage of all three instruments in relation to functional impacts, particularly distal impacts on ADLs and impacts on physical functioning.
Expert Clinician Interviews
All clinicians reported that severity of symptoms varies among RLBP1 RP patients, with the condition generally becoming more severe with age. The symptoms and impacts reported by the expert clinicians were broadly consistent with those reported by the patients. The experts also highlighted the loss of independence experienced by patients and confirmed the importance of environmental factors such as familiarity of environment and lighting conditions in mediating the impact of patients’ visual impairment. The expert clinicians commented that the symptoms of RLBP1 RP are widely similar to those of broader RP, but that RLBP1 RP patients start to lose central vision at an earlier age. The experts identified issues with the NEI VFQ-25 and LLQ that were consistent with those reported by patient participants.
To our knowledge, this is the first comprehensive qualitative study of the patient experience of RLBP1 RP. The symptoms and impacts reported in this study were consistent with those reported in the scientific literature relating to RP and RLBP1 RP and consistent with reports from expert clinicians [5, 19, 20, 22, 40, 41]. However, this study provides a more in-depth understanding of the patient experience of RLBP1 RP from the perspective of patients themselves than was previously available. Based on the literature, two RP patient pilot interviews and clinician interviews, it seems likely that the findings reported here may be generalized to RP as well as RLBP1 RP [42, 43]. However, further study in a sample of RP patients without the RLBP1 gene mutations is necessary to confirm whether the findings from this study can be extrapolated to the broader population.
Neither the NEI VFQ-25, the LLQ, nor the four items tested from the VAQ provide a comprehensive assessment of the RLBP1 RP patient experience, even if administered in combination. In addition, the inconsistencies in item and response option interpretation and limited relevance of some items suggest that, at a minimum, some revision of those instruments is recommended for use in this population.
While some previous studies have demonstrated an association between visual function scores and scores on the instruments in this population, thus supporting construct validity [22, 29, 30], other studies have highlighted issues with the face validity and psychometric properties of the instruments in other similar populations [44,45,46,47,48]. For example, a previous study in patients with Birdshot chorioretinopathy highlighted that the composite score of the NEI VFQ-25 was not statistically associated with decreased visual acuity. Results also suggested that the NEI VFQ-25 may have poor sensitivity at the lower end of the severity scale and may be prone to placebo effects. In addition, previous studies suggest, consistent with findings here, that the LLQ would not provide a full picture of the vision related QoL limitations of RLBP1 RP patients.
Since this research was conducted, the multi-luminance mobility test (MLMT), a performance related outcome assessment, has been developed and validated to assess ADLs in retinal diseases; data from performance outcome assessments such as the MLMT are valuable to complement data generated via PRO instruments [49, 50].
The findings reported provide valuable insight into the patient experience of RLBP1 RP and the assessment of content validity in three visual function PRO measures. However, findings should be interpreted in light of the limitations of the study. The sample size of the study is relatively small for qualitative research; however, the sample size reflects the rare nature of RLBP1 RP. Furthermore, based on the literature and two pilot interviews, it seems likely that the findings reported may generalize to RP as well as RLBP1 RP. However, for that claim to be credible it would be important to replicate the study in a larger sample of RP patients with a variety of mutations. Additional research doing just this is currently on-going. The research was conducted with patients recruited from Canada and Sweden in centers with a high prevalence of RLBP1 RP. Further research to confirm the generalizability of the findings for other cultures is warranted, particularly in relation to functional impairment, which may vary by country. Finally, the participants involved in the study were considered to have severe or very severe visual function limitations consistent with the early onset and rapid progression of RLBP1 RP; therefore, results may not fully represent younger participants with mild or moderate visual function limitations in RLBP1 RP. However, representation from each visual function severity level was obtained, and differences associated with the severity level of RLBP1 RP were consistent with those described by clinicians.
This study has identified issues with conceptual coverage, relevance and participant interpretability of items in the NEI VFQ-25, LLQ and VAQ in the context of RLBP1 RP. This is consistent with evidence from other studies regarding the inadequacy of these instruments in RP and similar diseases .
These findings suggest that modification of one or more existing instruments or development of a new non-syndromic RP specific instrument is warranted to adequately assess the symptoms and associated impacts in RLBP1 RP. Such an instrument would allow generation of data that would be of value to regulators, payers, prescribing clinicians and, most importantly, patients.
Council NC. NICE Citizens Council Report: ultra orphan drugs. London: National Institute for Health and Clinical Excellence; 2004.
Botelho PJ, Blinder KJ, Shahinfar S. Familial occurrence of retinitis punctata albescens and congenital sensorineural deafness. Am J Ophthalmol. 1999;128(2):246–7.
Demirci FYK, Rigatti BW, Mah TS, Gorin MB. A novel compound heterozygous mutation in the cellular retinaldehyde-binding protein gene (RLBP1) in a patient with retinitis punctata albescens. Am J Ophthalmol. 2004;138(1):171–3.
Dessalces E, Bocquet B, Bourien J, Zanlonghi X, Verdet R, Meunier I, et al. Early-onset foveal involvement in retinitis punctata albescens with mutations in RLBP1. JAMA Ophthalmol. 2013;131(10):1314–23.
Eichers ER, Green JS, Stockton DW, Jackman CS, Whelan J, McNamara JA, et al. Newfoundland rod-cone dystrophy, an early-onset retinal dystrophy, is caused by splice-junction mutations in RLBP1. Am J Hum Genet. 2002;70(4):955–64.
Fishman GA, Roberts MF, Derlacki DJ, Grimsby JL, Yamamoto H, Sharon D, et al. Novel mutations in the cellular retinaldehyde-binding protein gene (RLBP1) associated with retinitis punctata albescens. Arch Ophthalmol. 2004;122(1):70–5.
Flynn M, Bohnert D. Fundus albipunctatus and other flecked retina syndromes. J Am Optom Assoc. 1999;70(9):571–80.
Genead MA, Fishman GA, Lindeman M. Spectral-domain optical coherence tomography and fundus autofluorescence characteristics in patients with fundus albipunctatus and retinitis punctata albescens. Ophthalmic Genet. 2010;31(2):66–72.
Hipp S, Zobor G, Glöckle N, Mohr J, Kohl S, Zrenner E, et al. Phenotype variations of retinal dystrophies caused by mutations in the RLBP1 gene. Acta Ophthalmol. 2015;93(4):e281–6.
Zhang X, Wen F, Hu S, Yan H, Lin S. Fundus angiography findings of atypical retinitis pigmentosa. Chin Ophthalmic Res. 2004;22(3):299–301.
Humbert G, Delettre C, Sénéchal A, Bazalgette C, Barakat A, Bazalgette C, et al. Homozygous deletion related to Alu repeats in RLBP1 causes retinitis punctata albescens. Investig Ophthalmol Vis Sci. 2006;47(11):4719–24.
Katsanis N, Shroyer N, Lewis R, Cavender J, Al-Rajhi A, Jabak M, et al. Fundus albipunctatus and retinitis punctata albescens in a pedigree with an R150Q mutation in RLBP1. Clin Genet. 2001;59(6):424–9.
Maw MA, Kennedy B, Knight A, Bridges R, Roth KE, Mani E, et al. Mutation of the gene encoding cellular retinaldehyde–binding protein in autosomal recessive retinitis pigmentosa. Nat Genet. 1997;17(2):198–200.
Morimura H, Berson EL, Dryja TP. Recessive mutations in the RLBP1 gene encoding cellular retinaldehyde-binding protein in a form of retinitis punctata albescens. Investig Ophthalmol Vis Sci. 1999;40(5):1000–4.
Nakamura M, Lin J, Ito Y, Miyake Y. Novel mutation in RLBP1 gene in a Japanese patient with retinitis punctata albescens. Am J Ophthalmol. 2005;139(6):1133–5.
Naz S, Ali S, Riazuddin SA, Farooq T, Butt NH, Zafar AU, Khan SN, Husnain T, MacDonald IM, Sieving PA, Hejtmancik JF. Mutations in RLBP1 associated with fundus albipunctatus in consanguineous Pakistani families. Br J Ophthalmol. 2011;95(7):1019–24.
Nojima K, Hosono K, Zhao Y, Toshiba T, Hikoya A, Asai T, et al. Clinical features of a Japanese case with Bothnia dystrophy. Ophthalmic Genet. 2012;33(2):83–8.
van Genderen MM, Littink KW, van Schooneveld, MJ, Riemslag FCC, Keunen JEE, den Hollander AI, et al. Genotypes and phentotypes in fundus albipunctatus and retinitis punctata albescens [abstract]. In: Proceedings of the 50th ISCEV International Symposium; 5–8 June 2012; Valencia, Spain. Abstract number OS2-4.
Hartong DT, Berson EL, Dryja TP. Retinitis pigmentosa. Lancet. 2006;368(9549):1795–809.
Burstedt M, Sandgren O, Holmgren G, Forsman-Semb K. Bothnia dystrophy caused by mutations in the cellular retinaldehyde-binding protein gene (RLBP1) on chromosome 15q26. Investig Ophthalmol Vis Sci. 1999;40(5):995–1000.
Hamel C, Dessalces E, Meunier I. Retinitis punctata albescens. Inherited chorioretinal dystrophies. Berlin: Springer; 2014. p. 135–41.
Burstedt MS, Mönestam E. Self-reported quality of life in patients with retinitis pigmentosa and maculopathy of Bothnia type. Clin Ophthalmol (Auckland, NZ). 2010;4:147.
Lasch KE, Marquis P, Vigneux M, Abetz L, Arnould B, Bayliss M, et al. PRO development: rigorous qualitative research as the crucial foundation. Qual Life Res. 2010;19(8):1087–96.
US Department of Health and Human Services. Food and Drug Administration. 2009. Guidance for Industry: Patient-reported outcome measures: use in medical product development to support labeling claims. http://www.da.gov/downloads/drugs/guidanceComplianceRegulatoryInformation/Guidances/UCM193282.pdf. Accessed December 5, 2012.
Mangione CM, Lee PP, Gutierrez PR, Spritzer K, Berry S, Hays RD. Development of the 25-list-item national eye institute visual function questionnaire. Arch Ophthalmol. 2001;119(7):1050–8.
Mangione CM, Lee PP, Pitts J, Gutierrez P, Berry S, Hays RD. Psychometric properties of the National Eye Institute visual function questionnaire (NEI-VFQ). Arch Ophthalmol. 1998;116(11):1496–504.
Owsley C, McGwin G, Scilley K, Kallies K. Development of a questionnaire to assess vision problems under low luminance in age-related maculopathy. Investig Ophthalmol Vis Sci. 2006;47(2):528–35.
Sloane M, Ball K, Owsley C, Bruni J, Roenker D. The Visual Activities Questionnaire: developing an instrument for assessing problems in everyday visual tasks. Tech Dig Noninvasive Assess Vis Syst. 1992;1:26–9.
Burstedt MS, Mönestam E, Sandgren O. Associations between specific measures of vision and vision-related quality of life in patients with bothnia dystrophy, a defined type of retinitis pigmentosa. Retina. 2005;25(3):317–23.
Mullins A, Burstedt M, Green J, Whelan J, Sloesen B, Ni X, et al. Cross-sectional evaluation of patient-reported outcomes (PROs) in patients with RLBP1 retinitis pigmentosa enrolled in a Natural History Study. Investig Ophthalmol Vis Sci. 2017;58(8):3251.
Francis JJ, Johnston M, Robertson C, Glidewell L, Entwistle V, Eccles MP, et al. What is an adequate sample size? Operationalising data saturation for theory-based interview studies. Psychol Health. 2010;25(10):1229–45.
Guest G, Bunce A, Johnson L. How many interviews are enough? An experiment with data saturation and variability. Field Methods. 2006;18(1):59–82.
Latham K, Baranian M, Timmis MA, Pardhan S. Difficulties with goals of the Dutch ICF activity inventory: perceptions of those with retinitis pigmentosa and of those who support them. Investig Ophthalmol Vis Sci. 2015;56(4):2381–91.
Braun V, Clarke V. Using thematic analysis in psychology. Qual Res Psychol. 2006;3(2):77–101.
Hsieh HF, Shannon SE. Three approaches to qualitative content analysis. Qual Health Res. 2005;15(9):1277–88.
Kerr C, Nixon A, Wild D. Assessing and demonstrating data saturation in qualitative inquiry supporting patient-reported outcomes research. Expert Rev Pharmacoecon Outcomes Res. 2010;10(3):269–81.
Patrick DL, Burke LB, Gwaltney CJ, Leidy NK, Martin ML, Molsen E, et al. Content validity—establishing and reporting the evidence in newly developed patient-reported outcomes (PRO) instruments for medical product evaluation: ISPOR PRO Good Research Practices Task Force report: part 2—assessing respondent understanding. Value Health. 2011;14(8):978–88.
World Health Organisation. Change the definition of blindness. https://www.who.int/blindness/Change%20the%20Definition%20of%20Blindness.pdf. Accessed Dec 5, 2012.
Sousa MC, Biteli LG, Dorairaj S, Maslin JS, Leite MT, Prata TS. Suitability of the visual field index according to glaucoma severity. J Curr Glaucoma Pract. 2015;9(3):65.
Hamel C. Retinitis pigmentosa. Orphanet J Rare Dis. 2006;1(1):40.
Rivolta C, Sharon D, DeAngelis MM, Dryja TP. Retinitis pigmentosa and allied diseases: numerous diseases, genes, and inheritance patterns. Hum Mol Genet. 2002;11(10):1219–27.
Lowe J, Drasdo N. Patients’ responses to retinitis pigmentosa. Optom Vis Sci. 1992;69(3):182–5.
Senthil MP, Khadka J, Pesudovs K. Seeing through their eyes: lived experiences of people with retinitis pigmentosa. Eye. 2017;31(5):741.
Sugawara T, Hagiwara A, Hiramatsu A, Ogata K, Mitamura Y, Yamamoto S. Relationship between peripheral visual field loss and vision-related quality of life in patients with retinitis pigmentosa. Eye. 2010;24(4):535.
Finger RP, Fenwick E, Owsley C, Holz FG, Lamoureux EL. Visual functioning and quality of life under low luminance: evaluation of the German Low Luminance Questionnaire. Investig Ophthalmol Vis Sci. 2011;52(11):8241–9.
Mills RP. Correlation of quality of life with clinical symptoms and signs at the time of glaucoma diagnosis. Trans Am Ophthalmol Soc. 1998;96:753.
Gothwal VK, Wright TA, Lamoureux EL, Pesudovs K. Visual Activities Questionnaire: assessment of subscale validity for cataract surgery outcomes. J Cataract Refract Surg. 2009;35(11):1961–9.
Levinson RD, Monnet D, Yu F, Holland GN, Gutierrez P, Brezin AP. Longitudinal cohort study of patients with birdshot chorioretinopathy. V. Quality of life at baseline. Am J Ophthalmol. 2009;147(2):346–50.
Chung DC, McCague S, Yu ZF, Thill S, DiStefano-Pappas J, Bennett J, Cross D, Marshall K, Wellman J, High KA. Novel mobility test to assess functional vision in patients with inherited retinal dystrophies. Clin Exp Ophthalmol. 2018;46(3):247–59.
Russell S, Bennett J, Wellman JA, Chung DC, Yu Z-F, Tillman A, et al. Efficacy and safety of voretigene neparvovec (AAV2-hRPE65v2) in patients with RPE65-mediated inherited retinal dystrophy: a randomised, controlled, open-label, phase 3 trial. Lancet. 2017;390(10097):849–60.
Khadka J, McAlinden C, Pesudovs K. Quality assessment of ophthalmic questionnaires: review and recommendations. Optom Vis Sci. 2013;90(8):720–44.
The authors would like to thank the people with RLBP1 RP who participated in the interviews and provided valuable insight into their experience of living with this condition. The team would also like to thank the clinical sites that facilitated this interview study.
This study and manuscript development (including the journal’s Rapid Service and Open Access Fees) were fully funded by Novartis Pharma AG.
All named authors meet the International Committee of Medical Journal Editors (ICMJE) criteria for authorship for this article, take responsibility for the integrity of the work as a whole, and have given their approval for this version to be published.
All authors contributed to the design of the study. Chloe Tolley, Rob Arbuckle and Sarah Bentley were responsible for conducting the research, including development of study documents, data collection, data analysis and interpretation. Karen Holopigian, Kali Stasi, Brigitte Sloesen, Claudio Spera, Jean-Yves Delandes and Anmol Mullins contributed to the review of the study documents and deliverables. Chloe Tolley, Rob Arbuckle and Sarah Bentley wrote the manuscript, and all authors were responsible for reviewing and revising the manuscript and have given approval for this version to be published.
Chloe Tolley, Rob Arbuckle and Sarah Bentley are employees of Adelphi Values, the outcomes research consultancy commissioned by Novartis to conduct this study. Karen Holopigian, Kali Stasi, Brigitte Sloesen and Claudio Spera are employees of Novartis. Jean-Yves Deslandes and Anmol Mullins were employees of Novartis when the research was conducted. Jean-Yves Deslandes is now an employee of Horama, and Anmol Mullins is an employee of Shire. Jane Green and James Whelan are employees of Memorial University of Newfoundland, and Marie Burstedt is an employee of University of Umeå. Jane Green, James Whelan and Marie Burstedt were financially compensated by Novartis for participation in the research as consultants but were not financially compensated for the development of this manuscript.
Compliance with Ethics Guidelines
Ethical approval was obtained from the Newfoundland and Labrador Health Research Ethics Board in Canada (reference no. 2016.224) and Umea University in Sweden (reference no. 2016/357-31). Written informed consent from each participant was obtained prior to data collection.
Data generated from this study are not publicly available; additional data may be provided by the authors on reasonable request.
Enhanced Digital Features
To view enhanced digital features for this article go to https://doi.org/10.6084/m9.figshare.11865084.
About this article
Cite this article
Green, J., Tolley, C., Bentley, S. et al. Qualitative Interviews to Better Understand the Patient Experience and Evaluate Patient-Reported Outcomes (PRO) in RLBP1 Retinitis Pigmentosa (RLBP1 RP). Adv Ther 37, 2884–2901 (2020). https://doi.org/10.1007/s12325-020-01275-4
- Quality of life
- RLBP1 retinitis pigmentosa
- RLBP1 RP
- Visual functioning