FormalPara Key Summary Points

The findings presented in this study advance our understanding of the breadth of neurological signs/symptoms and the magnitude of the impacts of  Wilson disease (WD), which was not so well understood.

This work provides evidence to support the content validity of Part II of the Unified Wilson Disease Rating Scale (UWDRS).

This work also includes supportive evidence for the content validity of Part III of the UWDRS based on the patient perspective, which was not previously demonstrated.

The findings suggest that the UWDRS should be used in conjunction with other clinical outcome assessments that measure key hepatic and psychiatric signs/symptoms of WD as all three domains of symptoms are important to the patient experience and together provide a comprehensive assessment of the patient experience in WD.

Introduction

Wilson disease (WD) is an inherited autosomal recessive metabolic disorder of impaired copper transport. WD causes an accumulation of copper in multiple organs, namely the liver and brain, resulting in a range of neurological, psychiatric, and hepatic manifestations. WD can be managed with life-long treatment that reduces copper burden; however, without treatment the disease can be fatal [1].

The Unified Wilson Disease Rating Scale (UWDRS) was created by the EuroWilson consortium, in collaboration with the German Network of Hereditary Movement Disorders (GeNeMove), to address the lack of a standardized assessment for the neurological manifestations of WD [2]. This instrument was based on previously validated measures for neurological disorders that assess similar symptoms including the Barthel Index, Unified Parkinson’s Disease Rating Scale (UPDRS), Unified Huntington’s Disease Rating Scale (UHDRS), International Cooperative Ataxia Rating Scale (ICARS), and Burke-Fahn-Marsden Dystonia Rating Scale (BFM scale) [2,3,4]. On their own, none of these scales were able to reflect the multiple motor impairments caused by WD [3]. The UWDRS scale is split into three parts, Part I assesses level of consciousness, Part II is the patient-reported section, and Part III is a detailed neurological examination. Parts I and III are designed to be completed by a neurologist or other clinician, while Part II is to be completed by a patient, family member, or caregiver.

The UWDRS overall, including Part II, was, however, created without the direct input from patients, and to date, evidence that confirms the content validity and relevance of the instrument to patients does not exist. Utilizing patient input to evaluate content validity of an instrument is a crucial step as quantitative analysis of an instrument’s psychometric properties alone cannot confirm the appropriateness of an instrument to measure the patient experience in a clinical trial. The importance of establishing content validity in clinical outcomes assessments (COAs) from the patient perspective is increasingly emphasized by both the United States (US) Food and Drug Administration (FDA) and the European Medicines Agency (EMA). [5]

A key first step in assessing content validity is to understand the experiences of a patient population. A manuscript by Karantzoulis et al.[6] describes the development of a conceptual model of WD that includes the signs, symptoms, and impacts experienced by patients with WD. The conceptual model was created with data from a targeted literature review, clinician interviews, and concept elicitation (CE) interviews with patients with WD. The findings from that work highlighted the heterogenous presentation of WD, with patients experiencing a range of signs and symptoms including hepatic, neurological, and/or psychiatric, as well as many resulting impacts [6]. This published work also suggests that patients with WD fall into one of two groups based on their symptoms experienced: (1) patients who experience neurological, psychiatric, and hepatic symptoms and (2) patients who experience mostly hepatic and some psychiatric symptoms [6].

In the current work, we sought to evaluate the content validity of the UWDRS Part II—specifically regarding its coverage/ability to assess the neurological aspects of WD—using qualitative interviews and mapping of concepts identified to the items of the UWDRS to further augment the findings from the previously published work. Specifically, one-on-one interviews were conducted with patients who experience neurological signs/symptoms in order to better understand the many neurological manifestations of WD and how those impact how patients feel and function as well as to assess the conceptual coverage of these symptoms by the UWDRS. Concepts were mapped to Parts II and III of the UWDRS to assess conceptual coverage by the instrument. Cognitive debriefing (CD) interviews were conducted with patients to evaluate the patient-reported Part II items and only select Part III items of the UWDRS to confirm their relevance and clarity, to evaluate the content validity of the instrument. Not all Part III items were discussed given that this section is intended to be completed by a clinician.

Methods

Content validity can be defined as “the extent to which a scale or questionnaire represents the most relevant and important aspects of a concept in the context of a given measurement” [7]. Content validity can be assessed by utilizing two qualitative interview methodologies. CE interviews are conducted to understand the signs, symptoms, and impacts (“concepts”) of disease that patients experience and culminate in the creation of a conceptual model of the disease experience. CD interviews are conducted to determine if items in a selected instrument are relevant to the patient experience with the disease and are clearly and easily understood. This study was conducted in several steps, starting with CE interviews, then mapping concepts from the patient interviews to items in Parts II and III of the UWDRS, followed by CD interviews of Part II and select items of Part III. The methods for the CE interviews are the same as those in the previously published manuscript [6] with the addition of one inclusion criterion for targeted recruitment of patients who currently experience one or more neurological signs/symptoms. For full details, see Karantzoulis et al. [6].

Concept Elicitation Interviews

One-on-one CE interviews, lasting approximately 90–150 mins, were conducted with 16 patients diagnosed with WD who experience neurological signs/symptoms. The patients were recruited from the US, United Kingdom (UK), and Germany via two separate recruiting services, Global Perspectives (US and Germany) and Medicys (UK), as well as through patient advocacy groups (PAGs).

Patients completed an online screener or answered screener questions over the phone and provided consent prior to the start of the interview. Patients’ clinicians or clinicians’ offices also completed a confirmation of diagnosis form. To recruit a sample of patients with WD who experience neurological signs/symptoms, patients were asked, during screening, to select or report the neurological signs/symptoms that they previously and/or currently experience from a list including tremor, walking or movement problems, speech difficulties, drooling, lack of muscle coordination, handwriting deterioration, increased irritability/anger outburst, increased impulsivity, memory loss, loss of focus, and increased confusion. An “other” option was also included to allow patients to share other neurological signs/symptoms they previously and/or currently experience that may have not been listed. Patients who stated they experienced one or more neurological signs/symptoms previously and/or currently were included in the study. Further details on selection criteria are provided in the supplementary material (Additional file 1).

The interviews were conducted by trained interviewers in accordance with the 1975 Declaration of Helsinki and the regulations of the US FDA. All interview materials received a priori Independent Review Board (IRB) approval. During the interviews, patients spontaneously reported and described the signs, symptoms, and impacts they experience as a result of their WD, and then were probed on additional concepts by the moderator. The interviews were recorded, and transcripts were coded and analyzed using ATLAS.ti v8 qualitative analysis software. To assess the saturation of concepts, transcripts were arranged chronologically and then grouped into five waves. By the end of the fourth wave, no new concepts were mentioned, suggesting saturation was achieved.

In order to evaluate the changes in sign/symptom experience over time, patients were asked about the symptoms they currently experience and those previously experienced. Patients were later described as “ever experiencing” a concept if it was reported by the patient in the past, currently, or both. The degree of bother was asked for all concepts experienced currently and/or previously using a 0-10 scale, where 0 means not at all bothered by the concept and 10 means greatly bothered. Peak bother is the highest bother rating reported by the patient for that concept, regardless of when the concept was experienced (i.e., if that bother rating was reported in the context of the patient’s past or current experience with the concept). Average peak bother was calculated for each concept based on the total number of patients who provided a peak bother rating for that concept. Patients who did not provide a peak bother rating for a concept they experienced were not included in the average peak bother calculation. They were, however, factored into determining the number of patients ever experiencing each concept. A concept was deemed salient if it was mentioned by ≥ 8 patients ever experiencing and had an average peak bother rating of ≥ 5.

Concept/Instrument Mapping

Following the completion of CE data analysis, item mapping was conducted to determine the overlap of concepts from the CE patient interviews and the items from Parts II and III of the UWDRS. All neurological symptoms and all physical, emotional, and social/functional impacts were included in the mapping exercise.

Cognitive Debriefing Interviews

One-on-one CD interviews, lasting approximately 60–90 mins, were conducted with 13 patients with WD. The patients were recruited from the US and the UK via two separate recruiting vendors Global Perspectives (US) and Medicys (UK). Patients who previously participated in the CE interviews were contacted to be interviewed again, if available. All patients completed screening questions and informed consent to determine eligibility to participate in these interviews. Patients’ clinicians or clinicians’ offices completed a confirmation of diagnosis form. Unlike the CE interviews, patients who participated in the CD interviews were recruited regardless of whether or not they experience neurological signs/symptoms associated with their WD. This is representative of the WD patient population as a whole and is consistent with the sample from the previously published manuscript [6]. Further details on selection criteria are provided in the supplementary material (Additional file 1).

All Part II items of the UWDRS were debriefed to assess their relevance and clarity. Only four items (speech, handwriting, arising from a chair, and gait) from Part III were selected to be debriefed for relevance with patients in line with regulatory agency feedback to specifically explore key functional items. Moreover, Part III items were not fully discussed with patients in the same way as Part II given that Part III is intended to be completed by clinicians. The interviews were conducted by trained interviewers in accordance with the 1975 Declaration of Helsinki and the regulations of the US FDA. All interview materials received a priori IRB approval.

On the date and time of the interview, both the interviewer and patient joined a teleconference line with a screen-sharing platform allowing the patient to review the items one at a time, provide a response, and comment on the relevance and clarity of the items. Patients provided verbal consent to participate and be audio recorded prior to the start of the interview. The interviews were structured to include patient perceptions of relevance and clarity of all Part II items and the relevance of the four Part III items, as time allowed. For the Part II items, patients were first asked about the relevancy of an item to their experience with WD and were then given the opportunity to speak about the clarity of the items in an open-ended manner. In some instances, if a Part II item was not relevant to the patient, they were not further probed about its clarity.

Results

Patient Sample Demographics and Clinical History

CE interviews were conducted between January and April 2020. The 16 patients interviewed were from the US, UK, and Germany, and previously and/or currently experienced neurological signs/symptoms (Table 1). The sample included nine female patients and seven male patients ranging in age, including two patients under the age of 18. About half of the patients had completed some higher-level education (n = 9). Patients were diagnosed with WD between two and 38 years ago and all were symptomatic in the four months preceding the interview.

Table 1 Concept elicitation and cognitive debriefing patient sample summary
Full size table

CD interviews were conducted between June and October 2021 with 13 patients with WD from the US and UK (Table 1). The sample included seven female patients and six male patients, varying in age. Most patients had completed some higher-level education (n = 10). Patients were diagnosed with WD in the previous two to 45 years, and all were symptomatic in the four months preceding the interview.

Concept Elicitation Interviews

During the CE interviews, patients reported experiencing a range of signs, symptoms, and impacts. Seventy of the 74 (94.5%) identified (from literature review, clinician interviews, and ongoing patient interviews) signs/symptoms and 22 of the 23 (95.7%) identified impacts were reported by patients over the course of the interviews.

Analysis of CE interviews identified several neurological (7), psychiatric (6), and hepatic (4) signs/symptoms as salient (Tables 2, 3, 4). The salient neurological signs/symptoms included change in balance, change in walking, change in writing, drooling, other changes in speech, slurred speech, and tremor. The salient psychiatric sign/symptoms included anxiety, apathy, depression, irritability, changes in attention, and changes in thinking skills. The salient hepatic signs/symptoms included fatigue, joint pain, muscle cramping, and stomach pain.

Table 2 Number of patients ever experiencing concept in the neuro-enriched sample and average peak bother rating for neurological symptoms
Full size table
Table 3 Number of patients ever experiencing concept in the neuro-enriched sample and average peak bother rating for psychiatric symptoms
Full size table
Table 4 Number of patients ever experiencing concept in the neuro-enriched sample and average peak bother rating for hepatic symptoms
Full size table

Seven impacts of WD were also identified as salient (Table 5). These salient impacts included worried about the future, limits in physical function, impact on social life, impact on ability to work, impact on school performance, impact on family life, and worried about how perceived by others. These impacts spanned the three impact categories (physical, emotional, and social/functional).

Table 5 Number of patients ever experiencing concept in the neuro-enriched sample and average peak bother rating for impacts
Full size table

During the interviews, patients described many of the concepts as being highly bothersome. Approximately 81% of the neurological signs/symptoms reported by patients have an average peak bother rating of > 5. Additionally, 44% of these signs/symptoms were reported to have an average peak bother rating of > 7.

Concept/Instrument Mapping

All neurological signs and symptoms, as well as all impacts identified in the CE interviews were mapped to items of the UWDRS. The full mapping exercise can be found in the supplementary material (Additional file 2). Following the review of the impacts, they were categorized into three groups, physical, emotional, and social/functional. Mapping showed that there was almost complete coverage of the neurological signs/symptoms and impacts reported by patients with one or more items of Part II and Part III of the UWDRS.

Cognitive Debriefing Interviews

Except for the toilet use item, all 13 patients found all the items to be clear (Table 6). A few patients (n = 3) expressed concerns about what exactly “toilet use” was asking in terms of actual use or mobility. Most patients found the following items to be personally relevant mobility (n = 11), salivation (n = 9), dressing (n = 7), and toilet use (n = 7). Some patients found the following items to be personally relevant falling (n = 6), transfer (n = 6), swallowing (n = 5), feeding (n = 6), taking a bath or shower (n = 6), and grooming (n = 6).

Table 6 Summary of UWDRS Part II item debriefing
Full size table

Among the patients that were asked about Part III items, some of the patients found the four items to be relevant to their experience with WD (speech n = 4; handwriting n = 9; arising from chair n = 6; gait n = 6).

Discussion

This study aimed to evaluate the content validity of the UWDRS from the patient perspective, utilizing qualitative research methods. The results from the CE interviews with the “neuro-enriched sample” of patients with WD generally align with the results previously reported in Karantzoulis et al. [6] and other prior published literature on the WD patient experience regarding the signs, symptoms, and impacts [1, 8,9,10,11]. These findings are also in line with prior research supporting the usefulness of the UWDRS in patients with WD.

While patients with WD present with multisystem symptoms and impacts, this CE interview study highlighted a broad range of salient neurological signs/symptoms as well as impacts. The salient neurological concepts identified include change in balance, change in walking, change in writing, drooling, other changes in speech, slurred speech, and tremor. Only one other salient concept, dizziness, identified in the previous study [6], was not identified as salient in this study.

In addition to the large number of neurological concepts reported, patients described many of the concepts as being highly bothersome. Bothersome ratings provided highlight a high degree of bother associated with the neurological aspects of WD and negative impact on patients’ lives. Karantzoulis et al. [6] showed that not all patients with WD experience neurological signs/symptoms due to their WD. However, for those who do, this data underscores the importance of understanding how future treatments may affect these bothersome neurological concepts.

Mapping neurological concepts identified in the CE interviews with items of the UWDRS Parts II and III demonstrates almost complete coverage of all neurological signs and symptoms, and complete coverage of salient neurological signs and symptoms by the UWDRS. This data suggests that the UWDRS Parts II and III is comprehensive in its conceptual coverage of the neurological aspects of WD.

UWDRS Part II item debriefing with patients demonstrated that the Part II items of the UWDRS were generally clear and easily understood, and most patients considered items personally relevant to their experience with WD. Three patients expressed concerns regarding the clarity of the toilet use item, though the majority reported this item to be clear and relevant. These data provide supportive evidence for the content validity of Part II of the UWDRS. This suggests that patients enrolled in a clinical trial would understand and consider these Part II items to be relevant.

The main strengths of this study are the multiple streams of research conducted, including the CE interviews, concept/instrument mapping, and CD interviews. The CE interviews conducted with the “neuro-enriched sample” further support the results published in Karantzoulis et al. [6]. The number of patients interviewed and the heterogeneous experiences they reported further strengthens the body of evidence gathered to date.

This study has a few potential limitations. Patients actively volunteered to participate in this study, creating the possibility of selection bias in the sample. The cognitive debriefing interviews were conducted in English with patients primarily located in the US. Therefore, the results of the cognitive debriefing interviews may not be generalizable to other regions/languages for which the UWDRS may be deployed. Additionally, while some patients who reviewed the select Part III items found these items to be relevant, conclusions cannot be made regarding content validity of Part III from this study. In order to do so, the relevance and clarity of these items will need to be explored with individuals who would complete Part III, i.e., study clinicians.

Importantly this work, along with Karantzoulis et al. [6], highlights other important aspects of the patient experiences—specifically, psychiatric and hepatic signs/symptoms. The UWDRS does not cover these concepts and therefore does not assess the full patient experience. To evaluate the full patient experience, a robust COA strategy that includes the UWDRS and other clinical outcome assessments that map onto the hepatic and psychiatric symptoms is needed. Further, as reported in Karantzoulis et al. [6], only a portion of patients with WD experience neurological signs/symptoms. As such, scales and endpoints created from the UWDRS may be limited in a broad WD population, with a proportion of patients appearing “asymptomatic”, as the coverage of UWDRS is limited to neurological symptoms/impacts. Scales and endpoints derived from the UWDRS must take into consideration the heterogeneity of the WD population. Specifically, in some patients, there is a lack of neurological symptoms/impacts and there is variability even among those patients with neurological symptoms/impacts. Meaningfully derived scales/scores for the UWDRS will be critical for this population and instrument, given the heterogeneity of the population and the range of neurological symptoms/impacts measured.

Conclusion

Overall, the data from this study provides evidence for the content validity of Part II of the UWDRS as well as further evidence supporting the conceptual relevance and comprehensibility of select items of Part III of the UWDRS. Specifically, all salient neurological signs/symptoms identified in patient interviews are represented by one or more of the items in Part II and/or Part III of the UWDRS, suggesting that the UWDRS is comprehensive in its coverage of the neurological aspects of WD. Further, Part II of the UWDRS includes items that are confirmed to be well understood and relevant to patients with WD who have a history of neurological signs/symptoms. As the neurological concepts reported are noted by patients as very bothersome, having a COA like the UWDRS that has strong conceptual coverage, including for bothersome neurological symptoms, and has items that are relevant and understood by the patients supports its use in the context of a clinical trial to evaluate treatment effect. Importantly, the UWDRS should be used in conjunction with other COAs to provide a comprehensive assessment of the patient experience and scales and endpoints derived from the UWDRS (and other instruments) should take into consideration the heterogeneity of the population.