FormalPara Key Points for Decision Makers

With careful screening, persons with self-reported memory and thinking concerns can independently complete a simplified discrete-choice experiment survey.

The simplified discrete-choice experiment tool can be used to collect quantitative patient preference data for use in regulatory decision making and by other stakeholders interested in patient preferences.

1 Introduction

Understanding patients’ perspectives on current or potential treatment options is increasingly important to regulators and other stakeholders [1,2,3,4,5,6,7]; however, the impact of cognitive decline on a person’s ability to participate in stated-preference research is largely unknown. Alzheimer’s disease (AD) is a progressive neurodegenerative disease characterized by neuropathological hallmarks that can now be qualitatively and quantitatively assessed [8,9,10]. Clinical manifestations include gradual declines in memory, executive function, and language. Early cognitive symptoms emerge many years after AD neuropathological changes begin but may be subtle and involve no functional impairment.

Guidance from the US Food and Drug Administration on trials in early AD (an umbrella term for the clinical syndromes of mild cognitive impairment [MCI] and mild dementia with evidence of AD neuropathology) recommends that a treatment show a clinically meaningful effect but notes that clinical meaningfulness of changes on neuropsychological performance measures is unclear [10]. Previous clinical research with amyloid-antibody therapeutics in those with early AD investigated whether such treatments slowed the rate of cognitive decline with unconvincing results [11,12,13]. Recent trials of donanemab, lecanemab, and aducanumab have been more supportive of the potential for amyloid-antibody treatments to slow cognitive decline [14,15,16]. Clinically meaningful effects of such treatments in early AD may be difficult to measure using existing instruments, when only subtle and variable cognitive and functional deficits are detectable and when existing measures may not capture the impact that patients find important. [17, 18]. In this situation, the meaningfulness of changes in cognition irrespective of improvement in functional impairment may be assessed by the people affected, highlighting the importance of eliciting perspectives and preferences in early AD.

Preference studies can quantify risks patients are willing to accept to achieve specific treatment benefits and may help elucidate the clinical meaningfulness of those benefits [19, 20]. Discrete-choice experiments (DCEs) are a statistically rigorous method to collect preferences, allowing the consideration of multiple attributes at a time, and have informed regulatory decisions [21, 22]. For AD, some studies have reported on trade-offs people were willing to make among features of potential AD treatments; however, except for Oremus and colleagues [23], these studies were conducted with healthy adults [24,25,26] or caregivers [27, 28] rather than patients.

Because preference surveys can be cognitively challenging, the first objective of this study was determining whether a DCE survey of people with memory and thinking problems (i.e., subjective cognitive complaints and/or MCI) was feasible to reliably elicit preferences for delaying cognitive decline. The second objective was determining whether self-reported screening criteria could reliably identify a population with cognitive impairment consistent with early stages of an AD clinical syndrome who could understand and independently complete the DCE survey. To achieve these objectives, a preliminary DCE survey was developed on the basis of qualitative interviews [29] and assessed through two rounds of individual face-to-face pretest interviews. This article describes these pretest interview results and the interplay between the survey design and screening criteria, including the implication that a point along the AD cognitive impairment continuum may exist beyond which one cannot independently complete certain preference surveys.

2 Initial Survey Development

2.1 Survey Instrument

Previous qualitative interviews with participants self-reporting cognition problems consistent with early stages of an AD clinical syndrome (target sample) were conducted to identify meaningful benefits of a treatment that could slow the progression of memory and thinking problems and to assess participants’ reactions to several adverse event (AE) risks [29]. These interviews informed and tested potential formats and graphics for the current DCE survey.

The DCE survey offered target sample participants three options: two hypothetical medicines and no new medicine (opt out) (Fig. 1). The development of benefit and risk attributes is described below. The survey instrument also collected information about current memory and thinking problems, current/previous treatments for memory and thinking problems, participants’ ability to comprehend attribute descriptions, and demographic questions.

Fig. 1
figure 1

Example of a discrete-choice experiment question from the first day of round 1 pretest interviews

Full size image

2.1.1 Treatment Benefit Attribute

The benefit attribute was based on then-published clinical studies of treatments in symptomatic AD that suggested a potential slowing of cognitive decline [13, 30]. These experimental treatments resulted in smaller mean changes on the Alzheimer’s Disease Assessment Scale–Cognitive Subscale (ADAS-Cog) [average: 2 points] than changes observed with placebo at 18 months [12]. Therefore, the benefit attribute in this study was described as delaying or slowing decline in memory and thinking problems.

Cognitive decline, a multifaceted concept, was described in the survey using five daily-life examples of impacts to memory and thinking ability: financial management, medication management, finding the right words in conversation, remembering what you have read in a new book or magazine, and understanding verbal instructions. The choice of these five impacts was based on clinical data (natural history/clinical studies) demonstrating the ADAS-Cog items that were most likely impacted early in the AD clinical syndrome (e.g., item scores that decreased earliest within an 18-month clinical trial) [31,32,33]. The language used for the impacts was informed by previous research [34], input from practicing physicians, and previous qualitative interviews [29].

The survey described a person who had “no problems,” “some problems,” and “more problems” for each of the five impacts. “Some problems” and “more problems” reflected a specific change in the ease with which a person could complete the described impact (see Electronic Supplementary Material [ESM]). Cognitive decline was described as the time it would take to progress from having “some problems” with memory and thinking to having “more problems.” Although a minimal clinically important difference for ADAS-Cog has been published [35], no studies translate specific changes in ADAS-Cog scores to relevant practical changes in a patient’s life. Thus, the change from “some” to “more” problems was developed to represent a modest change in the ADAS-Cog and was based on medical judgment of what could reasonably occur over the natural course of AD and on observations from some clinical trials of investigational disease-modifying treatments for early AD [13, 32, 33, 36].

The opt out option was defined as a decline from “some problems” to “more problems” in 1 year. The two hypothetical medicines offered an additional 6 months, 12 months, or 18 months to the time until memory and thinking problems worsened. Time to progression was depicted graphically in the initial survey instrument (Fig. 1).

2.1.2 Treatment Risk Attributes

To capture the range of possible AEs, one mild AE and one severe AE were included. A common AE of antibody treatment is mild/moderate hypersensitivity reactions, which can manifest as flu-like symptoms (i.e., chills, fatigue, and muscle weakness). In previous qualitative interviews [29], participants perceived “flu-like symptoms” as worse than intended, thus the attribute description of “feeling tired and weak” was used as a mild AE associated with treatment. For a severe AE, recurrent noncardiac chest pain that felt like a heart attack, requiring a doctor visit, was selected based on its rare occurrence in solanezumab clinical trials [12]. A plausible duration or frequency range of the selected AEs, based on Siemers and colleagues [12] and the authors’ medical knowledge of solanezumab, was 1–4 days of feeling tired and weak and 1–6% frequency for recurrent noncardiac chest pain. These risk attribute levels were depicted using an icon array (Fig. 1).

Although many amyloid-targeting therapeutics in development are associated with amyloid-related imaging abnormalities [37,38,39], the clinical impact of amyloid-related imaging abnormalities was not well understood at the time this study was conducted [40]. Thus, the presence of amyloid-related imaging abnormalities was not included as a risk attribute in this study.

2.2 Human Subjects Protection

Before study initiation, the RTI International Institutional Review Board reviewed and approved the study materials (No. 14025, dated 11 July, 2017, amended 13 November, 2017). Each participant provided written informed consent, which included teach-back questions to verify participants understood key elements of the consent form.

3 Round 1 Pretest Interviews

3.1 Procedure

Fifteen in-person, 1-hour pretest interviews were conducted over 3 days. During each interview, participants were asked to read the survey aloud and discuss their responses to survey questions. Respondents were asked to assume they currently had “some problems” when they answered the DCE treatment questions.

Interviewers asked follow-up questions to assess participants’ understanding of survey questions and text. Survey text and content were revised between interviews to address identified issues. Additional revisions were made to the survey after completing all round 1 interviews.

3.2 Participant Screening

Inclusion and exclusion criteria were developed to recruit a sample of participants for an online survey. The goal of the criteria was to identify a group of participants who have a clinical syndrome consistent with early AD and who could be recruited from existing online panels or other databases of research participants, without requiring a clinician assessment. The inclusion and exclusion criteria were designed to identify people who had self-reported noticeable changes in memory and thinking that were significant enough to visit their doctor but not so severe that the changes would preclude the participant from completing a DCE survey.

Figure 2 depicts the flow of screening questions for recruiting round 1 pretest interview participants. The ESM provides round 1 screening questions. Although the research framework for the clinical and neuropathological diagnosis of AD has since evolved [9], the screening criteria included core clinical criteria for the diagnosis of MCI from the National Institute on Aging and the Alzheimer’s Association [40,41,42]. The participants’ level of cognitive decline was ascertained through participant-reported symptom and treatment history. Screening criteria were enhanced by requiring participants to recently have visited their doctor about their memory and thinking concerns and discussed using medications for treatment of these symptoms. Including those who discussed concerns with their doctor helped identify those more likely to have early signs of cognitive impairment. As physicians use a variety of terms with patients when discussing the cause or severity of cognitive complaints, and may avoid saying “Alzheimer’s disease” or “dementia” [43], screening on whether a discussion of AD symptomatic medication took place enabled the identification of those who may have early signs of cognitive impairment as judged by their physician but with whom the physician did not use “Alzheimer’s” or “dementia” terminology.

Fig. 2
figure 2

Inclusion and exclusion criteria for round 1 interviews. AD Alzheimer’s disease, ADLs activities of daily living, meds medications, MCI mild cognitive impairment, Rx prescription. Note: The screening criteria were revised for round 2 interviews. The revisions included the following: time since a person first spoke with their doctor about memory concerns was decreased from 4 years to 2 years, participants who reported taking memantine (Namenda) or the combination of memantine and donepezil (Namzaric) were excluded, and those self-reporting that they were not comfortable with a computer were ineligible

Full size image

For round 1 pretest interviews, potential participants were screened for eligibility via telephone. Re-administration of screening questions during the in-person interview provided a consistency assessment of self-reported responses. Screening question responses also were compared with the interviewers’ subjective assessment of the participant’s ability to understand the survey. Interviewers assessed whether differences in responses to screening questions during the telephone compared with in-person interviews correlated with the participant’s ability to understand the survey. Evaluation of whether screening questions could identify people able to independently complete the survey was also performed.

3.3 Results

The 15 participants in round 1 pretests averaged 64 years of age, with approximately half being women (Table 1). Most were married and had a 4-year college or graduate degree. Table 2 provides participants’ responses to some questions about their memory and thinking problems. The participant characteristics and types of problems reported in these interviews were consistent with a population diagnosed with MCI or mild AD (using Petersen or NINCDS-ADRDA criteria, respectively), with the possible exception of orientation [31, 44,45,46].

Table 1 Participant characteristics
Full size table
Table 2 Responses to selected questions related to memory and thinking problems
Full size table

On the basis of the judgment of experienced interviewers, five participants were unable to understand and complete the survey, even with assistance from the interviewer, and two needed substantial help. Primarily, these participants struggled to understand text describing the five impacts on memory and thinking problems, DCE attributes, and choice questions. Results from the remaining eight participants who independently completed the survey and changes made to the survey are summarized in Sects. 3.3.2 and 3.3.3. Out of the eight participants who were able to complete the survey independently, three met the eligibility criteria based on both the telephone and in-person screening. Another five reported answers during the in-person pretests that were different than the answers they reported during the telephone screening and that would have made them ineligible because their cognitive impacts reported during the in-person pretests were too mild.

3.3.1 Screening Questions

All participants provided at least one inconsistent response when completing screening questions in person compared with completing screening question during the initial telephone screening; however, not all of these inconsistencies would have precluded participants from the pretest interviews (Table 3). Participants most frequently provided inconsistent responses to when the participant first discussed memory and thinking concerns with their doctor, what the doctor told them was the cause of memory and thinking problems, and with which daily activities they need help.

Table 3 Consistency of responses to screening questions comparing telephone screening with responses at in-person interview in pretest interviewsa
Full size table

On the basis of responses to screening questions during in-person interviews, 11 of the 15 respondents would have been ineligible for the study, with eight reporting impacts that were too mild, two reporting impacts that were too advanced, and one reporting a stroke. Six of the seven participants who could not independently complete the survey would have been ineligible based on their responses to screening questions in person compared with responses via telephone screening. Specifically, two participants changed their answer about when they had first talked to their doctor to “more than 4 years ago,” which was an indicator of more severe disease and would have made them ineligible. Five of the seven reported currently taking a medicine for memory problems (not an exclusion criterion in round 1 interviews). Finally, three participants who could not complete the survey were the only participants self-reporting a formal AD diagnosis: “Alzheimer’s,” “early Alzheimer’s,” or “pre-Alzheimer’s.” No other notable patterns in types of reported memory and thinking concerns or other characteristics were identified to inform the differential ability to complete the survey.

3.3.2 Survey Instrument

For the eight participants who could independently complete the survey, all could answer questions about attribute descriptions, understand and remember the information presented, respond logically to the choice questions, and provide rationale for the choices they made. Specifically, these eight participants appeared to understand the distinctions among the different severities of memory and thinking problems (i.e., difference between “no problems,” “some problems,” and “more problems”) presented for each of the five impacts.

Despite understanding the text about “some problems” and “more problems,” it appeared participants did not follow instructions to answer the DCE questions as if they currently had “some problems.” During interviews, when asked what level of benefit they were thinking about, seven of these eight participants reported they were thinking about a medicine that would slow the decline from their current level of memory or thinking problems to a worse level, rather than imagining a decline from “some problems” to “more problems” as instructed. Nevertheless, participants understood and applied the concept of a modest incremental decline in cognitive problems when answering the DCE questions.

The eight participants who independently completed the survey accurately described trading different levels of AEs with a slowing of the decline in memory and thinking problems, indicating an understanding of choices offered in the DCE questions. However, some participants described having difficulty understanding the graphics used to present time until worsening of memory and thinking problems, as well as risk grids used to describe the risks of recurrent heart-related chest pain.

3.3.3 Changes to the Screening Procedure and Survey Instrument

The results from the round 1 interviews provided guidance on the changes to the screening criteria and screening process that might improve the ability of the screening questions to identify the target sample, as well as the changes to the survey instrument that would result in a survey that the target population could complete independently. Because seven participants in round 1 pretests could not independently complete the survey, and considering results from round 1 interviews, it seems a level of cognitive impairment exists beyond which participants cannot independently complete a DCE survey, and that cut-off may be distinguishable from those who can independently complete the survey. Thus, two adjustments were made to the screening approach for round 2 pretests: use of an online screener before the telephone screening, as the final goal was developing an online survey, and revision of two screening criteria to exclude those with more severe cognitive impairment. The revised screening criteria were the time since a person first spoke with their doctor about memory concerns (decreased from 4 years to 2 years) and exclusion of participants taking memantine (Namenda) or combination memantine/donepezil (Namzaric), given memantine is indicated for moderate-to-severe dementia (ESM). Finally, a question about self-reported ease of computer use for e-mail and websites was added; those uncomfortable with computer use were ineligible.

Three major changes were made to the DCE survey during and after round 1 pretest interviews. After the first six interviews, four with participants who were able to independently complete the survey, graphics illustrating “time until symptoms worsened” and risk grids depicting levels of noncardiac chest pain were replaced with text descriptions. Second, the question format was changed from offering a choice among two hypothetical medicines and no new medicine to offering a choice between no treatment and one hypothetical treatment to reduce the question complexity. Third, in the DCE question, the beneficial attribute was revised to describe a change from the participant’s current cognitive status and as the time until the participant’s memory and thinking problems got “noticeably worse.” Participants were observed to ignore instructions that they were to assume a hypothetical baseline and level of change using the terms “no problems,” “some problems,” and “more problems.” Therefore, these terms were removed from the DCE questions. Figure 3 presents an example of the revised DCE question used on the first day of round 2 pretest interviews.

Fig. 3
figure 3

Example of a discrete-choice experiment question from round 2 pretest interviews

Full size image

4 Round 2 Pretest Interviews

4.1 Procedure

In round 2 pretest interviews, a three-part sequential screening process was used: an online screener, telephone screening, and in-person rescreening to start the in-person interview. For the round 2 interviews, prespecified criteria were developed to assess the participants’ ability to independently complete the survey. Specifically, after each of the two practice DCE questions, one of the two interviewers asked the participant to explain the difference between two options presented in the DCE question. Each interviewer independently rated whether the participant was (1) able to explain the two options correctly without assistance, (2) able to explain the two options correctly after minor prompting, (3) able to explain the two options correctly after detailed prompting, or (4) not able to explain the two options correctly. The two interviewers also independently rated the participant’s ability to complete the full survey using a scale from 1, “the participant would not be able to complete the survey on their own or would need significant help,” to 5, “the participant would be able to complete the survey on their own with no more difficulty than what the interviewers would expect with a typical respondent to a DCE survey.”

4.2 Results

Twelve participants were recruited and interviewed. The participants had an average age of 59 years; most were female, married, and had a 4-year college or graduate degree (Table 1). Table 2 reports participants’ responses to some of the questions about memory and thinking problems, which were consistent with a population diagnosed with MCI or mild AD (using Petersen or NINCDS-ADRDA criteria, respectively) [31, 44,45,46].

All 12 participants in round 2 pretest interviews could understand and complete the revised survey instrument with little to no difficulty, based on the interviewers’ assessments and ratings (ESM). When comparing responses to screening questions across the three modes of administration (online, telephone, and in person), some inconsistencies in responses were noted. Four respondents would have been ineligible based on their responses during the in-person interviews (Table 3): three were too mildly affected and one reported brain trauma during the in-person interview.

5 Discussion

As the investigation of potentially disease-modifying AD treatments continues, understanding patient preferences for such treatments and willingness of patients to accept risks to attain treatment benefit will be a consideration in drug development and regulatory/health authority decision making. Designing a patient preference survey for disease-modifying AD treatments presents additional challenges compared with constructing a patient preference survey for populations that are not cognitively impaired.

On the basis of previous qualitative interviews [29], a preliminary DCE survey was developed to collect preferences from participants with cognitive concerns consistent with an early AD clinical syndrome. However, in round 1 pretests, seven of the 15 participants could not independently complete the survey. Screening question responses associated with difficulty completing the survey included questions about how long ago participants visited their doctor because of memory concerns and the use of medicines for memory and thinking concerns. Both suggest a higher degree of cognitive impairment for those unable to independently complete the survey compared with the intended target population.

Among those participants independently completing the survey, some struggled to understand the graphics used to communicate how long the treatment slowed cognitive decline and the chance a risk would be experienced. Anecdotally, we have encountered problems with time-to-event graphics in other surveys, although the graphics we have used to convey risk generally have been well understood [47]. The literature on the use of graphics to communicate risk and other attributes is mixed [48, 49]. It may be that typical graphics used in DCE surveys, such as risk grids, are less helpful in cognitively impaired populations who struggle with abstract thinking. More research on the best approach to communicate attributes would inform future survey design in this population.

While the eight participants in round 1 who completed the survey met eligibility criteria based on their responses to the telephone screening prior to in-person interviews, five others changed their answers to a few screening questions during the in-person pretest in a manner that would have made them ineligible because their cognitive impacts were too mild. However, these participants still provided useful feedback on the survey during the pretest interviews. Specifically, the input from these five participants helped clarify survey text and graphics that were confusing for many participants when compared with survey text and graphics that seemed confusing only for people with greater cognitive challenges. Researchers designing surveys for patients with cognitive decline might consider including respondents with milder problems during initial pretests of the survey instrument, thus identifying issues with the survey, before presenting the survey to patients with more advanced problems. The results highlighted the importance of carefully pretesting graphics used in any preference survey, to confirm that the graphics are easy to understand and support the interpretation of the attributes. The variability among individuals with cognitive decline may make designing and testing graphics difficult.

In round 2 pretest interviews, eligibility criteria regarding the time since speaking with a doctor and the use of symptomatic AD medicines were revised with the aim of targeting earlier stages of disease. In addition, this round of screening was conducted through an online screener, with a future goal to launch an online survey. Round 2 participants still discussed problems associated with cognitive impairment, consistent with an early AD clinical syndrome, but they were judged capable of independently completing the survey.

The main limitation of this work is the abstract beneficial attribute. There is no validated crosswalk between an instrument that has been consistently used in clinical trials such as the ADAS-Cog and a set of activities that are relevant and relatable to respondents. For this survey, categorization of ADAS-Cog-11 items from Appels and Scherder [50] and the scoring guide for the ADAS-Cog [34] combined were used to integrate the clinical and practical aspects related to incremental decline in memory and thinking. Specific levels for the five impacts were included to establish a reference level of memory and thinking problems and achieve a uniform baseline: hypothetically progressing from “some problems” (i.e., forget appointments once or twice a month) to “more problems” (i.e., forget appointments two to three times a week). When participants struggled to follow this instruction, a less uniform baseline was collected for the round 2 pretest, whereby participants were asked to use their current cognitive status as their baseline for the scenario rather than assuming a hypothetical baseline. Asking respondents to use their current cognitive status as the baseline makes it more difficult for the researchers to know how large a benefit each respondent assumed they would receive in exchange for the risks. This limitation may be present for any stated-preference survey designed to collect patient preferences on slowing cognitive decline given the complexity of cognition, the spectrum of AD clinical severity, and the diversity of patients with AD. The challenge participants faced by assuming a hypothetical baseline may be important for other health preference studies, as many studies ask respondents to place themselves in a hypothetical baseline. Again, careful testing of the survey instrument is important for the interpretation of the preference data gathered in a study.

Another limitation was the lack of formal assessment within the examples of whether moving from one level to the next would be rated as a “noticeably worse decline in memory and thinking.” Challenges of recall-based screening for those with cognitive impairment, while still present, are relatively minimized for the target population of an early AD clinical syndrome. On the basis of round 2 pretesting results, participants seemed capable of appropriate recall for study screening. Careful testing of screening questions is as important as testing other survey questions, to assure that the questions identify the sample of interest for the study. The survey has not been administered, thus results from the survey are not available.

6 Conclusions

This work is intended to provide guidance in patient preference research within the field of dementia. Taken together, our results suggest that the severity, not presence, of cognitive impairment determines whether a participant can independently complete a stated-preference survey. Thus, when recruiting those with cognitive impairment, care should be taken to characterize where participants may be along the clinical continuum of the disease impacting cognition. On the basis of round 2 pretest results, the refined screening questions and mode of screening successfully identified those with self-reported cognitive concerns consistent with an early AD clinical syndrome who could independently complete the refined DCE survey. The survey included careful drafting and revision of the attribute description that was meaningful and understandable to participants. Results also suggest that a simplified DCE survey for use in the target population is possible. Careful pretesting and inclusion of comprehension questions to assess respondent understanding are important for any study but are crucial for studies in populations with cognitive decline.

Patient preference research for disease-modifying AD treatments would be greatly enhanced by developing a clinical instrument suitable for both clinical measurement of cognitive and functional decline and patient preference studies that used relevant examples from a patient’s life. This study, along with ongoing work by Alzheimer’s Disease Patient and Caregiver Engagement [17], may inform regulators and other relevant decision makers on disease-modifying AD treatment assessments. Further, these results, possibly in combination with results forthcoming from IMI-PREFER (e.g., in those with neuromuscular disease [51]), may inform researchers and other stakeholders on preference research in persons with cognitive impairment.