FormalPara Key Summary Points

Digital health technologies such as wearable sensors allow another way to understand and measure the patient experience.

Clinical trial endpoints are evolving because of the rapid implementation of such digital health technologies into trials.

However, the implementation of these technologies is often disassociated from the patients perspective on their own health condition.

Here, we set out that clinical outcomes assessment science should be used to fully integrate digital health tools into clinical trials to create meaningful, patient-relevant endpoints.

A potential process flow is presented, and a need for pre-competitive collaboration is discussed.

Commentary

Endpoints in clinical trial design are the analysed parameters intended to test the efficacy of a study drug [1]. The field of endpoint development and validation, and trial design has rapidly evolved since the 2009 Food and Drug Administration (FDA) guidance [2], which described how to incorporate the patient voice into drug development and have patient-reported outcomes (PROs) included in a product label. Patient-centred endpoint development is an established process in patient focused drug development, starting with understanding what is important to patients (concept of interest), developing a way to measure this (conceptual framework), testing the resulting assessment’s measurement properties (validation) and adequacy to assess meaningful change (interpretation). It is now routine to measure the patient’s feelings or function in a clinical trial. Indeed, the 21st Centuries Cures Act mandates that the patient voice is incorporated into clinical trials and efforts must be undertaken to assess what matters to patients.

Digital health technologies (DHTs) such as apps, smartphones, sensors and wearable devices can be used as digital therapeutics to help administer treatment to patients, digital and remote monitoring of patient disease and medication compliance or as a measurement device for detecting the effects of an intervention in the context of a clinical trial. Here, we focus on the latter. The introduction of DHTs in the clinical outcomes assessment space may be perceived by some as a recent evolution, and while the pandemic has acted as an accelerator for DHTs to be adopted in clinical trials, their use in clinical trials has been growing for the last 10–15 years. DHTs can, either actively or passively, measure health-related data from the patient’s life, from heartbeats to blood oxygen, from steps to sleep. This ability is appealing when considering clinical trial endpoint development of a biomarker, defined by the FDA as “an indicator of normal biological processes, pathogenic processes, or responses to an exposure or intervention” [3]. The continuous measurement over an extended period of time increases the ability to detect change over time. However, the FDA definition also states that “a biomarker is not an assessment of how an individual feels, functions, or survives”; this is the realm of a clinical outcome assessment (COA) [2]. Therefore, despite being able to objectively catalogue a patient’s life, a key question for patient-centricity and COA endpoint development is whether DHTs can reliably and validly measure a patient’s lived experience.

The willingness and opportunities to include DHTs into clinical trials as endpoints has been rapidly increasing. One group that tracks the inclusion of digital health endpoints in trials is the Digital Medicine Society (DiMe). DiMe’s database [4] shows that > 49 individual clinical trials in Phase 2 or 3 have included DHTs measuring over 114 different endpoints as a secondary or even primary trial endpoint. Despite these advances, DHTs have yet to be leveraged in any FDA label claim.

In our opinion, the reason for the lack of approved labelling with DHTs is almost exclusively due to the well-intentioned, but misplaced energy associated with the excitement around these new tools. In an effort to be innovative, digital devices have been included in research programs without first establishing meaningful aspects of the individual’s health and clear delineation of whether assessments are biomarkers or rather COA [5] and being specific at the onset about what exactly the DHT is trying to measure [6, 7]. Furthermore, the research question the digital endpoint is going to address needs to be specified, with careful examination of how to analyse the data arising from the DHT in a way that relates the results back to an outcome that matters to patients. In short, we have the technological assessment tools, but not yet the associated meaningful digital endpoints that matter to patients.

To exemplify the issue, a search for applications of DHTs to measure digital outcomes on the aforementioned trial endpoint database reveals digital endpoints which assess, for example, step count or cough over a whole 24-h period, total sleep time and increase in blood oxygenation over time [4]. Despite these assessments being technically accurate, the authors do not note a single experience in their own lives where they have been conscious of their lived experience or quality of life improving because of changes in their daily step count or blood oxygenation. It is not that these elements are not important or not related to an individual’s health-related quality of life. In fact, it is likely quite the opposite: DHTs allow a way of measuring these concepts, assessing them in a way that was previously not possible and potentially using these as biomarkers for hard clinical endpoint assessment. Here, the problem is that these measures do not yet relate meaningfully to a patient’s life or to how patients themselves understand their feelings and function with a certain disease and treatment. As such, without linking the measured variable to the patient experience, they may be considered a digital biomarker, but not as a COA for the assessment of a patient-centred endpoint in the regulatory process. For example, a patient with a respiratory condition may want to be able to walk to the local store without having to stop to catch their breath. This act of walking to the local store is intrinsically linked to step count. At the same time, however, step count does not tell us anything about the patient’s ability to complete their journey in a manner which was important to them. A clinical trial program team could devise an endpoint around testing improvement in step count, could be successful in an assessment of this endpoint to show a treatment-related benefit but unable to translate that benefit into an outcome that matters to patients and ultimately regulators.

Nevertheless, there is an existing body of research which details the process needed to understand what matters to patients (conceptual model), how to convert this into a reliable and valid assessment, and how to use this assessment to develop a clinical trial endpoint which is meaningful to patients and ultimately regulators. This body of research is considered standard practice in pharmaceutical and biotech companies to develop patient-centred COAs. However, currently, this is mostly used in the process of developing questionnaires asked directly to the patient or to the patient’s caregiver or clinician. This process is, admittedly, more straightforward with questionnaires than with a digital outcome measured by a digital health technology tool. If a patient mentions an important concept, that concept can be formulated into an item in a questionnaire. However, there is not a huge leap from using a similar process in the design of digitally assessed COA endpoints.

It is imperative that research in the field of digital health starts with understanding what is important to patients as covered in detail elsewhere [6,7,8,9,10]. Once a meaningful aspect of health (i.e., the part of their health the patient wants to improve or does not want to worsen) is identified, a concept of interest can be defined and assessed. To build on our earlier example, a meaningful aspect of health in a respiratory condition might be the ability to move around the world freely without stopping for rest. A specific concept of interest might be uninterrupted walking capacity. Above, the hypothetical patient wanted to walk to the store without having to stop for breath. Once there is a good understanding of the meaningful aspect of health and the concept of interest, an assessment measure can be selected or designed, and a COA endpoint meaningful for patients can be derived. The selected assessment measure(s) may or may not include a DHT tool. The important part of this process is that a tool is selected which can relate to the patient experience and measures an outcome that matters to the patient. The full process is exemplified in Fig. 1.

Fig. 1
figure 1

A proposed pathway to defining a digital COA endpoint. This pathway is iterative in the sense that information gained in later stages may need to be explored again in the earlier stages of the work. For example, if as a result of the patient interviews, a device measuring walking bouts is chosen, it may be necessary to interview patients again to see their understanding of how walking bouts may relate to their life. Alternatively, if a device is selected, but perhaps has poor analytic validation, the research team could decide to re-review potential available devices rather than develop their own algorithm

Full size image

To complete the example started above, the patients may have expressed a desire to freely move without being impeded by their respiratory condition and needing to stop. Therefore, one patient-centric endpoint measure could be a questionnaire probing this concept of interest. However, in this case a DHT tool may be appropriate and offer additional insights from an objective assessment. Rather than measuring step count, a DHT tool could be used to measure continuous active minutes of movement in a single walking bout or the number of walking bouts recorded in each 1-h period of the day. These are just simple examples to help the reader to understand that these outcome measures are more closely related to the patients desired treatment goals (i.e., getting somewhere without having to stop and catch breath) than the overall number of steps. Digital outcome measures, like conventional COAs, benefit from concept elicitation interviews with patients, caregivers, and health care professionals for improvement of the developed digital outcome measure and confirmation of the measured concept of interest. This assessment can then be used to test a trial endpoint.

As shown in Fig. 1, the process does not end with understanding the patient. Once the variable of interest has been defined, a relevant sensor has been found and a potential device ascertained, further steps of validation are needed. The process of validation is described in detail elsewhere [11], but useability is a key issue that requires some attention. When considering the implementation of a device in a clinical trial, a preliminary study to understand whether patients can use the device appropriately is key to ensuring data are collected in a fair and non-biased way. Although onsite interviews with patients while they are interacting with the device can help make sure that they understand the basic functionality, there is merit in a more naturalistic approach. Useability studies that allow the patient to use the device in their day-to-day lives over a short period of time (e.g., a week) prior to being interviewed about their experience can lead to richer information. For example, a patient may often remove an accelerometer device due to discomfort, perhaps cannot install a sleep mat correctly or find they are unable to interact with the device app. This information can feed back into the iterative process of sensor and device selection. Furthermore, in the context of decentralized clinical trials, useability studies could offer an opportunity to test the logistics of conducting a partially or fully remote study. The investment at this stage is crucial and leads to less research waste at later stages in the trial.

We understand that this process could lead to different trials using different endpoints. This lack of standardization could hinder the ability to compare results across trials. Pre-competitive collaboration to bring together pharmaceutical companies and other stakeholders, such as that initiated by DiMe [12], is crucial. This pre-competitive work aims to conduct the science to define digital endpoints that are patient relevant, while making the research and resulting device specifications available to the broad scientific community. This information can then be used across research programs to allow for comparable results.

We strongly believe that patient-centred endpoint development is a process that starts, and ends, with the patient in mind. It is a highly iterative process from early drug development, throughout a product lifecycle and into the real world of clinical practice. DHTs are an excellent addition to the toolbox of collecting COA, but without patient input and a specific research question to be addressed, they are just collecting data points with limited probability of success and not delivering on patients’ needs and priorities.