Randomized clinical trials play an important role in the advancement of science, pursuit of improved health, and quest for more effective delivery of health care. Patients, policy makers, and the medical community recognize these trials as the gold standard in the development of evidence on which to base the delivery of medical care [1]. In spite of their importance, the conduct of these trials is challenging for a variety of reasons, including their cost and complexity to perform [2]. The field of applied clinical informatics is increasingly well-positioned to facilitate many of the traditional steps required for the successful conduct of a prospective randomized clinical trial, potentially catalyzing the efficient generation of high-quality medical evidence and incorporation of the medical record into a learning health system [3, 4].

Most randomized clinical trials follow a relatively set sequence of six steps. A population of patients is screened for eligibility by study inclusion and exclusion criteria. Those patients who meet criteria are considered for enrollment in the trial. Enrolled patients are randomized into study groups (i.e., control vs. intervention(s)). The assigned intervention is delivered and the rate of receipt of study intervention is recorded. Data are collected about patients’ response to the intervention over the course of the trial and outcomes at the end of the trial are measured. Data are synthesized and analyzed to allow interpretation of the results of the trial.

Now more than ever, there is burgeoning potential for clinical informatics to facilitate each of these steps. “Sniffing” applications can automatically screen populations of patients for eligibility, by comparing defined inclusion and exclusion criteria with data housed in an electronic medical record [5]. These applications can then be used to flag potential study participants [6]. Once patients are enrolled in a study, randomization assignment can be automatically generated using a real-time randomization scheme built into a study application. Delivery of the assigned intervention can also be facilitated by informatics (e.g., an advisor in the computerized provider electronic order (CPOE) entry system that directs providers to an assigned medication, or an automatic generation of an order to pharmacy, laboratory, etc.) [7] or the informatics applications themselves may be the study intervention (e.g., a clinical decision support system to aid providers in the management of a clinical condition) [8, 9]. Through the automated collection of clinical data from existing sources which make up the larger electronic medical record (e.g., CPOE, pharmacy logs, clinical monitoring systems, laboratory systems, billing systems, registration systems), informatics can provide information regarding compliance with the assigned study intervention (e.g., dispensing of assigned medication by pharmacy, administration by bedside nurse), changes over time in response to the intervention (e.g., changes in physiologic measures and laboratory values over the course of the study), and trial outcomes (e.g., death, hospital length of stay, development of organ dysfunction). Although these data are typically collected in raw form and then analyzed at defined time points in a study, informatics can also help synthesize, analyze, and present the data in real-time to facilitate monitoring and conduct of the study. (e.g., dashboards for study personnel detailing enrollment, compliance, and safety measures or interim analyses in Bayesian design studies).

The steps of a clinical trial that are best augmented by clinical informatics depend on the design of the trial and the electronic infrastructure available at the study performance site. Similarly, the potential limitations of an informatics approach at each of the steps differ depending on the specific trial design. A fundamental decision-point is whether the informatics applications themselves are a part of the study intervention, or are simply tools to facilitate examination of an unrelated question. If the study intervention is an informatics tool (such as a computerized clinical decision support system) then careful attention must be paid to best practices in the development of electronic tools. Such interventions should provide decision support that is embedded in an existing workflow, without relying on clinician initiative [10, 11]. Additionally, to be successful, these types of interventions should deliver decision support at the time of decision making, and provide actionable recommendations rather than just assessment [10, 11]. They should be integrated, to the degree possible, with the electronic systems being used and require minimal data entry on the part of the clinician. Attention to system speed, user interface, timing of the intervention, clinician time savings, startup costs, alert fatigue, information overload, local environment, periodic performance feedback, and the manner in which the tool is introduced to providers may be more important to the success of the trial than the actual content of the informatics intervention [9–12]. It is therefore critically important when reporting on the success or failure of an informatics tool intervention, to describe carefully the particular context in which it was implemented.

In contrast, when informatics are not part of the study intervention but are being used as tools to identify, enroll, and follow patients, the challenges are different [13]. In these cases, validation that the informatics approach is performing the assigned function in a manner comparable to how it would be performed in a traditional trial is essential. For example, if a “sniffer” is used to identify hospitalized patients eligible for a trial, but fails to screen the records of patients coded as “24 h observation,” the application may systematically introduce a selection bias that might not be readily evident to investigators from generated lists of eligible patients. Another example relates to study data collection. If body mass index is an outcome, traditional data collection by a study nurse might automatically convert values in which weight was recorded in kg but height in inches, whereas an informatics application might fail to identify unit conversion errors. Recognizing and appropriately handling the noise that accompanies the automatic extraction of data from electronic systems is crucial to detecting true signal from a study intervention.

In spite of these challenges, the deployment of informatics tools to facilitate the conduct of clinical trials has the potential to provide tremendous cost savings. A traditional budget for patient enrollment in an NIH-funded prospective randomized clinical trial performed in an inpatient setting is $10,000–40,000 per patient [14]. While some of this cost goes toward the study intervention itself (e.g., generation of intervention or distribution of placebo pills by a study pharmacy), the majority of the cost provides for study personnel and the time required to screen patients, enroll subjects, collect data throughout the trial, and maintain study databases. Leveraging informatics to automatically perform a portion of these functions may allow conduct of smaller trials without any funding, conduct of traditionally-sized trials at a fraction of the traditional cost, or the conduct of trials too large to previously be attempted.

There are also components of traditional prospective randomized clinical trials for which informatics have not yet been able to be leveraged. For trials requiring informed consent, there are provisions for consent submission electronically or by facsimile, but little progress has been made in identifying approaches to satisfactorily detail the risks and benefits of participation in order to obtain informed consent without the direct involvement of study personnel. However, there is promising work around the development of interactive systems which may enable patients to give informed consent electronically without the direct involvement of a study member [15]. In addition to the limitations involving consent, data collection via informatics approaches has largely been limited to data collected for clinical care or surveys. While this is consistent with the design of many pragmatic trials, many traditional randomized trials have collected granular physiologic and laboratory evaluations which have proven instrumental in understanding the study intervention and outcomes of the study. Developing new approaches to identify and utilize discarded clinical specimens and other routinely available samples may help bridge this gap and allow more detailed explanatory approaches via informatics-driven data collection.

In the future, we anticipate increased utilization of clinical informatics to facilitate the conduct of prospective randomized trials and optimize the medical record as a vital tool of the learning health system. Given the mounting financial pressures across the health care system and the relatively limited resources available for the conduct of clinical trials, the integration of informatics-based techniques will be essential to optimizing the value of clinical trials going forward. In the ideal state, our electronic systems will automatically and continuously generate new knowledge and recommendations that enable the delivery of better care. By understanding its strengths, limitations, and optimal orchestration, we have the opportunity to make informatics-enhanced clinical trials the new gold-standard in the generation of evidence in medicine.