In the real world, we lack evidence guiding the use of medications and devices in children. This lack of evidence arose out of the challenges of conducting clinical trials in children and other vulnerable populations and the historical decision (reversed in recent decades) to exclude children from clinical trials [1,2,3]. As a result, large percentages of drugs and devices used by children are unlabeled for pediatric use or for some age sub-groups [4,5,6]. Despite efforts through legislation and increased allocation of resources to clinical trials to attain labeling for all medications and devices used by children, it seems unlikely that it will be possible to conduct clinical trials for every medication used by children and for all relevant indications. The widespread off-label use of drugs and devices in children creates an ecosystem of real-world evidence (RWE) that may be the only source of information about efficacy and safety in children.

The recent focus on the potential of RWE to guide approval and use of new treatments may provide a much-needed solution [7, 8]. A broad definition of RWE is “information on health care that is derived from multiple sources outside typical clinical research settings, including electronic health records (EHRs), claims and billing data, product and disease registries, and data gathered through personal devices and health applications” [7]. The interest in RWE arises from the limitations inherent in relying on randomized controlled trials, as well as growing opportunities to acquire real-world data from a range of sources. These include the high costs and length of time required to conduct such trials, the narrow focus of most randomized trials regarding efficacy and safety, the inability to answer a broader range of questions regarding effectiveness, factors contributing to the efficacy–effectiveness gap and the full range of side effects, and difficulty in generalizing findings to “larger, more inclusive populations of patients, providers, and health care delivery systems or settings that reflect actual use in practice” [7, 8]. At the same time, the increasing use of EHRs and the development of linked data resources, standard patient-reported outcome measures, and E-health and M-health tools to collect data open efficient avenues of data acquisition that cover broad sectors of the population.

Those working on pediatric medication labeling often cite the mantra “Children are not just little adults” to stress the unique challenges in studying children. How would this mantra apply to RWE? Are there issues around RWE that are unique to studying medicines and devices in children? For the most part, it is reasonable to expect that considerations around the use of RWE in adult populations will apply to its use in children; however, a number of issues are unique to studying children.

The first and most critical of these is that many databases suited for studying adult populations may not meet minimal needs required for studying children. They may not include any children, may include only sub-populations of children, or may include small numbers of children. For example, in the USA, the Medicare databases that are a rich source of data when studying medications in people aged ≥65 years only include individuals aged <20 years if they have end-stage renal disease. Even if a database includes children, such databases in the USA, Canada, and Europe will have fewer children than adults because of the age structures of their populations, and sample sizes may be inadequate for studying children. In 2015 in the USA, 22.9% of the population was aged <18 years, and in most databases fewer than one in four records will be a child’s record. When we seek information about age sub-groups within the pediatric population (e.g., infants aged <2 years, or school-age children aged 5–11 years), the numbers become correspondingly smaller. This is critical because medication use is lower in children than in adults, and the numbers of children needed in such data sources to detect clinically meaningful differences in effects become larger. Development of databases to study sub-groups such as neonates can provide a resource for studying such sub-groups and accruing larger sample sizes for the sub-group. Examples include the International Network for Evaluation of Outcomes of Neonates (iNEO) and the Vermont Oxford Network.

It is intuitive that datasets need to have adequate numbers of children if they are to be useful in providing RWE about medications and devices used by children. It is less obvious that different types of data may be required when studying children. At the top of the list is birth date, specifically month and date of birth, a variable so basic that we overlook the challenge in accessing it in de-identified real-world databases. Databases are de-identified by removing personal information, and birth date is often one of the first variables stripped from a record. This may not be a huge problem when studying adults or older children, but birth date is an essential variable when studying newborns, infants, and young children, where we wish to study events over days, weeks, or months [9]. Close behind month and date of birth in importance are variables such as birth weight and gestational age, which are sometimes available in an EHR but often missing from administrative or claims databases. A comprehensive analysis of the data elements needed in pediatric EHRs has outlined functionalities with regards to newborn screening, vaccines, medications, and other domains; however, these recommendations have only been recently released and have not yet been fully implemented [10]. Thus, databases and data elements need to be adapted to the unique requirements of collecting RWE in pediatrics. A recent analysis of database systems for use in post-marketing surveillance of adverse events concluded, “There is a tremendous need for pediatric systems that include the essentials in measuring post-marketing safety: clinical detail at all ages of childhood, information on the number and descriptors of those exposed to medicines (denominator data), and a large enough system to detect rare AEs” [11].

Another challenge unique to studying medications in children is that of linking information in the child’s record to that of the mother, father, or siblings [9]. This is desirable when attempting to include variables such as age, height, and weight of the mother and father, maternal medication exposure during pregnancy, or risk factors such as substance abuse, obesity, socio-economic status, and other conditions, all of which may affect medication exposure and children’s health outcomes and introduce potential confounding. While information about socio-economic status may be missing in studies of adults, and proxy variables such as address and type of insurance can be used to indicate a child’s socio-economic status in some settings, pediatric studies require information about parental income, education, and related variables to determine the child’s socio-economic environment. Development of methodology to use RWE to address questions in children will need to outline best practices in implementing such linkages, and situations in which such linkages might not be required. Additionally, it will be important to understand the factors that may affect record linkage and biases that may ensue as a result of non-random success in record linkage. Linkage between maternal pregnancy record and infant’s record has been demonstrated to be feasible in the US Medicaid Analytic eXtract files [12]. The ability to link records varied greatly by state, reflecting differences in data systems, coverage, and other factors. In the UK, Harron et al. [13] demonstrated that the ability to link baby and mother records improved over time, from 91% in 2001–02 to 98% in 2012–13. Studies using record linkage will need to adjust for the factors, such as year and geography, that affect the probability of successful linkage and inclusion of a mother–infant pair in the study population. When parents and children receive healthcare insurance from different sources, the challenges to record linkage are greater. In 2015 in the USA, 39% of children (defined as aged <19 years) were covered by Medicaid or the Children’s Health Insurance program (CHIP) compared with 15% of adults aged 19–64 years [14]. In many of these cases, the parents may not be eligible for Medicaid or may receive health insurance through employers. As a result, data for children and parents may reside in separate databases, and record linkage from child to parents will require access to both databases. In the special case of neonates, greater efforts have been made to link the infant’s record with parental records. A recent review identified 516 studies reporting perinatal health record linkages internationally, with the largest number occurring in the Nordic countries, followed by the USA, the UK, Australia, and Canada [15].

In addition to issues around the measurement of outcomes in children, and the need to validate outcome measures for use in pediatric populations, whether in a clinical trial or in a real-world setting, the need for school performance data poses an additional challenge to researchers intending to use RWE in pediatric studies. Researchers studying medications and devices in children are often concerned about outcomes such as days missed from school, promotion to the next grade level, behavior issues, and test performance, all of which can be ascertained through school records. It is highly desirable to link data about drug and device utilization with outcome variables such as school performance to describe the real-world benefits and risks associated with medications and devices used by children [9]. School performance is an outcome of interest when studying psychotropic medications, liver transplants, and other conditions, and it is possible that some researchers may find it of interest to also describe extra-curricular performance and participation [16, 17].

Issues around the use of RWE in pediatrics fall into at least four categories: (1) identification of databases with adequate numbers of children in the age sub-groups of interest, (2) access to critical variables such as birth date, birth weight, and gestational age, (3) linkage to parental records for information about pre-natal exposures and socio-economic status, and (4) linkage to school records for information about outcomes such as missed school days, academic progress, and behavioral issues. If we can address the methodologic challenges of RWE, we may have new sources to inform regulatory decision making. This may be especially valuable with respect to labeling medications for children. As efforts increase to address the challenge of developing high-quality RWE, we must consider and address the special considerations and challenges in using RWE to answer questions about the effects of medications in children. Asking these questions now will ensure that advances in the use of RWE will benefit children as well as other sectors of the population.