Introduction

There is an increasing evidence that the introduction of the electronic health record (EHR) and the extraction of data from those systems have had a profound impact upon medical care, especially in the NICU. Furthermore, the use of such “Big Data” for the assessment of outcomes in thousands of infants and the evaluation of practice variation upon outcomes have already affected care in the NICU in a profound way. It is the purpose of this paper to examine how this change in medical care is progressing, what has already evolved with respect to neonatal outcome measures, and what can be expected in the near future.

The Electronic Health Record (EHR) in the NICU

The overall goals for EHRs, as outlined in the concept of “meaningful use” [1] by the United States Health IT Policy Committee, are relatively straightforward:

  • Improve quality, safety, efficiency, and reduce health care disparities.

  • Engage patients and families.

  • Improve care coordination.

  • Improve population and public health.

  • Ensure adequate privacy and security protection of personal health information [2]

In 1996, Pediatrix Medical Group anticipated the need for an EHR as the number of practices it managed continued to expand. With a growing patient population under its care (at present, nearly 25 % of neonates in intensive care units in the U.S. are cared for by Pediatrix clinicians), the company recognized the possibilities for both extraction of “meaningful” data in measuring outcomes, as well as the potential for examining how practice variation might affect those outcomes. Thus, the opportunity to develop a strategy for research investigations into comparative effectiveness measures was an early consideration in the development of a dedicated neonatal EHR and Data Warehouse.

Pediatrix has developed a proprietary EHR system called BabySteps® (Fig. 1). This EHR has served to gather data on a rapidly expanding patient population, while accurately coding for care according to American Academy of Pediatrics (AAP) Perinatal Section Coding Committee guidelines. Most importantly, however, early consideration was given for the best way to extract data for both clinical and research use, ultimately giving rise to the Pediatrix BabySteps Clinical Data Warehouse (CDW). At the present time, the CDW is believed to be one of the largest repositories of data on the neonate in the world, containing information on more than 1,025,000 infants and approximately 18,600,000 patient days. Because of the extent and depth of the data collected, the CDW has been queried not only within the organization for novel research observations, but also by the Food and Drug Administration (FDA), the National Institutes of Health (NIH), and the National Institute of Child Health and Human Development (NICHD), and numerous academic neonatology programs [3–6]. Many of these queries have resulted in publication in the peer-reviewed literature, and to date, more than 100 papers have been published that use the data from the CDW.

Fig. 1
figure 1

Part of the BabySteps admission note to the NICU

Full size image

Perhaps the most significant attestation to the value of this electronic system was provided by the American Board of Pediatrics (ABP), when it designated the CDW as an acceptable tool for use in their Maintenance of Certification (MOC) process, and named Pediatrix Medical Group as the initial Portfolio Sponsor in the MOC system introduced by the ABP in 2011.

BabySteps® Development and the Approach to Data Extraction

When Pediatrix Medical Group first implemented the installation of a proprietary electronic health record throughout its practices, several goals emerged. First, the EHR was intended to create a clear, easily-readable admission, discharge, and daily medical record note that was consistent throughout the organization (now numbering more than 200 practices, covering approximately 340 hospitals in 34 states plus Puerto Rico). In order to meet this goal, a series of concepts was defined as being critical for optimal EHR documentation, known as the 4 “C’s”:

  • Conciseness of notes

    • Reduce the daily note to the specific needs of the patient.

    • The daily note should, in general, only contain that day’s information.

    • Avoid carryover of excessive amounts of information from previous days.

    • Excessive verbiage should be excluded, since it breeds inconsistency in charting and increases liability.

  • Convey information to other caregivers

    • Notes must be easily readable.

    • Daily changes in the patient’s condition or the management plan should be immediately apparent to the reader.

    • Problems should be diagnosed, assessed, treated, resolved, and removed from daily notes as appropriate.

    • Simple recitation of numbers or laboratory reports does not constitute patient assessment, and further evidence of physician thought processes must be provided in the record.

  • Confirm clinical decisions

    • Notes should not simply recount numbers or events, but assess the patient’s clinical condition and provide a coherent approach.

    • Notes should be read carefully before being placed into the chart.

    • Confirmation of the clinical plan and the reasons for that plan are essential in assigning proper codes for the care being delivered.

  • Consistent internally

    • Notes should be consistent from the physical examination through the management plan. There should not be any discrepancies between the physical examination, laboratory values, radiographic studies, the assessment, or the plan for the patient.

    • Entry of information into fields, rather than as comments, is essential in refining the daily progress note and limiting inconsistency.

    • Inconsistencies are the most common problematic issue within the chart and are difficult to defend in malpractice cases.

There is a tendency for the EHR documentation to contain excessive detail, making evaluation of the daily note difficult, especially for consultants and reviewers, while potentially increasing liability risks. Clinicians sometimes erroneously believe that voluminous documentation aids in the care of the patient. Unfortunately, non-representative information is often carried forward unnecessarily, enhancing risks for liability. For example, cloning a physical examination over several days may indicate a “normal” exam from a prior period, yet later the note may have the child headed to the operating room. These inconsistencies undermine care and increase the risks for errors. To offset these problems, documentation training should be provided to all physicians and nurses in order to insure consistency.

In planning for automated data extraction, several important decisions had to be made. These processes are often overlooked during the adoption of the EHR, but are critical in the concept of “meaningful use.” Because every medical record note consists of drop down menus and text entryfields, questions arose if both parts of the note could be reliably used for data extraction. Ultimately, the issue was resolved in favor of drop downfield entry extraction only.

First, text areas of notes often are highly inconsistent in recording data. With no standardized definitions for many conditions, clinicians record diagnoses in a highly variable manner. Secondly, the authors assessed how a few specific diagnoses were designated within the text areas of the note to assess the variability of documentation. As an example, neonatal intraventricular hemorrhage (IVH) was mentioned in the text in more than 1,700 different ways! A small, partial list of these is shown in Table 1, all of which would need to be examined for if the text boxes were to be accurately reviewed. This process would dramatically slow data extraction, limiting the value of the CDW itself. It was therefore felt that limiting extraction to the drop down menu fields would be the optimal approach.

Table 1 Designation variability for intraventricular hemorrhage, partial listing
Full size table

Goals for Data Extraction from BabySteps© and Any Electronic Record

The ultimate objectives for data management were also identified. The goals for data must be carefully considered, since it is frustrating to have large volumes of data that are not useful. Most importantly, outcome data from any EHR should be automatically extracted from the record, not transcribed by hand for input into a database. Repeated transcription of data provides a serious potential source for error and undermines the value of the information in the database.

Automated data extraction has several additional benefits. Automated data extraction eliminates the bias that might creep into the information with multiple data extractors. For example, some referral centers may exclude some patients referred for high-level tertiary or quaternary care, since they affect outcomes negatively. Babies sent for laser surgery for Retinopathy of Prematurity (ROP), or infants with complex congenital malformations tend to adversely influence NICU outcomes and lengthen stays because of complex management issues. Shouldering the burden for these patients in a database often leads to results that appear less than optimal and they may be edited out. Automated extraction of information, therefore, leads to a much more precise data set, from which more accurate conclusions can be drawn.

With any attempt at data extraction, a number of issues become evident. Each practice needs to be aware of its evolving outcomes. Often, when one views outcome data for the first time, one is often left with a sense of disbelief. The goals of authors, therefore, have been to assist physicians in assessing their data so that they can focus on outcomes and work towards improvement, the primary reason for this information.

It is essential to provide outcome data as a graphic report, which yields a more indelible impression than a table. In addition, outcome data in the abstract are not meaningful, and the data must have some basis for comparison. Even with risk-adjusted data, if you do not know how your neonatal colleagues are doing, outcomes have little value. In authors company of that big size, the obvious comparison would be the rest of the patient population in Pediatrix Medical Group, at least as a starting point. Each report, therefore, would have to present outcomes from not only the individual practice, but from other Pediatrix practices as well. These outcomes had to be provided in an easily understandable format. One should be able to see at a glance how one’s NICU is doing compared to other NICUs.

Since outcome data were going to be used in a comparative manner, it was essential that the Clinical Data Warehouse be made HIPAA (Health Insurance and Portability and Accountability Act of 1996) compliant. The data set required eliminating all patient-identifying information. To accomplish this result, data “cleansing” excluded information such as day, month and year of birth, date of admission, date of specific therapy initiation, etc. Furthermore, events were recorded as days since birth to eliminate any possibility of discovery of a baby having an unusual event on a specific birth date that might be traceable. Research use of the de-identified data set is approved annually by the national Western IRB. In the publication of studies, however, authors also require IRB approval from any institution, company, or university whose investigators query the Data Warehouse. More than 100 peer-reviewed papers have been published during the last fifteen years with information from the CDW.

Elimination of selection bias was also important, since there is a tendency to avoid taking credit for patients who might have adverse outcomes, especially in transport situations. By collecting data on all patients, and allowing comparisons of inborn and outborn patient populations and NICUs of similar size, controls were built into the CDW to avoid situations in which large referral institutions were somewhat unfairly penalized for accepting the most difficult cases in transport. While some databases either risk-adjust or assign severity indices to outcomes, it was authors belief that the actual data, with selection of appropriate comparison groups, was a preferable way to evaluate outcomes. Finally, to examine any regional variations in outcomes that might be of interest, the CDW also allows grouping of all hospitals covered by an individual practice to allow comparison to other groups within the state or a region of the country.

BabySteps Data Warehouse Reports

The CDW is constantly evolving to provide the practicing physician with the most current data possible and as many report types as is practical for the management of the patient in that NICU. From clinician feedback, the CDW has become amarvelous tool for assessment of outcomes and quality improvement efforts. Data within the CDW is refreshed on a weekly basis.

At the present time, Table 2 shows the list of CDW reports that are currently available. Reports are categorized into several types:

Table 2 Current Pediatrix Clinical Data Warehouse reports
Full size table

Activity Reports

indicate basic demographic types of information.

Management Reports

reflect decision-making processes in patient care.

Morbidity and Mortality Reports

documentation of a variety of common types of outcomes of greatest interest in NICU patients.

Summary Reports

reveal a selected one-page snapshot of outcomes for a specific NICU, or network trends in a variety of outcomes that are constantly tracked.

The Summary Reports were developed as a request from the regional management teams, who are each responsible for approximately 40–70 practices in one of six regions of the country. So that they could get a sense of the quality of care being delivered in an individual unit, or all units within the region, the Summary Report was developed to provide a snapshot picture of overall outcomes and is shown in Fig. 2. Overall, the Summary Reports provide an instant snapshot of a year’s important outcome measures and can be assessed within minutes. Should the observer desire more data, he or she could then turn to the more detailed reports.

Fig. 2
figure 2

Annual Summary Report from the Clinical Data Warehouse with selected indicators in 14 different areas. The grey bars on each gauge represent the 33–66th outcome percentiles (middle third) for all Pediatrix Medical Group NICUs

Full size image

A typical Clinical Data Warehouse Report is illustrated in Fig. 3. Reports can be viewed for yearly, quarterly, or monthly time periods. Many morbidities, even in large NICUs, occur so infrequently that annual reports often work best, but the clinician has the option to examine an outcome during various time frames, which is helpful in quality improvement projects.

Fig. 3
figure 3

A typical Data Warehouse report for catheter-related infections. The rate of catheter-associated blood stream infections for an individual NICU is shown as connected dots between 2009–2013. The rate is shown as number of infections per 1,000 catheter line days. The grey background shading represents the 33rd–66th percentile for all High Volume (>450 discharges per year) Pediatrix Medical Group NICUs as a comparison

Full size image

The clinician may filter report results by multiple combinations of birth weight and gestational age selections, to facilitate detailed “drill down” for various outcomes. Results can also be filtered by admission status - inborn, outborn, or combined. The selected filtering parameters are used to provide the specific network comparison group. This can be even further refined by selection of high (>450 discharges annually), medium (225–450 discharges annually) or low (<225 discharges annually) volume NICUs as the comparison NICU group (Fig. 4).

Fig. 4
figure 4

The panel on the left shows the various options for reports that can be selected: Overall Report Group, Specific Report, and Report Format at the top of the Figure. The period of time can be selected by year, quarter, or month. Various gestational age or birth weight combinations can also be chosen to filter reports. Admission status (inborn or outborn) is available as an additional filter. Only surviving infants may also be chosen as a group to be examined. The comparison network group may be selected based on annual NICU volume, specific region, or state

Full size image

The report illustrated in Fig. 3 examines catheter-related blood stream infections between January 1, 2009 and December 31, 2013. The annual rate of infections, measured as the number of infections per 1,000 line days, is shown for this specific NICU as dots connected by a solid line from year to year. In the background in grey is the 33rd–66th percentile for all of Pediatrix Medical Group. As can be seen during this time period, the infection rate in this NICU was declining, as was the rate for all NICUs in Pediatrix, partly the result of an increasing CQI focus on reduction of catheter associated blood stream infections. It is apparent that run-chart methodology is extremely useful in assessing whether interventions in a specific NICU have value over time.

Some reports cannot be provided easily in graphic form in the Data Warehouse, but have great value for patient care and quality improvement nonetheless. The medication report is an example of a report in tabular format, again allowing the clinician to examine the use of the common neonatal medications in their NICU, while using the scope of Pediatrix Medical Group as the basis for comparison. Essentially all medication classes used in the NICU are examined in this report.

The Data Warehouse as a Quality Improvement Tool

As noted earlier, the Pediatrix Clinical Data Warehouse is valuable for looking at many common neonatal outcomes, as well as basic population data. While the provision of outcome data can itself have a strong effect in improving care, the effect can be magnified through the use of directed quality methodology, illustrating how an EHR can provide “meaningful use.”

The CDW has served as the basis for many quality improvement interventions during the past several years, and several toolkits have been devised to utilize this resource. In one of the most notable examples, in 2004–2005, a corporate-wide program was initiated that focused on reducing the rate of severe Retinopathy of Prematurity (ROP). This project, known as COMP-ROP, for Comprehensive Management of ROP, provided educational programs, nursing and physician assessment of their understanding of ROP pathophysiology, slide presentation, parent information, isolette stickers for correct oxygen level management, and several other informational programs for both caregivers and parents. The CDW was used to follow ROP rates, which have improved markedly since that time. From 2003 to 2013, a striking decrease in severe ROP (stage 3,4,5, or surgically treated) was seen in the Pediatrix Network. In infants with birth weights of 400–1,500 g, severe ROP dropped from 12 % in 2000 to 2.4 % in 2013 (Fig. 5). Given the fact that Pediatrix Medical Group admits approximately 20,000–25,000 extremely low birth weight (ELBW) and very low birth weight (VLBW) infants annually, the reduction in rates of severe ROP, the leading cause of childhood blindness, represents an important public health initiative, well documented in the Data Warehouse.

Fig. 5
figure 5

Network trend reports between 2000–2013 for severe retinopathy for prematurity (ROP) – Stages 3–5 and infants surgically treated

Full size image

Similarly, other Pediatrix quality improvement toolkits have also resulted in significant wide-scale outcome improvements in areas such as: improved antibiotic stewardship, decreased catheter-related infections (Fig. 3) and late-onset sepsis, enhanced growth, improved temperature stability from the delivery room to the NICU, decreased use of drugs with limited evidence of therapeutic value for the neonate (e.g., metoclopramide, H2 blockers, spironolactone), and reduction of necrotizing enterocolitis.

One of the most interesting recent reports recently made available in the CDW is the individual NICU comparison report (Fig. 6). While comparisons have been part of the visual reports in the Data Warehouse, the new report allows the identification of “outliers,” whose outcomes fall below acceptable standards.

Fig. 6
figure 6

Comparison report showing an individual NICU compared to other regional NICUs with respect to human milk use in the first week of life. The dark horizontal line is the average for all NICUs examined. The lighter lines represent one standard deviation from the mean. The group of NICUs on the right hand side of the figure that lie more than one standard deviation below the mean for human milk feeding may be considered “outliers” and be targeted for quality initiatives to improve human milk use rates

Full size image

In practice, Pediatrix corporate quality improvement projects have started with the identification of a pressing outcome concern to be addressed. The CQI team reviews the literature, defines the appropriate evidence necessary to support the project, and proceeds to build a toolkit. The toolkit contains reprints of key publications from the literature, slide presentations for the medical and nursing staffs, an operations manual that describes the methodology for rolling out the project, and any ancillary materials needed for project management. Toolkits are finally posted on the Pediatrix web site for the practices to review, use, or modify as needed. The toolkits do not represent specific recommendations on how to practice, but rather provide a series of suggested approaches for NICUs looking to improve their outcomes. Each practice can tailor the toolkit to their particular unit’s needs. Timelines are then established for projects and enrollment is initiated for units desiring to participate. The Data Warehouse serves as the primary method of following outcome data.

As seen from the data extracted from the Data Warehouse, a tool at this level of sophistication allows for meaningful healthcare improvement and cost savings. It further allows identification of practices that fail to meet (or exceed) goals for outcomes, referred to as an “outlier.” Many questions revolve around outlier practices: are they outliers that are doing exceptionally well, or exceptionally poorly; are they outliers in several areas or just an isolated part of practice; can relationships be drawn between a particular practice strategy and the outcome results; how can adverse outcomes be corrected, etc. The thoughtful application of evidence-based approaches with data can also add immeasurably to parent satisfaction. Hospital partners also appreciate knowing that the services they offer in the NICU produce the best possible results.

The BabySteps© Research Data Warehouse

Given that the BabySteps record is scoured daily for 563 data fields, when combined with the extraordinary volume of patients in this database (now >1,000,000), the potential for novel research observations from “big data” cannot be overlooked. In fact, the Data Warehouse has been queried by the NIH (National Institutes of Health) and the NICHD Neonatal Network, the Food and Drug Administration (FDA), major universities, and private corporations, who have sought data not available elsewhere. Some publications from the Data Warehouse are provided in the references.

As an example of the utility of the CDW, in 2005, the authors were contacted by the FDA, which was interested in knowing the 30 most commonly used medications in the NICU. These data were furnished to the FDA and published in Pediatrics [3]. It was also noted that three of the five most commonly used medications were antibiotics: ampicillin, gentamicin, and cefotaxime. Since the most common neonatal admission to the NICU is suspected septicemia, these drugs represented the two most common antibiotic regimens used for suspected sepsis, ampicillin and gentamicin or ampicillin and cefotaxime. The two approaches were initially thought to be equivalent. When the authors examined the outcomes in more detail, however, they discovered that the use of cefotaxime had nearly a two-fold (100 %) greater association with mortality at certain gestational ages than did gentamicin in this patient population [7] (Fig. 7). Extensive attempts to eliminate confounding variables through the use of logistic regression analysis strongly suggested that this finding was indeed correct. Because of this risk, the use of cefotaxime has fallen to an extremely low rate in the NICUs across the country. This observation, which might have gone unnoticed otherwise, became apparent only when large numbers of patients were evaluated. Most recently, the research CDW has been used to develop a screening tool for the potential development of NEC [8], examine the impact of duration of rupture of membranes on outcome of premature infants [9], assess the intrauterine growth [10] and growth of premature infants [11], examine clinical outcomes and caffeine use [12], follow trends in discharge planning needs for premature infants [13], re-examine changes in medication use in the NICU [14], evaluate renal function in the pre-term infant [15], and several others.

Fig. 7
figure 7

Adjusted odds ratios for mortality rate of ampicillin and cefotaxime compared to ampicillin and gentamicin. 95th confidence intervals are indicated [7]

Full size image

The authors continue to expand the Data Warehouse capabilities for research and constantly invite inquiries for it use. One of the greatest assets of this type of database is that not only does it provide answers to many outcome questions, but it can also serve to model prospective trials and studies in which targeted data collection is required. It is authors’ belief that there is much more information that can continue to be extracted and evaluated that will have additional significant benefits for the practice of newborn medicine.