Background

A hospital-based birth defects (BD) surveillance system captures birth defects that occur in selected hospitals within a defined geographic area [1]. Most low- and middle-income countries do not have accurate data on the prevalence of birth defects due to a lack of diagnostic capabilities, resources, and awareness of available services, as well as the absence of a birth defects surveillance system [2, 3]. Lack of routine, systematically-collected, accurate data can inhibit the ability to develop, monitor, and assess birth defects prevention and intervention activities [4].

Integrating a successful birth defects surveillance system into an existing hospital structure can be a daunting task. Challenges include, integration of surveillance activities in busy delivery units, burden on staff to conduct additional activities, such as; reporting BD data, recording the clinical presentation of any identified birth defect [5], rigorous reporting structures/hierarchy for surveillance midwives conducting birth defect activities, and limited space, equipment and supplies to successfully conduct surveillance activities [6]. Strengthening health workers’ capacity for identification and diagnosis of birth defects is crucial. It fosters development of competencies and skills needed to correctly diagnose birth defects and establish an effective hospital-based birth defects surveillance program.

The objectives of this study were to describe the process for strengthening capacity of health workers to diagnose external major birth defects in four large Ugandan hospitals and to examine the concordance of these diagnoses to those of birth defects subject matter experts (SMEs) at the U.S. Centers for Disease Control and Prevention (CDC) to assess the competence of surveillance staff in diagnosing birth defects and whether or not this improved over time. Positive predictive value (PPV) between 2015 and 2018 and 2019–2021 for both surveillance midwives and the Makerere University-Johns Hopkins University (MU-JHU) Research Collaboration clinical team were compared.

Methods

In 2015, Makerere University–Johns Hopkins University Research Collaboration, in collaboration with CDC, implemented a hospital-based birth defects surveillance system in Kampala, Uganda. This was the first birth defects surveillance system in Uganda. The surveillance system was designed to provide accurate prevalence estimates of 17 major external birth defects (i.e., anencephaly, iniencephaly, encephalocele, spina bifida, craniorachischisis, microcephaly, anophthalmia/microphthalmia, anotia/microtia, cleft palate alone, cleft lip alone, cleft lip with cleft palate, imperforate anus, hypospadias, talipes equinovarus, limb reduction, gastroschisis, and omphalocele) [7] at four large hospitals: Kawempe National Referral Hospital; Mengo Hospital; St. Francis Hospital, Nsambya; and Uganda Martyrs Hospital, Lubaga. Definitions of these birth defects are presented in Table 1 of the additional file. All birth defects were included - no exclusions were made based on diagnosis of a syndrome. Results are presented by birth defect and not by infant i.e. if an infant had more than one defect, then each defect was included as a separate case for that particular defect. The MU-JHU clinical team consisted of two pediatricians (the principal investigator and a co-investigator), one physician serving as the program manager and one hospital liaison coordinator.

Process for strengthening capacity of health workers

The initial training for MU-JHU clinical team included an online 7-week pre-course training and a 5-day in-person birth defects surveillance workshop conducted in 2015 by the International Clearinghouse for Birth Defects Surveillance and Research (ICBDSR) and the U.S. CDC [8]. The workshop focused on increasing awareness of the importance of birth defects surveillance, the need for establishing a comprehensive program to collect accurate and timely data, and the utilization of these data to plan for implementation and evaluation of birth defects prevention programs [8].

The trained MU-JHU clinical team and data managers collaborated with CDC to further develop the surveillance protocol, standard operating procedures (SOPs), data collection logistics, and data collection forms. Additional surveillance study staff (i.e., surveillance midwives, research assistants, study consultants) were recruited to conduct the initial examination of all newborns and collect data, support quality control/assurance, and conduct bedside confirmation of birth defects at the hospitals.

Surveillance study staff were trained by CDC and the MU-JHU clinical team on the study protocol, SOPs, good clinical practice, data collection, as well as the systematic process of examining all informative births for major external birth defects [6]. Additional engagement to further strengthen capacity of health workers to diagnose birth defects included: clinical workshop trainings [8], mentoring/coaching, customized birth defect protocol trainings, hands-on trainings with support from the MU-JHU clinical team/supervisors/experienced surveillance staff, data managers, and feedback from the MU-JHU clinical team to surveillance midwives on birth defects cases. Furthermore, formalized in-house continuous medical trainings and hands-on trainings, led by MU-JHU clinical team, were organized to upgrade professional skills and competence in diagnosing birth defects, among surveillance midwives.

Diagnosis and verification of the birth defects are done at several levels to ensure accuracy. First, the surveillance midwives conduct the initial examination of each newborn to identify any external birth defects. Newborns are examined when they are clinically stable and external birth defects are documented in the relevant study logs at the hospitals. Data are collected electronically on android-based tablets using the Open Data Kit (ODK) platform [5]. Next, selected study staff (i.e., study investigators, program manager/clinical consultants, hospital liaison coordinator, nurse coordinator and data managers) are notified about the identified birth defect through a phone call or text message. The program manager/clinical consultant, who can be an on-call clinician, pediatrician or obstetrician-gynecologist, performs bedside examination of the newborn to confirm the presence of any birth defects and determine if the birth defect is of interest to the study. If the program manager/clinical consultant on-call confirms that the birth defect is of interest, maternal informed consent is obtained by the surveillance midwife to obtain photographs of the fetus’ or newborn’s birth defect, which is then used for case review and classification by the MU-JHU clinical team and CDC birth defects SME.

In all hospitals, if the program manager/clinical consultant is not available to perform bedside confirmation or if consent for photographs of the case is not obtained, the surveillance midwife draws an illustration of the defect and writes a brief narrative description of the condition. This visual data along with the surveillance information are then entered into a tablet for further review by another surveillance midwife and then submitted to the MU-JHU central server. Thereafter, the birth defects records are extracted by the data managers and shared with the MU-JHU clinical team for review and classification.

This process is followed in all hospitals except at Kawempe National Referral Hospital, a high volume government hospital where up to 70 mothers give births daily and close to 24,000 babies are delivered annually [9]. The surveillance midwives at this referral hospital were further trained to conduct bedside confirmation of birth defects in absence of a program manager/on-call clinical consultant. If a surveillance midwife identifies a birth defect, she informs a second midwife on duty to also examine the baby. The birth defect is confirmed if there is agreement by the two midwives. If there is disagreement, a third midwife examines the newborn and a diagnosis is established based on two concurring assessments. If all three midwives disagree on the diagnosis, then the examination findings and all data (photographs, illustrations and narrative description) are submitted to the MU-JHU central server, and thereafter the birth defects records are extracted by the data managers and shared with the MU-JHU clinical team for review and classification [5].

After cases are reviewed by the MU-JHU clinical team, the compilation of birth defects pictures/illustrations, descriptions, diagnoses, and coding are sent to CDC birth defects SMEs by data mangers through encrypted files. Feedback from CDC birth defects SMEs along with any corrections to the diagnoses or coding are sent back to the MU-JHU clinical team as well as the surveillance midwives at the hospital level. Birth defects diagnoses are coded using the International Classification of Diseases (ICD) 10 classification system by the Royal College of Paediatrics and Child Health (RCPCH). All care of live newborns with birth defects is provided through routine care by the hospitals, which includes referral to specialists when available [3].

Statistical analysis

Birth defects surveillance data were used to calculate PPV of birth defects diagnosis at each level of review (i.e., surveillance midwives and MU-JHU clinical team), using the CDC subject matter experts’ review as the “gold standard.” PPV was calculated to assess how well the surveillance midwives and MU-JHU clinical team diagnosed and classified the identified birth defects compared to the CDC birth defects SME review. The PPVs for surveillance midwives were calculated as the proportion of a given birth defect diagnosed by the surveillance midwives that was confirmed by the CDC birth defects SME review. The PPVs for the MU-JHU clinical team were calculated as the proportion of a given birth defect classified by the MU-JHU clinical team that was confirmed by the CDC birth defects SME review. For both surveillance midwives and MU-JHU clinical team, the PPVs in the earlier periods of the surveillance activities (2015–2018) were compared to PPVs of the later periods (2019–2021) using the two-sided proportion test where the numbers were quite high. Where the occurrences of defects were rare, like craniorachischisis, a Fisher’s exact test was used to determine if classification improved over time, given the ongoing capacity building and additional staff trainings. STATA version 17 software (StataCorp. 2017. College Station, TX: StataCorp LLC) was used for analysis.

Results

Birth defects ascertainment was performed by 202 surveillance midwives at the hospital level and reviewed by four MU-JHU clinical team members (Table 1) and two birth defects SMEs at CDC. All four MU-JHU clinical team members had ≥ 4 years of surveillance experience. Among surveillance midwives, over 90% (184/202) had ≥ 2 years and 62% (125/202) had ≥ 4 years of surveillance experience (Table 1). Surveillance midwives examined 204,332 newborns during August 2015–December 2021. A total of 1,245 birth defects were identified, and 99% of these (1,232) were reviewed and confirmed by CDC birth defects SMEs. The remaining 1% were reviewed but unconfirmed due to poor photograph quality or insufficient supporting information.

Table 1 Surveillance staff from 2015 to end of 2021, Kampala, Uganda
Full size table

From 2015 to 2023, the surveillance midwives had a PPV of > 80% for 7 external birth defects: hypospadias (95%), omphalocele (90%), spina bifida (89%), anencephaly (87%), imperforate anus (86%), cleft lip with palate (82%) and gastroschisis (81%) (Table 2). Birth defects with the lowest PPVs, for surveillance midwives, were craniorachischisis (60%), encephalocele (66%) and cleft lip only (67%). The MU-JHU clinical team had a PPV of > 85% for 13 birth defects: anophthalmia/microphthalmia (100%), anotia/microtia (100%), iniencephaly (100%), hypospadias (98%), cleft lip only (97%), imperforate anus (93%), cleft lip with cleft palate (92%), talipes equinovarus (92%), spina bifida (91%), omphalocele (89%), anencephaly (89%), gastroschisis (88%), and limb reduction deficiencies (87%); while the lowest PPVs were observed for craniorachischisis (50%) and microcephaly (68.%) (Table 3).

Table 2 Positive predictive value (PPV) for diagnoses by surveillance midwives compared to birth defects subject matter experts at Centers for Disease Control and Prevention, August 2015 to December 2021, Kampala, Uganda
Full size table
Table 3 Positive predictive value (PPV) for MU-JHU clinical team diagnoses compared to birth defects subject matter experts at Centers for Disease Control and Prevention, August 2015 to December 2021, Kampala, Uganda
Full size table

Generally, higher PPVs were observed among surveillance midwives during the later period (2019–2021) compared to the earlier period of when the surveillance system began (2015–2018) (Table 4). Among surveillance midwives, PPV increased significantly for microcephaly (50–100%, p = 0.019), cleft lip with cleft palate (72.7–89.7%, p = 0.031), imperforate anus (70–100%, p = 0.025), and talipes equinovarus (70.9–81.7%, p = 0.023). A non-significant increase in PPV among surveillance midwives was observed for 10 other birth defects, including: anencephaly (85.7–89.5%, p = 0.347), encephalocele (61.1–71.4%,p = 0.271), spina bifida (85.7–95%, 0.069), anophthalmia/microphthalmia (71.4–73.3%, p = 0.454), anotia/microtia (76.2–78.6%, p = 0.435), cleft palate only (70.6–100%, p = 0.540), hypospadias (93.9–97.4%, p = 0.125), limb reduction deficiencies (68.2–80.6%, p = 0.115), gastroschisis (73.9–89.5%, p = 0.101) and omphalocele (86.7–100%, p = 0.062). For the MU-JHU clinical team, there was an increase in PPV from 2019 to 2021 compared to 2015–2018 for 5 birth defects, including: anencephaly (88.2–89.5%, p = 0.446), encephalocele (72.2–83.3%, p = 0.240), craniorachischisis (33.3–100%, p = 0.999), cleft lip with cleft palate (88.4–95.8%, p = 0.091), imperforate anus (92.3–94.1%, p = 0.422); although none were statistically significant. There were also decreases in PPV for nine birth defects, none of which were statistically significant (Table 5).

Table 4 Comparison of PPV of birth defects diagnoses by surveillance midwives between 2015–2018 and 2019–2021, Kampala, Uganda
Full size table
Table 5 Comparison of PPV of birth defects diagnoses by MU-JHU clinical team between 2015–2018 and 2019–2021, Kampala, Uganda
Full size table

Discussion

Successful birth defects surveillance programs require skilled and competent staff to identify and diagnose birth defects rapidly and accurately. The importance of birth defect capacity building has been reported in other resource limited settings [9]. For birth defects surveillance systems to provide quality data, comprehensive training, capacity building, and mentorship of health care workers and implementing staff are of the utmost importance [10]. To further strengthen surveillance capacity for accurate birth defects diagnoses, a mobile application for data collection could be used to assist with birth defect identification, description and coding. This tool would be invaluable as staff can access it in real-time as they are examining the baby.

In Uganda, strengthening and building capacity among different cadres involved with surveillance activities has been done through organized workshops with international birth defects experts, formalized in-house continuous medical trainings and hands on trainings, and self-learning. In addition, continuous quality improvement activities were implemented to address data quality gaps and ensure the accuracy and completeness of every surveillance record. Several quality control measures were implemented including real-time quality control by surveillance midwives before data submissions, review of submitted data for accuracy and completeness by data managers before inclusion in the study database, and supportive supervision by trained senior surveillance study staff and the MU-JHU clinical team. Accuracy of birth defects diagnosis was positively associated with increasing use of readily available birth defect documentation tools, particularly; the ICD-10 form for standardized coding of birth defects, and the quality control verification form for checking completeness of data variables [11].

One limitation faced by this surveillance program was the high turnover (37%) among surveillance midwives, as it is not unusual for midwives to change hospitals in resource-limited settings like Uganda [12]. However, the training and skills these midwives gained from working in birth defects surveillance will be taken to other hospitals, which can only strengthen the identification and classification of birth defects in Uganda.

Overall, results from this study showed that the MU-JHU clinical team had a higher PPV for more birth defects than the surveillance midwives. This may have been because there was no turnover in the clinical team at MU-JHU and that the team worked collaboratively, to review the information provided on identified birth defects. A second limitation was the low prevalence of certain birth defects in the surveillance system. The lowest PPV for the MU-JHU clinical team was observed for rare birth defects, such as craniorachischisis and microcephaly. Likewise, among surveillance midwives, the lowest PPV was only observed for craniorachischisis, encephalocele, and cleft lip. The low PPV among midwives, could be attributed to low prevalence of craniorachischisis, encephalocele and cleft lip during the surveillance study period. Additional training and skills-building are needed to help clinical and surveillance staff identify and diagnose rare birth defects. Surveillance teams with capacity similar to the MU-JHU clinical team would also benefit from an additional member with training in clinical genetics, especially for the diagnosis of complicated cases and syndromes.

This study also found that both the MU-JHU clinical team and the surveillance midwives have continuously increased their knowledge of birth defects diagnoses, which is evidenced by the increase in PPVs for most of the birth defects between the period 2015–2018 and 2019–2021. This may be attributed to the trainings that the MU-JHU clinical team and surveillance study staff were required to undergo before participating in surveillance activities, as well as additional training surveillance midwives underwent to conduct bedside birth defect confirmation in the absence of a clinical consultant. For surveillance midwives, strengthening capacity through confirmation by a second midwife could also have helped surveillance midwives gain confidence, experience and expertise in diagnosing birth defects by learning from their colleagues. However, there were some drops in rare defects like craniorachischisis though not statistically significant, potentially attributable to defects being rare, some defects being part of a syndrome and high surveillance midwives turn-over. Also, there may have been over-reporting of cases by surveillance midwives as they are trained to report any abnormality. Diagnoses of some defects may have been complicated by the rarity of a defect, the presence of multiple birth defects, or even a typical presentations of birth defects which would have made accurate identification more of a challenge for surveillance midwives. For some birth defects such as abdominal wall defects and orofacial clefts, the presence or absence of the defect was correctly noted; however, the type of defect may have required additional input by the clinical team at MU-JHU or by birth defects subject matter experts at CDC.

Compared to the period 2015–2018, additional measures were implemented in 2019–2021, to help hospital surveillance staff improve diagnosis and classification of birth defects. For example, during 2019–2021, a greater emphasis was placed on practical sessions and video trainings for anthropometric measurements, which improved skills among hospital surveillance staff for performing physical exams and ruling out abnormal length, weight, and head circumference, especially microcephaly. During this latter period, standard operating protocols were also modified for identification and coding of cases for inclusion in birth defects surveillance. A quality control verification form was put in place, according to identified gaps. Additionally, video conference calls were regularly held between surveillance midwives and MU-JHU clinical team, to provide feedback on incorrectly diagnosed birth defects.

One limitation faced by this surveillance program was the high turnover (37%) among surveillance midwives, as it is not unusual for midwives to change hospitals in resource-limited settings like Uganda [12]. However, the training and skills these midwives gained from working in birth defects surveillance will be taken to other hospitals, which can only strengthen the identification and classification of birth defects in Uganda.

Further improvements in PPV can be attributed to the feedback on birth defects diagnoses and coding, provided by CDC birth defects SMEs to the MU-JHU clinical team. This feedback was periodically given to surveillance study midwives, during meetings at their respective hospitals. The feedback process, for both the MU-JHU clinic team and the surveillance midwives, helped staff gain experience in diagnosing birth defects.

Conclusion

This study highlighted that birth defects surveillance, by front-line health care workers (midwives) in a low-middle income country like Uganda, is possible when midwives are exposed to comprehensive training, technical support and continuous professional development initiatives. Over the six years of the MU-JHU Research Collaboration Birth Defects Surveillance System, the diagnostic accuracy of the MU-JHU clinical team increased for five defects and decreased for nine defects, but none of the changes in PPV were statistically significant. Among the surveillance midwives, however, diagnostic accuracy increased across several birth defects, four of which were statistically significant. Empowering surveillance midwives to identify and confirm external birth defects is a strategy that is feasible, in a resource limited setting, where there is scarcity of consultants/doctors to provide confirmation.