Background

Cytomegalovirus (CMV) causes developmental defects at birth among 10% of infected babies, and 8.5–18% of asymptomatic newborns will develop sensorineural hearing loss (SNHL) [1,2]. The two main potential benefits of neonatal screening are early intervention to prevent the onset or progression of SNHL and the identification of infants at risk for late-onset or progressive SNHL [3]. Randomized control trials (RCT) and observational studies have reported that ganciclovir therapy begun in the neonatal period in symptomatically infected infants prevented hearing deterioration at 6 months [4-6].

The gold standard for the diagnosis of congenital CMV (cCMV) infection is positive results for viral isolation from urine and/or saliva collected during the first 3 weeks of life [7]. This method is not suitable for large scale screening. As an alternative, dried blood spots (DBS) polymerase chain reaction (PCR) assays are getting more and more attention, because specimens can be collected routinely and preserved easily; and because PCR can be automated. Certain researchers have suggested that this method is suitable for the retrospective diagnosis of cCMV infection in infants or children with hearing loss, mental retardation or other symptoms compatible with cCMV [8]. However, others have insisted the tests is not suitable for screening due to low sensitivity. In conclusion, there is no consensus on the screening performance of DBS PCR assay in screening for cCMV infection.

In the present meta-analysis, we systematically reviewed studies of the diagnostic performance of DBS PCR assays for cCMV infection to determine whether these assays are sufficiently effective to be used for screening neonates.

Methods

Search strategy

Shibata et al. [9] first reported a method for using DBS PCR to detect CMV DNA in 1994, so the publication time was limited from 1990 to 2014. We performed an electronic search of Medline (1990 to January 31, 2014), the Cochrane Library database (1990 to January 31, 2014), and the Science Citation Index (1990 to January 31, 2014) using the following search terms: “DBS” (or “dried blood spots”, “filter papers” or “Guthrie card”), and “congenital cytomegalovirus”.

Study eligibility

The inclusion criteria were as follows:

  1. 1.

    Studies that compared DBS PCR assays with the standard method for detecting cCMV infection. The DBS samples had to have been collected within the first week of life. The protocol for DBS PCR assays included DNA extraction from DBS samples and PCR amplification of CMV DNA. Reference standards were generally viral isolation from or PCR detection in urine and/or saliva collected within the first 3 weeks of life.

  2. 2.

    Studies with available data for constructing contingency tables for true positive (TP), false positive (FP), false negative (FN) and true negative (TN) determination.

The exclusion criteria were as follows:

  1. 1.

    Studies that did not compare DBS PCR testing with standard tests (viral isolation from urine and/or saliva) for diagnosing cCMV infection.

  2. 2.

    Studies that overlapped with the studies selected (i.e., studies from the same study group, institution, and period of inclusion).

  3. 3.

    Letters, editorials, expert opinions, and reviews without original data, and case reports.

Data extraction

A comprehensive search of the literature was performed by two of our authors, who each assessed which studies to retain based on the inclusion and exclusion criteria. Each investigator was blinded to the other’s selections. Discrepancies between the two were arbitrated by our senior investigator (Jihong Qian). All data were recorded, regarding the study setting, screening test properties and screening test results, such as TPs, FPs, FNs, and TNs. When several groups of CMV DNA tests were examined in one study, the data from the highest sensitivity group were chosen for analysis.

Limit of detection (LOD) calculation

8 of 14 including studies reported the protocols of DNA extraction and PCR, and the detection limit of PCR reaction (copies/1 ml whole blood or copies/1 reaction). We extracted data from 5 of the 8 studies directly [10-14], and referenced the handbook of PCR kit used in 3 of the 8 studies [2,15,16] without exact protocol. We calculated the LOD (copies/1 ml whole blood) by the method shown in Table 1, based on assuming a yield of 100% extraction.

Table 1 The calculation methods of LOD
Full size table

Qualitative assessment

Evidence quality was assessed by two independent investigators using the Quality Assessment of Diagnostic Accuracy Studies 2 (QUADAS-2) tool [17]. Briefly, the QUADAS-2 includes 4 key domains that are rated in terms of the risk of bias: patient selection, execution of the index test, reference standard, and flow of patients (in particular, whether there was an appropriate interval between the index test and the reference standard). Each domain is scored as having a high, low, or unclear risk of bias. Discrepancies regarding the risk of bias and other discrepancies were arbitrated by a third reviewer.

Test for heterogeneity

The values of I 2 and Cochrane -Q were assessed for heterogeneity among studies. I 2 values of 25%, 50% and 75% represent mild, moderate, and severe inconsistency, respectively.

Sensitivity analysis

In the sensitivity analysis, one study at a time was excluded from each analysis. Most of the results appeared to be robust to the influence of individual studies (Table 2). The diagnostic odds ratio (DOR) results also did not vary significantly when the reference changed.

Table 2 Sensitivity analysis
Full size table

Statistical methods

The statistical analysis was performed using Meta-Disc (version 1.4) and STATA 12.0 software. The pooled sensitivity, pooled specificity, positive likelihood ratio (LR+), negative likelihood ratio (LR-), positive predictive values (PPVs), negative predictive values (NPVs) and their 95% CIs were calculated using the random effect model and the fixed effect model [18,19]. The joint distribution of the TP rate (TPR) and FP rate (FPR) was analyzed using a summary receiver operating characteristic (SROC) curve. The performance of the DBS PCR assays was analyzed based on the DOR, the Q-statistic, and the area under the ROC curve (AUC) [20]. Heterogeneity among studies was analyzed using the I 2 statistic.

Results

Eligible studies

The searches for screening generated 102 articles after removal of duplicates. Through assessment for eligibility, 14 articles were included in the meta-analysis (details in Figure 1). In 2010, Boppana et al. [2] enrolled two groups of subjects, who were screened for cCMV using two different CMV DNA tests (single-primer real-time PCR and two-primer real-time PCR). This article was deemed to be two independent studies, and thus 15 studies were included in the present review. The main characteristics and details of the studies are reported in Table 3. Overall, 26007 neonates were enrolled, composed of 583 neonates identified as having cCMV infection and 25424 neonates without CMV infection, as determined using a reference standard.

Figure 1
figure 1

Flow chart showing the process of study selection. 14 articles were included in this meta-analysis.

Full size image
Table 3 Characteristics of the included studies
Full size table

Quality assessment

The quality of the eligible studies was assessed according to the QUADAS-2 criteria (see in Table 4). Four of the 15 studies were considered to having a low risk of bias in any of the domains [2,15,21]. Nine studies were assessed as having a high risk of patient selection bias, because the neonates with classical symptoms of cCMV infection, confirmed using reference standard might have been more easily diagnosed with cCMV [10-12,14,22-26]. Seven studies (5 with high risks and 2 with unclear risks) were identified as high risks in the index test domains [12-14,16,22,23,25]. In these studies, the researchers knew the results of the reference standard, which could have influenced the interpretation of the index test results. One study [11] had an applicability concern in the index test domains, because the study employed two PCR methods, conventional nested PCR designed to amplify one region in the gp58 gene and a commercial kit for the amplification of the IE gene. In 3 studies [11,12,23], there was several unclear risks because the study had not been published as open access or because the article did not describe related information, which were considered to be potential risks.

Table 4 Summary of the assessment of the included studies using QUADAS-2
Full size table

Test for heterogeneity

The threshold effect was denied by diagnostic threshold analysis (Spearman correlation = 0.415, P = 0.124). The I 2 statistics was 45.3% and the Cochrane-Q statistics was 25.57 (P = 0.0293) for the pooled DOR, suggesting that there was moderate heterogeneity among the included studies.

Screening performance

A total of 15 studies including 26007 DBS samples from neonates were assessed. The pooled sensitivity and specificity of the DBS tests were 0.844 (95% CI = 0.812–0.872) and 0.999 (95% CI = 0.998–0.999), respectively (Figure 2a, b). The pooled LR+ was 99.437 (95% CI = 45.666–216.523), and the pooled LR- was 0.110 (95% CI = 0.0424–0.289). The pooled PPV and NPV were 0.906 (95% CI = 0.835–0.948) and 0.991 (95%CI = 0.972–0.997), respectively. The pooled DOR was 1362.10 (95% CI = 566.91–3272.60) using the random effect model (Figure 2c). The AUC was 0.9953 (SE = 0.0023) with Q* = 0.9734 (SE = 0.0077) (Figure 2d), showing that the DBS tests for cCMV performed well. The pooled data were analyzed using fixed effect model, and the results were consistent with those of the analysis using the random effect model (data not shown).

Figure 2
figure 2

Summary of diagnostic performance estimates using random effect model. a. Pooled sensitivity of DBS PCR assays; The pooled sensitivity is 0.84. b. Pooled specificity of DBS PCR assays; The pooled specificity is 1.00. c. DOR for DBS PCR assays; The DOR is 1362.09. d SROC curve for cCMV diagnosis by DBS PCR assays. The AUC is 0.9953.

Full size image

Subgroup analysis and meta regression

A subgroup analysis of the study design (Table 5a) was performed because the 7 prospective studies [2,10,13,15,21,26] had lower sensitivity than the retrospective studies [11,12,14,16,22-25]. The results showed that the 7 prospective studies, with no significant heterogeneity had obviously lower sensitivity (0.623) than the retrospective studies (0.945). However, the diagnostic performance in the two subgroups of studies was similar, as evidenced by the DOR and the AUC. In addition, there was moderate heterogeneity in subgroup of retrospective, indicated there was other factors to affect the heterogeneity.

Table 5 Performance of subgroups of DBS PCR assays for screening cCMV
Full size table

The limit of detection (LOD) of DBS assays is critical to sensitivity. We extracted and obtained LOD data from 9 studies [2,10-16,22], whereas the remaining 6 studies had no LOD data. A subgroup analysis of the LOD was performed among a high-LOD group (LOD ≥ 1500 copies/ml) [12,14-16], a low-LOD group (LOD < 1500 copies/ml) [2,10,11,13] and others (the group of remaining studies without LOD data) [23-27]. The sensitivity in the high-LOD group was higher than that in the other two groups, so it is possible that low LOD led to a higher false positive rate (Table 5b).

The DBS samples were all not uniform, with different sample sizes, storage temperatures and areas of the samples tested. We chose to analyze the most important factor that may influence the sensitivity: the total surface area of DBS. Three groups, large area group (>25 mm2) [13,14,16,21,26], small area group (≤25 mm2) [2,10,11,22,23], and others (the group of remaining studies with unknown the area data) [12,15,24,25], were compared by subgroup analysis. The results indicated that the large surface area group had higher sensitivity (Table 5c).

We performed meta-regression (Table 6) to investigate whether the above factors influenced diagnostic performance. The LOD, PCR methods and area of DBS sample were the factors analyzed in the meta-regression. The results showed that only the PCR method was related to the performance. The number of included studies were not enough to perform meta-regression of 4 factors, therefore we exclude the “study design” factor. If the factor “study design” was added into the meta-regression, the results were not changed.

Table 6 Meta-regression to determine potential sources of heterogeneity
Full size table

Clinical aspects

In total, 5/14 studies [10,13,15,21,23] described the clinical features (symptomatic or asymptomatic) of the enrolled neonates. In asymptomatic infants confirmed as having cCMV infection, the positive rate of DBS PCR assays was 95.90% (117/122), whereas the rate in symptomatic infants was 96.67% (58/60). These values were not significantly different. (P = 0.579, Fisher’s exact test).

Discussion

The biggest benefit from screening for cCMV disease is the identification of asymptomatic infants who may develop late onset disease, with symptoms including auditory deterioration. In addition, diagnosing infants with symptomatic infections as early as possible will provide the opportunity to improve the outcomes.

A cost-effective, specific, and sensitive means of newborn screening would make it possible to screen on a large scale. In particular, specimens should be routinely collected from newborns, with CMV DNA kept stable for testing. The test must be amenable to automation at a relatively low cost. More importantly, the test needs to have high sensitivity, especially for neonatal screening. DBS samples are routinely collected for metabolic screening generally within 3 days after birth and PCR assays could be automatically run to test large numbers of specimens.

As shown in Table 3, the length of storage of DBS samples varies from 14 days to 18 years. The length of DBS storage is dependent on the newborn screening policies in different regions. Many states have written policies addressing the storage of DBSs [28]. Therrell et al. [29] surveyed 37 newborn screening programs in the United States with written policies regarding the length of storage, among which the storage time varied from 6 months to 23 years. Johansson et al. [22] has tested 12- to18- year old DBS samples, which was the longest duration of preserving the DBS, and the diagnostic sensitivity and specificity of the test in this study were 81.25% and 100%, respectively. The precision was not affected by the sample storage time. These findings suggest that CMV DNA in DBS samples is stable enough for testing.

Whether cards carrying DBSs with CMV infection will contaminate adjacent cards is another critical problem. In study by Johansson et al. [22], CMV DNA was detected in 6/32 DBS samples stored above or below the CMV positive ones, indicating a transfer of CMV DNA. The possible reason for contamination of adjacent cards may be longtime preservation. However, several studies, such as that by Walter et al., reported no cross-contamination during the period of study [30-33].

Although the pooled sensitivity (84.4%) and pooled specificity (99.9%), and especially the AUC (0.9953) presented better performance for the DBS PCR assays, the moderate heterogeneity (I 2 = 45.3%) of the included studies should not be ignored. The subgroup analysis indicated that the study design, LOD, and area of DBS sample all influenced the performance of the DBS PCR assays. The LOD in consideration of the purification is determined by CMV positive specimens in combination with a particular extraction method, followed by a probit analysis. A 95% detection limit denotes that there is a 95% probability that the minimum amount of CMV DNA in whole blood can be detected. In some of the studies, the LOD data was not described. We extracted the information of DNA extraction and PCR methods to calculate the LOD or translate to the number of copies in whole blood (details in Table 1). A higher LOD indicates a lower sensitivity of the PCR assays. The low-LOD subgroup had the higher DOR among all of the groups. In addition, the test performance in large area of DBS group was better than in the other two groups, suggesting that the extraction quality may affect the DBS PCR assays results.

The study design (prospective or retrospective) affected the prevalence of cCMV in the sample groups. In the prospective studies, the prevalence which influences the DOR was always lower than that in the retrospective ones, because more patients were selected in retrospective studies. The results showed that the pooled sensitivity of the prospective studies was 62.3%, which was lower than the 94.5% in retrospective studies. Although the DOR in the prospective studies was higher, “study design” was still tested as a confounder in the meta-regression (data not shown). This results may be explained by patient selection bias due to lack of blinding of the index test operators to the reference test results. For example, 7/8 retrospective studies had a higher risks of patient selection bias. In particular, these studies enrolled infants with CMV infection confirmed by gold standard or with suspected infection and negative controls confirmed not to have infection. The high sensitivity in retrospective studies indicates the test is more suitable for diagnosing cases with risk of long term sequelae and/or following up the children suffered with CMV infection.

We performed a meta-regression to analyze 3 factors to determine the source of heterogeneity. The results showed that the factor of “PCR method” led to inconsistency. The performance of PCR was influenced by several factors, such as DNA extraction, therefore the results indicated that PCR was one of source of heterogeneity. The other two factors were regarded as confounders in the meta-regression analysis.

If the viral load in whole blood cannot reach the detection threshold of the DBS PCR assay, the sensitivity will be affected. The viral load in whole blood from neonates with symptomatic cCMV infection was higher than that in asymptomatic neonates [26,34]. The sensitivity was lower in studies with consecutive cases, which probably enrolled more asymptomatic infants, such as those of Yamamoto et al. [21] and Boppana et al. [2] studies. In the present study, the rates of detection of CMV DNA in DBSs from symptomatic neonates and asymptomatic neonates were similar (95.90% and 96.67%, respectively), based on the data assembled from 5 studies that included information regarding the clinical status of the neonates. However, 2 of the 5 studies [15,23] selected infants with confirmed cCMV infection and performed the test with knowledge of the reference standard’s results, which could inflate the final performance of the test. Therefore, the relationship between the detection rate and clinical features should be explored using a large number of samples, to evaluate the exact value of DBS PCR assays in screening for cCMV infection.

The main limitation of this review is the quality of the included studies. The studies, such as that by Leruez-Ville et al. [12] selected neonates with compatible symptoms, which could have led to an overestimate of the sensitivity and specificity of the test. In addition, the sample sizes in 4 studies were less than 100, which probably resulted in the wide variability in the performance of the tests. Second, the DBSs in the studies were not uniform, including the sample size, different areas of sample, and storage conditions et al. We chose the most important factor, the area of DBS to analyze. The sample sizes may have affected the performance of the screening test, which is of concern given the low prevalence of cCMV 0.42% or 1%, in Canada [35] and Brazil [27], respectively. More studies on DBS assays performed among various prevalence levels of cCMV infection will be needed to evaluate their screening performance.

Conclusion

Giving sensitivity priority over specificity is particularly useful for epidemiologic surveys of both the prevalence of congenital infection and neonatal screening [23]. The sensitivity of DBS PCR assays may meet the need for retrospective diagnosis, but the diagnostic performance is not robust or sufficient for large-scale universal screening. A DBS sample could be collected and then sent to a laboratory for virology testing in areas that may not have the required equipment.