High-risk human papillomavirus (HPV) infection is a high risk factor for the development of cervical cancer, which is the second most common malignant tumor in women worldwide [1]. High-risk HPV diagnoses combined with liquid-based cytology analysis is considered to be the most effective method for early cervical cancer screening [2]. The first edition of the “Human papillomavirus laboratory manual” was published in 2009 by the World Health Organization (WHO) to provide guidance for high-risk HPV diagnoses [3]. HPV vaccination is currently the safest approach for preventing the development of cervical cancer [4]. The distribution of HPV type and the individual risk of each HPV type are two important factors that needed to be considered for developing the HPV vaccine against particular HPV genotypes within a country [5]. Convenient and accurate techniques for high-risk HPV detection and genotyping are urgently needed for HPV clinical diagnoses and epidemiological studies.

The hybridization capture detection for HPV genetic DNA or its PCR amplicon is a traditional approach for HPV genotyping [6,7,8,9]. However, it requires many operational steps, and PCR contamination is a common problem. A variety of methods based on real-time PCR have been developed in the past decade [10,11,12,13]. Compared with traditional hybridization capture detection, the real-time PCR platform possesses the advantages of convenience, high throughput, low time and cost, and low risk of false-positive results due to cross-contamination of PCR. Commercial real-time PCR kits are now widely adopted in clinical HPV diagnostic and research studies. The Cobas® HPV Test is a typical system that has been approved by the U.S. Food and Drug Administration (FDA).

In a previous study we reported a novel HPV genotyping method based on real-time PCR and melt curve analysis [14]. The MeltPro® HPV Test was developed with the same method, and can detect and genotype the 14 most common high-risk HPV types in a single-tube reaction. In this study, we performed a comparison of the MeltPro® HPV Test and the Cobas® HPV Test for the detection and genotyping of 14 high-risk HPV types in southeast China. The processes involved in this comparison study are shown in Fig. S1. Moreover, reproducibility is important for any newly developed method. A protocol for clinical validation of HPV assays, called “VALGENT”, was developed and successfully applied for the comparison of many commercial HPV tests [15,16,17,18]. Following this guideline, intra-laboratory and inter-laboratory reproducibility experiments were performed in this study.

For comparison of the MeltPro® HPV Test and the Cobas® HPV Test, a total of 1647 residual cervix cell samples were collected from individual women living in the southeast of China. Pregnant women were excluded from this study. All samples were collected at Zhongshan Hospital, Xiamen University (Xiamen, Fujian, China) with a ThinPrep liquid-based cytology system, in 2015, and stored at -20 °C for 2 weeks before analysis. The age of the patients is from 19 to 65 years old with a mean age of 32 years-old and a median of 31 years-old. All of the experiments in this comparison study were performed at Zhongshan Hospital. For the intra-laboratory and inter-laboratory reproducibility study of the MeltPro® HPV Test, a total of 540 residual cervix cell samples were collected at Zhongshan hospital by the same protocol in 2017. The intra-laboratory reproducibility study was performed at Zhongshan Hospital, while the inter-laboratory reproducibilities study was performed at Zhongshan Hospital and the Engineering Research Center of Molecular Diagnostics of the Ministry of Education, Xiamen University. The study protocol was approved by The Research Ethics Committee of Xiamen University.

For the comparison study, 1647 samples were assayed in a double-blinded fashion using the MeltPro® HPV Test (Zeesan Biotechnology Co., China) and the Cobas® HPV Test (Roche Diagnostics Co., Switzerland) on the same day. DNA extraction was automatically performed, separately, according to the protocols of these two systems. The MeltPro® HPV Test detected 14 high-risk HPV types (HPV-16, 18, 31, 33, 35, 39, 45, 51, 52, 56, 58, 59, 66, and 68) and provided specific genotyping for all 14 HPV types based on melt curve analysis. The glyceraldehyde-3-phosphate dehydrogenase (GAPDH) gene was used as an internal control in the MeltPro ® HPV Test. The PCR reaction and data processing steps of the MeltPro® HPV Test were performed in the SLAN 96 real-time PCR system. The Cobas® HPV Test detected the same 14 high-risk HPV types but only provided specific genotyping for HPV-16 and HPV-18, based on real-time PCR. The β-globulin gene was used as internal control in the Cobas® HPV Test. The PCR reaction and data processing steps of the Cobas® HPV Test were performed in a Cobas® 4800 real-time PCR system. The experimental conditions for The MeltPro® HPV Test and Cobas HPV Test followed the guidelines of their associated protocols. During each run, for both assays, a positive and negative control was included to ensure proper PCR reactions and that there was no carry-over contamination.

The HPV genotype for each sample was assayed using independent, automated-readout software supplied with the MeltPro® HPV system or Cobas® HPV systems. The diagnostic results from the MeltPro® HPV Test and the Cobas® HPV Test were compared to evaluate agreement. For all discrepant samples, the original data were processed manually. For cases with discrepant results of HPV-16 and HPV-18, confirmatory testing was performed using an HPV type-specific real-time PCR assay diagnostic kit designed only for HPV-16 and HPV-18 (Kehua Ltd., Shanghai, China).

For the intra-laboratory reproducibility study, 540 samples were assayed by the MeltPro® HPV Test twice on the same day at Zhongshan hospital, and the results of the first and second assay were compared. For the inter-laboratory reproducibility study, the same 540 samples were assayed one week later at the Engineering Research Center of Molecular Diagnostics of the Ministry of Education, and the results were compared with the first assay’s results from the intra-laboratory reproducibility study.

Statistical analysis for the comparison data was carried out using the SPSS statistical software (version 13.0, SPSS Inc., Chicago, IL). Epidemiological prevalence, for the 14 high-risk HPV types, was only calculated from results obtained with the MeltPro® HPV Test, which was capable of full genotyping.

We compared the results obtained from 1647 samples using the MeltPro® HPV Test and the Cobas® HPV Test for the detection of 14 high-risk HPV types without genotyping (Table 1). The overall agreement between these tests was 96.2% (1584/1647), and the kappa value was 0.881 (95% CI, 0.851-0.911). Since the MeltPro® HPV Test and the Cobas® HPV Test both provide HPV-16 and HPV-18 genotyping, we also evaluated the genotyping results for HPV-16 and HPV-18, within the comparison shown in Table 1. The agreement for the genotyping results for HPV-16 was 99.6% (1641/1647), and the kappa value was 0.946 (95% CI, 0.902-0.990). The agreement for the genotyping results for HPV-18 was 99.7% (1642/1647), and the kappa value was 0.904 (95% CI, 0.818-0.990). The results for the HPV detection, HPV-16 genotyping and HPV-18 genotyping using the MeltPro® HPV Test and the Cobas® HPV Test were not significantly different (McNemar’s Test, P value = 0.23 for HPV detection, P value = 0.69 for HPV-16 genotyping, P value = 1.00 for HPV-18 genotyping).

Table 1 Comparison of HPV detection, HPV-16 genotyping, and HPV-18 genotyping using the MeltPro® HPV Test and the Cobas® HPV Test
Full size table

For the 63 discrepant samples, we processed the original data manually. The quantitative PCR cycle (Cq) values for the 22 samples diagnosed as positive by the Cobas® HPV Test and negative by the MeltPro® HPV Test were very close to the cut-off Cq value for the Cobas® automated readout software. Moreover, 16 cases among these 22 samples showed weak melting curve signals in the MeltPro® HPV Test, but their melting curve readout (Rm) values were lower than the cut-off Rm value for the MeltPro® automated readout software, while the other 6 cases showed no melting curve signals. On the other hand, the Rm values for the 41 samples diagnosed as positive by the MeltPro® HPV Test and negative by the Cobas® HPV Test were also very close to the cut-off Rm value for the MeltPro® automated readout software. Moreover, 23 cases among these 41samples showed late amplification signals in the Cobas® HPV Test, but their Cq values occurred later than the cut-off Cq value for the Cobas® automated readout software, while the other 18 cases showed no amplification signals. Six discrepant samples infected by HPV-16 and 5 discrepant samples infected by HPV-18 were diagnosed again using a third party comparison method based on type-specific real-time PCR, and all of them were confirmed to be HPV-16 positive or HPV-18 positive (Table S1).

A visual comparison of the detection results obtained from the 1647 samples using the MeltPro® HPV Test and the Cobas® HPV Test is shown in Fig. 1. A total of 339 samples were diagnosed as HPV positive by the MeltPro® HPV Test, including 83 samples that were diagnosed as infected with HPV-16 or HPV-18. Among these 339 samples, 78.5% (266/339) samples were identified as having a single HPV type infection, whereas 21.5% (73/339) samples were infected by multiple HPV types. By comparison, a total of 321 samples were diagnosed as HPV positive by the Cobas® HPV Test, including 81 samples that were diagnosed with HPV-16 or HPV-18 infection. However, because the Cobas® HPV Test cannot genotype the other 12 high-risk HPV types, aside from HPV-16 and HPV-18, we could not calculate the number of samples infected by multiple HPV types.

Fig. 1
figure 1

Detection results for 1647 samples using the MeltPro® HPV Test and the Cobas® HPV Test. The left panel shows the statistical data for the HPV detection results, prasinous pie: HPV negative samples, pink pie: HPV positive samples. The right panel shows the statistical data for the HPV genotyping results, red pie: HPV-16, yellow pie: HPV-18, green pie: co-infection samples including HPV-16 or HPV-18, blue pie: HPV-other type (for the MetltPro® HPV Test, from dark to light: HPV-52, 58, 39, 51, 68, 66, 56, 59, 33, 31, 35, and 45), grey pie: co-infection samples excluding HPV-16 and HPV-18

Full size image

The distribution of 14 high-risk HPV types among these 1647 samples is shown in Fig. 2. HPV-52 (70 cases), HPV-58 (64 cases), and HPV-16 (59 cases) were the three most prevalent HPV types in this study. HPV-31 (8 cases) and HPV-45 (5 cases) were the two rarest HPV types. The distributions of the other types were HPV-39 (31 cases), HPV-68 (29 cases), HPV-18 (27 cases), HPV-51 (26 cases), HPV-56 (23 cases), HPV-59 (20 cases), HPV-33 (18 cases), HPV-66 (17 cases), and HPV-35 (15 cases). Since 21.5% of the positive samples were infected with multiple HPV types, the total number of HPV type cases (412 cases) was larger than the total number of HPV positive samples (339 samples).

Fig. 2
figure 2

The distribution of the HPV genotyping results using the MeltPro® HPV Test. Red bars: HPV-16 and HPV-18, blue bars: other high-risk HPV types

Full size image

Two repeats using parallel diagnosis of the 540 samples in an intra-laboratory study showed 97.8% agreement, with a kappa value of 0.947 (0.917-0.977, 95% CI). The intra-laboratory data for each HPV type is shown in Table S2. The inter-laboratory study of 540 samples showed 96.9% agreement, with a kappa value of 0.925 (0.889-0.961, 95% CI). The inter-laboratory data for each HPV type is shown in Table S3.

Regular screening for high-risk HPV is recommended for adult woman by the WHO for monitoring the development of cervical cancer [3]. In this study, we evaluated the clinical performance of the MeltPro® HPV Test using 1647 samples collected from women during routine medical examinations. The HPV detection results and HPV-16 and HPV18 genotyping results using the MeltPro® HPV Test were in strong agreement with data from the Cobas® HPV Test (Table 1).

All inconsistent cases between the MeltPro® HPV Test and the Cobas® HPV Test were found to be samples infected with a low viral load of HPV. 61.9% (16+23 cases/63 cases) of the negative results for the 63 discrepant cases showed weak detection signals below the cut-off values for the MeltPro® and Cobas® automated readout software. Both the MeltPro® HPV Test and the Cobas® HPV Test were designed for the simultaneous detection of 14 high-risk HPV types in a single reaction, which means that the amplification primers used cannot completely match the gene sequence for each HPV type. Differing amplification primers might be an important reason for the discrepant results obtained for these 63 samples using the two methods. HPV type-specific real-time PCR was used as a third detection method for these discrepant samples infected by HPV-16 or HPV-18 (Table S1). A third detection method was designed to contain two pairs of primers that perfectly matched HPV-16 or HPV-18 and should only detect these two HPV types. As shown in table S1, the detection results for HPV type-specific real-time PCR showed that 11 ‘suspicious’ samples infected by HPV-16 or HPV-18 were positive.

Compared to the Cobas® HPV system, the MeltPro® HPV Test provides full genotyping information for 14 high-risk HPV types. This means the MeltPro® HPV Test can identify the specific HPV type within a sample during the detection step (Fig. 1), which will benefit users conducting HPV epidemiological studies. In this study, HPV-16, HPV-52, and HPV-58 were the three most prevalent high-risk HPV types in the southeast of China (Fig. 2). This conclusion is consistent with previous reports by other researchers [19, 20]. Consequently, HPV-52 and HPV-58 should be considered for coverage during HPV vaccine development by Chinese scientists. The full genotyping of high-risk HPV can also help doctors determine whether a patient has been persistently infected by the same HPV type or infected multiple times by different HPV types.

In summary, the MeltPro® HPV Test and Cobas® HPV Test are comparable, with 96.2% agreement and a kappa coefficient of 0.881 for HPV detection. Although the MeltPro® HPV Test can identify more HPV types than the Cobas® HPV Test, their clinical performances were similar regarding HPV detection. Different cut-off values and the different amplification primers might be the cause of discrepant results between the two methods. The reproducibility of the MeltPro® HPV Test proved to be stable in both intra-laboratory and inter-laboratory studies. The MeltPro® HPV Test can provide full typing information and is an accurate, high-throughput, and low-cost method that can be used in the future for clinical HPV screening and genotyping.