Abstract
Background
Trachoma, the leading infectious cause of blindness worldwide, is caused by conjunctival Chlamydia trachomatis infection. Trachoma is diagnosed clinically by observation of conjunctival inflammation and/or scarring; however, there is evidence that monitoring C. trachomatis infection may be required for elimination programmes. There are many commercial and ‘in-house’ nucleic acid amplification tests for the detection of C. trachomatis DNA, but the majority have not been validated for use with ocular swabs. This study evaluated a commercial assay, the Fast-Track Vaginal swab kit, using conjunctival samples from trachoma-endemic areas. An objective, biostatistical-based method for binary classification of continuous PCR data was developed, to limit potential user-bias in diagnostic settings.
Methods
The Fast-Track Vaginal swab assay was run on 210 ocular swab samples from Guinea-Bissau and Tanzania. Fit of individual amplification curves to exponential or sigmoid models, derivative and second derivative of the curves and final fluorescence value were examined for utility in thresholding for determining positivity. The results from the Fast-Track Vaginal swab assay were evaluated against a commercial test (Amplicor CT/NG) and a non-commercial test (in-house droplet digital PCR), both of whose performance has previously been evaluated.
Results
Significant evidence of exponential amplification (R2 > 0.99) and final fluorescence > 0.15 were combined for thresholding. This objective approach identified a population of positive samples, however there were a subset of samples that amplified towards the end of the cycling protocol (at or later than 35 cycles), which were less clearly defined. The Fast-Track Vaginal swab assay showed good sensitivity against the commercial (95.71) and non-commercial (97.18) tests. Specificity was lower against both (90.00 and 96.55, respectively).
Conclusions
This study defined a simple, automated protocol for binary classification of continuous, real-time qPCR data, for use in an end-point diagnostic test. This method identified a population of positive samples, however, as with manual thresholding, a subset of samples that amplified towards the end of the cycling program were less easily classified. When used with ocular swabs, the Fast-Track Vaginal swab assay had good sensitivity for C. trachomatis detection, but lower specificity than the commercial and non-commercial assays it was evaluated against, possibly leading to false positives.
Similar content being viewed by others
Background
Chlamydia trachomatis is the most common bacterial sexually transmitted infection [1] and the leading infectious cause of blindness worldwide [2]. Trachoma, conjunctival C. trachomatis infection and subsequent disease, is responsible for visual impairment or blindness in an estimated 2.2 million people [2]. Trachoma is targeted for elimination by 2020 [3], using a series of interventions known as SAFE [4]: Surgery for trichiasis (inturned eyelashes), Antibiotics for infection (community mass drug administration, MDA), Facial cleanliness and Environmental improvement to reduce transmission. Control programmes use clinical diagnosis of trachoma for monitoring prevalence, chiefly signs of trachomatous inflammation-follicular (TF) and trachomatous inflammation-intense (TI). However, the correlation between TF/TI and conjunctival C. trachomatis infection lessens as prevalence declines [5, 6], suggesting tests for infection may be necessary to accurately monitor trachoma dynamics, particularly in low-endemicity and post-MDA settings [7].
Chlamydia trachomatis infection has historically been diagnosed through culture of the bacterium, antigen detection and direct cytological examination [8]. Currently, nucleic acid amplification tests (NAATs) are the gold standard for C. trachomatis detection as they are more sensitive and allow increased throughput. Many commercial and non-commercial assays are available; however, the majority have not been validated for use with ocular swabs. Incorporating tests for C. trachomatis infection into trachoma control programmes, rather than relying on clinical diagnosis alone, has the potential to reduce cost and increase success [9].
Positivity in NAATs based on real time qPCR assays, such as Artus C. trachomatis Plus RG PCR Kit (Qiagen, Hilden, Germany), is primarily based on samples ‘displaying an exponential trace’ [10] during thermal cycling, and there are a number of methods for binary classification of samples into positives and negatives. It is common for thresholds to be defined based on manual inspection of traces, as well as by comparison to positive and negative controls. Samples that demonstrate exponential amplification before this manually defined threshold are deemed positive. This inherently subjective method introduces user-bias.
This study evaluated a commercial assay, the Fast-Track Vaginal swab kit, for diagnosing C. trachomatis infection from ocular swabs. Additionally, the study aimed to define an objective method for binary classification of samples using the raw PCR amplification curves, rather than subjective thresholding by individual users.
Methods
Sample collection
Samples were collected from the upper tarsal conjunctiva using a Dacron polyester-tipped swab (Hardwood Products Company, Guilford, Maine). The swab was passed firmly four times across the conjunctiva with a quarter turn between each pass. All samples were kept on ice packs in the field until transfer to -80 °C the same day for storage until processing. Samples included were collected in Kilimanjaro Region, northern Tanzania [11] (109) and Bijagos Islands, Guinea-Bissau (101) as part of trachoma surveys.
DNA extraction and amplification
DNA was extracted from all swabs using the QIAmp DNA mini kit (Qiagen, Crawley, UK). For sample processing using the Fast-Track Diagnostic (FTD) Vaginal swab kit (Fast-Track Diagnostics, Esch-sur-Alzette, Luxembourg), 10 μl extracted DNA was amplified in a total reaction volume of 25 μl. Positive and negative controls were included in each run. Cycling conditions were as described in the manufacturer’s instructions. Raw fluorescence data was modeled to determine positivity as described below. Chlamydia trachomatis was detected using the Amplicor CT/NG kit (Roche Molecular Systems, Branchburg, NJ) with previously described modifications [12]. Samples whose absorbance values at 450 nm (A450) were ≥ 0.8 were considered positive while samples less than 0.2 A450 were considered negative. Samples for which the result was equivocal (≥ 0.2, < 0.8) were tested again in duplicate. The sample was only considered positive if the A450 of one of the retests was ≥0.8. DNA amplification by droplet digital PCR (ddPCR) was conducted as previously described [13]. A sample was considered positive if the 95% confidence interval of C. trachomatis plasmid copies per μl did not intersect zero.
Data analysis
The raw fluorescence data for the FTD Vaginal swab PCR were used for all analyses. Data for Amplicor and ddPCR were analysed as previously described [13, 14]. All analyses were performed using R version 3.3.2. Data were fitted to exponential and sigmoid models using the qpcR package. The mixtools package was used for mixtures modelling. FTD Vaginal swab PCR was evaluated against the Amplicor CT/NG and ddPCR assays, using the caret and psych packages to calculate sensitivity, specificity, positive predictive value, negative predictive value and Cohens kappa.
Results
Defining threshold for positivity
Two-hundred and ten upper tarsal conjunctival swabs were tested for the presence of C. trachomatis infection using the FTD Vaginal swab kit. Data from all 210 samples were used to determine an automated threshold by which samples could be considered as either positive or negative.
Raw amplification curves were fitted to exponential and sigmoid models (four-parameter logistic regression) and evaluated using their respective R2-values, to test for evidence of exponential amplification. Samples evidenced by a clear exponential trace and final fluorescence intensity, including all positive controls, fitted both models well. However, samples that had a final fluorescence less than or equivalent to the negative controls had variable R2-values, with many showing strong fits for one or both models. This discrepancy was demonstrated by poor correlation of fluorescence intensity at cycle 40 and the model R2-values (exponential model; correlation = 0.31, P-value = 0.0001 and sigmoid model; correlation = 0.35, P-value < 0.0001). Good model fit in samples that did not amplify was likely caused by instability in low-level/background fluorescence, leading to fluctuations that can be mistaken for amplification (Fig. 1). Using model fit alone would classify these non-amplifying samples as positive, dramatically reducing specificity. Similar results were found with the derivative and second derivative of the sigmoid models.
Sigmoid models representative of variation in amplification. Raw fluorescence results from samples processed using the FTD Vaginal swab kit. Sigmoid models representing: samples that did not amplify and do not fit a sigmoid model (a), samples that did not amplify but do fit a sigmoid model (b), and samples that did amplify and do fit a sigmoid model (c)
To overcome the limitations of only considering ‘clear’ exponential amplification, a minimum final fluorescence component was incorporated into the method for defining positive samples. The final fluorescence values for each sample were modelled onto mixtures of two or three normal distributions. Allowing three distributions provided the optimum fit (log-likelihood = 474) compared with two distributions (log-likelihood = 431). Both models identified samples with a final fluorescence above 0.15 units as a separate population. A mixture of three distributions also highlighted a population of samples with a final fluorescence between 0.05 and 0.15 units (Fig. 2).
Diagnostic evaluation of FTD Vaginal swab PCR
Finally, we conducted a diagnostic evaluation of the FTD Vaginal swab kit against previously validated commercial (Amplicor CT/NG [14]) and in-house (ddPCR [13]) assays (Table 1), using a subset of 100 samples. For diagnostic evaluation, two methods for identifying exponential amplification were combined, sigmoid model R2-value > 0.99 and final fluorescence > 0.15. Thirty of 100 samples were positive by Amplicor CT/NG; the FTD Vaginal swab kit and ddPCR correctly identified 27 of these, respectively calling three and two additional positive results. Twenty-nine of 100 samples were positive by ddPCR; the FTD Vaginal swab kit correctly identified 28 of these, additionally calling two further positive results. In-house ddPCR had greater sensitivity (97.14) compared with Amplicor CT/NG than previously reported [13].
The three Amplicor CT/NG-positive samples called as negative by the FTD Vaginal swab kit and by ddPCR were originally identified as equivocal samples before retesting, per the Amplicor protocol. Two of three had no positive droplets by ddPCR and a final fluorescence by the FTD Vaginal swab kit of less than 0.01. The remaining sample did not have enough positive droplets to be reliably called as positive by ddPCR, but had a final fluorescence above 0.1 by the FTD Vaginal swab kit.
Discussion
This study evaluated results for detection of C. trachomatis DNA from ocular swabs using the FTD Vaginal swab kit, a commercial PCR assay validated for use with urogenital samples, against previously validated commercial and in-house assays. Automated methods for determining thresholds for positivity from raw amplification curves were also explored. A composite of amplification curve fit and absolute level of amplification was determined to be the best method for the identification of positive results, although, as with setting a manual threshold, there was some ambiguity assigning samples that amplified at or later than 35 cycles. The FTD Vaginal swab kit performed well against both comparator assays; however specificity was notably lower versus Amplicor CT/NG.
Manual and inherently subjective binary classification of continuous real time qPCR data is a significant problem, as it creates unnecessary variability within and across assays. Visual inspection of curves focuses on identifying clear evidence of exponential/sigmoid amplification. This study found that observation of exponential amplification could not singularly define positivity, due to instability in low-level/background fluorescence being mistaken for true amplification. Including mixtures-model clusters defined using the final fluorescence value greatly improved identification of positive samples. For diagnostic evaluation, a sample was deemed positive if: (i) the amplification curve was fit strongly to a sigmoid model (R2-value > 0.99); and (ii) the sample clustered within the right-most population of a mixtures-model of three normal distributions (final fluorescence > 0.15). This objective method identified a population of positive samples; however there was some ambiguity in samples that appear to amplify late in the reaction (at or later than 35 cycles), a problem common to manual threshold setting and inspection. A subset of samples, which formed the middle population of the three distribution mixtures-model, described above, showed varying levels of amplification using the FTD Vaginal swab assay, with final fluorescence values between 0.05 and 0.15. Of interest, the sample in this subset with the highest final fluorescence was positive by Amplicor CT/NG, after an initial equivocal result. As with Amplicor CT/NG, samples that fit in this middle, less clear population should, ideally, be retested.
The FTD Vaginal swab assay performed well against both Amplicor CT/NG and ddPCR. Sensitivity was above 95% for both, with three and one false negatives, respectively. One of the false negatives, was a sample with a final fluorescence over 0.1, which was initially equivocal by Amplicor CT/NG. It is possible with retesting, as suggested above, this sample might have been positive. Specificity was slightly lower against both tests, notably down to 90% for Amplicor CT/NG, with three and two false positives, respectively. Compared with either assay, the specificity of FTD Vaginal swab assay is below that of various alternative NAATs for C. trachomatis [15].
Conclusion
Automated, unbiased classification of continuous real time qPCR data into binary results, for diagnostic purposes, is achievable using a simple set of biostatistical rules. The method described allows objective classification of results from qPCR, using the raw output from thermal cycling programs. FTD Vaginal swab PCR for use with ocular swabs has lower specificity than both Amplicor CT/NG and ddPCR for C. trachomatis detection, challenging its diagnostic utility with this sample type.
Abbreviations
- CT:
-
Chlamydia trachomatis
- ddPCR:
-
Droplet digital PCR
- DNA:
-
Deoxyribonucleic acid
- FTD:
-
Fast-Track Diagnostics
- MDA:
-
Mass drug administration
- NAAT:
-
Nucleic acid amplification test
- NG:
-
Neisseria gonorrheae
- PCR:
-
Polymerase chain reaction
- qPCR:
-
Quantitative PCR
References
Newman L, Rowley J, Vander Hoorn S, Wijesooriya NS, Unemo M, Low N, et al. Global estimates of the prevalence and incidence of four curable sexually transmitted infections in 2012 based on systematic review and global reporting. PLoS One. 2015;10(12):e0143304.
Bourne RR, Stevens GA, White RA, Smith JL, Flaxman SR, Price H, et al. Causes of vision loss worldwide, 1990-2010: a systematic analysis. Lancet Glob Health. 2013;1(6):e339–49.
WHO. Report of the first meeting of the WHO alliance for the global elimination of trachoma; 1997.
Emerson PM, Burton M, Solomon AW, Bailey R, Mabey D. The SAFE strategy for trachoma control: using operational research for policy, planning and implementation. Bull World Health Organ. 2006;84(8):613–9.
Baral K, Osaki S, Shreshta B, Panta CR, Boulter A, Pang F, et al. Reliability of clinical diagnosis in identifying infectious trachoma in a low-prevalence area of Nepal. Bull World Health Organ. 1999;77(6):461–6.
Ramadhani AM, Derrick T, Macleod D, Holland MJ, Burton MJ. The relationship between active trachoma and ocular Chlamydia trachomatis infection before and after mass antibiotic treatment. PLoS Negl Trop Dis. 2016;10(10):e0005080.
Butcher RM, Sokana O, Jack K, Macleod CK, Marks ME, Kalae E, et al. Low prevalence of conjunctival infection with Chlamydia trachomatis in a treatment-naive trachoma-endemic region of the Solomon Islands. PLoS Negl Trop Dis. 2016;10(9):e0004863.
Chernesky MA. The laboratory diagnosis of Chlamydia trachomatis infections. Can J Infect Dis Med Microbiol. 2005;16(1):39–44.
Harding-Esch E, Jofre-Bonet M, Dhanjal JK, Burr S, Edwards T, Holland M, et al. Costs of testing for ocular Chlamydia trachomatis infection compared to mass drug administration for trachoma in the Gambia: application of results from the PRET study. PLoS Negl Trop Dis. 2015;9(4):e0003670.
Manual: FTD Vaginal swab, version 2. Fast-Track Diagnostics. Esch-sur-Alzette; Luxembourg. 2016.
Burton MJ. Hu VH, Massae P, Burr SE, Chevallier C, Afwamba IA, et al. what is causing active trachoma? The role of nonchlamydial bacterial pathogens in a low prevalence setting. Invest Ophthalmol Vis Sci. 2011;52(8):6012–7.
Holland MJ, Jeffries D, Pattison M, Korr G, Gall A, Joof H, et al. Pathway-focused arrays reveal increased matrix metalloproteinase-7 (matrilysin) transcription in trachomatous trichiasis. Invest Ophthalmol Vis Sci. 2010;51(8):3893–902.
Roberts CH, Last AR, Molina-Gonzalez S, Cassama E, Butcher R, Nabicassa M, et al. Development and evaluation of a next-generation digital PCR diagnostic assay for ocular Chlamydia trachomatis infections. J Clin Microbiol. 2013;51(7):2195–203.
Cheng A, Qian Q, Kirby JE. Evaluation of the Abbott RealTime CT/NG assay in comparison to the Roche Cobas Amplicor CT/NG assay. J Clin Microbiol. 2011;49(4):1294–300.
Centers for Disease C, Prevention. Recommendations for the laboratory-based detection of Chlamydia trachomatis and Neisseria gonorrhoeae - 2014. MMWR Recomm Rep. 2014;63(RR-02):1–19.
Acknowledgements
The authors would like to acknowledge individuals in Guinea-Bissau and Tanzania who provided samples for this study and the local field teams involved in sample collection.
Funding
This study was funded by the Wellcome trust (grant number WT093368MA).
Availability of data and materials
The datasets generated and analysed during the current study are available from the corresponding author upon reasonable request.
Author information
Authors and Affiliations
Contributions
Study design: MJH and SEB. Sample collection: ARL and MJB. Sample processing: SEB. Data analysis: HP, MJH and SEB. Manuscript preparation: HP, MJH and SEB. Manuscript approval: HP, ARL, MJB, MJH and SEB. All authors read and approved the final manuscript.
Corresponding author
Ethics declarations
Ethics approval and consent to participate
Samples were collected from trachoma-endemic communities in Tanzania and Guinea-Bissau following ethical approval from the Kilimanjaro Christian Medical Centre (Tanzania), the National Institute for Medical Research in Tanzania, Comitê Nacional de Ética e Saúde in Guinea Bissau and London School of Hygiene and Tropical Medicine (UK). Written, informed consent was obtained from all participants prior to sample collection. In the case of children, consent was obtained from the parent or guardian.
Consent for publication
Not applicable.
Competing interests
The Fast-Track Vaginal swab kits were provided free of charge by Fast-Track Diagnostics. Fast-Track Diagnostics had no input on study design or manuscript preparation.
Publisher’s Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
Open Access This article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated.
About this article
Cite this article
Pickering, H., Holland, M.J., Last, A.R. et al. Evaluation of a Chlamydia trachomatis-specific, commercial, real-time PCR for use with ocular swabs. Parasites Vectors 11, 102 (2018). https://doi.org/10.1186/s13071-018-2686-y
Received:
Accepted:
Published:
DOI: https://doi.org/10.1186/s13071-018-2686-y