Skip to main content
SpringerLink
Log in

Use of Receiver Operating Characteristic Curves to Assess the Performance of a Microdilution Assay for Determination of Drug Susceptibility of Clinical Isolates of Mycobacterium tuberculosis

  • Article
  • Published:
European Journal of Clinical Microbiology and Infectious Diseases Aims and scope Submit manuscript

Abstract

The aim of this study was to apply receiver operating characteristic (ROC) analysis to the microplate Alamar blue assay, a recently developed alternative for drug susceptibility testing of mycobacteria. As this is a quantitative assay, its performance can be determined by ROC analysis, in which the area under the ROC curve represents a summary of test performance (the higher the area, the better the test's performance). Sixty isolates of Mycobacterium tuberculosis were tested by the microcolorimetric assay against six twofold dilutions of streptomycin, isoniazid, rifampin, and ethambutol. For each isolate, the susceptibility pattern was simultaneously established by the agar proportion method, the result of which represented the gold standard value for the ROC analysis. The critical concentration, area under the curve, and P value for each drug were determined by ROC curve analysis. The results of the assay were obtained in an average of 8 days of incubation. The performance of the assay was excellent for all four drugs: the area under the curves was >0.97, the P values were 0.000, and sensitivity was 94%, specificity 97%, predictive value for resistance ≥92%, predictive value for susceptibility 97%, and test efficiency 97%. According to ROC analysis, the microplate Alamar blue assay is a reliable method for determination of drug-susceptibility. Rapidity and cost efficiency are two additional qualities that make this test an excellent alternative for the drug susceptibility testing of Mycobacterium tuberculosis. The ROC curve analysis is a robust statistical approach for evaluating the performance of new quantitative methods for determination of drug sensitivity of Mycobacterium tuberculosis isolates.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1.
Fig. 2.
Fig. 3.
Fig. 4.

Similar content being viewed by others

References

  1. Cooksey RC, Crawford T, Jacobs WR, Shinnick TM (1993) A rapid method for screening antimicrobial agents for activities against a strain ofMycobacterium tuberculosis expressing firefly luciferase. Antimicrob Agents Chemother 37:1349–1352

    Google Scholar 

  2. Riska PF, Su Y, Bardarov S, Freundlich L, Sarkis G, Hatfull G, Carrière C, Kumar V, Chan J, Jacobs WR (1999) Rapid film-based determination of antibiotic susceptibilities ofMycobacterium tuberculosis by using a luciferase reporter phage and the Bronx Box. J Clin Microbiol 37:1144–1149

    CAS  PubMed  Google Scholar 

  3. Walters SB, Hanna BA (1996) Testing of susceptibility ofMycobacterium tuberculosis to isoniazid and rifampin by Mycobacterium growth indicator tube method. J Clin Microbiol 34:1565–1567

    CAS  PubMed  Google Scholar 

  4. Palomino JC, Traore H, Fissette K, Portaels F (1999) Evaluation of mycobacteria growth indicator tube (MGIT) for drug susceptibility testing ofMycobacterium tuberculosis. Int J Tuberc Lung Dis 3:344–348

    CAS  PubMed  Google Scholar 

  5. Wagner M, Mills K (1996) Testing ofMycobacterium tuberculosis susceptibility to ethambutol, isoniazid, rifampin and streptomycin by using E test. J Clin Microbiol 34:1672–1676

    PubMed  Google Scholar 

  6. Norden MA, Kurzynski TA, Bownds SE, Callister SM, Schell RF (1995) Rapid susceptibility testing ofMycobacterium tuberculosis (H37Ra) by flow cytometry. J Clin Microbiol 33:1231–1237

    CAS  PubMed  Google Scholar 

  7. Moore AV, Kirk SM, Callister SM, Mazurek GH, Schell RF (1999) Safe determination of susceptibility ofMycobacterium tuberculosis to antimycobacterial agents by flow cytometry. J Clin Microbiol 37:479–483

    CAS  PubMed  Google Scholar 

  8. Eltringham IJ, Wilson SM, Drobniewski FA (1999) Evaluation of a bacteriophage-based assay (phage amplified biologically assay) as a rapid screen for resistance to isoniazid, ethambutol, streptomycin, pyrazinamide and ciprofloxacin among clinical isolates of Mycobacterium tuberculosis. J Clin Microbiol 37:3528–3532

    CAS  PubMed  Google Scholar 

  9. Gomez-Flores R, Gupta S, Tamez-Guerra R, Mehta RT (1995) Determination of MICs forMycobacterium avium-M. intracelullare complex in liquid medium by a colorimetric method. J Clin Microbiol 33:1842–1846

    CAS  PubMed  Google Scholar 

  10. Mshana RN, Tadesse G, Abate G, Miörner H (1998) Use of 3(-4,5-dimethylthiazol-2-yl)-2,5-diphenyl tetrazolium bromide (MTT) for rapid detection of rifampicin-resistantMycobacterium tuberculosis. J Clin Micobiol 36:1214–1219

    CAS  Google Scholar 

  11. Yajko DM, Madej JJ, Lancaster MV, Sanders CA, Cawthon VL, Gee B, Babst A, Hadley WK (1995) Colorimetric method for determining MICs of antimicrobial agents for Mycobacterium tuberculosis. J Clin Microbiol 33:2324–2327

    CAS  PubMed  Google Scholar 

  12. Collins LA, Franzblau SG (1997) Microplate Alamar blue assay versus BACTEC 460 system for high-throughput screening of compounds againstMycobacterium tuberculosis and Mycobacterium avium. Antimicrob Agents Chemother 41:1004–1009

    CAS  PubMed  Google Scholar 

  13. Franzblau SG, Witzig RS, McLaughlin JC, Torres P, Madico G, Hernandez A, Degnan MT, Cook MB, Quenzer VK, Ferguson RM, Gilman RH (1998) Rapid, low-technology MIC determination with clinicalMycobacterium tuberculosis isolates by using the microplate Alamar blue assay. J Clin Microbiol 36:362–366

    CAS  PubMed  Google Scholar 

  14. Ahmed SA, Gogal RM, Walsh JE (1994) A new rapid and simple non-radioactive assay to monitor and determine the proliferation of lymphocytes: an alternative to [3H] thymidine incorporation assay. J Immunol Methods 170:211–224

    CAS  PubMed  Google Scholar 

  15. Pfaller MA, Vu Q, Lancaster M, Espinel-Ingroff A, Fothergill VL, Grant C, McGinnis MR, Pasarell L, Rinaldi MG, Steele-Moore L (1994) Multisite reproducibility of colorimetric broth microdilution method for antifungal susceptibility testing of yeast isolates. J Clin Microbiol 32:1625–1628

    CAS  PubMed  Google Scholar 

  16. Griner PF, Mayewski RJ, Mushlin AI, Greenland P (1981) Selection and interpretation of diagnostic tests and procedures. Principles and applications. Ann Intern Med 94:557–592

    CAS  PubMed  Google Scholar 

  17. Swets AJ (1988) Measuring the accuracy of diagnostic systems. Science 240:1285–1293

    CAS  PubMed  Google Scholar 

  18. Dawson-Saunders B, Trapp RG (1994) Basic and clinical biostatistics. Appleton and Lange, Norwalk, CT, p 275

  19. van der Schouw YT, Verbeek ALM, Rouijs SHJ (1995) Guidelines for the assessment of new diagnostics tests. Invest Radiol 30:334–340

    PubMed  Google Scholar 

  20. Mitchell MF, Cantor SB, Brookner C, Utzinger U, Schottenfeld D, Richards-Kortum R (1999) Screening for squamous intraepithelial lesions with fluorescence spectroscopy. Obstet Gynecol 94:889–896

    Article  CAS  PubMed  Google Scholar 

  21. U.S. Food and Drug Administration, Center for Devices and Radiological Health (2001) Premarket applications for digital mammography systems; final guidance for industry and FDA. U.S. Food and Drug Administration, Washington DC, document no. 983

  22. Inderlied CB, Salfinger M (1995) Antimicrobial agents and susceptibility tests: mycobacteria. In: Murray PR, Baron EJ, Pfaller MA, Tenover FC, Yolken RH (eds) Manual of clinical microbiology. American Society for Microbiology, Washington DC, pp 1385–1404

  23. Rastogi N, Goh KS, David HL (1989) Drug susceptibility testing in tuberculosis: a comparison of the proportion methods using Löwenstein-Jensen, Middlebrook 7H10 and 7H11 agar media and a radiometric method. Res Microbiol 140:405–417

    Article  CAS  PubMed  Google Scholar 

  24. National Committee for Clinical Laboratory Standards (2000) Susceptibility testing of mycobacteria, nocardia and other aerobic actinomycetes. Tentative standard M24-T2, 2nd edn. NCCLS, Wayne, PA

  25. Jorgensen JH (1993) Selection criteria for an antimicrobial susceptibility testing system. J Clin Microbiol 31:2841–2844

    CAS  PubMed  Google Scholar 

  26. Jorgensen JH, Ferraro MJ (1998) Antimicrobial susceptibility testing: general principles and contemporary practices. Clin Infect Dis 26:973–980

    CAS  PubMed  Google Scholar 

  27. Heifets L (2000) Conventional methods for antimycobacterial susceptibility testing of Mycobacterium tuberculosis. In: Bastian I, Portaels F (eds) Multidrug-resistant tuberculosis. Kluwer, Dordrecht, pp 133–144

  28. Palomino JC, Portaels F (1999) Simple procedure for drug susceptibility testing of Mycobacterium tuberculosis using a commercial colorimetric assay. Eur J Clin Microbiol Infect Dis 18:380–383

    Article  CAS  PubMed  Google Scholar 

Download references

Acknowledgements

We thank Faustina Ramírez and Daniel Morales for technical assistance. This study was supported by the Consejo Nacional de Ciencia y Tecnologia (CONACYT) project 26448-M and by the Direccion de Estudios de Posgrado e Investigacion del Instituto Politecnico Nacional (DEPI-IPN) projects 970539 and 980682. A.E.M. and J.T.L. are SNI fellows. J.L.H. and F.Q.P. are Comision de Operacion y Fomento de Actividades Academicas (COFAA), Estimulo al Desempeño Docente (EDD), and SNI fellows.

Author information

Authors and Affiliations

  1. Departamento de Inmunología, Instituto Politécnico Nacional, Escuela Nacional de Ciencias Biológicas, Carpio y Plan de Ayala, Colonia Santo Tomás 11340, Mexico City, Mexico

    J. Luna-Herrera, G. Martínez-Cabrera, F. Quesada-Pascual & R. Delgadillo-Polanco

  2. Hospital de Pediatria, Centro Médico Nacional Siglo XXI, Instituto Mexicano del Seguro Social, Mexico City, Mexico

    R. Parra-Maldonado, J. A. Enciso-Moreno & J. Torres-López

  3. Pharmacology Department, University of Illinois at Chicago, Chicago, IL, USA

    S. G. Franzblau

Authors
  1. J. Luna-Herrera
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. G. Martínez-Cabrera
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. R. Parra-Maldonado
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. J. A. Enciso-Moreno
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. J. Torres-López
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. F. Quesada-Pascual
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. R. Delgadillo-Polanco
    View author publications

    You can also search for this author in PubMed Google Scholar

  8. S. G. Franzblau
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to J. Luna-Herrera.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Luna-Herrera, J., Martínez-Cabrera, G., Parra-Maldonado, R. et al. Use of Receiver Operating Characteristic Curves to Assess the Performance of a Microdilution Assay for Determination of Drug Susceptibility of Clinical Isolates of Mycobacterium tuberculosis . Eur J Clin Microbiol Infect Dis 22, 21–27 (2003). https://doi.org/10.1007/s10096-002-0855-5

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10096-002-0855-5

Keywords

Search

Navigation