Skip to main content
SpringerLink
Log in

Ensemble Classifiers for Predicting HIV-1 Resistance from Three Rule-Based Genotypic Resistance Interpretation Systems

Journal of Medical Systems volume 41, Article number: 155 (2017) Cite this article

Abstract

Resistance to antiretrovirals (ARVs) is a major problem faced by HIV-infected individuals. Different rule-based algorithms were developed to infer HIV-1 susceptibility to antiretrovirals from genotypic data. However, there is discordance between them, resulting in difficulties for clinical decisions about which treatment to use. Here, we developed ensemble classifiers integrating three interpretation algorithms: Agence Nationale de Recherche sur le SIDA (ANRS), Rega, and the genotypic resistance interpretation system from Stanford HIV Drug Resistance Database (HIVdb). Three approaches were applied to develop a classifier with a single resistance profile: stacked generalization, a simple plurality vote scheme and the selection of the interpretation system with the best performance. The strategies were compared with the Friedman’s test and the performance of the classifiers was evaluated using the F-measure, sensitivity and specificity values. We found that the three strategies had similar performances for the selected antiretrovirals. For some cases, the stacking technique with naïve Bayes as the learning algorithm showed a statistically superior F-measure. This study demonstrates that ensemble classifiers can be an alternative tool for clinical decision-making since they provide a single resistance profile from the most commonly used resistance interpretation systems.

This is a preview of subscription content, access via your institution.

Access options

Buy single article

Instant access to the full article PDF.

USD 39.95

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Fig. 1
Fig. 2

References

  1. Paredes, R., and Clotet, B., Clinical management of HIV-1 resistance. Antivir. Res. 85:245–265, 2010. https://doi.org/10.1016/j.antiviral.2009.09.015.

    Article  CAS  PubMed  Google Scholar 

  2. Bronze, M., Steegen, K., Wallis, C.L., de Wolf, H., Papathanasopoulos, M.A., van Houtte, M., Stevens, W.S., de Wit, T.R., and Stuyver, L.J., Hiv-1 phenotypic reverse transcriptase inhibitor drug resistance test interpretation is not dependent on the subtype of the virus backbone. PLoS One, 2012. https://doi.org/10.1371/journal.pone.0034708.

  3. Hertogs, K., de Béthune, M.P., Miller, V., Ivens, T., Schel, P., Van Cauwenberge, A., Van Den Eynde, C., Van Gerwen, V., Azijn, H., Van Houtte, M., Peeters, F., Staszewski, S., Conant, M., Bloor, S., Kemp, S., Larder, B., and Pauwels, R., A rapid method for simultaneous detection of phenotypic resistance to inhibitors of protease and reverse transcriptase in recombinant human immunodeficiency virus type 1 isolates from patients treated with antiretroviral drugs. Antimicrob. Agents Chemother. 42:269–276, 1998.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  4. Beerenwinkel, N., Schmidt, B., Walter, H., Kaiser, R., Lengauer, T., Hoffmann, D., Korn, K., and Selbig, J., Diversity and complexity of HIV-1 drug resistance: A bioinformatics approach to predicting phenotype from genotype. Proc. Natl. Acad. Sci. U. S. A. 99:8271–8276, 2002. https://doi.org/10.1073/pnas.112177799.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  5. Vercauteren, J., and Vandamme, A.M., Algorithms for the interpretation of HIV-1 genotypic drug resistance information. Antivir. Res. 71:335–342, 2006. https://doi.org/10.1016/j.antiviral.2006.05.003.

    Article  CAS  PubMed  Google Scholar 

  6. Ravela, J., Betts, B.J., Brun-Vézinet, F., Vandamme, A.-M., Descamps, D., van Laethem, K., Smith, K., Schapiro, J.M., Winslow, D.L., Reid, C., and Shafer, R.W., HIV-1 protease and reverse transcriptase mutation patterns responsible for discordances between genotypic drug resistance interpretation algorithms. J. Acquir. Immune Defic. Syndr. 33:8–14, 2003.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  7. De Luca, A., Cingolani, A., Di Giambenedetto, S., Trotta, M.P., Baldini, F., Rizzo, M.G., Bertoli, A., Liuzzi, G., Narciso, P., Murri, R., Ammassari, A., Perno, C.F., and Antinori, A., Variable prediction of antiretroviral treatment outcome by different systems for interpreting genotypic human immunodeficiency virus type 1 drug resistance. J. Infect. Dis. 183:1934–1943, 2003. https://doi.org/10.1086/375355.

    Google Scholar 

  8. Yebra, G., de Mulder, M., del Romero, J., Rodríguez, C., and Holguín, A., HIV-1 non-B subtypes: High transmitted NNRTI-resistance in Spain and impaired genotypic resistance interpretation due to variability. Antivir. Res. 85:409–417, 2010. https://doi.org/10.1016/j.antiviral.2009.11.010.

    Article  CAS  PubMed  Google Scholar 

  9. Zazzi, M., Romano, L., Venturi, G., Shafer, R.W., Reid, C., Dal Bello, F., Parolin, C., Palù, G., and Valensin, P.E., Comparative evaluation of three computerized algorithms for prediction of antiretroviral susceptibility from HIV type 1 genotype. J. Antimicrob. Chemother. 53:356–360, 2004. https://doi.org/10.1093/jac/dkh021.

    Article  CAS  PubMed  Google Scholar 

  10. Meynard, J.-L., Vray, M., Morand-Joubert, L., Race, E., Descamps, D., Peytavin, G., Matheron, S., Lamotte, C., Guiramand, S., Costagliola, D., Brun-Vézinet, F., Clavel, F., and Girard, P.-M., Phenotypic or genotypic resistance testing for choosing antiretroviral therapy after treatment failure: A randomized trial. AIDS. 16:727–736, 2002.

    Article  PubMed  Google Scholar 

  11. Van Laethem, K., De Luca, A., Antinori, A., Cingolani, A., Perno, C.F., and Vandamme, A.M., A genotypic drug resistance interpretation algorithm that significantly predicts therapy response in HIV-1-infected patients. Antivir. Ther. 7:123–129, 2002.

    PubMed  Google Scholar 

  12. Shafer, R.W., Genotypic testing for human immunodeficiency virus type 1 drug resistance. Society. 15:247–277, 2002. https://doi.org/10.1128/CMR.15.2.247.

    CAS  Google Scholar 

  13. Yashik, S., and Maurice, M., Predicting a single HIV drug resistance measure from three international interpretation gold standards. Asian Pac. J. Trop. Med. 5:566–572, 2012. https://doi.org/10.1016/S1995-7645(12)60100-X.

    Article  CAS  PubMed  Google Scholar 

  14. Wagner, S., Kurz, M., and Klimkait, T., Algorithm evolution for drug resistance prediction: Comparison of systems for HIV-1 genotyping. Antivir. Ther. 20:661–665, 2015. https://doi.org/10.3851/IMP2947.

    Article  CAS  PubMed  Google Scholar 

  15. Cunha, P., Moura, D.C., Guevara López, M.A., Guerra, C., Pinto, D., and Ramos, I., Impact of ensemble learning in the assessment of skeletal maturity. J. Med. Syst. 38:87, 2014. https://doi.org/10.1007/s10916-014-0087-0.

    Article  PubMed  Google Scholar 

  16. Whalen, S., Pandey, G., A comparative analysis of ensemble classifiers: Case studies in genomics. In: 2013 I.E. 13th Int. Conf. Data Min. IEEE, pp 807–816, 2013.

  17. Heider, D., Dybowski, J.N., Wilms, C., and Hoffmann, D., A simple structure-based model for the prediction of HIV-1 co-receptor tropism. BioData Min. 7:14, 2014. https://doi.org/10.1186/1756-0381-7-14.

    Article  PubMed  PubMed Central  Google Scholar 

  18. Dybowski, J.N., Riemenschneider, M., Hauke, S., Pyka, M., Verheyen, J., Hoffmann, D., and Heider, D., Improved Bevirimat resistance prediction by combination of structural and sequence-based classifiers. BioData Min. 4:26, 2011. https://doi.org/10.1186/1756-0381-4-26.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  19. Kilic, N., Hosgormez, E., Automatic Estimation of Osteoporotic Fracture Cases by Using Ensemble Learning Approaches. 40:61, 2016. doi: https://doi.org/10.1007/s10916-015-0413-1

  20. Rhee, S.-Y., Gonzales, M.J., Kantor, R., Betts, B.J., Ravela, J., and Shafer, R.W., Human immunodeficiency virus reverse transcriptase and protease sequence database. Nucleic Acids Res. 31:298–303, 2003. https://doi.org/10.1093/nar/gkg100.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  21. Monogram Biosciences., Phenosense HIV Drug Resistance Assay. 1–2, 2014.

  22. He, H., and Garcia, E.A., Learning from imbalanced data. IEEE Trans. Knowl. Data Eng. 21:1263–1284, 2009. https://doi.org/10.1109/TKDE.2008.239.

    Article  Google Scholar 

  23. Lunardon, N., Menardi, G., Maintainer, N. T., Package “ROSE.” 6, 2015.

  24. Džeroski, S., and Ženko, B., Is combining classifiers with stacking better than selecting the best one? Mach. Learn. 54:255–273, 2004. https://doi.org/10.1023/B:MACH.0000015881.36452.6e.

    Article  Google Scholar 

  25. Rokach, L., Ensemble methods for classifiers. Data Min. Knowl. Discov. Handb.:957–980, 2005. https://doi.org/10.1007/0-387-25465-X_45.

  26. Wolpert, D.H., Stacked generalization. Neural Netw. 5:241–259, 1992. https://doi.org/10.1016/S0893-6080(05)80023-1.

    Article  Google Scholar 

  27. Gr, S.D., Paliouras, G., Spyropoulos, C.D., Gr, C.D., and Gr, M.U., Combining information extraction systems using voting and stacked generalization Georgios Sigletos Georgios Paliouras. J. Mach. Learn. Res. 6:1751–1782, 2005.

    Google Scholar 

  28. Ting, K. M., and Witten, I. H., Stacked generalization: When does it work? 866–871, 1997.

  29. Leung, K. M., Naive bayesian classifier. Polytech. Univ. Dep. Comput. Sci. Risk Eng, 2007.

  30. Tan, S., Neighbor-weighted K-nearest neighbor for unbalanced text corpus. Expert Syst. Appl. 28:667–671, 2005. https://doi.org/10.1016/j.eswa.2004.12.023.

    Article  Google Scholar 

  31. Kuhn, M., Package “caret.” 151,156, 2016.

  32. Roever, C., Raabe, N., Luebke, K., Ligges, U., Szepannek, G., and Zentgraf, M., Package “klaR.” 33, 2015.

  33. Schliep, K., Hechenbichler, K., and Lizee, A., Package “kknn.” 2016 5.

  34. Demšar, J., Statistical comparisons of classifiers over multiple data sets. J. Mach. Learn. Res. 7:1–30, 2006.

    Google Scholar 

  35. Holm, S., A simple sequentially Rejective multiple test procedure. Scand. J. Stat. 6:65–70, 1979.

    Google Scholar 

  36. Beerenwinkel, N., D Umer, M., Oette, M., Korn, K., Hoffmann, D., Kaiser, R., Lengauer, T., Selbig, J., and Walter, H., Geno2pheno: Estimating phenotypic drug resistance from HIV-1 genotypes. Nucleic Acids Res. 31:3850–3855, 2003. https://doi.org/10.1093/nar/gkg575.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  37. Vergne, L., Snoeck, J., Aghokeng, A., Maes, B., Valea, D., Delaporte, E., Vandamme, A.M., Peeters, M., and Van Laethem, K., Genotypic drug resistance interpretation algorithms display high levels of discordance when applied to non-B strains from HIV-1 naive and treated patients. FEMS Immunol. Med. Microbiol. 46:53–62, 2006. https://doi.org/10.1111/j.1574-695X.2005.00011.x.

    Article  CAS  PubMed  Google Scholar 

  38. Poonpiriya, V., Sungkanuparph, S., Leechanachai, P., Pasomsub, E., Watitpun, C., Chunhakan, S., and Chantratita, W., A study of seven rule-based algorithms for the interpretation of HIV-1 genotypic resistance data in Thailand. J. Virol. Methods. 151:79–86, 2008. https://doi.org/10.1016/j.jviromet.2008.03.017.

    Article  CAS  PubMed  Google Scholar 

  39. Weiss, G. M., Provost, F., The effect of class distribution on classifier learning, 2001

  40. Japkowicz, N., Learning from imbalanced data sets: A comparison of various strategies. In: Work. Notes AAAI’00 work. pp 10–15, 2000.

  41. Estabrooks, A., Jo, T., and Japkowicz, N., A multiple resampling method for learning from imbalanced data sets. Comput. Intell. 20:18–36, 2004. https://doi.org/10.1111/j.0824-7935.2004.t01-1-00228.x.

    Article  Google Scholar 

  42. Snoeck, J., Kantor, R., Shafer, R.W., Van Laethem, K., Deforche, K., Carvalho, A.P., Wynhoven, B., Soares, M.A., Cane, P., Clarke, J., Pillay, C., Sirivichayakul, S., Ariyoshi, K., Holguin, A., Rudich, H., Rodrigues, R., Bouzas, M.B., Brun-Vézinet, F., Reid, C., Cahn, P., Brigido, L.F., Grossman, Z., Soriano, V., Sugiura, W., Phanuphak, P., Morris, L., Weber, J., Pillay, D., Tanuri, A., Harrigan, R.P., Camacho, R., Schapiro, J.M., Katzenstein, D., and Vandamme, A.M., Discordances between interpretation algorithms for genotypic resistance to protease and reverse transcriptase inhibitors of human immunodeficiency virus are subtype dependent. Antimicrob. Agents Chemother. 50:694–701, 2006. https://doi.org/10.1128/AAC.50.2.694-701.2006.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

Download references

Acknowledgements

We want to acknowledge FAPERJ (Carlos Chagas Filho Foundation for Research Support of the State of Rio de Janeiro), CNPq Brazil (National Counsel of Technological and Scientific Development) and CAPES (Coordination for the Improvement of Higher-Education Personnel) for the financial support provided for this research.

Funding

Letícia Raposo is funded by Carlos Chagas Filho Foundation for Research Support of the State of Rio de Janeiro (FAPERJ, grant 221,340) and was funded by National Counsel of Technological and Scientific Development (CNPq Brazil, grant 131,968/2012–2).

Author information

Authors and Affiliations

  1. Biomedical Engineering Program, Federal University of Rio de Janeiro – UFRJ, Ilha do Fundão, Rio de Janeiro, RJ, Brazil

    Letícia M. Raposo & Flavio F. Nobre

Authors
  1. Letícia M. Raposo
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Flavio F. Nobre
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Letícia M. Raposo.

Ethics declarations

Conflict of Interest

The authors declare that they have no conflict of interest.

Ethical Approval

This article does not contain any studies with human participants or animals performed by any of the authors.

Additional information

This article is part of the Topical Collection on Systems-Level Quality Improvement

Rights and permissions

Reprints and Permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Raposo, L.M., Nobre, F.F. Ensemble Classifiers for Predicting HIV-1 Resistance from Three Rule-Based Genotypic Resistance Interpretation Systems. J Med Syst 41, 155 (2017). https://doi.org/10.1007/s10916-017-0802-8

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1007/s10916-017-0802-8

Keywords

search

Navigation