Here, we give an overview of the protein-ligand binding portion of the Statistical Assessment of Modeling of Proteins and Ligands 4 (SAMPL4) challenge, which focused on predicting binding of HIV integrase inhibitors in the catalytic core domain. The challenge encompassed three components—a small “virtual screening” challenge, a binding mode prediction component, and a small affinity prediction component. Here, we give summary results and statistics concerning the performance of all submissions at each of these challenges. Virtual screening was particularly challenging here in part because, in contrast to more typical virtual screening test sets, the inactive compounds were tested because they were thought to be likely binders, so only the very top predictions performed significantly better than random. Pose prediction was also quite challenging, in part because inhibitors in the set bind to three different sites, so even identifying the correct binding site was challenging. Still, the best methods managed low root mean squared deviation predictions in many cases. Here, we give an overview of results, highlight some features of methods which worked particularly well, and refer the interested reader to papers in this issue which describe specific submissions for additional details.
This is a preview of subscription content, access via your institution.
Buy single article
Instant access to the full article PDF.
Tax calculation will be finalised during checkout.
Subscribe to journal
Immediate online access to all issues from 2019. Subscription will auto renew annually.
Tax calculation will be finalised during checkout.
The SAMPL4 challenge was designed, run and evaluated by the Mobley lab with some help from Kim Branson, so when this report uses the word “we” to refer to an action relating to challenge design, logistics, and analysis, it refers to these authors—specifically, Mobley, Branson, Su, Lim, Wymer, and Liu.
The challenge began with 322 compounds, 260 non-binders, and 62 binders, but due to errors and redundancies, final analysis was run on 305 compounds and 56 binders.
Abram ME, Hluhanich RM, Goodman DD, Andreatta KN, Margot NA, Ye L, Niedziela-Majka A, Barnes TL, Novikov N, Chen X, Svarovskaia ES, McColl DJ, White KL, Miller MD (2013) Impact of primary Elvitegravir resistance-associated mutations in HIV-1 integrase on drug susceptibility and viral replication fitness. Antimicrob Agents Chemother 57(6):2654–2663
Benson ML, Faver JC, Ucisik MN, Dashti DS, Zheng Z, Merz KM Jr (2012) Prediction of trypsin/molecular fragment binding affinities by free energy decomposition and empirical scores. J Comput Aided Mol Des 26(5):647–659
Blow DM (2002) Rearrangement of Cruickshank’s formulae for the diffraction-component precision index. Acta Crystallogr Sect D Biol Crystallogr 58(5):792–797
Cherepanov P, Ambrosio ALB, Rahman S, Ellenberger T, Engelman A (2005) Structural basis for the recognition between HIV-1 integrase and transcriptional coactivator p75. Proc Natl Acad Sci USA 102(48):17,308–17,313
Chodera JD, Mobley DL, Shirts MR, Dixon RW, Branson K, Pande VS (2011) Alchemical free energy methods for drug discovery: progress and challenges. Curr Opin Struct Biol 21(2):150–160
Christ F, Shaw S, Demeulemeester J, Desimmie BA, Marchand A, Butler S, Smets W, Chaltin P, Westby M, Debyser Z, Pickford C (2012) Small-molecule inhibitors of the LEDGF/p75 binding site of integrase block HIV replication and modulate integrase multimerization. Antimicrob Agents Chemother 56(8):4365–4374
Christ F, Voet A, Marchand A, Nicolet S, Desimmie BA, Marchand D, Bardiot D, Vander Veken NJ, Van Remoortel B, Strelkov SV, De Maeyer M, Chaltin P, Debyser Z (2010) Rational design of small-molecule inhibitors of the LEDGF/p75-integrase interaction and HIV replication. Nat Meth 6(6):442–448
Coleman RG, Sterling T, Weiss DR (2014) SAMPL4 & DOCK3.7: lessons for automated docking procedures. J Comput Aided Mol Des. doi:10.1007/s10822-014-9722-6
Corbeil CR, Sulea T, Purisima EO (2010) Rapid prediction of solvation free energy. 2. The first-shell hydration (FiSH) continuum model. J Chem Theory Comput 6(5):1622–1637
De Clercq E (1999) Perspectives of non-nucleoside reverse transcriptase inhibitors (NNRTIs) in the therapy of HIV-1 infection. Il Farmaco 54(1–2):26–45
Dewdney TG, Wang Y, Kovari IA, Reiter SJ, Kovari LC (2013) Reduced HIV-1 integrase flexibility as a mechanism for raltegravir resistance. J Struct Biol 184:245–250
Engelman A, Cherepanov P (2012) The structural biology of HIV-1: mechanistic and therapeutic insights. Nat Rev Microbiol 10(4):279–290
Ewing TJ, Makino S, Skillman AG, Kuntz ID (2001) DOCK 4.0: search strategies for automated molecular docking of flexible molecule databases. J Comput Aided Mol Des 15(5):411–428
Friesner RA, Murphy RB, Repasky MP, Frye LL, Greenwood JR, Halgren TA, Sanschagrin PC, Mainz DT (2006) Extra precision glide: docking and scoring incorporating a model of hydrophobic enclosure for proteinligand complexes. J Med Chem 49(21):6177–6196
Gallicchio E, Deng N, He P, Perryman AL, Santiago DN, Forli S, Olson AJ, Levy RM (2014) Virtual screening of integrase inhibitors by large scale binding free energy calculations. J Comput Aided Mol Des. doi:10.1007/s10822-014-9711-9
Gallicchio E, Lapelosa M, Levy RM (2010) The binding energy distribution analysis method (BEDAM) for the estimation of protein-ligand binding affinities. J Chem Theory Comput 6(9):2961–2977
Geretti AM, Armenia D, Ceccherini-Silberstein F (2012) Emerging patterns and implications of HIV-1 integrase inhibitor resistance. Curr Opin Infect Dis 25(6):677–686. doi:10.1097/QCO.0b013e32835a1de7
Greenwald J, Le V, Butler SL, Bushman FD, Choe S (1999) The mobility of an HIV-1 integrase active site loop is correlated with catalytic activity. Biochemistry 38(28):8892–8898
Hare S, Maertens GN, Cherepanov P (2012) 3[prime]-Processing and strand transfer catalysed by retroviral integrase in crystallo. EMBO J 31(13):3020–3028
Hare S, Smith SJ, Métifiot M, Jaxa-Chamiec A, Pommier Y, Hughes SH, Cherepanov P (2011) Structural and functional analyses of the second-generation integrase strand transfer inhibitor dolutegravir (S/GSK1349572). Mol Pharmacol 80(4):565–572
Hare S, Vos AM, Clayton RF, Thuring JW, Cummings MD, Cherepanov P (2010) Molecular mechanisms of retroviral integrase inhibition and the evolution of viral resistance. Proc Natl Acad Sci 107(46):20,057–20,062
Hawkins PCD, Nicholls A (2012) Conformer generation with OMEGA: learning from the data set and the analysis of failures. J Chem Inf Model 52(11):2919–2936
Hawkins PCD, Skillman AG, Warren GL, Ellingson BA, Stahl MT (2010) Conformer generation with OMEGA: algorithm and validation using high quality structures from the Protein Databank and Cambridge Structural Database. J Chem Inf Model 50(4):572–584
Hogues H, Sulea T, Purisima EO (2014) Exuastive docking and solvated interaction energy scoring: lessons learned from the SAMPL4 challenge. J Comput Aided Mol Des. doi:10.1007/s10822-014-9715-5
Japrung D, Leartsakulpanich U, Chusacultanachai S, Yuthavong Y (2007) Conflicting requirements of Plasmodium falciparum dihydrofolate reductase mutations conferring resistance to pyrimethamine-WR99210 combination. Antimicrob Agents Chemother 51(12):4356–4360
Jurado KA, Wang H, Slaughter A, Feng L, Kessl JJ, Koh Y, Wang W, Ballandras-Colas A, Patel PA, Fuchs JR, Kvaratskhelia M, Engelman A (2013) Allosteric integrase inhibitor potency is determined through the inhibition of HIV-1 particle maturation. Proc Natl Acad Sci 110(21):8690–8695
Kessl JJ, Jena N, Koh Y, Taskent-Sezgin H, Slaughter A, Feng L, de Silva S, Wu L, Le Grice SFJ, Engelman A, Fuchs JR, Kvaratskhelia M (2012) Multimode, cooperative mechanism of action of allosteric HIV-1 integrase inhibitors. J Biol Chem 287(20):16,801–16,811
Krishnan L, Engelman A (2012) Retroviral integrase proteins and HIV-1 DNA integration. J Biol Chem 287(49):40,858–40,866
Kuhn B, Kollman PA (2000) Binding of a diverse set of ligands to avidin and streptavidin: an accurate quantitative prediction of their relative affinities by a combination of molecular mechanics and continuum solvent models. J Med Chem 43(20):3786–3791
Kulp JL, Blumenthal SN, Wang Q, Bryan RL, Guarnieri F (2012) A fragment-based approach to the SAMPL3 challenge. J Comput Aided Mol Des 26(5):583–594
Kumar A, Zhang KYJ (2012) Computational fragment-based screening using RosettaLigand: the SAMPL3 challenge. J Comput Aided Mol Des 26(5):603–616
Kuntz ID, Chen K, Sharp KA, Kollman PA (1999) The maximal affinity of ligands. Proc Natl Acad Sci 96(18):9997–10,002
Lindorff-Larsen K, Piana S, Palmo K, Maragakis P, Klepeis JL, Dror RO, Shaw DE (2010) Improved side-chain torsion potentials for the Amber ff99SB protein force field. Proteins pp NA–NA 78(8):1950–1958
Maertens GN, Hare S, Cherepanov P (2010) The mechanism of retroviral integration from X-ray structures of its key intermediates. Nature 468(7321):326–329
MarvinSketch version 5.8.2 (2013) ChemAxon. http://www.chemaxon.com/products/marvin/marvinsketch/
Métifiot M, Maddali K, Johnson BC, Hare S, Smith SJ, Zhao XZ, Marchand C, Burke TR, Hughes SH, Cherepanov P, Pommier Y (2013) Activities, crystal structures, and molecular dynamics of dihydro-1H-isoindole derivatives, inhibitors of HIV-1 integrase. ACS Chem Biol 8(1):209–217
Mobley DL, Wymer KL, Lim NM (2014) Blind prediction of solvation free energies from the SAMPL4 challenge. J Comput Aided Mol Des
Naïm M, Bhat S, Rankin KN, Dennis S, Chowdhury SF, Siddiqi I, Drabik P, Sulea T, Bayly CI, Jakalian A (2007) Solvated interaction energy (SIE) for scoring protein-ligand binding affinities. 1. Exploring the parameter space. J Chem Inf Model 47(1):122–133
Newman J, Dolezal O, Fazio V, Caradoc-Davies T, Peat TS (2012) The DINGO dataset: a comprehensive set of data for the SAMPL challenge. J Comput Aided Mol Des 26(5):497–503
Nicholls A, Mobley DL, Guthrie JP, Chodera JD, Bayly CI, Cooper MD, Pande VS (2008) Predicting small-molecule solvation free energies: an informal blind test for computational chemistry. J Med Chem 51(4):769–779
OpenEye Python Toolkits. http://www.eyesopen.com (2013)
Peat TS, Dolezal O, Newman J, Mobley DL, Deadman JJ (2014) Interrogating HIV integrase for compounds that bind—a SAMPL4 challenge. J Comput Aided Mol Des. doi:10.1007/s10822-014-9721-7
Peat TS, Warren G (2013) Personal Communication. E-mail exchange
Perryman AL, Forli S, Morris GM, Burt C, Cheng Y, Palmer MJ, Whitby K, McCammon JA, Phillips C, Olson AJ (2010) A dynamic model of HIV integrase inhibition and drug resistance. J Mol Biol 397(2):600–615
Quashie PK, Mesplède T, Han YS, Veres T, Osman N, Hassounah S, Sloan R, Xu HT, Wainberg MA (2013) Biochemical analysis of the role of G118R-linked dolutegravir drug resistance substitutions in HIV-1 integrase. Antimicrob Agents Chemother 57(12):6223–6235
Quashie PK, Mesplède T, Wainberg MA (2013) Evolution of HIV integrase resistance mutations. Curr Opin Infect Dis 26(1):43–49. doi:10.1097/QCO.0b013e32835ba81c
Skillman AG, Warren GL, Nicholls A (2008) SAMPL at first glance: So much data, so little time…. http://www.eyesopen.com/2008_cup_presentations/CUP9_Skillman.pdf
Sulea T, Cui Q, Purisima EO (2011) Solvated interaction energy (SIE) for scoring protein–ligand binding affinities. 2. Benchmark in the CSAR-2010 scoring exercise. J Chem Inf Model 51(9):2066–2081
Sulea T, Hogues H, Purisima EO (2012) Exhaustive search and solvated interaction energy (SIE) for virtual screening and affinity prediction. J Comput Aided Mol Des 26(5):617–633
Surpateanu G, Iorga BI (2012) Evaluation of docking performance in a blinded virtual screening of fragment-like trypsin inhibitors. J Comput Aided Mol Des 26(5):595–601
Trott O, Olson AJ (2010) AutoDock Vina: improving the speed and accuracy of docking with a new scoring function, efficient optimization, and multithreading. J Comput Chem 31(2):455–461
Truchon J, Bayly CI (2007) Evaluating virtual screening methods: good and bad metrics for the “early recognition” problem. J Chem Inf Model 47(2):488–508
Tsiang M, Jones GS, Niedziela-Majka A, Kan E, Lansdon EB, Huang W, Hung M, Samuel D, Novikov N, Xu Y, Mitchell M, Guo H, Babaoglu K, Liu X, Geleziunas R, Sakowicz R (2012) New class of HIV-1 integrase (IN) inhibitors with a dual mode of action. J Biol Chem 287(25):21,189–21,203
Voet ARD, Kumar A, Berenger F, Zhang KYJ (2014) Combining in cerebra and in silico approaches for virtual screening and pose prediction in SAMPL4. J Comput Aided Mol Des. doi:10.1007/s10822-013-9702-2
Wainberg MA, Mesplède T, Quashie PK (2012) The development of novel HIV integrase inhibitors and the problem of drug resistance. Curr Opin Virol 2(5):656–662
Wang J, Wolf R, Caldwell J, Kollman P, Case D (2004) Development and testing of a general amber force field. J Comput Chem 25(9):1157–1174
Wang R, Liu L, Lai L, Tang Y (1998) SCORE: a new empirical method for estimating the binding affinity of a protein-ligand complex. J Mol Model 4(12):379–394
Zhang J, Adrian FJ, Jahnke W, Cowan-Jacob SW, Li AG, Iacob RE, Sim T, Powers J, Dierks C, Sun F, Guo GR, Ding Q, Okram B, Choi Y, Wojciechowski A, Deng X, Liu G, Fendrich G, Strauss A, Vajpai N, Grzesiek S, Tuntland T, Liu Y, Bursulaya B, Azam M, Manley PW, Engen JR, Daley GQ, Warmuth M, Gray NS (2010) Targeting Bcr-Abl by combining allosteric with ATP-binding-site inhibitors. Nature 463(7280):501–506
We acknowledge the financial support of the National Institutes of Health (1R15GM096257-01A1 to DLM and R01 GM073087 and P50 GM103368 to AJO), and computing support from the UCI GreenPlanet cluster, supported in part by NSF Grant CHE-0840513. We are also grateful to OpenEye Scientific Software for support for SAMPL, including for the meeting and for logistical help with the website, and in particular would like to thank Matt Geballe for help with the website and submissions and for helpful discussions, and Paul Hawkins, Greg Warren, and Geoff Skillman for helpful discussions and pointers on analysis. We are also thankful to Tom Peat (CSIRO) and colleagues for the experimental data which made the integrase portion of SAMPL possible, and helped initiate SAMPL4.
Electronic supplementary material
Below is the link to the electronic supplementary material.
About this article
Cite this article
Mobley, D.L., Liu, S., Lim, N.M. et al. Blind prediction of HIV integrase binding from the SAMPL4 challenge. J Comput Aided Mol Des 28, 327–345 (2014). https://doi.org/10.1007/s10822-014-9723-5
- HIV integrase
- Binding mode
- Virtual screening
- Pose prediction