Skip to main content

Advertisement

SpringerLink
Log in

D3R grand challenge 2015: Evaluation of protein–ligand pose and affinity predictions

  • Published:
Journal of Computer-Aided Molecular Design Aims and scope Submit manuscript

Abstract

The Drug Design Data Resource (D3R) ran Grand Challenge 2015 between September 2015 and February 2016. Two targets served as the framework to test community docking and scoring methods: (1) HSP90, donated by AbbVie and the Community Structure Activity Resource (CSAR), and (2) MAP4K4, donated by Genentech. The challenges for both target datasets were conducted in two stages, with the first stage testing pose predictions and the capacity to rank compounds by affinity with minimal structural data; and the second stage testing methods for ranking compounds with knowledge of at least a subset of the ligand–protein poses. An additional sub-challenge provided small groups of chemically similar HSP90 compounds amenable to alchemical calculations of relative binding free energy. Unlike previous blinded Challenges, we did not provide cognate receptors or receptors prepared with hydrogens and likewise did not require a specified crystal structure to be used for pose or affinity prediction in Stage 1. Given the freedom to select from over 200 crystal structures of HSP90 in the PDB, participants employed workflows that tested not only core docking and scoring technologies, but also methods for addressing water-mediated ligand–protein interactions, binding pocket flexibility, and the optimal selection of protein structures for use in docking calculations. Nearly 40 participating groups submitted over 350 prediction sets for Grand Challenge 2015. This overview describes the datasets and the organization of the challenge components, summarizes the results across all submitted predictions, and considers broad conclusions that may be drawn from this collaborative community endeavor.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10

Similar content being viewed by others

References

  1. Smith RD, Dunbar JB Jr, Ung PM et al (2011) CSAR benchmark exercise of 2010: combined evaluation across all submitted scoring functions. J Chem Inf Model 51:2115–2131. doi:10.1021/ci200269q

    Article  CAS  Google Scholar 

  2. Damm-Ganamet KL, Smith RD, Dunbar JB Jr et al (2013) CSAR benchmark exercise 2011–2012: evaluation of results from docking and relative ranking of blinded congeneric series. J Chem Inf Model 53:1853–1870. doi:10.1021/ci400025f

    Article  CAS  Google Scholar 

  3. Smith RD, Damm-Ganamet KL, Dunbar JB Jr et al (2016) CSAR benchmark exercise 2013: evaluation of results from a combined computational protein design, docking, and scoring/ranking challenge. J Chem Inf Model 56:1022–1031. doi:10.1021/acs.jcim.5b00387

    Article  CAS  Google Scholar 

  4. Carlson HA, Smith RD, Damm-Ganamet KL et al (2016) CSAR 2014: a benchmark exercise using unpublished data from pharma. J Chem Inf Model 56:1063–1077. doi:10.1021/acs.jcim.5b00523

  5. Solit DB, Rosen N (2006) Hsp90: a novel target for cancer therapy. Curr Top Med Chem 6:1205–1214

    Article  CAS  Google Scholar 

  6. Travers J, Sharp S, Workman P (2012) HSP90 inhibition: two-pronged exploitation of cancer dependencies. Drug Discov Today 17:242–252. doi:10.1016/j.drudis.2011.12.021

    Article  CAS  Google Scholar 

  7. Crawford TD, Ndubaku CO, Chen H et al (2014) Discovery of selective 4-amino-pyridopyrimidine inhibitors of MAP4K4 using fragment-based lead identification and optimization. J Med Chem 57:3484–3493. doi:10.1021/jm500155b

    Article  CAS  Google Scholar 

  8. Pearl LH, Prodromou C (2006) Structure and mechanism of the Hsp90 molecular chaperone machinery. Annu Rev Biochem 75:271–294. doi:10.1146/annurev.biochem.75.103004.142738

    Article  CAS  Google Scholar 

  9. Tembe BL, McCammon JA (1984) Ligand-receptor interactions. Comput Chem 8:281–283. doi:10.1016/0097-8485(84)85020-2

    Article  Google Scholar 

  10. Ndubaku CO, Crawford TD, Chen H et al (2015) Structure-based design of GNE-495, a potent and selective MAP4K4 inhibitor with efficacy in retinal angiogenesis. ACS Med Chem Lett 6:913–918. doi:10.1021/acsmedchemlett.5b00174

    Article  CAS  Google Scholar 

  11. Schroder P, Forster T, Kleine S et al (2015) Neuritogenic militarinone-inspired 4-hydroxypyridones target the stress pathway kinase MAP4K4. AngewChemIntEdEngl 54:12398–12403

    Google Scholar 

  12. Wang L, Stanley M, Boggs J et al (2014) Fragment-based identification and optimization of a class of potent pyrrolo[2,1-f][1, 2, 4]triazine MAP4K4 inhibitors. BioorgMedChemLett 24:4546–4552

    CAS  Google Scholar 

  13. Guimaraes CR, Rai BK, Munchhof MJ et al (2011) Understanding the impact of the P-loop conformation on kinase selectivity. J Chem Inf Model 51:1199–1204. doi:10.1021/ci200153c

    Article  CAS  Google Scholar 

  14. Huth JR, Park C, Petros AM et al (2007) Discovery and design of novel HSP90 inhibitors using multiple fragment-based design strategies. Chem Biol Drug Des 70:1–12. doi:10.1111/j.1747-0285.2007.00535.x

    Article  CAS  Google Scholar 

  15. Dunbar JB Jr, Smith RD, Damm-Ganamet KL et al (2013) CSAR data set release 2012: ligands, affinities, complexes, and docking decoys. J Chem Inf Model 53:1842–1852. doi:10.1021/ci4000486

    Article  CAS  Google Scholar 

  16. Bruncko M, Tahir SK, Song X et al (2010) N-aryl-benzimidazolones as novel small molecule HSP90 inhibitors. Bioorg Med Chem Lett 20:7503–7506. doi:10.1016/j.bmcl.2010.10.010

    Article  CAS  Google Scholar 

  17. Wang L, Stanley M, Boggs JW et al (2014) Fragment-based identification and optimization of a class of potent pyrrolo[2,1-f][1, 2, 4]triazine MAP4K4 inhibitors. Bioorg Med Chem Lett 24:4546–4552. doi:10.1016/j.bmcl.2014.07.071

    Article  CAS  Google Scholar 

  18. Molecular Operating Environment (MOE) 2013.08; Chemical Computing Group Inc., 1010 Sherbooke St. West, Suite #910, Montreal, QC, Canada, H3A 2R7, 2016.

  19. Read RJ, Adams PD, Arendall WB 3rd et al (2011) A new generation of crystallographic validation tools for the protein data bank. Structure 19:1395–1412. doi:10.1016/j.str.2011.08.006

    Article  CAS  Google Scholar 

  20. Yang H, Peisach E, Westbrook JD et al (2016) DCC: a Swiss army knife for structure factor analysis and validation. J Appl Crystallogr 49:1081–1084. doi:10.1107/S1600576716004428

    Article  CAS  Google Scholar 

  21. OpenEye Toolkits 2016. Jun.1 OpenEye Scientific Software, Santa Fe, NM. http://www.eyesopen.com

  22. Mobley DL, Liu S, Lim NM et al (2014) Blind prediction of HIV integrase binding from the SAMPL4 challenge. J Comput Aided Mol Des 28:327–345. doi:10.1007/s10822-014-9723-5

    Article  CAS  Google Scholar 

  23. Brown SP, Muchmore SW, Hajduk PJ (2009) Healthy skepticism: assessing realistic model performance. Drug Discov Today 14:420–427

    Article  Google Scholar 

  24. Antosiewicz J, McCammon JA, Gilson MK (1994) Prediction of Ph-dependent properties of proteins. J Mol Biol 238:415–436. doi:10.1006/jmbi.1994.1301

    Article  CAS  Google Scholar 

  25. Klopman G, Li J-Y, Wang S, Dimayuga M (1994) Computer automated log P calculations based on an extended group contribution approach. Computer automated log P calculations based on an extended group contribution approach. J Chem Inf Comput Sci. 34:752–781. doi:10.1021/ci00020a009

  26. Ekins S, Bunin BA (2013) The Collaborative Drug Discovery (CDD) database. Methods Mol Biol 993:139–154. doi:10.1007/978-1-62703-342-8_10

    Article  CAS  Google Scholar 

  27. Yung-Chi C, Prusoff WH (1973) Relationship between the inhibition constant (KI) and the concentration of inhibitor which causes 50 per cent inhibition (I50) of an enzymatic reaction. Biochem Pharmacol 22:3099–3108. doi:10.1016/0006-2952(73)90196-2

    Article  Google Scholar 

  28. Muddana HS, Fenley AT, Mobley DL, Gilson MK (2014) The SAMPL4 host-guest blind prediction challenge: an overview. J Comput Aided Mol Des 28:305–317. doi:10.1007/s10822-014-9735-1

    Article  CAS  Google Scholar 

  29. Papadatos G, Gaulton A, Hersey A, Overington JP (2015) Activity, assay and target data curation and quality in the ChEMBL database. J Comput Aided Mol Des 29:885–896. doi:10.1007/s10822-015-9860-5

    Article  CAS  Google Scholar 

  30. Hawkins PC, Skillman AG, Warren GL et al (2010) Conformer generation with OMEGA: algorithm and validation using high quality structures from the Protein Databank and Cambridge Structural Database. J Chem Inf Model 50:572–584. doi:10.1021/ci100031x

    Article  CAS  Google Scholar 

  31. Kumar A, Zhang KYJ (2016) A pose prediction approach based on ligand 3D shape similarity. J Comput Aided Mol Des 30:457–469. doi:10.1007/s10822-016-9923-2

    Article  CAS  Google Scholar 

  32. Kumar A, Zhang KYJ (2016) Prospective evaluation of shape similarity based pose prediction method in D3R Grand Challenge 2015. J Comput Aided Mol Des. doi:10.1007/s10822-016-9931-2

    Google Scholar 

  33. Rush TS 3rd, Grant JA, Mosyak L, Nicholls A (2005) A shape-based 3-D scaffold hopping method and its application to a bacterial protein–protein interaction. J Med Chem 48:1489–1495. doi:10.1021/jm040163o

    Article  CAS  Google Scholar 

  34. Cozzetto D, Tramontano A (2005) Relationship between multiple sequence alignments and quality of protein comparative models. Proteins 58:151–157. doi:10.1002/prot.20284

    Article  CAS  Google Scholar 

  35. Kelley BP, Brown SP, Warren GL, Muchmore SW (2015) POSIT: flexible shape-guided docking for pose prediction. J Chem Inf Model. 55:1771-80. doi:10.1021/acs.jcim.5b00142

  36. Kim R, Skolnick J (2008) Assessment of programs for ligand binding affinity prediction. J Comput Chem 29:1316–1331. doi:10.1002/jcc.20893

    Article  CAS  Google Scholar 

  37. Gumbart JC, Roux B, Chipot C (2013) Efficient determination of protein–protein standard binding free energies from first principles. J Chem Theory Comput 9:3789–3798. doi:10.1021/ct400273t

    Article  CAS  Google Scholar 

  38. Gumbart JC, Roux B, Chipot C (2013) Standard binding free energies from computer simulations: what is the best strategy? J Chem Theory Comput 9:794–802. doi:10.1021/ct3008099

    Article  CAS  Google Scholar 

  39. Gooljarsingh LT, Fernandes C, Yan K et al (2006) A biochemical rationale for the anticancer effects of Hsp90 inhibitors: slow, tight binding inhibition by geldanamycin and its analogues. Proc Natl Acad Sci 103:7625–7630. doi:10.1073/pnas.0602650103

    Article  CAS  Google Scholar 

  40. Gaulton A, Bellis LJ, Bento AP et al (2012) ChEMBL: a large-scale bioactivity database for drug discovery. Nucleic Acids Res 40:D1100–D1107. doi:10.1093/nar/gkr777

    Article  CAS  Google Scholar 

  41. Liu T, Lin Y, Wen X et al (2007) BindingDB: a web-accessible database of experimentally determined protein–ligand binding affinities. Nucleic Acids Res 35:D198–D201. doi:10.1093/nar/gkl999

    Article  CAS  Google Scholar 

  42. Kim S, Thiessen PA, Bolton EE et al (2016) PubChem substance and compound databases. Nucleic Acids Res 44:D1202–D1213. doi:10.1093/nar/gkv951

    Article  Google Scholar 

Download references

Acknowledgments

This work was supported by National Institutes of Health (NIH) grant 1U01GM111528 for the Drug Design Data Resource (D3R) and U01 GM086873 to the Community Structure Activity Resource (CSAR). We are grateful to Dr. Seth Harris and to Genentech, Inc. for their collaboration and generous donation of the MAP4K4 dataset and Dr. Phil Hadjuk of Abbvie for the HSP90 dataset. We also thank OpenEye Scientific Software for generously donating the use of their software and Jenny Chong for her assistance in preparation of figures for the manuscript. The contents of this publication are solely the responsibility of the authors and do not necessarily represent the official views of the NIH. MKG has an equity interest in, and is a co-founder and scientific advisor of, VeraChem LLC. REA has equity interest in and is a co-founder of Actavalon, Inc., VAF has equity interest in Actavalon, Inc.

Author information

Author notes

  1. Victoria A. Feher

    Present address: Schrodinger, Inc., New York, NY, USA

Authors and Affiliations

  1. Drug Design Data Resource, Center for Research in Biological Systems, University of California, San Diego, 9500 Gilman Drive, La Jolla, CA, 92093, USA

    Symon Gathiaka, Shuai Liu, Michael Chiu, Rommie E. Amaro, Victoria A. Feher & Michael K. Gilson

  2. RCSB Protein Data Bank, Center for Integrative Proteomics Research, Institute for Quantitative Biomedicine, Department Chemistry and Chemical Biology, Rutgers, The State University of New Jersey, 174 Frelinghuysen Road, Piscataway, NJ, 08854, USA

    Huanwang Yang & Stephen K. Burley

  3. Center for Structural Biology, Life Sciences Institute, University of Michigan, 210 Washtenaw Avenue, Ann Arbor, MI, 48109-2216, USA

    Jeanne A. Stuckey, You Na Kang, Jim Delproposto & Ginger Kubish

  4. Department of Medicinal Chemistry, University of Michigan, 428 Church St., Ann Arbor, MI, 48109-1065, USA

    James B. Dunbar Jr. & Heather A. Carlson

  5. Department of Pharmacy, Skaggs School of Pharmacy and Pharmaceutical Sciences, University of California, San Diego, 9500 Gilman Drive, La Jolla, CA, 92093, USA

    Stephen K. Burley & Michael K. Gilson

  6. San Diego Supercomputer Center, University of California, San Diego, 9500 Gilman Drive, La Jolla, CA, 92093, USA

    Stephen K. Burley

  7. Relay Therapeutics, 215 First St, Cambridge, MA, 20142, USA

    W. Patrick Walters

  8. Department of Chemistry and Biochemistry, University of California, San Diego, 9500 Gilman Drive, La Jolla, CA, 92093, USA

    Rommie E. Amaro, Victoria A. Feher & Michael K. Gilson

Authors
  1. Symon Gathiaka
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Shuai Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Michael Chiu
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Huanwang Yang
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Jeanne A. Stuckey
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. You Na Kang
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Jim Delproposto
    View author publications

    You can also search for this author in PubMed Google Scholar

  8. Ginger Kubish
    View author publications

    You can also search for this author in PubMed Google Scholar

  9. James B. Dunbar Jr.
    View author publications

    You can also search for this author in PubMed Google Scholar

  10. Heather A. Carlson
    View author publications

    You can also search for this author in PubMed Google Scholar

  11. Stephen K. Burley
    View author publications

    You can also search for this author in PubMed Google Scholar

  12. W. Patrick Walters
    View author publications

    You can also search for this author in PubMed Google Scholar

  13. Rommie E. Amaro
    View author publications

    You can also search for this author in PubMed Google Scholar

  14. Victoria A. Feher
    View author publications

    You can also search for this author in PubMed Google Scholar

  15. Michael K. Gilson
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding authors

Correspondence to Rommie E. Amaro, Victoria A. Feher or Michael K. Gilson.

Electronic supplementary material

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Gathiaka, S., Liu, S., Chiu, M. et al. D3R grand challenge 2015: Evaluation of protein–ligand pose and affinity predictions. J Comput Aided Mol Des 30, 651–668 (2016). https://doi.org/10.1007/s10822-016-9946-8

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10822-016-9946-8

Keywords

Search

Navigation