Journal of Computer-Aided Molecular Design

, Volume 32, Issue 1, pp 1–20 | Cite as

D3R Grand Challenge 2: blind prediction of protein–ligand poses, affinity rankings, and relative binding free energies

  • Zied Gaieb
  • Shuai Liu
  • Symon Gathiaka
  • Michael Chiu
  • Huanwang Yang
  • Chenghua Shao
  • Victoria A. Feher
  • W. Patrick Walters
  • Bernd Kuhn
  • Markus G. Rudolph
  • Stephen K. Burley
  • Michael K. GilsonEmail author
  • Rommie E. Amaro


The Drug Design Data Resource (D3R) ran Grand Challenge 2 (GC2) from September 2016 through February 2017. This challenge was based on a dataset of structures and affinities for the nuclear receptor farnesoid X receptor (FXR), contributed by F. Hoffmann-La Roche. The dataset contained 102 IC50 values, spanning six orders of magnitude, and 36 high-resolution co-crystal structures with representatives of four major ligand classes. Strong global participation was evident, with 49 participants submitting 262 prediction submission packages in total. Procedurally, GC2 mimicked Grand Challenge 2015 (GC2015), with a Stage 1 subchallenge testing ligand pose prediction methods and ranking and scoring methods, and a Stage 2 subchallenge testing only ligand ranking and scoring methods after the release of all blinded co-crystal structures. Two smaller curated sets of 18 and 15 ligands were developed to test alchemical free energy methods. This overview summarizes all aspects of GC2, including the dataset details, challenge procedures, and participant results. We also consider implications for progress in the field, while highlighting methodological areas that merit continued development. Similar to GC2015, the outcome of GC2 underscores the pressing need for methods development in pose prediction, particularly for ligand scaffolds not currently represented in the Protein Data Bank (, and in affinity ranking and scoring of bound ligands.


D3R Docking Scoring Ligand ranking Alchemical methods Farnesoid X receptor Blinded prediction challenge 



This work was supported by National Institutes of Health (NIH) grant 1U01GM111528 for the Drug Design Data Resource (D3R). We also thank OpenEye Scientific Software for generously donating the use of their software. RCSB Protein Data Bank is supported by NSF, NIH, and DOE (NSF DBI-1338415). The contents of this publication are solely the responsibility of the authors and do not necessarily represent the official views of the NIH. MKG has an equity interest in, and is a co-founder and scientific advisor of, VeraChem LLC. REA has equity interest in and is a co-founder and scientific advisor of Actavalon, Inc.; VAF has equity interest in Actavalon, Inc. and PW has an equity interest in Relay Pharmaceuticals, Inc. In August 2016, VAF joined Schrodinger, Inc., and after that point was no longer involved in D3R activities. In June 2017, SL joined Silicon Therapeutics, Inc., and after that point was no longer involved in D3R activities.

Supplementary material

10822_2017_88_MOESM1_ESM.docx (50.1 mb)
Supplementary material 1 (DOCX 51288 KB)
10822_2017_88_MOESM2_ESM.pdf (5.6 mb)
Supplementary material 2 (PDF 5686 KB)
10822_2017_88_MOESM3_ESM.pdf (47 kb)
Supplementary material 3 (PDF 48 KB)


  1. 1.
    Sliwoski G, Kothiwale S, Meiler J, Lowe EW (2014) Computational methods in drug discovery. Pharmacol Rev 66:334–395. CrossRefGoogle Scholar
  2. 2.
    Amaro RE, Baron R, McCammon JA (2008) An improved relaxed complex scheme for receptor flexibility in computer-aided drug design. J Comput Aided Mol Des 22:693–705. CrossRefGoogle Scholar
  3. 3.
    Jorgensen WL (2004) The many roles of computation in drug discovery. Science 303:1813–1818. CrossRefGoogle Scholar
  4. 4.
    Carlson HA (2016) Lessons learned over four benchmark exercises from the community structure–activity resource. J Chem Inf Model 56:951–954. CrossRefGoogle Scholar
  5. 5.
    Carlson HA, Smith RD, Damm-Ganamet KL et al (2016) CSAR 2014: a benchmark exercise using unpublished data from pharma. J Chem Inf Model 56:1063–1077. CrossRefGoogle Scholar
  6. 6.
    Smith RD, Damm-Ganamet KL, Dunbar JB et al (2016) CSAR benchmark exercise 2013: evaluation of results from a combined computational protein design, docking, and scoring/ranking challenge. J Chem Inf Model 56:1022–1031. CrossRefGoogle Scholar
  7. 7.
    Damm-Ganamet KL, Smith RD, Dunbar JB et al (2013) CSAR benchmark exercise 2011–2012: evaluation of results from docking and relative ranking of blinded congeneric series. J Chem Inf Model 53:1853–1870. CrossRefGoogle Scholar
  8. 8.
    Smith RD, Dunbar JB, Ung PM-U et al (2011) CSAR Benchmark Exercise of 2010: Combined Evaluation Across All Submitted Scoring Functions. J Chem Inf Model 51:2115–2131. CrossRefGoogle Scholar
  9. 9.
    Gathiaka S, Liu S, Chiu M et al (2016) D3R grand challenge 2015: evaluation of protein–ligand pose and affinity predictions. J Comput Aided Mol Des 30:651–668. CrossRefGoogle Scholar
  10. 10.
    Makishima M, Okamoto AY, Repa JJ et al (1999) Identification of a nuclear receptor for bile acids. Science 284:1362–1365. CrossRefGoogle Scholar
  11. 11.
    Parks DJ, Blanchard SG, Bledsoe RK et al (1999) Bile acids: natural ligands for an orphan nuclear receptor. Science 284:1365–1368.. CrossRefGoogle Scholar
  12. 12.
    Wang H, Chen J, Hollister K et al (1999) Endogenous bile acids are ligands for the nuclear receptor FXR/BAR. Mol Cell 3:543–553. CrossRefGoogle Scholar
  13. 13.
    Lu TT, Makishima M, Repa JJ et al (2000) Molecular basis for feedback regulation of bile acid synthesis by nuclear receptors. Mol Cell 6:507–515. CrossRefGoogle Scholar
  14. 14.
    Gardès C, Blum D, Bleicher K et al (2011) Studies in mice, hamsters, and rats demonstrate that repression of hepatic apoA-I expression by taurocholic acid in mice is not mediated by the farnesoid-X-receptor. J Lipid Res 52:1188–1199. CrossRefGoogle Scholar
  15. 15.
    Richter HGF, Benson GM, Bleicher KH et al (2011) Optimization of a novel class of benzimidazole-based farnesoid X receptor (FXR) agonists to improve physicochemical and ADME properties. Bioorg Med Chem Lett 21:1134–1140. CrossRefGoogle Scholar
  16. 16.
    Richter HGF, Benson GM, Blum D et al (2011) Discovery of novel and orally active FXR agonists for the potential treatment of dyslipidemia & diabetes. Bioorg Med Chem Lett 21:191–194. CrossRefGoogle Scholar
  17. 17.
    Feng S, Yang M, Zhang Z et al (2009) Identification of an N-oxide pyridine GW4064 analog as a potent FXR agonist. Bioorg Med Chem Lett 19:2595–2598. CrossRefGoogle Scholar
  18. 18.
    Tembe BL, Mc Cammon JA (1984) Ligand-receptor interactions. Comput Chem 8:281–283. CrossRefGoogle Scholar
  19. 19.
    Nichols JS, Parks DJ, Consler TG, Blanchard SG (1998) Development of a scintillation proximity assay for peroxisome proliferator-activated receptor ligand binding domain. Anal Biochem 257:112–119. CrossRefGoogle Scholar
  20. 20.
    Warkentin M, Thorne RE (2007) A general method for hyperquenching protein crystals. J Struct Funct Genom 8:141–144. CrossRefGoogle Scholar
  21. 21.
    Otwinowski Z, Minor W (1997) Processing of X-ray diffraction data collected in oscillation mode. Methods Enzymol 276:307–326. CrossRefGoogle Scholar
  22. 22.
    Kabsch W (2010) XDS. Acta Crystallogr D 66:125–132. CrossRefGoogle Scholar
  23. 23.
    McCoy AJ, Grosse-Kunstleve RW, Adams PD et al (2007) Phaser crystallographic software. J Appl Crystallogr 40:658–674. CrossRefGoogle Scholar
  24. 24.
    Winn MD, Murshudov GN, Papiz MZ (2003) Macromolecular TLS refinement in REFMAC at moderate resolutions. Methods Enzymol 374:300–321. CrossRefGoogle Scholar
  25. 25.
    Blanc E, Roversi P, Vonrhein C et al (2004) Refinement of severely incomplete structures with maximum likelihood in BUSTER–TNT. Acta Crystallogr D 60:2210–2221. CrossRefGoogle Scholar
  26. 26.
    Krissinel E, Henrick K (2004) Secondary-structure matching (SSM), a new tool for fast protein structure alignment in three dimensions. Acta Crystallogr 60:2256–2268. Google Scholar
  27. 27.
    Small-molecule Drug Discovery Suite 2017-4 (2017) Schrödinger, LLC, New York, NYGoogle Scholar
  28. 28.
    Jacobson MP, Pincus DL, Rapp CS et al (2004) A hierarchical approach to all-atom protein loop prediction. Proteins Struct Funct Bioinform 55:351–367. CrossRefGoogle Scholar
  29. 29.
    Jacobson MP, Friesner RA, Xiang Z, Honig B (2002) On the role of the crystal environment in determining protein side-chain conformations. J Mol Biol 320:597–608. CrossRefGoogle Scholar
  30. 30.
    Yung-Chi C, Prusoff WH (1973) Relationship between the inhibition constant (KI) and the concentration of inhibitor which causes 50 per cent inhibition (I50) of an enzymatic reaction. Biochem Pharmacol 22:3099–3108. CrossRefGoogle Scholar
  31. 31.
    Ekins S, Bunin B (2013) The collaborative drug discovery (CDD) database. In: Kortagere S (ed) In silico models for drug discovery. Humana Press, New York, pp 139–154CrossRefGoogle Scholar
  32. 32.
    Viswanadhan VN, Ghose AK, Revankar GR, Robins RK (1989) Atomic physicochemical parameters for three dimensional structure directed quantitative structure-activity relationships. 4. Additional parameters for hydrophobic and dispersive interactions and their application for an automated superposition of certain naturally occurring nucleoside antibiotics. J Chem Inf Comput Sci 29:163–172. CrossRefGoogle Scholar
  33. 33.
    Duan J, Dixon SL, Lowrie JF, Sherman W (2010) Analysis and comparison of 2D fingerprints: insights into database screening performance using eight fingerprint methods. J Mol Graph Model 29:157–170. CrossRefGoogle Scholar
  34. 34.
    Sastry M, Lowrie JF, Dixon SL, Sherman W (2010) Large-scale systematic analysis of 2D fingerprint methods and parameters to improve virtual screening enrichments. J Chem Inf Model 50:771–784. CrossRefGoogle Scholar
  35. 35.
    Kuhn B, Tichý M, Wang L et al (2017) Prospective evaluation of free energy calculations for the prioritization of cathepsin L inhibitors. J Med Chem 60:2485–2497. CrossRefGoogle Scholar
  36. 36.
    Keränen H, Pérez-Benito L, Ciordia M et al (2017) Acylguanidine beta secretase 1 inhibitors: a combined experimental and free energy perturbation study. J Chem Theory Comput 13:1439–1453. CrossRefGoogle Scholar
  37. 37.
    Lenselink EB, Louvel J, Forti AF et al (2016) Predicting binding affinities for GPCR ligands using free-energy perturbation. ACS Omega 1:293–304. CrossRefGoogle Scholar
  38. 38.
    Abel R, Mondal S, Masse C et al (2017) Accelerating drug discovery through tight integration of expert molecular design and predictive scoring. Curr Opin Struct Biol 43:38–44. CrossRefGoogle Scholar
  39. 39.
    Chipot C (2014) Frontiers in free-energy calculations of biological systems. Wiley Interdiscip Rev 4:71–89. Google Scholar
  40. 40.
    Aldeghi M, Heifetz A, Bodkin MJ et al (2015) Accurate calculation of the absolute free energy of binding for drug molecules. Chem Sci 7:207–218. CrossRefGoogle Scholar
  41. 41.
    Aldeghi M, Heifetz A, Bodkin MJ et al (2017) Predictions of ligand selectivity from absolute binding free energy calculations. J Am Chem Soc 139:946–957. CrossRefGoogle Scholar
  42. 42.
    Mishra SK, Calabró G, Loeffler HH et al (2015) Evaluation of selected classical force fields for alchemical binding free energy calculations of protein-carbohydrate complexes. J Chem Theory Comput 11:3333–3345. CrossRefGoogle Scholar
  43. 43.
    Trott O, Olson AJ (2010) AutoDock Vina: Improving the speed and accuracy of docking with a new scoring function, efficient optimization, and multithreading. J Comput Chem 31:455–461. Google Scholar
  44. 44.
    Bannan CC, Burley KH, Chiu M et al (2016) Blind prediction of cyclohexane–water distribution coefficients from the SAMPL5 challenge. J Comput Aided Mol Des 30:927–944. CrossRefGoogle Scholar
  45. 45.
    Yin J, Henriksen NM, Slochower DR et al (2017) Overview of the SAMPL5 host–guest challenge: are we doing better?. J Comput Aided Mol Des 31:1–19. CrossRefGoogle Scholar
  46. 46.
    Muddana HS, Fenley AT, Mobley DL, Gilson MK (2014) The SAMPL4 host–guest blind prediction challenge: an overview. J Comput Aided Mol Des 28:305–317. CrossRefGoogle Scholar

Copyright information

© Springer International Publishing AG, part of Springer Nature 2017

Authors and Affiliations

  • Zied Gaieb
    • 1
  • Shuai Liu
    • 1
  • Symon Gathiaka
    • 2
  • Michael Chiu
    • 1
  • Huanwang Yang
    • 3
  • Chenghua Shao
    • 3
  • Victoria A. Feher
    • 1
  • W. Patrick Walters
    • 4
  • Bernd Kuhn
    • 5
  • Markus G. Rudolph
    • 5
  • Stephen K. Burley
    • 3
  • Michael K. Gilson
    • 1
    Email author
  • Rommie E. Amaro
    • 1
  1. 1.Drug Design Data ResourceUniversity of CaliforniaSan Diego, La JollaUSA
  2. 2.Merck & Co., Inc.BostonUSA
  3. 3.RCSB Protein Data Bank, Institute for Quantitative Biomedicine, Rutgers, The State University of New JerseyPiscatawayUSA
  4. 4.Relay TherapeuticsCambridgeUSA
  5. 5.Roche Pharmaceutical Research and Early Development (pRED), Roche Innovation Center BaselF. Hoffmann-La Roche Ltd.BaselSwitzerland

Personalised recommendations