Journal of Computer-Aided Molecular Design

, Volume 28, Issue 4, pp 305–317 | Cite as

The SAMPL4 host–guest blind prediction challenge: an overview

  • Hari S. Muddana
  • Andrew T. Fenley
  • David L. MobleyEmail author
  • Michael K. GilsonEmail author


Prospective validation of methods for computing binding affinities can help assess their predictive power and thus set reasonable expectations for their performance in drug design applications. Supramolecular host–guest systems are excellent model systems for testing such affinity prediction methods, because their small size and limited conformational flexibility, relative to proteins, allows higher throughput and better numerical convergence. The SAMPL4 prediction challenge therefore included a series of host–guest systems, based on two hosts, cucurbit[7]uril and octa-acid. Binding affinities in aqueous solution were measured experimentally for a total of 23 guest molecules. Participants submitted 35 sets of computational predictions for these host–guest systems, based on methods ranging from simple docking, to extensive free energy simulations, to quantum mechanical calculations. Over half of the predictions provided better correlations with experiment than two simple null models, but most methods underperformed the null models in terms of root mean squared error and linear regression slope. Interestingly, the overall performance across all SAMPL4 submissions was similar to that for the prior SAMPL3 host–guest challenge, although the experimentalists took steps to simplify the current challenge. While some methods performed fairly consistently across both hosts, no single approach emerged as consistent top performer, and the nonsystematic nature of the various submissions made it impossible to draw definitive conclusions regarding the best choices of energy models or sampling algorithms. Salt effects emerged as an issue in the calculation of absolute binding affinities of cucurbit[7]uril-guest systems, but were not expected to affect the relative affinities significantly. Useful directions for future rounds of the challenge might involve encouraging participants to carry out some calculations that replicate each others’ studies, and to systematically explore parameter options.


SAMPL4 Host-guest Cucurbit[7]uril Octa-acid Binding Prediction Blind challenge 



We thank OpenEye software for providing logistical support in the form of web hosting, technical support, and some financial support for the SAMPL4 workshop. We also thank Jay Ponder (Washington University) and Teng Lin (Schrödinger) for helpful discussions about analyzing absolute versus relative free energy predictions, and Vijay Pande for hosting the workshop at Stanford University. DLM acknowledges the financial support of the National Institutes of Health (1R15GM096257-01A1) and appreciates the support of the GreenPlanet computing facility at UC Irvine, supported in part by NSF CHE-0840513. MKG acknowledges funding from National Institute of General Medical Sciences (GM61300). The contents of this publication are solely the responsibility of the authors and do not necessarily represent the official views of the NIGMS, NIH, or NSF.

Supplementary material

10822_2014_9735_MOESM1_ESM.docx (1020 kb)
Supplementary material 1 (DOCX 1020 kb)


  1. 1.
    Kitchen DB, Decornez H, Furr JR, Bajorath J (2004) Docking and scoring in virtual screening for drug discovery: methods and applications. Nat Rev Drug Discov 3(11):935–949CrossRefGoogle Scholar
  2. 2.
    Warren GL, Andrews CW, Capelli A-M, Clarke B, LaLonde J, Lambert MH, Lindvall M, Nevins N, Semus SF, Senger S (2006) A critical assessment of docking programs and scoring functions. J Med Chem 49(20):5912–5931CrossRefGoogle Scholar
  3. 3.
    Gilson MK, Zhou H-X (2007) Calculation of protein-ligand binding affinities. Annu Rev Biophys Biomol Struct 36:21–42CrossRefGoogle Scholar
  4. 4.
    Damm-Ganamet KL, Smith RD, Dunbar JB Jr, Stuckey JA, Carlson HA (2013) CSAR benchmark exercise 2011–2012: evaluation of results from docking and relative ranking of blinded congeneric series. J Chem Inf Model 53(8):1853–1870CrossRefGoogle Scholar
  5. 5.
    Guthrie JP (2009) A blind challenge for computational solvation free energies: introduction and overview. J Phys Chem B 113(14):4501–4507CrossRefGoogle Scholar
  6. 6.
    Geballe MT, Skillman AG, Nicholls A, Guthrie JP, Taylor PJ (2010) The SAMPL2 blind prediction challenge: introduction and overview. J Comput Aided Mol Des 24(4):259–279CrossRefGoogle Scholar
  7. 7.
    Skillman AG (2012) SAMPL3: blinded prediction of host–guest binding affinities, hydration free energies, and trypsin inhibitors. J Comput Aided Mol Des 26(5):473–474CrossRefGoogle Scholar
  8. 8.
    Dunbar JB, Smith RD, Damm-Ganamet KL, Ahmed A, Esposito EX, Delproposto J, Chinnaswamy K, Kang Y-N, Kubish G, Gestwicki JE (2013) CSAR dataset release 2012: ligands, affinities, complexes, and docking decoys. J Chem Inf Model 53(8):1842–1852CrossRefGoogle Scholar
  9. 9.
    Kryshtafovych A, Monastyrskyy B, Fidelis K (2013) CASP prediction center infrastructure and evaluation measures in CASP10 and CASP ROLL. Proteins Struct Funct Bioinf 82(S2):7–13Google Scholar
  10. 10.
    Nielsen JE, Gunner M, García‐Moreno E (2011) The pKa cooperative: a collaborative effort to advance structure-based calculations of pKa values and electrostatic effects in proteins. Proteins Struct Funct Bioinf 79(12):3249–3259Google Scholar
  11. 11.
    Nicholls A, Mobley DL, Guthrie JP, Chodera JD, Bayly CI, Cooper MD, Pande VS (2008) Predicting small-molecule solvation free energies: an informal blind test for computational chemistry. J Med Chem 51(4):769–779CrossRefGoogle Scholar
  12. 12.
    Skillman AG, Geballe MT, Nicholls A (2010) SAMPL2 challenge: prediction of solvation energies and tautomer ratios. J Comput Aided Mol Des 24(4):257–258CrossRefGoogle Scholar
  13. 13.
    Muddana HS, Varnado CD, Bielawski CW, Urbach AR, Isaacs L, Geballe MT, Gilson MK (2012) Blind prediction of host–guest binding affinities: a new SAMPL3 challenge. J Comput Aided Mol Des 26(5):475–487CrossRefGoogle Scholar
  14. 14.
    Mobley DL, Wymer KL, Lim NM (2014) Blind prediction of solvation free energies from the SAMPL4 challenge. J Comput Aided Mol Des. doi: 10.1007/s10822-014-9718-2
  15. 15.
    Mobley DL, Liu S, Lim NM, Wymer KL, Perryman AL, Forli S, Deng N, Su J, Branson K, Olson A (2014) Blind prediction of HIV integrase binding from the SAMPL4 challenge. J Comput Aided Mol Des. doi: 10.1007/s10822-014-9723-5
  16. 16.
    Chen W, Chang C-E, Gilson MK (2004) Calculation of cyclodextrin binding affinities: energy, entropy, and implications for drug design. Biophys J 87(5):3035–3049CrossRefGoogle Scholar
  17. 17.
    Chang C-E, Gilson MK (2004) Free energy, entropy, and induced fit in host-guest recognition: calculations with the second-generation mining minima algorithm. J Am Chem Soc 126(40):13156–13164CrossRefGoogle Scholar
  18. 18.
    Rekharsky MV, Mori T, Yang C, Ko YH, Selvapalam N, Kim H, Sobransingh D, Kaifer AE, Liu S, Isaacs L (2007) A synthetic host-guest system achieves avidin-biotin affinity by overcoming enthalpy–entropy compensation. Proc Natl Acad Sci USA 104(52):20737–20742CrossRefGoogle Scholar
  19. 19.
    Muddana HS, Gilson MK (2012) Prediction of SAMPL3 host–guest binding affinities: evaluating the accuracy of generalized force-fields. J Comput Aided Mol Des 26(5):517–525CrossRefGoogle Scholar
  20. 20.
    Nguyen CN, Young TK, Gilson MK (2012) Grid inhomogeneous solvation theory: hydration structure and thermodynamics of the miniature receptor cucurbit[7]uril. J Chem Phys 137:044101CrossRefGoogle Scholar
  21. 21.
    Biedermann F, Uzunova VD, Scherman OA, Nau WM, De Simone A (2012) Release of high-energy water as an essential driving force for the high-affinity binding of cucurbit[n]urils. J Am Chem Soc 134(37):15318–15323CrossRefGoogle Scholar
  22. 22.
    Liu S, Ruspic C, Mukhopadhyay P, Chakrabarti S, Zavalij PY, Isaacs L (2005) The cucurbit[n]uril family: prime components for self-sorting systems. J Am Chem Soc 127(45):15959–15967CrossRefGoogle Scholar
  23. 23.
    Gibb CL, Gibb BC (2004) Well-defined, organic nanoenvironments in water: the hydrophobic effect drives a capsular assembly. J Am Chem Soc 126(37):11408–11409CrossRefGoogle Scholar
  24. 24.
    Cao L, Isaacs L (2014) Absolute and relative binding affinity of cucurbit[7]uril towards a series of cationic guests. Supramol Chem. doi: 10.1080/10610278.2013.852674
  25. 25.
    Gibb CL, Gibb BC (2014) Binding of cyclic carboxylates to octa-acid deep-cavity cavitand. J Comput Aided Mol Des. doi: 10.1007/s10822-013-9690-2
  26. 26.
    Wang J, Wolf RM, Caldwell JW, Kollman PA, Case DA (2004) Development and testing of a general amber force field. J Comput Chem 25(9):1157–1174CrossRefGoogle Scholar
  27. 27.
    Jakalian A, Bush BL, Jack DB, Bayly CI (2000) Fast, efficient generation of high-quality atomic charges. AM1-BCC model: I. Method. J Comput Chem 21(2):132–146CrossRefGoogle Scholar
  28. 28.
    Jakalian A, Jack DB, Bayly CI (2002) Fast, efficient generation of high-quality atomic charges. AM1-BCC model: II. Parameterization and validation. J Comput Chem 23(16):1623–1641CrossRefGoogle Scholar
  29. 29.
    Zheng L, Yang W (2012) Practically efficient and robust free energy calculations: double-integration orthogonal space tempering. J Chem Theory Comput 8(3):810–823CrossRefGoogle Scholar
  30. 30.
    Jorgensen WL, Chandrasekhar J, Madura JD, Impey RW, Klein ML (1983) Comparison of simple potential functions for simulating liquid water. J Chem Phys 79:926CrossRefGoogle Scholar
  31. 31.
    Sun Y, Kollman PA (1995) Hydrophobic solvation of methane and nonbond parameters of the TIP3P water model. J Comput Chem 16(9):1164–1169CrossRefGoogle Scholar
  32. 32.
    Bennett CH (1976) Efficient estimation of free energy differences from Monte Carlo data. J Comput Phys 22(2):245–268CrossRefGoogle Scholar
  33. 33.
    Ponder JW, Wu C, Ren P, Pande VS, Chodera JD, Schnieders MJ, Haque I, Mobley DL, Lambrecht DS, DiStasio RA Jr (2010) Current status of the AMOEBA polarizable force field. J Phys Chem B 114(8):2549–2564CrossRefGoogle Scholar
  34. 34.
    Ren P, Ponder JW (2003) Polarizable atomic multipole water model for molecular mechanics simulation. J Phys Chem B 107(24):5933–5947CrossRefGoogle Scholar
  35. 35.
    Grimme S (2012) Supramolecular binding thermodynamics by dispersion-corrected density functional theory. Chem Eur J 18(32):9955–9964CrossRefGoogle Scholar
  36. 36.
    Muddana H, Yin J, Sapra N, Fenley A, Gilson M (2014) Blind prediction of SAMPL4 cucurbit[7]uril binding affinities with the mining minima method. J Comput Aided Mol Des. doi: 10.1007/s10822-014-9726-2
  37. 37.
    Muddana HS, Gilson MK (2012) Calculation of host–guest binding affinities using a quantum-mechanical energy model. J Chem Theory Comput 8(6):2023–2033CrossRefGoogle Scholar
  38. 38.
    Hogues H, Sulea T, Purisima E (2014) Exhaustive docking and solvated interaction energy scoring: lessons learned from the SAMPL4 challenge. J Comput Aided Mol Des. doi: 10.1007/s10822-014-9715-5
  39. 39.
    Naïm M, Bhat S, Rankin KN, Dennis S, Chowdhury SF, Siddiqi I, Drabik P, Sulea T, Bayly CI, Jakalian A (2007) Solvated interaction energy (SIE) for scoring protein-ligand binding affinities. 1. Exploring the parameter space. J Chem Inf Model 47(1):122–133CrossRefGoogle Scholar
  40. 40.
    Corbeil CR, Sulea T, Purisima EO (2010) Rapid prediction of solvation free energy. 2. The first-shell hydration (FiSH) continuum model. J Chem Theory Comput 6(5):1622–1637CrossRefGoogle Scholar
  41. 41.
    Mikulskis P, Cioloboc D, Andrejić M, Khare S, Brorsson J, Genheden S, Mata RA, Söderhjelm P, Ryde U (2014) Free-energy perturbation and quantum mechanical study of SAMPL4 octa-acid host-guest binding energies. J Comput Aided Mol Des. doi: 10.1007/s10822-014-9739-x
  42. 42.
    Gallicchio E, Lapelosa M, Levy RM (2010) Binding energy distribution analysis method (BEDAM) for estimation of protein—ligand binding affinities. J Chem Theory Comput 6(9):2961–2977CrossRefGoogle Scholar
  43. 43.
    Jorgensen WL, Maxwell DS, Tirado-Rives J (1996) Development and testing of the OPLS all-atom force field on conformational energetics and properties of organic liquids. J Am Chem Soc 118(45):11225–11236CrossRefGoogle Scholar
  44. 44.
    Gallicchio E, Paris K, Levy RM (2009) The AGBNP2 implicit solvation model. J Chem Theory Comput 5(9):2544–2564CrossRefGoogle Scholar
  45. 45.
    Coleman RG, Sterling T, Weiss DR (2014) SAMPL4 & DOCK 3.7: lessons for automated docking procedures. J Comput Aided Mol Des. doi: 10.1007/s10822-014-9722-6
  46. 46.
    Kuntz I, Chen K, Sharp K, Kollman P (1999) The maximal affinity of ligands. Proc Natl Acad Sci USA 96(18):9997–10002CrossRefGoogle Scholar
  47. 47.
    Rogers KE, Ortiz-Sánchez JM, Baron R, Fajer M, de Oliveira CsAF, McCammon JA (2012) On the role of dewetting transitions in host–guest binding free energy calculations. J Chem Theory Comput 9(1):46–53CrossRefGoogle Scholar
  48. 48.
    Muddana HS, Sapra NV, Fenley AT, Gilson MK (2013) The electrostatic response of water to neutral polar solutes: implications for continuum solvent modeling. J Chem Phys 138:224504CrossRefGoogle Scholar
  49. 49.
    Horn HW, Swope WC, Pitera JW, Madura JD, Dick TJ, Hura GL, Head-Gordon T (2004) Development of an improved four-site water model for biomolecular simulations: TIP4P-Ew. J Chem Phys 120:9665CrossRefGoogle Scholar
  50. 50.
    Ong W, Kaifer AE (2004) Salt effects on the apparent stability of the cucurbit[7]uril-methyl viologen inclusion complex. J Org Chem 69(4):1383–1385CrossRefGoogle Scholar
  51. 51.
    Hsiao Y-W, Söderhjelm P (2014) Prediction of SAMPL4 host-guest binding affinities using funnel metadynamics. J Comput Aided Mol Des. doi: 10.1007/s10822-014-9724-4
  52. 52.
    Monroe JI, Shirts MR (2014) Converging free energies of binding in cucurbit[7]uril and octa-acid host-guest systems from SAMPL4 using expanded ensemble simulations. J Comput Aided Mol Des. doi: 10.1007/s10822-014-9716-4
  53. 53.
    Coleman RG, Carchia M, Sterling T, Irwin JJ, Shoichet BK (2013) Ligand pose and orientational sampling in molecular docking. PLoS ONE 8(10):e75992CrossRefGoogle Scholar
  54. 54.
    Zwanzig RW (1954) High-temperature equation of state by a perturbation method. I. Nonpolar gases. J Chem Phys 22:1420CrossRefGoogle Scholar
  55. 55.
    Kollman P (1993) Free energy calculations: applications to chemical and biochemical phenomena. Chem Rev 93(7):2395–2417CrossRefGoogle Scholar
  56. 56.
    Kumar S, Rosenberg JM, Bouzida D, Swendsen RH, Kollman PA (1992) The weighted histogram analysis method for free-energy calculations on biomolecules. I. The method. J Comput Chem 13(8):1011–1021CrossRefGoogle Scholar
  57. 57.
    Lyubartsev A, Martsinovski A, Shevkunov S, Vorontsov-Velyaminov P (1992) New approach to Monte Carlo calculation of the free energy: method of expanded ensembles. J Chem Phys 96:1776CrossRefGoogle Scholar
  58. 58.
    König G, Boresch S (2011) Non-Boltzmann sampling and Bennett’s acceptance ratio method: how to profit from bending the rules. J Comput Chem 32(6):1082–1090CrossRefGoogle Scholar
  59. 59.
    Hawkins G, Giesen D, Lynch G, Chambers C, Rossi I, Storer J, Li J, Zhu T, Thompson J, Winget P (2004) AMSOL-version 7.1. University of Minnesota, Minneapolis, based in part on AMPAC-version 2Google Scholar
  60. 60.
    Grimme S (2011) Density functional theory with London dispersion corrections. Wiley Interdiscip Rev Comput Mol Sci 1(2):211–228CrossRefGoogle Scholar
  61. 61.
    Sure R, Grimme S (2013) Corrected small basis set Hartree-Fock method for large systems. J Comput Chem 34(19):1672–1685CrossRefGoogle Scholar
  62. 62.
    Řezáč J, Fanfrlík J, Salahub D, Hobza P (2009) Semiempirical quantum chemical PM6 method augmented by dispersion and H-bonding correction terms reliably describes various types of noncovalent complexes. J Chem Theory Comput 5(7):1749–1760CrossRefGoogle Scholar
  63. 63.
    Korth M (2010) Third-generation hydrogen-bonding corrections for semiempirical QM methods and force fields. J Chem Theory Comput 6(12):3808–3816CrossRefGoogle Scholar
  64. 64.
    Momany FA, Rone R (1992) Validation of the general purpose QUANTA® 3.2/CHARMm® force field. J Comput Chem 13(7):888–900CrossRefGoogle Scholar
  65. 65.
    Gilson MK, Gilson HS, Potter MJ (2003) Fast assignment of accurate partial atomic charges: an electronegativity equalization method that accounts for alternate resonance forms. J Chem Inf Comput Sci 43(6):1982–1997CrossRefGoogle Scholar
  66. 66.
    Vanommeslaeghe K, Hatcher E, Acharya C, Kundu S, Zhong S, Shim J, Darian E, Guvench O, Lopes P, Vorobyov I (2010) CHARMM general force field: a force field for drug-like molecules compatible with the CHARMM all-atom additive biological force fields. J Comput Chem 31(4):671–690Google Scholar
  67. 67.
    Bayly CI, Cieplak P, Cornell W, Kollman PA (1993) A well-behaved electrostatic potential based method using charge restraints for deriving atomic charges: the RESP model. J Phys Chem 97(40):10269–10280CrossRefGoogle Scholar
  68. 68.
    Hampel C, Werner HJ (1996) Local treatment of electron correlation in coupled cluster theory. J Chem Phys 104:6286CrossRefGoogle Scholar
  69. 69.
    Purisima EO (1998) Fast summation boundary element method for calculating solvation free energies of macromolecules. J Comput Chem 19(13):1494–1504CrossRefGoogle Scholar
  70. 70.
    Klamt A (1995) Conductor-like screening model for real solvents: a new approach to the quantitative calculation of solvation phenomena. J Phys Chem 99(7):2224–2235CrossRefGoogle Scholar
  71. 71.
    Klamt A, Schüürmann G (1993) COSMO: a new approach to dielectric screening in solvents with explicit expressions for the screening energy and its gradient. J Chem Soc Perkin Trans 2(5):799–805CrossRefGoogle Scholar
  72. 72.
    Sitkoff D, Sharp KA, Honig B (1994) Accurate calculation of hydration free energies using macroscopic solvent models. J Phys Chem 98(7):1978–1988CrossRefGoogle Scholar
  73. 73.
    Chang CE, Gilson MK (2003) Tork: conformational analysis method for molecules and complexes. J Comput Chem 24(16):1987–1998CrossRefGoogle Scholar
  74. 74.
    Laio A, Parrinello M (2002) Escaping free-energy minima. Proc Natl Acad Sci USA 99(20):12562–12566CrossRefGoogle Scholar
  75. 75.
    Limongelli V, Bonomi M, Parrinello M (2013) Funnel metadynamics as accurate binding free-energy method. Proc Natl Acad Sci USA 110(16):6358–6363CrossRefGoogle Scholar
  76. 76.
    Moghaddam S, Yang C, Rekharsky M, Ko YH, Kim K, Inoue Y, Gilson MK (2011) New Ultrahigh Affinity Host–Guest Complexes of Cucurbit[7]uril with Bicyclo [2.2. 2] octane and Adamantane Guests: Thermodynamic Analysis and Evaluation of M2 Affinity Calculations. J Am Chem Soc 133(10):3570–3581CrossRefGoogle Scholar

Copyright information

© Springer International Publishing Switzerland 2014

Authors and Affiliations

  1. 1.Skaggs School of Pharmacy and Pharmaceutical SciencesUniversity of California San DiegoLa JollaUSA
  2. 2.Departments of Pharmaceutical Sciences and ChemistryUniversity of California IrvineIrvineUSA

Personalised recommendations