Journal of Computer-Aided Molecular Design

, Volume 30, Issue 3, pp 219–227 | Cite as

Incorporating specificity into optimization: evaluation of SPA using CSAR 2014 and CASF 2013 benchmarks

  • Zhiqiang Yan
  • Jin WangEmail author


Scoring functions of protein–ligand interactions are widely used in computationally docking software and structure-based drug discovery. Accurate prediction of the binding energy between the protein and the ligand is the main task of the scoring function. The accuracy of a scoring function is normally evaluated by testing it on the benchmarks of protein–ligand complexes. In this work, we report the evaluation analysis of an improved version of scoring function SPecificity and Affinity (SPA). By testing on two independent benchmarks Community Structure-Activity Resource (CSAR) 2014 and Comparative Assessment of Scoring Functions (CASF) 2013, the assessment shows that SPA is relatively more accurate than other compared scoring functions in predicting the interactions between the protein and the ligand. We conclude that the inclusion of the specificity in the optimization can effectively suppress the competitive state on the funnel-like binding energy landscape, and make SPA more accurate in identifying the “native” conformation and scoring the binding decoys. The evaluation of SPA highlights the importance of binding specificity in improving the accuracy of the scoring functions.


Scoring function Protein–ligand interaction Binding specificity Binding affinity SPA CASF CSAR 



This work was supported by National Natural Science Foundation of China (Grant nos. 21403208, 91227114, 11174105, 21190040, 91430217). The authors thank Computing Center of Jilin Province for computational support.

Supplementary material

10822_2016_9897_MOESM1_ESM.docx (235 kb)
Supplementary material 1 (docx 234 KB)


  1. 1.
    Dickson M, Gagnon JP (2004) Key factors in the rising cost of new drug discovery and development. Nat Rev Drug Discov 3:417–429CrossRefGoogle Scholar
  2. 2.
    Shoichet BK (2004) Virtual screening of chemical libraries. Nature 432:862–865CrossRefGoogle Scholar
  3. 3.
    Sousa S, Cerqueira N, Fernandes P, Ramos M (2010) Virtual screening in drug design and development. Comb Chem High Throughput Screen 13:442–453CrossRefGoogle Scholar
  4. 4.
    Cosconati S, Forli S, Perryman AL, Harris R, Goodsell DS et al (2010) Virtual screening with autodock: theory and practice. Expert Opin Drug Dis 5:597–607CrossRefGoogle Scholar
  5. 5.
    Bello M, Martínez-Archundia M, Correa-Basurto J (2013) Automated docking for novel drug discovery. Expert Opin Drug Dis 8:821–834CrossRefGoogle Scholar
  6. 6.
    Kitchen DB, Decornez H, Furr JR, Bajorath J (2004) Docking and scoring in virtual screening for drug discovery: methods and applications. Nat Rev Drug Discov 3:935–949CrossRefGoogle Scholar
  7. 7.
    Danishuddin M, Khan AU (2015) Structure based virtual screening to discover putative drug candidates: necessary considerations and successful case studies. Methods 71:135–145CrossRefGoogle Scholar
  8. 8.
    Huang SY, Grinter SZ, Zou X (2010) Scoring functions and their evaluation methods for protein–ligand docking: recent advances and future directions. Phys Chem Chem Phys 12:12899–12908CrossRefGoogle Scholar
  9. 9.
    Wang JC, Lin JH (2013) Scoring functions for prediction of protein–ligand interactions. Curr Pharm Des 19:2174–2182CrossRefGoogle Scholar
  10. 10.
    Liu J, Wang R (2015) Classification of current scoring functions. J Chem Inf Model 55:475–482CrossRefGoogle Scholar
  11. 11.
    Huang N, Shoichet BK, Irwin JJ (2006) Benchmarking sets for molecular docking. J Med Chem 49:6789–6801CrossRefGoogle Scholar
  12. 12.
    Smith RD, Dunbar JB Jr, Ung PMU, Esposito EX, Yang CY et al (2011) Csar benchmark exercise of 2010: combined evaluation across all submitted scoring functions. J Chem Inf Model 51:2115–2131CrossRefGoogle Scholar
  13. 13.
    Damm-Ganamet KL, Smith RD, Dunbar JB Jr, Stuckey JA, Carlson HA (2013) Csar benchmark exercise 2011–2012: evaluation of results from docking and relative ranking of blinded congeneric series. J Chem Inf Model 53:1853–1870CrossRefGoogle Scholar
  14. 14.
    Li Y, Han L, Liu Z, Wang R (2014) Comparative assessment of scoring functions on an updated benchmark: II. Evaluation methods and general results. J Chem Inf Model 54:1717–1736CrossRefGoogle Scholar
  15. 15.
    Janin J (1995) Principles of protein–protein recognition from structure to thermodynamics. Biochimie 77:497–505CrossRefGoogle Scholar
  16. 16.
    Wang J, Verkhivker GM (2003) Energy landscape theory, funnels, specificity, and optimal criterion of biomolecular binding. Phys Rev Lett 90:188101CrossRefGoogle Scholar
  17. 17.
    Wang J, Zheng X, Yang Y, Drueckhammer D, Yang W et al (2007) Quantifying intrinsic specificity: a potential complement to affinity in drug screening. Phys Rev Lett 99:198101CrossRefGoogle Scholar
  18. 18.
    Havranek JJ, Harbury PB (2003) Automated design of specificity in molecular recognition. Nat Struct Mol Biol 10:45–52CrossRefGoogle Scholar
  19. 19.
    Shifman J, Mayo S (2003) Exploring the origins of binding specificity through the computational redesign of calmodulin. Proc Natl Acad Sci USA 100:13274CrossRefGoogle Scholar
  20. 20.
    Kortemme T, Joachimiak LA, Bullock AN, Schuler AD, Stoddard BL et al (2004) Computational redesign of protein–protein interaction specificity. Nat Struct Mol Biol 11:371–379CrossRefGoogle Scholar
  21. 21.
    Bolon DN, Grant RA, Baker TA, Sauer RT (2005) Specificity versus stability in computational protein design. Proc Natl Acad Sci USA 102:12724–12729CrossRefGoogle Scholar
  22. 22.
    Ashworth J, Havranek J, Duarte C, Sussman D, Monnat R et al (2006) Computational redesign of endonuclease dna binding and cleavage specificity. Nature 441:656–659CrossRefGoogle Scholar
  23. 23.
    Grigoryan G, Reinke AW, Keating AE (2009) Design of protein-interaction specificity gives selective bzip-binding peptides. Nature 458:859–864CrossRefGoogle Scholar
  24. 24.
    Yan Z, Zheng X, Wang E, Wang J (2013) Thermodynamic and kinetic specificities of ligand binding. Chem Sci 4:2387–2395CrossRefGoogle Scholar
  25. 25.
    Zhang J, Zheng F, Grigoryan G (2014) Design and designability of protein-based assemblies. Curr Opin Struct Biol 27:79–86CrossRefGoogle Scholar
  26. 26.
    Yan Z, Wang J (2012) Specificity quantification of biomolecular recognition and its implication for drug discovery. Sci Rep 2:309Google Scholar
  27. 27.
    Yan Z, Guo L, Hu L, Wang J (2013) Specificity and affinity quantification of protein–protein interactions. Bioinformatics 29:1127–1133CrossRefGoogle Scholar
  28. 28.
    Yan Z, Wang J (2013) Optimizing scoring function of protein–nucleic acid interactions with both affinity and specificity. PloS ONE 8:e74443CrossRefGoogle Scholar
  29. 29.
    Yan Z, Wang J (2015) Optimizing the affinity and specificity of ligand binding with the inclusion of solvation effect. Proteins 83:1632–1642CrossRefGoogle Scholar
  30. 30.
    Bryngelson JD, Onuchic JN, Socci ND, Wolynes PG (1995) Funnels, pathways, and the energy landscape of protein folding: a synthesis. Proteins 21:167–195CrossRefGoogle Scholar
  31. 31.
    Janin J (1996) Quantifying biological specificity: the statistical mechanics of molecular recognition. Proteins 25:438–445CrossRefGoogle Scholar
  32. 32.
    Rejto PA, Verkhivker GM (1996) Unraveling principles of lead discovery: from unfrustrated energy landscapes to novel molecular anchors. Proc Natl Acad Sci USA 93:8945–8950CrossRefGoogle Scholar
  33. 33.
    Miller DW, Dill KA (2008) Ligand binding to proteins: the binding landscape model. Protein Sci 6:2166–2179CrossRefGoogle Scholar
  34. 34.
    Tsai CJ, Kumar S, Ma B, Nussinov R (1999) Folding funnels, binding funnels, and protein function. Protein Sci 8:1181–1190CrossRefGoogle Scholar
  35. 35.
    Dominy BN, Shakhnovich EI (2004) Native atom types for knowledge-based potentials: application to binding energy prediction. J Med Chem 47:4538–4558CrossRefGoogle Scholar
  36. 36.
    Liu Z, Dominy BN, Shakhnovich EI (2004) Structural mining: self-consistent design on flexible protein-peptide docking and transferable binding affinity potential. J Am Chem Soc 126:8515–8528CrossRefGoogle Scholar
  37. 37.
    Levy Y, Wolynes PG, Onuchic JN (2004) Protein topology determines binding mechanism. Proc Natl Acad Sci USA 101:511–516CrossRefGoogle Scholar
  38. 38.
    Koppensteiner W, Sippl MJ (1998) Knowledge-based potentials-back to the roots. Biochemistry 63:247–252Google Scholar
  39. 39.
    Shen Q, Xiong B, Zheng M, Luo X, Luo C et al (2010) Knowledge-based scoring functions in drug design: 2. Can the knowledge base be enriched? J Chem Inf Model 51:386–397CrossRefGoogle Scholar
  40. 40.
    Wang R, Fang X, Lu Y, Wang S (2004) The pdbbind database: collection of binding affinities for protein–ligand complexes with known three-dimensional structures. J Med Chem 47:2977–2980CrossRefGoogle Scholar
  41. 41.
    Wang R, Fang X, Lu Y, Yang CY, Wang S (2005) The pdbbind database: methodologies and updates. J Med Chem 48:4111–4119CrossRefGoogle Scholar
  42. 42.
    Liu Z, Li Y, Han L, Li J, Liu J et al (2015) PDB-wide collection of binding data: current status of the PDBbind database. Bioinformatics 31:405–412CrossRefGoogle Scholar
  43. 43.
    Liu Y, Zhao L, Li W, Zhao D, Song M et al (2013) Fipsdock: a new molecular docking technique driven by fully informed swarm optimization algorithm. J Comput Chem 34:67–75CrossRefGoogle Scholar
  44. 44.
    Guo L, Yan Z, Zheng X, Hu L, Yang Y et al (2014) A comparison of various optimization algorithms of protein–ligand docking programs by fitness accuracy. J Mol Model 20:1–10Google Scholar
  45. 45.
    Jiang L, Gao Y, Mao F, Liu Z, Lai L (2002) Potential of mean force for protein–protein interaction studies. Proteins 46:190–196CrossRefGoogle Scholar
  46. 46.
    Zhang C, Liu S, Zhu Q, Zhou Y (2005) A knowledge-based energy function for protein–ligand, protein–protein, and protein–DNA complexes. J Med Chem 48:2325–2335CrossRefGoogle Scholar
  47. 47.
    Liu Z, Mao F, Guo J, Yan B, Wang P et al (2005) Quantitative evaluation of protein–DNA interactions using an optimized knowledge-based potential. Nucleic Acids Res 33:546–558CrossRefGoogle Scholar
  48. 48.
    Su Y, Zhou A, Xia X, Li W, Sun Z (2009) Quantitative prediction of protein–protein binding affinity with a potential of mean force considering volume correction. Protein Sci 18:2550–2558CrossRefGoogle Scholar
  49. 49.
    Huang SY, Zou X (2014) A knowledge-based scoring function for protein–RNA interactions derived from a statistical mechanics-based iterative method. Nucleic Acids Res 42:e55–e55CrossRefGoogle Scholar
  50. 50.
    Li Y, Liu Z, Li J, Han L, Liu J et al (2014) Comparative assessment of scoring functions on an updated benchmark: I. Compilation of the test set. J Chem Inf Model 54:1700–1716CrossRefGoogle Scholar
  51. 51.
    OLBoyle NM, Banck M, James CA, Morley C et al (2011) Open babel: an open chemical toolbox. J Cheminf 3:33CrossRefGoogle Scholar
  52. 52.
    Shan Y, Kim ET, Eastwood MP, Dror RO, Seeliger MA et al (2011) How does a drug molecule find its target binding site? J Am Chem Soc 133:9181–9183CrossRefGoogle Scholar

Copyright information

© Springer International Publishing Switzerland 2016

Authors and Affiliations

  1. 1.State Key Laboratory of Electroanalytical Chemistry Changchun Institute of Applied ChemistryChinese Academy of SciencesChangchunChina
  2. 2.Department of Chemistry and PhysicsState University of New York at Stony BrookStony BrookUSA

Personalised recommendations