Evaluating docking programs: keeping the playing field level



Over recent years many enrichment studies have been published which purport to rigorously compare the performance of two or more docking protocols. It has become clear however that such studies often have flaws within their methodologies, which cast doubt on the rigour of the conclusions. Setting up such comparisons is fraught with difficulties and no best mode of practice is available to guide the experimenter. Careful choice of structural models and ligands appropriate to those models is important. The protein structure should be representative for the target. In addition the set of active ligands selected should be appropriate to the structure in cases where different forms of the protein bind different classes of ligand. Binding site definition is also an area in which errors arise. Particular care is needed in deciding which crystallographic waters to retain and again this may be predicated by knowledge of the likely binding modes of the ligands making up the active ligand list. Geometric integrity of the ligand structures used is clearly important yet it is apparent that published sets of actives + decoys may contain sometimes high proportions of incorrect structures. Choice of protocol for docking and analysis needs careful consideration as many programs can be tweaked for optimum performance. Should studies be run using ‘black box’ protocols supplied by the software provider? Lastly, the correct method of analysis of enrichment studies is a much discussed topic at the moment. However currently promoted approaches do not consider a crucial aspect of a successful virtual screen, namely that a good structural diversity of hits be returned. Overall there is much to consider in the experimental design of enrichment studies. Hopefully this study will be of benefit in helping others plan such experiments.


Docking Enrichment Factor Xa GOLD Protein–ligand interaction Thrombin Virtual screening 



Cambridge Crystallographic Data Centre


Research collaboratory for structural bioinformatics


Protein data bank


Cyclooxygenase 2


Oestrogen receptor


Secretory phospholipase 2


Root mean square deviation


Virtual screening


Absorption, distribution, metabolism, excretion


Received operating characteristic


Area under curve



The author thanks Willem Nissink and Noel O’Boyle for assistance in setting up the docking studies presented here, Hongming Chen of AstraZeneca for supplying structure files and protocols for these studies and Richard Sykes for the GoldMine program used to carry out the analysis. Robin Taylor and Jason Cole are thanked for making perceptive and useful comments on the manuscript, and Simon Bowden, Marcel Verdonk, Chris Murray and Paul Mortensen are thanked for useful discussions.


  1. 1.
    Cole JC, Murray CW, Nissink JWM, Taylor RD, Taylor R (2005) Proteins 60:325CrossRefGoogle Scholar
  2. 2.
    Kellenberger E, Rodrigo J, Muller P, Rognan D (2004) Proteins 57:225CrossRefGoogle Scholar
  3. 3.
    Perola E, Walters WP, Charifson P (2007) J Chem Inf Model 47:251CrossRefGoogle Scholar
  4. 4.
    Kontoyianni M, McClellan LM, Sokol GS (2004) J Med Chem 47:558CrossRefGoogle Scholar
  5. 5.
    Warren GL, Andrews CW, Capelli A-M, Clarke B, LaLonde J, Lamber MH, Lindvall M, Nevins N, Semus S, Senger S, Tedesco G, Wall ID, Woolven JM, Peishoff CE, Head MS (2006) J Med Chem 49:5912CrossRefGoogle Scholar
  6. 6.
    Chen H, Lyne PD, Giordanetto F, Lovell T, Li J (2006) J Chem Inf Model 46:401CrossRefGoogle Scholar
  7. 7.
    Perola E, Walters SP, Charifson PS (2004) Proteins 56:235CrossRefGoogle Scholar
  8. 8.
    Berman HM, Westbrook J, Feng Z, Gilliland G, Bhat TN, Weissig I, Shindyalov IN, Bourne PE (2000) Nucleic Acids Res 28:235CrossRefGoogle Scholar
  9. 9.
    Jones G, Willett P, Glen R, Leach AR, Taylor R (1997) J Mol Biol 267:727CrossRefGoogle Scholar
  10. 10.
    Kramer B, Rarey M, Lengauer T (1999) Proteins 37:22CrossRefGoogle Scholar
  11. 11.
    Nissink JWM, Murray CW, Hartshorn MJ, Verdonk ML, Cole JC, Taylor R (2002) Proteins 49:457CrossRefGoogle Scholar
  12. 12.
    Hartshorn MJ, Verdonk ML, Chessari G, Brewerton SC, Mooij WTM, Mortensen PN, Murray CW (2007) J Med Chem 50:726CrossRefGoogle Scholar
  13. 13.
    Friesner RA, Murphy RB, Repasky MP, Frye LL, Greenwood JR, Halgren TA, Sanschagrin PC, Mainz DT (2006) J Med Chem 49:6177CrossRefGoogle Scholar
  14. 14.
    Hendlich M, Bergner A, Günther J, Klebe K (2003) J Mol Biol 326:607CrossRefGoogle Scholar
  15. 15.
    Pike ACW, Brzozowski AM, Walton J, Hubbard RE, Thorsell A-G, Li Y-L, Gustafsson J-A, Carlqusit M (2001) Structure 9(2):145CrossRefGoogle Scholar
  16. 16.
    Schulz-Gasch TA, Stahl M (2003) J Mol Model 9:47Google Scholar
  17. 17.
    Verdonk ML, Chessari G, Cole JC, Hartshorn M, Murray CW, Nissink JWM, Taylor RD, Taylor R (2007) J Med Chem 50:726CrossRefGoogle Scholar
  18. 18.
    Tucker TJ, Brady SF, Lumma WC, Lewis SD, Gardell SJ, Naylor-Olsen AM, Yan Y, Sisko JT, Stauffer KJ, Lucas BJ, Lynch JJ, Cook JJ, Stranieri MT, Holahan MA, Lyle EA, Baskin EP, Chen I-W, Dancheck KB, Krueger JA, Cooper CM, Vacca JP (1998) J Med Chem 41:3210CrossRefGoogle Scholar
  19. 19.
    Verdonk ML, Chessari G, Cole JC, Hartshorn MC, Murray CW, Nissink JWM, Taylor RD, Taylor R (2005) J Med Chem 48:6504CrossRefGoogle Scholar
  20. 20.
    Onodera K, Satou K, Hirota H. (2007) J Chem Inf Model 47:1609CrossRefGoogle Scholar
  21. 21.
    Taken from I. Dramburg et al from http://cisrg.shef.ac.uk/shef2004/abstracts.htm
  22. 22.
    Huang N, Shoichet BK, Irwin JJ (2006) J Med Chem 49(23):6789CrossRefGoogle Scholar
  23. 23.
    Communicated in a seminar by M. Nevins, 234th ACS National Meeting, COMP 150Google Scholar
  24. 24.
    Truchon J-F, Bayly CI (2007) J Chem Inf Model 47:488CrossRefGoogle Scholar
  25. 25.
    Sheridan RP, Singh SB, Fluder EM, Kearsley SJ (2001) J Chem Inf Comput Sci 41:1395CrossRefGoogle Scholar
  26. 26.
    Barker EJ, Gardiner EJ, Gillet VJ, Kitts P, Morris J (2003) J Chem Inf Comput Sci 43:346CrossRefGoogle Scholar
  27. 27.
    Good AC (2007) 234th ACS meeting 2007, Abs 266Google Scholar
  28. 28.
    Olah M, Mracec M, Ostopovici L, Rad R, Bora A, Hadaruga N, Olah I, Banda M, Simon Z, Mracec M, Oprea TI (2004) In: Oprea TI (eds) Chemoinformatics in drug discovery. Wiley-VCH, New York, p 223Google Scholar

Copyright information

© Springer Science+Business Media B.V. 2008

Authors and Affiliations

  1. 1.Cambridge Crystallographic Data CentreCambridgeUK

Personalised recommendations