Scoring and lessons learned with the CSAR benchmark using an improved iterative knowledge-based scoring function

Based on a statistical mechanics-based iterative method, we have extracted a set of distance-dependent, all-atom pairwise potentials for protein-ligand interactions from the crystal structures of 1300 protein-ligand complexes. The iterative method circumvents the long-standing reference state problem in knowledge-based scoring functions. The resulted scoring function, referred to as ITScore 2.0, has been tested with the CSAR (Community Structure-Activity Resource, 2009 release) benchmark of 345 diverse protein-ligand complexes. ITScore 2.0 achieved a Pearson correlation of R(2) = 0.54 in binding affinity prediction. A comparative analysis has been done on the scoring performances of ITScore 2.0, the van der Waals (VDW) scoring function, the VDW with heavy atoms only, and the force field (FF) scoring function of DOCK which consists of a VDW term and an electrostatic term. The results reveal several important factors that affect the scoring performances, which could be helpful for the improvement of scoring functions.     

An iterative knowledge-based scoring function to predict protein-ligand interactions: II. Validation of the scoring function

We have developed an iterative knowledge-based scoring function (ITScore) to describe protein-ligand interactions. Here, we assess ITScore through extensive tests on native structure identification, binding affinity prediction, and virtual database screening. Specifically, ITScore was first applied to a test set of 100 protein-ligand complexes constructed by Wang et al. (J Med Chem 2003, 46, 2287), and compared with 14 other scoring functions. The results show that ITScore yielded a high success rate of 82% on identifying native-like binding modes under the criterion of rmsd < or = 2 A for each top-ranked ligand conformation. The success rate increased to 98% if the top five conformations were considered for each ligand. In the case of binding affinity prediction, ITScore also obtained a good correlation for this test set (R = 0.65). Next, ITScore was used to predict binding affinities of a second diverse test set of 77 protein-ligand complexes prepared by Muegge and Martin (J Med Chem 1999, 42, 791), and compared with four other widely used knowledge-based scoring functions. ITScore yielded a high correlation of R2 = 0.65 (or R = 0.81) in the affinity prediction. Finally, enrichment tests were performed with ITScore against four target proteins using the compound databases constructed by Jacobsson et al. (J Med Chem 2003, 46, 5781). The results were compared with those of eight other scoring functions. ITScore yielded high enrichments in all four database screening tests. ITScore can be easily combined with the existing docking programs for the use of structure-based drug design.     

Statistical potential for modeling and ranking of protein-ligand interactions

Applications in structural biology and medicinal chemistry require protein-ligand scoring functions for two distinct tasks: (i) ranking different poses of a small molecule in a protein binding site and (ii) ranking different small molecules by their complementarity to a protein site. Using probability theory, we developed two atomic distance-dependent statistical scoring functions: PoseScore was optimized for recognizing native binding geometries of ligands from other poses and RankScore was optimized for distinguishing ligands from nonbinding molecules. Both scores are based on a set of 8,885 crystallographic structures of protein-ligand complexes but differ in the values of three key parameters. Factors influencing the accuracy of scoring were investigated, including the maximal atomic distance and non-native ligand geometries used for scoring, as well as the use of protein models instead of crystallographic structures for training and testing the scoring function. For the test set of 19 targets, RankScore improved the ligand enrichment (logAUC) and early enrichment (EF(1)) scores computed by DOCK 3.6 for 13 and 14 targets, respectively. In addition, RankScore performed better at rescoring than each of seven other scoring functions tested. Accepting both the crystal structure and decoy geometries with all-atom root-mean-square errors of up to 2 ? from the crystal structure as correct binding poses, PoseScore gave the best score to a correct binding pose among 100 decoys for 88% of all cases in a benchmark set containing 100 protein-ligand complexes. PoseScore accuracy is comparable to that of DrugScore(CSD) and ITScore/SE and superior to 12 other tested scoring functions. Therefore, RankScore can facilitate ligand discovery, by ranking complexes of the target with different small molecules; PoseScore can be used for protein-ligand complex structure prediction, by ranking different conformations of a given protein-ligand pair. The statistical potentials are available through the Integrative Modeling Platform (IMP) software package (http://salilab.org/imp) and the LigScore Web server (http://salilab.org/ligscore/).     

DSX: a knowledge-based scoring function for the assessment of protein-ligand complexes

We introduce the new knowledge-based scoring function DSX that consists of distance-dependent pair potentials, novel torsion angle potentials, and newly defined solvent accessible surface-dependent potentials. DSX pair potentials are based on the statistical formalism of DrugScore, extended by a much more specialized set of atom types. The original DrugScore-like reference state is rather unstable with respect to modifications in the used atom types. Therefore, an important method to overcome this problem and to allow for robust results when deriving pair potentials for arbitrary sets of atom types is presented. A validation based on a carefully prepared test set is shown, enabling direct comparison to the majority of other popular scoring functions. Here, DSX features superior performance with respect to docking- and ranking power and runtime requirements. Furthermore, the beneficial combination with torsion angle-dependent and desolvation-dependent potentials is demonstrated. DSX is robust, flexible, and capable of working together with special features of popular docking engines, e.g., flexible protein residues in AutoDock or GOLD. The program is freely available to the scientific community and can be downloaded from our Web site www.agklebe.de .     

An improved PMF scoring function for universally predicting the interactions of a ligand with protein, DNA, and RNA

An improved potential mean force (PMF) scoring function, named KScore, has been developed by using 23 redefined ligand atom types and 17 protein atom types, as well as 28 newly introduced atom types for nucleic acids (DNA and RNA). Metal ions and water molecules embedded in the binding sites of receptors are considered explicitly by two newly defined atom types. The individual potential terms were devised on the basis of the high-resolution crystal and NMR structures of 2,422 protein-ligand complexes, 300 DNA-ligand complexes, and 97 RNA-ligand complexes. The optimized atom pairwise distances and minima of the potentials overcome some of the disadvantages and ambiguities of current PMF potentials; thus, they more reasonably explain the atomic interaction between receptors and ligands. KScore was validated against five test sets of protein-ligand complexes and two sets of nucleic-acid-ligand complexes. The results showed acceptable correlations between KScore scores and experimentally determined binding affinities (log K i's or binding free energies). In particular, KScore can be used to rank the binding of ligands with metalloproteins; the linear correlation coefficient ( R) for the test set is 0.65. In addition to reasonably ranking protein-ligand interactions, KScore also yielded good results for scoring DNA/RNA--ligand interactions; the linear correlation coefficients for DNA-ligand and RNA-ligand complexes are 0.68 and 0.81, respectively. Moreover, KScore can appropriately reproduce the experimental structures of ligand-receptor complexes. Thus, KScore is an appropriate scoring function for universally ranking the interactions of ligands with protein, DNA, and RNA.     

DrugScoreRNA--knowledge-based scoring function to predict RNA-ligand interactions

There is growing interest in RNA as a drug target due to its widespread involvement in biological processes. To exploit the power of structure-based drug-design approaches, novel scoring and docking tools need to be developed that can efficiently and reliably predict binding modes and binding affinities of RNA ligands. We report for the first time the development of a knowledge-based scoring function to predict RNA-ligand interactions (DrugScoreRNA). Based on the formalism of the DrugScore approach, distance-dependent pair potentials are derived from 670 crystallographically determined nucleic acid-ligand and -protein complexes. These potentials display quantitative differences compared to those of DrugScore (derived from protein-ligand complexes) and DrugScoreCSD (derived from small-molecule crystal data). When used as an objective function for docking 31 RNA-ligand complexes, DrugScoreRNA generates "good" binding geometries (rmsd (root mean-square deviation) < 2 A) in 42% of all cases on the first scoring rank. This is an improvement of 44% to 120% when compared to DrugScore, DrugScoreCSD, and an RNA-adapted AutoDock scoring function. Encouragingly, good docking results are also obtained for a subset of 20 NMR structures not contained in the knowledge-base to derive the potentials. This clearly demonstrates the robustness of the potentials. Binding free energy landscapes generated by DrugScoreRNA show a pronounced funnel shape in almost 3/4 of all cases, indicating the reduced steepness of the knowledge-based potentials. Docking with DrugScoreRNA can thus be expected to converge fast to the global minimum. Finally, binding affinities were predicted for 15 RNA-ligand complexes with DrugScoreRNA. A fair correlation between experimental and computed values is found (RS = 0.61), which suffices to distinguish weak from strong binders, as is required in virtual screening applications. DrugScoreRNA again shows superior predictive power when compared to DrugScore, DrugScoreCSD, and an RNA-adapted AutoDock scoring function.     

Knowledge-based scoring functions in drug design: 2. Can the knowledge base be enriched?

Fast and accurate predicting of the binding affinities of large sets of diverse protein?ligand complexes is an important, yet extremely challenging, task in drug discovery. The development of knowledge-based scoring functions exploiting structural information of known protein?ligand complexes represents a valuable contribution to such a computational prediction. In this study, we report a scoring function named IPMF that integrates additional experimental binding affinity information into the extracted potentials, on the assumption that a scoring function with the "enriched" knowledge base may achieve increased accuracy in binding affinity prediction. In our approach, the functions and atom types of PMF04 were inherited to implicitly capture binding effects that are hard to model explicitly, and a novel iteration device was designed to gradually tailor the initial potentials. We evaluated the performance of the resultant IPMF with a diverse set of 219 protein-ligand complexes and compared it with seven scoring functions commonly used in computer-aided drug design, including GLIDE, AutoDock4, VINA, PLP, LUDI, PMF, and PMF04. While the IPMF is only moderately successful in ranking native or near native conformations, it yields the lowest mean error of 1.41 log K(i)/K(d) units from measured inhibition affinities and the highest Pearson's correlation coefficient of R(p)2 0.40 for the test set. These results corroborate our initial supposition about the role of "enriched" knowledge base. With the rapid growing volume of high-quality structural and interaction data in the public domain, this work marks a positive step toward improving the accuracy of knowledge-based scoring functions in binding affinity prediction.     

Predicting protein-ligand binding affinities using novel geometrical descriptors and machine-learning methods

Inspired by the concept of knowledge-based scoring functions, a new quantitative structure-activity relationship (QSAR) approach is introduced for scoring protein-ligand interactions. This approach considers that the strength of ligand binding is correlated with the nature of specific ligand/binding site atom pairs in a distance-dependent manner. In this technique, atom pair occurrence and distance-dependent atom pair features are used to generate an interaction score. Scoring and pattern recognition results obtained using Kernel PLS (partial least squares) modeling and a genetic algorithm-based feature selection method are discussed.     

Specific empirical free energy function for automated docking of carbohydrates to proteins

We present an automated docking protocol specifically optimized to predict the structure and affinity of a protein-carbohydrate complex. A scoring function was developed based on a training set of 30 protein-carbohydrate complexes of known structure and affinity. Combinations of several models for hydrogen bonding, torsional entropy loss, and solvation were tested for their ability to fit the training set data, and the best model was used with AutoDock. The electrostatic empirical coefficient is larger than in a previously obtained model using a training set comprised of various types of protein-ligand complexes, indicating that electrostatic interactions play a more important role in determining the affinity between a carbohydrate and a protein. The differences in the relative weighting of the empirical coefficients in the model yields predicted free energies for the training set with a standard error of 1.403 kcal/mol. The new scoring function was tested on 17 Aspergillus niger glucoamylase inhibitors for which binding energies had been determined experimentally. Free energies of complex formation were predicted with a residual standard error of 1.101 kcal/mol. The new scoring function therefore provides a robust method for predicting free energies of formation and optimal conformations of carbohydrate-protein complexes.     

Ligand Identification Scoring Algorithm (LISA)

A central problem in de novo drug design is determining the binding affinity of a ligand with a receptor. A new scoring algorithm is presented that estimates the binding affinity of a protein-ligand complex given a three-dimensional structure. The method, LISA (Ligand Identification Scoring Algorithm), uses an empirical scoring function to describe the binding free energy. Interaction terms have been designed to account for van der Waals (VDW) contacts, hydrogen bonding, desolvation effects, and metal chelation to model the dissociation equilibrium constants using a linear model. Atom types have been introduced to differentiate the parameters for VDW, H-bonding interactions, and metal chelation between different atom pairs. A training set of 492 protein-ligand complexes was selected for the fitting process. Different test sets have been examined to evaluate its ability to predict experimentally measured binding affinities. By comparing with other well-known scoring functions, the results show that LISA has advantages over many existing scoring functions in simulating protein-ligand binding affinity, especially metalloprotein-ligand binding affinity. Artificial Neural Network (ANN) was also used in order to demonstrate that the energy terms in LISA are well designed and do not require extra cross terms.     

Information theory-based scoring function for the structure-based prediction of protein-ligand binding affinity

The development and validation of a new knowledge based scoring function (SIScoreJE) to predict binding energy between proteins and ligands is presented. SIScoreJE efficiently predicts the binding energy between a small molecule and its protein receptor. Protein-ligand atomic contact information was derived from a Non-Redundant Data set (NRD) of over 3000 X-ray crystal structures of protein-ligand complexes. This information was classified for individual "atom contact pairs" (ACP) which is used to calculate the atomic contact preferences. In addition to the two schemes generated in this study we have assessed a number of other common atom-type classification schemes. The preferences were calculated using an information theoretic relationship of joint entropy. Among 18 different atom-type classification schemes "ScoreJE Atom Type set2" (SATs2) was found to be the most suitable for our approach. To test the sensitivity of the method to the inclusion of solvent, Single-body Solvation Potentials (SSP) were also derived from the atomic contacts between the protein atom types and water molecules modeled using AQUARIUS2. Validation was carried out using an evaluation data set of 100 protein-ligand complexes with known binding energies to test the ability of the scoring functions to reproduce known binding affinities. In summary, it was found that a combined SSP/ScoreJE (SIScoreJE) performed significantly better than ScoreJE alone, and SIScoreJE and ScoreJE performed better than GOLD::GoldScore, GOLD::ChemScore, and XScore.     

Ligand-specific scoring functions: improved ranking of docking solutions

Molecular docking is a powerful computational method that has been widely used in many biomolecular studies to predict geometry of a protein-ligand complex. However, while its conformational search algorithms are usually able to generate correct conformation of a ligand in the binding site, the scoring methods often fail to discriminate it among many false variants. We propose to treat this problem by applying more precise ligand-specific scoring filters to re-rank docking solutions. In this way specific features of interactions between protein and different types of compounds can be implicitly taken into account. New scoring functions were constructed including hydrogen bonds, hydrophobic and hydrophilic complementarity terms. These scoring functions also discriminate ligands by the size of the molecule, the total hydrophobicity, and the number of peptide bonds for peptide ligands. Weighting coefficients of the scoring functions were adjusted using a training set of 60 protein-ligand complexes. The proposed method was then tested on the results of docking obtained for an additional 70 complexes. In both cases the success rate was 5-8% better compared to the standard functions implemented in popular docking software.     

Bootstrap-based consensus scoring method for protein-ligand docking

To improve the performance of a single scoring function used in a protein-ligand docking program, we developed a bootstrap-based consensus scoring (BBCS) method, which is based on ensemble learning. BBCS combines multiple scorings, each of which has the same function form but different energy-parameter sets. These multiple energy-parameter sets are generated in two steps: (1) generation of training sets by a bootstrap method and (2) optimization of energy-parameter set by a Z-score approach, which is based on energy landscape theory as used in protein folding, against each training set. In this study, we applied BBCS to the FlexX scoring function. Using given 50 complexes, we generated 100 training sets and obtained 100 optimized energy-parameter sets. These parameter sets were tested against 48 complexes different from the training sets. BBCS was shown to be an improvement over single scoring when using a parameter set optimized by the same Z-score approach. Comparing BBCS with the original FlexX scoring function, we found that (1) the success rate of recognizing the crystal structure at the top relative to decoys increased from 33.3% to 52.1% and that (2) the rank of the crystal structure improved for 54.2% of the complexes and worsened for none. We also found that BBCS performed better than conventional consensus scoring (CS).     

Lessons learned from participating in D3R 2016 Grand Challenge 2: compounds targeting the farnesoid X receptor

D3R 2016 Grand Challenge 2 focused on predictions of binding modes and affinities for 102 compounds against the farnesoid X receptor (FXR). In this challenge, two distinct methods, a docking-based method and a template-based method, were employed by our team for the binding mode prediction. For the new template-based method, 3D ligand similarities were calculated for each query compound against the ligands in the co-crystal structures of FXR available in Protein Data Bank. The binding mode was predicted based on the co-crystal protein structure containing the ligand with the best ligand similarity score against the query compound. For the FXR dataset, the template-based method achieved a better performance than the docking-based method on the binding mode prediction. For the binding affinity prediction, an in-house knowledge-based scoring function ITScore2 and MM/PBSA approach were employed. Good performance was achieved for MM/PBSA, whereas the performance of ITScore2 was sensitive to ligand composition, e.g. the percentage of carbon atoms in the compounds. The sensitivity to ligand composition could be a clue for the further improvement of our knowledge-based scoring function.     

Further development and validation of empirical scoring functions for structure-based binding affinity prediction

New empirical scoring functions have been developed to estimate the binding affinity of a given protein-ligand complex with known three-dimensional structure. These scoring functions include terms accounting for van der Waals interaction, hydrogen bonding, deformation penalty, and hydrophobic effect. A special feature is that three different algorithms have been implemented to calculate the hydrophobic effect term, which results in three parallel scoring functions. All three scoring functions are calibrated through multivariate regression analysis of a set of 200 protein-ligand complexes and they reproduce the binding free energies of the entire training set with standard deviations of 2.2 kcal/mol, 2.1 kcal/mol, and 2.0 kcal/mol, respectively. These three scoring functions are further combined into a consensus scoring function, X-CSCORE. When tested on an independent set of 30 protein-ligand complexes, X-CSCORE is able to predict their binding free energies with a standard deviation of 2.2 kcal/mol. The potential application of X-CSCORE to molecular docking is also investigated. Our results show that this consensus scoring function improves the docking accuracy considerably when compared to the conventional force field computation used for molecular docking.     

A scoring function for docking ligands to low-resolution protein structures

We present a docking method that uses a scoring function for protein-ligand docking that is designed to maximize the docking success rate for low-resolution protein structures. We find that the resulting scoring function parameters are very different depending on whether they were optimized for high- or low-resolution protein structures. We show that this docking method can be successfully applied to predict the ligand-binding site of low-resolution structures. For a set of 25 protein-ligand complexes, in 76% of the cases, more than 50% of ligand-contacting residues are correctly predicted (using receptor crystal structures where the binding site is unspecified). Using decoys of the receptor structures having a 4 A RMSD from the native structure, for the same set of complexes, in 72% of the cases, we obtain at least one correctly predicted ligand-contacting residue. Furthermore, using an 81-protein-ligand set described by Jain, in 76 (93.8%) cases, the algorithm correctly predicts more than 50% of the ligand-contacting residues when native protein structures are used. Using 3 A RMSD from native decoys, in all but two cases (97.5%), the algorithm predicts at least one ligand-binding residue correctly. Finally, compared to the previously published Dolores method, for 298 protein-ligand pairs, the number of cases in which at least half of the specific contacts are correctly predicted is more than four times greater.     

AIScore chemically diverse empirical scoring function employing quantum chemical binding energies of hydrogen-bonded complexes

In this work we report on a novel scoring function that is based on the LUDI model and focuses on the prediction of binding affinities. AIScore extends the original FlexX scoring function using a chemically diverse set of hydrogen-bonded interactions derived from extensive quantum chemical ab initio calculations. Furthermore, we introduce an algorithmic extension for the treatment of multifurcated hydrogen bonds (XFurcate). Charged and resonance-assisted hydrogen bond energies and hydrophobic interactions as well as a scaling factor for implicit solvation were fitted to experimental data. To this end, we assembled a set of 101 protein-ligand complexes with known experimental binding affinities. Tightly bound water molecules in the active site were considered to be an integral part of the binding pocket. Compared to the original FlexX scoring function, AIScore significantly improves the prediction of the binding free energies of the complexes in their native crystal structures. In combination with XFurcate, AIScore yields a Pearson correlation coefficient of R P = 0.87 on the training set. In a validation run on the PDBbind test set we achieved an R P value of 0.46 for 799 attractively scored complexes, compared to a value of R P = 0.17 and 739 bound complexes obtained with the FlexX original scoring function. The redocking capability of AIScore, on the other hand, does not fully reach the good performance of the original FlexX scoring function. This finding suggests that AIScore should rather be used for postscoring in combination with the standard FlexX incremental ligand construction scheme.     

Role of binding entropy in the refinement of protein-ligand docking predictions: analysis based on the use of 11 scoring functions

We present results of testing the ability of eleven popular scoring functions to predict native docked positions using a recently developed method (Ruvinsky and Kozintsev, J Comput Chem 2005, 26, 1089) for estimation the entropy contributions of relative motions to protein-ligand binding affinity. The method is based on the integration of the configurational integral over clusters obtained from multiple docked positions. We use a test set of 100 PDB protein-ligand complexes and ensembles of 101 docked positions generated by (Wang et al. J Med Chem 2003, 46, 2287) for each ligand in the test set. To test the suggested method we compared the averaged root-mean square deviations (RMSD) of the top-scored ligand docked positions, accounting and not accounting for entropy contributions, relative to the experimentally determined positions. We demonstrate that the method increases docking accuracy by 10-21% when used in conjunction with the AutoDock scoring function, by 2-25% with G-Score, by 7-41% with D-Score, by 0-8% with LigScore, by 1-6% with PLP, by 0-12% with LUDI, by 2-8% with F-Score, by 7-29% with ChemScore, by 0-9% with X-Score, by 2-19% with PMF, and by 1-7% with DrugScore. We also compared the performance of the suggested method with the method based on ranking by cluster occupancy only. We analyze how the choice of a clustering-RMSD and a low bound of dense clusters impacts on docking accuracy of the scoring methods. We derive optimal intervals of the clustering-RMSD for 11 scoring functions.     

BLEEP—potential of mean force describing protein–ligand interactions: I. Generating potential

We have developed BLEEP (biomolecular ligand energy evaluation protocol), an atomic level potential of mean force (PMF) describing protein–ligand interactions. The pair potentials for BLEEP have been derived from high-resolution X-ray structures of protein–ligand complexes in the Brookhaven Protein Data Bank (PDB), with a careful treatment of homology. The use of a broad variety of protein–ligand structures in the derivation phase gives BLEEP more general applicability than previous potentials, which have been based on limited classes of complexes, and thus represents a significant step forward. We calculate the distance distributions in protein–ligand interactions for all 820 possible pairs that can be chosen from our set of 40 different atom types, including polar hydrogen. We then use a reverse Boltzmann methodology to convert these into energy-like pair potential functions. Two versions of BLEEP are calculated, one including and one excluding interactions between protein and water. The pair potentials are found to have the expected forms; polar and hydrogen bonding interactions show minima at short range, around 3.0 Å, whereas a typical hydrophobic interaction is repulsive at this distance, with values above 4.0 Å being preferred. ©1999 John Wiley & Sons, Inc. J Comput Chem 20: 1165–1176, 1999     

Improving binding mode and binding affinity predictions of docking by ligand-based search of protein conformations: evaluation in D3R grand challenge 2015

The growing number of protein–ligand complex structures, particularly the structures of proteins co-bound with different ligands, in the Protein Data Bank helps us tackle two major challenges in molecular docking studies: the protein flexibility and the scoring function. Here, we introduced a systematic strategy by using the information embedded in the known protein–ligand complex structures to improve both binding mode and binding affinity predictions. Specifically, a ligand similarity calculation method was employed to search a receptor structure with a bound ligand sharing high similarity with the query ligand for the docking use. The strategy was applied to the two datasets (HSP90 and MAP4K4) in recent D3R Grand Challenge 2015. In addition, for the HSP90 dataset, a system-specific scoring function (ITScore2_hsp90) was generated by recalibrating our statistical potential-based scoring function (ITScore2) using the known protein–ligand complex structures and the statistical mechanics-based iterative method. For the HSP90 dataset, better performances were achieved for both binding mode and binding affinity predictions comparing with the original ITScore2 and with ensemble docking. For the MAP4K4 dataset, although there were only eight known protein–ligand complex structures, our docking strategy achieved a comparable performance with ensemble docking. Our method for receptor conformational selection and iterative method for the development of system-specific statistical potential-based scoring functions can be easily applied to other protein targets that have a number of protein–ligand complex structures available to improve predictions on binding.