Evaluation and application of multiple scoring functions for a virtual screening experiment

In order to identify novel chemical classes of factor Xa inhibitors, five scoring functions (FlexX, DOCK, GOLD, ChemScore and PMF) were engaged to evaluate the multiple docking poses generated by FlexX. The compound collection was composed of confirmed potent factor Xa inhibitors and a subset of the LeadQuest screening compound library. Except for PMF the other four scoring functions succeeded in reproducing the crystal complex (PDB code: 1FAX). During virtual screening the highest hit rate (80%) was demonstrated by FlexX at an energy cutoff of -40 kJ/mol, which is about 40-fold over random screening (2.06%). Limited results suggest that presenting more poses of a single molecule to the scoring functions could deteriorate their enrichment factors. A series of promising scaffolds with favorable binding scores was retrieved from LeadQuest. Consensus scoring by pair-wise intersection failed to enrich the hit rate yielded by single scorings (i.e. FlexX). We note that reported successes of consensus scoring in hit rate enrichment could be artificial because their comparisons were based on a selected subset of single scoring and a markedly reduced subset of double or triple scoring. The findings presented in this report are based upon a single biological system and support further studies.     

Assessing the discriminatory power of scoring functions for virtual screening

The efficiency of scoring functions for hit identification is usually quantified in terms of enrichment factors and enrichment curves. Close inspection of simulated and real score distributions from virtual screening, however, suggests that 'analysis of variance' (ANOVA) is a more reliable method for assessing their performance. Using ANOVA to quantify the discriminatory power of scoring functions with respect to ligands, decoys, and a reproducible reference database has the potential to facilitate the advancement of scoring functions significantly.     

Structure-based virtual screening with supervised consensus scoring: evaluation of pose prediction and enrichment factors

Since the evaluation of ligand conformations is a crucial aspect of structure-based virtual screening, scoring functions play significant roles in it. However, it is known that a scoring function does not always work well for all target proteins. When one cannot know which scoring function works best against a target protein a priori, there is no standard scoring method to know it even if 3D structure of a target protein-ligand complex is available. Therefore, development of the method to achieve high enrichments from given scoring functions and 3D structure of protein-ligand complex is a crucial and challenging task. To address this problem, we applied SCS (supervised consensus scoring), which employs a rough linear correlation between the binding free energy and the root-mean-square deviation (rmsd) of a native ligand conformations and incorporates protein-ligand binding process with docked ligand conformations using supervised learning, to virtual screening. We evaluated both the docking poses and enrichments of SCS and five scoring functions (F-Score, G-Score, D-Score, ChemScore, and PMF) for three different target proteins: thymidine kinase (TK), thrombin (thrombin), and peroxisome proliferator-activated receptor gamma (PPARgamma). Our enrichment studies show that SCS is competitive or superior to a best single scoring function at the top ranks of screened database. We found that the enrichments of SCS could be limited by a best scoring function, because SCS is obtained on the basis of the five individual scoring functions. Therefore, it is concluded that SCS works very successfully from our results. Moreover, from docking pose analysis, we revealed the connection between enrichment and average centroid distance of top-scored docking poses. Since SCS requires only one 3D structure of protein-ligand complex, SCS will be useful for identifying new ligands.     

Supervised consensus scoring for docking and virtual screening

Docking programs are widely used to discover novel ligands efficiently and can predict protein-ligand complex structures with reasonable accuracy and speed. However, there is an emerging demand for better performance from the scoring methods. Consensus scoring (CS) methods improve the performance by compensating for the deficiencies of each scoring function. However, conventional CS and existing scoring functions have the same problems, such as a lack of protein flexibility, inadequate treatment of salvation, and the simplistic nature of the energy function used. Although there are many problems in current scoring functions, we focus our attention on the incorporation of unbound ligand conformations. To address this problem, we propose supervised consensus scoring (SCS), which takes into account protein-ligand binding process using unbound ligand conformations with supervised learning. An evaluation of docking accuracy for 100 diverse protein-ligand complexes shows that SCS outperforms both CS and 11 scoring functions (PLP, F-Score, LigScore, DrugScore, LUDI, X-Score, AutoDock, PMF, G-Score, ChemScore, and D-score). The success rates of SCS range from 89% to 91% in the range of rmsd < 2 A, while those of CS range from 80% to 85%, and those of the scoring functions range from 26% to 76%. Moreover, we also introduce a method for judging whether a compound is active or inactive with the appropriate criterion for virtual screening. SCS performs quite well in docking accuracy and is presumably useful for screening large-scale compound databases before predicting binding affinity.     

Consensus scoring criteria for improving enrichment in virtual screening

MOTIVATION: Virtual screening of molecular compound libraries is a potentially powerful and inexpensive method for the discovery of novel lead compounds for drug development. The major weakness of virtual screening-the inability to consistently identify true positives (leads)-is likely due to our incomplete understanding of the chemistry involved in ligand binding and the subsequently imprecise scoring algorithms. It has been demonstrated that combining multiple scoring functions (consensus scoring) improves the enrichment of true positives. Previous efforts at consensus scoring have largely focused on empirical results, but they have yet to provide a theoretical analysis that gives insight into real features of combinations and data fusion for virtual screening. RESULTS: We demonstrate that combining multiple scoring functions improves the enrichment of true positives only if (a) each of the individual scoring functions has relatively high performance and (b) the individual scoring functions are distinctive. Notably, these two prediction variables are previously established criteria for the performance of data fusion approaches using either rank or score combinations. This work, thus, establishes a potential theoretical basis for the probable success of data fusion approaches to improve yields in in silico screening experiments. Furthermore, it is similarly established that the second criterion (b) can, in at least some cases, be functionally defined as the area between the rank versus score plots generated by the two (or more) algorithms. Because rank-score plots are independent of the performance of the individual scoring function, this establishes a second theoretically defined approach to determining the likely success of combining data from different predictive algorithms. This approach is, thus, useful in practical settings in the virtual screening process when the performance of at least two individual scoring functions (such as in criterion a) can be estimated as having a high likelihood of having high performance, even if no training sets are available. We provide initial validation of this theoretical approach using data from five scoring systems with two evolutionary docking algorithms on four targets, thymidine kinase, human dihydrofolate reductase, and estrogen receptors of antagonists and agonists. Our procedure is computationally efficient, able to adapt to different situations, and scalable to a large number of compounds as well as to a greater number of combinations. Results of the experiment show a fairly significant improvement (vs single algorithms) in several measures of scoring quality, specifically "goodness-of-hit" scores, false positive rates, and "enrichment". This approach (available online at http://gemdock.life. nctu.edu.tw/dock/download.php) has practical utility for cases where the basic tools are known or believed to be generally applicable, but where specific training sets are absent.     

Impact of scoring functions on enrichment in docking-based virtual screening: an application study on renin inhibitors

The docking program LigandFit/Cerius(2) has been used to perform shape-based virtual screening of databases against the aspartic protease renin, a target of determined three-dimensional structure. The protein structure was used in the induced fit binding conformation that occurs when renin is bound to the highly active renin inhibitor 1 (IC(50) = 2 nM). The scoring was calculated using several different scoring functions in order to get insight into the predictability of the magnitude of binding interactions. A database of 1000 diverse and druglike compounds, comprised of 990 members of a virtual database generated by using the iLib diverse software and 10 known active renin inhibitors, was docked flexibly and scored to determine appropriate scoring functions. All seven scoring functions used (LigScore1, LigScore2, PLP1, PLP2, JAIN, PMF, LUDI) were able to retrieve at least 50% of the active compounds within the first 20% (200 molecules) of the entire test database. A hit rate of 90% in the top 1.4% resulted using the quadruple consensus scoring of LigScore2, PLP1, PLP2, and JAIN. Additionally, a focused database was created with the iLib diverse software and used for the same procedure as the test database. Docking and scoring of the 990 focused compounds and the 10 known actives were performed. A hit rate of 100% in the top 8.4% resulted with use of the triple consensus scoring of PLP1, PLP2, and PMF. As expected, a ranking of the known active compounds within the focused database compared to the test database was observed. Adequate virtual screening conditions were derived empirically. They can be used for proximate docking and scoring application of compounds with putative renin inhibiting potency.     

Evaluation of binary QSAR models derived from LUDI and MOE scoring functions for structure based virtual screening

In today's world of high-throughput in silico screening, the development of virtual screening methodologies to prioritize small molecules as new chemical entities (NCEs) for synthesis is of current interest. Among several approaches to virtual screening, structure-based virtual screening has been considered the most effective. However the problems associated with the ranking of potential solutions in terms of scoring functions remains one of the major bottlenecks in structure-based virtual screening technology. It has been suggested that scoring functions may be used as filters for distinguishing binders from nonbinders instead of accurately predicting their binding free energies. Subsequently, several improvements have been made in this area, which include the use of multiple rather than single scoring functions and application of either consensus or multivariate statistical methods or both to improve the discrimination between binders and nonbinders. In view of it, the discriminative ability (distinguishing binders from nonbinders) of binary QSAR models derived using LUDI and MOE scoring functions has been compared with the models derived by Jacobbsson et al. on five data sets viz. estrogen receptor alphamimics (ERalpha_mimics), estrogen receptor alphatoxins (ERalpha_toxins), matrix metalloprotease 3 inhibitors (MMP-3), factor Xa inhibitors (fXa), and acetylcholine esterase inhibitors (AChE). The overall analyses reveal that binary QSAR is comparable to the PLS discriminant analysis, rule-based, and Bayesian classification methods used by Jacobsson et al. Further the scoring functions implemented in LUDI and MOE can score a wide range of protein-ligand interactions and are comparable to the scoring functions implemented in ICM and Cscore. Thus the binary QSAR models derived using LUDI and MOE scoring functions may be useful as a preliminary screening layer in a multilayered virtual screening paradigm.     

Cheminformatics meets molecular mechanics: a combined application of knowledge-based pose scoring and physical force field-based hit scoring functions improves the accuracy of structure-based virtual screening

Poor performance of scoring functions is a well-known bottleneck in structure-based virtual screening (VS), which is most frequently manifested in the scoring functions' inability to discriminate between true ligands vs known nonbinders (therefore designated as binding decoys). This deficiency leads to a large number of false positive hits resulting from VS. We have hypothesized that filtering out or penalizing docking poses recognized as non-native (i.e., pose decoys) should improve the performance of VS in terms of improved identification of true binders. Using several concepts from the field of cheminformatics, we have developed a novel approach to identifying pose decoys from an ensemble of poses generated by computational docking procedures. We demonstrate that the use of target-specific pose (scoring) filter in combination with a physical force field-based scoring function (MedusaScore) leads to significant improvement of hit rates in VS studies for 12 of the 13 benchmark sets from the clustered version of the Database of Useful Decoys (DUD). This new hybrid scoring function outperforms several conventional structure-based scoring functions, including XSCORE::HMSCORE, ChemScore, PLP, and Chemgauss3, in 6 out of 13 data sets at early stage of VS (up 1% decoys of the screening database). We compare our hybrid method with several novel VS methods that were recently reported to have good performances on the same DUD data sets. We find that the retrieved ligands using our method are chemically more diverse in comparison with two ligand-based methods (FieldScreen and FLAP::LBX). We also compare our method with FLAP::RBLB, a high-performance VS method that also utilizes both the receptor and the cognate ligand structures. Interestingly, we find that the top ligands retrieved using our method are highly complementary to those retrieved using FLAP::RBLB, hinting effective directions for best VS applications. We suggest that this integrative VS approach combining cheminformatics and molecular mechanics methodologies may be applied to a broad variety of protein targets to improve the outcome of structure-based drug discovery studies.     

FRED pose prediction and virtual screening accuracy

Results of a previous docking study are reanalyzed and extended to include results from the docking program FRED and a detailed statistical analysis of both structure reproduction and virtual screening results. FRED is run both in a traditional docking mode and in a hybrid mode that makes use of the structure of a bound ligand in addition to the protein structure to screen molecules. This analysis shows that most docking programs are effective overall but highly inconsistent, tending to do well on one system and poorly on the next. Comparing methods, the difference in mean performance on DUD is found to be statistically significant (95% confidence) 61% of the time when using a global enrichment metric (AUC). Early enrichment metrics are found to have relatively poor statistical power, with 0.5% early enrichment only able to distinguish methods to 95% confidence 14% of the time.     

A neural network based virtual high throughput screening test for the prediction of CNS activity

A virtual high throughput screening test to identify potentially CNS-active drugs has been developed. Discrimination was based on the knowledge available in databases containing CNS-active (Cipsline from Prous Science) and inactive compounds (Chemical Directory from Sigma-Aldrich). Molecular structures were represented using 2D Unit y fingerprints and a feedforward neural network was trained to classify molecules regarding their CNS activity. The parameterized network was validated by reclassification of the training set elements, by the classification of a test set preselected from the Prous database, and also by the prediction of activity for known CNS drugs not used in the training set but available in the Medchem database (Daylight). These tests revealed that our neural net recognized at least 89% of CNS-active compounds and would be suitable for use in our virtual screening protocol.     

Homology model-based virtual screening for GPCR ligands using docking and target-biased scoring

The current study investigates the combination of two recently reported techniques for the improvement of homology model-based virtual screening for G-protein coupled receptor (GPCR) ligands. First, ligand-supported homology modeling was used to generate receptor models that were in agreement with mutagenesis data and structure-activity relationship information of the ligands. Second, interaction patterns from known ligands to the receptor were applied for scoring and rank ordering compounds from a virtual library using ligand-receptor interaction fingerprint-based similarity (IFS). Our approach was evaluated in retrospective virtual screening experiments for antagonists of the metabotropic glutamate receptor (mGluR) subtype 5. The results of our approach were compared to the results obtained by conventional scoring functions (Dock-Score, PMF-Score, Gold-Score, ChemScore, and FlexX-Score). The IFS lead to significantly higher enrichment rates, relative to the competing scoring functions. Though using a target-biased scoring approach, the results were not biased toward the chemical classes of the reference structures. Our results indicate that the presented approach has the potential to serve as a general setup for successful structure-based GPCR virtual screening.     

New scoring functions for virtual screening from molecular dynamics simulations with a quantum-refined force-field (QRFF-MD). Application to cyclin-dependent kinase 2

A recently introduced new methodology based on ultrashort (50-100 ps) molecular dynamics simulations with a quantum-refined force-field (QRFF-MD) is here evaluated in its ability both to predict protein-ligand binding affinities and to discriminate active compounds from inactive ones. Physically based scoring functions are derived from this approach, and their performance is compared to that of several standard knowledge-based scoring functions. About 40 inhibitors of cyclin-dependent kinase 2 (CDK2) representing a broad chemical diversity were considered. The QRFF-MD method achieves a correlation coefficient, R(2), of 0.55, which is significantly better than that obtained by a number of traditional approaches in virtual screening but only slightly better than that obtained by consensus scoring (R(2) = 0.50). Compounds from the Available Chemical Directory, along with the known active compounds, were docked into the ATP binding site of CDK2 using the program Glide, and the 650 ligands from the top scored poses were considered for a QRFF-MD analysis. Combined with structural information extracted from the simulations, the QRFF-MD methodology results in similar enrichment of known actives compared to consensus scoring. Moreover, a new scoring function is introduced that combines a QRFF-MD based scoring function with consensus scoring, which results in substantial improvement on the enrichment profile.     

Physics-based scoring of protein-ligand complexes: enrichment of known inhibitors in large-scale virtual screening

We demonstrate that using an all-atom molecular mechanics force field combined with an implicit solvent model for scoring protein-ligand complexes is a promising approach for improving inhibitor enrichment in the virtual screening of large compound databases. The rescoring method is evaluated by the extent to which known binders for nine diverse, therapeutically relevant enzymes are enriched against a background of approximately 100,000 drug-like decoys. The improvement in enrichment is most robust and dramatic within the top 1% of the ranked database, that is, the first thousand compounds; below the first few percent of the ranked database, there is little overall improvement. The improved early enrichment is likely due to the more realistic treatment of ligand and receptor desolvation in the rescoring procedure. We also present anecdotal but encouraging results assessing the ability of the rescoring method to predict specificity of inhibitors for structurally related proteins.     

Evaluation of docking performance in a blinded virtual screening of fragment-like trypsin inhibitors

In this study, we have "blindly" assessed the ability of several combinations of docking software and scoring functions to predict the binding of a fragment-like library of bovine trypsine inhibitors. The most suitable protocols (involving Gold software and GoldScore scoring function, with or without rescoring) were selected for this purpose using a training set of compounds with known biological activities. The selected virtual screening protocols provided good results with the SAMPL3-VS dataset, showing enrichment factors of about 10 for Top 20 compounds. This methodology should be useful in difficult cases of docking, with a special emphasis on the fragment-based virtual screening campaigns.     

Structure based virtual screening of the Ebola virus trimeric glycoprotein using consensus scoring

Ebola virus (EBOV) causes zoonotic viral infection with a potential risk of global spread and a highly fatal effect on humans. Till date, no drug has gotten market approval for the treatment of Ebola virus disease (EVD), and this perhaps allows the use of both experimental and computational approaches in the antiviral drug discovery process. The main target of potential vaccines that are recently undergoing clinical trials is trimeric glycoprotein (GP) of the EBOV and its exact crystal structure was used in this structure based virtual screening study, with the aid of consensus scoring to select three possible hit compounds from about 36 million compounds in MCULE’s database. Amongst these three compounds, (5R)-5-[[5-(4-chlorophenyl)-1,2,4-oxadiazol-3-yl]methyl]-N-[(4-methoxyphenyl)methyl]-4,5-dihydroisoxazole-3-carboxamide (SC-2, C₂₁H₁₉ClN₄O₄) showed good features with respect to drug likeness, ligand efficiency metrics, solubility, absorption and distribution properties and non-carcinogenicity to emerge as the most promising compound that can be optimized to lead compound against the GP EBOV. The binding mode showed that SC-2 is well embedded within the trimeric chains of the GP EBOV with molecular interactions with some amino acids. The SC-2 hit compound, upon its optimization to lead, might be a good potential candidate with efficacy against the EBOV pathogen and subsequently receive necessary approval to be used as antiviral drug for the treatment of EVD.     

Further development and validation of empirical scoring functions for structure-based binding affinity prediction

New empirical scoring functions have been developed to estimate the binding affinity of a given protein-ligand complex with known three-dimensional structure. These scoring functions include terms accounting for van der Waals interaction, hydrogen bonding, deformation penalty, and hydrophobic effect. A special feature is that three different algorithms have been implemented to calculate the hydrophobic effect term, which results in three parallel scoring functions. All three scoring functions are calibrated through multivariate regression analysis of a set of 200 protein-ligand complexes and they reproduce the binding free energies of the entire training set with standard deviations of 2.2 kcal/mol, 2.1 kcal/mol, and 2.0 kcal/mol, respectively. These three scoring functions are further combined into a consensus scoring function, X-CSCORE. When tested on an independent set of 30 protein-ligand complexes, X-CSCORE is able to predict their binding free energies with a standard deviation of 2.2 kcal/mol. The potential application of X-CSCORE to molecular docking is also investigated. Our results show that this consensus scoring function improves the docking accuracy considerably when compared to the conventional force field computation used for molecular docking.     

Reducing false-positive rates in virtual screening via cancellation of systematic errors in the scoring function

Here we propose an over-the-hood docking method that compensates for systematic errors in the docking force fields. This method explicitly estimates the interaction energy of the ligand with the protein surface and uses it as a baseline to estimate the actual binding energy in the active site. It improves the accuracy of virtual screening in the LeadFinder package by up to 48%.