Bayesian analysis and inference from QSAR predictive model results

QSAR models have been under development for decades but acceptance and utilization of model results have been slow, in part, because there is no widely accepted metric for assessing their reliability. We reapply a method commonly used in quantitative epidemiology and medical decision-making for evaluating the results of screening tests to assess reliability of a QSAR model. It quantifies the accuracy (expressed as sensitivity and specificity) of QSAR models as conditional probabilities of correct and incorrect classification of chemical characteristic, given a true characteristic. Using Bayes formula, these conditional probabilities are combined with prior information to generate a posterior distribution to determine the probability a specific chemical has a particular characteristic, given a model prediction. As an example, we apply this approach to evaluate the predictive reliability of a CATABOL model and base on it a "ready" and "not ready" biodegradability classification. Finally, we show how predictive capability of the model can be improved by sequential use of two models, the first one with high sensitivity and the second with high specificity.     

Rational selection of training and test sets for the development of validated QSAR models

Quantitative Structure–Activity Relationship (QSAR) models are used increasingly to screen chemical databases and/or virtual chemical libraries for potentially bioactive molecules. These developments emphasize the importance of rigorous model validation to ensure that the models have acceptable predictive power. Using k nearest neighbors (kNN) variable selection QSAR method for the analysis of several datasets, we have demonstrated recently that the widely accepted leave-one-out (LOO) cross-validated R² (q²) is an inadequate characteristic to assess the predictive ability of the models [Golbraikh, A., Tropsha, A. Beware of q2! J. Mol. Graphics Mod. 20, 269-276, (2002)]. Herein, we provide additional evidence that there exists no correlation between the values of q ² for the training set and accuracy of prediction (R ²) for the test set and argue that this observation is a general property of any QSAR model developed with LOO cross-validation. We suggest that external validation using rationally selected training and test sets provides a means to establish a reliable QSAR model. We propose several approaches to the division of experimental datasets into training and test sets and apply them in QSAR studies of 48 functionalized amino acid anticonvulsants and a series of 157 epipodophyllotoxin derivatives with antitumor activity. We formulate a set of general criteria for the evaluation of predictive power of QSAR models.     

Experimental determination and prediction of bilitranslocase transport activity

The transport activity of a membrane protein, bilitranslocase (T.C. # 2.A.65.1.1), which acts as a transporter of bilirubin from blood to liver cells, was experimentally determined for a large set of various endogenous compounds, drugs, purine and pyrimidine derivatives. On these grounds, the structure-activity models were developed following the OECD principles of QSAR models and their predictive ability for new chemicals was evaluated. The applicability domain of the models was estimated by Euclidean distances criteria according to the applied modeling method. The selection of the most influential structural variables was an important stage in the adopted modeling methodology. The interpretation of selected variables was performed in order to get an insight into the mechanism of transport through the cell membrane via bilitranslocase. Validation of the optimized models was performed by a previously determined validation set. The classification model was build to separate active from inactive compounds. The resulting accuracy, sensitivity, and specificity were 0.73, 0.89, and 0.64, respectively. Only active compounds were used to develop a predictive model for bilitranslocase inhibition constants. The model showed good predictive ability; Root Mean Squared error of the validation set, RMS(V)=0.29 log units.     

Local and global quantitative structure-activity relationship modeling and prediction for the baseline toxicity

The predictive accuracy of the model is of the most concern for computational chemists in quantitative structure-activity relationship (QSAR) investigations. It is hypothesized that the model based on analogical chemicals will exhibit better predictive performance than that derived from diverse compounds. This paper develops a novel scheme called "clustering first, and then modeling" to build local QSAR models for the subsets resulted from clustering of the training set according to structural similarity. For validation and prediction, the validation set and test set were first classified into the corresponding subsets just as those of the training set, and then the prediction was performed by the relevant local model for each subset. This approach was validated on two independent data sets by local modeling and prediction of the baseline toxicity for the fathead minnow. In this process, hierarchical clustering was employed for cluster analysis, k-nearest neighbor for classification, and partial least squares for the model generation. The statistical results indicated that the predictive performances of the local models based on the subsets were much superior to those of the global model based on the whole training set, which was consistent with the hypothesis. This approach proposed here is promising for extension to QSAR modeling for various physicochemical properties, biological activities, and toxicities.