Evaluating virtual screening methods: good and bad metrics for the "early recognition" problem

Many metrics are currently used to evaluate the performance of ranking methods in virtual screening (VS), for instance, the area under the receiver operating characteristic curve (ROC), the area under the accumulation curve (AUAC), the average rank of actives, the enrichment factor (EF), and the robust initial enhancement (RIE) proposed by Sheridan et al. In this work, we show that the ROC, the AUAC, and the average rank metrics have the same inappropriate behaviors that make them poor metrics for comparing VS methods whose purpose is to rank actives early in an ordered list (the "early recognition problem"). In doing so, we derive mathematical formulas that relate those metrics together. Moreover, we show that the EF metric is not sensitive to ranking performance before and after the cutoff. Instead, we formally generalize the ROC metric to the early recognition problem which leads us to propose a novel metric called the Boltzmann-enhanced discrimination of receiver operating characteristic that turns out to contain the discrimination power of the RIE metric but incorporates the statistical significance from ROC and its well-behaved boundaries. Finally, two major sources of errors, namely, the statistical error and the "saturation effects", are examined. This leads to practical recommendations for the number of actives, the number of inactives, and the "early recognition" importance parameter that one should use when comparing ranking methods. Although this work is applied specifically to VS, it is general and can be used to analyze any method that needs to segregate actives toward the front of a rank-ordered list.     

"Venetian blinds" mechanism of solvation/desolvation in palladium(II) wheel-and-axle organic-inorganic diols

A family of organic-inorganic wheel-and-axle diols (Pd(LOH)(2)Cl(2), Pd(LOH)(2)(CH(3))Cl, Pd(LOH)(2)(CH(3)COO)(2), LOH = alpha-(4-pyridyl)benzhydrol) and several corresponding solvates are synthesized and characterized by single-crystal X-ray diffraction analysis. Their structures are compared to investigate the factors governing the modes of solid state association, the propensity to clathration, and the structural basis of guest inclusion. In all the complexes, the palladium coordination is a slightly distorted square. The LOH ligands coordinate Pd(2+) by means of the 4-pyridyl ring. In the chloride complexes solvation occurs with a 1:2 host/guest ratio by hydrogen bonding between the terminal -OH groups of the complex diol and one acceptor atom on the guest, and it is further assisted by guest stacking between host aryl rings. All solvates are organized in layers with practically invariant metrics, while the layers may be assembled in different arrangements. The structures of the nonsolvate compounds are related to the metrics of the solvate forms by rotation of the complex molecules within the layer plane. In all cases the nonsolvates are completely converted into the corresponding crystalline solvate forms by exposure to the vapor of the guest, and conversely they are quantitatively recovered from the solvate upon removal of the guest by mild conditions. On the basis of the structural data, it is proposed that the solvation/desolvation process proceeds by a concerted rotation of the complex molecules in the layer plane. The structural analysis of Pd(LOH)(2)(CH(3)COO)(2) and of its tetrahydrofuran monosolvate form suggests that the first step of the solid/gas solvation process may imply the clathration of 1 mol of guest between the aryl rings, which successively triggers the collective reorientation of the host molecules.     

Assessment of analytical testing: The impact of metrics for the sustainable measurement of pharmaceuticals

Analytical metrics allow a periodic, healthy examination of methods and how they impact the environment, while ensuring that procedures are reproducible, reliable, repeatable, transferable, and of high quality. This article provides a review of current trends with analytical metrics, such as the analytical method green score (AMGS). More importantly, the AMGS impact on sample preparation, development strategy, and energy consumption considerations are provided. Finally, recent uses of metrics with specific relevance to the pharmaceutical research and development area through method transfer are covered.     

Comparing distance metrics for rotation using the k‐nearest neighbors algorithm for entropy estimation

Distance metrics facilitate a number of methods for statistical analysis. For statistical mechanical applications, it is useful to be able to compute the distance between two different orientations of a molecule. However, a number of distance metrics for rotation have been employed, and in this study, we consider different distance metrics and their utility in entropy estimation using the k‐nearest neighbors (KNN) algorithm. This approach shows a number of advantages over entropy estimation using a histogram method, and the different approaches are assessed using uniform randomly generated data, biased randomly generated data, and data from a molecular dynamics (MD) simulation of bulk water. The results identify quaternion metrics as superior to a metric based on the Euler angles. However, it is demonstrated that samples from MD simulation must be independent for effective use of the KNN algorithm and this finding impacts any application to time series data. © 2013 Wiley Periodicals, Inc.     

Multiplex_QA: an exploratory quality assessment tool for multiplexed electrophoretic assays

Multiplex_QA is a data analysis tool for visualizing short- and long-term changes in the performance of multiplexed electrophoretic assays, particularly the commercial short tandem repeat (STR) kits used by the human forensic identity community. A number of quality metrics are calculated from the signal collected for the internal size standard included in nearly all multiplex assays. These quality metrics are related to the signal intensity, symmetry, retention, resolution, and noise of data collected by capillary electrophoresis systems. Interlocking graphical displays enable the identification of changes in the quality metrics with time, evaluation of relationships among the metrics, and detailed examination of electropherographic features of particularly interesting analyses. While primarily intended for exploring which metrics are most useful for documenting data quality, the current version of the tool is sufficiently robust for use by forensic scientists with an interest in data analysis and access to a fast desktop computer.     

A family of metrics for biopolymers based on counting independent sets

We introduce a new family of metrics for graphs of fixed size, based on counting-independent sets. Our definition is simpler and easier to calculate than the edge ideal metric family defined by Llabrés and Rosselló without loosing any of its abstract properties. We contrast them on some examples with graphs that represent protein secondary and three-dimensional (3D) structures. We conclude that although the edge ideal metrics are faster to calculate on some sparse graphs, in general, the independent set metrics are more tractable.     

Combining electron-neutral building blocks with intramolecular "conformational locks" affords stable, high-mobility p- and n-channel polymer semiconductors

Understanding the relationship between molecular/macromolecular architecture and organic thin film transistor (TFT) performance is essential for realizing next-generation high-performance organic electronics. In this regard, planar π-conjugated, electron-neutral (i.e., neither highly electron-rich nor highly electron-deficient) building blocks represent a major goal for polymeric semiconductors, however their realization presents synthetic challenges. Here we report that an easily accessible (minimal synthetic steps), electron-neutral thienyl-vinylene (TVT)-based building block having weak intramolecular S···O "conformational locks" affords a new class of stable, structurally planar, solution-processable, high-mobility, molecular, and macromolecular semiconductors. The attraction of merging the weak TVT electron richness with supramolecular planarization is evident in the DFT-computed electronic structures, favorable MO energetics, X-ray diffraction-derived molecular structures, experimental lattice coehesion metrics, and excellent TFT performance. TVT-based polymer TFTs exhibit stable carrier mobilities in air as high as 0.5 and 0.05 cm(2)/V·s (n- and p-type, respectively). All-TVT polymer-based complementary inverter circuitry exhibiting high voltage gains (~50) and ring oscillator circuitry with high f(osc)(~1.25 kHz) is readily fabricated from these materials by simple inkjet printing.     

Metrics of separation performance in chromatography. Part 1. Definitions and application to static analyses

Earlier introduced metrics of separation performance are described in a systematic way. After providing the definitions of the metrics suitable for a broad variety of applications, the study focuses on static analyses (isothermal GC, isocratic LC, etc.) and their general separation performance. Statistically expected number of resolved (adequately separated) single-component peaks is treated as the ultimate metric of general separation performance of chromatographic analysis. That number depends on the peak capacity of the analysis and the number of components in a test mixture. The peak capacity, in turn, depends on the separation capacity of a column and the lowest separation required by the data-analysis system for resolving poorly separated peaks. The separation capacity is a special case of a broader metric of the separation measure which is a function of column efficiency, solute separability, and the level of the solute interaction with a column stationary phase. The formulae for theoretical prediction of all these metrics for arbitrary pairs of peaks in static analyses are derived. To provide a better insight into the basic metrics of the separation performance, additional metrics such as the solute discrimination (relative difference in solute velocities), utilization of separability (solute discrimination per unit of their separability), specific separation (the separation per unit of separability), and others are defined and found for static analyses.     

A complexity-based measure and its application to phylogenetic analysis

In this article, we propose two well-defined distance metrics of biological sequences based on a universal complexity profile. To illustrate our metrics, phylogenetic trees of 18 Eutherian mammals from comparison of their mtDNA sequences and 24 coronaviruses using the whole genomes are constructed. The resulting monophyletic clusters agree well with the established taxonomic groups.     

Ligand efficiency metrics considered harmful

Ligand efficiency metrics are used in drug discovery to normalize biological activity or affinity with respect to physicochemical properties such as lipophilicity and molecular size. This Perspective provides an overview of ligand efficiency metrics and summarizes thermodynamics of protein–ligand binding. Different classes of ligand efficiency metric are critically examined and the study concludes with suggestions for alternative ways to account for physicochemical properties when prioritizing and optimizing leads.     

Metric engineering of perfluorocarbon-hydrocarbon layered solids driven by the halogen bonding

The halogen bonding driven self-assembly of 1,4-dicyanobutane and 1,6-dicyanohexane with a range of perfluorinated telechelic diiodoalkanes yields layered 1D, infinite chain co-crystals. The structure and metrics of these co-crystals are remarkably predictable.     

Pushing and pulling electrons: the effect on the heat of formation of trifluoromethyl compounds

The effect on heats of formation, of conjoined or proximate functional groups which can interact via polar or resonance effects, is examined using the –CF₃ group as a standard. Two metrics are applied: the difference in heat of formation of G–CF₃ and G–CH₃, where –G is a wide range of functional groups, and also the deviation of the heat of formation of G–CF₃ from the average of the heats of formation of G–G and CF₃–CF₃. This latter metric reveals both stabilizing and destabilizing effects on the heat of formation, of up to 60 kcal/mol, depending on the polar and resonance nature of the –G structure. The possibility of using such metrics as a correction the group additivity values is examined.     

“The hidden power and competitive advantage of applying green chemistry metrics”

Green chemistry (GC) metrics provide insight into the relative waste, time and cost implications of pharmaceutical chemical processes and serve to guide scientists in the strategic application of resources to develop more efficient and sustainable processes. Examples of the application of GC metrics in evaluating pharmaceutical process efficiency and the subsequent development toward improvement exist in abundance from journals such as Organic Process Research and Development, Green Chemistry, or as encompassed by the winning examples from the ACS GCI Pharmaceutical Roundtable's [1] Peter J. Dunn award [2] or the US EPA's Green Chemistry Challenge award [3]. By their nature, GC metrics are continuously evolving but justify the necessary, unceasing investment in understanding and application as they offer unique, opportunistic insight serving to guide scientific resource deployment when developing greener pharmaceutical, chemical processes.     

Implementation of Statistical Process Control for Proteomic Experiments Via LC MS/MS

Statistical process control (SPC) is a robust set of tools that aids in the visualization, detection, and identification of assignable causes of variation in any process that creates products, services, or information. A tool has been developed termed Statistical Process Control in Proteomics (SProCoP) which implements aspects of SPC (e.g., control charts and Pareto analysis) into the Skyline proteomics software. It monitors five quality control metrics in a shotgun or targeted proteomic workflow. None of these metrics require peptide identification. The source code, written in the R statistical language, runs directly from the Skyline interface, which supports the use of raw data files from several of the mass spectrometry vendors. It provides real time evaluation of the chromatographic performance (e.g., retention time reproducibility, peak asymmetry, and resolution), and mass spectrometric performance (targeted peptide ion intensity and mass measurement accuracy for high resolving power instruments) via control charts. Thresholds are experiment- and instrument-specific and are determined empirically from user-defined quality control standards that enable the separation of random noise and systematic error. Finally, Pareto analysis provides a summary of performance metrics and guides the user to metrics with high variance. The utility of these charts to evaluate proteomic experiments is illustrated in two case studies.

Scientometric assessment of the progress of innovative pharmacy in Russia

The importance and potential of scientometric assessment of the progress of innovative pharmacy is discussed. Scientometric publications analyzing and forecasting new domestic drug R&D in historical perspective and in contemporary Russia are considered. The role of systemic informational analysis of science as a new methodic tool for research metrics is described.     

Comparing normal modes across different models and scales: Hessian reduction versus coarse‐graining

Dimension reduction is often necessary when attempting to reach longer length and time scales in molecular simulations. It is realized by constraining degrees of freedom or by coarse‐graining the system. When evaluating the accuracy of a dimensional reduction, there is a practical challenge: the models yield vectors with different lengths, making a comparison by calculating their dot product impossible. This article investigates mapping procedures for normal mode analysis. We first review a horizontal mapping procedure for the reduced Hessian techniques, which projects out degrees of freedom. We then design a vertical mapping procedure for the “implosion” of the all‐atom (AA) Hessian to a coarse‐grained scale that is based upon vibrational subsystem analysis. This latter method derives both effective force constants and an effective kinetic tensor. Next, a series of metrics is presented for comparison across different scales, where special attention is given to proper mass‐weighting. The dimension‐dependent metrics, which require prior mapping for proper evaluation, are frequencies, overlap of normal mode vectors, probability similarity, Hessian similarity, collectivity of modes, and thermal fluctuations. The dimension‐independent metrics are shape derivatives, elastic modulus, vibrational free energy differences, heat capacity, and projection on a predefined basis set. The power of these metrics to distinguish between reasonable and unreasonable models is tested on a toy alpha helix system and a globular protein; both are represented at several scales: the AA scale, a Gō‐like model, a canonical elastic network model, and a network model with intentionally unphysical force constants. Published 2012 Wiley Periodicals, Inc.