Global trends in quality specifications: a report from the Stockholm Conference

 In Stockholm a conference entitled: "Strategies to Set Global Quality Specifications in Laboratory Medicine" was held in April 1999. The primary aim in organizing the Conference was to provide a vehicle for reaching consensus on global quality specifications in laboratory medicine. This objective was achieved and a lively constructive debate, after the presentations were complete, led to agreement on the principles laid down in the Consensus Statement. The International Federation of Clinical Chemistry (IFCC), the International Union of Pure and Applied Chemistry (IUPAC) and the World Health Organization (WHO) kindly sponsored the Conference but it must be noted that the Consensus Statement reflects the views of the presenters and registrants who participated in the Conference and does not necessarily represent those of the sponsoring bodies. This paper reports on the standardization efforts so far, the Stockholm Consensus Conference, the ISO uncertainty concept and the consensus reality. A hierarchy of quality specifications in laboratory medicine was defined and agreed on. Received: 15 April 2000 · Accepted: 15 April 2000     

Ligand and structure‐based models for the prediction of ligand‐receptor affinities and virtual screenings: Development and application to the β2‐adrenergic receptor

In this study, we evaluated the applicability of ligand‐based and structure‐based models to quantitative affinity predictions and virtual screenings for ligands of the β₂‐adrenergic receptor, a G protein‐coupled receptor (GPCR). We also devised and evaluated a number of consensus models obtained through partial least square regressions, to combine the strengths of the individual components. In all cases, the bioactive conformation of each ligand was derived from molecular docking at the crystal structure of the receptor. We identified the most effective models applicable to the different scenarios, in the presence or in the absence of a training set. For ranking the affinity of closely related analogs when a training set is available, a ligand‐based consensus model (LI‐CM) seems to be the best choice, while the structure‐based MM‐GBSA score seems the best alternative in the absence of a training set. For virtual screening purposes, the structure‐based MM‐GBSA score was found to be the method of choice. Consensus models consistently had performances superior or close to those of the best individual components, and were endowed with a significantly increased robustness. Given multiple models with no a priori knowledge of their predictive capabilities, constructing a consensus model ensures results very close to those that the best model alone would have yielded. © 2009 Wiley Periodicals, Inc. J Comput Chem 2010     

Metrological reference values for estimating measurement bias in clinical laboratory sciences

Clinical laboratory quantities are measured for monitoring or diagnostic purposes. In both cases, a modification of the measurement bias can generate a false interpretation of measurement results. On the other hand, in clinical laboratory sciences, one of the most frequently used metrological reference value for calculating measurement bias is a conventional value that corresponds to the called consensus value. But this selection probably is not the best one, and some clinical and biological considerations should be made to decide the requirement for maximum permissible measurement bias and to decide the more appropriate metrological reference value. In the clinical laboratory, the medical relevance of the measurement bias of any measurement system in use depends on the measurement bias with which the biological reference values were produced. This dependence is due to the necessity that, for interpretation purposes, the current measurement bias should be the same that is included in the biological reference limits. For this reason, it is necessary to control the changes of bias during the life span of a measuring system. Several scenarios are described for the different estimations of bias.     

The quality of consensus values

The definition of a metrological term “quality” is discussed in relation to a series of consensus values pertaining to a set of measurement results from an actual PT exercise. It was concluded that the definition used by ISO in quality management systems could be readily interpreted in the metrological sphere.     

Measurement uncertainty and doping control in sport. Part 2: Metrological traceability aspects

The assessment of (non-)compliance in doping control and in particular the appreciation of uncertainty of measurement in such an assessment has become a subject of debate. In a previous paper, the requirements for evaluating measurement uncertainty have been addressed. The debate now focuses on the estimation of the standard uncertainty. The completeness of an uncertainty budget is discussed in view of the (metrological) traceability of the result. It is concluded that despite the fact that the new doping rules dictate that A-tests and B-tests be performed in the same laboratory, the uncertainty budget should encompass all factors, including between-laboratory effects. The assessment of (non-)compliance should be based on a one-sided t-test. There is also consensus about the level of confidence to be applied (99%), despite the fact that the regulations state 95% as the appropriate level.     

Quality specifications in laboratory medicine – current consensus views

Every analytical method used in laboratory medicine can be fully described in terms of its performance characteristics. Ideally, quality specifications should be available for all of these, particularly precision and bias. Specifications for these can be set using a variety of strategies. Consideration of the clinical settings of monitoring individual patients and diagnosis using reference intervals shows that generally applicable quality specifications can be based on the components of biological variation, namely, within-subject [CV_I] and between-subject [CV_G] variation. Current consensus is that precision should be <1/2CV_I and bias should be <1/4[CV_I ²+CV_G ²]^1/2. This strategy has advantages in that data on components of biological variation are easily available on more than 180 quantities. Dissemination of information on application of objective quality specifications needs attention from those involved in publication, manufacturers and organisers of external quality assessment schemes.     

Comparability of analytical results obtained in proficiency testing based on a metrological approach

A “yes–no” type of criterion is proposed for the assessment of comparability of proficiency testing (PT) results when the PT scheme is based on a metrological approach, i.e. on the use of a reference material as the test sample, etc. The criterion tests a null hypothesis concerning the insignificance of a bias of the mean of the results from a traceable value certified in the reference material used for the PT. Reliability of such assessment is determined by the probabilities of not rejecting the null hypothesis when it is true, and rejecting it when it is false (the alternative hypothesis is true). It is shown that a number of chemical, metrological and statistical reasons should be taken into account for careful formulation of the hypotheses, enabling the avoidance of an erroneous assessment of the comparability. The criterion can be helpful for PT providers and laboratory accreditation bodies in analysis of PT results.     

Characterization of the measurement error structure in 1D H NMR data for metabolomics studies

NMR-based metabolomics is characterized by high throughput measurements of the signal intensities of complex mixtures of metabolites in biological samples by assaying, typically, bio-fluids or tissue homogenates. The ultimate goal is to obtain relevant biological information regarding the dissimilarity in patho-physiological conditions that the samples experience. For a long time now, this information has been obtained through the analysis of measured NMR signals via multivariate statistics.NMR data are quite complex and the use of such multivariate statistical methods as principal components analysis (PCA) for their analysis assumes that the data are multivariate normal with errors that are identical, independent and normally distributed (i.e. iid normal). There is a consensus that these assumptions are not always true for these data and, thus, several methods have been devised to transform the data or weight them prior to analysis by PCA. The structure of NMR measurement noise, or the extent to which violations of error homoscedasticity affect PCA results have neither been characterized nor investigated.A comprehensive characterization of measurement uncertainties in NMR based metabolomics was achieved in this work using an experiment designed to capture contributions of several sources of error to the total variance in the measurements. The noise structure was found to be heteroscedastic and highly correlated with spectral characteristics that are similar to the mean of the spectra and their standard deviation. A model was subsequently developed that potentially allows errors in NMR measurements to be accurately estimated without the need for extensive replication.     

A study of reproducibility of guanidination-dimethylation labeling and liquid chromatography matrix-assisted laser desorption ionization mass spectrometry for relative proteome quantification

The combination of dimethylation after guanidination (2MEGA) isotope labeling with microbore liquid chromatography (LC)-matrix-assisted laser desorption ionization (MALDI) MS and MS/MS [C. Ji, N. Guo, L. Li, J. Proteome Res. 4 (2005) 2099] has been reported as a promising strategy for abundance ratio-dependent quantitative proteome analysis. A critical step in using this integrated strategy is to set up the abundance ratio threshold of peptide pairs, above which the peptide pairs are used for quantifying and identifying the protein that is considered to be differentially expressed between two different samples. The threshold is determined by technical variation (i.e., the overall abundance ratio variation caused by the experimental process including sample workup, MS analysis and data processing) as well as biological variation (i.e., the abundance ratio variation caused by the biological process including cell growth), which can be defined and assessed by a coefficient of variation (CV). We have designed experiments and measured three different levels of variations, starting with the same membrane protein preparation, the same batch of cells and three batches of cells from the same cell line grown under the same conditions, respectively. It is shown that technical variation from the experimental processes involved in 2MEGA labeling LC-MALDI MS has a CV of <15%. In addition, the measured biological variation from cell growth was much smaller than the measured technical variation. From the studies of the occurrence rate of outliers in the distribution of the abundance ratio data within a comparative dataset of peptide pairs, it is concluded that, to compare the proteome changes between two sets of cultured cells without the use of replicate experiments, a relative abundance ratio of greater than 2X or less than 0.5X (X is the average abundance ratio of the dataset) on peptide pairs can be used as a stringent threshold to quantify and identify differentially expressed proteins with high confidence.     

Some Latin American experiences concerning teaching of chemical metrology

In this paper, 15 years of the experiences acquired concerning the teaching of chemical metrology in Latin America are presented. These include postgraduate and undergraduate activities developed in eight countries. The combination of theoretic and practical activities and the sequence of learning from metrological, statistical, and chemometrical backgrounds up to practical activities in personal computers are basic and motivate the learning process. Care is taken to promote the metrological approach and thinking in analytical chemistry. The learning of computing techniques plays an important role, combining graphic and numerical techniques for data analysis. The role of examples during the teaching process is analyzed and recognized. The introduction of a general model of errors permits one to approach different topics on a metrological basis. The metrological approach of uncertainty based on the theory of errors permits one to develop the topic. Undergraduate students acquire a basic metrological knowledge and other experiences are also presented. Recommendations for undergraduate and postgraduate programs are pointed out.     

GFscore: a general nonlinear consensus scoring function for high-throughput docking

Most of the recent published works in the field of docking and scoring protein/ligand complexes have focused on ranking true positives resulting from a Virtual Library Screening (VLS) through the use of a specified or consensus linear scoring function. In this work, we present a methodology to speed up the High Throughput Screening (HTS) process, by allowing focused screens or for hitlist triaging when a prohibitively large number of hits is identified in the primary screen, where we have extended the principle of consensus scoring in a nonlinear neural network manner. This led us to introduce a nonlinear Generalist scoring Function, GFscore, which was trained to discriminate true positives from false positives in a data set of diverse chemical compounds. This original Generalist scoring Function is a combination of the five scoring functions found in the CScore package from Tripos Inc. GFscore eliminates up to 75% of molecules, with a confidence rate of 90%. The final result is a Hit Enrichment in the list of molecules to investigate during a research campaign for biological active compounds where the remaining 25% of molecules would be sent to in vitro screening experiments. GFscore is therefore a powerful tool for the biologist, saving both time and money.     

Method comparison on estimation for an unknown specimen in a heteroscedastic controlled calibration model

In analytical chemistry applications, statistical calibration models are commonly used to estimate the true value of an unknown specimen. In this article, we consider a heteroscedastic controlled calibration model in which both dependent and independent variables are subject to heteroscedastic measurement errors. The main task of using this model is to estimate the true value of an unknown regressor (independent variable) under the condition that a set of observations on its corresponding response (dependent variable) is available. We introduce four estimation methods to the problem of interest, including generalized least squares (GLS), modified least squares, corrected score, and expectation maximization‐based (EM‐based) methods. Furthermore, an interval estimation based on an asymptotic method is also derived. We compare their performance through detailed simulation studies. In consequence, GLS and EM‐based methods are recommended in practical use. A real data example is given to illustrate the application of the calibration model. Copyright © 2013 John Wiley & Sons, Ltd.     

Performance evaluation for proficiency testing with a limited number of participants

Proficiency testing (PT) is an essential tool for laboratories to assess their competency. Also, participation in PT has become one of the mandatory requirements for laboratory to seek accreditation according to ISO/IEC 17025. For this reason, the effectiveness of performance evaluation by PT scheme is of great concern for the participants and for accreditation bodies as well. In practice, owing to unavailability of other appropriate alternatives, PT scheme providers may have to choose using consensus values to evaluate the performance of participants. However, such consensus values approach was not recommended by relevant international guidelines for PT schemes with limited number of participants. With the use of Monte Carlo simulation technique, this study attempted to investigate the effectiveness of using consensus values for performance evaluation in PT schemes with limited number of participants. The simulation process was schemed according to the statistical model provided by ISO 5725-1 for laboratory measurement results, which covered components like method bias, laboratory bias, and measurement precision. The effectiveness of the consensus value approach was expressed as the percentage of participants in a simulation run could get the same evaluation result, either satisfactory or unsatisfactory, against the “true value.” The findings indicated that the number of participants, choice of consensus values, mass fraction of analyte, method bias, laboratory bias, and measurement repeatability of participating laboratories would all affect the effectiveness of the consensus value approach but at different extent. However, under certain circumstances, use of consensus value could still be considered as an acceptable approach for performance evaluation even the number of participants was limited. Some of the findings were further verified using real data from PT schemes where appropriate certified reference materials or reliable reference values were available.     

The estimation and use of measurement uncertainty for a drug substance test procedure validated according to USP <1225>

The vision for metrology at the United States Pharmacopeia (USP) is for international recognition, harmonization, and official acceptance of all USP reference standards that are based on sound, scientific, metrological principles, such as measurement uncertainty (MU). Pharmaceutical testing laboratories will need to estimate the MU for their test procedures. This paper demonstrates how to estimate MU for a test procedure validated following the requirements in USP General Chapter <1225>, Validation of Compendial Procedures. The test procedure determines the assay and impurities for a pharmaceutical drug substance. Pharmaceutical manufacturing companies are required to test the drug substance used in their drug products. In this example, a new test procedure was developed and validated following the requirements in USP <1225>. The MU was estimated using the data from the test procedure validation. The MU estimate determined that the test procedure was fit for use with one determination, avoiding the expense of duplicate determinations. The MU estimate was used to set up decision rules for comparing test data to the assay and impurity specifications. The information from the MU estimate was used to monitor the routine use of the test procedure using control charts that assess accuracy and precision of each run based upon actual test procedure performance capability.     

Core trees and consensus fragment sequences for molecular representation and similarity analysis

A new type of molecular representation is introduced that is based on activity class characteristic substructures extracted from random fragment populations. Mapping of characteristic substructures is used to determine atom match rates in active molecules. Comparison of match rates of bonded atoms defines a hierarchical molecular fragmentation scheme. Active compounds are encoded as fragmentation pathways isolated from core trees. These paths are amenable to biological sequence alignment methods in combination with substructure-based scoring functions. From multiple core path alignments, consensus fragment sequences are derived that represent compound activity classes. Consensus fragment sequences weighted by increasing structural specificity can also be used to map molecules and search databases for active compounds.     

A <Emphasis Type="Italic">k</Emphasis>-nearest neighbor classification of hERG K<Superscript>+</Superscript> channel blockers

A series of 172 molecular structures that block the hERG K⁺ channel were used to develop a classification model where, initially, eight types of PaDEL fingerprints were used for k-nearest neighbor model development. A consensus model constructed using Extended-CDK, PubChem and Substructure count fingerprint-based models was found to be a robust predictor of hERG activity. This consensus model demonstrated sensitivity and specificity values of 0.78 and 0.61 for the internal dataset compounds and 0.63 and 0.54 for the external (PubChem) dataset compounds, respectively. This model has identified the highest number of true positives (i.e. 140) from the PubChem dataset so far, as compared to other published models, and can potentially serve as a basis for the prediction of hERG active compounds. Validating this model against FDA-withdrawn substances indicated that it may even be useful for differentiating between mechanisms underlying QT prolongation.     

A consensus least squares support vector regression (LS-SVR) for analysis of near-infrared spectra of plant samples

Consensus modeling of combining the results of multiple independent models to produce a single prediction avoids the instability of single model. Based on the principle of consensus modeling, a consensus least squares support vector regression (LS-SVR) method for calibrating the near-infrared (NIR) spectra was proposed. In the proposed approach, NIR spectra of plant samples were firstly preprocessed using discrete wavelet transform (DWT) for filtering the spectral background and noise, then, consensus LS-SVR technique was used for building the calibration model. With an optimization of the parameters involved in the modeling, a satisfied model was achieved for predicting the content of reducing sugar in plant samples. The predicted results show that consensus LS-SVR model is more robust and reliable than the conventional partial least squares (PLS) and LS-SVR methods.