多个离群值鉴别方法在定量校正中的应用

本文使用统计量Fk进行多元线性回归分析体系离群值的鉴别,将该方法用于紫外光度法直接校正定量波长的选择及液相压电传感器阵列定量体系校正样本的选择,选择前后定量分析结果表明,方法可行。     

如何统一Q检验法对一个以上可疑值的取舍问题

在定量分析工作中,通常要对同一试样做几份平行测定,然后求出平均值.但所测结果总会有大有小,如果数据中出现显著性差异,即有的数据特大或特小(称为可疑值或离群值),是否都能参加平均值的计算呢?这就需要用统计学方法进行检验,不得随意弃去或保留可疑值.取舍可疑值的方法很多,其中Q检验是一种简便易行、比较常用的方法.具体怎么检验,与可疑值的分布情况有关.可疑值的分布,一般可分三种情况:     

用质控图法和稳健统计法评估热裂解-原子吸收光谱法测量固体样品汞含量的不确定度

基于实验室长期积累的质控数据评估测量不确定度的top-down方法具有广泛应用前景,质控图法和稳健统计法是其中常见方法。质控图法要求测量数据中不含离群值,对测量数据时间顺序有明确要求。稳健统计法是指不用识别、剔除离群值,直接应用全部测量数据,将离群值对统计分析结果影响降低到最小的统计分析方法。基于质控图法、稳健统计和实验室长期积累的质控数据对固体样品汞含量热裂解-原子吸收光谱分析方法的测量不确定度进行了评估,对两种方法计算结果、实验室评估的测量不确定度和标准样品标称值的不确定度进行了比较。计算结果表明,两种方法结果基本吻合,基于实验室质控数据期间精密度评估的不确定度明显小于标准样品标称值,结果合理。     

用质控图法和稳健统计法评估固体样品汞含量热裂解-原子吸收法测量不确定度

基于实验室长期积累的质控数据评估测量不确定度的top-down方法具有广泛应用前景,质控图法和稳健统计法是其中常见方法。质控图法要求测量数据中不含离群值,对测量数据时间顺序有明确要求。稳健统计法是指不用识别、剔除离群值,直接应用全部测量数据,将离群值对统计分析结果影响降低到最小的统计分析方法。本文基于质控图法、稳健统计和实验室长期积累的质控数据对固体样品汞含量热裂解-原子吸收分析方法的测量不确定度进行了评估,对两种方法计算结果、实验室评估的测量不确定度和标准样品标称值的不确定度进行了比较。计算结果表明,两种方法结果基本吻合,基于实验室质控数据期间精密度评估的不确定度明显小于标准样品标称值,结果合理。     

统计学在理化检验中的应用第七讲一组和多组测定值的统计处理(续)

2　多组实验观测值的处理在对某一量进行实际测量时 ,经常采用多种方法或由很多实验室进行分析测试。因此得到很多组实验数据。现假设 ,由 m个测定方法或 m个实验室提供 m组数据 ,即 x1 1 ,x1 2 ,… ,x1 n1,平均值 x1 ,单次测量的标准偏差 S1 ;x2 1 ,x2 2 ,… ,x2 n2 ,平均值 x2 ,单次测量的标准偏差 S2 ;… ;xm1 ,xm2 ,… ,xmnm,平均值xm,单次测量的标准偏差 Sm。对这样的 m组数据应该如何进行统计处理呢 ?2 .1　当数据为等精度时观测值的统计处理2 .1 .1　等精度检验要检验 m组数据之间是否等精度有很多种统计检验方式可供使用 ,较常用…     

计算机在测试数据统计处理中的应用——Ⅰ、分析测试数据可靠性的检验(上)

测试数据可靠性的检验包括离群值检验,精密度的比较和系统误差的检验,检验的目的是判断测试过程中是否存在异常值和系统误差。本文介绍了用于上述目的的各种常用的统计检验方法,给出了用BASIC语言编写的程序,最后用实例说明了它们各自的应用。     

碲化铜中碲含量测定方法的精密度

在有色金属行业标准碲化铜化学分析方法起草过程中,拟定了还原分离–硫酸亚铁铵返滴定法测定碲化铜中碲含量,并组织11家实验室参加了协同实验,得到碲化铜中碲的测定结果。按照GB/T 6379.2–2004标准的规范要求,采用曼德尔k图和h图、格拉布斯检验和柯克伦检验法对数据的一致性和离群值进行检查,计算出该方法精密度试验中重复性限(r)和再现性限(R)的结果。实验证明,统计计算的得到的r和R能反映该方法适用性的真实情况。该方法可以作为有色金属行业标准推广使用。     

实验室常用数据处理程序的PC-1500多功能程序

本文介绍为分析室常用数据处理方法所编制的PC-1500袖珍计算机使用的多功能程序。可分别打印出两组测定数据的测定次数N、平均值(?)、标准偏差S及变异系数C.V.,还可对原始数据进行Grubbs检验以给出各组剔除极端值后的N、(?)、S和C.V.及(?)的置信区间(α=0.05);对F和t检验可打印出计算值和α=0.05的临界F值与t值及判断结果;可为配对试验结果进行t检验。所编程序基本上涉及到各种常用数据的处理,使用灵活、直观而又方便。     

地下水中有机磷农药测量方法准确度的质量评价

采用5个浓度水平样品、通过7家实验室进行协同评定试验,验证了地下水中敌敌畏、甲拌磷、乐果、甲基对硫磷、马拉硫磷、毒死蜱、对硫磷等有机磷农药分析方法的稳定性与准确性。地下水中有机磷组分选用二氯甲烷与丙酮混合溶液(丙酮溶液的比例约为5%)进行液液萃取,利用气相色谱火焰光度检测器进行检测。每一浓度级别的水溶液样品分别进行3组全过程检测。由于个别实验室或数据可能与其他实验室或其他数据明显不一致从而影响统计处理,因此对这些数值进行了一致性和离群值检验,并应用柯克伦和格拉布斯法则对测量方法与结果的准确度进行了确定。结果表明,地下水中,7种有机磷农药在20~2 000 ng/L浓度范围内,经柯克伦法则与格拉布斯法则检验,参与统计的个别测量数据因离群被剔除后计算结果较好。协同验证试验中,有机磷各组分在5个浓度水平的基体加标平均回收率为91.1%~109%,相对标准偏差为2.2%~12.6%,各样品中替代物磷酸三苯酯的回收率为77.8%~119%,方法准确、可靠。     

自举法在协同试验数据处理中的应用

正协同试验是分析方法标准化过程中的重要环节,在该过程中,组织者将均匀样品分发到多个实验室,由其开展平行测试。组织者对回收的测试结果进行离群值检验与处理后,利用方差分析计算重复性标准差sr和再现性标准差sR[1]。这两项参数是方法应用中进行质量控制的重要依据,一般情况下(置信度为95%),实验室内重复性条件下经两次独立测试得到的结果之差不得超过2.8 sr,两个实验室分别测试得到的结果之差不得超过2.8 sR,否则     

A new strategy of outlier detection for QSAR/QSPR

The crucial step of building a high performance QSAR/QSPR model is the detection of outliers in the model. Detecting outliers in a multivariate point cloud is not trivial, especially when several outliers coexist in the model. The classical identification methods do not always identify them, because they are based on the sample mean and covariance matrix influenced by the outliers. Moreover, existing methods only lay stress on some type of outliers but not all the outliers. To avoid these problems and detect all kinds of outliers simultaneously, we provide a new strategy based on Monte‐Carlo cross‐validation, which was termed as the MC method. The MC method inherently provides a feasible way to detect different kinds of outliers by establishment of many cross‐predictive models. With the help of the distribution of predictive residuals such obtained, it seems to be able to reduce the risk caused by the masking effect. In addition, a new display is proposed, in which the absolute values of mean value of predictive residuals are plotted versus standard deviations of predictive residuals. The plot divides the data into normal samples, y direction outliers and X direction outliers. Several examples are used to demonstrate the detection ability of MC method through the comparison of different diagnostic methods. © 2009 Wiley Periodicals, Inc. J Comput Chem, 2010     

Outliers in SAR and QSAR: Is unusual binding mode a possible source of outliers?

A lead optimization is usually carried out by structure-activity relationship (SAR) and/or quantitative structure-activity relationship (QSAR) studies. One of the assumptions in SAR and QSAR studies is that similar analogs bind to the same binding site in a similar binding mode. One often observes that there are outliers, especially in QSAR. However, most QSAR studies are carried out focusing their attention to the development of QSAR and leave the outliers without much attention. We searched a number of ligand-bound X-ray crystal structures from the protein structure database to find evidences that could indicate a possible source of outliers in SAR or QSAR. Our results show that unusual binding mode could be a source of outliers. Electronic supplementary material The online version of this article (doi:) contains supplementary material, which is available to authorized users.     

NMF-Based Approach for Missing Values Imputation of Mass Spectrometry Metabolomics Data

In mass spectrometry (MS)-based metabolomics, missing values (NAs) may be due to different causes, including sample heterogeneity, ion suppression, spectral overlap, inappropriate data processing, and instrumental errors. Although a number of methodologies have been applied to handle NAs, NA imputation remains a challenging problem. Here, we propose a non-negative matrix factorization (NMF)-based method for NA imputation in MS-based metabolomics data, which makes use of both global and local information of the data. The proposed method was compared with three commonly used methods: k-nearest neighbors (kNN), random forest (RF), and outlier-robust (ORI) missing values imputation. These methods were evaluated from the perspectives of accuracy of imputation, retrieval of data structures, and rank of imputation superiority. The experimental results showed that the NMF-based method is well-adapted to various cases of data missingness and the presence of outliers in MS-based metabolic profiles. It outperformed kNN and ORI and showed results comparable with the RF method. Furthermore, the NMF method is more robust and less susceptible to outliers as compared with the RF method. The proposed NMF-based scheme may serve as an alternative NA imputation method which may facilitate biological interpretations of metabolomics data.     

Use of cluster separation indices and the influence of outliers: application of two new separation indices,the modified silhouette index and the overlap coefficient to simulated data and mouse urine metabolomic profiles

To quantify separate classes, four indices are compared namely the Davies Bouldin index, the silhouette width and two new approaches described in this paper, the modified silhouette width index based on the proportion of objects with a positive silhouette width and the Overlap Coefficient. Four sets of simulated datasets are described, each in turn, consisting of 15 sets of data of varying degrees of overlap, and differing in the nature of outliers. Three experimental datasets consisting of the gas chromatography mass spectrometry of extracts from mouse urine obtained to study the effect of different environmental (stress), physiological (diet) and developmental (age) factors on their metabolic profiles are also described. The paper discusses the robustness of each approach to outliers, and to allow assessment of class separation for each index. The two modifications protect against outliers. Copyright © 2008 John Wiley & Sons, Ltd.     

Robustified least squares support vector classification

Support vector machine (SVM) algorithms are a popular class of techniques to perform classification. However, outliers in the data can result in bad global misclassification percentages. In this paper, we propose a method to identify such outliers in the SVM framework. A specific robust classification algorithm is proposed adjusting the least squares SVM (LS‐SVM). This yields better classification performance for heavily tailed data and data containing outliers. Copyright © 2009 John Wiley & Sons, Ltd.     

Outliers in SAR and QSAR: 2. Is a flexible binding site a possible source of outliers?

Structure-activity relationship (SAR) and/or quantitative structure-activity relationship (QSAR) studies play an important role in a lead optimization of drug discovery research. When there is a lack of ligand-bound protein structural information, one of the assumptions in SAR and QSAR studies is that similar analogs bind to the same binding site in a similar binding mode. In such studies, outliers have often been observed, especially in QSAR. However, most of these studies have focused their attention on the development of QSAR and left outliers unattended. We searched ligand-bound X-ray crystal structures from the protein structure database to find evidences that could indicate a possible source of outliers in SAR or QSAR. Our results showed the possibility of conformational changes in a flexible binding site as one possible source of outliers. Electronic supplementary material The online version of this article (doi:) contains supplementary material, which is available to authorized users.     

A new hybrid strategy for constructing a robust calibration model for near-infrared spectral analysis

A new hybrid algorithm is proposed for construction of a high-quality calibration model for near-infrared (NIR) spectra that is robust against both spectral interference (including background and noise) and multiple outliers. The algorithm is a combination of continuous wavelet transform (CWT) and a modified iterative reweighted PLS (mIRPLS) procedure. In the proposed algorithm the spectral interference is filtered by CWT at the first stage then mIRPLS is proposed to detect the multiple outliers in the CWT domain. Compared with the original IRPLS method, mIRPLS does not need to adjust variable parameters to achieve optimum calibration results, which makes it very convenient to perform in practice. The final PLS model is constructed robustly because both the spectral interference and multiple outliers are eliminated. In order to validate the effectiveness and universality of the algorithm, it was applied to two different sets of NIR spectra. The results indicate that the proposed strategy can greatly enhance the robustness and predictive ability of NIR spectral analysis.     

Robust regression techniques A useful alternative for the detection of outlier data in chemical analysis

The validation of an analytical procedure means the evaluation of some performance criteria such as accuracy, sensitivity, linear range, capability of detection, selectivity, calibration curve, etc. This implies the use of different statistical methodologies, some of them related with statistical regression techniques, which may be robust or not. The presence of outlier data has a significant effect on the determination of sensitivity, linear range or capability of detection amongst others, when these figures of merit are evaluated with non-robust methodologies.In this paper some of the robust methods used for calibration in analytical chemistry are reviewed: the Huber M-estimator; the Andrews, Tukey and Welsh GM-estimators; the fuzzy estimators; the constrained M-estimators, CM; the least trimmed squares, LTS. The paper also shows that the mathematical properties of the least median squares (LMS) regression can be of great interest in the detection of outlier data in chemical analysis. A comparative analysis is made of the results obtained by applying these regression methods to synthetic and real data. There is also a review of some applications where this robust regression works in a suitable and simple way that proves very useful to secure an objective detection of outliers. The use of a robust regression is recommended in ISO 5725-5.     

Robust regression and outlier detection in the evaluation of robustness tests with different experimental designs

Robustness tests are usually based on an experimental design approach. As designed experiments generally lead to a large variability among the results, erroneous results are often not readily detected. As a consequence, the ordinary least squares (OLS) estimates of the effects of the robustness test can be biased. Here, two robustness tests are studied, which both contain a suspicious result. Moreover, simulated datasets are considered to examine the influence of the extent of the outlier as well as the influence of multiple outliers. On the one hand, different methods are applied to inspect the results of the experiments for outliers: the half-normal plot of the OLS residuals, the normal probability plot of the effects and a method, which is based on experimental design reconstruction. On the other hand, two robust regression methods are applied to calculate the effects with a minimum influence of possible outliers. The different methods are compared and it is evaluated under which circumstances they can be applied.     

Comparison of methods for outlier detection and their effects on the classification results for a particular data base

One of the drawbacks for using linear discriminant analysis (LDA) is the presence of outliers. Some methods of detecting outliers are compared and applied to a particular data base. When multivariate methods (multinormal distribution procedure and Hawkins' procedure) were applied, the two subsets produced did not differ greatly. Assumptions needed for the application of LDA were evaluated for each subset. Classification ability, feature selection and prediction ability were considered for each subset. Results for each subset were quite different. Hawkins' procedure seems the better method for detecting outliers.