Influence diagnostic for censored regression models

This paper proposes a diagnostic measure to detect outliers either in the response variable or in the design space both for Schmee-Hahn estimator and Buckley-James estimator in censored regressions. The diagnostics consists of the leverage point and standardized predicted residual.     

On the stability of robust filter-cleaners

We study the dependence on initial conditions of two recursive filters for cleaning a contaminated time series from additive outliers. We show that the function in the recursive equation is in general not contractive, but nevertheless there exists a stationary solution and two iterates with arbitrary initial conditions coincide after some random time T₀. However T₀ may be quite large.     

Adaptive Penalized Weighted Least Absolute Deviations Estimation for the Accelerated Failure Time Model

The accelerated failure time model always offers a valuable complement to the traditional Cox proportional hazards model due to its direct and meaningful interpretation. We propose a variable selection method in the context of the accelerated failure time model for survival data, which can simultaneously complete variable selection and parameter estimation. Meanwhile, the proposed method can deal with the potential outliers in survival times as well as heteroscedastic model errors, which are frequently encountered in practice. Specifically, utilizing the general nonconvex penalty, we propose the adaptive penalized weighted least absolute deviation estimator for the accelerated failure time model. Under some regularity conditions, we show that the proposed method yields consistent estimator and possesses the oracle property. In addition, we propose a new algorithm to compute the estimate in the high dimensional settings, and evaluate the practical utility of the proposed method through extensive simulation studies and two real examples.     

Leak Detection in Water Pipes Based on Maximum Entropy Version of Least Square Twin K-Class Support Vector Machine

Numerous novel improved support vector machine (SVM) methods are used in leak detection of water pipelines at present. The least square twin K-class support vector machine (LST-KSVC) is a novel simple and fast multi-classification method. However, LST-KSVC has a non-negligible drawback that it assigns the same classification weights to leak samples, including outliers that affect classification, these outliers are often situated away from the main leak samples. To overcome this shortcoming, the maximum entropy (MaxEnt) version of the LST-KSVC is proposed in this paper, called the MLT-KSVC algorithm. In this classification approach, classification weights of leak samples are calculated based on the MaxEnt model. Different sample points are assigned different weights: large weights are assigned to primary leak samples and outliers are assigned small weights, hence the outliers can be ignored in the classification process. Leak recognition experiments prove that the proposed MLT-KSVC algorithm can reduce the impact of outliers on the classification process and avoid the misclassification color block drawback in linear LST-KSVC. MLT-KSVC is more accurate compared with LST-KSVC, TwinSVC, TwinKSVC, and classic Multi-SVM.     

基于约束概念格的天体光谱局部离群数据挖掘系统

寻找特殊的、未知的天体是人类探索宇宙奥妙所追求的目标之一,天体光谱数据挖掘是实现该目标的一种有效方法。约束概念格是一种新的概念格结构,具有构造效率高、提取知识针对性和实用性强等特点。针对天体光谱数据在特征子空间中的局部偏离,采用VC++ 6.0和Oracle 9i作为开发工具,设计与实现了基于约束概念格的天体光谱局部离群数据挖掘系统,并对软件模块功能和体系结构,以及天体光谱数据预处理、约束概念格构造方法、基于链表结构的概念格构造、局部离群数据挖掘方法等关键技术进行了详细描述。运行结果表明,该系统实现天体光谱数据局部离群数据挖掘是可行的、有价值的,从而为寻找未知的、特殊的天体提供了一种新途径。     

Choosing a Linear Model with a Random Number of Change-Points and Outliers

The problem of determining a normal linear model with possible perturbations, viz. change-points and outliers, is formulated as a problem of testing multiple hypotheses, and a Bayes invariant optimal multi-decision procedure is provided for detecting at most k (k > 1) such perturbations. The asymptotic form of the procedure is a penalized log-likelihood procedure which does not depend on the loss function nor on the prior distribution of the shifts under fairly mild assumptions. The term which penalizes too large a number of changes (or outliers) arises mainly from realistic assumptions about their occurrence. It is different from the term which appears in Akaikes or Schwarz criteria, although it is of the same order as the latter. Some concrete numerical examples are analyzed.     

基于网格技术的孤立点数据挖掘

在算法LOF和GridLOF的基础上提出了改进的GPOD算法，介绍了实行的步骤，给出主要的算法代码，并给出算法的实现部分和实验结果，最后分析了算法的性能，验证了改进后算法的先进性。     

The two-filter formula for smoothing and an implementation of the Gaussian-sum smoother

A Gaussian-sum smoother is developed based on the two filter formula for smoothing. This facilitates the application of non-Gaussian state space modeling to diverse problems in time series analysis. It is especially useful when a higher order state vector is required and the application of the non-Gaussian smoother based on direct numerical computation is impractical. In particular, applications to the non-Gaussian seasonal adjustment of economic time series and to the modeling of seasonal time series with several outliers are shown.     

The forward search and data visualisation

Summary  A statistical analysis using the forward search produces many graphs. For multivariate data an appreciable proportion of these are a variety of plots of the Mahalanobis distances of the individual observations during the search. Each unit, originally a point inv-dimensional space, is then represented by a curve in two dimensions connecting the almostn values of the distance for each unit calculated during the search. Our task is now to recognise and classify these curves: we may find several clusters of data, or outliers or some unexpected, non-normal, structure. We look at the plots from five data sets. Statistical techniques in clude cluster analysis and transformations to multivariate normality.     

NMF-Based Approach for Missing Values Imputation of Mass Spectrometry Metabolomics Data

In mass spectrometry (MS)-based metabolomics, missing values (NAs) may be due to different causes, including sample heterogeneity, ion suppression, spectral overlap, inappropriate data processing, and instrumental errors. Although a number of methodologies have been applied to handle NAs, NA imputation remains a challenging problem. Here, we propose a non-negative matrix factorization (NMF)-based method for NA imputation in MS-based metabolomics data, which makes use of both global and local information of the data. The proposed method was compared with three commonly used methods: k-nearest neighbors (kNN), random forest (RF), and outlier-robust (ORI) missing values imputation. These methods were evaluated from the perspectives of accuracy of imputation, retrieval of data structures, and rank of imputation superiority. The experimental results showed that the NMF-based method is well-adapted to various cases of data missingness and the presence of outliers in MS-based metabolic profiles. It outperformed kNN and ORI and showed results comparable with the RF method. Furthermore, the NMF method is more robust and less susceptible to outliers as compared with the RF method. The proposed NMF-based scheme may serve as an alternative NA imputation method which may facilitate biological interpretations of metabolomics data.