Nonparametric variable selection and its application to additive models

Variable selection for multivariate nonparametric regression models usually involves parameterized approximation for nonparametric functions in the objective function. However, this parameterized approximation often increases the number of parameters significantly, leading to the “curse of dimensionality” and inaccurate estimation. In this paper, we propose a novel and easily implemented approach to do variable selection in nonparametric models without parameterized approximation, enabling selection consistency to be achieved. The proposed method is applied to do variable selection for additive models. A two-stage procedure with selection and adaptive estimation is proposed, and the properties of this method are investigated. This two-stage algorithm is adaptive to the smoothness of the underlying components, and the estimation consistency can reach a parametric rate if the underlying model is really parametric. Simulation studies are conducted to examine the performance of the proposed method. Furthermore, a real data example is analyzed for illustration.

     

Volterra-type models for nonlinear systems identification

In this work, multi-input multi-output (MIMO) nonlinear process identification is dealt with. In particular, two Volterra-type models are discussed in the context of system identification. These models are: Memory Polynomial (MP) and Modified Generalized Memory Polynomial (MGMP), which can be considered as a generalization of Hammerstein and Wiener models, respectively. Both of them are appealing representations as they allow to describe larger model sets with less parametric complexity. Simulation example is given to illustrate the quality of the obtained models.     

Nonparametric conditional autoregressive expectile model via neural network with applications to estimating financial risk

The parametric conditional autoregressive expectiles (CARE) models have been developed to estimate expectiles, which can be used to assess value at risk and expected shortfall. The challenge lies in parametric CARE modeling is the specification of a parametric form. To avoid any model misspecification, we propose a nonparametric CARE model via neural network. The nonparametric CARE model can be estimated by a classical gradient based nonlinear optimization algorithm, and the consistency of nonparametric conditional expectile estimators is established. We then apply the nonparametric CARE model to estimating value at risk and expected shortfall of six stock indices. Empirical results for the new model is competitive with those classical models and parametric CARE models. Copyright © 2016 John Wiley & Sons, Ltd.     

A new distribution-free adaptive sample size control chart for a finite production horizon and its application in monitoring fill volume of soft drink beverage bottles

Nonparametric control charts have received increasing attention in process monitoring. In this article, a new nonparametric sign (SN) control chart with variable sample size (VSS) for a finite horizon process is developed. The novelty of this research lies in the incorporation of the VSS technique into the nonparametric SN chart for a finite horizon process, hence, resulting in the development of a more sensitive nonparametric short run chart. The statistical performance of the new nonparametric VSS SN control chart is evaluated and compared with the existing fixed sample size (FSS) SN chart for a finite horizon process. The charts' performances are compared using the truncated average run length (TARL) and truncated standard deviation of the run length (TSDRL) criteria. The results obtained show that the nonparametric VSS SN short run chart is always quicker than the FSS SN short run chart in detecting process shifts for various underlying process distributions, hence, reducing scrap and rework cost. Finally, an application of the proposed control charting scheme is shown through a real-life example on the fill volume of soft drink beverage bottles.     

Covariate selection for semiparametric hazard function regression models

We study a flexible class of nonproportional hazard function regression models in which the influence of the covariates splits into the sum of a parametric part and a time-dependent nonparametric part. We develop a method of covariate selection for the parametric part by adjusting for the implicit fitting of the nonparametric part. Asymptotic consistency of the proposed covariate selection method is established, leading to asymptotically normal estimators of both parametric and nonparametric parts of the model in the presence of covariate selection. The approach is applied to a real data set and a simulation study is presented.     

半参数广义线性混合效应模型的估计及其渐近性质

半参数广义线性混合效应模型在心理学、生物育种、医学等领域有广泛的应用. Zhang(1998)用最大惩罚似然函数的方法(MPLE)对模型的参数和非参数部分进行了估计, 而Zhang (1998) MPLE方法只适用于正态数据模型. 对于泊松等常用的模型, 常的方法是将随机效应看作缺失数据, 再引入EM算法. 本文基于McCulloch 1997)提出的MCNR算法, 此算法推广到半参数广义线性混合效应模型中并得到相应的估计算法. 于非参数部分, 本文采用P样条拟合并利用GCV方法选取光滑参数, 时证明了所得估计的相合性和渐近正态性. 最后, 过模拟和实例与其它算法作比较验证本文估计方法的有效性.     

Semiparametric mixtures of regressions with single-index for model based clustering

In this article, we propose two classes of semiparametric mixture regression models with single-index for model based clustering. Unlike many semiparametric/nonparametric mixture regression models that can only be applied to low dimensional predictors, the new semiparametric models can easily incorporate high dimensional predictors into the nonparametric components. The proposed models are very general, and many of the recently proposed semiparametric/nonparametric mixture regression models are indeed special cases of the new models. Backfitting estimates and the corresponding modified EM algorithms are proposed to achieve optimal convergence rates for both parametric and nonparametric parts. We establish the identifiability results of the proposed two models and investigate the asymptotic properties of the proposed estimation procedures. Simulation studies are conducted to demonstrate the finite sample performance of the proposed models. Two real data applications using the new models reveal some interesting findings.

     

Nonparametric likelihood ratio goodness-of-fit tests for survival data

Berk and Jones (Z. Wahrsch. Verw. Gebiete 47 (1979) 47) described a nonparametric likelihood test of uniformity that is more efficient, in Bahadur's sense, than any weighted Kolmogorov-Smirnov test at any alternative. This article shows how to obtain a nonparametric likelihood test of a general parametric family for incomplete survival data. A nonparametric likelihood ratio test process is employed to measure the discrepancy between a parametric family and the observed data. Large sample properties of the likelihood ratio test process are studied under both the null and alternative hypotheses. A Monte Carlo simulation method is proposed to estimate its null distribution. We show how to produce a likelihood ratio graphical check as well as a formal test of a parametric family based on the developed theory. Our method is developed for the right-censorship model, but can be easily extended to some other survival models. Illustrations are given using both real and simulated data.     

Nonparametric Bayesian topic modelling with the hierarchical Pitman–Yor processes

The Dirichlet process and its extension, the Pitman–Yor process, are stochastic processes that take probability distributions as a parameter. These processes can be stacked up to form a hierarchical nonparametric Bayesian model. In this article, we present efficient methods for the use of these processes in this hierarchical context, and apply them to latent variable models for text analytics. In particular, we propose a general framework for designing these Bayesian models, which are called topic models in the computer science community. We then propose a specific nonparametric Bayesian topic model for modelling text from social media. We focus on tweets (posts on Twitter) in this article due to their ease of access. We find that our nonparametric model performs better than existing parametric models in both goodness of fit and real world applications.     

Functional inference in semiparametric models using the piggyback bootstrap

This paper introduces the “piggyback bootstrap.” Like the weighted bootstrap, this bootstrap procedure can be used to generate random draws that approximate the joint sampling distribution of the parametric and nonparametric maximum likelihood estimators in various semiparametric models, but the dimension of the maximization problem for each bootstrapped likelihood is smaller. This reduction results in significant computational savings in comparison to the weighted bootstrap. The procedure can be stated quite simply. First obtain a valid random draw for the parametric component of the model. Then take the draw for the nonparametric component to be the maximizer of the weighted bootstrap likelihood with the parametric component fixed at the parametric draw. We prove the procedure is valid for a class of semiparametric models that includes frailty regression models airsing in survival analysis and biased sampling models that have application to vaccine efficacy trials. Bootstrap confidence sets from the piggyback, and weighted bootstraps are compared for biased sampling data from simulated vaccine efficacy trials.     

Nonlinear parametric models from volterra kernels measurements

Methods for nonlinear system identification are often classified, based on the employed model form, into parametric (nonlinear differential or difference equations) and nonparametric (functional expansions). These methods exhibit distinct sets of advantages and disadvantages that have motivated comparative studies and point to potential benefits from combined use. Fundamental to these studies are the mathematical relations between nonlinear differential (or difference, in discrete time) equations (NDE) and Volterra functional expansions (VFE) of the class of nonlinear systems for which both model forms exist, in continuous or discrete time. Considerable work has been done in obtaining the VFE's of a broad class of NDE's, which can be used to make the transition from nonparametric models (obtained from experimental input-output data) to more compact parametric models. This paper presents a methodology by which this transition can be made in discrete time. Specifically, a method is proposed for obtaining a parametric NARMAX (Nonlinear Auto-Regressive Moving-Average with exogenous input) model from Volterra kernels estimated by use of input-output data.     

Empirical Likelihood Inference for Functional Coefficient ARCH-M Model

Empirical likelihood inference for parametric and nonparametric parts in functional coefficient ARCH-M models is investigated in this paper. Firstly, the kernel smoothing technique is used to estimate coefficient function δ(x). In this way we obtain an estimated function with parameter β.Secondly, the empirical likelihood method is developed to estimate the parameter β. An estimated empirical log-likelohood ratio is proved to be asymptotically standard chi-squred, and the maximum empirical likelihood estimation(MELE) for β is shown to be asymptotically normal. Finally, based on the MELE of β, the empirical likelihood approach is again applied to reestimate the nonparametric part δ(x). The empirical log-likelohood ratio for δ(x) is proved to be also asymptotically standard chi-squred. Simulation study shows that the proposed method works better than the normal approximation method in terms of average areas of confidence regions for β, and the empirical likelihood confidence belt for δ(x) performs well.     

Strong Consistency of the Maximum Product of Spacings Estimates with Applications in Nonparametrics and in Estimation of Unimodal Densities

Distributions with unimodal densities are among the most commonly used in practice. However, for many unimodal distribution families the likelihood functions may be unbounded, thereby leading to inconsistent estimates. The maximum product of spacings (MPS) method, introduced by Cheng and Amin and independently by Ranneby, has been known to give consistent and asymptotically normal estimators in many parametric situations where the maximum likelihood method fails. In this paper, strong consistency theorems for the MPS method are obtained under general conditions which are comparable to the conditions of Bahadur and Wang for the maximum likelihood method. The consistency theorems obtained here apply to both parametric models and some nonparametric models. In particular, in any unimodal distribution family the asymptotic MPS estimator of the underlying unimodal density is shown to be universally L¹ consistent without any further conditions (in parametric or nonparametric settings).     

Statistical inference in partially-varying-coefficient single-index model

Consider a varying-coefficient single-index model which consists of two parts: the linear part with varying coefficients and the nonlinear part with a single-index structure, and are hence termed as varying-coefficient single-index models. This model includes many important regression models such as single-index models, partially linear single-index models, varying-coefficient model and varying-coefficient partially linear models as special examples. In this paper, we mainly study estimating problems of the varying-coefficient vector, the nonparametric link function and the unknown parametric vector describing the single-index in the model. A stepwise approach is developed to obtain asymptotic normality estimators of the varying-coefficient vector and the parametric vector, and estimators of the nonparametric link function with a convergence rate. The consistent estimator of the structural error variance is also obtained. In addition, asymptotic pointwise confidence intervals and confidence regions are constructed for the varying coefficients and the parametric vector. The bandwidth selection problem is also considered. A simulation study is conducted to evaluate the proposed methods, and real data analysis is also used to illustrate our methods.     

Nonparametric regression with current status data

We apply nonparametric regression to current status data, which often arises in survival analysis and reliability analysis. While no parametric assumption on the distributions has been imposed, most authors have employed parametric models like linear models to measure the covariate effects on failure times in regression analysis with current status data. We construct a nonparametric estimator of the regression function by modifying the maximum rank correlation (MRC) estimator. Our estimator can deal with the cases where the other estimators do not work. We present the asymptotic bias and the asymptotic distribution of the estimator by adapting a result on equicontinuity of degenerate U-processes to the setup of this paper.     

B-Spline Estimation for Varying-Coefficient Single-Index Model

The varying-coefficient single-index models (VCSIM) have been applied in many fields since they combine the advantages of single-index models and varying-coefficient models. In this paper, their estimation method is proposed based on B-spline approximation technique and two calculation methods can be used. The first one is to directly calculate the parametric and nonparametric parts simultaneously by Newton-Raphson iteration algorithm. The second one is to calculate the two parts by profile method individually. We suggest that the second method is for our preference when the large amount of parameters are involved, otherwise the first method will be more convenient. Two simulated examples are given to illustrate the performances of the proposed estimation methodologies and calculation procedures.     

Bayesian semiparametric analysis for a single item maintenance optimization

We address the problem of a finite horizon single item maintenance optimization structured as a combination of preventive and corrective maintenance in a nuclear power plant environment. We present Bayesian semiparametric models to estimate the failure time distribution and costs involved. The objective function for the optimization is the expected total cost of maintenance over the pre-defined finite time horizon. Typically, the mathematical modeling of failure times are based on parametric models. These models fail to capture the true underlying relationships in the data; indeed, under a parametric assumption, the hazard rates are treated as unimodal, which, as shown in this paper, is incorrect. Importantly, assuming an increasing failure rate, as is typically done, we show, is way off the mark in the present context. Since hazard and cost estimates feed into the optimization phase, from a risk management perspective, potentially gross errors, resulting from purely parametric models, can be obviated. We show the effectiveness of our approach using real data from the South Texas Project Nuclear Operating Company (STPNOC) located in Bay City, Texas.     

A two-stage approach to semilinear in-slide models

The semilinear in-slide models (SLIMs) have been shown to be effective methods for normalizing microarray data [J. Fan, P. Tam, G. Vande Woude, Y. Ren, Normalization and analysis of cDNA micro-arrays using within-array replications applied to neuroblastoma cell response to a cytokine, Proceedings of the National Academy of Science (2004) 1135-1140]. Using a backfitting method, [J. Fan, H. Peng, T. Huang, Semilinear high-dimensional model for normalization of microarray data: a theoretical analysis and partial consistency, Journal of American Statistical Association, 471, (2005) 781-798] proposed a profile least squares (PLS) estimation for the parametric and nonparametric components. The general asymptotic properties for their estimator is not developed. In this paper, we consider a new approach, two-stage estimation, which enables us to establish the asymptotic normalities for both of the parametric and nonparametric component estimators. We further propose a plug-in bandwidth selector using the asymptotic normality of the nonparametric component estimator. The proposed method allow for the modeling of the aggregated SLIMs case where we can explicitly show that taking the aggregated information into account can improve both of the parametric and nonparametric component estimator by the proposed two-stage approach. Some simulation studies are conducted to illustrate the finite sample performance of the proposed procedures.     

检验两个部分线性模型中非参函数相等

We propose the test statistic to check whether the nonparametric func-tions in two partially linear models are equality or not in this paper. We estimate the nonparametric function both in null hypothesis and the alternative by the local linear method, where we ignore the parametric components, and then estimate the parameters by the two stage method. The test statistic is derived, and it is shown to be asymptotically normal under the null hypothesis.     

Diffusion and survival models for the process of entry into marriage

In this paper different survival models with a non‐parametric hazard rate function are applied to the process of entry into marriage. The hazard function of the Hernes model of the marriage process as well as the log‐logistic survival model are both derivable from a differential equation for a social diffusion process. The log‐logistic model might be appropriate for modelling the marriage rate because of its non‐monotonic hazard function. Moreover, in light of our analysis, application of this model to the marriage process can be justified by the theoretical rationale of a process of social diffusion. Both models are tested using German age‐at‐marriage data and the U.S. cohort data analyzed by Hernes (1972). It can be shown that the three parameter Hernes model yields a good fit while the log‐logistic model, with only two parameters, leads to a moderate approximation of the data.