Propagation-Separation Approach for Local Likelihood Estimation

The paper presents a unified approach to local likelihood estimation for a broad class of nonparametric models, including e.g. the regression, density, Poisson and binary response model. The method extends the adaptive weights smoothing (AWS) procedure introduced in Polzehl and Spokoiny (2000) in context of image denoising. The main idea of the method is to describe a greatest possible local neighborhood of every design point X_i in which the local parametric assumption is justified by the data. The method is especially powerful for model functions having large homogeneous regions and sharp discontinuities. The performance of the proposed procedure is illustrated by numerical examples for density estimation and classification. We also establish some remarkable theoretical nonasymptotic results on properties of the new algorithm. This includes the ``propagation' property which particularly yields the root-n consistency of the resulting estimate in the homogeneous case. We also state an ``oracle' result which implies rate optimality of the estimate under usual smoothness conditions and a ``separation' result which explains the sensitivity of the method to structural changes.     

Nonparametric variable selection and its application to additive models

Variable selection for multivariate nonparametric regression models usually involves parameterized approximation for nonparametric functions in the objective function. However, this parameterized approximation often increases the number of parameters significantly, leading to the “curse of dimensionality” and inaccurate estimation. In this paper, we propose a novel and easily implemented approach to do variable selection in nonparametric models without parameterized approximation, enabling selection consistency to be achieved. The proposed method is applied to do variable selection for additive models. A two-stage procedure with selection and adaptive estimation is proposed, and the properties of this method are investigated. This two-stage algorithm is adaptive to the smoothness of the underlying components, and the estimation consistency can reach a parametric rate if the underlying model is really parametric. Simulation studies are conducted to examine the performance of the proposed method. Furthermore, a real data example is analyzed for illustration.

     

Covariate selection for semiparametric hazard function regression models

We study a flexible class of nonproportional hazard function regression models in which the influence of the covariates splits into the sum of a parametric part and a time-dependent nonparametric part. We develop a method of covariate selection for the parametric part by adjusting for the implicit fitting of the nonparametric part. Asymptotic consistency of the proposed covariate selection method is established, leading to asymptotically normal estimators of both parametric and nonparametric parts of the model in the presence of covariate selection. The approach is applied to a real data set and a simulation study is presented.     

On sparse estimation for semiparametric linear transformation models

Semiparametric linear transformation models have received much attention due to their high flexibility in modeling survival data. A useful estimating equation procedure was recently proposed by Chen et al. (2002) [21] for linear transformation models to jointly estimate parametric and nonparametric terms. They showed that this procedure can yield a consistent and robust estimator. However, the problem of variable selection for linear transformation models has been less studied, partially because a convenient loss function is not readily available under this context. In this paper, we propose a simple yet powerful approach to achieve both sparse and consistent estimation for linear transformation models. The main idea is to derive a profiled score from the estimating equation of Chen et al. [21], construct a loss function based on the profile scored and its variance, and then minimize the loss subject to some shrinkage penalty. Under regularity conditions, we have shown that the resulting estimator is consistent for both model estimation and variable selection. Furthermore, the estimated parametric terms are asymptotically normal and can achieve a higher efficiency than that yielded from the estimation equations. For computation, we suggest a one-step approximation algorithm which can take advantage of the LARS and build the entire solution path efficiently. Performance of the new procedure is illustrated through numerous simulations and real examples including one microarray data.     

Statistical inference using a weighted difference-based series approach for partially linear regression models

Partially linear regression models with fixed effects are useful tools for making econometric analyses and normalizing microarray data. Baltagi and Li (2002) [7] proposed a computation friendly difference-based series estimation (DSE) for them. We show that the DSE is not asymptotically efficient in most cases and further propose a weighted difference-based series estimation (WDSE). The weights in it do not involve any unknown parameters. The asymptotic properties of the resulting estimators are established for both balanced and unbalanced cases, and it is shown that they achieve a semiparametric efficient boundary. Additionally, we propose a variable selection procedure for identifying significant covariates in the parametric part of the semiparametric fixed-effects regression model. The method is based on a combination of the nonconcave penalization (Fan and Li, 2001 [13]) and weighted difference-based series estimation techniques. The resulting estimators have the oracle property; that is, they can correctly identify the true model as if the true model (the subset of variables with nonvanishing coefficients) were known in advance. Simulation studies are conducted and an application is given to demonstrate the finite sample performance of the proposed procedures.     

Semi-varying coefficient models with a diverging number of components

Semiparametric models with both nonparametric and parametric components have become increasingly useful in many scientific fields, due to their appropriate representation of the trade-off between flexibility and efficiency of statistical models. In this paper we focus on semi-varying coefficient models (a.k.a. varying coefficient partially linear models) in a “large n, diverging p” situation, when both the number of parametric and nonparametric components diverges at appropriate rates, and we only consider the case p=o(n). Consistency of the estimator based on B-splines and asymptotic normality of the linear components are established under suitable assumptions. Interestingly (although not surprisingly) our analysis shows that the number of parametric components can diverge at a faster rate than the number of nonparametric components and the divergence rates of the number of the nonparametric components constrain the allowable divergence rates of the parametric components, which is a new phenomenon not established in the existing literature as far as we know. Finally, the finite sample behavior of the estimator is evaluated by some Monte Carlo studies.     

Statistical inference in partially-varying-coefficient single-index model

Consider a varying-coefficient single-index model which consists of two parts: the linear part with varying coefficients and the nonlinear part with a single-index structure, and are hence termed as varying-coefficient single-index models. This model includes many important regression models such as single-index models, partially linear single-index models, varying-coefficient model and varying-coefficient partially linear models as special examples. In this paper, we mainly study estimating problems of the varying-coefficient vector, the nonparametric link function and the unknown parametric vector describing the single-index in the model. A stepwise approach is developed to obtain asymptotic normality estimators of the varying-coefficient vector and the parametric vector, and estimators of the nonparametric link function with a convergence rate. The consistent estimator of the structural error variance is also obtained. In addition, asymptotic pointwise confidence intervals and confidence regions are constructed for the varying coefficients and the parametric vector. The bandwidth selection problem is also considered. A simulation study is conducted to evaluate the proposed methods, and real data analysis is also used to illustrate our methods.     

A two-stage approach to semilinear in-slide models

The semilinear in-slide models (SLIMs) have been shown to be effective methods for normalizing microarray data [J. Fan, P. Tam, G. Vande Woude, Y. Ren, Normalization and analysis of cDNA micro-arrays using within-array replications applied to neuroblastoma cell response to a cytokine, Proceedings of the National Academy of Science (2004) 1135-1140]. Using a backfitting method, [J. Fan, H. Peng, T. Huang, Semilinear high-dimensional model for normalization of microarray data: a theoretical analysis and partial consistency, Journal of American Statistical Association, 471, (2005) 781-798] proposed a profile least squares (PLS) estimation for the parametric and nonparametric components. The general asymptotic properties for their estimator is not developed. In this paper, we consider a new approach, two-stage estimation, which enables us to establish the asymptotic normalities for both of the parametric and nonparametric component estimators. We further propose a plug-in bandwidth selector using the asymptotic normality of the nonparametric component estimator. The proposed method allow for the modeling of the aggregated SLIMs case where we can explicitly show that taking the aggregated information into account can improve both of the parametric and nonparametric component estimator by the proposed two-stage approach. Some simulation studies are conducted to illustrate the finite sample performance of the proposed procedures.     

Variable selection for semiparametric varying coefficient partially linear errors-in-variables models

This paper focuses on the variable selections for semiparametric varying coefficient partially linear models when the covariates in the parametric and nonparametric components are all measured with errors. A bias-corrected variable selection procedure is proposed by combining basis function approximations with shrinkage estimations. With appropriate selection of the tuning parameters, the consistency of the variable selection procedure and the oracle property of the regularized estimators are established. A simulation study and a real data application are undertaken to evaluate the finite sample performance of the proposed method.     

Nonparametric methods for unbalanced multivariate data and many factor levels

We propose different nonparametric tests for multivariate data and derive their asymptotic distribution for unbalanced designs in which the number of factor levels tends to infinity (large a, small n_i case). Quasi gratis, some new parametric multivariate tests suitable for the large a asymptotic case are also obtained. Finite sample performances are investigated and compared in a simulation study. The nonparametric tests are based on separate rankings for the different variables. In the presence of outliers, the proposed nonparametric methods have better power than their parametric counterparts. Application of the new tests is demonstrated using data from plant pathology.     

A Computational Approach for Full Nonparametric Bayesian Inference Under Dirichlet Process Mixture Models

Widely used parametric generalized linear models are, unfortunately, a somewhat limited class of specifications. Nonparametric aspects are often introduced to enrich this class, resulting in semiparametric models. Focusing on single or k-sample problems, many classical nonparametric approaches are limited to hypothesis testing. Those that allow estimation are limited to certain functionals of the underlying distributions. Moreover, the associated inference often relies upon asymptotics when nonparametric specifications are often most appealing for smaller sample sizes. Bayesian nonparametric approaches avoid asymptotics but have, to date, been limited in the range of inference. Working with Dirichlet process priors, we overcome the limitations of existing simulation-based model fitting approaches which yield inference that is confined to posterior moments of linear functionals of the population distribution. This article provides a computational approach to obtain the entire posterior distribution for more general functionals. We illustrate with three applications: investigation of extreme value distributions associated with a single population, comparison of medians in a k-sample problem, and comparison of survival times from different populations under fairly heavy censoring.     

Asymptotic distributions of two “synthetic data” estimators for censored single-index models

The censored single-index model provides a flexible way for modelling the association between a response and a set of predictor variables when the response variable is randomly censored and the link function is unknown. It presents a technique for “dimension reduction” in semiparametric censored regression models and generalizes the existing accelerated failure time models for survival analysis. This paper proposes two methods for estimation of single-index models with randomly censored samples. We first transform the censored data into synthetic data or pseudo-responses unbiasedly, then obtain estimates of the index coefficients by the rOPG or rMAVE procedures of Xia (2006) [1]. Finally, we estimate the unknown nonparametric link function using techniques for univariate censored nonparametric regression. The estimators for the index coefficients are shown to be root-n consistent and asymptotically normal. In addition, the estimator for the unknown regression function is a local linear kernel regression estimator and can be estimated with the same efficiency as the parameters are known. Monte Carlo simulations are conducted to illustrate the proposed methodologies.     

Functional semiparametric partially linear model with autoregressive errors

In this paper, we introduce a functional semiparametric model, where a real-valued random variable is explained by the sum of a unknown linear combination of the components of a multivariate random variable and an unknown transformation of a functional random variable. The errors can be autocorrelated. We focus here on the parametric estimation of the coefficients in the linear combination. First, we use a nonparametric kernel method to remove the effect of the functional explanatory variable. Then, we use generalized least squares approach to obtain an estimator of these coefficients. Under some technical assumptions, we prove consistency and asymptotic normality of our estimator. Finally, we present Monte Carlo simulations that illustrate these characteristics.     

Nonlinear weighted least squares estimation of a three-parameter Weibull density with a nonparametric start

This paper is concerned with the parameter estimation problem for the three-parameter Weibull density which is widely employed as a model in reliability and lifetime studies. Our approach is a combination of nonparametric and parametric methods. The basic idea is to start with an initial nonparametric density estimate which needs to be as good as possible, and then apply the nonlinear least squares method to estimate the unknown parameters. As a main result, a theorem on the existence of the least squares estimate is obtained. Some simulations are given to show that our approach is satisfactory if the initial density is of good enough quality.     

Asymptotic efficient estimation in semiparametric nonlinear regression models

§１ＩｎｔｒｏｄｕｃｔｉｏｎＣｏｎｓｉｄｅｒｔｈｅｆｉｘｅｄｄｅｓｉｇｎｓｅｍｉｐａｒａｍｅｔｒｉｃｎｏｎｌｉｎｅａｒｒｅｇｒｅｓｓｉｏｎｍｏｄｅｌｓｇｉｖｅｎｂｙｙｉ＝ｆ（ｘｉ,θ）＋λ（ｔｉ）＋εｉ,ｉ＝１,．．．,ｎ,（１）ｗｈｅｒｅｆ（,）ｉ...     

Statistical inference for panel data semiparametric partially linear regression models with heteroscedastic errors

We consider a panel data semiparametric partially linear regression model with an unknown parameter vector for the linear parametric component, an unknown nonparametric function for the nonlinear component, and a one-way error component structure which allows unequal error variances (referred to as heteroscedasticity). We develop procedures to detect heteroscedasticity and one-way error component structure, and propose a weighted semiparametric least squares estimator (WSLSE) of the parametric component in the presence of heteroscedasticity and/or one-way error component structure. This WSLSE is asymptotically more efficient than the usual semiparametric least squares estimator considered in the literature. The asymptotic properties of the WSLSE are derived. The nonparametric component of the model is estimated by the local polynomial method. Some simulations are conducted to demonstrate the finite sample performances of the proposed testing and estimation procedures. An example of application on a set of panel data of medical expenditures in Australia is also illustrated.     

Robust and efficient variable selection for semiparametric partially linear varying coefficient model based on modal regression

Semiparametric partially linear varying coefficient models (SPLVCM) are frequently used in statistical modeling. With high-dimensional covariates both in parametric and nonparametric part for SPLVCM, sparse modeling is often considered in practice. In this paper, we propose a new estimation and variable selection procedure based on modal regression, where the nonparametric functions are approximated by $B$ -spline basis. The outstanding merit of the proposed variable selection procedure is that it can achieve both robustness and efficiency by introducing an additional tuning parameter (i.e., bandwidth $h$ ). Its oracle property is also established for both the parametric and nonparametric part. Moreover, we give the data-driven bandwidth selection method and propose an EM-type algorithm for the proposed method. Monte Carlo simulation study and real data example are conducted to examine the finite sample performance of the proposed method. Both the simulation results and real data analysis confirm that the newly proposed method works very well.     

Local polynomial maximum likelihood estimation for Pareto-type distributions

We discuss the estimation of the tail index of a heavy-tailed distribution when covariate information is available. The approach followed here is based on the technique of local polynomial maximum likelihood estimation. The generalized Pareto distribution is fitted locally to exceedances over a high specified threshold. The method provides nonparametric estimates of the parameter functions and their derivatives up to the degree of the chosen polynomial. Consistency and asymptotic normality of the proposed estimators will be proven under suitable regularity conditions. This approach is motivated by the fact that in some applications the threshold should be allowed to change with the covariates due to significant effects on scale and location of the conditional distributions. Using the asymptotic results we are able to derive an expression for the asymptotic mean squared error, which can be used to guide the selection of the bandwidth and the threshold. The applicability of the method will be demonstrated with a few practical examples.     

Variable selection via generalized SELO-penalized linear regression models

The seamless-L_0(SELO) penalty is a smooth function on [0, ∞) that very closely resembles the L_0 penalty, which has been demonstrated theoretically and practically to be effective in nonconvex penalization for variable selection. In this paper, we first generalize SELO to a class of penalties retaining good features of SELO, and then propose variable selection and estimation in linear models using the proposed generalized SELO(GSELO) penalized least squares(PLS) approach. We show that the GSELO-PLS procedure possesses the oracle property and consistently selects the true model under some regularity conditions in the presence of a diverging number of variables. The entire path of GSELO-PLS estimates can be efficiently computed through a smoothing quasi-Newton(SQN) method. A modified BIC coupled with a continuation strategy is developed to select the optimal tuning parameter. Simulation studies and analysis of a clinical data are carried out to evaluate the finite sample performance of the proposed method. In addition, numerical experiments involving simulation studies and analysis of a microarray data are also conducted for GSELO-PLS in the high-dimensional settings.     

Nonparametric Bayesian Assessment of the Order of Dependence for Binary Sequences

This article discusses inference on the order of dependence in binary sequences. The proposed approach is based on the notion of partial exchangeability of order k. A partially exchangeable binary sequence of order k can be represented as a mixture of Markov chains. The mixture is with respect to the unknown transition probability matrix θ. We use this defining property to construct a semiparametric model for binary sequences by assuming a nonparametric prior on the transition matrix θ. This enables us to consider inference on the order of dependence without constraint to a particular parametric model. Implementing posterior simulation in the proposed model is complicated by the fact that the dimension of θ changes with the order of dependence k. We discuss appropriate posterior simulation schemes based on a pseudo prior approach. We extend the model to include covariates by considering an alternative parameterization as an autologistic regression which allows for a straightforward introduction of covariates. The regression on covariates raises the additional inference problem of variable selection. We discuss appropriate posterior simulation schemes, focusing on inference about the order of dependence. We discuss and develop the model with covariates only to the extent needed for such inference.