The nonparametric estimation of long memory spatio-temporal random field models

This paper considers the local linear estimation of a multivariate regression function and its derivatives for a stationary long memory(long range dependent) nonparametric spatio-temporal regression model.Under some mild regularity assumptions, the pointwise strong convergence, the uniform weak consistency with convergence rates and the joint asymptotic distribution of the estimators are established. A simulation study is carried out to illustrate the performance of the proposed estimators.     

Censored multiple regression by the method of average derivatives

This paper proposes a technique [termed censored average derivative estimation (CADE)] for studying estimation of the unknown regression function in nonparametric censored regression models with randomly censored samples. The CADE procedure involves three stages: firstly-transform the censored data into synthetic data or pseudo-responses using the inverse probability censoring weighted (IPCW) technique, secondly estimate the average derivatives of the regression function, and finally approximate the unknown regression function by an estimator of univariate regression using techniques for one-dimensional nonparametric censored regression. The CADE provides an easily implemented methodology for modelling the association between the response and a set of predictor variables when data are randomly censored. It also provides a technique for “dimension reduction” in nonparametric censored regression models. The average derivative estimator is shown to be root-n consistent and asymptotically normal. The estimator of the unknown regression function is a local linear kernel regression estimator and is shown to converge at the optimal one-dimensional nonparametric rate. Monte Carlo experiments show that the proposed estimators work quite well.     

DIMENSION-REDUCTION TYPE TESTFOR LINEARITY OF ASTOCHASTIC REGRESSION MODEL

1.IntroductionLinearregressionmodelsarewidelyusedinstatisticalanalysisofexperimentalandobservationaldata,thatis,oneoftenemploysastandardlinearmodely=or K: E,a.s.,(1.1)todostatisticalanalysis,whereydenotesascalaroutcomevariableand2denotesaP-dimensionalcolumnvectorofregressorvariables.Thismodelmeansthattheprojectionofthepdimensionalexplanatory2ontotheone-dimensionalsubspaceadZcapturesalltheinformationweneedtoknowabouttheoutcomevariabley.Thisisadimension-reductionmodel.Hencewemayreachthegoalofd…     

Nonparametric variable selection and its application to additive models

Variable selection for multivariate nonparametric regression models usually involves parameterized approximation for nonparametric functions in the objective function. However, this parameterized approximation often increases the number of parameters significantly, leading to the “curse of dimensionality” and inaccurate estimation. In this paper, we propose a novel and easily implemented approach to do variable selection in nonparametric models without parameterized approximation, enabling selection consistency to be achieved. The proposed method is applied to do variable selection for additive models. A two-stage procedure with selection and adaptive estimation is proposed, and the properties of this method are investigated. This two-stage algorithm is adaptive to the smoothness of the underlying components, and the estimation consistency can reach a parametric rate if the underlying model is really parametric. Simulation studies are conducted to examine the performance of the proposed method. Furthermore, a real data example is analyzed for illustration.

     

Consistency of the regression estimator with functional data under long memory conditions

We study the nonparametric regression estimation when the explanatory variable takes values in some abstract functional space. We establish some asymptotic results and we give the (pointwise and uniform) convergence of the kernel type estimator constructed from functional data under long memory conditions.     

Heteroscedasticity checks for regression models

For checking on heteroscedasticity in regression models, a unified approach is proposed to constructing test statistics in parametric and nonparametric regression models. For nonparametric regression, the test is not affected sensitively by the choice of smoothing parameters which are involved in estimation of the nonparametric regression function. The limiting null distribution of the test statistic remains the same in a wide range of the smoothing parameters. When the covariate is one-dimensional, the tests are, under some conditions, asymptotically distribution-free. In the high-dimensional cases, the validity of bootstrap approximations is investigated. It is shown that a variant of the wild bootstrap is consistent while the classical bootstrap is not in the general case, but is applicable if some extra assumption on conditional variance of the squared error is imposed. A simulation study is performed to provide evidence of how the tests work and compare with tests that have appeared in the literature. The approach may readily be extended to handle partial linear, and linear autoregressive models.     

��ָ���λ���ع�ı���ѡ��

多元非参数分位数回归常常是难于估计的, 为了降低维数同时保持非参数估计的灵活性, 人们常常用单指标的方法模拟响应变量的条件分位数. 本文主要研究单指标分位数回归的变量选择. 以最小化平均损失估计为基础, 我们通过最小化具有SCAD惩罚项的平均损失进行变量选择和参数估计. 在正则条件下, 得到了单指标分位数回归SCAD变量选择的Oracle性质, 给出了SCAD变量选择的计算方法, 并通过模拟研究说明了本文所提方法变量选择的样本性质.     

Variance function estimation in multivariate nonparametric regression with fixed design

Variance function estimation in multivariate nonparametric regression is considered and the minimax rate of convergence is established in the iid Gaussian case. Our work uses the approach that generalizes the one used in [A. Munk, Bissantz, T. Wagner, G. Freitag, On difference based variance estimation in nonparametric regression when the covariate is high dimensional, J. R. Stat. Soc. B 67 (Part 1) (2005) 19-41] for the constant variance case. As is the case when the number of dimensions d=1, and very much contrary to standard thinking, it is often not desirable to base the estimator of the variance function on the residuals from an optimal estimator of the mean. Instead it is desirable to use estimators of the mean with minimal bias. Another important conclusion is that the first order difference based estimator that achieves minimax rate of convergence in the one-dimensional case does not do the same in the high dimensional case. Instead, the optimal order of differences depends on the number of dimensions.     

Nonparametric estimation of location parameter after a preliminary test on regression in the multivariate case

For a simple multivariate regression model, nonparametric estimation of the (vector of) intercept following a preliminary test on the regression vector is considered. Along with the asymptotic distribution of these estimators, their asymptotic bias and dispersion matrices are studied and allied efficiency results are presented.     

Estimating the error distribution in nonparametric multiple regression with applications to model testing

In this paper we consider the estimation of the error distribution in a heteroscedastic nonparametric regression model with multivariate covariates. As estimator we consider the empirical distribution function of residuals, which are obtained from multivariate local polynomial fits of the regression and variance functions, respectively. Weak convergence of the empirical residual process to a Gaussian process is proved. We also consider various applications for testing model assumptions in nonparametric multiple regression. The model tests obtained are able to detect local alternatives that converge to zero at an n^−1/2-rate, independent of the covariate dimension. We consider in detail a test for additivity of the regression function.     

Error Process Indexed by Bandwidth Matrices in Multivariate Local Linear Smoothing

We focus on nonparametric multivariate regression function estimation by locally weighted least squares. The asymptotic behavior for a sequence of error processes indexed by bandwidth matrices is derived. We discuss feasible data-driven consistent estimators minimizing asymptotic mean squared error or efficient estimators reducing asymptotic bias at points where opposite sign curvatures of the regression function are present in different directions.     

Bayesian Nonparametric Modeling for Multivariate Ordinal Regression

Univariate or multivariate ordinal responses are often assumed to arise from a latent continuous parametric distribution, with covariate effects that enter linearly. We introduce a Bayesian nonparametric modeling approach for univariate and multivariate ordinal regression, which is based on mixture modeling for the joint distribution of latent responses and covariates. The modeling framework enables highly flexible inference for ordinal regression relationships, avoiding assumptions of linearity or additivity in the covariate effects. In standard parametric ordinal regression models, computational challenges arise from identifiability constraints and estimation of parameters requiring nonstandard inferential techniques. A key feature of the nonparametric model is that it achieves inferential flexibility, while avoiding these difficulties. In particular, we establish full support of the nonparametric mixture model under fixed cut-off points that relate through discretization the latent continuous responses with the ordinal responses. The practical utility of the modeling approach is illustrated through application to two datasets from econometrics, an example involving regression relationships for ozone concentration, and a multirater agreement problem. Supplementary materials with technical details on theoretical results and on computation are available online.     

On confidence bands for multivariate nonparametric regression

In a multivariate nonparametric regression problem with fixed, deterministic design asymptotic, uniform confidence bands for the regression function are constructed. The construction of the bands is based on the asymptotic distribution of the maximal deviation between a suitable nonparametric estimator and the true regression function which is derived by multivariate strong approximation methods and a limit theorem for the supremum of a stationary Gaussian field over an increasing system of sets. The results are derived for a general class of estimators which includes local polynomial estimators as a special case. The finite sample properties of the proposed asymptotic bands are investigated by means of a small simulation study.     

Semiparametric mixtures of regressions with single-index for model based clustering

In this article, we propose two classes of semiparametric mixture regression models with single-index for model based clustering. Unlike many semiparametric/nonparametric mixture regression models that can only be applied to low dimensional predictors, the new semiparametric models can easily incorporate high dimensional predictors into the nonparametric components. The proposed models are very general, and many of the recently proposed semiparametric/nonparametric mixture regression models are indeed special cases of the new models. Backfitting estimates and the corresponding modified EM algorithms are proposed to achieve optimal convergence rates for both parametric and nonparametric parts. We establish the identifiability results of the proposed two models and investigate the asymptotic properties of the proposed estimation procedures. Simulation studies are conducted to demonstrate the finite sample performance of the proposed models. Two real data applications using the new models reveal some interesting findings.

     

Estimation of additive quantile regression

We consider the nonparametric estimation problem of conditional regression quantiles with high-dimensional covariates. For the additive quantile regression model, we propose a new procedure such that the estimated marginal effects of additive conditional quantile curves do not cross. The method is based on a combination of the marginal integration technique and non-increasing rearrangements, which were recently introduced in the context of estimating a monotone regression function. Asymptotic normality of the estimates is established with a one-dimensional rate of convergence and the finite sample properties are studied by means of a simulation study and a data example.     

Asymptotic properties of local polynomial regression with missing data and correlated errors

The main objective of this work is the nonparametric estimation of the regression function with correlated errors when observations are missing in the response variable. Two nonparametric estimators of the regression function are proposed. The asymptotic properties of these estimators are studied; expresions for the bias and the variance are obtained and the joint asymptotic normality is established. A simulation study is also included.     

Multivariate Adaptive Splines for Analysis of Longitudinal Data

Abstract

An essential feature of longitudinal data is the existence of autocorrelation among the observations from the same unit or subject. Two-stage random-effects linear models are commonly used to analyze longitudinal data. These models are not flexible enough, however, for exploring the underlying data structures and, especially, for describing time trends. Semi-parametric models have been proposed recently to accommodate general time trends. But these semi-parametric models do not provide a convenient way to explore interactions among time and other covariates although such interactions exist in many applications. Moreover, semi-parametric models require specifying the design matrix of the covariates (time excluded). We propose nonparametric models to resolve these issues. To fit nonparametric models, we use the novel technique of the multivariate adaptive regression splines for the estimation of mean curve and then apply an EM-like iterative procedure for covariance estimation. After giving a general algorithm of model building, we show how to design a fast algorithm. We use both simulated and published data to illustrate the use of our proposed method.     

Covariate-Adjusted Nonparametric Regression for Time Series

The covariate-adjusted regression model was initially proposed for the situations where both the predictors and the response variables are not directly observed, but are distorted by some common observable covariates. In this paper, we investigate a covariate-adjusted nonparametric regression (CANR) model and consider the proposed model on time series setting. We develop a two-step estimation procedure to estimate the regression function. The asymptotic property of the proposed estimation is investigated under the -mixing conditions. Both the real data and simulated examples are provided for illustration.     

Successive direction extraction for estimating the central subspace in a multiple-index regression

In this paper we propose a dimension reduction method for estimating the directions in a multiple-index regression based on information extraction. This extends the recent work of Yin and Cook [X. Yin, R.D. Cook, Direction estimation in single-index regression, Biometrika 92 (2005) 371-384] who introduced the method and used it to estimate the direction in a single-index regression. While a formal extension seems conceptually straightforward, there is a fundamentally new aspect of our extension: We are able to show that, under the assumption of elliptical predictors, the estimation of multiple-index regressions can be decomposed into successive single-index estimation problems. This significantly reduces the computational complexity, because the nonparametric procedure involves only a one-dimensional search at each stage. In addition, we developed a permutation test to assist in estimating the dimension of a multiple-index regression.     

Fast Nonparametric Quantile Regression With Arbitrary Smoothing Methods

The calculation of nonparametric quantile regression curve estimates is often computationally intensive, as typically an expensive nonlinear optimization problem is involved. This article proposes a fast and easy-to-implement method for computing such estimates. The main idea is to approximate the costly nonlinear optimization by a sequence of well-studied penalized least squares-type nonparametric mean regression estimation problems. The new method can be paired with different nonparametric smoothing methods and can also be applied to higher dimensional settings. Therefore, it provides a unified framework for computing different types of nonparametric quantile regression estimates, and it also greatly broadens the scope of the applicability of quantile regression methodology. This wide applicability and the practical performance of the proposed method are illustrated with smoothing spline and wavelet curve estimators, for both uni- and bivariate settings. Results from numerical experiments suggest that estimates obtained from the proposed method are superior to many competitors. This article has supplementary material online.