Statistical inference for panel data semiparametric partially linear regression models with heteroscedastic errors

We consider a panel data semiparametric partially linear regression model with an unknown parameter vector for the linear parametric component, an unknown nonparametric function for the nonlinear component, and a one-way error component structure which allows unequal error variances (referred to as heteroscedasticity). We develop procedures to detect heteroscedasticity and one-way error component structure, and propose a weighted semiparametric least squares estimator (WSLSE) of the parametric component in the presence of heteroscedasticity and/or one-way error component structure. This WSLSE is asymptotically more efficient than the usual semiparametric least squares estimator considered in the literature. The asymptotic properties of the WSLSE are derived. The nonparametric component of the model is estimated by the local polynomial method. Some simulations are conducted to demonstrate the finite sample performances of the proposed testing and estimation procedures. An example of application on a set of panel data of medical expenditures in Australia is also illustrated.     

Statistical inference in a panel data semiparametric regression model with serially correlated errors

We consider a panel data semiparametric partially linear regression model with an unknown vector β of regression coefficients, an unknown nonparametric function g(·) for nonlinear component, and unobservable serially correlated errors. The correlated errors are modeled by a vector autoregressive process which involves a constant intraclass correlation. Applying the pilot estimators of β and g(·), we construct estimators of the autoregressive coefficients, the intraclass correlation and the error variance, and investigate their asymptotic properties. Fitting the error structure results in a new semiparametric two-step estimator of β, which is shown to be asymptotically more efficient than the usual semiparametric least squares estimator in terms of asymptotic covariance matrix. Asymptotic normality of this new estimator is established, and a consistent estimator of its asymptotic covariance matrix is presented. Furthermore, a corresponding estimator of g(·) is also provided. These results can be used to make asymptotically efficient statistical inference. Some simulation studies are conducted to illustrate the finite sample performances of these proposed estimators.     

Quantile regression for longitudinal data

The penalized least squares interpretation of the classical random effects estimator suggests a possible way forward for quantile regression models with a large number of “fixed effects”. The introduction of a large number of individual fixed effects can significantly inflate the variability of estimates of other covariate effects. Regularization, or shrinkage of these individual effects toward a common value can help to modify this inflation effect. A general approach to estimating quantile regression models for longitudinal data is proposed employing ?₁ regularization methods. Sparse linear algebra and interior point methods for solving large linear programs are essential computational tools.     

On an asymptotically more efficient estimation of the single-index model

In this note, we revisit the single-index model with heteroscedastic error, and recommend an estimating equation method in terms of transferring restricted least squares to unrestricted least squares: the estimator of the index parameter is asymptotically more efficient than existing estimators in the literature in the sense that it is of a smaller limiting variance.     

Sparse estimation in functional linear regression

As a useful tool in functional data analysis, the functional linear regression model has become increasingly common and been studied extensively in recent years. In this paper, we consider a sparse functional linear regression model which is generated by a finite number of basis functions in an expansion of the coefficient function. In this model, we do not specify how many and which basis functions enter the model, thus it is not like a typical parametric model where predictor variables are pre-specified. We study a general framework that gives various procedures which are successful in identifying the basis functions that enter the model, and also estimating the resulting regression coefficients in one-step. We adopt the idea of variable selection in the linear regression setting where one adds a weighted L₁ penalty to the traditional least squares criterion. We show that the procedures in our general framework are consistent in the sense of selecting the model correctly, and that they enjoy the oracle property, meaning that the resulting estimators of the coefficient function have asymptotically the same properties as the oracle estimator which uses knowledge of the underlying model. We investigate and compare several methods within our general framework, via a simulation study. Also, we apply the methods to the Canadian weather data.     

Difference based estimation for partially linear regression models with measurement errors

This paper is concerned with the estimating problem of the partially linear regression models where the linear covariates are measured with additive errors. A difference based estimation is proposed to estimate the parametric component. We show that the resulting estimator is asymptotically unbiased and achieves the semiparametric efficiency bound if the order of the difference tends to infinity. The asymptotic normality of the resulting estimator is established as well. Compared with the corrected profile least squares estimation, the proposed procedure avoids the bandwidth selection. In addition, the difference based estimation of the error variance is also considered. For the nonparametric component, the local polynomial technique is implemented. The finite sample properties of the developed methodology is investigated through simulation studies. An example of application is also illustrated.     

Limit distributions of least squares estimators in linear regression models with vague concepts

Linear regression models with vague concepts extend the classical single equation linear regression models by admitting observations in form of fuzzy subsets instead of real numbers. They have lately been introduced (cf. [V. Krätschmer, Induktive Statistik auf Basis unscharfer Meßkonzepte am Beispiel linearer Regressionsmodelle, unpublished postdoctoral thesis, Faculty of Law and Economics of the University of Saarland, Saarbrücken, 2001; V. Krätschmer, Least squares estimation in linear regression models with vague concepts, Fuzzy Sets and Systems, accepted for publication]) to improve the empirical meaningfulness of the relationships between the involved items by a more sensitive attention to the problems of data measurement, in particular, the fundamental problem of adequacy. The parameters of such models are still real numbers, and a method of estimation can be applied which extends directly the ordinary least squares method. In another recent contribution (cf. [V. Krätschmer, Strong consistency of least squares estimation in linear regression models with vague concepts, J. Multivar. Anal., accepted for publication]) strong consistency and -consistency of this generalized least squares estimation have been shown. The aim of the paper is to complete these results by an investigation of the limit distributions of the estimators. It turns out that the classical results can be transferred, in some cases even asymptotic normality holds.     

Nonparametric lack-of-fit tests for parametric mean-regression models with censored data

We developed two kernel smoothing based tests of a parametric mean-regression model against a nonparametric alternative when the response variable is right-censored. The new test statistics are inspired by the synthetic data and the weighted least squares approaches for estimating the parameters of a (non)linear regression model under censoring. The asymptotic critical values of our tests are given by the quantiles of the standard normal law. The tests are consistent against fixed alternatives, local Pitman alternatives and uniformly over alternatives in Hölder classes of functions of known regularity.     

Model selection by sequentially normalized least squares

Model selection by means of the predictive least squares (PLS) principle has been thoroughly studied in the context of regression model selection and autoregressive (AR) model order estimation. We introduce a new criterion based on sequentially minimized squared deviations, which are smaller than both the usual least squares and the squared prediction errors used in PLS. We also prove that our criterion has a probabilistic interpretation as a model which is asymptotically optimal within the given class of distributions by reaching the lower bound on the logarithmic prediction errors, given by the so called stochastic complexity, and approximated by BIC. This holds when the regressor (design) matrix is non-random or determined by the observed data as in AR models. The advantages of the criterion include the fact that it can be evaluated efficiently and exactly, without asymptotic approximations, and importantly, there are no adjustable hyper-parameters, which makes it applicable to both small and large amounts of data.     

Empirical likelihood analysis of the Buckley-James estimator

The censored linear regression model, also referred to as the accelerated failure time (AFT) model when the logarithm of the survival time is used as the response variable, is widely seen as an alternative to the popular Cox model when the assumption of proportional hazards is questionable. Buckley and James [Linear regression with censored data, Biometrika 66 (1979) 429-436] extended the least squares estimator to the semiparametric censored linear regression model in which the error distribution is completely unspecified. The Buckley-James estimator performs well in many simulation studies and examples. The direct interpretation of the AFT model is also more attractive than the Cox model, as Cox has pointed out, in practical situations. However, the application of the Buckley-James estimation was limited in practice mainly due to its illusive variance. In this paper, we use the empirical likelihood method to derive a new test and confidence interval based on the Buckley-James estimator of the regression coefficient. A standard chi-square distribution is used to calculate the P-value and the confidence interval. The proposed empirical likelihood method does not involve variance estimation. It also shows much better small sample performance than some existing methods in our simulation studies.     

Asymptotic equivalence for nonparametric generalized linear models

Summary. We establish that a non-Gaussian nonparametric regression model is asymptotically equivalent to a regression model with Gaussian noise. The approximation is in the sense of Le Cam's deficiency distance Δ; the models are then asymptotically equivalent for all purposes of statistical decision with bounded loss. Our result concerns a sequence of independent but not identically distributed observations with each distribution in the same real-indexed exponential family. The canonical parameter is a value f(t _i ) of a regression function f at a grid point t _i (nonparametric GLM). When f is in a H?lder ball with exponent we establish global asymptotic equivalence to observations of a signal Γ(f(t)) in Gaussian white noise, where Γ is related to a variance stabilizing transformation in the exponential family. The result is a regression analog of the recently established Gaussian approximation for the i.i.d. model. The proof is based on a functional version of the Hungarian construction for the partial sum process. Received: 4 February 1997     

Empirical likelihood for a varying coefficient partially linear model with diverging number of parameters

The purpose of this paper is two-fold. First, for the estimation or inference about the parameters of interest in semiparametric models, the commonly used plug-in estimation for infinite-dimensional nuisance parameter creates non-negligible bias, and the least favorable curve or under-smoothing is popularly employed for bias reduction in the literature. To avoid such strong structure assumptions on the models and inconvenience of estimation implementation, for the diverging number of parameters in a varying coefficient partially linear model, we adopt a bias-corrected empirical likelihood (BCEL) in this paper. This method results in the distribution of the empirical likelihood ratio to be asymptotically tractable. It can then be directly applied to construct confidence region for the parameters of interest. Second, different from all existing methods that impose strong conditions to ensure consistency of estimation when diverging the number of the parameters goes to infinity as the sample size goes to infinity, we provide techniques to show that, other than the usual regularity conditions, the consistency holds under moment conditions alone on the covariates and error with a diverging rate being even faster than those in the literature. A simulation study is carried out to assess the performance of the proposed method and to compare it with the profile least squares method. A real dataset is analyzed for illustration.     

Local linear regression for functional predictor and scalar response

The aim of this work is to introduce a new nonparametric regression technique in the context of functional covariate and scalar response. We propose a local linear regression estimator and study its asymptotic behaviour. Its finite-sample performance is compared with a Nadayara-Watson type kernel regression estimator and with the linear regression estimator via a Monte Carlo study and the analysis of two real data sets. In all the scenarios considered, the local linear regression estimator performs better than the kernel one, in the sense that the mean squared prediction error is lower.     

Empirical likelihood for single-index models

The empirical likelihood method is especially useful for constructing confidence intervals or regions of the parameter of interest. This method has been extensively applied to linear regression and generalized linear regression models. In this paper, the empirical likelihood method for single-index regression models is studied. An estimated empirical log-likelihood approach to construct the confidence region of the regression parameter is developed. An adjusted empirical log-likelihood ratio is proved to be asymptotically standard chi-square. A simulation study indicates that compared with a normal approximation-based approach, the proposed method described herein works better in terms of coverage probabilities and areas (lengths) of confidence regions (intervals).     

Single-index quantile regression

Nonparametric quantile regression with multivariate covariates is a difficult estimation problem due to the “curse of dimensionality”. To reduce the dimensionality while still retaining the flexibility of a nonparametric model, we propose modeling the conditional quantile by a single-index function , where a univariate link function g₀(⋅) is applied to a linear combination of covariates , often called the single-index. We introduce a practical algorithm where the unknown link function g₀(⋅) is estimated by local linear quantile regression and the parametric index is estimated through linear quantile regression. Large sample properties of estimators are studied, which facilitate further inference. Both the modeling and estimation approaches are demonstrated by simulation studies and real data applications.     

Depth estimators and tests based on the likelihood principle with application to regression

We investigate depth notions for general models which are derived via the likelihood principle. We show that the so-called likelihood depth for regression in generalized linear models coincides with the regression depth of Rousseeuw and Hubert (J. Amer. Statist. Assoc. 94 (1999) 388) if the dependent observations are appropriately transformed. For deriving tests, the likelihood depth is extended to simplicial likelihood depth. The simplicial likelihood depth is always a U-statistic which is in some cases not degenerated. Since the U-statistic is degenerated in the most cases, we demonstrate that nevertheless the asymptotic distribution of the simplicial likelihood depth and thus asymptotic α-level tests for general types of hypotheses can be derived. The tests are distribution-free. We work out the method for linear and quadratic regression.     

Sharp adaptive estimation of quadratic functionals

Estimation of a quadratic functional of a function observed in the Gaussian white noise model is considered. A data-dependent method for choosing the amount of smoothing is given. The method is based on comparing certain quadratic estimators with each other. It is shown that the method is asymptotically sharp or nearly sharp adaptive simultaneously for the “regular” and “irregular” region. We consider l_p bodies and construct bounds for the risk of the estimator which show that for p=4 the estimator is exactly optimal and for example when p ∈[3,100], then the upper bound is at most 1.055 times larger than the lower bound. We show the connection of the estimator to the theory of optimal recovery. The estimator is a calibration of an estimator which is nearly minimax optimal among quadratic estimators. Writing of this article was financed by Deutsche Forschungsgemeinschaft under project MA1026/6-2, CIES, France, and Jenny and AnttiWihuri Foundation.     

Affine-invariant aligned rank tests for the multivariate general linear model with VARMA errors

We develop optimal rank-based procedures for testing affine-invariant linear hypotheses on the parameters of a multivariate general linear model with elliptical VARMA errors. We propose a class of optimal procedures that are based either on residual (pseudo-)Mahalanobis signs and ranks, or on absolute interdirections and lift-interdirection ranks, i.e., on hyperplane-based signs and ranks. The Mahalanobis versions of these procedures are strictly affine-invariant, while the hyperplane-based ones are asymptotically affine-invariant. Both versions generalize the univariate signed rank procedures proposed by Hallin and Puri (J. Multivar. Anal. 50 (1994) 175), and are locally asymptotically most stringent under correctly specified radial densities. Their AREs with respect to Gaussian procedures are shown to be convex linear combinations of the AREs obtained in Hallin and Paindaveine (Ann. Statist. 30 (2002) 1103; Bernoulli 8 (2002) 787) for the pure location and purely serial models, respectively. The resulting test statistics are provided under closed form for several important particular cases, including multivariate Durbin-Watson tests, VARMA order identification tests, etc. The key technical result is a multivariate asymptotic linearity result proved in Hallin and Paindaveine (Asymptotic linearity of serial and nonserial multivariate signed rank statistics, submitted).     

Spline-based sieve maximum likelihood estimation in the partly linear model under monotonicity constraints

We study a spline-based likelihood method for the partly linear model with monotonicity constraints. We use monotone B-splines to approximate the monotone nonparametric function and apply the generalized Rosen algorithm to compute the estimators jointly. We show that the spline estimator of the nonparametric component achieves the possible optimal rate of convergence under the smooth assumption and that the estimator of the regression parameter is asymptotically normal and efficient. Moreover, a spline-based semiparametric likelihood ratio test is established to make inference of the regression parameter. Also an observed profile information method to consistently estimate the standard error of the spline estimator of the regression parameter is proposed. A simulation study is conducted to evaluate the finite sample performance of the proposed method. The method is illustrated by an air pollution study.