Statistical inference using a weighted difference-based series approach for partially linear regression models

Partially linear regression models with fixed effects are useful tools for making econometric analyses and normalizing microarray data. Baltagi and Li (2002) [7] proposed a computation friendly difference-based series estimation (DSE) for them. We show that the DSE is not asymptotically efficient in most cases and further propose a weighted difference-based series estimation (WDSE). The weights in it do not involve any unknown parameters. The asymptotic properties of the resulting estimators are established for both balanced and unbalanced cases, and it is shown that they achieve a semiparametric efficient boundary. Additionally, we propose a variable selection procedure for identifying significant covariates in the parametric part of the semiparametric fixed-effects regression model. The method is based on a combination of the nonconcave penalization (Fan and Li, 2001 [13]) and weighted difference-based series estimation techniques. The resulting estimators have the oracle property; that is, they can correctly identify the true model as if the true model (the subset of variables with nonvanishing coefficients) were known in advance. Simulation studies are conducted and an application is given to demonstrate the finite sample performance of the proposed procedures.     

Bias-corrected empirical likelihood in a multi-link semiparametric model

In this paper, we investigate the empirical likelihood for constructing a confidence region of the parameter of interest in a multi-link semiparametric model when an infinite-dimensional nuisance parameter exists. The new model covers the commonly used varying coefficient, generalized linear, single-index, multi-index, hazard regression models and their generalizations, as its special cases. Because of the existence of the infinite-dimensional nuisance parameter, the classical empirical likelihood with plug-in estimation cannot be asymptotically distribution-free, and the existing bias correction is not extendable to handle such a general model. We then propose a link-based correction approach to solve this problem. This approach gives a general rule of bias correction via an inner link, and consists of two parts. For the model whose estimating equation contains the score functions that are easy to estimate, we use a centering for the scores to correct the bias; for the model of which the score functions are of complex structure, a bias-correction procedure using simpler functions instead of the scores is given without loss of asymptotic efficiency. The resulting empirical likelihood shares the desired features: it has a chi-square limit and, under-smoothing technique, high order kernel and parameter estimation are not needed. Simulation studies are carried out to examine the performance of the new method.     

Structural test in regression on functional variables

Many papers deal with structural testing procedures in multivariate regression. More recently, various estimators have been proposed for regression models involving functional explanatory variables. Thanks to these new estimators, we propose a theoretical framework for structural testing procedures adapted to functional regression. The procedures introduced in this paper are innovative and make the link between former works on functional regression and others on structural testing procedures in multivariate regression. We prove asymptotic properties of the level and the power of our procedures under general assumptions that cover a large scope of possible applications: tests for no effect, linearity, dimension reduction, …     

Conditional and unconditional methods for selecting variables in linear mixed models

In the problem of selecting the explanatory variables in the linear mixed model, we address the derivation of the (unconditional or marginal) Akaike information criterion (AIC) and the conditional AIC (cAIC). The covariance matrices of the random effects and the error terms include unknown parameters like variance components, and the selection procedures proposed in the literature are limited to the cases where the parameters are known or partly unknown. In this paper, AIC and cAIC are extended to the situation where the parameters are completely unknown and they are estimated by the general consistent estimators including the maximum likelihood (ML), the restricted maximum likelihood (REML) and other unbiased estimators. We derive, related to AIC and cAIC, the marginal and the conditional prediction error criteria which select superior models in light of minimizing the prediction errors relative to quadratic loss functions. Finally, numerical performances of the proposed selection procedures are investigated through simulation studies.     

Conditionally specified models and dimension reduction in the exponential families

We consider informative dimension reduction for regression problems with random predictors. Based on the conditional specification of the model, we develop a methodology for replacing the predictors with a smaller number of functions of the predictors. We apply the method to the case where the inverse conditional model is in the linear exponential family. For such an inverse model and the usual Normal forward regression model it is shown that, for any number of predictors, the sufficient summary has dimension two or less. In addition, we develop a test of dimensionality. The relationship of our method with the existing dimension reduction theory based on the marginal distribution of the predictors is discussed.     

A profile-type smoothed score function for a varying coefficient partially linear model

The varying coefficient partially linear model is considered in this paper. When the plug-in estimators of coefficient functions are used, the resulting smoothing score function becomes biased due to the slow convergence rate of nonparametric estimations. To reduce the bias of the resulting smoothing score function, a profile-type smoothed score function is proposed to draw inferences on the parameters of interest without using the quasi-likelihood framework, the least favorable curve, a higher order kernel or under-smoothing. The resulting profile-type statistic is still asymptotically Chi-squared under some regularity conditions. The results are then used to construct confidence regions for the parameters of interest. A simulation study is carried out to assess the performance of the proposed method and to compare it with the profile least-squares method. A real dataset is analyzed for illustration.     

Functional-coefficient partially linear regression model

In this paper, the functional-coefficient partially linear regression (FCPLR) model is proposed by combining nonparametric and functional-coefficient regression (FCR) model. It includes the FCR model and the nonparametric regression (NPR) model as its special cases. It is also a generalization of the partially linear regression (PLR) model obtained by replacing the parameters in the PLR model with some functions of the covariates. The local linear technique and the integrated method are employed to give initial estimators of all functions in the FCPLR model. These initial estimators are asymptotically normal. The initial estimator of the constant part function shares the same bias as the local linear estimator of this function in the univariate nonparametric model, but the variance of the former is bigger than that of the latter. Similarly, initial estimators of every coefficient function share the same bias as the local linear estimates in the univariate FCR model, but the variance of the former is bigger than that of the latter. To decrease the variance of the initial estimates, a one-step back-fitting technique is used to obtain the improved estimators of all functions. The improved estimator of the constant part function has the same asymptotic normality property as the local linear nonparametric regression for univariate data. The improved estimators of the coefficient functions have the same asymptotic normality properties as the local linear estimates in FCR model. The bandwidths and the smoothing variables are selected by a data-driven method. Both simulated and real data examples related to nonlinear time series modeling are used to illustrate the applications of the FCPLR model.     

Estimation in generalized linear models for functional data via penalized likelihood

We analyze in a regression setting the link between a scalar response and a functional predictor by means of a Functional Generalized Linear Model. We first give a theoretical framework and then discuss identifiability of the model. The functional coefficient of the model is estimated via penalized likelihood with spline approximation. The L² rate of convergence of this estimator is given under smoothness assumption on the functional coefficient. Heuristic arguments show how these rates may be improved for some particular frameworks.     

Model checking in errors-in-variables regression

This paper discusses a class of minimum distance tests for fitting a parametric regression model to a class of regression functions in the errors-in-variables model. These tests are based on certain minimized distances between a nonparametric regression function estimator and a deconvolution kernel estimator of the conditional expectation of the parametric model being fitted. The paper establishes the asymptotic normality of the proposed test statistics under the null hypothesis and that of the corresponding minimum distance estimators. We also prove the consistency of the proposed tests against a fixed alternative and obtain the asymptotic distributions for general local alternatives. Simulation studies show that the testing procedures are quite satisfactory in the preservation of the finite sample level and in terms of a power comparison.     

Thresholding projection estimators in functional linear models

We consider the problem of estimating the regression function in functional linear regression models by proposing a new type of projection estimators which combine dimension reduction and thresholding. The introduction of a threshold rule allows us to get consistency under broad assumptions as well as minimax rates of convergence under additional regularity hypotheses. We also consider the particular case of Sobolev spaces generated by the trigonometric basis which permits us to get easily mean squared error of prediction as well as estimators of the derivatives of the regression function. We prove that these estimators are minimax and rates of convergence are given for some particular cases.     

A comparison of minimax and least squares estimators in linear regression with polyhedral prior information

We consider the linear regression model where prior information in the form of linear inequalities restricts the parameter space to a polyhedron. Since the linear minimax estimator has, in general, to be determined numerically, it was proposed to minimize an upper bound of the maximum risk instead. The resulting so-called quasiminimax estimator can be easily calculated in closed form. Unfortunately, both minimax estimators may violate the prior information. Therefore, we consider projection estimators which are obtained by projecting the estimate in an optional second step. The performance of these estimators is investigated in a Monte Carlo study together with several least squares estimators, including the inequality restricted least squares estimator. It turns out that both the projected and the unprojected quasiminimax estimators have the best average performance.     

Statistical inference for panel data semiparametric partially linear regression models with heteroscedastic errors

We consider a panel data semiparametric partially linear regression model with an unknown parameter vector for the linear parametric component, an unknown nonparametric function for the nonlinear component, and a one-way error component structure which allows unequal error variances (referred to as heteroscedasticity). We develop procedures to detect heteroscedasticity and one-way error component structure, and propose a weighted semiparametric least squares estimator (WSLSE) of the parametric component in the presence of heteroscedasticity and/or one-way error component structure. This WSLSE is asymptotically more efficient than the usual semiparametric least squares estimator considered in the literature. The asymptotic properties of the WSLSE are derived. The nonparametric component of the model is estimated by the local polynomial method. Some simulations are conducted to demonstrate the finite sample performances of the proposed testing and estimation procedures. An example of application on a set of panel data of medical expenditures in Australia is also illustrated.     

Nonconcave penalized inverse regression in single-index models with high dimensional predictors

In this paper we aim to estimate the direction in general single-index models and to select important variables simultaneously when a diverging number of predictors are involved in regressions. Towards this end, we propose the nonconcave penalized inverse regression method. Specifically, the resulting estimation with the SCAD penalty enjoys an oracle property in semi-parametric models even when the dimension, p_n, of predictors goes to infinity. Under regularity conditions we also achieve the asymptotic normality when the dimension of predictor vector goes to infinity at the rate of p_n=o(n^1/3) where n is sample size, which enables us to construct confidence interval/region for the estimated index. The asymptotic results are augmented by simulations, and illustrated by analysis of an air pollution dataset.     

Exact distributions of MLEs of regression coefficients in GMANOVA-MANOVA model

This paper studies the exact distributions of the MLEs of the regression coefficient matrices in a GMANOVA-MANOVA model with normal error. The unique conditions for linear functions of the MLEs of regression coefficient matrices are presented, and the exact density functions or characteristic functions for these linear functions are derived.     

Admissible prediction in superpopulation models with random regression coefficients under matrix loss function

Admissible prediction problems in finite populations with arbitrary rank under matrix loss function are investigated. For the general random effects linear model, we obtained the necessary and sufficient conditions for a linear predictor of the linearly predictable variable to be admissible in the two classes of homogeneous linear predictors and all linear predictors and the class that contains all predictors, respectively. Moreover, we prove that the best linear unbiased predictors (BLUPs) of the population total and the finite population regression coefficient are admissible under different assumptions of superpopulation models respectively.     

Estimation of the regression operator from functional fixed-design with correlated errors

We consider the estimation of the regression operator r in the functional model: Y=r(x)+ε, where the explanatory variable x is of functional fixed-design type, the response Y is a real random variable and the error process ε is a second order stationary process. We construct the kernel type estimate of r from functional data curves and correlated errors. Then we study their performances in terms of the mean square convergence and the convergence in probability. In particular, we consider the cases of short and long range error processes. When the errors are negatively correlated or come from a short memory process, the asymptotic normality of this estimate is derived. Finally, some simulation studies are conducted for a fractional autoregressive integrated moving average and for an Ornstein-Uhlenbeck error processes.     

Influence of forecasting electricity prices in the optimization of complex hydrothermal systems

This paper proposes a new method for addressing the short-term optimal operation of a generation company, fully adapted to represent the characteristics of the new competitive markets. We propose an efficient and highly accurate novel method for next-day price forecasting. We model the functional time series with a linear autoregressive functional model which formulates the relationships between each daily function of prices and the functions of previous days. For the optimization problem (formulated within the framework of nonsmooth analysis using Pontryagin’s Maximum Principle), we propose a new method that uses diverse mathematical techniques (the Shooting Method, Euler’s Method, the Cyclic Coordinate Descent Method). These techniques are well known for the case of functions, but are adapted here to the case of functionals and are efficiently combined to provide a novel contribution. Finally, the paper presents the results of applying our method to a price-taker company in the Spanish electricity market.     

Affine-invariant aligned rank tests for the multivariate general linear model with VARMA errors

We develop optimal rank-based procedures for testing affine-invariant linear hypotheses on the parameters of a multivariate general linear model with elliptical VARMA errors. We propose a class of optimal procedures that are based either on residual (pseudo-)Mahalanobis signs and ranks, or on absolute interdirections and lift-interdirection ranks, i.e., on hyperplane-based signs and ranks. The Mahalanobis versions of these procedures are strictly affine-invariant, while the hyperplane-based ones are asymptotically affine-invariant. Both versions generalize the univariate signed rank procedures proposed by Hallin and Puri (J. Multivar. Anal. 50 (1994) 175), and are locally asymptotically most stringent under correctly specified radial densities. Their AREs with respect to Gaussian procedures are shown to be convex linear combinations of the AREs obtained in Hallin and Paindaveine (Ann. Statist. 30 (2002) 1103; Bernoulli 8 (2002) 787) for the pure location and purely serial models, respectively. The resulting test statistics are provided under closed form for several important particular cases, including multivariate Durbin-Watson tests, VARMA order identification tests, etc. The key technical result is a multivariate asymptotic linearity result proved in Hallin and Paindaveine (Asymptotic linearity of serial and nonserial multivariate signed rank statistics, submitted).     

Empirical Bayes predictive densities for high-dimensional normal models

This paper addresses the problem of estimating the density of a future outcome from a multivariate normal model. We propose a class of empirical Bayes predictive densities and evaluate their performances under the Kullback–Leibler (KL) divergence. We show that these empirical Bayes predictive densities dominate the Bayesian predictive density under the uniform prior and thus are minimax under some general conditions. We also establish the asymptotic optimality of these empirical Bayes predictive densities in infinite-dimensional parameter spaces through an oracle inequality.     

Autoregressive process modeling via the Lasso procedure

The Lasso is a popular model selection and estimation procedure for linear models that enjoys nice theoretical properties. In this paper, we study the Lasso estimator for fitting autoregressive time series models. We adopt a double asymptotic framework where the maximal lag may increase with the sample size. We derive theoretical results establishing various types of consistency. In particular, we derive conditions under which the Lasso estimator for the autoregressive coefficients is model selection consistent, estimation consistent and prediction consistent. Simulation study results are reported.