Sparse estimation in functional linear regression

As a useful tool in functional data analysis, the functional linear regression model has become increasingly common and been studied extensively in recent years. In this paper, we consider a sparse functional linear regression model which is generated by a finite number of basis functions in an expansion of the coefficient function. In this model, we do not specify how many and which basis functions enter the model, thus it is not like a typical parametric model where predictor variables are pre-specified. We study a general framework that gives various procedures which are successful in identifying the basis functions that enter the model, and also estimating the resulting regression coefficients in one-step. We adopt the idea of variable selection in the linear regression setting where one adds a weighted L₁ penalty to the traditional least squares criterion. We show that the procedures in our general framework are consistent in the sense of selecting the model correctly, and that they enjoy the oracle property, meaning that the resulting estimators of the coefficient function have asymptotically the same properties as the oracle estimator which uses knowledge of the underlying model. We investigate and compare several methods within our general framework, via a simulation study. Also, we apply the methods to the Canadian weather data.     

On asymptotics of t-type regression estimation in multiple linear model

We consider a robust estimator (t-type regression estimator) of multiple linear regression model by maximizing marginal likelihood of a scaled t-type error t-distribution. The marginal likelihood can also be applied to the de-correlated response when the within-subject correlation can be consistently estimated from an initial estimate of the model based on the independent working assumption. This paper shows that such a t-type estimator is consistent.     

Empirical Bayes estimation in a multiple linear regression model

Summary Estimation of the vector β of the regression coefficients in a multiple linear regressionY=Xβ+ε is considered when β has a completely unknown and unspecified distribution and the error-vector ε has a multivariate standard normal distribution. The optimal estimator for β, which minimizes the overall mean squared error, cannot be constructed for use in practice. UsingX, Y and the information contained in the observation-vectors obtained fromn independent past experiences of the problem, (empirical Bayes) estimators for β are exhibited. These estimators are compared with the optimal estimator and are shown to be asymptotically optimal. Estimators asymptotically optimal with rates nearO(n ⁻¹) are constructed. Supported in part by a Natural Sciences and Engineering Research Council of Canada grant.     

Efficient estimation of a varying-coefficient partially linear binary regression model

This article considers a semiparametric varying-coefficient partially linear binary regression model. The semiparametric varying-coefficient partially linear regression binary model which is a generalization of binary regression model and varying-coefficient regression model that allows one to explore the possibly nonlinear effect of a certain covariate on the response variable. A Sieve maximum likelihood estimation method is proposed and the asymptotic properties of the proposed estimators are discussed. One of our main objects is to estimate nonparametric component and the unknowen parameters simultaneously. It is easier to compute, and the required computation burden is much less than that of the existing two-stage estimation method. Under some mild conditions, the estimators are shown to be strongly consistent. The convergence rate of the estimator for the unknown smooth function is obtained, and the estimator for the unknown parameter is shown to be asymptotically efficient and normally distributed. Simulation studies are carried out to investigate the performance of the proposed method.     

Polynomial spline estimation for partial functional linear regression models

Because of its orthogonality, interpretability and best representation, functional principal component analysis approach has been extensively used to estimate the slope function in the functional linear model. However, as a very popular smooth technique in nonparametric/semiparametric regression, polynomial spline method has received little attention in the functional data case. In this paper, we propose the polynomial spline method to estimate a partial functional linear model. Some asymptotic results are established, including asymptotic normality for the parameter vector and the global rate of convergence for the slope function. Finally, we evaluate the performance of our estimation method by some simulation studies.     

空间半参数变系数部分线性回归中的分位数估计

本文研究了空间数据变系数部分线性回归中的分位数估计. 模型中的参数估计量通过未知系数函数的分段多项式逼近得到, 而未知系数函数的估计量通过将参数估计量代入模型中并通过局部线性逼近得到. 文中推导了未知参数向量估计量的渐近分布, 并建立了未知系数函数估计量在内点及边界点的渐近分布. 通过Monte Carlo 模拟研究了估计量的有限样本性质.     

On prediction rate in partial functional linear regression

We consider a prediction of a scalar variable based on both a function-valued variable and a finite number of real-valued variables. For the estimation of the regression parameters, which include the infinite dimensional function as well as the slope parameters for the real-valued variables, it is inevitable to impose some kind of regularization. We consider two different approaches, which are shown to achieve the same convergence rate of the mean squared prediction error under respective assumptions. One is based on functional principal components regression (FPCR) and the alternative is functional ridge regression (FRR) based on Tikhonov regularization. Also, numerical studies are carried out for a simulation data and a real data.     

On rates of convergence in functional linear regression

This paper investigates the rate of convergence of estimating the regression weight function in a functional linear regression model. It is assumed that the predictor as well as the weight function are smooth and periodic in the sense that the derivatives are equal at the boundary points. Assuming that the functional data are observed at discrete points with measurement error, the complex Fourier basis is adopted in estimating the true data and the regression weight function based on the penalized least-squares criterion. The rate of convergence is then derived for both estimators. A simulation study is also provided to illustrate the numerical performance of our approach, and to make a comparison with the principal component regression approach.     

On restricted linear estimation for regression in stochastic processes

In this paper the problem of restricted linear estimation for regression in stochastic processes is analyzed from different viewpoints, using RKHS methods. Of special interest is a relationship with an extended regression problem. Applications of the results to finite dimensional situations are also given.     

Bayesian bandwidth estimation for a semi-functional partial linear regression model with unknown error density

In the context of semi-functional partial linear regression model, we study the problem of error density estimation. The unknown error density is approximated by a mixture of Gaussian densities with means being the individual residuals, and variance a constant parameter. This mixture error density has a form of a kernel density estimator of residuals, where the regression function, consisting of parametric and nonparametric components, is estimated by the ordinary least squares and functional Nadaraya–Watson estimators. The estimation accuracy of the ordinary least squares and functional Nadaraya–Watson estimators jointly depends on the same bandwidth parameter. A Bayesian approach is proposed to simultaneously estimate the bandwidths in the kernel-form error density and in the regression function. Under the kernel-form error density, we derive a kernel likelihood and posterior for the bandwidth parameters. For estimating the regression function and error density, a series of simulation studies show that the Bayesian approach yields better accuracy than the benchmark functional cross validation. Illustrated by a spectroscopy data set, we found that the Bayesian approach gives better point forecast accuracy of the regression function than the functional cross validation, and it is capable of producing prediction intervals nonparametrically.     

On asymptotically efficient estimation for a semiparametric regression model

1.IntroductionConsiderthemodelY=X"0 g(T) E,(1'1)whereX"~(xl,',xo)areexplanatoryvariablesthatenterlinearly,Pisakx1vectorofunknownparameters,Tisanotherexplanatoryvariablesthatentersinanonlinearfashion,g')isanunknownsmoothfunctionofTinR',(X,T)andeareindependent,andeistheerrorwithmean0andvariancea2.Trangesoveranondegeneratecompact1-dimensionalilltervalC*;withoutlossofgenerality,C*=[0,1].Chenl2]discussedasymptoticnormalityofestimatorsP.of0byusingpiecewisepolynthacaltoapproximateg.Speckmanls…     

The convergence rates of empirical Bayes estimation in a multiple linear regression model

Empirical Bayes (EB) estimation of the parameter vector =(,²) in a multiple linear regression modelY=X+ is considered, where is the vector of regression coefficient, N(0,² I) and ² is unknown. In this paper, we have constructed the EB estimators of by using the kernel estimation of multivariate density function and its partial derivatives. Under suitable conditions it is shown that the convergence rates of the EB estimators areO(n ^{-(k-1)(k-2)/k(2k+p+1)}), where the natural numberk3, 1/3<<1, andp is the dimension of vector .The project is supported by the National Natural Science Foundation of China.     

On the estimation of prediction errors in linear regression models

Estimating the prediction error is a common practice in the statistical literature. Under a linear regression model, lete be the conditional prediction error andê be its estimate. We use (ê, e), the correlation coefficient betweene andê, to measure the performance of a particular estimation method. Reasons are given why correlation is chosen over the more popular mean squared error loss. The main results of this paper conclude that it is generally not possible to obtain good estimates of the prediction error. In particular, we show that (ê, e)=O(n ^–1/2) whenn . When the sample size is small, we argue that high values of (ê, e) can be achieved only when the residual error distribution has very heavy tails and when no outlier presents in the data. Finally, we show that in order for (ê, e) to be bounded away from zero asymptotically,ê has to be biased.     

Statistical inference in the partial functional linear expectile regression model

As extensions of means, expectiles embrace all the distribution information of a random variable.The expectile regression is computationally friendlier because the asymmetric least square loss function is differentiable everywhere. This regression also enables effective estimation of the expectiles of a response variable when potential explanatory variables are given. In this study, we propose the partial functional linear expectile regression model. The slope function and constant coefficients ...     

Robust functional estimation in the multivariate partial linear model

Annals of the Institute of Statistical Mathematics - We consider the problem of adaptive estimation of the functional component in a partial linear model where the argument of the function is...     

Consistent estimation in a linear regression problem with random errors in coefficients

We consider the linear regression model in the case when the independent variables are measured with errors, while the variances of the main observations depend on an unknown parameter. In the case of normally distributed replicated regressors we propose and study new classes of two-step estimates for the main unknown parameter. We find consistency and asymptotic normality conditions for first-step estimates and an asymptotic normality condition for second-step estimates. We discuss conditions under which these estimates have the minimal asymptotic variance.     

On estimation and inference in a partially linear hazard model with varying coefficients

We study estimation and inference in a marginal proportional hazards model that can handle (1) linear effects, (2) non-linear effects and (3) interactions between covariates. The model under consideration is an amalgamation of three existing marginal proportional hazards models studied in the literature. Developing an estimation and inference procedure with desirable properties for the amalgamated model is rather challenging due to the co-existence of all three effects listed above. Much of the existing literature has avoided the problem by considering narrow versions of the model. The object of this paper is to show that an estimation and inference procedure that accommodates all three effects is within reach. We present a profile partial-likelihood approach for estimating the unknowns in the amalgamated model with the resultant estimators of the unknown parameters being root- \(n\) consistent and the estimated functions achieving optimal convergence rates. Asymptotic normality is also established for the estimators.