Efficient on-line estimation of autoregressive parameters

New procedures for estimating autoregressive parameters in AR(m) models are proposed. The proposed method allows for incorporation of auxiliary information into the estimation process and produces estimation procedures, which are consistent and asymptotically efficient under certain regularity conditions. Also, these procedures are naturally on-line and do not require storing all the data. Theoretical results are presented in the case when m = 1. Two important particular cases are considered in detail: linear procedures and likelihood procedures with the LS truncations. A specific example is also presented to briefly discuss some practical aspects of applications of the procedures of this type.     

Nonparametric regression estimation with general parametric error covariance

The asymptotic distribution for the local linear estimator in nonparametric regression models is established under a general parametric error covariance with dependent and heterogeneously distributed regressors. A two-step estimation procedure that incorporates the parametric information in the error covariance matrix is proposed. Sufficient conditions for its asymptotic normality are given and its efficiency relative to the local linear estimator is established. We give examples of how our results are useful in some recently studied regression models. A Monte Carlo study confirms the asymptotic theory predictions and compares our estimator with some recently proposed alternative estimation procedures.     

Estimation and test procedures for composite quantile regression with covariates missing at random

In this paper, we study the weighted composite quantile regression (WCQR) for general linear model with missing covariates. We propose the WCQR estimation and bootstrap test procedures for unknown parameters. Simulation studies and a real data analysis are conducted to examine the finite performance of our proposed methods.     

Local linear regression for data with AR errors

In many statistical applications, data are collected over time, and they are likely correlated. In this paper, we investigate how to incorporate the correlation information into the local linear regression. Under the assumption that the error process is an auto-regressive process, a new estimation procedure is proposed for the nonparametric regression by using local linear regression method and the profile least squares techniques. We further propose the SCAD penalized profile least squares method to determine the order of auto-regressive process. Extensive Monte Carlo simulation studies are conducted to examine the finite sample performance of the proposed procedure, and to compare the performance of the proposed procedures with the existing one. From our empirical studies, the newly proposed procedures can dramatically improve the accuracy of naive local linear regression with working-independent error structure. We illustrate the proposed methodology by an analysis of real data set.     

Robust estimation for partially linear models with large-dimensional covariates

We are concerned with robust estimation procedures to estimate the parameters in partially linear models with large-dimensional covariates. To enhance the interpretability, we suggest implementing a nonconcave regularization method in the robust estimation procedure to select important covariates from the linear component. We establish the consistency for both the linear and the nonlinear components when the covariate dimension diverges at the rate of o(n1/2), where n is the sample size. We show that the robust estimate of linear component performs asymptotically as well as its oracle counterpart which assumes the baseline function and the unimportant covariates were known a priori. With a consistent estimator of the linear component, we estimate the nonparametric component by a robust local linear regression. It is proved that the robust estimate of nonlinear component performs asymptotically as well as if the linear component were known in advance.Comprehensive simulation studies are carried out and an application is presented to examine the fnite-sample performance of the proposed procedures.     

Estimation and test of restricted linear EV model with nonignorable missing covariates

This paper deals with estimation and test procedures for restricted linear errors-invariables (EV) models with nonignorable missing covariates. We develop a restricted weighted corrected least squares (WCLS) estimator based on the propensity score, which is fitted by an exponentially tilted likelihood method. The limiting distributions of the proposed estimators are discussed when tilted parameter is known or unknown. To test the validity of the constraints, we construct two test procedures based on corrected residual sum of squares and empirical likelihood method and derive their asymptotic properties. Numerical studies are conducted to examine the finite sample performance of our proposed methods.     

Non-parametric estimation from simultaneous renewal–failure–degradation data with competing risks

A joint model for linear degradation and competing failure data with partial renewals is proposed. Non-parametric estimation procedures for failure intensities and failure probabilities as functions of degradation level are given. Asymptotic properties of the estimators are investigated. To cite this article: V. Bagdonavi?ius et al., C. R. Acad. Sci. Paris, Ser. I 342 (2006).     

带线性约束的回归模型参数估计的新研究

针对一般带约束的最小二乘估计(ORLSE)在参数估计中处理复共线性的不足,引入随机线性约束,提出了约束k-d估计方法。在均方误差(MSE)下,讨论了它的性质,得到了四个主要结果,与带约束的最小二乘估计ORLSE、约束岭估计(RRE)和约束型Liu估计比较,得出更好的结论。     

带线性约束的回归模型参数的Liu估计方法

针对一般带约束的最小二乘估计(ORLSE)在参数估计中处理复共线性的不足,通过引入附加随机线性约束,提出了约束Liu型估计方法.经理论证明,此方法在均方误差下较已经提出的ORLSE和约束岭估计(RRE)效果更好,并且它可以作为两种方法的推广形式.讨论了其中k, d的确定原则,并将此方法推广到了广义形式和典则形式下的估计公式.     

Statistical inference for panel data semiparametric partially linear regression models with heteroscedastic errors

We consider a panel data semiparametric partially linear regression model with an unknown parameter vector for the linear parametric component, an unknown nonparametric function for the nonlinear component, and a one-way error component structure which allows unequal error variances (referred to as heteroscedasticity). We develop procedures to detect heteroscedasticity and one-way error component structure, and propose a weighted semiparametric least squares estimator (WSLSE) of the parametric component in the presence of heteroscedasticity and/or one-way error component structure. This WSLSE is asymptotically more efficient than the usual semiparametric least squares estimator considered in the literature. The asymptotic properties of the WSLSE are derived. The nonparametric component of the model is estimated by the local polynomial method. Some simulations are conducted to demonstrate the finite sample performances of the proposed testing and estimation procedures. An example of application on a set of panel data of medical expenditures in Australia is also illustrated.     

Statistical inference using a weighted difference-based series approach for partially linear regression models

Partially linear regression models with fixed effects are useful tools for making econometric analyses and normalizing microarray data. Baltagi and Li (2002) [7] proposed a computation friendly difference-based series estimation (DSE) for them. We show that the DSE is not asymptotically efficient in most cases and further propose a weighted difference-based series estimation (WDSE). The weights in it do not involve any unknown parameters. The asymptotic properties of the resulting estimators are established for both balanced and unbalanced cases, and it is shown that they achieve a semiparametric efficient boundary. Additionally, we propose a variable selection procedure for identifying significant covariates in the parametric part of the semiparametric fixed-effects regression model. The method is based on a combination of the nonconcave penalization (Fan and Li, 2001 [13]) and weighted difference-based series estimation techniques. The resulting estimators have the oracle property; that is, they can correctly identify the true model as if the true model (the subset of variables with nonvanishing coefficients) were known in advance. Simulation studies are conducted and an application is given to demonstrate the finite sample performance of the proposed procedures.     

回归系数的广义根方估计及其模拟

文献[1,2]中提出了回归系数的根方估计~(k),当回归自变量间存在复共线关系时,~(k)较回归系数的最小二乘估计有所改善,本文将根方估计作一拓广,得出了回归系数的广义根方估计~(K),其中K为对角阵,文中证明了广义根方估计~(K)较~(k)能更有效地改善最小二乘估计,并给出了广义根方估计的显式解,在此基础上,提出了广义根方估计的显式解和一种确定k_i的方法。     

Maximum Likelihood Estimation of the Markov-Switching GARCH Model Based on a General Collapsing Procedure

The Markov-switching GARCH model allows for a GARCH structure with time-varying parameters. This flexibility is unfortunately undermined by a path dependence problem which complicates the parameter estimation process. This problem led to the development of computationally intensive estimation methods and to simpler techniques based on an approximation of the model, known as collapsing procedures. This article develops an original algorithm to conduct maximum likelihood inference in the Markov-switching GARCH model, generalizing and improving previously proposed collapsing approaches. A new relationship between particle filtering and collapsing procedures is established which reveals that this algorithm corresponds to a deterministic particle filter. Simulation and empirical studies show that the proposed method allows for a fast and accurate estimation of the model.     

带线性约束的线性模型中回归系数的平衡LS估计

基于平衡损失的思想和最小二乘统一理论,对带线性约束的一般线性模型提出了一种全面度量估计优良性的标准．给出了此标准下模型中回归系数线性函数的约束广义平衡LS估计,并得到了约束广义平衡LS估计唯一性的一个充分条件．     

部分线性变系数模型改进的加权混合Profile-Liu估计

研究半参数部分线性变系数模型的有偏估计,当回归模型参数部分自变量存在多重共线性时,在随机线性约束条件下,融合Profile最小二乘估计、加权混合估计和Liu估计构造回归模型参数分量改进的加权混合Profile-Liu估计,并在一定正则条件下证明估计量的渐近性质,最后利用蒙特卡洛数值模拟验证所提出估计量的有限样本表现性.     

Measures of influence for the functional linear model with scalar response

This paper studies how to identify influential observations in the functional linear model in which the predictor is functional and the response is scalar. Measurement of the effects of a single observation on estimation and prediction when the model is estimated by the principal components method is undertaken. For that, three statistics are introduced for measuring the influence of each observation on estimation and prediction of the functional linear model with scalar response that are generalizations of the measures proposed for the standard regression model by [D.R. Cook, Detection of influential observations in linear regression, Technometrics 19 (1977) 15-18; D. Peña, A new statistic for influence in linear regression, Technometrics 47 (2005) 1-12] respectively. A smoothed bootstrap method is proposed to estimate the quantiles of the influence measures, which allows us to point out which observations have the larger influence on estimation and prediction. The behavior of the three statistics and the quantile estimation bootstrap based method is analyzed via a simulation study. Finally, the practical use of the proposed statistics is illustrated by the analysis of a real data example, which show that the proposed measures are useful for detecting heterogeneity in the functional linear model with scalar response.     

Nonlinear logistic discrimination via regularized radial basis functions for classifying high-dimensional data

A flexible nonparametric method is proposed for classifying high- dimensional data with a complex structure. The proposed method can be regarded as an extended version of linear logistic discriminant procedures, in which the linear predictor is replaced by a radial-basis-expansion predictor. Radial basis functions with a hyperparameter are used to take the information on covariates and class labels into account; this was nearly impossible within the previously proposed hybrid learning framework. The penalized maximum likelihood estimation procedure is employed to obtain stable parameter estimates. A crucial issue in the model-construction process is the choice of a suitable model from candidates. This issue is examined from information-theoretic and Bayesian viewpoints and we employed Ando et al. (Japanese Journal of Applied Statistics, 31, 123–139, 2002)’s model evaluation criteria. The proposed method is available not only for the high-dimensional data but also for the variable selection problem. Real data analysis and Monte Carlo experiments show that our proposed method performs well in classifying future observations in practical situations. The simulation results also show that the use of the hyperparameter in the basis functions improves the prediction performance.     

Wavelet-Based Weighted LASSO and Screening Approaches in Functional Linear Regression

One useful approach for fitting linear models with scalar outcomes and functional predictors involves transforming the functional data to wavelet domain and converting the data-fitting problem to a variable selection problem. Applying the LASSO procedure in this situation has been shown to be efficient and powerful. In this article, we explore two potential directions for improvements to this method: techniques for prescreening and methods for weighting the LASSO-type penalty. We consider several strategies for each of these directions which have never been investigated, either numerically or theoretically, in a functional linear regression context. We compare the finite-sample performance of the proposed methods through both simulations and real-data applications with both 1D signals and 2D image predictors. We also discuss asymptotic aspects. We show that applying these procedures can lead to improved estimation and prediction as well as better stability. Supplementary materials for this article are available online.     

Robust estimation for nonparametric generalized regression

This paper focuses on nonparametric regression estimation for the parameters of a discrete or continuous distribution, such as the Poisson or Gamma distributions, when anomalous data are present. The proposal is a natural extension of robust methods developed in the setting of parametric generalized linear models. Robust estimators bounding either large values of the deviance or of the Pearson residuals are introduced and their asymptotic behaviour is derived. Through a Monte Carlo study, for the Poisson and Gamma distributions, the finite properties of the proposed procedures are investigated and their performance is compared with that of the classical ones. A resistant cross-validation method to choose the smoothing parameter is also considered.     

Estimation of an optimal solution of a LP problem with unknown objective function

We consider a linear programming problem with unknown objective function. Random observations related to the unknown objective function are sequentially available. We define a stochastic algorithm, based on the simplex method, that estimates an optimal solution of the linear programming problem. It is shown that this algorithm converges with probability one to the set of optimal solutions and that its failure probability is of order inversely proportional to the sample size. We also introduce stopping criteria for the algorithm. The asymptotic normality of some suitably defined residuals is also analyzed. The proposed estimation algorithm is motivated by the stochastic approximation algorithms but it introduces a generalization of these techniques when the linear programming problem has several optimal solutions. The proposed algorithm is also close to the stochastic quasi-gradient procedures, though their usual assumptions are weakened.Mathematics Subject Classification (2000): 90C05, 62L20, 90C15Acknowledgments. I would like to thank two unknown referees for their fruitful suggestions that have helped to improve the paper.