Optimal generalized ridge estimator under the generalized cross-validation criterion in linear regression

In this short paper, we mainly aim to study the generalized ridge estimator in a linear regression model. Through matrix techniques including Hadamard product and derivative of a vector, the globally optimal generalized ridge estimator is derived under the generalized cross-validation criterion from the theoretical point of view. It will be seen that the notion of linearized ridge estimator plays an important role in the process. A numerical example is applied to illustrate the main results of the paper.     

The general linear model and the generalized singular value decomposition

A new derivation is given for the generalized singular value decomposition of two matrices X and F having the same number of rows. It is shown how this decomposition reveals the structure of the general Gauss-Markov linear model (y, Xβ, σ²FF′), and exhibits the structure and solution of the generalized linear least squares problem used to provide the best linear unbiased estimator for the model. The decomposition is used to prove optimality of the estimator and to reveal the structure of the covariance matrix of the error of the estimator.     

Statistical inference in a panel data semiparametric regression model with serially correlated errors

We consider a panel data semiparametric partially linear regression model with an unknown vector β of regression coefficients, an unknown nonparametric function g(·) for nonlinear component, and unobservable serially correlated errors. The correlated errors are modeled by a vector autoregressive process which involves a constant intraclass correlation. Applying the pilot estimators of β and g(·), we construct estimators of the autoregressive coefficients, the intraclass correlation and the error variance, and investigate their asymptotic properties. Fitting the error structure results in a new semiparametric two-step estimator of β, which is shown to be asymptotically more efficient than the usual semiparametric least squares estimator in terms of asymptotic covariance matrix. Asymptotic normality of this new estimator is established, and a consistent estimator of its asymptotic covariance matrix is presented. Furthermore, a corresponding estimator of g(·) is also provided. These results can be used to make asymptotically efficient statistical inference. Some simulation studies are conducted to illustrate the finite sample performances of these proposed estimators.     

Generalized Levinson-Durbin sequences, binomial coefficients and autoregressive estimation

For a discrete time second-order stationary process, the Levinson-Durbin recursion is used to determine the coefficients of the best linear predictor of the observation at time k+1, given k previous observations, best in the sense of minimizing the mean square error. The coefficients determined by the recursion define a Levinson-Durbin sequence. We also define a generalized Levinson-Durbin sequence and note that binomial coefficients form a special case of a generalized Levinson-Durbin sequence. All generalized Levinson-Durbin sequences are shown to obey summation formulas which generalize formulas satisfied by binomial coefficients. Levinson-Durbin sequences arise in the construction of several autoregressive model coefficient estimators. The least squares autoregressive estimator does not give rise to a Levinson-Durbin sequence, but least squares fixed point processes, which yield least squares estimates of the coefficients unbiased to order 1/T, where T is the sample length, can be combined to construct a Levinson-Durbin sequence. By contrast, analogous fixed point processes arising from the Yule-Walker estimator do not combine to construct a Levinson-Durbin sequence, although the Yule-Walker estimator itself does determine a Levinson-Durbin sequence. The least squares and Yule-Walker fixed point processes are further studied when the mean of the process is a polynomial time trend that is estimated by least squares.     

A stochastic restricted ridge regression estimator

Groß [J. Groß, Restricted ridge estimation, Statistics & Probability Letters 65 (2003) 57–64] proposed a restricted ridge regression estimator when exact restrictions are assumed to hold. When there are stochastic linear restrictions on the parameter vector, we introduce a new estimator by combining ideas underlying the mixed and the ridge regression estimators under the assumption that the errors are not independent and identically distributed. Apart from [J. Groß, Restricted ridge estimation, Statistics & Probability Letters 65 (2003) 57–64], we call this new estimator as the stochastic restricted ridge regression (SRRR) estimator. The performance of the SRRR estimator over the mixed estimator in respect of the variance and the mean square error matrices is examined. We also illustrate our findings with a numerical example. The shrinkage generalized least squares (GLS) and the stochastic restricted shrinkage GLS estimators are proposed.     

Generalized least squares estimation of the functional multivariate linear errors-in-variables model

Estimators of the parameters of the functional multivariate linear errors-in-variables model are obtained by the application of generalized least squares to the sample matrix of mean squares and products. The generalized least squares estimators are shown to be consistent and asymptotically multivariate normal. Relationships between generalized least squares estimation of the functional model and of the structural model are demonstrated. It is shown that estimators constructed under the assumption of normal x are appropriate for fixed x.     

Iterative Weighted Semiparametric Least Squares Estimation in Repeated Measurement Partially Linear Regression Models

Consider a repeated measurement partially linear regression model with an unknown vector parameter β, an unknown function g(.), and unknown heteroscedastic error variances. In order to improve the semiparametric generalized least squares estimator (SGLSE) of β, we propose an iterative weighted semiparametric least squares estimator (IWSLSE) and show that it improves upon the SGLSE in terms of asymptotic covariance matrix. An adaptive procedure is given to determine the number of iterations. We also show that when the number of replicates is less than or equal to two, the IWSLSE can not improve upon the SGLSE. These results are generalizations of those in [2] to the case of semiparametric regressions.     

回归系数的广义岭型主相关估计及其优良性

In this paper, we propose a new biased estimator of the regression parameters, the generalized ridge and principal correlation estimator. We present its some properties and prove that it is superior to LSE （least squares estimator）, principal correlation estimator, ridge and principal correlation estimator under MSE （mean squares error） and PMC （Pitman closeness） criterion, respectively.     

Second-order risk comparison of SLSE with GLSE and MLE in a regression with serial correlation

First, the second-order bias of the estimator of the autoregressive parameter based on the ordinary least squares residuals in a linear model with serial correlation is given. Second, the second-order expansion of the risk matrix of a generalized least squares estimator with the above estimated parameter is obtained. This expansion is the same as that based on a suitable estimator of the autoregressive parameter independent of the sample. Third, it is shown that the risk matrix of the generalized least squares estimator is asymptotically equivalent to that of the maximum likelihood estimator up to the second order. Last, a sufficient condition is given for the term due to the estimation of the autoregressive parameter in this expansion to vanish under Grenander's condition for the explanatory variates.     

Delete-group Jackknife Estimate in Partially Linear Regression Models with Heteroscedasticity

Abstract Consider a partially linear regression model with an unknown vector parameter β,an unknownfunction g(.),and unknown heteroscedastic error variances.Chen,You proposed a semiparametric generalizedleast squares estimator(SGLSE)for β,which takes the heteroscedasticity into account to increase efficiency.Forinference based on this SGLSE,it is necessary to construct a consistent estimator for its asymptotic covariancematrix.However,when there exists within-group correlation, the traditional delta method and the delete-1jackknife estimation fail to offer such a consistent estimator.In this paper, by deleting grouped partial residualsa delete-group jackknife method is examined.It is shown that the delete-group jackknife method indeed canprovide a consistent estimator for the asymptotic covariance matrix in the presence of within-group correlations.This result is an extension of that in[21].     

Statistical inference for panel data semiparametric partially linear regression models with heteroscedastic errors

We consider a panel data semiparametric partially linear regression model with an unknown parameter vector for the linear parametric component, an unknown nonparametric function for the nonlinear component, and a one-way error component structure which allows unequal error variances (referred to as heteroscedasticity). We develop procedures to detect heteroscedasticity and one-way error component structure, and propose a weighted semiparametric least squares estimator (WSLSE) of the parametric component in the presence of heteroscedasticity and/or one-way error component structure. This WSLSE is asymptotically more efficient than the usual semiparametric least squares estimator considered in the literature. The asymptotic properties of the WSLSE are derived. The nonparametric component of the model is estimated by the local polynomial method. Some simulations are conducted to demonstrate the finite sample performances of the proposed testing and estimation procedures. An example of application on a set of panel data of medical expenditures in Australia is also illustrated.     

Bernstein–Frechet inequalities for the parameter of the first order autoregressive process

The autoregressive process takes an important part in predicting problems leading to decision making. In practice, we use the least squares method to estimate the parameter of the autoregressive process. In the case of the first order autoregressive process, we know that the least squares estimator converges in probability to the unknown parameter θ. In this Note, we show that the least squares estimator converges almost completely to θ and so we construct the inequalities of type Bernstein–Frechet for the coefficient of the first order autoregressive process. Using these inequalities a confidence interval is then obtained. To cite this article: A. Dahmani, M. Tari, C. R. Acad. Sci. Paris, Ser. I 340 (2005).     

变系数部分线性模型的加权随机约束s-K估计

研究随机约束条件下半参数变系数部分线性模型的参数估计问题,当回归模型线性部分变量存在多重共线性时,基于Profile最小二乘方法、s-K估计和加权混合估计构造参数向量的加权随机约束s-K估计,并在均方误差矩阵准则下给出新估计量优于s-K估计和加权混合估计的充要条件,最后通过蒙特卡洛数值模拟验证所提出估计量的有限样本性质.     

A Gel’fand-type spectral radius formula and stability of linear constrained switching systems

Using ergodic theory, in this paper we present a Gel’fand-type spectral radius formula which states that the joint spectral radius is equal to the generalized spectral radius for a matrix multiplicative semigroup S⁺ restricted to a subset that need not carry the algebraic structure of S⁺ This generalizes the Berger–Wang formula. Using it as a tool, we study the absolute exponential stability of a linear switched system driven by a compact subshift of the one-sided Markov shift associated to S.     

Model reduction for large-scale dynamical systems via equality constrained least squares

In this paper, we present a new method of model reduction for large-scale dynamical systems, which belongs to the SVD-Krylov based method category. It is a two-sided projection where one side reflects the Krylov part and the other side reflects the SVD (observability gramian) part. The reduced model matches the first r+i Markov parameters of the full order model, and the remaining ones approximate in a least squares sense without being explicitly computed, where r is the order of the reduced system, and i is a nonnegative integer such that 1≤i<r. The reduced system minimizes a weighted ?₂ error. By the definition of a shift operator, the proposed approximation is also obtained by solving an equality constrained least squares problem. Moreover, the method is generalized for moment matching at arbitrary interpolation points. Several numerical examples verify the effectiveness of the approach.     

A subclass of bayes linear estimators that are minimax

In the linear regression model with ellipsoidal parameter constraints, the problem of estimating the unknown parameter vector is studied. A well-described subclass of Bayes linear estimators is proposed in the paper. It is shown that for each member of this subclass, a generalized quadratic risk function exists so that the estimator is minimax. Moreover, some of the proposed Bayes linear estimators are admissible with respect to all possible generalized quadratic risks. Also, a necessary and sufficient condition is given to ensure that the considered Bayes linear estimator improves the least squares estimator over the whole ellipsoid whatever generalized risk function is chosen.     

Moderate Deviations for M-estimators in Linear Models with φ-mixing Errors

In this paper, the moderate deviations for the M-estimators of regression parameter in a linear model are obtained when the errors form a strictly stationary φ-mixing sequence. The results are applied to study many different types of M-estimators such as Huber's estimator, Lp-regression estimator, least squares estimator and least absolute deviation estimator.     

线性模型中F-检验的稳健性

本文讨论了一般线性模型中关于均值参数β的线性假设基于广义最小二乘估计的F-检验统计量的稳健性问题.主要研究了当误差的协方差矩阵含有参数时,设计阵可以列降秩情况下的F-检验统计量的稳健性,得到了F(V(θ))为该假设下F-检验统计量的误差协方差矩阵的最大类.并讨论了分块线性模型中,关于分块参数的线性假设的F-检验统计量的稳健性.     

Fundamental solution for linear two-point boundary value problem

The quadratic functional minimization with differential restrictions represented by the command linear systems is considered. The optimal solution determination implies the solving of a linear problem with two points boundary values. The proposed method implies the construction of a fundamental solution S(t)—a n×n matrix—and of a vector h(t) defining an adjoint variable λ(t) depending of the state variable x(t). From the extremum necessary conditions it is obtained the Riccati matrix differential equation having the S(t) as unknown fundamental solution is obtained. The paper analyzes the existence of the Riccati equation solution S(t) and establishes as the optimal solution of the proposed optimum problem. Also a superior limit of the minimum for the considered quadratic functionals class are evaluated.     

Asymptotic efficiency of ridge estimator in linear and semiparametric linear models

The linear model with a growing number of predictors arises in many contemporary scientific endeavor. In this article, we consider the commonly used ridge estimator in linear models. We propose analyzing the ridge estimator for a finite sample size n and a growing dimension p. The existence and asymptotic normality of the ridge estimator are established under some regularity conditions when p→∞. It also occurs that a strictly linear model is inadequate when some of the relations are believed to be of certain linear form while others are not easily parameterized, and thus a semiparametric partial linear model is considered. For these semiparametric partial linear models with p>n, we develop a procedure to estimate the linear coefficients as if the nonparametric part is not present. The asymptotic efficiency of the proposed estimator for the linear component is studied for p→∞. It is shown that the proposed estimator of the linear component asymptotically performs very well.