Bias of LS estimators in nonlinear regression models with constraints. Part I: General Case

We derive expressions for the asymptotic approximation of the bias of the least squares estimators in nonlinear regression models with parameters which are subject to nonlinear equality constraints.The approach suggested modifies the normal equations of the estimator, and approximates them up to o _p(N ^–1), where N is the number of observations. The bias equations so obtained are solved under different assumptions on constraints and on the model. For functions of the parameters the invariance of the approximate bias with respect to reparametrisations is demonstrated. Singular models are considered as well, in which case the constraints may serve either to identify the parameters, or eventually to restrict the parameter space.     

The rate of convergence for the least squares estimator in nonlinear regression model with dependent errors

We study the parameter estimation in a nonlinear regression model with a general error's structure,strong consistency and strong consistency rate of the least squares estimator are obtained.     

Approximation by randomly weighting method in censored regression model

Censored regression (“Tobit”) models have been in common use, and their linear hypothesis testings have been widely studied. However, the critical values of these tests are usually related to quantities of an unknown error distribution and estimators of nuisance parameters. In this paper, we propose a randomly weighting test statistic and take its conditional distribution as an approximation to null distribution of the test statistic. It is shown that, under both the null and local alternative hypotheses, conditionally asymptotic distribution of the randomly weighting test statistic is the same as the null distribution of the test statistic. Therefore, the critical values of the test statistic can be obtained by randomly weighting method without estimating the nuisance parameters. At the same time, we also achieve the weak consistency and asymptotic normality of the randomly weighting least absolute deviation estimate in censored regression model. Simulation studies illustrate that the performance of our proposed resampling test method is better than that of central chi-square distribution under the null hypothesis. This work was supported by National Natural Science Foundation of China (Grant No. 10471136), PhD Program Foundation of the Ministry of Education of China, and Special Foundations of the Chinese Academy of Sciences and University of Science and Technology of China     

Nonparametric regression under dependent errors with infinite variance

We consider local least absolute deviation (LLAD) estimation for trend functions of time series with heavy tails which are characterised via a symmetric stable law distribution. The setting includes both causal stable ARMA model and fractional stable ARIMA model as special cases. The asymptotic limit of the estimator is established under the assumption that the process has either short or long memory autocorrelation. For a short memory process, the estimator admits the same convergence rate as if the process has the finite variance. The optimal rate of convergencen ^−2/5 is obtainable by using appropriate bandwidths. This is distinctly different from local least squares estimation, of which the convergence is slowed down due to the existence of heavy tails. On the other hand, the rate of convergence of the LLAD estimator for a long memory process is always slower thann ^−2/5 and the limit is no longer normal.     

Bias of LS estimators in nonlinear regression models with constraints. Part II: Biadditive Models

General results giving approximate bias for nonlinear models with constrained parameters are applied to bilinear models in ANOVA framework, called biadditive models. Known results on the information matrix and the asymptotic variance matrix of the parameters are summarized, and the Jacobians and Hessians of the response and of the constraints are derived. These intermediate results are the basis for any subsequent second order study of the model. Despite the large number of parameters involved, bias formulae turn out to be quite simple due to the orthogonal structure of the model. In particular, the response estimators are shown to be approximately unbiased. Some simulations assess the validity of the approximations.     

Asymptotic theory of least squares estimator in a nonregular nonlinear regression model

The asymptotic properties of the least squares estimator are derived for a nonregular nonlinear model via the study of weak convergence of the least squares process. This approach was adapted earlier by the author in the smooth case. The model discussed here is not amenable to analysis via the normal equations and Taylor expansions used by earlier authors.     

Equality and inequality constrained optimization algorithms with convergent stepsizes

In this paper, we consider two algorithms for nonlinear equality and inequality constrained optimization. Both algorithms utilize stepsize strategies based on differentiable penalty functions and quadratic programming subproblems. The essential difference between the algorithms is in the stepsize strategies used. The objective function in the quadratic subproblem includes a linear term that is dependent on the penalty functions. The quadratic objective function utilizes an approximate Hessian of the Lagrangian augmented by the penalty functions. In this approximation, it is possible to ignore the second-derivative terms arising from the constraints in the penalty functions.The penalty parameter is determined using a strategy, slightly different for each algorithm, that ensures boundedness as well as a descent property. In particular, the boundedness follows as the strategy is always satisfied for finite values of the parameter.These properties are utilized to establish global convergence and the condition under which unit stepsizes are achieved. There is also a compatibility between the quadratic objective function and the stepsize strategy to ensure the consistency of the properties for unit steps and subsequent convergence rates.This research was funded by SERC and ESRC research contracts. The author is grateful to Professors Laurence Dixon and David Mayne for their comments. The numerical results in the paper were obtained using a program written by Mr. Robin Becker.     

A GENERALIZED PROJECTION-SUCCESSIVE LINEAREQUATIONS ALGORITHM FOR NONLINEARLY EQUALITy AND INEQUALITY CONSTRAINED OPTIMIZATION AND ITS RATE OF CONVERGENCE

In this paper, a new superlinearly convergent algorithm is presented for optimization problems with general nonlineer equality and inequality Constraints, Comparing with other methods for these problems, the algorithm has two main advantages. First, it doesn‘t solve anyquadratic programming (QP), and its search directions are determined by the generalized projection technique and the solutions of two systems of linear equations. Second, the sequential points generated by the algoritbh satisfy all inequity constraints and its step-length is computed by the straight line search,The algorithm is proved to possesa global and auperlinear convergence.     

Truncated Estimator of Asymptotic Covariance Matrix in Partially Linear Models with Heteroscedastic Errors

A partially linear regression model with heteroscedastic and/or serially correlated errors is studied here. It is well known that in order to apply the semiparametric least squares estimation （SLSE） to make statistical inference a consistent estimator of the asymptotic covariance matrix is needed. The traditional residual-based estimator of the asymptotic covariance matrix is not consistent when the errors are heteroscedastic and/or serially correlated. In this paper we propose a new estimator by truncating, which is an extension of the procedure in White. This estimator is shown to be consistent when the truncating parameter converges to infinity with some rate.     

Strong universal consistency of smooth kernel regression estimates

The paper deals with kernel estimates of Nadaraya-Watson type for a regression function with square integrable response variable. For usual bandwidth sequences and smooth nonnegative kernels, e.g., Gaussian and quartic kernels, strongL ₂-consistency is shown without any further condition on the underlying distribution. The proof uses a Tauberian theorem for Cesàro summability.     

部分线性变系数模型改进的加权混合Profile-Liu估计

研究半参数部分线性变系数模型的有偏估计,当回归模型参数部分自变量存在多重共线性时,在随机线性约束条件下,融合Profile最小二乘估计、加权混合估计和Liu估计构造回归模型参数分量改进的加权混合Profile-Liu估计,并在一定正则条件下证明估计量的渐近性质,最后利用蒙特卡洛数值模拟验证所提出估计量的有限样本表现性.     

The rate of convergence of the least squares estimator in a non-linear regression model with dependent errors

The rate of convergence of the least squares estimator in a non-linear regression model with errors forming either a φ-mixing or strong mixing process is obtained. Strong consistency of the least squares estimator is obtained as a corollary.     

On convergence of LAD estimates in autoregression with infinite variance

The least absolute deviation estimates L(N), from N data points, of the autoregressive constants a = (a₁, …, a_q)′ for a stationary autoregressive model, are shown to have the property that N^σ(L(N) ? a) converge to zero in probability, for $\text{σ <}\text{1}\text{α}$, where the disturbances are i.i.d., attracted to a stable law of index α, 1 ≤ α < 2, and satisfy some other conditions.     

Double Shrinkage Estimation of Common Coefficients in Two Regression Equations with Heteroscedasticity

The problem of estimating the common regression coefficients is addressed in this paper for two regression equations with possibly different error variances. The feasible generalized least squares (FGLS) estimators have been believed to be admissible within the class of unbiased estimators. It is, nevertheless, established that the FGLS estimators are inadmissible in light of minimizing the covariance matrices if the dimension of the common regression coefficients is greater than or equal to three. Double shrinkage unbiased estimators are proposed as possible candidates of improved procedures.     

On the robustness of least squares procedures in regression models

The criterion robustness of the standard likelihood ratio test (LRT) under the multivariate normal regression model and also the inference robustness of the same test under the univariate set up are established for certain nonnormal distributions of errors. Restricting attention to the normal distribution of errors in the context of univariate regression models, conditions on the design matrix are established under which the usual LRT of a linear hypothesis (under homoscedasticity of errors) remains valid if the errors have an intraclass covariance structure. The conditions hold in the case of some standard designs. The relevance of C. R. Rao's (1967 In Proceedings Fifth Berkeley Symposium on Math. Stat. and Prob., Vol. 1, pp. 355–372) and G. Zyskind's (1967, Ann. Math. Statist.38 1092–1110) conditions in this context is discussed.