Bias of LS estimators in nonlinear regression models with constraints. Part I: General Case

We derive expressions for the asymptotic approximation of the bias of the least squares estimators in nonlinear regression models with parameters which are subject to nonlinear equality constraints.The approach suggested modifies the normal equations of the estimator, and approximates them up to o _p(N ^–1), where N is the number of observations. The bias equations so obtained are solved under different assumptions on constraints and on the model. For functions of the parameters the invariance of the approximate bias with respect to reparametrisations is demonstrated. Singular models are considered as well, in which case the constraints may serve either to identify the parameters, or eventually to restrict the parameter space.     

Efficient statistical inference for partially nonlinear errors-in-variables models

In this paper, we consider the partially nonlinear errors-in-variables models when the non- parametric component is measured with additive error. The profile nonlinear least squares estimator of unknown parameter and the estimator of nonparametric component are constructed, and their asymptotic properties are derived under general assumptions. Finite sample performances of the proposed statistical inference procedures are illustrated by Monte Carlo simulation studies.     

Heteroscedasticity check in nonlinear semiparametric models based on nonparametric variance function

The assumption of homoscedasticity has received much attention in classical analysis of regression. Heteroscedasticity tests have been well studied in parametric and nonparametric regressions. The aim of this paper is to present a test of heteroscedasticity for nonlinear semiparametric regression models with nonparametric variance function. The validity of the proposed test is illustrated by two simulated examples and a real data example.     

Asymptotics of estimates in constrained nonlinear regression with long-range dependent innovations

The purpose of this paper is to investigate the asymptotic properties of the least squares estimates (L ₂-estimates) and the least absolute deviation estimates (L ₁-estimates) of the parameters of a nonlinear regression model subject to a set of equality and inequality restrictions, which has a long-range dependent stationary process as its stochastic errors. Then we will compare the asymptotic relative efficiencies of the above estimators.     

LAD estimation for nonlinear regression models with randomly censored data

The least absolute deviations (LAD) estimation for nonlinear regression models with randomly censored data is studied and the asymptotic properties of LAD estimators such as consistency, boundedness in probability and asymptotic normality are established. Simulation results show that for the problems with censored data, LAD estimation performs much more robustly than the least squares estimation.     

On construction of improved estimators in multiple-design multivariate linear models under general restriction

Consider a set ofp equations Y_i = X_ii + _i,i=1,...,p, where the rows of the random error matrix (₁,..., _p):n × p are mutually independent and identically distributed with ap-variate distribution functionF(x) having null mean and finite positive definite variance-covariance matrix . We are mainly interested in an improvement upon a feasible generalized least squares estimator (FGLSE) for = ( ₁ ,..., _p ) when it is a priori suspected thatC=c_o may hold. For this problem, Saleh and Shiraishi (1992,Nonparametric Statistics and Related Topics (ed. A. K. Md. E. Saleh), 269–279, North-Holland, Amsterdam) investigated the property of estimators such as the shrinkage estimator (SE), the positive-rule shrinkage estimator (PSE) in the light of their asymptotic distributional risks associated with the Mahalanobis loss function. We consider a general form of estimators and give a sufficient condition for proposed estimators to improve on FGLSE with respect to their asymptotic distributional quadratic risks (ADQR). The relative merits of these estimators are studied in the light of the ADQR under local alternatives. It is shown that the SE, the PSE and the Kubokawa-type shrinkage estimator (KSE) outperform the FGLSE and that the PSE is the most effective among the four estimators considered underC=c_o. It is also observed that the PSE and the KSE fairly improve over the FGLSE. Lastly, the construction of estimators improved on a generalized least squares estimator is studied, assuming normality when is known.     

The rate of convergence for the least squares estimator in nonlinear regression model with dependent errors

We study the parameter estimation in a nonlinear regression model with a general error's structure,strong consistency and strong consistency rate of the least squares estimator are obtained.     

Delete-group Jackknife Estimate in Partially Linear Regression Models with Heteroscedasticity

Abstract Consider a partially linear regression model with an unknown vector parameter β,an unknownfunction g(.),and unknown heteroscedastic error variances.Chen,You proposed a semiparametric generalizedleast squares estimator(SGLSE)for β,which takes the heteroscedasticity into account to increase efficiency.Forinference based on this SGLSE,it is necessary to construct a consistent estimator for its asymptotic covariancematrix.However,when there exists within-group correlation, the traditional delta method and the delete-1jackknife estimation fail to offer such a consistent estimator.In this paper, by deleting grouped partial residualsa delete-group jackknife method is examined.It is shown that the delete-group jackknife method indeed canprovide a consistent estimator for the asymptotic covariance matrix in the presence of within-group correlations.This result is an extension of that in[21].     

Asymptotic theory of least squares estimator in a nonregular nonlinear regression model

The asymptotic properties of the least squares estimator are derived for a nonregular nonlinear model via the study of weak convergence of the least squares process. This approach was adapted earlier by the author in the smooth case. The model discussed here is not amenable to analysis via the normal equations and Taylor expansions used by earlier authors.     

Iterative Weighted Semiparametric Least Squares Estimation in Repeated Measurement Partially Linear Regression Models

Consider a repeated measurement partially linear regression model with an unknown vector parameter β, an unknown function g(.), and unknown heteroscedastic error variances. In order to improve the semiparametric generalized least squares estimator (SGLSE) of β, we propose an iterative weighted semiparametric least squares estimator (IWSLSE) and show that it improves upon the SGLSE in terms of asymptotic covariance matrix. An adaptive procedure is given to determine the number of iterations. We also show that when the number of replicates is less than or equal to two, the IWSLSE can not improve upon the SGLSE. These results are generalizations of those in [2] to the case of semiparametric regressions.     

Consistency and asymptotic normality of profilekernel and backfitting estimators in semiparametric reproductive dispersion nonlinear models

Semiparametric reproductive dispersion nonlinear model (SRDNM) is an extension of nonlinear reproductive dispersion models and semiparametric nonlinear regression models, and includes semiparametric nonlinear model and semiparametric generalized linear model as its special cases. Based on the local kernel estimate of nonparametric component, profile-kernel and backfitting estimators of parameters of interest are proposed in SRDNM, and theoretical comparison of both estimators is also investigated in this paper. Under some regularity conditions, strong consistency and asymptotic normality of two estimators are proved. It is shown that the backfitting method produces a larger asymptotic variance than that for the profile-kernel method. A simulation study and a real example are used to illustrate the proposed methodologies. This work was supported by National Natural Science Foundation of China (Grant Nos. 10561008, 10761011), Natural Science Foundation of Department of Education of Zhejiang Province (Grant No. Y200805073), PhD Special Scientific Research Foundation of Chinese University (Grant No. 20060673002) and Program for New Century Excellent Talents in University (Grant No. NCET-07-0737)     

Latent scale linear models for multivariate ordinal responses and analysis by the method of weighted least squares

Summary A multivariate latent scale linear model is defined for multivariate ordered categorical responses and inference procedures based on the weighted least squares method are developed. Several applications of the model are suggested and illustrated through an analysis of real data. Asymptotic properties of the weighted least squares method are examined and some consequences of misspecification of the model are also discussed.     

APPROXIMATE POWER OF HETEROSCEDASTICITY TEST IN NONLINEAR MODELS WITH ARIMA（0,1,0） ERRORS

This paper presents an approach for estimating power of the score test, based on an asymptotic approximation to the power of the score test under contiguous alternatives. The method is applied to the problem of power calculations for the score test of heteroscedasticity in European rabbit data （Ratkowsky, 1983）. Simulation studies are presented which indicate that the asymptotic approximation to the finite-sample situation is good over a wide range of parameter configurations.     

Truncated Estimator of Asymptotic Covariance Matrix in Partially Linear Models with Heteroscedastic Errors

A partially linear regression model with heteroscedastic and/or serially correlated errors is studied here. It is well known that in order to apply the semiparametric least squares estimation （SLSE） to make statistical inference a consistent estimator of the asymptotic covariance matrix is needed. The traditional residual-based estimator of the asymptotic covariance matrix is not consistent when the errors are heteroscedastic and/or serially correlated. In this paper we propose a new estimator by truncating, which is an extension of the procedure in White. This estimator is shown to be consistent when the truncating parameter converges to infinity with some rate.     

A Unified Approach to Second Order Optimality Criteria in Nonlinear Design of Experiments

In the nonlinear regression model we consider the optimal design problem with a second order design D-criterion. Our purpose is to present a general approach to this problem, which includes the asymptotic second order bias and variance criterion of the least squares estimator and criteria using the volume of confidence regions based on different statistics. Under assumptions of regularity for these statistics a second order approximation of the volume of these regions is derived which is proposed as a quadratic optimality criterion. These criteria include volumes of confidence regions based on the u _n - representable statistics. An important difference between the criteria presented in this paper and the second order criteria commonly employed in the recent literature is that the former criteria are independent of the vector of residuals. Moreover, a refined version of the commonly applied criteria is obtained, which also includes effects of nonlinearity caused by third derivatives of the response function.     

Approximation to the distribution of the least squares estimators in two dimensional cosine models by randomly weighted bootstrap

Recently, Kundu and Gupta （Metrika, 48：83 C 97, 1998） established the asymptotic normality of the least squares estimators in the two dimensional cosine model. In this paper, we give the approximation to the general least squares estimators by using random weights which is called the Bayesian bootstrap or the random weighting method by Rubin （Annals of Statistics, 9：130 C 134, 1981） and Zheng （Acta Math. Appl. Sinica （in Chinese）, 10（2）： 247 C 253, 1987）. A simulation study shows that this approximation works very well.     

Application in stochastic volatility models of nonlinear regression with stochastic design

In regression model with stochastic design, the observations have been primarily treated as a simple random sample from a bivariate distribution. It is of enormous practical significance to generalize the situation to stochastic processes. In this paper, estimation and hypothesis testing problems in stochastic volatility model are considered, when the volatility depends on a nonlinear function of the state variable of other stochastic process, but the correlation coefficient |ρ|≠±1. The methods are applied to estimate the volatility of stock returns from Shanghai stock exchange. Copyright © 2009 John Wiley & Sons, Ltd.     

Regularized Adaptation of Fuzzy Inference Systems. Modelling the Opinion of a Medical Expert about Physical Fitness: An Application

This study presents a new approach to adaptation of Sugeno type fuzzy inference systems using regularization, since regularization improves the robustness of standard parameter estimation algorithms leading to stable fuzzy approximation. The proposed method can be used for modelling, identification and control of physical processes. A recursive method for on-line identification of fuzzy parameters employing Tikhonov regularization is suggested. The power of approach was shown by applying it to the modelling, identification, and adaptive control problems of dynamic processes. The proposed approach was used for modelling of human-decisions (experience) with a fuzzy inference system and for the fuzzy approximation of physical fitness with real world medical data.     

Stability of the Minimizers of Least Squares with a Non-Convex Regularization. Part II: Global Behavior

We address estimation problems where the sought-after solution is defined as the minimizer of an objective function composed of a quadratic data-fidelity term and a regularization term. We especially focus on non-convex and possibly non-smooth regularization terms because of their ability to yield good estimates. This work is dedicated to the stability of the minimizers of such piecewise C^m, with m ≥ 2, non-convex objective functions. It is composed of two parts. In the previous part of this work we considered general local minimizers. In this part we derive results on global minimizers. We show that the data domain contains an open, dense subset such that for every data point therein, the objective function has a finite number of local minimizers, and a unique global minimizer. It gives rise to a global minimizer function which is C^m-1 everywhere on an open and dense subset of the data domain.     

A Generalization of Rao's Covariance Structure with Applications to Several Linear Models

This paper presents a generalization of Rao's covariance structure. In a general linear regression model, we classify the error covariance structure into several categories and investigate the efficiency of the ordinary least squares estimator (OLSE) relative to the Gauss–Markov estimator (GME). The classification criterion considered here is the rank of the covariance matrix of the difference between the OLSE and the GME. Hence our classification includes Rao's covariance structure. The results are applied to models with special structures: a general multivariate analysis of variance model, a seemingly unrelated regression model, and a serial correlation model.