Bias of LS estimators in nonlinear regression models with constraints. Part II: Biadditive Models

General results giving approximate bias for nonlinear models with constrained parameters are applied to bilinear models in ANOVA framework, called biadditive models. Known results on the information matrix and the asymptotic variance matrix of the parameters are summarized, and the Jacobians and Hessians of the response and of the constraints are derived. These intermediate results are the basis for any subsequent second order study of the model. Despite the large number of parameters involved, bias formulae turn out to be quite simple due to the orthogonal structure of the model. In particular, the response estimators are shown to be approximately unbiased. Some simulations assess the validity of the approximations.     

Asymptotics of estimates in constrained nonlinear regression with long-range dependent innovations

The purpose of this paper is to investigate the asymptotic properties of the least squares estimates (L ₂-estimates) and the least absolute deviation estimates (L ₁-estimates) of the parameters of a nonlinear regression model subject to a set of equality and inequality restrictions, which has a long-range dependent stationary process as its stochastic errors. Then we will compare the asymptotic relative efficiencies of the above estimators.     

On construction of improved estimators in multiple-design multivariate linear models under general restriction

Consider a set ofp equations Y_i = X_ii + _i,i=1,...,p, where the rows of the random error matrix (₁,..., _p):n × p are mutually independent and identically distributed with ap-variate distribution functionF(x) having null mean and finite positive definite variance-covariance matrix . We are mainly interested in an improvement upon a feasible generalized least squares estimator (FGLSE) for = ( ₁ ,..., _p ) when it is a priori suspected thatC=c_o may hold. For this problem, Saleh and Shiraishi (1992,Nonparametric Statistics and Related Topics (ed. A. K. Md. E. Saleh), 269–279, North-Holland, Amsterdam) investigated the property of estimators such as the shrinkage estimator (SE), the positive-rule shrinkage estimator (PSE) in the light of their asymptotic distributional risks associated with the Mahalanobis loss function. We consider a general form of estimators and give a sufficient condition for proposed estimators to improve on FGLSE with respect to their asymptotic distributional quadratic risks (ADQR). The relative merits of these estimators are studied in the light of the ADQR under local alternatives. It is shown that the SE, the PSE and the Kubokawa-type shrinkage estimator (KSE) outperform the FGLSE and that the PSE is the most effective among the four estimators considered underC=c_o. It is also observed that the PSE and the KSE fairly improve over the FGLSE. Lastly, the construction of estimators improved on a generalized least squares estimator is studied, assuming normality when is known.     

On the robustness of least squares procedures in regression models

The criterion robustness of the standard likelihood ratio test (LRT) under the multivariate normal regression model and also the inference robustness of the same test under the univariate set up are established for certain nonnormal distributions of errors. Restricting attention to the normal distribution of errors in the context of univariate regression models, conditions on the design matrix are established under which the usual LRT of a linear hypothesis (under homoscedasticity of errors) remains valid if the errors have an intraclass covariance structure. The conditions hold in the case of some standard designs. The relevance of C. R. Rao's (1967 In Proceedings Fifth Berkeley Symposium on Math. Stat. and Prob., Vol. 1, pp. 355–372) and G. Zyskind's (1967, Ann. Math. Statist.38 1092–1110) conditions in this context is discussed.     

The rate of convergence for the least squares estimator in nonlinear regression model with dependent errors

We study the parameter estimation in a nonlinear regression model with a general error's structure,strong consistency and strong consistency rate of the least squares estimator are obtained.     

Consistency and asymptotic normality of profilekernel and backfitting estimators in semiparametric reproductive dispersion nonlinear models

Semiparametric reproductive dispersion nonlinear model (SRDNM) is an extension of nonlinear reproductive dispersion models and semiparametric nonlinear regression models, and includes semiparametric nonlinear model and semiparametric generalized linear model as its special cases. Based on the local kernel estimate of nonparametric component, profile-kernel and backfitting estimators of parameters of interest are proposed in SRDNM, and theoretical comparison of both estimators is also investigated in this paper. Under some regularity conditions, strong consistency and asymptotic normality of two estimators are proved. It is shown that the backfitting method produces a larger asymptotic variance than that for the profile-kernel method. A simulation study and a real example are used to illustrate the proposed methodologies. This work was supported by National Natural Science Foundation of China (Grant Nos. 10561008, 10761011), Natural Science Foundation of Department of Education of Zhejiang Province (Grant No. Y200805073), PhD Special Scientific Research Foundation of Chinese University (Grant No. 20060673002) and Program for New Century Excellent Talents in University (Grant No. NCET-07-0737)     

SEM modeling with singular moment matrices Part III: GLS estimation

We discuss generalized least squares (GLS) and maximum likelihood (ML) estimation for structural equations models (SEM), when the sample moment matrices are possibly singular. This occurs in several instances, for example, for panel data when there are more panel waves than independent replications or for time series data where the number of time points is large, but only one unit is observed. In previous articles, it was shown that ML estimation of the SEM is possible by using a correct Gaussian likelihood function. In this article, the usual GLS fit function is modified so that it is also defined for singular sample moment matrices S. In large samples, GLS and ML estimation perform similarly, and the modified GLS approach is a good alternative when S becomes nearly singular. Both GLS approaches do not work for N = 1, since here S = 0 and the modified GLS approach yields biased estimates. In conclusion, ML estimation (and pseudo ML under misspecification) is recommended for all sample sizes including N = 1.     

Asymptotic expansions and higher order properties of semi-parametric estimators in a system of simultaneous equations

Asymptotic expansions are made for the distributions of the Maximum Empirical Likelihood (MEL) estimator and the Estimating Equation (EE) estimator (or the Generalized Method of Moments (GMM) in econometrics) for the coefficients of a single structural equation in a system of linear simultaneous equations, which corresponds to a reduced rank regression model. The expansions in terms of the sample size, when the non-centrality parameters increase proportionally, are carried out to O(n⁻¹). Comparisons of the distributions of the MEL and GMM estimators are made. Also, we relate the asymptotic expansions of the distributions of the MEL and GMM estimators to the corresponding expansions for the Limited Information Maximum Likelihood (LIML) and the Two-Stage Least Squares (TSLS) estimators. We give useful information on the higher order properties of alternative estimators including the semi-parametric inefficiency factor under the homoscedasticity assumption.     

Asymptotic properties of a maximum likelihood estimator with data from a Gaussian process

We consider an estimation problem with observations from a Gaussian process. The problem arises from a stochastic process modeling of computer experiments proposed recently by Sacks, Schiller, and Welch. By establishing various representations and approximations to the corresponding log-likelihood function, we show that the maximum likelihood estimator of the identifiable parameter θσ² is strongly consistent and converges weakly (when normalized by √n) to a normal random variable, whose variance does not depend on the selection of sample points. Some extensions to regression models are also obtained.     

ECLES: A general method for local editing of parameters with linear constraints

We describe ECLES (Editing by Constrained LEast Squares), a general method for interactive local editing of objects that are defined by a list of parameters subject to a set of linear or affine constraints. The method is intended for situations where each edit action affects only a small set of the parameters; some of which (the “anchors”) are to be set to new given values, whereas the rest (the “derived” ones) are to be modified so as to preserve the constraints. We use exact integer arithmetic in order to reliably determine solvability, to detect and eliminate redundancies in the set of constraints, and to ensure that the solution exactly satisfies the constraints. We also use constrained least squares to choose a suitable solution when the constraints allow multiple solutions. Unlike the usual finite element approach, the method allows editing of any set of anchors with any sufficiently large set of derived parameters. As an example of application, we show how the method can be used to edit smooth ( ${\mathsf{C}}^1$) deformations of geometric mesh models.     

Application in stochastic volatility models of nonlinear regression with stochastic design

In regression model with stochastic design, the observations have been primarily treated as a simple random sample from a bivariate distribution. It is of enormous practical significance to generalize the situation to stochastic processes. In this paper, estimation and hypothesis testing problems in stochastic volatility model are considered, when the volatility depends on a nonlinear function of the state variable of other stochastic process, but the correlation coefficient |ρ|≠±1. The methods are applied to estimate the volatility of stock returns from Shanghai stock exchange. Copyright © 2009 John Wiley & Sons, Ltd.     

Stability of the Minimizers of Least Squares with a Non-Convex Regularization. Part I: Local Behavior

Many estimation problems amount to minimizing a piecewise C^m objective function, with m ≥ 2, composed of a quadratic data-fidelity term and a general regularization term. It is widely accepted that the minimizers obtained using non-convex and possibly non-smooth regularization terms are frequently good estimates. However, few facts are known on the ways to control properties of these minimizers. This work is dedicated to the stability of the minimizers of such objective functions with respect to variations of the data. It consists of two parts: first we consider all local minimizers, whereas in a second part we derive results on global minimizers. In this part we focus on data points such that every local minimizer is isolated and results from a C^m-1 local minimizer function, defined on some neighborhood. We demonstrate that all data points for which this fails form a set whose closure is negligible.     

Asymptotics for models of non‐stationary diffusion in domains with a surface distribution of obstacles

We consider a time‐dependent model for the diffusion of a substance through an incompressible fluid in a perforated domain Ω_?, with n = 3,4. The fluid flows in a domain containing a periodical set of “obstacles” (Ω\Ω_?) placed along an inner (n ? 1)‐dimensional manifold . The size of the obstacles is much smaller than the size of the characteristic period ?. An advection term appears in the partial differential equation linking the fluid velocity with the concentration, while we assume a nonlinear adsorption law on the boundary of the obstacles. This law involves a monotone nonlinear function σ of the concentration and a large adsorption parameter. The “critical adsorption parameter” depends on the size of the obstacles , and, for different sizes, we derive the time‐dependent homogenized models. These models contain a “strange term” in the transmission conditions on Σ, which is a nonlinear function and inherits the properties of σ. The case in which the fluid velocity and the concentration do not interact is also considered for n ≥ 3.     

SEM Modeling with Singular Moment Matrices Part I: ML-Estimation of Time Series

A structural equation model (SEM) with deterministic intercepts is introduced. The Gaussian likelihood function does not contain determinants of sample moment matrices and is thus well-defined for only one statistical unit. The SEM is applied to the dynamic state space model and compared with the Kalman filter (KF) approach. The likelihood of both methods are shown to be equivalent, but for long time series numerical problems occur in the SEM approach, which are traced to the inversion of the latent state covariance matrix. Both approaches are compared on several aspects. The SEM approach is now open for idiographic (N = 1) analysis and estimation of panel data with correlated units.     

Consistency and asymptotic normality of the maximum quasi-likelihood estimator in quasi-likelihood nonlinear models with random regressors

This paper proposes some regularity conditions, which result in the existence, strong consistency and asymptotic normality of maximum quasi-likelihood estimator （MQLE） in quasi-likelihood nonlinear models （QLNM） with random regressors. The asymptotic results of generalized linear models （GLM） with random regressors are generalized to QLNM with random regressors.     

APPROXIMATE POWER OF HETEROSCEDASTICITY TEST IN NONLINEAR MODELS WITH ARIMA（0,1,0） ERRORS

This paper presents an approach for estimating power of the score test, based on an asymptotic approximation to the power of the score test under contiguous alternatives. The method is applied to the problem of power calculations for the score test of heteroscedasticity in European rabbit data （Ratkowsky, 1983）. Simulation studies are presented which indicate that the asymptotic approximation to the finite-sample situation is good over a wide range of parameter configurations.     

Stability of the Minimizers of Least Squares with a Non-Convex Regularization. Part II: Global Behavior

We address estimation problems where the sought-after solution is defined as the minimizer of an objective function composed of a quadratic data-fidelity term and a regularization term. We especially focus on non-convex and possibly non-smooth regularization terms because of their ability to yield good estimates. This work is dedicated to the stability of the minimizers of such piecewise C^m, with m ≥ 2, non-convex objective functions. It is composed of two parts. In the previous part of this work we considered general local minimizers. In this part we derive results on global minimizers. We show that the data domain contains an open, dense subset such that for every data point therein, the objective function has a finite number of local minimizers, and a unique global minimizer. It gives rise to a global minimizer function which is C^m-1 everywhere on an open and dense subset of the data domain.     

SEM Modeling with Singular Moment Matrices Part II: ML-Estimation of Sampled Stochastic Differential Equations

Linear stochastic differential equations are expressed as an exact discrete model (EDM) and estimated with structural equation models (SEMs) and the Kalman filter (KF) algorithm. The oversampling approach is introduced in order to formulate the EDM on a time grid which is finer than the sampling intervals. This leads to a simple computation of the nonlinear parameter functionals of the EDM. For small discretization intervals, the functionals can be linearized, and standard software permitting only linear parameter restrictions can be used. However, in this case the SEM approach must handle large matrices leading to degraded performance and possible numerical problems. The methods are compared using coupled linear random oscillators with time-varying parameters and irregular sampling times.     

The rate of convergence of the least squares estimator in a non-linear regression model with dependent errors

The rate of convergence of the least squares estimator in a non-linear regression model with errors forming either a φ-mixing or strong mixing process is obtained. Strong consistency of the least squares estimator is obtained as a corollary.     

The Essential Spectrum of a System of Singular Ordinary Differential Operators of Mixed Order. Part I: The General Problem and an Almost Regular Case

The paper develops the notion of signed contents and measures on the basis of a new definition. It form a common roof for some well-known partial notions which appeared to be incompatible. The paper also redefines the notion of angular pairs of contents and measures, to the effect that the two notions are connected via an attended Jordan decomposition theorem. The decisive idea is a new difference formation for contents and measures.