Moderate deviations of generalized method of moments and empirical likelihood estimators

This paper studies moderate deviation behaviors of the generalized method of moments and generalized empirical likelihood estimators for generalized estimating equations, where the number of equations can be larger than the number of unknown parameters. We consider two cases for the data generating probability measure: the model assumption and local contaminations or deviations from the model assumption. For both cases, we characterize the first-order terms of the moderate deviation error probabilities of these estimators. Our moderate deviation analysis complements the existing literature of the local asymptotic analysis and misspecification analysis for estimating equations, and is useful to evaluate power and robust properties of statistical tests for estimating equations which typically involve some estimators for nuisance parameters.     

REML estimation for binary data in GLMMs

The restricted maximum likelihood (REML) procedure is useful for inferences about variance components in mixed linear models. However, its extension to hierarchical generalized linear models (HGLMs) is often hampered by analytically intractable integrals. Numerical integration such as Gauss-Hermite quadrature (GHQ) is generally not recommended when the dimensionality of the integral is high. With binary data various extensions of the REML method have been suggested, but they have had unsatisfactory biases in estimation. In this paper we propose a statistically and computationally efficient REML procedure for the analysis of binary data, which is applicable over a wide class of models and design structures. We propose a bias-correction method for models such as binary matched pairs and discuss how the REML estimating equations for mixed linear models can be modified to implement more general models.     

Autoregressive frequency detection using Regularized Least Squares

Tracking of an unknown frequency embedded in noise is widely applied in a variety of applications. Unknown frequencies can be obtained by approximating generalized spectral density of a periodic process by an autoregressive (AR) model. The advantage is that an AR model has a simple structure and its parameters can be easily estimated iteratively, which is crucial for online (real-time) applications. Typically, the order of the AR approximation is chosen by information criteria. However, with an increase of a sample size, model order may change, which leads to re-estimation of all model parameters. We propose a new iterative procedure for frequency detection based on a regularization of an empirical information matrix. The suggested method enables to avoid the repeated model selection as well as parameter estimation steps and therefore optimize computational costs. The asymptotic properties of the proposed regularized AR (RAR) frequency estimates are derived and performance of RAR is evaluated by numerical examples.     

Unbiased estimation in the multivariate natural exponential family with simple quadratic variance function

We give expansions for the unbiased estimator of a parametric function of the mean vector in a multivariate natural exponential family with simple quadratic variance function. This expansion is given in terms of a system of multivariate orthogonal polynomials with respect to the density of the sample mean. We study some limit properties of the system of orthogonal polynomials. We show that these properties are useful to establish the limit distribution of unbiased estimators.     

Fixed-gain estimation in continuous time

We prove that the parameter estimation error of continuous-time linear stochastic systems that is obtained in connection with a fixed-gain estimation method can be written as a stochastic integral plus a residual term, the moments of which are of order+o(1) where is the forgetting factor.     

Parameter estimation under ambiguity and contamination with the spurious model

Recently, we proposed variants as a statistical model for treating ambiguity. If data are extracted from an object with a machine then it might not be able to give a unique safe answer due to ambiguity about the correct interpretation of the object. On the other hand, the machine is often able to produce a finite number of alternative feature sets (of the same object) that contain the desired one. We call these feature sets variants of the object. Data sets that contain variants may be analyzed by means of statistical methods and all chapters of multivariate analysis can be seen in the light of variants. In this communication, we focus on point estimation in the presence of variants and outliers. Besides robust parameter estimation, this task requires also selecting the regular objects and their valid feature sets (regular variants). We determine the mixed MAP-ML estimator for a model with spurious variants and outliers as well as estimators based on the integrated likelihood. We also prove asymptotic results which show that the estimators are nearly consistent.The problem of variant selection turns out to be computationally hard; therefore, we also design algorithms for efficient approximation. We finally demonstrate their efficacy with a simulated data set and a real data set from genetics.     

UMVU estimation of the ratio of powers of normal generalized variances under correlation

We consider estimation of the ratio of arbitrary powers of two normal generalized variances based on two correlated random samples. First, the result of Iliopoulos [Decision theoretic estimation of the ratio of variances in a bivariate normal distribution, Ann. Inst. Statist. Math. 53 (2001) 436-446] on UMVU estimation of the ratio of variances in a bivariate normal distribution is extended to the case of the ratio of any powers of the two variances. Motivated by these estimators’ forms we derive the UMVU estimator in the multivariate case. We show that it is proportional to the ratio of the corresponding powers of the two sample generalized variances multiplied by a function of the sample canonical correlations. The mean squared errors of the derived UMVU estimator and the maximum likelihood estimator are compared via simulation for some special cases.     

New insights into best linear unbiased estimation and the optimality of least-squares

New results in matrix algebra applied to the fundamental bordered matrix of linear estimation theory pave the way towards obtaining additional and informative closed-form expressions for the best linear unbiased estimator (BLUE). The results prove significant in several respects. Indeed, more light is shed on the BLUE structure and on the working of the OLS estimator under nonsphericalness in (possibly) singular models.     

Estimation in mixed effects model with errors in variables

In this paper, we consider a linear mixed-effects model with measurement errors in both fixed and random effects and find the moment of estimators for the parameters of interest. The strong consistency and asymptotic normality of the estimators are obtained under regularity conditions. Moreover, we obtain the strong consistent estimators of the asymptotic covariance matrices involved in the limiting theory. Simulations are reported for illustration.     

Model selection by sequentially normalized least squares

Model selection by means of the predictive least squares (PLS) principle has been thoroughly studied in the context of regression model selection and autoregressive (AR) model order estimation. We introduce a new criterion based on sequentially minimized squared deviations, which are smaller than both the usual least squares and the squared prediction errors used in PLS. We also prove that our criterion has a probabilistic interpretation as a model which is asymptotically optimal within the given class of distributions by reaching the lower bound on the logarithmic prediction errors, given by the so called stochastic complexity, and approximated by BIC. This holds when the regressor (design) matrix is non-random or determined by the observed data as in AR models. The advantages of the criterion include the fact that it can be evaluated efficiently and exactly, without asymptotic approximations, and importantly, there are no adjustable hyper-parameters, which makes it applicable to both small and large amounts of data.     

Analysis of variance and linear contrasts in experimental design with generalized secant hyperbolic distribution

We consider one-way classification model in experimental design when the errors have generalized secant hyperbolic distribution. We obtain efficient and robust estimators for block effects by using the modified maximum likelihood estimation (MML) methodology. A test statistic analogous to the normal-theory F statistic is defined to test block effects. We also define a test statistic for testing linear contrasts. It is shown that test statistics based on MML estimators are efficient and robust. The methodology readily extends to unbalanced designs.     

Efficiencies of tests and estimators forp-order autoregressive processes when the error distribution is nonnormal

Summary We considerpth order autoregressive time series where the shocks need not be normal. By employing the concept of contiguity, we obtain the sysmptotic power for tests of hypothesis concerning the autoregressive parameters. Our approach allows consideration of the double exponential and other thicker-tailed distributions for the shocks. We derive a new result in the contiguity framework that leads directly to an expression for the Pitman efficiencies of tests as well as estimators. The numerical values of the efficiencies suggest a lack of robustness for the normal theory least squares estimators when the shock distribution is thick tailed or an outlier prone mixed normal. An important alternative test statistic is proposed that competes with the normal theory tests. This research was supported by the Office of Naval Research under Grant No. N00014-78-C-0722 and by the Army Research Office.     

Estimation of a tail index based on minimum density power divergence

In this paper, we consider the minimum density power divergence estimator for the tail index of heavy tailed distributions in strong mixing processes. It is shown that the estimator is consistent and asymptotically normal under regularity conditions. The simulation results demonstrate that the estimator is robust in the presence of outliers.     

On Bayes estimators with uniform priors on spheres and their comparative performance with maximum likelihood estimators for estimating bounded multivariate normal means

For independently distributed observables: X_i∼N(θ_i,σ²),i=1,…,p, we consider estimating the vector θ=(θ₁,…,θ_p)^′ with loss ‖d−θ‖² under the constraint , with known τ₁,…,τ_p,σ²,m. In comparing the risk performance of Bayesian estimators δ_α associated with uniform priors on spheres of radius α centered at (τ₁,…,τ_p) with that of the maximum likelihood estimator , we make use of Stein’s unbiased estimate of risk technique, Karlin’s sign change arguments, and a conditional risk analysis to obtain for a fixed (m,p) necessary and sufficient conditions on α for δ_α to dominate . Large sample determinations of these conditions are provided. Both cases where all such δ_α’s and cases where no such δ_α’s dominate are elicited. We establish, as a particular case, that the boundary uniform Bayes estimator δ_m dominates if and only if m≤k(p) with , improving on the previously known sufficient condition of Marchand and Perron (2001) [3] for which . Finally, we improve upon a universal dominance condition due to Marchand and Perron, by establishing that all Bayesian estimators δ_π with π spherically symmetric and supported on the parameter space dominate whenever m≤c₁(p) with .     

Modified Whittle estimation of multilateral models on a lattice

In the estimation of parametric models for stationary spatial or spatio-temporal data on a d-dimensional lattice, for d?2, the achievement of asymptotic efficiency under Gaussianity, and asymptotic normality more generally, with standard convergence rate, faces two obstacles. One is the “edge effect”, which worsens with increasing d. The other is the possible difficulty of computing a continuous-frequency form of Whittle estimate or a time domain Gaussian maximum likelihood estimate, due mainly to the Jacobian term. This is especially a problem in “multilateral” models, which are naturally expressed in terms of lagged values in both directions for one or more of the d dimensions. An extension of the discrete-frequency Whittle estimate from the time series literature deals conveniently with the computational problem, but when subjected to a standard device for avoiding the edge effect has disastrous asymptotic performance, along with finite sample numerical drawbacks, the objective function lacking a minimum-distance interpretation and losing any global convexity properties. We overcome these problems by first optimizing a standard, guaranteed non-negative, discrete-frequency, Whittle function, without edge-effect correction, providing an estimate with a slow convergence rate, then improving this by a sequence of computationally convenient approximate Newton iterations using a modified, almost-unbiased periodogram, the desired asymptotic properties being achieved after finitely many steps. The asymptotic regime allows increase in both directions of all d dimensions, with the central limit theorem established after re-ordering as a triangular array. However our work offers something new for “unilateral” models also. When the data are non-Gaussian, asymptotic variances of all parameter estimates may be affected, and we propose consistent, non-negative definite estimates of the asymptotic variance matrix.     

Parametric estimation of a bivariate stable Lévy process

We propose a parametric model for a bivariate stable Lévy process based on a Lévy copula as a dependence model. We estimate the parameters of the full bivariate model by maximum likelihood estimation. As an observation scheme we assume that we observe all jumps larger than some ε>0 and base our statistical analysis on the resulting compound Poisson process. We derive the Fisher information matrix and prove asymptotic normality of all estimates when the truncation point ε→0. A simulation study investigates the loss of efficiency because of the truncation.     

An estimation method for the Neyman chi-square divergence with application to test of hypotheses

We propose a new definition of the Neyman chi-square divergence between distributions. Based on convexity properties and duality, this version of the χ² is well suited both for the classical applications of the χ² for the analysis of contingency tables and for the statistical tests in parametric models, for which it is advocated to be robust against outliers.We present two applications in testing. In the first one, we deal with goodness-of-fit tests for finite and infinite numbers of linear constraints; in the second one, we apply χ²-methodology to parametric testing against contamination.     

Quasi-maximum likelihood estimation in GARCH processes when some coefficients are equal to zero

The asymptotic distribution of the quasi-maximum likelihood (QML) estimator is established for generalized autoregressive conditional heteroskedastic (GARCH) processes, when the true parameter may have zero coefficients. This asymptotic distribution is the projection of a normal vector distribution onto a convex cone. The results are derived under mild conditions. For an important subclass of models, no moment condition is imposed on the GARCH process. The main practical implication of these results concerns the estimation of overidentified GARCH models.     

Who's afraid of reduced-rank parameterizations of multivariate models? Theory and example

Reduced-rank restrictions can add useful parsimony to coefficient matrices of multivariate models, but their use is limited by the daunting complexity of the methods and their theory. The present work takes the easy road, focusing on unifying themes and simplified methods. For Gaussian and non-Gaussian (GLM, GAM, mixed normal, etc.) multivariate models, the present work gives a unified, explicit theory for the general asymptotic (normal) distribution of maximum likelihood estimators (MLE). MLE can be complex and computationally hard, but we show a strong asymptotic equivalence between MLE and a relatively simple minimum (Mahalanobis) distance estimator. The latter method yields particularly simple tests of rank, and we describe its asymptotic behavior in detail. We also examine the method's performance in simulation and via analytical and empirical examples.