Adaptive quasi-likelihood estimate in generalized linear models

This paper gives a thorough theoretical treatment on the adaptive quasi-likelihood estimate of the parameters in the generalized linear models. The unknown covariance matrix of the response variable is estimated by the sample. It is shown that the adaptive estimator defined in this paper is asymptotically most efficient in the sense that it is asymptotic normal, and the covariance matrix of the limit distribution coincides with the one for the quasi-likelihood estimator for the case that the covariance matrix of the response variable is completely known.     

A note on improving quadratic inference functions using a linear shrinkage approach

In some commonly used longitudinal clinical trials designs, the quadratic inference functions (QIF) method fails to work due to non-invertible estimation of the optimal weighting matrix. We propose a modified QIF method, in which the optimal weighting matrix is estimated by a linear shrinkage estimator, replacing the sample covariance matrix. We prove that the linear shrinkage estimator is consistent and asymptotically optimal under the expected quadratic loss, and will have more stable numerical performance than the sample covariance matrix. Simulations show that numerical improvements are acquired in light of a higher percentage of convergence, and smaller standard errors and mean square errors of parameter estimates.     

Asymptotic Improvement of the Usual Confidence Set in a Multivariate Normal Distribution with Unknown Variance

We consider confidence sets for the mean of a multivariate normal distribution with unknown covariance matrix of the formσ²I. The coverage probability of the usual confidence set is shown to be improved asymptotically by centering at a shrinkage estimator.     

Statistical inference in a panel data semiparametric regression model with serially correlated errors

We consider a panel data semiparametric partially linear regression model with an unknown vector β of regression coefficients, an unknown nonparametric function g(·) for nonlinear component, and unobservable serially correlated errors. The correlated errors are modeled by a vector autoregressive process which involves a constant intraclass correlation. Applying the pilot estimators of β and g(·), we construct estimators of the autoregressive coefficients, the intraclass correlation and the error variance, and investigate their asymptotic properties. Fitting the error structure results in a new semiparametric two-step estimator of β, which is shown to be asymptotically more efficient than the usual semiparametric least squares estimator in terms of asymptotic covariance matrix. Asymptotic normality of this new estimator is established, and a consistent estimator of its asymptotic covariance matrix is presented. Furthermore, a corresponding estimator of g(·) is also provided. These results can be used to make asymptotically efficient statistical inference. Some simulation studies are conducted to illustrate the finite sample performances of these proposed estimators.     

Efficient simulation of tail probabilities of sums of correlated lognormals

We consider the problem of efficient estimation of tail probabilities of sums of correlated lognormals via simulation. This problem is motivated by the tail analysis of portfolios of assets driven by correlated Black-Scholes models. We propose two estimators that can be rigorously shown to be efficient as the tail probability of interest decreases to zero. The first estimator, based on importance sampling, involves a scaling of the whole covariance matrix and can be shown to be asymptotically optimal. A further study, based on the Cross-Entropy algorithm, is also performed in order to adaptively optimize the scaling parameter of the covariance. The second estimator decomposes the probability of interest in two contributions and takes advantage of the fact that large deviations for a sum of correlated lognormals are (asymptotically) caused by the largest increment. Importance sampling is then applied to each of these contributions to obtain a combined estimator with asymptotically vanishing relative error.     

The Distribution of Robust Distances

Mahalanobis-type distances in which the shape matrix is derived from a consistent, high-breakdown robust multivariate location and scale estimator have an asymptotic chi-squared distribution as is the case with those derived from the ordinary covariance matrix. For example, Rousseeuw's minimum covariance determinant (MCD) is a robust estimator with a high breakdown. However, even in quite large samples, the chi-squared approximation to the distances of the sample data from the MCD center with respect to the MCD shape is poor. We provide an improved F approximation that gives accurate outlier rejection points for various sample sizes.     

Estimation of the entropy of a multivariate normal distribution

Motivated by problems in molecular biosciences wherein the evaluation of entropy of a molecular system is important for understanding its thermodynamic properties, we consider the efficient estimation of entropy of a multivariate normal distribution having unknown mean vector and covariance matrix. Based on a random sample, we discuss the problem of estimating the entropy under the quadratic loss function. The best affine equivariant estimator is obtained and, interestingly, it also turns out to be an unbiased estimator and a generalized Bayes estimator. It is established that the best affine equivariant estimator is admissible in the class of estimators that depend on the determinant of the sample covariance matrix alone. The risk improvements of the best affine equivariant estimator over the maximum likelihood estimator (an estimator commonly used in molecular sciences) are obtained numerically and are found to be substantial in higher dimensions, which is commonly the case for atomic coordinates in macromolecules such as proteins. We further establish that even the best affine equivariant estimator is inadmissible and obtain Stein-type and Brewster–Zidek-type estimators dominating it. The Brewster–Zidek-type estimator is shown to be generalized Bayes.     

Second-order nonlinear least squares estimation

The ordinary least squares estimation is based on minimization of the squared distance of the response variable to its conditional mean given the predictor variable. We extend this method by including in the criterion function the distance of the squared response variable to its second conditional moment. It is shown that this “second-order” least squares estimator is asymptotically more efficient than the ordinary least squares estimator if the third moment of the random error is nonzero, and both estimators have the same asymptotic covariance matrix if the error distribution is symmetric. Simulation studies show that the variance reduction of the new estimator can be as high as 50% for sample sizes lower than 100. As a by-product, the joint asymptotic covariance matrix of the ordinary least squares estimators for the regression parameter and for the random error variance is also derived, which is only available in the literature for very special cases, e.g. that random error has a normal distribution. The results apply to both linear and nonlinear regression models, where the random error distributions are not necessarily known.     

Second-order risk comparison of SLSE with GLSE and MLE in a regression with serial correlation

First, the second-order bias of the estimator of the autoregressive parameter based on the ordinary least squares residuals in a linear model with serial correlation is given. Second, the second-order expansion of the risk matrix of a generalized least squares estimator with the above estimated parameter is obtained. This expansion is the same as that based on a suitable estimator of the autoregressive parameter independent of the sample. Third, it is shown that the risk matrix of the generalized least squares estimator is asymptotically equivalent to that of the maximum likelihood estimator up to the second order. Last, a sufficient condition is given for the term due to the estimation of the autoregressive parameter in this expansion to vanish under Grenander's condition for the explanatory variates.     

Trimmed minimax estimator of a covariance matrix

Summary In the problem of estimating the covariance matrix of a multivariate normal population, James and Stein (Proc. Fourth Berkeley Symp. Math. Statist. Prob.,1, 361–380, Univ. of California Press) obtained a minimax estimator under a scale invariant loss. In this paper we propose an orthogonally invariant trimmed estimator by solving certain differential inequality involving the eigenvalues of the sample covariance matrix. The estimator obtained, truncates the extreme eigenvalues first and then shrinks the larger and expands the smaller sample eigenvalues. Adaptive version of the trimmed estimator is also discussed. Finally some numerical studies are performed using Monte Carlo simulation method and it is observed that the trimmed estimate shows a substantial improvement over the minimax estimator. The second author's research was supported by NSF Grant Number MCS 82-12968.     

Characteristic polynomials of sample covariance matrices: The non-square case

We consider the sample covariance matrices of large data matrices which have i.i.d. complex matrix entries and which are non-square in the sense that the difference between the number of rows and the number of columns tends to infinity. We show that the second-order correlation function of the characteristic polynomial of the sample covariance matrix is asymptotically given by the sine kernel in the bulk of the spectrum and by the Airy kernel at the edge of the spectrum. Similar results are given for real sample covariance matrices.     

Jackknifing in generalized linear models

In a generalized linear model, the jackknife estimator of the asymptotic covariance matrix of the maximum likelihood estimator is shown to be consistent. The corresponding jackknife studentized statistic is asymptotically normal. In addition, these results remain true even if there exist unequal dispersion parameters in the model. On the other hand, the variance estimator and the studentized statistic based on the standard method (substitution and linearization) do not enjoy this robustness property against the presence of unequal dispersion parameters.This research was supported by an Operating Grant from the Natural Science and Engineering Research Council of Canada.     

On uniform asymptotic normality of sequential least squares estimators for the parameters in a stable AR(p)

For a stable autoregressive process of order p with unknown vector parameter θ, it is shown that under a sequential sampling scheme with the stopping time defined by the trace of the observed Fisher information matrix, the least-squares estimator of θ is asymptotically normally distributed uniformly in θ belonging to any compact set in the parameter region.     

Sample Covariance Matrix for Random Vectors with Heavy Tails

We compute the asymptotic distribution of the sample covariance matrix for independent and identically distributed random vectors with regularly varying tails. If the tails of the random vectors are sufficiently heavy so that the fourth moments do not exist, then the sample covariance matrix is asymptotically operator stable as a random element of the vector space of symmetric matrices.     

Asymptotic distributions of robust shape matrices and scales

It has been frequently observed in the literature that many multivariate statistical methods require the covariance or dispersion matrix Σ of an elliptical distribution only up to some scaling constant. If the topic of interest is not the scale but only the shape of the elliptical distribution, it is not meaningful to focus on the asymptotic distribution of an estimator for Σ or another matrix Γ∝Σ. In the present work, robust estimators for the shape matrix and the associated scale are investigated. Explicit expressions for their joint asymptotic distributions are derived. It turns out that if the joint asymptotic distribution is normal, the estimators presented are asymptotically independent for one and only one specific choice of the scale function. If it is non-normal (this holds for example if the estimators for the shape matrix and scale are based on the minimum volume ellipsoid estimator) only the scale function presented leads to asymptotically uncorrelated estimators. This is a generalization of a result obtained by Paindaveine [D. Paindaveine, A canonical definition of shape, Statistics and Probability Letters 78 (2008) 2240-2247] in the context of local asymptotic normality theory.     

A test of the hypothesis of partial common principal components

A test of the equality of the first h eigenvectors of covariance matrices of several populations is constructed without the assumption that the sampled distributions are Gaussian. It is proved that the test statistic is asymptotically chi-square distributed. In this general setting, an explicit formula for column space of the asymptotic covariance matrix of the sample eigenvectors is derived and the rank of this matrix is computed. An essential assumption in deriving the asymptotic distribution of the presented test statistic is the existence of the finite fourth moments and the simplicity of the h largest eigenvalues of population covariance matrices, which makes possible to use the formulas for derivatives of eigenvectors of symmetric matrices.     

On the limiting spectral distribution of the covariance matrices of time-lagged processes

We consider two continuous-time Gaussian processes, one being partially correlated to a time-lagged version of the other. We first give the limiting spectral distribution for the covariance matrices of the increments of the processes when the span between two observations tends to zero. Then, we derive the limiting distribution of the eigenvalues of the sample covariance matrices. This result is obtained when the number of paths of the processes is asymptotically proportional to the number of observations for each single path. As an application, we use the second moment of this distribution together with auxiliary volatility and correlation estimates to construct an adaptive estimator of the time lag between the two processes. Finally, we provide an asymptotic theory for our estimation procedure.     

Fixed Width Confidence Region for the Mean of a Multivariate Normal Distribution

Srivastava gave an asymptotically efficient and consistent sequential procedure to obtain a fixed-width confidence region for the mean vector of any p-dimensional random vector with finite second moments. For normally distributed random vectors, Srivastava and Bhargava showed that the specified coverage probability is attained independent of the width, the mean vector, and the covariance matrix by taking a finite number of observations over and above T prescribed by the sequential rule. However, the problem of showing that E(T−n₀) is bounded, where n₀ is the sample size required if the covariance matrix were known, has not been available. In this paper, we not only show that it is bounded but obtain sharper estimates of E(T) on the lines of Woodroofe. We also give an asymptotic expansion of the coverage probability. Similar results have recently been obtained by Nagao under the assumption that the covariance matrix Σ=∑^k_i=1 σ_iA_i and ∑^k_i=1 A_i=I, where A_i's are known matrices. We obtain the results of this paper under the sole assumption that the largest latent root of Σ is simple.     

Shrinkage estimation in the frequency domain of multivariate time series

In this paper on developing shrinkage for spectral analysis of multivariate time series of high dimensionality, we propose a new nonparametric estimator of the spectral matrix with two appealing properties. First, compared to the traditional smoothed periodogram our shrinkage estimator has a smaller L₂ risk. Second, the proposed shrinkage estimator is numerically more stable due to a smaller condition number. We use the concept of “Kolmogorov” asymptotics where simultaneously the sample size and the dimensionality tend to infinity, to show that the smoothed periodogram is not consistent and to derive the asymptotic properties of our regularized estimator. This estimator is shown to have asymptotically minimal risk among all linear combinations of the identity and the averaged periodogram matrix. Compared to existing work on shrinkage in the time domain, our results show that in the frequency domain it is necessary to take the size of the smoothing span as “effective sample size” into account. Furthermore, we perform extensive Monte Carlo studies showing the overwhelming gain in terms of lower L₂ risk of our shrinkage estimator, even in situations of oversmoothing the periodogram by using a large smoothing span.     

Bayes estimation of number of signals

Bayes estimation of the number of signals, q, based on a binomial prior distribution is studied. It is found that the Bayes estimate depends on the eigenvalues of the sample covariance matrix S for white-noise case and the eigenvalues of the matrix S ₂ (S ₁+A)^–1 for the colored-noise case, where S ₁ is the sample covariance matrix of observations consisting only noise, S ₂ the sample covariance matrix of observations consisting both noise and signals and A is some positive definite matrix. Posterior distributions for both the cases are derived by expanding zonal polynomial in terms of monomial symmetric functions and using some of the important formulae of James (1964, Ann. Math. Statist., 35, 475–501).