Influence functions and efficiencies of the canonical correlation and vector estimates based on scatter and shape matrices

In this paper, the influence functions and limiting distributions of the canonical correlations and coefficients based on affine equivariant scatter matrices are developed for elliptically symmetric distributions. General formulas for limiting variances and covariances of the canonical correlations and canonical vectors based on scatter matrices are obtained. Also the use of the so-called shape matrices in canonical analysis is investigated. The scatter and shape matrices based on the affine equivariant Sign Covariance Matrix as well as the Tyler's shape matrix serve as examples. Their finite sample and limiting efficiencies are compared to those of the Minimum Covariance Determinant estimators and S-estimator through theoretical and simulation studies. The theory is illustrated by an example.     

Relative efficiencies of estimates using patterned covariances or correlations in the multivariate normal estimation problem

Summary The relative efficiency of maximum likelihood estimates is studied when taking advantage of underlying linear patterns in the covariances or correlations when estimating covariance matrices. We compare the variances of estimates of the covariance matrix obtained under two nested patterns with the assumption that the more restricted pattern is the true state. Formulas for the asymptotic variances are given which are exact for linear covariance patterns when explicit maximum likelihood estimates exist. Several specific examples are given using complete symmetry, circular symmetry and general covariance patterns as well as an example involving a covariance matrix with a linear pattern in the correlations.     

Spatial design matrices and associated quadratic forms: structure and properties

The paper provides significant simplifications and extensions of results obtained by Gorsich, Genton, and Strang (J. Multivariate Anal. 80 (2002) 138) on the structure of spatial design matrices. These are the matrices implicitly defined by quadratic forms that arise naturally in modelling intrinsically stationary and isotropic spatial processes. We give concise structural formulae for these matrices, and simple generating functions for them. The generating functions provide formulae for the cumulants of the quadratic forms of interest when the process is Gaussian, second-order stationary and isotropic. We use these to study the statistical properties of the associated quadratic forms, in particular those of the classical variogram estimator, under several assumptions about the actual variogram.     

Asymptotic theory for maximum deviations of sample covariance matrix estimates

We consider asymptotic distributions of maximum deviations of sample covariance matrices, a fundamental problem in high-dimensional inference of covariances. Under mild dependence conditions on the entries of the data matrices, we establish the Gumbel convergence of the maximum deviations. Our result substantially generalizes earlier ones where the entries are assumed to be independent and identically distributed, and it provides a theoretical foundation for high-dimensional simultaneous inference of covariances.     

Robust weighted orthogonal regression in the errors-in-variables model

This paper focuses on robust estimation in the structural errors-in-variables (EV) model. A new class of robust estimators, called weighted orthogonal regression estimators, is introduced. Robust estimators of the parameters of the EV model are simply derived from robust estimators of multivariate location and scatter such as the M-estimators, the S-estimators and the MCD estimator. The influence functions of the proposed estimators are calculated and shown to be bounded. Moreover, we derive the asymptotic distributions of the estimators and illustrate the results on simulated examples and on a real-data set.     

On the structure of the quadratic subspace in discriminant analysis

The concept of quadratic subspace is introduced as a helpful tool for dimension reduction in quadratic discriminant analysis (QDA). It is argued that an adequate representation of the quadratic subspace may lead to better methods for both data representation and classification. Several theoretical results describe the structure of the quadratic subspace, that is shown to contain some of the subspaces previously proposed in the literature for finding differences between the class means and covariances. A suitable assumption of orthogonality between location and dispersion subspaces allows us to derive a convenient reduced version of the full QDA rule. The behavior of these ideas in practice is illustrated with three real data examples.     

Asymptotic distribution of Wishart matrix for block-wise dispersion of population eigenvalues

This paper deals with the asymptotic distribution of Wishart matrix and its application to the estimation of the population matrix parameter when the population eigenvalues are block-wise infinitely dispersed. We show that the appropriately normalized eigenvectors and eigenvalues asymptotically generate two Wishart matrices and one normally distributed random matrix, which are mutually independent. For a family of orthogonally equivariant estimators, we calculate the asymptotic risks with respect to the entropy or the quadratic loss function and derive the asymptotically best estimator among the family. We numerically show (1) the convergence in both the distributions and the risks are quick enough for a practical use, (2) the asymptotically best estimator is robust against the deviation of the population eigenvalues from the block-wise infinite dispersion.     

An asymptotic theory for sample covariances of Bernoulli shifts

Covariances play a fundamental role in the theory of stationary processes and they can naturally be estimated by sample covariances. There is a well-developed asymptotic theory for sample covariances of linear processes. For nonlinear processes, however, many important problems on their asymptotic behaviors are still unanswered. The paper presents a systematic asymptotic theory for sample covariances of nonlinear time series. Our results are applied to the test of correlations.     

Bounded influence regression using high breakdown scatter matrices

In this paper we estimate the parameters of a regression model using S-estimators of multivariate location and scatter. The approach is proven to be Fisher-consistent, and the influence functions are derived. The corresponding asymptotic variances are obtained and it is shown how they can be estimated in practice. A comparison with other recently proposed robust regression estimators is made.     

An asymptotic approximation for EPMC in linear discriminant analysis based on two-step monotone missing samples

In this paper, we consider the expected probabilities of misclassification (EPMC) in the linear discriminant function (LDF) based on two-step monotone missing samples and derive an asymptotic approximation for the EPMC with an explicit form for the considered LDF. For this purpose, we also provide some results of the expectations for the inverted Wishart matrices in this paper. Finally, we conduct the Monte Carlo simulation for evaluating our result.     

Estimating cumulative incidence functions when the life distributions are constrained

In competing risks studies, the Kaplan-Meier estimators of the distribution functions (DFs) of lifetimes and the corresponding estimators of cumulative incidence functions (CIFs) are used widely when no prior information is available for these distributions. In some cases better estimators of the DFs of lifetimes are available when they obey some inequality constraints, e.g., if two lifetimes are stochastically or uniformly stochastically ordered, or some functional of a DF obeys an inequality in an empirical likelihood estimation procedure. If the restricted estimator of a lifetime differs from the unrestricted one, then the usual estimators of the CIFs will not add up to the lifetime estimator. In this paper we show how to estimate the CIFs in this case. These estimators are shown to be strongly uniformly consistent. In all cases we consider, when the inequality constraints are strict the asymptotic properties of the restricted and the unrestricted estimators are the same, thus providing the asymptotic properties of the restricted estimators essentially “free of charge”. We give an example to illustrate our procedure.     

On a class of free Lévy laws related to a regression problem

The free Meixner laws arise as the distributions of orthogonal polynomials with constant-coefficient recursions. We show that these are the laws of the free pairs of random variables which have linear regressions and quadratic conditional variances when conditioned with respect to their sum. We apply this result to describe free Lévy processes with quadratic conditional variances, and to prove a converse implication related to asymptotic freeness of random Wishart matrices.     

Testing nonparametric and semiparametric hypotheses in vector stationary processes

We propose a general nonparametric approach for testing hypotheses about the spectral density matrix of multivariate stationary time series based on estimating the integrated deviation from the null hypothesis. This approach covers many important examples from interrelation analysis such as tests for noncorrelation or partial noncorrelation. Based on a central limit theorem for integrated quadratic functionals of the spectral matrix, we derive asymptotic normality of a suitably standardized version of the test statistic under the null hypothesis and under fixed as well as under sequences of local alternatives. The results are extended to cover also parametric and semiparametric hypotheses about spectral density matrices, which includes as examples goodness-of-fit tests and tests for separability.     

Innovations algorithm asymptotics for periodically stationary time series with heavy tails

The innovations algorithm can be used to obtain parameter estimates for periodically stationary time series models. In this paper we compute the asymptotic distribution for these estimates in the case where the underlying noise sequence has infinite fourth moment but finite second moment. In this case, the sample covariances on which the innovations algorithm are based are known to be asymptotically stable. The asymptotic results developed here are useful to determine which model parameters are significant. In the process, we also compute the asymptotic distributions of least squares estimates of parameters in an autoregressive model.     

Asymptotic expansions in mean and covariance structure analysis

Asymptotic expansions of the distributions of parameter estimators in mean and covariance structures are derived. The parameters may be common to, or specific in means and covariances of observable variables. The means are possibly structured by the common/specific parameters. First, the distributions of the parameter estimators standardized by the population asymptotic standard errors are expanded using the single- and the two-term Edgeworth expansions. In practice, the pivotal statistic or the Studentized estimator with the asymptotically distribution-free standard error is of interest. An asymptotic distribution of the pivotal statistic is also derived by the Cornish-Fisher expansion. Simulations are performed for a factor analysis model with nonzero factor means to see the accuracy of the asymptotic expansions in finite samples.     

Optimal discriminant functions for normal populations

A class of discriminant rules which includes Fisher’s linear discriminant function and the likelihood ratio criterion is defined. Using asymptotic expansions of the distributions of the discriminant functions in this class, we derive a formula for cut-off points which satisfy some conditions on misclassification probabilities, and derive the optimal rules for some criteria. Some numerical experiments are carried out to examine the performance of the optimal rules for finite numbers of samples.     

Conditional and unconditional methods for selecting variables in linear mixed models

In the problem of selecting the explanatory variables in the linear mixed model, we address the derivation of the (unconditional or marginal) Akaike information criterion (AIC) and the conditional AIC (cAIC). The covariance matrices of the random effects and the error terms include unknown parameters like variance components, and the selection procedures proposed in the literature are limited to the cases where the parameters are known or partly unknown. In this paper, AIC and cAIC are extended to the situation where the parameters are completely unknown and they are estimated by the general consistent estimators including the maximum likelihood (ML), the restricted maximum likelihood (REML) and other unbiased estimators. We derive, related to AIC and cAIC, the marginal and the conditional prediction error criteria which select superior models in light of minimizing the prediction errors relative to quadratic loss functions. Finally, numerical performances of the proposed selection procedures are investigated through simulation studies.     

Robust estimation of periodic autoregressive processes in the presence of additive outliers

This paper suggests a robust estimation procedure for the parameters of the periodic AR (PAR) models when the data contains additive outliers. The proposed robust methodology is an extension of the robust scale and covariance functions given in, respectively, Rousseeuw and Croux (1993) [28], and Ma and Genton (2000) [23] to accommodate periodicity. These periodic robust functions are used in the Yule-Walker equations to obtain robust parameter estimates. The asymptotic central limit theorems of the estimators are established, and an extensive Monte Carlo experiment is conducted to evaluate the performance of the robust methodology for periodic time series with finite sample sizes. The quarterly Fraser River data was used as an example of application of the proposed robust methodology. All the results presented here give strong motivation to use the methodology in practical situations in which periodically correlated time series contain additive outliers.     

Asymptotic results in segmented multiple regression

This paper studies the asymptotic behavior of the least squares estimators in segmented multiple regression. For a model with more than one partitioning variable, each of which has one or more change-points, we study the asymptotic properties of the estimated change-points and regression coefficients. Using techniques in empirical process theory, we prove the consistency of the least squares estimators and also establish the asymptotic normality of the estimated regression coefficients. For the estimated change-points, we obtain their consistency at the rates of or 1/n, with or without continuity constraints, respectively. The change-points estimated under the continuity constraints are also shown to asymptotically have a multivariate normal distribution. For the case where the regression mean functions are not assumed to be continuous at the change-points, the asymptotic distribution of the estimated change-points involves a step function process, whose distribution does not follow a well-known distribution.     

A van Trees inequality for estimators on manifolds

Van Trees’ Bayesian version of the Cramér-Rao inequality is generalised here to the context of smooth loss functions on manifolds and estimation of parameters of interest. This extends the multivariate van Trees inequality of Gill and Levit (1995) [R.D. Gill, B.Y. Levit, Applications of the van Trees inequality: a Bayesian Cramér-Rao bound, Bernoulli 1 (1995) 59-79]. In addition, the intrinsic Cramér-Rao inequality of Hendriks (1991) [H. Hendriks, A Cramér-Rao type lower bound for estimators with values in a manifold, J. Multivariate Anal. 38 (1991) 245-261] is extended to cover estimators which may be biased. The quantities used in the new inequalities are described in differential-geometric terms. Some examples are given.