Canonical correlation for stochastic processes

A general notion of canonical correlation is developed that extends the classical multivariate concept to include function-valued random elements X$X$ and Y$Y$. The approach is based on the polar representation of a particular linear operator defined on reproducing kernel Hilbert spaces corresponding to the random functions X$X$ and Y$Y$. In this context, canonical correlations and variables are limits of finite-dimensional subproblems thereby providing a seamless transition between Hotelling’s original development and infinite-dimensional settings. Several infinite-dimensional treatments of canonical correlations that have been proposed for specific problems are shown to be special cases of this general formulation. We also examine our notion of canonical correlation from a large sample perspective and show that the asymptotic behavior of estimators can be tied to that of estimators from standard, finite-dimensional, multivariate analysis.     

Some properties of canonical correlations and variates in infinite dimensions

In this paper the notion of functional canonical correlation as a maximum of correlations of linear functionals is explored. It is shown that the population functional canonical correlation is in general well defined, but that it is a supremum rather than a maximum, so that a pair of canonical variates may not exist in the spaces considered. Also the relation with the maximum eigenvalue of an associated pair of operators and the corresponding eigenvectors is not in general valid. When the inverses of the operators involved are regularized, however, all of the above properties are restored. Relations between the actual population quantities and their regularized versions are also established. The sample functional canonical correlations can be regularized in a similar way, and consistency is shown at a fixed level of the regularization parameter.     

The likelihood ratio tests for the dimensionality of regression coefficients

In this paper we give a unified derivation of the likelihood ratio (LR) statistics for testing the hypothesis on the dimensionality of regression coefficients under a usual MANOVA model. We also derive the LR statistics under a general MANOVA model and study their asymptotic null and nonnull distributions. Further it is shown that the test statistic used by Bartlett [4] for testing the hypothesis that the last p?k canonical correlations are all zero is the LR statistic.     

Separation theorems for singular values of matrices and their applications in multivariate analysis

Separation theorems for singular values of a matrix, similar to the Poincaré separation theorem for the eigenvalues of a Hermitian matrix, are proved. The results are applied to problems in approximating a given r.v. by an r.v. in a specified class. In particular, problems of canonical correlations, reduced rank regression, fitting an orthogonal random variable (r.v.) to a given r.v., and estimation of residuals in the Gauss-Markoff model are discussed. In each case, a solution is obtained by minimizing a suitable norm. In some cases a common solution is shown to minimize a wide class of norms known as unitarily invariant norms introduced by von Neumann.     

A high-dimensional test for the equality of the smallest eigenvalues of a covariance matrix

For the test of sphericity, Ledoit and Wolf [Ann. Statist. 30 (2002) 1081-1102] proposed a statistic which is robust against high dimensionality. In this paper, we consider a natural generalization of their statistic for the test that the smallest eigenvalues of a covariance matrix are equal. Some inequalities are obtained for sums of eigenvalues and sums of squared eigenvalues. These bounds permit us to obtain the asymptotic null distribution of our statistic, as the dimensionality and sample size go to infinity together, by using distributional results obtained by Ledoit and Wolf [Ann. Statist. 30 (2002) 1081-1102]. Some empirical results comparing our test with the likelihood ratio test are also given.     

Inference for the invariance of canonical analysis under linear transformations

Using a recent result about the invariance problem in linear canonical analysis (LCA), we introduce a criterion by means of which one can see if this invariance holds when the related random vectors are transformed by linear maps. Then, the estimation of this criterion is considered as well as the problem of testing for invariance of LCA. Particularly, a new test for additional information in canonical analysis is proposed and simulations are used to gain understanding of the finite sample performance of this test and to compare it with the likelihood ratio test.     

Asymptotic expansions of the distributions of estimators in canonical correlation analysis under nonnormality

Asymptotic expansions of the distributions of typical estimators in canonical correlation analysis under nonnormality are obtained. The expansions include the Edgeworth expansions up to order O(1/n) for the parameter estimators standardized by the population standard errors, and the corresponding expansion by Hall's method with variable transformation. The expansions for the Studentized estimators are also given using the Cornish-Fisher expansion and Hall's method. The parameter estimators are dealt with in the context of estimation for the covariance structure in canonical correlation analysis. The distributions of the associated statistics (the structure of the canonical variables, the scaled log likelihood ratio and Rozeboom's between-set correlation) are also expanded. The robustness of the normal-theory asymptotic variances of the sample canonical correlations and associated statistics are shown when a latent variable model holds. Simulations are performed to see the accuracy of the asymptotic results in finite samples.     

Generalized canonical analysis for time series

Canonical correlation analysis is shown to be equivalent to the problem of estimating a linear regression matrix, B₀, of less than full rank. After reparameterizing B₀ some estimates of the new parameters, obtained by solving an eigenvalue problem and closely related to canonical correlations and vectors, are found to be consistent and efficient when the residuals are mutually independent. When the residuals are generated by an autocorrelated, stationary time series these estimates are still consistent and obey a central limit theorem, but they are no longer efficient. Alternative, more general estimates are suggested which are efficient in the presence of serial correlation. Asymptotic theory and iterative computational procedures for these estimates are given. A likelihoodratio test for the rank of B₀ is seen to be an extension of a familiar test for canonical correlations.     

A chi-square test for dimensionality with non-Gaussian data

The classical theory for testing the null hypothesis that a set of canonical correlation coefficients is zero leads to a chi-square test under the assumption of multi-normality. The test has been used in the context of dimension reduction. In this paper, we study the limiting distribution of the test statistic without the normality assumption, and obtain a necessary and sufficient condition for the chi-square limiting distribution to hold. Implications of the result are also discussed for the problem of dimension reduction.     

A generalization of the growth curve model which allows missing data

This study presents methods for estimating and testing hypotheses about linear functions of the unknown parameters in a generalization of the growth curve model which allows missing data. The estimators proposed are best asymptotically normal (BAN). A testing method for large samples is described which uses a test criterion given in general form by Wald. The asymptotic null distribution of the test statistic is a central chi-square variable. A BAN estimator of a linear vector function of the unknown parameters of the expectation model and consistent estimators of the variance-covariance parameters are required for computation.     

Measures of influence for the functional linear model with scalar response

This paper studies how to identify influential observations in the functional linear model in which the predictor is functional and the response is scalar. Measurement of the effects of a single observation on estimation and prediction when the model is estimated by the principal components method is undertaken. For that, three statistics are introduced for measuring the influence of each observation on estimation and prediction of the functional linear model with scalar response that are generalizations of the measures proposed for the standard regression model by [D.R. Cook, Detection of influential observations in linear regression, Technometrics 19 (1977) 15-18; D. Peña, A new statistic for influence in linear regression, Technometrics 47 (2005) 1-12] respectively. A smoothed bootstrap method is proposed to estimate the quantiles of the influence measures, which allows us to point out which observations have the larger influence on estimation and prediction. The behavior of the three statistics and the quantile estimation bootstrap based method is analyzed via a simulation study. Finally, the practical use of the proposed statistics is illustrated by the analysis of a real data example, which show that the proposed measures are useful for detecting heterogeneity in the functional linear model with scalar response.     

Estimation for parameters of interest in random effects growth curve models

In this paper, we consider the general growth curve model with multivariate random effects covariance structure and provide a new simple estimator for the parameters of interest. This estimator is not only convenient for testing the hypothesis on the corresponding parameters, but also has higher efficiency than the least-square estimator and the improved two-stage estimator obtained by Rao under certain conditions. Moreover, we obtain the necessary and sufficient condition for the new estimator to be identical to the best linear unbiased estimator. Examples of its application are given.     

Analysis of variance and linear contrasts in experimental design with generalized secant hyperbolic distribution

We consider one-way classification model in experimental design when the errors have generalized secant hyperbolic distribution. We obtain efficient and robust estimators for block effects by using the modified maximum likelihood estimation (MML) methodology. A test statistic analogous to the normal-theory F statistic is defined to test block effects. We also define a test statistic for testing linear contrasts. It is shown that test statistics based on MML estimators are efficient and robust. The methodology readily extends to unbalanced designs.     

Influence functions and efficiencies of the canonical correlation and vector estimates based on scatter and shape matrices

In this paper, the influence functions and limiting distributions of the canonical correlations and coefficients based on affine equivariant scatter matrices are developed for elliptically symmetric distributions. General formulas for limiting variances and covariances of the canonical correlations and canonical vectors based on scatter matrices are obtained. Also the use of the so-called shape matrices in canonical analysis is investigated. The scatter and shape matrices based on the affine equivariant Sign Covariance Matrix as well as the Tyler's shape matrix serve as examples. Their finite sample and limiting efficiencies are compared to those of the Minimum Covariance Determinant estimators and S-estimator through theoretical and simulation studies. The theory is illustrated by an example.     

Representations of best linear unbiased estimators in the Gauss-Markoff model with a singular dispersion matrix

In the general Gauss-Markoff model (Y, Xβ, σ²V), when V is singular, there exist linear functions of Y which vanish with probability 1 imposing some restrictions on Y as well as on the unknown β. In all earlier work on linear estimation, representations of best-linear unbiased estimators (BLUE's) are obtained under the assumption: “L′Y is unbiased for Xβ ? L′X = X.” Such a condition is not, however, necessary. The present paper provides all possible representations of the BLUE's some of which violate the condition L′X = X. Representations of X for given classes of BLUE's are also obtained.     

Preserving multivariate dispersion: An application to the Wishart distribution

Univariate dispersive ordering has been extensively characterized by many authors over the last two decades. However, the multivariate version lacks extensive analysis. In this paper, sufficient and necessary conditions are given to preserve the strong multivariate dispersion order through properties of the corresponding transformation. Finally, these results are applied to the Wishart distribution which can be viewed as “the spread of the dispersion”.     

A test for additional information in canonical correlation analysis

Summary In canonical correlation analysis a hypothesis concerning the relevance of a subset of variables from each of the two given variable sets is formulated. The likelihood ratio statistic for the hypothesis and an asymptotic expansion for its null distribution are obtained. In discriminant analysis various alternative forms of a hypothesis concerning the relevance of a specified variable subset are also discussed.     

A multivariate nonparametric test of independence

A new nonparametric approach to the problem of testing the joint independence of two or more random vectors in arbitrary dimension is developed based on a measure of association determined by interpoint distances. The population independence coefficient takes values between 0 and 1, and equals zero if and only if the vectors are independent. We show that the corresponding statistic has a finite limit distribution if and only if the two random vectors are independent; thus we have a consistent test for independence. The coefficient is an increasing function of the absolute value of product moment correlation in the bivariate normal case, and coincides with the absolute value of correlation in the Bernoulli case. A simple modification of the statistic is affine invariant. The independence coefficient and the proposed statistic both have a natural extension to testing the independence of several random vectors. Empirical performance of the test is illustrated via a comparative Monte Carlo study.     

High-dimensional asymptotic expansions for the distributions of canonical correlations

This paper examines asymptotic distributions of the canonical correlations between and with q≤p, based on a sample of size of N=n+1. The asymptotic distributions of the canonical correlations have been studied extensively when the dimensions q and p are fixed and the sample size N tends toward infinity. However, these approximations worsen when q or p is large in comparison to N. To overcome this weakness, this paper first derives asymptotic distributions of the canonical correlations under a high-dimensional framework such that q is fixed, m=n−p→∞ and c=p/n→c₀∈[0,1), assuming that and have a joint (q+p)-variate normal distribution. An extended Fisher’s z-transformation is proposed. Then, the asymptotic distributions are improved further by deriving their asymptotic expansions. Numerical simulations revealed that our approximations are more accurate than the classical approximations for a large range of p,q, and n and the population canonical correlations.     

Multivariate analysis of variance with fewer observations than the dimension

In this article, we consider the problem of testing a linear hypothesis in a multivariate linear regression model which includes the case of testing the equality of mean vectors of several multivariate normal populations with common covariance matrix Σ, the so-called multivariate analysis of variance or MANOVA problem. However, we have fewer observations than the dimension of the random vectors. Two tests are proposed and their asymptotic distributions under the hypothesis as well as under the alternatives are given under some mild conditions. A theoretical comparison of these powers is made.