Asymptotic expansions of the distributions of estimators in canonical correlation analysis under nonnormality

Asymptotic expansions of the distributions of typical estimators in canonical correlation analysis under nonnormality are obtained. The expansions include the Edgeworth expansions up to order O(1/n) for the parameter estimators standardized by the population standard errors, and the corresponding expansion by Hall's method with variable transformation. The expansions for the Studentized estimators are also given using the Cornish-Fisher expansion and Hall's method. The parameter estimators are dealt with in the context of estimation for the covariance structure in canonical correlation analysis. The distributions of the associated statistics (the structure of the canonical variables, the scaled log likelihood ratio and Rozeboom's between-set correlation) are also expanded. The robustness of the normal-theory asymptotic variances of the sample canonical correlations and associated statistics are shown when a latent variable model holds. Simulations are performed to see the accuracy of the asymptotic results in finite samples.     

Asymptotic expansions of test statistics for dimensionality and additional information in canonical correlation analysis when the dimension is large

This paper examines asymptotic expansions of test statistics for dimensionality and additional information in canonical correlation analysis based on a sample of size N=n+1 on two sets of variables, i.e.,  and . These problems are related to dimension reduction. The asymptotic approximations of the statistics have been studied extensively when dimensions p₁ and p₂ are fixed and the sample size N tends to infinity. However, the approximations worsen as p₁ and p₂ increase. This paper derives asymptotic expansions of the test statistics when both the sample size and dimension are large, assuming that and have a joint (p₁+p₂)-variate normal distribution. Numerical simulations revealed that this approximation is more accurate than the classical approximation as the dimension increases.     

A test for additional information in canonical correlation analysis

Summary In canonical correlation analysis a hypothesis concerning the relevance of a subset of variables from each of the two given variable sets is formulated. The likelihood ratio statistic for the hypothesis and an asymptotic expansion for its null distribution are obtained. In discriminant analysis various alternative forms of a hypothesis concerning the relevance of a specified variable subset are also discussed.     

Inference in canonical correlation analysis

The asymptotic behavior, for large sample size, is given for the distribution of the canonical correlation coefficients. The result is used to examine the Bartlett-Lawley test that the residual population canonical correlation coefficients are zero. A marginal likelihood function for the population coefficients is obtained and the maximum marginal likelihood estimates are shown to provide a bias correction.     

Generalized canonical analysis for time series

Canonical correlation analysis is shown to be equivalent to the problem of estimating a linear regression matrix, B₀, of less than full rank. After reparameterizing B₀ some estimates of the new parameters, obtained by solving an eigenvalue problem and closely related to canonical correlations and vectors, are found to be consistent and efficient when the residuals are mutually independent. When the residuals are generated by an autocorrelated, stationary time series these estimates are still consistent and obey a central limit theorem, but they are no longer efficient. Alternative, more general estimates are suggested which are efficient in the presence of serial correlation. Asymptotic theory and iterative computational procedures for these estimates are given. A likelihoodratio test for the rank of B₀ is seen to be an extension of a familiar test for canonical correlations.     

Third-order power comparisons for a class of tests for multivariate linear hypothesis under general distributions

The purpose of this paper is, in multivariate linear regression model (Part I) and GMANOVA model (Part II), to investigate the effect of nonnormality upon the nonnull distributions of some multivariate test statistics under normality. It is shown that whatever the underlying distributions, the difference of local powers up to order N⁻¹ after either Bartlett’s type adjustment or Cornish-Fisher’s type size adjustment under nonnormality coincides with that in Anderson [An Introduction to Multivariate Statistical Analysis, 2nd ed. and 3rd ed., Wiley, New York, 1984, 2003] under normality. The derivation of asymptotic expansions is based on the differential operator associated with the multivariate linear regression model under general distributions. The performance of higher-order results in finite samples, including monotone Bartlett’s type adjustment and monotone Cornish-Fisher’s type size adjustment, is examined using simulation studies.     

Consistent tuning parameter selection in high dimensional sparse linear regression

An exhaustive search as required for traditional variable selection methods is impractical in high dimensional statistical modeling. Thus, to conduct variable selection, various forms of penalized estimators with good statistical and computational properties, have been proposed during the past two decades. The attractive properties of these shrinkage and selection estimators, however, depend critically on the size of regularization which controls model complexity. In this paper, we consider the problem of consistent tuning parameter selection in high dimensional sparse linear regression where the dimension of the predictor vector is larger than the size of the sample. First, we propose a family of high dimensional Bayesian Information Criteria (HBIC), and then investigate the selection consistency, extending the results of the extended Bayesian Information Criterion (EBIC), in Chen and Chen (2008) to ultra-high dimensional situations. Second, we develop a two-step procedure, the SIS+AENET, to conduct variable selection in p>n situations. The consistency of tuning parameter selection is established under fairly mild technical conditions. Simulation studies are presented to confirm theoretical findings, and an empirical example is given to illustrate the use in the internet advertising data.     

Functional canonical analysis for square integrable stochastic processes

We study the extension of canonical correlation from pairs of random vectors to the case where a data sample consists of pairs of square integrable stochastic processes. Basic questions concerning the definition and existence of functional canonical correlation are addressed and sufficient criteria for the existence of functional canonical correlation are presented. Various properties of functional canonical analysis are discussed. We consider a canonical decomposition, in which the original processes are approximated by means of their canonical components.     

A high-dimensional test for the equality of the smallest eigenvalues of a covariance matrix

For the test of sphericity, Ledoit and Wolf [Ann. Statist. 30 (2002) 1081-1102] proposed a statistic which is robust against high dimensionality. In this paper, we consider a natural generalization of their statistic for the test that the smallest eigenvalues of a covariance matrix are equal. Some inequalities are obtained for sums of eigenvalues and sums of squared eigenvalues. These bounds permit us to obtain the asymptotic null distribution of our statistic, as the dimensionality and sample size go to infinity together, by using distributional results obtained by Ledoit and Wolf [Ann. Statist. 30 (2002) 1081-1102]. Some empirical results comparing our test with the likelihood ratio test are also given.     

Some properties of canonical correlations and variates in infinite dimensions

In this paper the notion of functional canonical correlation as a maximum of correlations of linear functionals is explored. It is shown that the population functional canonical correlation is in general well defined, but that it is a supremum rather than a maximum, so that a pair of canonical variates may not exist in the spaces considered. Also the relation with the maximum eigenvalue of an associated pair of operators and the corresponding eigenvectors is not in general valid. When the inverses of the operators involved are regularized, however, all of the above properties are restored. Relations between the actual population quantities and their regularized versions are also established. The sample functional canonical correlations can be regularized in a similar way, and consistency is shown at a fixed level of the regularization parameter.     

High-dimensional asymptotic expansions for the distributions of canonical correlations

This paper examines asymptotic distributions of the canonical correlations between and with q≤p, based on a sample of size of N=n+1. The asymptotic distributions of the canonical correlations have been studied extensively when the dimensions q and p are fixed and the sample size N tends toward infinity. However, these approximations worsen when q or p is large in comparison to N. To overcome this weakness, this paper first derives asymptotic distributions of the canonical correlations under a high-dimensional framework such that q is fixed, m=n−p→∞ and c=p/n→c₀∈[0,1), assuming that and have a joint (q+p)-variate normal distribution. An extended Fisher’s z-transformation is proposed. Then, the asymptotic distributions are improved further by deriving their asymptotic expansions. Numerical simulations revealed that our approximations are more accurate than the classical approximations for a large range of p,q, and n and the population canonical correlations.     

Similar tests for covariance structures in multivariate linear models

Nyblom (J. Multivariate Anal. 76 (2001) 294) has derived locally best invariant test for the covariance structure in a multivariate linear model. The class of invariant tests obtained by Nyblom [9] does not coincide with the class of similar tests for this testing set-up. This paper extends some of the results of Nyblom [9] by deriving the locally best similar tests for the covariance structure. Moreover, it develops a saddlepoint approximation to optimal weighted average power similar tests (i.e. tests which maximize a weighted average power).     

A test for elliptical symmetry

This paper presents a statistic for testing the hypothesis of elliptical symmetry. The statistic also provides a specialized test of multivariate normality. We obtain the asymptotic distribution of this statistic under the null hypothesis of multivariate normality, and give a bootstrapping procedure for approximating the null distribution of the statistic under an arbitrary elliptically symmetric distribution. We present simulation results to examine the accuracy of the asymptotic distribution and the performance of the bootstrapping procedure. Finally, for selected alternatives, we compare the power of our test statistic with that of recently proposed tests for elliptical symmetry given by Manzotti et al. [A statistic for testing the null hypothesis of elliptical symmetry, J. Multivariate Anal. 81 (2002) 274-285] and Schott [Testing for elliptical symmetry in covariance-matrix-based analyses, Statist. Probab. Lett. 60 (2002) 395-404], and with that of the well known tests for multivariate normality of Mardia [Measures of multivariate skewness and kurtosis with applications, Biometrika 57 (1970) 519-530] and Baringhaus and Henze [A consistent test for multivariate normality based on the empirical characteristic function, Metrika 35 (1988) 339-348].     

Inference under functional proportional and common principal component models

In many situations, when dealing with several populations with different covariance operators, equality of the operators is assumed. Usually, if this assumption does not hold, one estimates the covariance operator of each group separately, which leads to a large number of parameters. As in the multivariate setting, this is not satisfactory since the covariance operators may exhibit some common structure. In this paper, we discuss the extension to the functional setting of the common principal component model that has been widely studied when dealing with multivariate observations. Moreover, we also consider a proportional model in which the covariance operators are assumed to be equal up to a multiplicative constant. For both models, we present estimators of the unknown parameters and we obtain their asymptotic distribution. A test for equality against proportionality is also considered.     

Some high-dimensional tests for a one-way MANOVA

A statistic is proposed for testing the equality of the mean vectors in a one-way multivariate analysis of variance. The asymptotic null distribution of this statistic, as both the sample size and the number of variables go to infinity, is shown to be normal. Thus, this test can be used when the number of variables is not small relative to the sample size. In particular, it can be used when the number of variables exceeds the degrees of freedom for error, a situation in which standard MANOVA tests are invalid. A related statistic, also having an asymptotic normal distribution, is developed for tests concerning the dimensionality of the hyperplane formed by the population mean vectors. The finite sample size performances of the normal approximations are evaluated in a simulation study.     

Quadratic prediction problems in multivariate linear models

Linear and quadratic prediction problems in finite populations have become of great interest to many authors recently. In the present paper, we mainly aim to extend the problem of quadratic prediction from a general linear model, of form , to a multivariate linear model, denoted by with . Firstly, the optimal invariant quadratic unbiased (OIQU) predictor and the optimal invariant quadratic (potentially) biased (OIQB) predictor of for any particular symmetric nonnegative definite matrix satisfying are derived. Secondly, we consider predicting and . The corresponding restricted OIQU predictor and restricted OIQB predictor for them are given. In addition, we also offer four concluding remarks. One concerns the generalization of predicting and , and the others are concerned with three possible extensions from multivariate linear models to growth curve models, to restricted multivariate linear models, and to matrix elliptical linear models.     

Efficiency of test for independence after Box-Cox transformation

We consider the efficiency and the power of the normal theory test for independence after a Box-Cox transformation. We obtain an expression for the correlation between the variates after a Box-Cox transformation in terms of the correlation on the normal scale. We discuss the efficiency of test of independence after a Box-Cox transformation and show that for the family considered it is always more efficient to conduct the test of independence based on Pearson correlation coefficient after transformation to normality. Power of test of independence before and after a Box-Cox transformation is studied for a finite sample size using Monte Carlo simulation. Our results show that we can increase the power of the normal-theory test for independence after estimating the transformation parameter from the data. The procedure has application for generating non-negative random variables with prescribed correlation.     

Estimation for a marginal generalized single-index longitudinal model

In this paper, we suggest an estimating equations based approach to study a general single-index model with a given out-layer link for longitudinal data and treat the classical one as its special case. Within a wide range of bandwidths which is for estimating the inner-layer nonparametric link, the root-n consistency of the estimator of the index can be proved. The estimation efficiency can be achieved even when there is an infinite-dimensional nuisance parameter to be estimated. The performance of the new method is assessed through the comparison with other existing methods and through an application to an epileptic seizure study.