Rank estimation in reduced-rank regression

Reduced rank regression assumes that the coefficient matrix in a multivariate regression model is not of full rank. The unknown rank is traditionally estimated under the assumption of normal responses. We derive an asymptotic test for the rank that only requires the response vector have finite second moments. The test is extended to the nonconstant covariance case. Linear combinations of the components of the predictor vector that are estimated to be significant for modelling the responses are obtained.     

On a dimension reduction regression with covariate adjustment

In this paper, we consider a semiparametric modeling with multi-indices when neither the response nor the predictors can be directly observed and there are distortions from some multiplicative factors. In contrast to the existing methods in which the response distortion deteriorates estimation efficacy even for a simple linear model, the dimension reduction technique presented in this paper interestingly does not have to account for distortion of the response variable. The observed response can be used directly whether distortion is present or not. The resulting dimension reduction estimators are shown to be consistent and asymptotically normal. The results can be employed to test whether the central dimension reduction subspace has been estimated appropriately and whether the components in the basis directions in the space are significant. Thus, the method provides an alternative for determining the structural dimension of the subspace and for variable selection. A simulation study is carried out to assess the performance of the proposed method. The analysis of a real dataset demonstrates the potential usefulness of distortion removal.     

Robust dimension reduction based on canonical correlation

The canonical correlation (CANCOR) method for dimension reduction in a regression setting is based on the classical estimates of the first and second moments of the data, and therefore sensitive to outliers. In this paper, we study a weighted canonical correlation (WCANCOR) method, which captures a subspace of the central dimension reduction subspace, as well as its asymptotic properties. In the proposed WCANCOR method, each observation is weighted based on its Mahalanobis distance to the location of the predictor distribution. Robust estimates of the location and scatter, such as the minimum covariance determinant (MCD) estimator of Rousseeuw [P.J. Rousseeuw, Multivariate estimation with high breakdown point, Mathematical Statistics and Applications B (1985) 283-297], can be used to compute the Mahalanobis distance. To determine the number of significant dimensions in WCANCOR, a weighted permutation test is considered. A comparison of SIR, CANCOR and WCANCOR is also made through simulation studies to show the robustness of WCANCOR to outlying observations. As an example, the Boston housing data is analyzed using the proposed WCANCOR method.     

A semiparametric density estimator based on elliptical distributions

In the paper we study a semiparametric density estimation method based on the model of an elliptical distribution. The method considered here shows a way to overcome problems arising from the curse of dimensionality. The optimal rate of the uniform strong convergence of the estimator under consideration coincides with the optimal rate for the usual one-dimensional kernel density estimator except in a neighbourhood of the mean. Therefore the optimal rate does not depend on the dimension. Moreover, asymptotic normality of the estimator is proved.     

Sparse principal component analysis via regularized low rank matrix approximation

Principal component analysis (PCA) is a widely used tool for data analysis and dimension reduction in applications throughout science and engineering. However, the principal components (PCs) can sometimes be difficult to interpret, because they are linear combinations of all the original variables. To facilitate interpretation, sparse PCA produces modified PCs with sparse loadings, i.e. loadings with very few non-zero elements. In this paper, we propose a new sparse PCA method, namely sparse PCA via regularized SVD (sPCA-rSVD). We use the connection of PCA with singular value decomposition (SVD) of the data matrix and extract the PCs through solving a low rank matrix approximation problem. Regularization penalties are introduced to the corresponding minimization problem to promote sparsity in PC loadings. An efficient iterative algorithm is proposed for computation. Two tuning parameter selection methods are discussed. Some theoretical results are established to justify the use of sPCA-rSVD when only the data covariance matrix is available. In addition, we give a modified definition of variance explained by the sparse PCs. The sPCA-rSVD provides a uniform treatment of both classical multivariate data and high-dimension-low-sample-size (HDLSS) data. Further understanding of sPCA-rSVD and some existing alternatives is gained through simulation studies and real data examples, which suggests that sPCA-rSVD provides competitive results.     

Variable screening in predicting clinical outcome with high-dimensional microarrays

Statistical modeling is an important area of biomarker research of important genes for new drug targets, drug candidate validation, disease diagnoses, personalized treatment, and prediction of clinical outcome of a treatment. A widely adopted technology is the use of microarray data that are typically very high dimensional. After screening chromosomes for relative genes using methods such as quantitative trait locus mapping, there may still be a few thousands of genes related to the clinical outcome of interest. On the other hand, the sample size (the number of subjects) in a clinical study is typically much smaller. Under the assumption that only a few important genes are actually related to the clinical outcome, we propose a variable screening procedure to eliminate genes having negligible effects on the clinical outcome. Once the dimension of microarray data is reduced to a manageable number relative to the sample size, one can select a final set of genes via a well-known variable selection method such as the cross-validation. We establish the asymptotic consistency of the proposed variable screening procedure. Some simulation results are also presented.     

Separation theorems for singular values of matrices and their applications in multivariate analysis

Separation theorems for singular values of a matrix, similar to the Poincaré separation theorem for the eigenvalues of a Hermitian matrix, are proved. The results are applied to problems in approximating a given r.v. by an r.v. in a specified class. In particular, problems of canonical correlations, reduced rank regression, fitting an orthogonal random variable (r.v.) to a given r.v., and estimation of residuals in the Gauss-Markoff model are discussed. In each case, a solution is obtained by minimizing a suitable norm. In some cases a common solution is shown to minimize a wide class of norms known as unitarily invariant norms introduced by von Neumann.     

Sufficient dimension reduction for the conditional mean with a categorical predictor in multivariate regression

Recent sufficient dimension reduction methodologies in multivariate regression do not have direct application to a categorical predictor. For this, we define the multivariate central partial mean subspace and propose two methodologies to estimate it. The first method uses the ordinary least squares. Chi-squared distributed statistics for dimension tests are constructed, and an estimate of the target subspace is consistent and efficient. Moreover, the effects of continuous predictors can be tested without assuming any model. The second method extends Iterative Hessian Transformation to this context. For dimension estimation, permutation tests are used. Simulated and real data examples for illustrating various properties of the proposed methods are presented.     

A chi-square test for dimensionality with non-Gaussian data

The classical theory for testing the null hypothesis that a set of canonical correlation coefficients is zero leads to a chi-square test under the assumption of multi-normality. The test has been used in the context of dimension reduction. In this paper, we study the limiting distribution of the test statistic without the normality assumption, and obtain a necessary and sufficient condition for the chi-square limiting distribution to hold. Implications of the result are also discussed for the problem of dimension reduction.     

Nonparametric tests for conditional independence in two-way contingency tables

Testing for the independence between two categorical variables R and S forming a contingency table is a well-known problem: the classical chi-square and likelihood ratio tests are used. Suppose now that for each individual a set of p characteristics is also observed. Those explanatory variables, likely to be associated with R and S, can play a major role in their possible association, and it can therefore be interesting to test the independence between R and S conditionally on them. In this paper, we propose two nonparametric tests which generalise the chi-square and the likelihood ratio ideas to this case. The procedure is based on a kernel estimator of the conditional probabilities. The asymptotic law of the proposed test statistics under the conditional independence hypothesis is derived; the finite sample behaviour of the procedure is analysed through some Monte Carlo experiments and the approach is illustrated with a real data example.     

Local efficiency of a Cramér-von Mises test of independence

Deheuvels proposed a rank test of independence based on a Cramér-von Mises functional of the empirical copula process. Using a general result on the asymptotic distribution of this process under sequences of contiguous alternatives, the local power curve of Deheuvels’ test is computed in the bivariate case and compared to that of competing procedures based on linear rank statistics. The Gil-Pelaez inversion formula is used to make additional comparisons in terms of a natural extension of Pitman's measure of asymptotic relative efficiency.     

Asymptotically efficient two-sample rank tests for modal directions on spheres

A general class of optimal and distribution-free rank tests for the two-sample modal directions problem on (hyper-) spheres is proposed, along with an asymptotic distribution theory for such spherical rank tests. The asymptotic optimality of the spherical rank tests in terms of power-equivalence to the spherical likelihood ratio tests is studied, while the spherical Wilcoxon rank test, an important case for the class of spherical rank tests, is further investigated. A data set is reanalyzed and some errors made in previous studies are corrected. On the usual sphere, a lower bound on the asymptotic Pitman relative efficiency relative to Hotelling’s T²-type test is established, and a new distribution for which the spherical Wilcoxon rank test is optimal is also introduced.     

Functional semiparametric partially linear model with autoregressive errors

In this paper, we introduce a functional semiparametric model, where a real-valued random variable is explained by the sum of a unknown linear combination of the components of a multivariate random variable and an unknown transformation of a functional random variable. The errors can be autocorrelated. We focus here on the parametric estimation of the coefficients in the linear combination. First, we use a nonparametric kernel method to remove the effect of the functional explanatory variable. Then, we use generalized least squares approach to obtain an estimator of these coefficients. Under some technical assumptions, we prove consistency and asymptotic normality of our estimator. Finally, we present Monte Carlo simulations that illustrate these characteristics.     

Estimation of the precision matrix of a singular Wishart distribution and its application in high-dimensional data

In this article, the Stein-Haff identity is established for a singular Wishart distribution with a positive definite mean matrix but with the dimension larger than the degrees of freedom. This identity is then used to obtain estimators of the precision matrix improving on the estimator based on the Moore-Penrose inverse of the Wishart matrix under the Efron-Morris loss function and its variants. Ridge-type empirical Bayes estimators of the precision matrix are also given and their dominance properties over the usual one are shown using this identity. Finally, these precision estimators are used in a quadratic discriminant rule, and it is shown through simulation that discriminant methods based on the ridge-type empirical Bayes estimators provide higher correct classification rates.     

Asymptotic optimal inference for multivariate branching-Markov processes via martingale estimating functions and mixed normality

Multivariate tree-indexed Markov processes are discussed with applications. A Galton-Watson super-critical branching process is used to model the random tree-indexed process. Martingale estimating functions are used as a basic framework to discuss asymptotic properties and optimality of estimators and tests. The limit distributions of the estimators turn out to be mixtures of normals rather than normal. Also, the non-null limit distributions of standard test statistics such as Wald, Rao’s score, and likelihood ratio statistics are shown to have mixtures of non-central chi-square distributions. The models discussed in this paper belong to the local asymptotic mixed normal family. Consequently, non-standard limit results are obtained.     

A generalization of the growth curve model which allows missing data

This study presents methods for estimating and testing hypotheses about linear functions of the unknown parameters in a generalization of the growth curve model which allows missing data. The estimators proposed are best asymptotically normal (BAN). A testing method for large samples is described which uses a test criterion given in general form by Wald. The asymptotic null distribution of the test statistic is a central chi-square variable. A BAN estimator of a linear vector function of the unknown parameters of the expectation model and consistent estimators of the variance-covariance parameters are required for computation.     

Self-consistent estimation of censored quantile regression

The principle of self-consistency has been employed to estimate regression quantile with randomly censored response. The asymptotic studies for this type of approach was established only recently, partly due to the complex forms of the current self-consistent estimators of censored regression quantiles. Of interest, how the self-consistent estimation of censored regression quantiles is connected to the alternative martingale-based approach still remains uncovered. In this paper, we propose a new formulation of self-consistent censored regression quantiles based on stochastic integral equations. The proposed representation of censored regression quantiles entails a clearly defined estimation procedure. More importantly, it greatly simplifies the theoretical investigations. We establish the large sample equivalence between the proposed self-consistent estimators and the existing estimator derived from martingale-based estimating equations. The connection between the new self-consistent estimation approach and the available self-consistent algorithms is also elaborated.     

Asymptotic distribution of Wishart matrix for block-wise dispersion of population eigenvalues

This paper deals with the asymptotic distribution of Wishart matrix and its application to the estimation of the population matrix parameter when the population eigenvalues are block-wise infinitely dispersed. We show that the appropriately normalized eigenvectors and eigenvalues asymptotically generate two Wishart matrices and one normally distributed random matrix, which are mutually independent. For a family of orthogonally equivariant estimators, we calculate the asymptotic risks with respect to the entropy or the quadratic loss function and derive the asymptotically best estimator among the family. We numerically show (1) the convergence in both the distributions and the risks are quick enough for a practical use, (2) the asymptotically best estimator is robust against the deviation of the population eigenvalues from the block-wise infinite dispersion.     

The rank of the covariance matrix of an evanescent field

Evanescent random fields arise as a component of the 2D Wold decomposition of homogeneous random fields. Besides their theoretical importance, evanescent random fields have a number of practical applications, such as in modeling the observed signal in the space-time adaptive processing (STAP) of airborne radar data. In this paper we derive an expression for the rank of the low-rank covariance matrix of a finite dimension sample from an evanescent random field. It is shown that the rank of this covariance matrix is completely determined by the evanescent field spectral support parameters, alone. Thus, the problem of estimating the rank lends itself to a solution that avoids the need to estimate the rank from the sample covariance matrix. We show that this result can be immediately applied to considerably simplify the estimation of the rank of the interference covariance matrix in the STAP problem.     

Multivariate analysis of variance with fewer observations than the dimension

In this article, we consider the problem of testing a linear hypothesis in a multivariate linear regression model which includes the case of testing the equality of mean vectors of several multivariate normal populations with common covariance matrix Σ, the so-called multivariate analysis of variance or MANOVA problem. However, we have fewer observations than the dimension of the random vectors. Two tests are proposed and their asymptotic distributions under the hypothesis as well as under the alternatives are given under some mild conditions. A theoretical comparison of these powers is made.