Density Estimation on the Stiefel Manifold

This paper develops the theory of density estimation on the Stiefel manifoldV_{k, m}, whereV_{k, m}is represented by the set ofm×kmatricesXsuch thatX′X=I_k, thek×kidentity matrix. The density estimation by the method of kernels is considered, proposing two classes of kernel density estimators with small smoothing parameter matrices and for kernel functions of matrix argument. Asymptotic behavior of various statistical measures of the kernel density estimators is investigated for small smoothing parameter matrix and/or for large sample size. Some decompositions of the Stiefel manifoldV_{k, m}play useful roles in the investigation, and the general discussion is applied and examined for a special kernel function. Alternative methods of density estimation are suggested, using decompositions ofV_{k, m}.     

Concentrated matrix Langevin distributions

This paper concerns the matrix Langevin distributions, exponential-type distributions defined on the two manifolds of our interest, the Stiefel manifold V_k,m and the manifold P_k,m−k of m×m orthogonal projection matrices idempotent of rank k which is equivalent to the Grassmann manifold G_k,m−k. Asymptotic theorems are derived when the concentration parameters of the distributions are large. We investigate the asymptotic behavior of distributions of some (matrix) statistics constructed based on the sample mean matrices in connection with testing hypotheses of the orientation parameters, and obtain asymptotic results in the estimation of large concentration parameters and in the classification of the matrix Langevin distributions.     

High dimensional limit theorems and matrix decompositions on the Stiefel manifold

The main purpose of this paper is to investigate high dimensional limiting behaviors, as m becomes infinite (m → ∞), of matrix statistics on the Stiefel manifold V_{k, m}, which consists of m × k (m ≥ k) matrices X such that X′X = I_k. The results extend those of Watson. Let X be a random matrix on V_{k, m}. We present a matrix decomposition of X as the sum of mutually orthogonal singular value decompositions of the projections P X and P X, where and are each a subspace of R^m of dimension p and their orthogonal compliment, respectively (p ≥ k and m ≥ k + p). Based on this decomposition of X, the invariant measure on V_{k, m} is expressed as the product of the measures on the component subspaces. Some distributions related to these decompositions are obtained for some population distributions on V_{k, m}. We show the limiting normalities, as m → ∞, of some matrix statistics derived from the uniform distribution and the distributions having densities of the general forms f(P X) and f(m^1/2P X) on V_{k, m}. Subsequently, applications of these high dimensional limit theorems are considered in some testing problems.     

Elementary Symmetric Polynomials in Random Variables

The subject of the paper is the probability-theoretic properties of elementary symmetric polynomials σ _k of arbitrary degree k in random variables X _i (i=1,2,…,m) defined on special subsets of commutative rings ℛ _m with identity of finite characteristic m. It is shown that the probability distributions of the random elements σ _k (X ₁,…,X _m ) tend to a limit when m→∞ if X ₁,…,X _m form a Markov chain of finite degree μ over a finite set of states V, V⊂ℛ _m , with positive conditional probabilities. Moreover, if all the conditional probabilities exceed a prescribed positive number α, the limit distributions do not depend on the choice of the chain.      

Modifiable low‐rank approximation to a matrix

A truncated ULV decomposition (TULVD) of an m×n matrix X of rank k is a decomposition of the form X = ULV^T+E, where U and V are left orthogonal matrices, L is a k×k non‐singular lower triangular matrix, and E is an error matrix. Only U,V, L, and ∥E∥_F are stored, but E is not stored. We propose algorithms for updating and downdating the TULVD. To construct these modification algorithms, we also use a refinement algorithm based upon that in (SIAM J. Matrix Anal. Appl. 2005; 27 (1):198–211) that reduces ∥E∥_F, detects rank degeneracy, corrects it, and sharpens the approximation. Copyright © 2009 John Wiley & Sons, Ltd.     

On the Lusternik-Schnirelmann category of Stiefel manifolds

We determine the Lusternik-Schnirelmann category of real Stiefel manifolds Vn,k and quaternionic Stiefel manifolds Xn,k for n?2k which is equal to the cup-length of the mod 2 cohomology of Vn,k and the integer cohomology of Xn,k, respectively.     

On the Expected Number of Shadow Vertices of the Convex Hull of Random Points

Leta₁, . . . ,a_mbe independent random points in ⁿthat are independent and identically distributed spherically symmetrical in ⁿ. Moreover, letXbe the random polytope generated as the convex hull ofa₁, . . . ,a_mand letL_kbe an arbitraryk-dimensional subspace of ⁿwith 2 ≤k≤n− 1. LetX_kbe the orthogonal projection image ofXinL_k. We call those vertices ofXwhose projection images inL_kare vertices ofX_kshadow vertices ofXwith respect to the subspaceL_k. We derive a distribution independent sharp upper bound for the expected number of shadow vertices ofXinL_k.     

On a sub‐Stiefel Procrustes problem arising in computer vision

A sub‐Stiefel matrix is a matrix that results from deleting simultaneously the last row and the last column of an orthogonal matrix. In this paper, we consider a Procrustes problem on the set of sub‐Stiefel matrices of order n. For n = 2, this problem has arisen in computer vision to solve the surface unfolding problem considered by R. Fereirra, J. Xavier and J. Costeira. An iterative algorithm for computing the solution of the sub‐Stiefel Procrustes problem for an arbitrary n is proposed, and some numerical experiments are carried out to illustrate its performance. For these purposes, we investigate the properties of sub‐Stiefel matrices. In particular, we derive two necessary and sufficient conditions for a matrix to be sub‐Stiefel. We also relate the sub‐Stiefel Procrustes problem with the Stiefel Procrustes problem and compare it with the orthogonal Procrustes problem. Copyright © 2015 John Wiley & Sons, Ltd.     

Strong convergence rates for the estimation of a covariance operator for associated samples

Let X _n , n ≥ 1, be a strictly stationary associated sequence of random variables, with common continuous distribution function F. Using histogram type estimators we consider the estimation of the two-dimensional distribution function of (X ₁,X _k+1) as well as the estimation of the covariance function of the limit empirical process induced by the sequence X _n , n ≥ 1. Assuming a convenient decrease rate of the covariances Cov(X ₁,X _n+1), n ≥ 1, we derive uniform strong convergence rates for these estimators. The condition on the covariance structure of the variables is satisfied either if Cov(X ₁,X _n+1) decreases polynomially or if it decreases geometrically, but as we could expect, under the latter condition we are able to establish faster convergence rates. For the two-dimensional distribution function the rate of convergence derived under a geometrical decrease of the covariances is close to the optimal rate for independent samples.      

Minimax estimation of the mean of spherically symmetric distributions under general quadratic loss

For X one observation on a p-dimensional (p ≥ 4) spherically symmetric (s.s.) distribution about θ, minimax estimators whose risks dominate the risk of X (the best invariant procedure) are found with respect to general quadratic loss, L(δ, θ) = (δ − θ)′ D(δ − θ) where D is a known p × p positive definite matrix. For C a p × p known positive definite matrix, conditions are given under which estimators of the form δ_a,r,C,D(X) = (I − (ar(|X|²)) D^−1/2CD^1/2 |X|⁻²)X are minimax with smaller risk than X. For the problem of estimating the mean when n observations X₁, X₂, …, X_n are taken on a p-dimensional s.s. distribution about θ, any spherically symmetric translation invariant estimator, δ(X₁, X₂, …, X_n), with have a s.s. distribution about θ. Among the estimators which have these properties are best invariant estimators, sample means and maximum likelihood estimators. Moreover, under certain conditions, improved robust estimators can be found.     

Approximation of Projections of Random Vectors

Let X be a d-dimensional random vector and X _θ its projection onto the span of a set of orthonormal vectors {θ ₁,…,θ _k }. Conditions on the distribution of X are given such that if θ is chosen according to Haar measure on the Stiefel manifold, the bounded-Lipschitz distance from X _θ to a Gaussian distribution is concentrated at its expectation; furthermore, an explicit bound is given for the expected distance, in terms of d, k, and the distribution of X, allowing consideration not just of fixed k but of k growing with d. The results are applied in the setting of projection pursuit, showing that most k-dimensional projections of n data points in ℝ ^d are close to Gaussian, when n and d are large and k=clog (d) for a small constant c.     

On some regression relationships relating sample means and successive maxima

In recent years, a number of papers have appeared in which characterizations of a series of distributions in terms of regression properties of order statistics were obtained. In particular, characterizations of the Student’s t ₂-distribution in terms of linear regression relation-ships, including the sample median X _{2, 3} and the sample mean T ₃, were obtained in 2002. Later, similar results were also established for samples of larger sizes, and characterizations of families of distributions containing the t ₂-distribution were put forward. These characterizations are based, in particular, on linear regression relationships relating sample means T _{2k + 1} and sample medians X _{k + 1, 2k + 1}, k = 1, 2, ...     

Simultaneous estimation of eigenvalues

The problem of simultaneous estimation of eigenvalues of covariance matrix is considered for one and two sample problems under a sum of squared error loss. New classes of estimators are obtained which dominate the best multiple of the sample eigenvalues in terms of risk. These estimators shrink or expand the sample eigenvalues towards their geometric mean. Similar results are obtained for the estimation of eigenvalues of the precision matrix and the residual matrix when the original covariance matrix is partitioned into two groups. As a consequence, a new estimator of trace of the covariance matrix is obtained.The results are extended to two sample problem where two Wishart distributions are independently observed, say, S _i W _p ( _i , k _i ), i=1, 2, and eigenvalues of _{1 2} ^-1 are estimated simultaneously. Finally, some numerical calculations are done to obtain the amount of risk improvement.     

Some optimization problems in multivariate statistics

Interesting and important multivariate statistical problems containing principal component analysis, statistical visualization and singular value decomposition, furthermore, one of the basic theorems of linear algebra, the matrix spectral theorem, the characterization of the structural stability of dynamical systems and many others lead to a new class of global optimization problems where the question is to find optimal orthogonal matrices. A special class is where the problem consists in finding, for any 2kn, the dominant k-dimensional eigenspace of an n×n symmetric matrix A in R ⁿ where the eigenspaces are spanned by the k largest eigenvectors. This leads to the maximization of a special quadratic function on the Stiefel manifold M _n,k . Based on the global Lagrange multiplier rule developed in Rapcsák (1997) and the paper dealing with Stiefel manifolds in optimization theory (Rapcsák, 2002), the global optimality conditions of this smooth optimization problem are obtained, then they are applied in concrete cases.     

A Characterization of a Certain Class of Compact Metric Spaces

Compact metric spaces χ of such a kind, that ??_f =??(X), are characterized, ??(X) is the σ-field of BOREL sets and ??_f(X) is the field generated by all open subset of X. Our main result is Theorem 5: If χ is a compact metric space, then the following conditions are equivalent:

• 1 ??_f(X) =??(X).
• 2 card (X) ≦x₀ and there are k, m?N such that card (X^(k)) = m.
• 3 There are k, m?N such that χ is homeomorphic to ω^k · m + 1.

     

Asymptotically optimal selection of a piecewise polynomial estimator of a regression function

Let (X, Y) be a pair of random variables such that X = (X₁,…, X_d) ranges over a nondegenerate compact d-dimensional interval C and Y is real-valued. Let the conditional distribution of Y given X have mean θ(X) and satisfy an appropriate moment condition. It is assumed that the distribution of X is absolutely continuous and its density is bounded away from zero and infinity on C. Without loss of generality let C be the unit cube. Consider an estimator of θ having the form of a piecewise polynomial of degree k_n based on m_n^d cubes of length 1/m_n, where the m_n^d(_d^k_n+d) coefficients are chosen by the method of least squares based on a random sample of size n from the distribution of (X, Y). Let (k_n, m_n) be chosen by the FPE procedure. It is shown that the indicated estimator has an asymptotically minimal squared error of prediction if θ is not of the form of piecewise polynomial.     

Efficient and Practical Algorithms for Sequential Modular Decomposition

A module of an undirected graph G = (V, E) is a set X of vertices that have the same set of neighbors in V\X. The modular decomposition is a unique decomposition of the vertices into nested modules. We give a practical algorithm with an O(n + mα(m, n)) time bound and a variant with a linear time bound.     

Order Determination for Multivariate Autoregressive Processes Using Resampling Methods

LetX₁, …, X_nbe observations from a multivariate AR(p) model with unknown orderp. A resampling procedure is proposed for estimating the orderp. The classical criteria, such as AIC and BIC, estimate the orderpas the minimizer of the function[formula]wherenis the sample size,kis the order of the fitted model, Σ²_kis an estimate of the white noise covariance matrix, andC_nis a sequence of specified constants (for AIC,C_n=2m²/n, for Hannan and Quinn's modification of BIC,C_n=2m²(ln ln n)/n, wheremis the dimension of the data vector). A resampling scheme is proposed to estimate an improved penalty factorC_n. Conditional on the data, this procedure produces a consistent estimate ofp. Simulation results support the effectiveness of this procedure when compared with some of the traditional order selection criteria. Comments are also made on the use of Yule–Walker as opposed to conditional least squares estimations for order selection.     

Décomposition d"une théorie du potentiel en des sous-espaces elliptiques et paraboliques

Let V be a proper kernel on a measurable space (X, B), and E _V the cone of excessive functions generated by V. We give a necessary and sufficient condition to decompose the space (X, E _V ) in an ordered and countable family of subspaces (X _n, E _{V n} ), which are elliptic or parabolic. The X _i 's are finely open and measurable and form a partition of X. The kernel W = V _i on X _i is subordinate to V, and has a triangular matrix. Résumé. Soit V un noyau propre sur un espace mesurable (X, B), et E _V le cône des fonctions excessives engendré par V. Nous donnons une condition nécéssaire et suffisante pour décomposer l"espace (X, E _V ) en une famille dénombrable et ordonnée (X _n , E _{V n} ) de sous-espaces elliptiques ou paraboliques. Les X _i sont des ouverts fins mesurables et forment une partition de X. Le noyau W = V _i sur X _i est subordonné à V, et sa matrice est triangulaire.     

Simultaneous estimation of location parameters of the distribution with finite support

Summary LetX _i ,i=1,..., p be theith component of thep×1 vectorX=(X ₁,X ₂,...,X _p )′. Suppose thatX ₁,X ₂,...,X _p are independent and thatX _i has a probability density which is positive on a finite interval, is symmetric about θ _i and has the same variance. In estimation of the location vector θ=(θ₁, θ₂,...,θ _p )′ under the squared error loss function explicit estimators which dominateX are obtained by using integration by parts to evaluate the risk function. Further, explicit dominating estimators are given when the distributions ofX _i ′s are mixture of two uniform distributions. For the loss function such an estimator is also given when the distributions ofX _i ′s are uniform distributions.