Common Principal Components for Dependent Random Vectors

Let the kp-variate random vector X be partitioned into k subvectors X_i of dimension p each, and let the covariance matrix Ψ of X be partitioned analogously into submatrices Ψ_ij. The common principal component (CPC) model for dependent random vectors assumes the existence of an orthogonal p by p matrix β such that β^tΨ_ijβ is diagonal for all (i, j). After a formal definition of the model, normal theory maximum likelihood estimators are obtained. The asymptotic theory for the estimated orthogonal matrix is derived by a new technique of choosing proper subsets of functionally independent parameters.     

On improved estimators of the generalized variance

Treated in this paper is the problem of estimating with squared error loss the generalized variance | Σ | from a Wishart random matrix S: p × p W_p(n, Σ) and an independent normal random matrix X: p × k N(ξ, Σ I_k) with ξ(p × k) unknown. Denote the columns of X by X₍₁₎ ,…, X_(k) and set ψ⁽⁰⁾(S, X) = {(n − p + 2)!/(n + 2)!} | S |, ψ⁽ⁱ⁾(X, X) = min[ψ⁽ⁱ⁻¹⁾(S, X), {(n − p + i + 2)!/(n + i + 2)!} | S + X₍₁₎ X′₍₁₎ + + X_(i) X′_(i) |] and Ψ⁽ⁱ⁾(S, X) = min[ψ⁽⁰⁾(S, X), {(n − p + i + 2)!/(n + i + 2)!}| S + X₍₁₎ X′₍₁₎ + + X_(i) X′_(i) |], i = 1,…,k. Our result is that the minimax, best affine equivariant estimator ψ⁽⁰⁾(S, X) is dominated by each of Ψ⁽ⁱ⁾(S, X), i = 1,…,k and for every i, ψ⁽ⁱ⁾(S, X) is better than ψ⁽ⁱ⁻¹⁾(S, X). In particular, ψ^(k)(S, X) = min[{(n − p + 2)!/(n + 2)!} | S |, {(n − p + 2)!/(n + 2)!} | S + X₍₁₎X′₍₁₎|,…,| {(n − p + k + 2)!/(n + k + 2)!} | S + X₍₁₎X′₍₁₎ + + X_(k)X′_(k)|] dominates all other ψ's. It is obtained by considering a multivariate extension of Stein's result (Ann. Inst. Statist. Math. 16, 155–160 (1964)) on the estimation of the normal variance.     

α-Stable characterization of Banach spaces (1 < α < 2)

This work characterizes some subclasses of α-stable (0 < α < 1) Banach spaces in terms of the extendibility to Radon laws of certain α-stable cylinder measures. These result extend the work of S. Chobanian and V. Tarieladze (J. Multivar. Anal.7, 183–203 (1977)). For these spaces it is shown that every Radon stable measure is the continuous image of a stable measure on a suitable L_β space with β = α(1 − α)⁻¹. The latter result extends some work of Garling (Ann. Probab.4, 600–611 (1976)) and Jain (Proceedings, Symposia in Pure Math. XXXI, p. 55–65, Amer. Math. Soc., Providence, R.I.).     

Nonparametric Empirical Bayes Estimation of the Matrix Parameter of the Wishart Distribution

We consider independent pairs (X₁, Σ₁), (X₂, Σ₂), …, (X_n, Σ_n), where eachΣ_iis distributed according to some unknown density functiong(Σ) and, givenΣ_i=Σ,X_ihas conditional density functionq(xΣ) of the Wishart type. In each pair the first component is observable but the second is not. After the (n+1)th observationX_n+1is obtained, the objective is to estimateΣ_n+1corresponding toX_n+1. This estimator is called the empirical Bayes (EB) estimator ofΣ. An EB estimator ofΣis constructed without any parametric assumptions ong(Σ). Its posterior mean square risk is examined, and the estimator is demonstrated to be pointwise asymptotically optimal.     

Test of association between multivariate stable vectors

Based on observations of d-dimensional random vectors in the domain of attraction of a stable distribution with (multi-)index α = (α₁, …, α_d), an estimator for the dependence function of the α_i-stable variables is constructed. The estimator utilizes the α-tail-estimator and an estimator of the spectral measure of the α-stable law. This estimator gives rise to a test of association of the stable components and various quantitative measures of association.     

On stochastic integral representation of stable processes with sample paths in Banach spaces

Certain path properties of a symmetric α-stable process X(t) = ∫_Sh(t, s) dM(s), t T, are studied in terms of the kernel h. The existence of an appropriate modification of the kernel h enables one to use results from stable measures on Banach spaces in studying X. Bounds for the moments of the norm of sample paths of X are obtained. This yields definite bounds for the moments of a double α-stable integral. Also, necessary and sufficient conditions for the absolute continuity of sample paths of X are given. Along with the above stochastic integral representation of stable processes, the representation of stable random vectors due to[13], Ann. Probab.9, 624–632) is extensively used and the relationship between these two representations is discussed.     

Positive definite symmetric functions on finite dimensional spaces. I. Applications of the Radon transform

An n-dimensional random vector X is said (Cambanis, S., Keener, R., and Simons, G. (1983). J. Multivar. Anal., 13 213–233) to have an α-symmetric distribution, α > 0, if its characteristic function is of the form φ(|ξ₁|^α + … + |ξ_n|^α). Using the Radon transform, integral representations are obtained for the density functions of certain absolutely continuous α-symmetric distributions. Series expansions are obtained for a class of apparently new special functions which are encountered during this study. The Radon transform is also applied to obtain the densities of certain radially symmetric stable distributions on ⁿ. A new class of “zonally” symmetric stable laws on ⁿ is defined, and series expansions are derived for their characteristic functions and densities.     

Third-order asymptomic properties of a class of test statistics under a local alternative

Suppose that {X_i; I = 1, 2, …,} is a sequence of p-dimensional random vectors forming a stochastic process. Let p_n, θ(X_n), X_n ^np, be the probability density function of X_n = (X₁, …, X_n) depending on θ Θ, where Θ is an open set of ¹. We consider to test a simple hypothesis H : θ = θ₀ against the alternative A : θ ≠ θ₀. For this testing problem we introduce a class of tests , which contains the likelihood ratio, Wald, modified Wald, and Rao tests as special cases. Then we derive the third-order asymptotic expansion of the distribution of T under a sequence of local alternatives. Using this result we elucidate various third-order asymptotic properties of T (e.g., Bartlett's adjustments, third-order asymptotically most powerful properties). Our results are very general, and can be applied to the i.i.d. case, multivariate analysis, and time series analysis. Two concrete examples will be given. One is a Gaussian ARMA process (dependent case), and the other is a nonlinear regression model (non-identically distributed case).     

On the conditional probability density functions of multivariate uniform random vectors and multivariate normal random vectors

It is shown that the conditional probability density function of Y₁ given (1/n) Σ_i=1ⁿ Y_i=1Y_i^t = Σ, where Y₁, Y₂,…, Y_n are i.i.d, p-variate uniform random vectors with mean 0 equals to that of Y₁ given (1/n) Σ_i=1ⁿ Y_iY_i^t,…, Y_n are i.i.d, p-variate normal random vectors with mean 0 and covariance matrix Σ.     

Robust M-estimators of location vectors

Let X₁,…,X_n be i.i.d. random vectors in R^m, let θεR^m be an unknown location parameter, and assume that the restriction of the distribution of X₁−θ to a sphere of radius d belongs to a specified neighborhood of distributions spherically symmetric about 0. Under regularity conditions on and d, the parameter θ in this model is identifiable, and consistent M-estimators of θ (i.e., solutions of Σ_i=1ⁿψ(|X_i− |)(X_i− )=0) are obtained by using “re-descenders,” i.e., ψ's wh satisfy ψ(x)=0 for x≥c. An iterative method for solving for is shown to produce consistent and asymptotically normal estimates of θ under all distributions in . The following asymptotic robustness problem is considered: finding the ψ which is best among the re-descenders according to Huber's minimax variance criterion.     

Almost sure central limit theorem for partial sums and maxima

Let X, X₁, X₂, … be i.i.d. random variables with nondegenerate common distribution function F, satisfying EX = 0, EX² = 1. Let X_i and M_n = max{X_i, 1 ≤ i ≤ n }. Suppose there exists constants a_n > 0, b_n ∈ R and a nondegenrate distribution G (y) such that Then, we have almost surely, where f (x, y) denotes the bounded Lipschitz 1 function and Φ(x) is the standard normal distribution function (© 2009 WILEY‐VCH Verlag GmbH & Co. KGaA, Weinheim)     

Central limit theorem for stochastically continuous processes. Convergence to stable limit

LetX={X(t), t[0, 1]} be a stochastically continuous cadlag process. Assume that thek dimensional finite joint distributions ofX are in the domain of normal attraction of strictlyp-stable, 0<p<2, measure onR ^k for all 1k<. For functionsf, g such that _p (|X(x–X(u)|) >g(u–s) and _p (|X(s–X(t|)|X(t)–X(u|)>f(u–s), 0 s t u 1, conditions are found which imply that the distributions –(n ^–1/p (X ₁+···+X _n )),n1, converge weakly inD[0, 1] to the distribution of ap-stable process. HereX ₁,X ₂, ... are independent copies ofX and _p (Z)=sup _t<0 t ^pP{|Z|<t} denotes the weakpth moment of a random variable Z.     

Consistency of a recursive nearest neighbor regression function estimate

Let (X, Y) be an ^d × -valued random vector and let (X₁, Y₁),…,(X_N, Y_N) be a random sample drawn from its distribution. Divide the data sequence into disjoint blocks of length l₁, …, l_n, find the nearest neighbor to X in each block and call the corresponding couple (X_i^*, Y_i^*). It is shown that the estimate m_n(X) = Σ_{i = 1}ⁿ w_niY_i^*/Σ_{i = 1}ⁿ w_ni of m(X) = E{Y|X} satisfies E{|m_n(X) − m(X)|^p} 0 (p ≥ 1) whenever E{|Y|^p} < ∞, l_n ∞, and the triangular array of positive weights {w_ni} satisfies sup_{i ≤ n}w_ni/Σ_{i = 1}ⁿ w_ni 0. No other restrictions are put on the distribution of (X, Y). Also, some distribution-free results for the strong convergence of E{|m_n(X) − m(X)|^p|X₁, Y₁,…, X_N, Y_N} to zero are included. Finally, an application to the discrimination problem is considered, and a discrimination rule is exhibited and shown to be strongly Bayes risk consistent for all distributions.     

Characterizations of multivariate normality II. Through linear regressions

It is established that a vector (X′₁, X′₂, …, X′_k) has a multivariate normal distribution if (i) for each X_i the regression on the rest is linear, (ii) the conditional distribution of X₁ about the regression does not depend on the rest of the variables, and (iii) the conditional distribution of X₂ about the regression does not depend on the rest of the variables, provided that the regression coefficients satisfy some more conditions that those given by [4]J. Multivar. Anal. 6 81–94].     

On the Conditional Variance for Scale Mixtures of Normal Distributions

For a scale mixture of normal vector, X=A^1/2G, where X, G ⁿ and A is a positive variable, independent of the normal vector G, we obtain that the conditional variance covariance, Cov(X₂ X₁), is always finite a.s. for m2, where X₁ ⁿ and m<n, and remains a.s. finite even for m=1, if and only if the square root moment of the scale factor is finite. It is shown that the variance is not degenerate as in the Gaussian case, but depends upon a function S_A, m(·) for which various properties are derived. Application to a uniform and stable scale of normal distributions are also given.     

Representations for partially exchangeable arrays of random variables

Consider an array of random variables (X_i,j), 1 ≤ i,j < ∞, such that permutations of rows or of columns do not alter the distribution of the array. We show that such an array may be represented as functions f(α, ξ_i, η_j, λ_i,j) of underlying i.i.d, random variables. This result may be useful in characterizing arrays with additional structure. For example, we characterize random matrices whose distribution is invariant under orthogonal rotation, confirming a conjecture of Dawid.     

Admissibility and complete class results for the multinomial estimation problem with entropy and squared error loss

Let X ≡ (X₁, …, X_t) have a multinomial distribution based on N trials with unknown vector of cell probabilities p ≡ (p₁, …, p_t). This paper derives admissibility and complete class results for the problem of simultaneously estimating p under entropy loss (EL) and squared error loss (SEL). Let and f(x¦p) denote the (t − 1)-dimensional simplex, the support of X and the probability mass function of X, respectively. First it is shown that δ is Bayes w.r.t. EL for prior P if and only if δ is Bayes w.r.t. SEL for P. The admissible rules under EL are proved to be Bayes, a result known for the case of SEL. Let Q denote the class of subsets of of the form T = _j=1^kF_j where k ≥ 1 and each F_j is a facet of which satisfies: F a facet of such that F naF_jF ncT. The minimal complete class of rules w.r.t. EL when N ≥ t − 1 is characterized as the class of Bayes rules with respect to priors P which satisfy P( ⁰) = 1, ξ(x) ≡ ∫ f(x¦p) P(dp) > 0 for all x in {x : sup ^₀ f(x¦p) > 0} for some ⁰ in Q containing all the vertices of . As an application, the maximum likelihood estimator is proved to be admissible w.r.t. EL when the estimation problem has parameter space Θ = but it is shown to be inadmissible for the problem with parameter space Θ = ( minus its vertices). This is a severe form of “tyranny of boundary.” Finally it is shown that when N ≥ t − 1 any estimator δ which satisfies δ(x) > 0 x is admissible under EL if and only if it is admissible under SEL. Examples are given of nonpositive estimators which are admissible under SEL but not under EL and vice versa.     

Rates of convergence for classes of functions: The non-i.i.d. case

Let X_i, i ≥ 1, be a sequence of φ-mixing random variables with values in a sample space (X, A). Let L(X_i) = P_(i) for all i ≥ 1 and let _n, n ≥ 1, be classes of real-valued measurable functions on (X, A). Given any function g on (X, A), let S_n(g) = Σ_{i = 1}ⁿ {g(X_i) − Eg(X_i)}. Under weak metric entropy conditions on _n and under growth conditions on both the mixing coefficients and the maximal variance V V(n) max_{i ≤ n} sup_{g n} ∫ g² dP_(i), we show that there is a numerical constant U < ∞ such that

Full-size image

a.s. ^*, where _{i = 1}^xP_(i) and H H(n) is the square root of the entropy of the class _n. Additionally, the rate of convergence H⁻¹(n/V)^1/2 cannot, in general, be improved upon. Applications of this result are considered.     

Selection in Monotone Matrices and Computingkth Nearest Neighbors

Anm×nmatrix =(a_{i, j}), 1≤i≤mand 1≤j≤n, is called atotally monotonematrix if for alli₁, i₂, j₁, j₂, satisfying 1≤i₁<i₂≤m, 1≤j₁<j₂≤n.[formula]We present an[formula]time algorithm to select thekth smallest item from anm×ntotally monotone matrix for anyk≤mn. This is the first subquadratic algorithm for selecting an item from a totally monotone matrix. Our method also yields an algorithm of the same time complexity for ageneralized row-selection problemin monotone matrices. Given a setS={p₁,…, p_n} ofnpoints in convex position and a vectork={k₁,…, k_n}, we also present anO(n^4/3log^c n) algorithm to compute thek_ith nearest neighbor ofp_ifor everyi≤n; herecis an appropriate constant. This algorithm is considerably faster than the one based on a row-selection algorithm for monotone matrices. If the points ofSare arbitrary, then thek_ith nearest neighbor ofp_i, for alli≤n, can be computed in timeO(n^7/5 log^c n), which also improves upon the previously best-known result.     

Estimation of a Mean Vector in a Two-Sample Problem

We consider the problem of estimating a p-dimensional vector μ₁ based on independent variables X₁, X₂, and U, where X₁ is N_p(μ₁, σ²Σ₁), X₂ is N_p(μ₂, σ²Σ₂), and U is σ²χ²_n (Σ₁ and Σ₂ are known). A family of minimax estimators is proposed. Some of these estimators can be obtained via Bayesian arguments as well. Comparisons between our results and the one of Ghosh and Sinha (1988, J. Multivariate Anal.27 206-207) are presented.