Characterizations of multivariate normality. I. Through independence of some statistics

It is established that a vector variable (X₁, …, X_k) has a multivariate normal distribution if for each X_i the regression on the rest is linear and the conditional distribution about the regression does not depend on the rest of the variables, provided the regression coefficients satisfy some mild conditions. The result is extended to the case where X_i themselves are vector variables.     

Some properties of BLUE in a linear model and canonical correlations associated with linear transformations

Let (x, Xβ, V) be a linear model and let A′ = (A′₁, A′₂) be a p × p nonsingular matrix such that A₂X = 0, Rank A₂ = p − Rank X. We represent the BLUE and its covariance matrix in alternative forms under the conditions that the number of unit canonical correlations between y₁ ( = A₁x) and y₂ ( = A₂x) is zero. For the second problem, let x′ = (x′₁, x′₂) and let a g-inverse V⁻ of V be written as (V⁻)′ = (A′₁, A′₂). We investigate the reations (if any) between the nonzero canonical correlations {1 ₁ … ₁ > 0} due to y₁ ( = A₁x) and y₂ ( = A₂x), and the nonzero canonical correlations {1 λ₁ … λ_v+r > 0} due to x₁ and x₂. We answer some of the questions raised by Latour et al. (1987, in Proceedings, 2nd Int. Tampere Conf. Statist. (T. Pukkila and S. Puntanen, Eds.), Univ. of Tampere, Finland) in the case of the Moore-Penrose inverse V⁺ = (A′₁, A′₂) of V.     

A characterization of spherical distributions

It is shown that when the random vector X in Rⁿ has a mean and when the conditional expectation E(u′X|v′X) = 0 for all vectors u, v Rⁿ which satisfy u′v = 0, then the distribution of X is orthogonally invariant. A version of this characterization is also established when X does not have a mean vector.     

Likelihood-Based Local Polynomial Fitting for Single-Index Models

The parametric generalized linear model assumes that the conditional distribution of a response Y given a d-dimensional covariate X belongs to an exponential family and that a known transformation of the regression function is linear in X. In this paper we relax the latter assumption by considering a nonparametric function of the linear combination β^TX, say η₀(β^TX). To estimate the coefficient vector β and the nonparametric component η₀ we consider local polynomial fits based on kernel weighted conditional likelihoods. We then obtain an estimator of the regression function by simply replacing β and η₀ in η₀(β^TX) by these estimators. We derive the asymptotic distributions of these estimators and give the results of some numerical experiments.     

Nonparametric Empirical Bayes Estimation of the Matrix Parameter of the Wishart Distribution

We consider independent pairs (X₁, Σ₁), (X₂, Σ₂), …, (X_n, Σ_n), where eachΣ_iis distributed according to some unknown density functiong(Σ) and, givenΣ_i=Σ,X_ihas conditional density functionq(xΣ) of the Wishart type. In each pair the first component is observable but the second is not. After the (n+1)th observationX_n+1is obtained, the objective is to estimateΣ_n+1corresponding toX_n+1. This estimator is called the empirical Bayes (EB) estimator ofΣ. An EB estimator ofΣis constructed without any parametric assumptions ong(Σ). Its posterior mean square risk is examined, and the estimator is demonstrated to be pointwise asymptotically optimal.     

Stochastic comparisons of order statistics from heterogeneous populations, with applications in reliability

The basic result of the paper states: Let F₁, …, F_n, F₁^′,…, F_n^′ have proportional hazard functions with λ₁ ,…, λ_n , λ₁^′ ,…, λ_n^′ as the constants of proportionality. Let X₍₁₎ ≤ … ≤ X_(n) (X₍₁₎^′ ≤ … ≤ X_(n)^′) be the order statistics in a sample of size n from the heterogeneous populations {F₁ ,…, F_n}({F₁^′ ,…, F_n^′}). Then (λ₁ ,…, λ_n) majorizes (λ₁^′ ,…, λ_n^′) implies that (X₍₁₎ ,…, X_(n)) is stochastically larger than (X₍₁₎^′ ,…, X_(n)^′). Earlier results stochastically comparing individual order statistics are shown to be special cases. Applications of the main result are made in the study of the robustness of standard estimates of the failure rate of the exponential distribution, when observations actually come from a set of heterogeneous exponential distributions. Further applications are made to the comparisons of linear combinations of Weibull random variables and of binomial random variables.     

Asymptotic distributions of regression and autoregression coefficients with martingale difference disturbances

In this paper a form of the Lindeberg condition appropriate for martingale differences is used to obtain asymptotic normality of statistics for regression and autoregression. The regression model is y_t = Bz_t + v_t. The unobserved error sequence {v_t} is a sequence of martingale differences with conditional covariance matrices {Σ_t} and satisfying sup_{t=1,…, n} {v′_tv_tI(v′_tv_t>a) |z_t, v_t−1, z_t−1, …} 0 as a → ∞. The sample covariance of the independent variables z₁, …, z_n, is assumed to have a probability limit M, constant and nonsingular; max_t=1,…,nz′_tz_t/n 0. If (1/n)Σ_t=1ⁿΣ_t Σ, constant, then √nvec( _n−B) N(0,M⁻¹Σ) and _n Σ. The autoregression model is x_t = Bx_{t − 1} + v_t with the maximum absolute value of the characteristic roots of B less than one, the above conditions on {v_t}, and (1/n)Σ_t=max(r,s)+1(Σ_tv_t−1−rv′_t−1−s) δ_rs(ΣΣ), where δ_rs is the Kronecker delta. Then √nvec( _n−B) N(0,Γ⁻¹Σ), where Γ = Σ_{s = 0}^∞B^sΣ(B′)^s.     

Singular random matrix decompositions: Jacobians

For a singular random matrix X, we find the Jacobians associated to the following decompositions: QR, Polar, Singular Value (SVD), L′U, L′DM and modified QR (QDR). Similarly, for the cross-product matrix S=X′X we find the Jacobians of the Spectral, Cholesky's, L′DL and symmetric nonnegative definite square root decompositions.     

The problem of identification of parameters by the distribution of the maximum random variable

Suppose that X₁, X₂,…, X_n are independently distributed according to certain distributions. Does the distribution of the maximum of {X₁, X₂,…, X_n} uniquely determine their distributions? In the univariate case, a general theorem covering the case of Cauchy random variables is given here. Also given is an affirmative answer to the above question for general bivariate normal random variables with non-zero correlations. Bivariate normal random variables with nonnegative correlations were considered earlier in this context by T. W. Anderson and S. G. Ghurye.     

Vertex Set Partitions Preserving Conservativeness

Let G be an undirected graph and ={X₁, …, X_n} be a partition of V(G). Denote by G/ the graph which has vertex set {X₁, …, X_n}, edge set E, and is obtained from G by identifying vertices in each class X_i of the partition . Given a conservative graph (G, w), we study vertex set partitions preserving conservativeness, i.e., those for which (G/ , w) is also a conservative graph. We characterize the conservative graphs (G/ , w), where is a terminal partition of V(G) (a partition preserving conservativeness which is not a refinement of any other partition of this kind). We prove that many conservative graphs admit terminal partitions with some additional properties. The results obtained are then used in new unified short proofs for a co-NP characterization of Seymour graphs by A. A. Ageev, A. V. Kostochka, and Z. Szigeti (1997, J. Graph Theory34, 357–364), a theorem of E. Korach and M. Penn (1992, Math. Programming55, 183–191), a theorem of E. Korach (1994, J. Combin. Theory Ser. B62, 1–10), and a theorem of A. V. Kostochka (1994, in “Discrete Analysis and Operations Research. Mathematics and its Applications (A. D. Korshunov, Ed.), Vol. 355, pp. 109–123, Kluwer Academic, Dordrecht).     

On the uniform distribution of sequences of random variables

Summary This paper deals with the almost sure uniform distribution (modulo 1) of sequences of random variables. In the case where the law of the increments X _n+h –X _n of the sequence X ₀, X ₁, does not depend on n, sufficient conditions are given to assure the uniform distribution (modulo 1) with probability one. As an illustrative example the partial sums of a sequence of independent, identically distributed variables is considered.     

On Thompson's Theorem

Let p be a prime divisor of the order of a finite group G. Thompson (1970, J. Algebra14, 129–134) has proved the following remarkable result: a finite group G is p-nilpotent if the degrees of all its nonlinear irreducible characters are divisible by p (in fact, in that case G is solvable). In this note, we prove that a group G, having only one nonlinear irreducible character of p′-degree is a cyclic extension of Thompson's group. This result is a consequence of the following theorem: A nonabelian simple group possesses two nonlinear irreducible characters χ₁ and χ₂ of distinct degrees such that p does not divide χ₁(1)χ₂(1) (here p is arbitrary but fixed). Our proof depends on the classification of finite simple groups. Some properties of solvable groups possessing exactly two nonlinear irreducible characters of p′-degree are proved. Some open questions are posed.     

Minimax estimation of means of multivariate normal mixtures

Assume X = (X₁, …, X_p)′ is a normal mixture distribution with density w.r.t. Lebesgue measure, , where Σ is a known positive definite matrix and F is any known c.d.f. on (0, ∞). Estimation of the mean vector under an arbitrary known quadratic loss function _Q(θ, a) = (a − θ)′ Q(a − θ), Q a positive definite matrix, is considered. An unbiased estimator of risk is obatined for an arbitrary estimator, and a sufficient condition for estimators to be minimax is then achieved. The result is applied to modifying all the Stein estimators for the means of independent normal random variables to be minimax estimators for the problem considered here. In particular the results apply to the Stein class of limited translation estimators.     

Minimax estimation of the mean of spherically symmetric distributions under general quadratic loss

For X one observation on a p-dimensional (p ≥ 4) spherically symmetric (s.s.) distribution about θ, minimax estimators whose risks dominate the risk of X (the best invariant procedure) are found with respect to general quadratic loss, L(δ, θ) = (δ − θ)′ D(δ − θ) where D is a known p × p positive definite matrix. For C a p × p known positive definite matrix, conditions are given under which estimators of the form δ_a,r,C,D(X) = (I − (ar(|X|²)) D^−1/2CD^1/2 |X|⁻²)X are minimax with smaller risk than X. For the problem of estimating the mean when n observations X₁, X₂, …, X_n are taken on a p-dimensional s.s. distribution about θ, any spherically symmetric translation invariant estimator, δ(X₁, X₂, …, X_n), with have a s.s. distribution about θ. Among the estimators which have these properties are best invariant estimators, sample means and maximum likelihood estimators. Moreover, under certain conditions, improved robust estimators can be found.     

On improved estimators of the generalized variance

Treated in this paper is the problem of estimating with squared error loss the generalized variance | Σ | from a Wishart random matrix S: p × p W_p(n, Σ) and an independent normal random matrix X: p × k N(ξ, Σ I_k) with ξ(p × k) unknown. Denote the columns of X by X₍₁₎ ,…, X_(k) and set ψ⁽⁰⁾(S, X) = {(n − p + 2)!/(n + 2)!} | S |, ψ⁽ⁱ⁾(X, X) = min[ψ⁽ⁱ⁻¹⁾(S, X), {(n − p + i + 2)!/(n + i + 2)!} | S + X₍₁₎ X′₍₁₎ + + X_(i) X′_(i) |] and Ψ⁽ⁱ⁾(S, X) = min[ψ⁽⁰⁾(S, X), {(n − p + i + 2)!/(n + i + 2)!}| S + X₍₁₎ X′₍₁₎ + + X_(i) X′_(i) |], i = 1,…,k. Our result is that the minimax, best affine equivariant estimator ψ⁽⁰⁾(S, X) is dominated by each of Ψ⁽ⁱ⁾(S, X), i = 1,…,k and for every i, ψ⁽ⁱ⁾(S, X) is better than ψ⁽ⁱ⁻¹⁾(S, X). In particular, ψ^(k)(S, X) = min[{(n − p + 2)!/(n + 2)!} | S |, {(n − p + 2)!/(n + 2)!} | S + X₍₁₎X′₍₁₎|,…,| {(n − p + k + 2)!/(n + k + 2)!} | S + X₍₁₎X′₍₁₎ + + X_(k)X′_(k)|] dominates all other ψ's. It is obtained by considering a multivariate extension of Stein's result (Ann. Inst. Statist. Math. 16, 155–160 (1964)) on the estimation of the normal variance.     

More on Stochastic Comparisons and Dependence among Concomitants of Order Statistics

For a sample of iid observations {(X_i, Y_i)} from an absolutely continuous distribution, the multivariate dependence of concomitants Y_[]=(Y_[1], Y_[2], …, Y_[n]) and the stochastic order of subsets of Y_[] are studied. If (X, Y) is totally positive dependent of order 2, Y_[] is multivariate totally positive dependent of order 2. If the conditional hazard rate function of Y given X, h_YX(yx), is decreasing in x for every y, Y_[] is multivariate right corner set increasing. And if Y is stochastically increasing in X, the concomitants are increasing in multivariate stochastic order.     

On confidence sequences for the mean vector of a multivariate normal distribution

Let X₁, X₂,… be idd random vectors with a multivariate normal distribution N(μ, Σ). A sequence of subsets {R_n(a₁, a₂,…, a_n), n ≥ m} of the space of μ is said to be a (1 − α)-level sequence of confidence sets for μ if P(μ R_n(X₁, X₂,…, X_n) for every n ≥ m) ≥ 1 − α. In this note we use the ideas of Robbins Ann. Math. Statist. 41 (1970) to construct confidence sequences for the mean vector μ when Σ is either known or unknown. The constructed sequence R_n(X₁, X₂, …, X_n) depends on Mahalanobis' or Hotelling's according as Σ is known or unknown. Confidence sequences for the vector-valued parameter in the general linear model are also given.     

Some extensions of the Kantorovich inequality and statistical applications

Kantorovich gave an upper bound to the product of two quadratic forms, (X′AX) (X′A⁻¹X), where X is an n-vector of unit length and A is a positive definite matrix. Bloomfield, Watson and Knott found the bound for the product of determinants |X′AX| |X′A⁻¹X| where X is n × k matrix such that X′X = I_k. In this paper we determine the bounds for the traces and determinants of matrices of the type X′AYY′A⁻¹X, X′B²X(X′BCX)⁻¹ X′C²X(X′BCX)⁻¹ where X and Y are n × k matrices such that X′X = Y′Y = I_k and A, B, C are given matrices satisfying some conditions. The results are applied to the least squares theory of estimation.     

Updating of moments

Consider a statistical model, given by the distribution of the observation X, conditional on the parameter θ, and the prior distribution of the parameter θ. Let H_x denote the function that maps the prior mean and the prior covariance matrix into the posterior mean and the posterior covariance matrix, when X = x is observed. We prove that if the conditional distribution of X belongs to an exponential family, then the function H_x characterizes the distribution of Xθ.     

On the approximate behavior of the posterior distribution for an extreme multivariate observation

The behavior of the posterior for a large observation is considered. Two basic situations are discussed; location vectors and natural parameters.Let X = (X₁, X₂, …, X_n) be an observation from a multivariate exponential distribution with that natural parameter Θ = (Θ₁, Θ₂, …, Θ_n). Let θ_x^* be the posterior mode. Sufficient conditions are presented for the distribution of Θ − θ_x^* given X = x to converge to a multivariate normal with mean vector 0 as |x| tends to infinity. These same conditions imply that E(Θ | X = x) − θ_x^* converges to the zero vector as |x| tends to infinity.The posterior for an observation X = (X₁, X₂, …, X_n is considered for a location vector Θ = (Θ₁, Θ₂, …, Θ_n) as x gets large along a path, γ, in Rⁿ. Sufficient conditions are given for the distribution of γ(t) − Θ given X = γ(t) to converge in law as t → ∞. Slightly stronger conditions ensure that γ(t) − E(Θ | X = γ(t)) converges to the mean of the limiting distribution.These basic results about the posterior mean are extended to cover other estimators. Loss functions which are convex functions of absolute error are considered. Let δ be a Bayes estimator for a loss function of this type. Generally, if the distribution of Θ − E(Θ | X = γ(t)) given X = γ(t) converges in law to a symmetric distribution as t → ∞, it is shown that δ(γ(t)) − E(Θ | X = γ(t)) → 0 as t → ∞.