On the approximate behavior of the posterior distribution for an extreme multivariate observation

The behavior of the posterior for a large observation is considered. Two basic situations are discussed; location vectors and natural parameters.Let X = (X₁, X₂, …, X_n) be an observation from a multivariate exponential distribution with that natural parameter Θ = (Θ₁, Θ₂, …, Θ_n). Let θ_x^* be the posterior mode. Sufficient conditions are presented for the distribution of Θ − θ_x^* given X = x to converge to a multivariate normal with mean vector 0 as |x| tends to infinity. These same conditions imply that E(Θ | X = x) − θ_x^* converges to the zero vector as |x| tends to infinity.The posterior for an observation X = (X₁, X₂, …, X_n is considered for a location vector Θ = (Θ₁, Θ₂, …, Θ_n) as x gets large along a path, γ, in Rⁿ. Sufficient conditions are given for the distribution of γ(t) − Θ given X = γ(t) to converge in law as t → ∞. Slightly stronger conditions ensure that γ(t) − E(Θ | X = γ(t)) converges to the mean of the limiting distribution.These basic results about the posterior mean are extended to cover other estimators. Loss functions which are convex functions of absolute error are considered. Let δ be a Bayes estimator for a loss function of this type. Generally, if the distribution of Θ − E(Θ | X = γ(t)) given X = γ(t) converges in law to a symmetric distribution as t → ∞, it is shown that δ(γ(t)) − E(Θ | X = γ(t)) → 0 as t → ∞.     

L1-optimal estimates for a regression type function in R

Let X₁, X₂, …, X_n be random vectors that take values in a compact set in R^d, d ≥ 1. Let Y₁, Y₂, …, Y_n be random variables (“the responses”) which conditionally on X₁ = x₁, …, X_n = x_n are independent with densities f(y | x_i, θ(x_i)), i = 1, …, n. Assuming that θ lives in a sup-norm compact space Θ_q,d of real valued functions, an optimal L₁-consistent estimator of θ is constructed via empirical measures. The rate of convergence of the estimator to the true parameter θ depends on Kolmogorov's entropy of Θ_q,d.     

Asymptotic expansions of the distributions of some test statistics for Gaussian ARMA processes

Let {X_t} be a Gaussian ARMA process with spectral density f_θ(λ), where θ is an unknown parameter. The problem considered is that of testing a simple hypothesis H:θ = θ₀ against the alternative A:θ ≠ θ₀. For this problem we propose a class of tests , which contains the likelihood ratio (LR), Wald (W), modified Wald (MW) and Rao (R) tests as special cases. Then we derive the χ² type asymptotic expansion of the distribution of T up to order n⁻¹, where n is the sample size. Also we derive the χ² type asymptotic expansion of the distribution of T under the sequence of alternatives A_n: θ = θ₀ + /√n, ε > 0. Then we compare the local powers of the LR, W, MW, and R tests on the basis of their asymptotic expansions.     

On confidence sequences for the mean vector of a multivariate normal distribution

Let X₁, X₂,… be idd random vectors with a multivariate normal distribution N(μ, Σ). A sequence of subsets {R_n(a₁, a₂,…, a_n), n ≥ m} of the space of μ is said to be a (1 − α)-level sequence of confidence sets for μ if P(μ R_n(X₁, X₂,…, X_n) for every n ≥ m) ≥ 1 − α. In this note we use the ideas of Robbins Ann. Math. Statist. 41 (1970) to construct confidence sequences for the mean vector μ when Σ is either known or unknown. The constructed sequence R_n(X₁, X₂, …, X_n) depends on Mahalanobis' or Hotelling's according as Σ is known or unknown. Confidence sequences for the vector-valued parameter in the general linear model are also given.     

Stochastic comparisons of order statistics from heterogeneous populations, with applications in reliability

The basic result of the paper states: Let F₁, …, F_n, F₁^′,…, F_n^′ have proportional hazard functions with λ₁ ,…, λ_n , λ₁^′ ,…, λ_n^′ as the constants of proportionality. Let X₍₁₎ ≤ … ≤ X_(n) (X₍₁₎^′ ≤ … ≤ X_(n)^′) be the order statistics in a sample of size n from the heterogeneous populations {F₁ ,…, F_n}({F₁^′ ,…, F_n^′}). Then (λ₁ ,…, λ_n) majorizes (λ₁^′ ,…, λ_n^′) implies that (X₍₁₎ ,…, X_(n)) is stochastically larger than (X₍₁₎^′ ,…, X_(n)^′). Earlier results stochastically comparing individual order statistics are shown to be special cases. Applications of the main result are made in the study of the robustness of standard estimates of the failure rate of the exponential distribution, when observations actually come from a set of heterogeneous exponential distributions. Further applications are made to the comparisons of linear combinations of Weibull random variables and of binomial random variables.     

On improved estimators of the generalized variance

Treated in this paper is the problem of estimating with squared error loss the generalized variance | Σ | from a Wishart random matrix S: p × p W_p(n, Σ) and an independent normal random matrix X: p × k N(ξ, Σ I_k) with ξ(p × k) unknown. Denote the columns of X by X₍₁₎ ,…, X_(k) and set ψ⁽⁰⁾(S, X) = {(n − p + 2)!/(n + 2)!} | S |, ψ⁽ⁱ⁾(X, X) = min[ψ⁽ⁱ⁻¹⁾(S, X), {(n − p + i + 2)!/(n + i + 2)!} | S + X₍₁₎ X′₍₁₎ + + X_(i) X′_(i) |] and Ψ⁽ⁱ⁾(S, X) = min[ψ⁽⁰⁾(S, X), {(n − p + i + 2)!/(n + i + 2)!}| S + X₍₁₎ X′₍₁₎ + + X_(i) X′_(i) |], i = 1,…,k. Our result is that the minimax, best affine equivariant estimator ψ⁽⁰⁾(S, X) is dominated by each of Ψ⁽ⁱ⁾(S, X), i = 1,…,k and for every i, ψ⁽ⁱ⁾(S, X) is better than ψ⁽ⁱ⁻¹⁾(S, X). In particular, ψ^(k)(S, X) = min[{(n − p + 2)!/(n + 2)!} | S |, {(n − p + 2)!/(n + 2)!} | S + X₍₁₎X′₍₁₎|,…,| {(n − p + k + 2)!/(n + k + 2)!} | S + X₍₁₎X′₍₁₎ + + X_(k)X′_(k)|] dominates all other ψ's. It is obtained by considering a multivariate extension of Stein's result (Ann. Inst. Statist. Math. 16, 155–160 (1964)) on the estimation of the normal variance.     

Minimax estimation of the mean of spherically symmetric distributions under general quadratic loss

For X one observation on a p-dimensional (p ≥ 4) spherically symmetric (s.s.) distribution about θ, minimax estimators whose risks dominate the risk of X (the best invariant procedure) are found with respect to general quadratic loss, L(δ, θ) = (δ − θ)′ D(δ − θ) where D is a known p × p positive definite matrix. For C a p × p known positive definite matrix, conditions are given under which estimators of the form δ_a,r,C,D(X) = (I − (ar(|X|²)) D^−1/2CD^1/2 |X|⁻²)X are minimax with smaller risk than X. For the problem of estimating the mean when n observations X₁, X₂, …, X_n are taken on a p-dimensional s.s. distribution about θ, any spherically symmetric translation invariant estimator, δ(X₁, X₂, …, X_n), with have a s.s. distribution about θ. Among the estimators which have these properties are best invariant estimators, sample means and maximum likelihood estimators. Moreover, under certain conditions, improved robust estimators can be found.     

MU-Estimation and Smoothing

In the M-estimation theory developed by Huber (1964, Ann. Math. Statist.43, 1449–1458), the parameter under estimation is the value of θ which minimizes the expectation of what is called a discrepancy measure (DM) δ(X, θ) which is a function of θ and the underlying random variable X. Such a setting does not cover the estimation of parameters such as the multivariate median defined by Oja (1983) and Liu (1990), as the value of θ which minimizes the expectation of a DM of the type δ(X₁, …, X_m, θ) where X₁, …, X_m are independent copies of the underlying random variable X. Arcones et al. (1994, Ann. Statist.22, 1460–1477) studied the estimation of such parameters. We call such an M-type MU-estimation (or μ-estimation for convenience). When a DM is not a differentiable function of θ, some complexities arise in studying the properties of estimators as well as in their computation. In such a case, we introduce a new method of smoothing the DM with a kernel function and using it in estimation. It is seen that smoothing allows us to develop an elegant approach to the study of asymptotic properties and possibly apply the Newton–Raphson procedure in the computation of estimators.     

An identity on the maximum of a set of random variables

It is shown that if X₁, X₂, …, X_n are symmetric random variables and max(X₁, …, X_n)⁺ = max(0, X₁, …, X_n), then E[max(X₁,…,X_n)⁺]=[max(X₁,X₁,+X₂,+X₁,+X₃,…X₁,+X_n)⁺], and in the case of independent identically distributed symmetric random variables, E[max(X₁, X₂)⁺] = E[(X₁)⁺] + (1/2)E[(X₁ + X₂)⁺], so that for independent standard normal random variables, E[max(X₁, X₂)⁺] = (1/√2π)[1 + (1/√2)].     

Robust M-estimators of location vectors

Let X₁,…,X_n be i.i.d. random vectors in R^m, let θεR^m be an unknown location parameter, and assume that the restriction of the distribution of X₁−θ to a sphere of radius d belongs to a specified neighborhood of distributions spherically symmetric about 0. Under regularity conditions on and d, the parameter θ in this model is identifiable, and consistent M-estimators of θ (i.e., solutions of Σ_i=1ⁿψ(|X_i− |)(X_i− )=0) are obtained by using “re-descenders,” i.e., ψ's wh satisfy ψ(x)=0 for x≥c. An iterative method for solving for is shown to produce consistent and asymptotically normal estimates of θ under all distributions in . The following asymptotic robustness problem is considered: finding the ψ which is best among the re-descenders according to Huber's minimax variance criterion.     

Asymptotic properties of Cramér-Smirnov statistics—A new approach

Let Y₁,…, Y_n be independent identically distributed random variables with distribution function F(x, θ), θ = (θ′₁, θ′₂), where θ_i (i = 1, 2) is a vector of p_i components, p = p₁ + p₂ and for θI, an open interval in ^p, F(x, θ) is continuous. In the present paper the author shows that the asymptotic distribution of modified Cramér-Smirnov statistic under H_n: θ₁ = θ₁₀ + n^−1/2γ, θ₂ unspecified, where γ is a given vector independent of n, is the distribution of a sum of weighted noncentral χ₁² variables whose weights are eigenvalues of a covariance function of a Gaussian process and noncentrality parameters are Fourier coefficients of the mean function of the Gaussian process. Further, the author exploits the special form of the covariance function by using perturbation theory to obtain the noncentrality parameters and the weights. The technique is applicable to other goodness-of-fit statistics such as U² [G. S. Watson, Biometrika 48 (1961), 109–114].     

On the Maximal Multiplicity of Parts in a Random Integer Partition

We study the asymptotic behavior of the maximal multiplicity μ_n = μ_n(λ) of the parts in a partition λ of the positive integer n, assuming that λ is chosen uniformly at random from the set of all such partitions. We prove that πμ_n/(6n)^1/2 converges weakly to max _jX_j/j as n→∞, where X₁, X₂, … are independent and exponentially distributed random variables with common mean equal to 1.2000 Mathematics Subject Classification: Primary—05A17; Secondary—11P82, 60C05, 60F05     

Admissibility and complete class results for the multinomial estimation problem with entropy and squared error loss

Let X ≡ (X₁, …, X_t) have a multinomial distribution based on N trials with unknown vector of cell probabilities p ≡ (p₁, …, p_t). This paper derives admissibility and complete class results for the problem of simultaneously estimating p under entropy loss (EL) and squared error loss (SEL). Let and f(x¦p) denote the (t − 1)-dimensional simplex, the support of X and the probability mass function of X, respectively. First it is shown that δ is Bayes w.r.t. EL for prior P if and only if δ is Bayes w.r.t. SEL for P. The admissible rules under EL are proved to be Bayes, a result known for the case of SEL. Let Q denote the class of subsets of of the form T = _j=1^kF_j where k ≥ 1 and each F_j is a facet of which satisfies: F a facet of such that F naF_jF ncT. The minimal complete class of rules w.r.t. EL when N ≥ t − 1 is characterized as the class of Bayes rules with respect to priors P which satisfy P( ⁰) = 1, ξ(x) ≡ ∫ f(x¦p) P(dp) > 0 for all x in {x : sup ^₀ f(x¦p) > 0} for some ⁰ in Q containing all the vertices of . As an application, the maximum likelihood estimator is proved to be admissible w.r.t. EL when the estimation problem has parameter space Θ = but it is shown to be inadmissible for the problem with parameter space Θ = ( minus its vertices). This is a severe form of “tyranny of boundary.” Finally it is shown that when N ≥ t − 1 any estimator δ which satisfies δ(x) > 0 x is admissible under EL if and only if it is admissible under SEL. Examples are given of nonpositive estimators which are admissible under SEL but not under EL and vice versa.     

A uniform bound for the deviation of empirical distribution functions

If X₁, …, X_n are independent R^d-valued random vectors with common distribution function F, and if F_n is the empirical distribution function for X₁, …, X_n, then, among other things, it is shown that P{sup_x F_n(x) ε} 2e²(2n)^de^−2nε2 for all nε² ≥ d². The inequality remains valid if the X_i are not identically distributed and F(x) is replaced by Σ_iP{X_i ≤ x}/n.     

Asymptotic distributions of regression and autoregression coefficients with martingale difference disturbances

In this paper a form of the Lindeberg condition appropriate for martingale differences is used to obtain asymptotic normality of statistics for regression and autoregression. The regression model is y_t = Bz_t + v_t. The unobserved error sequence {v_t} is a sequence of martingale differences with conditional covariance matrices {Σ_t} and satisfying sup_{t=1,…, n} {v′_tv_tI(v′_tv_t>a) |z_t, v_t−1, z_t−1, …} 0 as a → ∞. The sample covariance of the independent variables z₁, …, z_n, is assumed to have a probability limit M, constant and nonsingular; max_t=1,…,nz′_tz_t/n 0. If (1/n)Σ_t=1ⁿΣ_t Σ, constant, then √nvec( _n−B) N(0,M⁻¹Σ) and _n Σ. The autoregression model is x_t = Bx_{t − 1} + v_t with the maximum absolute value of the characteristic roots of B less than one, the above conditions on {v_t}, and (1/n)Σ_t=max(r,s)+1(Σ_tv_t−1−rv′_t−1−s) δ_rs(ΣΣ), where δ_rs is the Kronecker delta. Then √nvec( _n−B) N(0,Γ⁻¹Σ), where Γ = Σ_{s = 0}^∞B^sΣ(B′)^s.     

Consistency of a recursive nearest neighbor regression function estimate

Let (X, Y) be an ^d × -valued random vector and let (X₁, Y₁),…,(X_N, Y_N) be a random sample drawn from its distribution. Divide the data sequence into disjoint blocks of length l₁, …, l_n, find the nearest neighbor to X in each block and call the corresponding couple (X_i^*, Y_i^*). It is shown that the estimate m_n(X) = Σ_{i = 1}ⁿ w_niY_i^*/Σ_{i = 1}ⁿ w_ni of m(X) = E{Y|X} satisfies E{|m_n(X) − m(X)|^p} 0 (p ≥ 1) whenever E{|Y|^p} < ∞, l_n ∞, and the triangular array of positive weights {w_ni} satisfies sup_{i ≤ n}w_ni/Σ_{i = 1}ⁿ w_ni 0. No other restrictions are put on the distribution of (X, Y). Also, some distribution-free results for the strong convergence of E{|m_n(X) − m(X)|^p|X₁, Y₁,…, X_N, Y_N} to zero are included. Finally, an application to the discrimination problem is considered, and a discrimination rule is exhibited and shown to be strongly Bayes risk consistent for all distributions.     

Checking the adequacy of the multivariate semiparametric location shift model

Let X,X₁,…,X_m,…, Y,Y₁,…,Y_n,… be independent d-dimensional random vectors, where the X_j are i.i.d. copies of X, and the Y_k are i.i.d. copies of Y. We study a class of consistent tests for the hypothesis that Y has the same distribution as X+μ for some unspecified . The test statistic L is a weighted integral of the squared modulus of the difference of the empirical characteristic functions of and Y₁,…,Y_n, where is an estimator of μ. An alternative representation of L is given in terms of an L²-distance between two nonparametric density estimators. The finite-sample and asymptotic null distribution of L is independent of μ. Carried out as a bootstrap or permutation procedure, the test is asymptotically of a given size, irrespective of the unknown underlying distribution. A large-scale simulation study shows that the permutation procedure performs better than the bootstrap.     

Asymptotically optimal selection of a piecewise polynomial estimator of a regression function

Let (X, Y) be a pair of random variables such that X = (X₁,…, X_d) ranges over a nondegenerate compact d-dimensional interval C and Y is real-valued. Let the conditional distribution of Y given X have mean θ(X) and satisfy an appropriate moment condition. It is assumed that the distribution of X is absolutely continuous and its density is bounded away from zero and infinity on C. Without loss of generality let C be the unit cube. Consider an estimator of θ having the form of a piecewise polynomial of degree k_n based on m_n^d cubes of length 1/m_n, where the m_n^d(_d^k_n+d) coefficients are chosen by the method of least squares based on a random sample of size n from the distribution of (X, Y). Let (k_n, m_n) be chosen by the FPE procedure. It is shown that the indicated estimator has an asymptotically minimal squared error of prediction if θ is not of the form of piecewise polynomial.     

A logarithmic Gauss curvature flow and the Minkowski problem

Let X₀ be a smooth uniformly convex hypersurface and f a postive smooth function in Sⁿ. We study the motion of convex hypersurfaces X(·,t) with initial X(·,0)=θX₀ along its inner normal at a rate equal to log(K/f) where K is the Gauss curvature of X(·,t). We show that the hypersurfaces remain smooth and uniformly convex, and there exists θ^*>0 such that if θ<θ^*, they shrink to a point in finite time and, if θ>θ^*, they expand to an asymptotic sphere. Finally, when θ=θ^*, they converge to a convex hypersurface of which Gauss curvature is given explicitly by a function depending on f(x).     

The problem of identification of parameters by the distribution of the maximum random variable

Suppose that X₁, X₂,…, X_n are independently distributed according to certain distributions. Does the distribution of the maximum of {X₁, X₂,…, X_n} uniquely determine their distributions? In the univariate case, a general theorem covering the case of Cauchy random variables is given here. Also given is an affirmative answer to the above question for general bivariate normal random variables with non-zero correlations. Bivariate normal random variables with nonnegative correlations were considered earlier in this context by T. W. Anderson and S. G. Ghurye.