On the approximate behavior of the posterior distribution for an extreme multivariate observation

The behavior of the posterior for a large observation is considered. Two basic situations are discussed; location vectors and natural parameters.Let X = (X₁, X₂, …, X_n) be an observation from a multivariate exponential distribution with that natural parameter Θ = (Θ₁, Θ₂, …, Θ_n). Let θ_x^* be the posterior mode. Sufficient conditions are presented for the distribution of Θ − θ_x^* given X = x to converge to a multivariate normal with mean vector 0 as |x| tends to infinity. These same conditions imply that E(Θ | X = x) − θ_x^* converges to the zero vector as |x| tends to infinity.The posterior for an observation X = (X₁, X₂, …, X_n is considered for a location vector Θ = (Θ₁, Θ₂, …, Θ_n) as x gets large along a path, γ, in Rⁿ. Sufficient conditions are given for the distribution of γ(t) − Θ given X = γ(t) to converge in law as t → ∞. Slightly stronger conditions ensure that γ(t) − E(Θ | X = γ(t)) converges to the mean of the limiting distribution.These basic results about the posterior mean are extended to cover other estimators. Loss functions which are convex functions of absolute error are considered. Let δ be a Bayes estimator for a loss function of this type. Generally, if the distribution of Θ − E(Θ | X = γ(t)) given X = γ(t) converges in law to a symmetric distribution as t → ∞, it is shown that δ(γ(t)) − E(Θ | X = γ(t)) → 0 as t → ∞.     

Asymptotic properties of Cramér-Smirnov statistics—A new approach

Let Y₁,…, Y_n be independent identically distributed random variables with distribution function F(x, θ), θ = (θ′₁, θ′₂), where θ_i (i = 1, 2) is a vector of p_i components, p = p₁ + p₂ and for θI, an open interval in ^p, F(x, θ) is continuous. In the present paper the author shows that the asymptotic distribution of modified Cramér-Smirnov statistic under H_n: θ₁ = θ₁₀ + n^−1/2γ, θ₂ unspecified, where γ is a given vector independent of n, is the distribution of a sum of weighted noncentral χ₁² variables whose weights are eigenvalues of a covariance function of a Gaussian process and noncentrality parameters are Fourier coefficients of the mean function of the Gaussian process. Further, the author exploits the special form of the covariance function by using perturbation theory to obtain the noncentrality parameters and the weights. The technique is applicable to other goodness-of-fit statistics such as U² [G. S. Watson, Biometrika 48 (1961), 109–114].     

Inequalities for predictive ratios and posterior variances in natural exponential families

The predictive ratio is considered as a measure of spread for the predictive distribution. It is shown that, in the exponential families, ordering according to the predictive ratio is equivalent to ordering according to the posterior covariance matrix of the parameters. This result generalizes an inequality due to Chaloner and Duncan who consider the predictive ratio for a beta-binomial distribution and compare it with a predictive ratio for the binomial distribution with a degenerate prior. The predictive ratio at x₁ and x₂ is defined to be p_g(x₁)p_g(x₂)/[p_g( )]² = h_g(x₁, x₂), where p_g(x₁) = ∫ ƒ(x₁θ) g(θ) dθ is the predictive distribution of x₁ with respect to the prior g. We prove that h_g(x₁, x₂) ≥ h_g^*(x₁, x₂) for all x₁ and x₂ if ƒ(xθ) is in the natural exponential family and Cov_gx(θ) ≥ Cov_g^*x(θ) in the Loewner sense, for all x on a straight line from x₁ to x₂. We then restrict the class of prior distributions to the conjugate class and ask whether the posterior covariance inequality obtains if g and g^* differ in that the “sample size”     

Robust M-estimators of location vectors

Let X₁,…,X_n be i.i.d. random vectors in R^m, let θεR^m be an unknown location parameter, and assume that the restriction of the distribution of X₁−θ to a sphere of radius d belongs to a specified neighborhood of distributions spherically symmetric about 0. Under regularity conditions on and d, the parameter θ in this model is identifiable, and consistent M-estimators of θ (i.e., solutions of Σ_i=1ⁿψ(|X_i− |)(X_i− )=0) are obtained by using “re-descenders,” i.e., ψ's wh satisfy ψ(x)=0 for x≥c. An iterative method for solving for is shown to produce consistent and asymptotically normal estimates of θ under all distributions in . The following asymptotic robustness problem is considered: finding the ψ which is best among the re-descenders according to Huber's minimax variance criterion.     

Minimax estimation of the mean of spherically symmetric distributions under general quadratic loss

For X one observation on a p-dimensional (p ≥ 4) spherically symmetric (s.s.) distribution about θ, minimax estimators whose risks dominate the risk of X (the best invariant procedure) are found with respect to general quadratic loss, L(δ, θ) = (δ − θ)′ D(δ − θ) where D is a known p × p positive definite matrix. For C a p × p known positive definite matrix, conditions are given under which estimators of the form δ_a,r,C,D(X) = (I − (ar(|X|²)) D^−1/2CD^1/2 |X|⁻²)X are minimax with smaller risk than X. For the problem of estimating the mean when n observations X₁, X₂, …, X_n are taken on a p-dimensional s.s. distribution about θ, any spherically symmetric translation invariant estimator, δ(X₁, X₂, …, X_n), with have a s.s. distribution about θ. Among the estimators which have these properties are best invariant estimators, sample means and maximum likelihood estimators. Moreover, under certain conditions, improved robust estimators can be found.     

L1-optimal estimates for a regression type function in R

Let X₁, X₂, …, X_n be random vectors that take values in a compact set in R^d, d ≥ 1. Let Y₁, Y₂, …, Y_n be random variables (“the responses”) which conditionally on X₁ = x₁, …, X_n = x_n are independent with densities f(y | x_i, θ(x_i)), i = 1, …, n. Assuming that θ lives in a sup-norm compact space Θ_q,d of real valued functions, an optimal L₁-consistent estimator of θ is constructed via empirical measures. The rate of convergence of the estimator to the true parameter θ depends on Kolmogorov's entropy of Θ_q,d.     

Minimax estimation of means of multivariate normal mixtures

Assume X = (X₁, …, X_p)′ is a normal mixture distribution with density w.r.t. Lebesgue measure, , where Σ is a known positive definite matrix and F is any known c.d.f. on (0, ∞). Estimation of the mean vector under an arbitrary known quadratic loss function _Q(θ, a) = (a − θ)′ Q(a − θ), Q a positive definite matrix, is considered. An unbiased estimator of risk is obatined for an arbitrary estimator, and a sufficient condition for estimators to be minimax is then achieved. The result is applied to modifying all the Stein estimators for the means of independent normal random variables to be minimax estimators for the problem considered here. In particular the results apply to the Stein class of limited translation estimators.     

Local polynomial fitting under association

We consider the estimation of multivariate regression functions r(x₁,…,x_d) and their partial derivatives up to a total order p1 using high-order local polynomial fitting. The processes {Y_i,X_i} are assumed to be (jointly) associated. Joint asymptotic normality is established for the estimates of the regression function r and all its partial derivatives up to the total order p. Expressions for the bias and variance/covariance matrix (of the asymptotic distribution) are given.     

A logarithmic Gauss curvature flow and the Minkowski problem

Let X₀ be a smooth uniformly convex hypersurface and f a postive smooth function in Sⁿ. We study the motion of convex hypersurfaces X(·,t) with initial X(·,0)=θX₀ along its inner normal at a rate equal to log(K/f) where K is the Gauss curvature of X(·,t). We show that the hypersurfaces remain smooth and uniformly convex, and there exists θ^*>0 such that if θ<θ^*, they shrink to a point in finite time and, if θ>θ^*, they expand to an asymptotic sphere. Finally, when θ=θ^*, they converge to a convex hypersurface of which Gauss curvature is given explicitly by a function depending on f(x).     

Generalized Hodges-Lehmann estimators for the analysis of variance

For X_i, …, X_n a random sample and K(·, ·) a symmetric kernel this paper considers large sample properties of location estimator satisfying , . Asymptotic normality of is obtained and two forms of interval estimators for parameter θ satisfying EK(X₁ − θ, X₂ − θ) = 0, are discussed. Consistent estimation of the variance parameters is obtained which permits the construction of asymptotically distribution free procedures. The p-variate and multigroup extension is accomplished to provide generalized one-way MANOVA. Monte Carlo results are included.     

A uniform bound for the deviation of empirical distribution functions

If X₁, …, X_n are independent R^d-valued random vectors with common distribution function F, and if F_n is the empirical distribution function for X₁, …, X_n, then, among other things, it is shown that P{sup_x F_n(x) ε} 2e²(2n)^de^−2nε2 for all nε² ≥ d². The inequality remains valid if the X_i are not identically distributed and F(x) is replaced by Σ_iP{X_i ≤ x}/n.     

Third-order asymptomic properties of a class of test statistics under a local alternative

Suppose that {X_i; I = 1, 2, …,} is a sequence of p-dimensional random vectors forming a stochastic process. Let p_n, θ(X_n), X_n ^np, be the probability density function of X_n = (X₁, …, X_n) depending on θ Θ, where Θ is an open set of ¹. We consider to test a simple hypothesis H : θ = θ₀ against the alternative A : θ ≠ θ₀. For this testing problem we introduce a class of tests , which contains the likelihood ratio, Wald, modified Wald, and Rao tests as special cases. Then we derive the third-order asymptotic expansion of the distribution of T under a sequence of local alternatives. Using this result we elucidate various third-order asymptotic properties of T (e.g., Bartlett's adjustments, third-order asymptotically most powerful properties). Our results are very general, and can be applied to the i.i.d. case, multivariate analysis, and time series analysis. Two concrete examples will be given. One is a Gaussian ARMA process (dependent case), and the other is a nonlinear regression model (non-identically distributed case).     

Limit theorems for sums of random exponentials

We study limiting distributions of exponential sums as t→∞, N→∞, where (X_i) are i.i.d. random variables. Two cases are considered: (A) ess sup X_i = 0 and (B) ess sup X_i = ∞. We assume that the function h(x)= -log P{X_i>x} (case B) or h(x) = -log P {X_i>-1/x} (case A) is regularly varying at ∞ with index 1 < ϱ <∞ (case B) or 0 < ϱ < ∞ (case A). The appropriate growth scale of N relative to t is of the form , where the rate function H₀(t) is a certain asymptotic version of the function (case B) or (case A). We have found two critical points, λ₁<λ₂, below which the Law of Large Numbers and the Central Limit Theorem, respectively, break down. For 0 < λ < λ₂, under the slightly stronger condition of normalized regular variation of h we prove that the limit laws are stable, with characteristic exponent α = α (ϱ, λ) ∈ (0,2) and skewness parameter β ≡ 1.Research supported in part by the DFG grants 436 RUS 113/534 and 436 RUS 113/722.Mathematics Subject Classification (2000): 60G50, 60F05, 60E07     

Likelihood-Based Local Polynomial Fitting for Single-Index Models

The parametric generalized linear model assumes that the conditional distribution of a response Y given a d-dimensional covariate X belongs to an exponential family and that a known transformation of the regression function is linear in X. In this paper we relax the latter assumption by considering a nonparametric function of the linear combination β^TX, say η₀(β^TX). To estimate the coefficient vector β and the nonparametric component η₀ we consider local polynomial fits based on kernel weighted conditional likelihoods. We then obtain an estimator of the regression function by simply replacing β and η₀ in η₀(β^TX) by these estimators. We derive the asymptotic distributions of these estimators and give the results of some numerical experiments.     

Global nonparametric estimation of conditional quantile functions and their derivatives

Let (X, Y) be a random vector such that X is d-dimensional, Y is real valued, and θ(X) is the conditional αth quantile of Y given X, where α is a fixed number such that 0 < α < 1. Assume that θ is a smooth function with order of smoothness p > 0, and set r = (p − m)/(2p + d), where m is a nonnegative integer smaller than p. Let T(θ) denote a derivative of θ of order m. It is proved that there exists estimate of T(θ), based on a set of i.i.d. observations (X₁, Y₁), …, (X_n, Y_n), that achieves the optimal nonparametric rate of convergence n^−r in L_q-norms (1 ≤ q < ∞) restricted to compacts under appropriate regularity conditions. Further, it has been shown that there exists estimate of T(θ) that achieves the optimal rate (n/log n)^−r in L_∞-norm restricted to compacts.     

Common Principal Components for Dependent Random Vectors

Let the kp-variate random vector X be partitioned into k subvectors X_i of dimension p each, and let the covariance matrix Ψ of X be partitioned analogously into submatrices Ψ_ij. The common principal component (CPC) model for dependent random vectors assumes the existence of an orthogonal p by p matrix β such that β^tΨ_ijβ is diagonal for all (i, j). After a formal definition of the model, normal theory maximum likelihood estimators are obtained. The asymptotic theory for the estimated orthogonal matrix is derived by a new technique of choosing proper subsets of functionally independent parameters.     

MU-Estimation and Smoothing

In the M-estimation theory developed by Huber (1964, Ann. Math. Statist.43, 1449–1458), the parameter under estimation is the value of θ which minimizes the expectation of what is called a discrepancy measure (DM) δ(X, θ) which is a function of θ and the underlying random variable X. Such a setting does not cover the estimation of parameters such as the multivariate median defined by Oja (1983) and Liu (1990), as the value of θ which minimizes the expectation of a DM of the type δ(X₁, …, X_m, θ) where X₁, …, X_m are independent copies of the underlying random variable X. Arcones et al. (1994, Ann. Statist.22, 1460–1477) studied the estimation of such parameters. We call such an M-type MU-estimation (or μ-estimation for convenience). When a DM is not a differentiable function of θ, some complexities arise in studying the properties of estimators as well as in their computation. In such a case, we introduce a new method of smoothing the DM with a kernel function and using it in estimation. It is seen that smoothing allows us to develop an elegant approach to the study of asymptotic properties and possibly apply the Newton–Raphson procedure in the computation of estimators.     

Set-Indexed Conditional Empirical and Quantile Processes Based on Dependent Data

We consider a conditional empirical distribution of the form F_n(C x)=∑ⁿ_t=1 ω_n(X_t−x) I{Y_tC} indexed by C , where {(X_t, Y_t), t=1, …, n} are observations from a strictly stationary and strong mixing stochastic process, {ω_n(X_t−x)} are kernel weights, and is a class of sets. Under the assumption on the richness of the index class in terms of metric entropy with bracketing, we have established uniform convergence and asymptotic normality for F_n(· x). The key result specifies rates of convergences for the modulus of continuity of the conditional empirical process. The results are then applied to derive Bahadur–Kiefer type approximations for a generalized conditional quantile process which, in the case with independent observations, generalizes and improves earlier results. Potential applications in the areas of estimating level sets and testing for unimodality (or multimodality) of conditional distributions are discussed.     

The Asymptotic Distribution of Sample Autocorrelations for a Class of Linear Filters

We consider a stationary time series {X_t} given by X_t = Σ_kψ_kZ_{t − k}, where the driving stream {Z_t} consists of independent and identically distributed random variables with mean zero and finite variance. Under the assumption that the filtering weights ψ_k are squared summable and that the spectral density of {X_t} is squared integrable, it is shown that the asymptotic distribution of the sequence of sample autocorrelation functions is normal with covariance matrix determined by the well-known Bartlett formula. This result extends classical theorems by Bartlett (1964, J. Roy Statist. Soc. Supp.8 27-41, 85-97) and Anderson and Walker (1964, Ann. Math. Statist.35 1296-1303), which were derived under the assumption that the filtering sequence {ψ_k] is summable.     

Asymptotic expansions of the distributions of some test statistics for Gaussian ARMA processes

Let {X_t} be a Gaussian ARMA process with spectral density f_θ(λ), where θ is an unknown parameter. The problem considered is that of testing a simple hypothesis H:θ = θ₀ against the alternative A:θ ≠ θ₀. For this problem we propose a class of tests , which contains the likelihood ratio (LR), Wald (W), modified Wald (MW) and Rao (R) tests as special cases. Then we derive the χ² type asymptotic expansion of the distribution of T up to order n⁻¹, where n is the sample size. Also we derive the χ² type asymptotic expansion of the distribution of T under the sequence of alternatives A_n: θ = θ₀ + /√n, ε > 0. Then we compare the local powers of the LR, W, MW, and R tests on the basis of their asymptotic expansions.