Consistency of a recursive nearest neighbor regression function estimate

Let (X, Y) be an ^d × -valued random vector and let (X₁, Y₁),…,(X_N, Y_N) be a random sample drawn from its distribution. Divide the data sequence into disjoint blocks of length l₁, …, l_n, find the nearest neighbor to X in each block and call the corresponding couple (X_i^*, Y_i^*). It is shown that the estimate m_n(X) = Σ_{i = 1}ⁿ w_niY_i^*/Σ_{i = 1}ⁿ w_ni of m(X) = E{Y|X} satisfies E{|m_n(X) − m(X)|^p} 0 (p ≥ 1) whenever E{|Y|^p} < ∞, l_n ∞, and the triangular array of positive weights {w_ni} satisfies sup_{i ≤ n}w_ni/Σ_{i = 1}ⁿ w_ni 0. No other restrictions are put on the distribution of (X, Y). Also, some distribution-free results for the strong convergence of E{|m_n(X) − m(X)|^p|X₁, Y₁,…, X_N, Y_N} to zero are included. Finally, an application to the discrimination problem is considered, and a discrimination rule is exhibited and shown to be strongly Bayes risk consistent for all distributions.     

Exponential bounds of mean error for the kernel estimates of regression functions

Let (X, Y), (X₁, Y₁), …, (X_n, Y_n) be i.d.d. R^r × R-valued random vectors with E|Y| < ∞, and let Q_n(x) be a kernel estimate of the regression function Q(x) = E(Y|X = x). In this paper, we establish an exponential bound of the mean deviation between Q_n(x) and Q(x) given the training sample Zⁿ = (X₁, Y₁, …, X_n, Y_n), under conditions as weak as possible.     

Minimax estimation of the mean of spherically symmetric distributions under general quadratic loss

For X one observation on a p-dimensional (p ≥ 4) spherically symmetric (s.s.) distribution about θ, minimax estimators whose risks dominate the risk of X (the best invariant procedure) are found with respect to general quadratic loss, L(δ, θ) = (δ − θ)′ D(δ − θ) where D is a known p × p positive definite matrix. For C a p × p known positive definite matrix, conditions are given under which estimators of the form δ_a,r,C,D(X) = (I − (ar(|X|²)) D^−1/2CD^1/2 |X|⁻²)X are minimax with smaller risk than X. For the problem of estimating the mean when n observations X₁, X₂, …, X_n are taken on a p-dimensional s.s. distribution about θ, any spherically symmetric translation invariant estimator, δ(X₁, X₂, …, X_n), with have a s.s. distribution about θ. Among the estimators which have these properties are best invariant estimators, sample means and maximum likelihood estimators. Moreover, under certain conditions, improved robust estimators can be found.     

Global nonparametric estimation of conditional quantile functions and their derivatives

Let (X, Y) be a random vector such that X is d-dimensional, Y is real valued, and θ(X) is the conditional αth quantile of Y given X, where α is a fixed number such that 0 < α < 1. Assume that θ is a smooth function with order of smoothness p > 0, and set r = (p − m)/(2p + d), where m is a nonnegative integer smaller than p. Let T(θ) denote a derivative of θ of order m. It is proved that there exists estimate of T(θ), based on a set of i.i.d. observations (X₁, Y₁), …, (X_n, Y_n), that achieves the optimal nonparametric rate of convergence n^−r in L_q-norms (1 ≤ q < ∞) restricted to compacts under appropriate regularity conditions. Further, it has been shown that there exists estimate of T(θ) that achieves the optimal rate (n/log n)^−r in L_∞-norm restricted to compacts.     

L1-optimal estimates for a regression type function in R

Let X₁, X₂, …, X_n be random vectors that take values in a compact set in R^d, d ≥ 1. Let Y₁, Y₂, …, Y_n be random variables (“the responses”) which conditionally on X₁ = x₁, …, X_n = x_n are independent with densities f(y | x_i, θ(x_i)), i = 1, …, n. Assuming that θ lives in a sup-norm compact space Θ_q,d of real valued functions, an optimal L₁-consistent estimator of θ is constructed via empirical measures. The rate of convergence of the estimator to the true parameter θ depends on Kolmogorov's entropy of Θ_q,d.     

A Kernel Estimator of a Conditional Quantile

Let (X₁, Y₁), (X₂, Y₂), …, be two-dimensional random vectors which are independent and distributed as (X, Y). For 0<p<1, letξ(px) be the conditionalpth quantile ofYgivenX=x; that is,ξ(px)=inf{y : P(YyX=x)p}. We consider the problem of estimatingξ(px) from the data (X₁, Y₁), (X₂, Y₂), …, (X_n, Y_n). In this paper, a new kernel estimator ofξ(px) is proposed. The asymptotic normality and a law of the iterated logarithm are obtained.     

Deconvolving a Density from Partially Contaminated Observations

We consider the problem of estimating a continuous bounded probability density function when independent data X₁, ..., X_n from the density are partially contaminated by measurement error. In particular, the observations Y₁, ..., Y_n are such that P(Y_j = X_j) = p and P(Y_j = X_j + ε_j) = 1 − p, where the errors ε_j are independent (of each other and of the X_j) and identically distributed from a known distribution. When p = 0 it is well known that deconvolution via kernel density estimators suffers from notoriously slow rates of convergence. For normally distributed ε_j the best possible rates are of logarithmic order pointwise and in mean square error. In this paper we demonstrate that for merely partially(0 < p <1) contaminated observations (where of course it is unknown which observations are contaminated and which are not) under mild conditions almost sure rates of order O(((log h⁻¹)/nh)^1/2) with h = h(n) = const(log n/n)^1/5 are achieved for convergence in L_∞-norm. This is equal to the optimal rate available in ordinary density estimation from direct uncontaminated observations (p = 1). A corresponding result is obtained for the mean integrated squared error.     

Correlations of functions of normal variables

Let (X₁, X₂,…, X_k, Y₁, Y₂,…, Y_k) be multivariate normal and define a matrix C by C_ij = cov(X_i, Y_j). If (i) (X₁,…, X_k) = (Y₁,…, Y_k) and (ii) C is symmetric positive definite, then 0 < varf(X₁,…, X_k) < ∞ corr(f(X₁,…, X_k),f(Y₁,…, Y_k)) > 0. Condition (i) is necessary for the conclusion. The sufficiency of (i) and (ii) follows from an infinite-dimensional version, which can also be applied to a pair of jointly normal Brownian motions.     

On the approximate behavior of the posterior distribution for an extreme multivariate observation

The behavior of the posterior for a large observation is considered. Two basic situations are discussed; location vectors and natural parameters.Let X = (X₁, X₂, …, X_n) be an observation from a multivariate exponential distribution with that natural parameter Θ = (Θ₁, Θ₂, …, Θ_n). Let θ_x^* be the posterior mode. Sufficient conditions are presented for the distribution of Θ − θ_x^* given X = x to converge to a multivariate normal with mean vector 0 as |x| tends to infinity. These same conditions imply that E(Θ | X = x) − θ_x^* converges to the zero vector as |x| tends to infinity.The posterior for an observation X = (X₁, X₂, …, X_n is considered for a location vector Θ = (Θ₁, Θ₂, …, Θ_n) as x gets large along a path, γ, in Rⁿ. Sufficient conditions are given for the distribution of γ(t) − Θ given X = γ(t) to converge in law as t → ∞. Slightly stronger conditions ensure that γ(t) − E(Θ | X = γ(t)) converges to the mean of the limiting distribution.These basic results about the posterior mean are extended to cover other estimators. Loss functions which are convex functions of absolute error are considered. Let δ be a Bayes estimator for a loss function of this type. Generally, if the distribution of Θ − E(Θ | X = γ(t)) given X = γ(t) converges in law to a symmetric distribution as t → ∞, it is shown that δ(γ(t)) − E(Θ | X = γ(t)) → 0 as t → ∞.     

Prophet regions and sharp inequalities for pth absolute moments of martingales

Exact comparisons are made relating E|Y₀|^p, E|Y_n−1|^p, and E(max_j≤n−1 |Y_j|^p), valid for all martingales Y₀,…,Y_n−1, for each p ≥ 1. Specifically, for p > 1, the set of ordered triples {(x, y, z) : X = E|Y₀|^p, Y = E |Y_n−1|^p, and Z = E(max_j≤n−1 |Y_j|^p) for some martingale Y₀,…,Y_n−1} is precisely the set {(x, y, z) : 0≤x≤y≤z≤Ψ_n,p(x, y)}, where Ψ_n,p(x, y) = xψ_n,p(y/x) if x > 0, and = a_n−1,py if x = 0; here ψ_n,p is a specific recursively defined function. The result yields families of sharp inequalities, such as E(max_j≤n−1 |Y_j|^p) + ψ_n,p^*(a) E |Y₀|^p ≤ aE |Y_n−1|^p, valid for all martingales Y₀,…,Y_n−1, where ψ_n,p^* is the concave conjugate function of ψ_n,p. Both the finite sequence and infinite sequence cases are developed. Proofs utilize moment theory, induction, conjugate function theory, and functional equation analysis.     

On the transition from a Markov chain to a continuous time process

Starting from a real-valued Markov chain X₀,X₁,…,X_n with stationary transition probabilities, a random element {Y(t);t[0, 1]} of the function space D[0, 1] is constructed by letting Y(k/n)=X_k, k= 0,1,…,n, and assuming Y (t) constant in between. Sample tightness criteria for sequences {Y(t);t[0,1]};_n of such random elements in D[0, 1] are then given in terms of the one-step transition probabilities of the underlying Markov chains. Applications are made to Galton-Watson branching processes.     

Checking the adequacy of the multivariate semiparametric location shift model

Let X,X₁,…,X_m,…, Y,Y₁,…,Y_n,… be independent d-dimensional random vectors, where the X_j are i.i.d. copies of X, and the Y_k are i.i.d. copies of Y. We study a class of consistent tests for the hypothesis that Y has the same distribution as X+μ for some unspecified . The test statistic L is a weighted integral of the squared modulus of the difference of the empirical characteristic functions of and Y₁,…,Y_n, where is an estimator of μ. An alternative representation of L is given in terms of an L²-distance between two nonparametric density estimators. The finite-sample and asymptotic null distribution of L is independent of μ. Carried out as a bootstrap or permutation procedure, the test is asymptotically of a given size, irrespective of the unknown underlying distribution. A large-scale simulation study shows that the permutation procedure performs better than the bootstrap.     

On improved estimators of the generalized variance

Treated in this paper is the problem of estimating with squared error loss the generalized variance | Σ | from a Wishart random matrix S: p × p W_p(n, Σ) and an independent normal random matrix X: p × k N(ξ, Σ I_k) with ξ(p × k) unknown. Denote the columns of X by X₍₁₎ ,…, X_(k) and set ψ⁽⁰⁾(S, X) = {(n − p + 2)!/(n + 2)!} | S |, ψ⁽ⁱ⁾(X, X) = min[ψ⁽ⁱ⁻¹⁾(S, X), {(n − p + i + 2)!/(n + i + 2)!} | S + X₍₁₎ X′₍₁₎ + + X_(i) X′_(i) |] and Ψ⁽ⁱ⁾(S, X) = min[ψ⁽⁰⁾(S, X), {(n − p + i + 2)!/(n + i + 2)!}| S + X₍₁₎ X′₍₁₎ + + X_(i) X′_(i) |], i = 1,…,k. Our result is that the minimax, best affine equivariant estimator ψ⁽⁰⁾(S, X) is dominated by each of Ψ⁽ⁱ⁾(S, X), i = 1,…,k and for every i, ψ⁽ⁱ⁾(S, X) is better than ψ⁽ⁱ⁻¹⁾(S, X). In particular, ψ^(k)(S, X) = min[{(n − p + 2)!/(n + 2)!} | S |, {(n − p + 2)!/(n + 2)!} | S + X₍₁₎X′₍₁₎|,…,| {(n − p + k + 2)!/(n + k + 2)!} | S + X₍₁₎X′₍₁₎ + + X_(k)X′_(k)|] dominates all other ψ's. It is obtained by considering a multivariate extension of Stein's result (Ann. Inst. Statist. Math. 16, 155–160 (1964)) on the estimation of the normal variance.     

Asymptotically optimal selection of a piecewise polynomial estimator of a regression function

Let (X, Y) be a pair of random variables such that X = (X₁,…, X_d) ranges over a nondegenerate compact d-dimensional interval C and Y is real-valued. Let the conditional distribution of Y given X have mean θ(X) and satisfy an appropriate moment condition. It is assumed that the distribution of X is absolutely continuous and its density is bounded away from zero and infinity on C. Without loss of generality let C be the unit cube. Consider an estimator of θ having the form of a piecewise polynomial of degree k_n based on m_n^d cubes of length 1/m_n, where the m_n^d(_d^k_n+d) coefficients are chosen by the method of least squares based on a random sample of size n from the distribution of (X, Y). Let (k_n, m_n) be chosen by the FPE procedure. It is shown that the indicated estimator has an asymptotically minimal squared error of prediction if θ is not of the form of piecewise polynomial.     

Stochastic comparisons of order statistics from heterogeneous populations, with applications in reliability

The basic result of the paper states: Let F₁, …, F_n, F₁^′,…, F_n^′ have proportional hazard functions with λ₁ ,…, λ_n , λ₁^′ ,…, λ_n^′ as the constants of proportionality. Let X₍₁₎ ≤ … ≤ X_(n) (X₍₁₎^′ ≤ … ≤ X_(n)^′) be the order statistics in a sample of size n from the heterogeneous populations {F₁ ,…, F_n}({F₁^′ ,…, F_n^′}). Then (λ₁ ,…, λ_n) majorizes (λ₁^′ ,…, λ_n^′) implies that (X₍₁₎ ,…, X_(n)) is stochastically larger than (X₍₁₎^′ ,…, X_(n)^′). Earlier results stochastically comparing individual order statistics are shown to be special cases. Applications of the main result are made in the study of the robustness of standard estimates of the failure rate of the exponential distribution, when observations actually come from a set of heterogeneous exponential distributions. Further applications are made to the comparisons of linear combinations of Weibull random variables and of binomial random variables.     

Generalized Hodges-Lehmann estimators for the analysis of variance

For X_i, …, X_n a random sample and K(·, ·) a symmetric kernel this paper considers large sample properties of location estimator satisfying , . Asymptotic normality of is obtained and two forms of interval estimators for parameter θ satisfying EK(X₁ − θ, X₂ − θ) = 0, are discussed. Consistent estimation of the variance parameters is obtained which permits the construction of asymptotically distribution free procedures. The p-variate and multigroup extension is accomplished to provide generalized one-way MANOVA. Monte Carlo results are included.     

On confidence sequences for the mean vector of a multivariate normal distribution

Let X₁, X₂,… be idd random vectors with a multivariate normal distribution N(μ, Σ). A sequence of subsets {R_n(a₁, a₂,…, a_n), n ≥ m} of the space of μ is said to be a (1 − α)-level sequence of confidence sets for μ if P(μ R_n(X₁, X₂,…, X_n) for every n ≥ m) ≥ 1 − α. In this note we use the ideas of Robbins Ann. Math. Statist. 41 (1970) to construct confidence sequences for the mean vector μ when Σ is either known or unknown. The constructed sequence R_n(X₁, X₂, …, X_n) depends on Mahalanobis' or Hotelling's according as Σ is known or unknown. Confidence sequences for the vector-valued parameter in the general linear model are also given.     

Third-order asymptomic properties of a class of test statistics under a local alternative

Suppose that {X_i; I = 1, 2, …,} is a sequence of p-dimensional random vectors forming a stochastic process. Let p_n, θ(X_n), X_n ^np, be the probability density function of X_n = (X₁, …, X_n) depending on θ Θ, where Θ is an open set of ¹. We consider to test a simple hypothesis H : θ = θ₀ against the alternative A : θ ≠ θ₀. For this testing problem we introduce a class of tests , which contains the likelihood ratio, Wald, modified Wald, and Rao tests as special cases. Then we derive the third-order asymptotic expansion of the distribution of T under a sequence of local alternatives. Using this result we elucidate various third-order asymptotic properties of T (e.g., Bartlett's adjustments, third-order asymptotically most powerful properties). Our results are very general, and can be applied to the i.i.d. case, multivariate analysis, and time series analysis. Two concrete examples will be given. One is a Gaussian ARMA process (dependent case), and the other is a nonlinear regression model (non-identically distributed case).     

Asymptotic properties of Cramér-Smirnov statistics—A new approach

Let Y₁,…, Y_n be independent identically distributed random variables with distribution function F(x, θ), θ = (θ′₁, θ′₂), where θ_i (i = 1, 2) is a vector of p_i components, p = p₁ + p₂ and for θI, an open interval in ^p, F(x, θ) is continuous. In the present paper the author shows that the asymptotic distribution of modified Cramér-Smirnov statistic under H_n: θ₁ = θ₁₀ + n^−1/2γ, θ₂ unspecified, where γ is a given vector independent of n, is the distribution of a sum of weighted noncentral χ₁² variables whose weights are eigenvalues of a covariance function of a Gaussian process and noncentrality parameters are Fourier coefficients of the mean function of the Gaussian process. Further, the author exploits the special form of the covariance function by using perturbation theory to obtain the noncentrality parameters and the weights. The technique is applicable to other goodness-of-fit statistics such as U² [G. S. Watson, Biometrika 48 (1961), 109–114].     

Some New Results on Stochastic Comparisons of Spacings from Heterogeneous Exponential Distributions

Some new results are obtained on stochastic orderings between random vectors of spacings from heterogeneous exponential distributions and homogeneous ones. LetD₁, …, D_nbe the normalized spacings associated with independent exponential random variablesX₁, …,X_n, whereX_ihas hazard rateλ_i,i=1, 2, …, n. LetD*₁, …, D*_nbe the normalized spacings of a random sampleY₁, …, Y_nof sizenfrom an exponential distribution with hazard rateλ=∑ⁿ_i=1 λ_i/n. It is shown that for anyn2, the random vector (D₁, …, D_n) is greater than the random vector (D*₁, …, D*_n) in the sense of multivariate likelihood ratio ordering. It also follows from the results proved in this paper that for anyjbetween 2 andn, the survival function ofX_j:n−X_1:nis Schur convex.