On Bayes estimators with uniform priors on spheres and their comparative performance with maximum likelihood estimators for estimating bounded multivariate normal means

For independently distributed observables: X_i∼N(θ_i,σ²),i=1,…,p, we consider estimating the vector θ=(θ₁,…,θ_p)^′ with loss ‖d−θ‖² under the constraint , with known τ₁,…,τ_p,σ²,m. In comparing the risk performance of Bayesian estimators δ_α associated with uniform priors on spheres of radius α centered at (τ₁,…,τ_p) with that of the maximum likelihood estimator , we make use of Stein’s unbiased estimate of risk technique, Karlin’s sign change arguments, and a conditional risk analysis to obtain for a fixed (m,p) necessary and sufficient conditions on α for δ_α to dominate . Large sample determinations of these conditions are provided. Both cases where all such δ_α’s and cases where no such δ_α’s dominate are elicited. We establish, as a particular case, that the boundary uniform Bayes estimator δ_m dominates if and only if m≤k(p) with , improving on the previously known sufficient condition of Marchand and Perron (2001) [3] for which . Finally, we improve upon a universal dominance condition due to Marchand and Perron, by establishing that all Bayesian estimators δ_π with π spherically symmetric and supported on the parameter space dominate whenever m≤c₁(p) with .     

Asymptotic properties of computationally efficient alternative estimators for a class of multivariate normal models

Parameters of Gaussian multivariate models are often estimated using the maximum likelihood approach. In spite of its merits, this methodology is not practical when the sample size is very large, as, for example, in the case of massive georeferenced data sets. In this paper, we study the asymptotic properties of the estimators that minimize three alternatives to the likelihood function, designed to increase the computational efficiency. This is achieved by applying the information sandwich technique to expansions of the pseudo-likelihood functions as quadratic forms of independent normal random variables. Theoretical calculations are given for a first-order autoregressive time series and then extended to a two-dimensional autoregressive process on a lattice. We compare the efficiency of the three estimators to that of the maximum likelihood estimator as well as among themselves, using numerical calculations of the theoretical results and simulations.     

Fiducial inference on the largest mean of a multivariate normal distribution

Inference on the largest mean of a multivariate normal distribution is a surprisingly difficult and unexplored topic. Difficulties arise when two or more of the means are simultaneously the largest mean. Our proposed solution is based on an extension of R.A. Fisher’s fiducial inference methods termed generalized fiducial inference. We use a model selection technique along with the generalized fiducial distribution to allow for equal largest means and alleviate the overestimation that commonly occurs. Our proposed confidence intervals for the largest mean have asymptotically correct frequentist coverage and simulation results suggest that they possess promising small sample empirical properties. In addition to the theoretical calculations and simulations we also applied this approach to the air quality index of the four largest cities in the northeastern United States (Baltimore, Boston, New York, and Philadelphia).     

Finite sample tail behavior of multivariate location estimators

A finite sample performance measure of multivariate location estimators is introduced based on “tail behavior”. The tail performance of multivariate “monotone” location estimators and the halfspace depth based “non-monotone” location estimators including the Tukey halfspace median and multivariate L-estimators is investigated. The connections among the finite sample performance measure, the finite sample breakdown point, and the halfspace depth are revealed. It turns out that estimators with high breakdown point or halfspace depth have “appealing” tail performance. The tail performance of the halfspace median is very appealing and also robust against underlying population distributions, while the tail performance of the sample mean is very sensitive to underlying population distributions. These findings provide new insights into the notions of the halfspace depth and breakdown point and identify the important role of tail behavior as a quantitative measure of robustness in the multivariate location setting.     

Estimation of the mean vector of a multivariate normal distribution: subspace hypothesis

This paper considers the estimation of the mean vector θ of a p-variate normal distribution with unknown covariance matrix Σ when it is suspected that for a p×r known matrix B the hypothesis θ=Bη, η∈R^r may hold. We consider empirical Bayes estimators which includes (i) the unrestricted unbiased (UE) estimator, namely, the sample mean vector (ii) the restricted estimator (RE) which is obtained when the hypothesis θ=Bη holds (iii) the preliminary test estimator (PTE), (iv) the James-Stein estimator (JSE), and (v) the positive-rule Stein estimator (PRSE). The biases and the risks under the squared loss function are evaluated for all the five estimators and compared. The numerical computations show that PRSE is the best among all the five estimators even when the hypothesis θ=Bη is true.     

Minimax Bayes estimators of a multivariate normal mean

In three or more dimensions it is well known that the usual point estimator for the mean of a multivariate normal distribution is minimax but not admissible with respect to squared Euclidean distance loss. This paper gives sufficient conditions on the prior distribution under which the Bayes estimator has strictly lower risk than the usual estimator. Examples are given for which the posterior density is useful in the formation of confidence sets.     

Shrinkage estimation in the frequency domain of multivariate time series

In this paper on developing shrinkage for spectral analysis of multivariate time series of high dimensionality, we propose a new nonparametric estimator of the spectral matrix with two appealing properties. First, compared to the traditional smoothed periodogram our shrinkage estimator has a smaller L₂ risk. Second, the proposed shrinkage estimator is numerically more stable due to a smaller condition number. We use the concept of “Kolmogorov” asymptotics where simultaneously the sample size and the dimensionality tend to infinity, to show that the smoothed periodogram is not consistent and to derive the asymptotic properties of our regularized estimator. This estimator is shown to have asymptotically minimal risk among all linear combinations of the identity and the averaged periodogram matrix. Compared to existing work on shrinkage in the time domain, our results show that in the frequency domain it is necessary to take the size of the smoothing span as “effective sample size” into account. Furthermore, we perform extensive Monte Carlo studies showing the overwhelming gain in terms of lower L₂ risk of our shrinkage estimator, even in situations of oversmoothing the periodogram by using a large smoothing span.     

Asymptotic properties of Bayes estimators for Gaussian Itô-processes with noisy observations

The estimation of a real parameter θ in a linear stochastic differential equation of the simple type is investigated, based on noisy, time continuous observations of X_t. Sufficient conditions on the continuous functions β and σ are given such that the (conditionally normal) Bayes estimators of θ satisfy certain error bounds and are strongly consistent.     

Estimation of a tail index based on minimum density power divergence

In this paper, we consider the minimum density power divergence estimator for the tail index of heavy tailed distributions in strong mixing processes. It is shown that the estimator is consistent and asymptotically normal under regularity conditions. The simulation results demonstrate that the estimator is robust in the presence of outliers.     

On the limiting spectral distribution of the covariance matrices of time-lagged processes

We consider two continuous-time Gaussian processes, one being partially correlated to a time-lagged version of the other. We first give the limiting spectral distribution for the covariance matrices of the increments of the processes when the span between two observations tends to zero. Then, we derive the limiting distribution of the eigenvalues of the sample covariance matrices. This result is obtained when the number of paths of the processes is asymptotically proportional to the number of observations for each single path. As an application, we use the second moment of this distribution together with auxiliary volatility and correlation estimates to construct an adaptive estimator of the time lag between the two processes. Finally, we provide an asymptotic theory for our estimation procedure.     

Efficiencies of tests and estimators forp-order autoregressive processes when the error distribution is nonnormal

Summary We considerpth order autoregressive time series where the shocks need not be normal. By employing the concept of contiguity, we obtain the sysmptotic power for tests of hypothesis concerning the autoregressive parameters. Our approach allows consideration of the double exponential and other thicker-tailed distributions for the shocks. We derive a new result in the contiguity framework that leads directly to an expression for the Pitman efficiencies of tests as well as estimators. The numerical values of the efficiencies suggest a lack of robustness for the normal theory least squares estimators when the shock distribution is thick tailed or an outlier prone mixed normal. An important alternative test statistic is proposed that competes with the normal theory tests. This research was supported by the Office of Naval Research under Grant No. N00014-78-C-0722 and by the Army Research Office.     

On the distribution of penalized maximum likelihood estimators: The LASSO, SCAD, and thresholding

We study the distributions of the LASSO, SCAD, and thresholding estimators, in finite samples and in the large-sample limit. The asymptotic distributions are derived for both the case where the estimators are tuned to perform consistent model selection and for the case where the estimators are tuned to perform conservative model selection. Our findings complement those of Knight and Fu [K. Knight, W. Fu, Asymptotics for lasso-type estimators, Annals of Statistics 28 (2000) 1356–1378] and Fan and Li [J. Fan, R. Li, Variable selection via non-concave penalized likelihood and its oracle properties, Journal of the American Statistical Association 96 (2001) 1348–1360]. We show that the distributions are typically highly non-normal regardless of how the estimator is tuned, and that this property persists in large samples. The uniform convergence rate of these estimators is also obtained, and is shown to be slower than n^−1/2 in case the estimator is tuned to perform consistent model selection. An impossibility result regarding estimation of the estimators’ distribution function is also provided.     

Asymptotic expansions of maximum likelihood estimators for small diffusions via the theory of Malliavin-Watanabe

Summary The asymptotic expansions of the probability distributions of statistics for the small diffusion are derived by means of the Malliavin calculus. From this the second order efficiency of the maximum likelihood estimator is proved.The research was supported in part by Grant-in-Aid for Encouragement of Young Scientists from the Ministry of Education, Science and Culture     

Preliminary test estimation for spectra

For a general non-Gaussian stationary linear process, quasi-maximum likelihood estimation of a subset of the parameters of the spectral density is considered when the complementary subset is suspected to be superfluous. A preliminary test quasi-maximum likelihood estimator (q-MLE) of parameters is introduced and, in the light of its mean square error, is compared with the restricted and unrestricted q-MLE.     

Asymptotic expansion of Bayes estimators for small diffusions

Summary Using the Malliavin calculus we derived asymptotic expansion of the distributions of the Bayes estimators for small diffusions. The second order efficiency of the Bayes estimator is proved.     

Enhanced one-sided confidence regions for a multivariate location parameter

If a one-sided test for a multivariate location parameter is inverted, the resulting confidence region may have an unpleasant shape. In particular, if the null and alternative hypothesis are both composite and complementary, the confidence region usually does not resemble the alternative parameter region in shape, but rather a reflected version of the null parameter region.We illustrate this effect and show one possibility of obtaining confidence regions for the location parameter that are smaller and have a more suitable shape for the type of problems investigated. This method is based on the closed testing principle applied to a family of nested hypotheses.     

Who's afraid of reduced-rank parameterizations of multivariate models? Theory and example

Reduced-rank restrictions can add useful parsimony to coefficient matrices of multivariate models, but their use is limited by the daunting complexity of the methods and their theory. The present work takes the easy road, focusing on unifying themes and simplified methods. For Gaussian and non-Gaussian (GLM, GAM, mixed normal, etc.) multivariate models, the present work gives a unified, explicit theory for the general asymptotic (normal) distribution of maximum likelihood estimators (MLE). MLE can be complex and computationally hard, but we show a strong asymptotic equivalence between MLE and a relatively simple minimum (Mahalanobis) distance estimator. The latter method yields particularly simple tests of rank, and we describe its asymptotic behavior in detail. We also examine the method's performance in simulation and via analytical and empirical examples.     

Bayesian analysis of multivariate t linear mixed models using a combination of IBF and Gibbs samplers

The multivariate linear mixed model (MLMM) has become the most widely used tool for analyzing multi-outcome longitudinal data. Although it offers great flexibility for modeling the between- and within-subject correlation among multi-outcome repeated measures, the underlying normality assumption is vulnerable to potential atypical observations. We present a fully Bayesian approach to the multivariate t linear mixed model (MtLMM), which is a robust extension of MLMM with the random effects and errors jointly distributed as a multivariate t distribution. Owing to the introduction of too many hidden variables in the model, the conventional Markov chain Monte Carlo (MCMC) method may converge painfully slowly and thus fails to provide valid inference. To alleviate this problem, a computationally efficient inverse Bayes formulas (IBF) sampler coupled with the Gibbs scheme, called the IBF-Gibbs sampler, is developed and shown to be effective in drawing samples from the target distributions. The issues related to model determination and Bayesian predictive inference for future values are also investigated. The proposed methodologies are illustrated with a real example from an AIDS clinical trial and a careful simulation study.