Efficient algorithms for generating truncated multivariate normal distributions

Sampling from a truncated multivariate normal distribution (TMVND) constitutes the core computational module in fitting many statistical and econometric models. We propose two efficient methods, an iterative data augmentation (DA) algorithm and a non-iterative inverse Bayes formulae (IBF) sampler, to simulate TMVND and generalize them to multivariate normal distributions with linear inequality constraints. By creating a Bayesian incomplete-data structure, the posterior step of the DA algorithm directly generates random vector draws as opposed to single element draws, resulting obvious computational advantage and easy coding with common statistical software packages such as S-PLUS, MATLAB and GAUSS. Furthermore, the DA provides a ready structure for implementing a fast EM algorithm to identify the mode of TMVND, which has many potential applications in statistical inference of constrained parameter problems. In addition, utilizing this mode as an intermediate result, the IBF sampling provides a novel alternative to Gibbs sampling and eliminates problems with convergence and possible slow convergence due to the high correlation between components of a TMVND. The DA algorithm is applied to a linear regression model with constrained parameters and is illustrated with a published data set. Numerical comparisons show that the proposed DA algorithm and IBF sampler are more efficient than the Gibbs sampler and the accept-reject algorithm.     

Solution of a problem on the identification of parameters by the distribution of the maximum random variable: a multivariate normal case

In this paper we solve the problem of unique factorization of products ofn-variate nonsingular normal distributions with covariance matrices of the form , _ij =p _{i j} forij, = _i ² ,j=j,p0.     

On the Convergence of Successive Substitution Sampling

Abstract

The problem of finding marginal distributions of multidimensional random quantities has many applications in probability and statistics. Many of the solutions currently in use are very computationally intensive. For example, in a Bayesian inference problem with a hierarchical prior distribution, one is often driven to multidimensional numerical integration to obtain marginal posterior distributions of the model parameters of interest. Recently, however, a group of Monte Carlo integration techniques that fall under the general banner of successive substitution sampling (SSS) have proven to be powerful tools for obtaining approximate answers in a very wide variety of Bayesian modeling situations. Answers may also be obtained at low cost, both in terms of computer power and user sophistication. Important special cases of SSS include the “Gibbs sampler” described by Gelfand and Smith and the “IP algorithm” described by Tanner and Wong. The major problem plaguing users of SSS is the difficulty in ascertaining when “convergence” of the algorithm has been obtained. This problem is compounded by the fact that what is produced by the sampler is not the functional form of the desired marginal posterior distribution, but a random sample from this distribution. This article gives a general proof of the convergence of SSS and the sufficient conditions for both strong and weak convergence, as well as a convergence rate. We explore the connection between higher-order eigenfunctions of the transition operator and accelerated convergence via good initial distributions. We also provide asymptotic results for the sampling component of the error in estimating the distributions of interest. Finally, we give two detailed examples from familiar exponential family settings to illustrate the theory.     

Finite-sample inference with monotone incomplete multivariate normal data, I

We consider problems in finite-sample inference with two-step, monotone incomplete data drawn from , a multivariate normal population with mean and covariance matrix . We derive a stochastic representation for the exact distribution of , the maximum likelihood estimator of . We obtain ellipsoidal confidence regions for through T², a generalization of Hotelling’s statistic. We derive the asymptotic distribution of, and probability inequalities for, T² under various assumptions on the sizes of the complete and incomplete samples. Further, we establish an upper bound for the supremum distance between the probability density functions of and , a normal approximation to .     

删失截断情形下失效率变点模型的Bayes参数估计

通过添加部分缺失寿命变量数据,得到了删失截断情形下失效率变点模型相对简单的似然函数.讨论了所添加缺失数据变量的概率分布和随机抽样方法.利用Monte Carlo EM算法对未知参数进行了迭代.结合Metropolis-Hastings算法对参数的满条件分布进行了Gibbs抽样,基于Gibbs样本对参数进行估计,详细介绍了MCMC方法的实施步骤.随机模拟试验的结果表明各参数Bayes估计的精度较高.     

Supervised learning of multivariate skew normal mixture models with missing information

We establish computationally flexible tools for the analysis of multivariate skew normal mixtures when missing values occur in data. To facilitate the computation and simplify the theoretical derivation, two auxiliary permutation matrices are incorporated into the model for the determination of observed and missing components of each observation and are manifestly effective in reducing the computational complexity. We present an analytically feasible EM algorithm for the supervised learning of parameters as well as missing observations. The proposed mixture analyzer, including the most commonly used Gaussian mixtures as a special case, allows practitioners to handle incomplete multivariate data sets in a wide range of considerations. The methodology is illustrated through a real data set with varying proportions of synthetic missing values generated by MCAR and MAR mechanisms and shown to perform well on classification tasks.     

A note on characterizations of multivariate stable distributions

Several characterizations of multivariate stable distributions together with a characterization of multivariate normal distributions and multivariate stable distributions with Cauchy marginals are given. These are related to some standard characterizations of marcinkiewicz.Research supported, in part, by the Air Force Office of Scientific Research under Contract AFOSR 84-0113. Reproduction in whole or part is permitted for any purpose of the United States Government.     

Finite-sample inference with monotone incomplete multivariate normal data, II

We continue our recent work on inference with two-step, monotone incomplete data from a multivariate normal population with mean and covariance matrix . Under the assumption that is block-diagonal when partitioned according to the two-step pattern, we derive the distributions of the diagonal blocks of and of the estimated regression matrix, . We represent in terms of independent matrices; derive its exact distribution, thereby generalizing the Wishart distribution to the setting of monotone incomplete data; and obtain saddlepoint approximations for the distributions of and its partial Iwasawa coordinates. We prove the unbiasedness of a modified likelihood ratio criterion for testing , where is a given matrix, and obtain the null and non-null distributions of the test statistic. In testing , where and are given, we prove that the likelihood ratio criterion is unbiased and obtain its null and non-null distributions. For the sphericity test, , we obtain the null distribution of the likelihood ratio criterion. In testing we show that a modified locally most powerful invariant statistic has the same distribution as a Bartlett-Pillai-Nanda trace statistic in multivariate analysis of variance.     

A characterization of multivariate normal distribution and its application

Let X₁, X₂, …, X_n be i.i.d. d-dimensional random vectors with a continuous density. Let and . In this paper we find that the distribution of Z_k (or Y_k) can be used for characterizing multivariate normal distribution. This characterization can be employed for testing multivariate normality in terms of the so-called transformation method.     

Joint Bayesian Analysis of Factor Scores and Structural Parameters in the Factor Analysis Model

A Bayesian approach is developed to assess the factor analysis model. Joint Bayesian estimates of the factor scores and the structural parameters in the covariance structure are obtained simultaneously. The basic idea is to treat the latent factor scores as missing data and augment them with the observed data in generating a sequence of random observations from the posterior distributions by the Gibbs sampler. Then, the Bayesian estimates are taken as the sample means of these random observations. Expressions for implementing the algorithm are derived and some statistical properties of the estimates are presented. Some aspects of the algorithm are illustrated by a real example and the performance of the Bayesian procedure is studied using simulation.     

Reversible algorithm of simulating multivariate densities with multi-hump

To simulate a multivariate density with multi-hump, Markov chain Monte Carlo method encounters the obstacle of escaping from one hump to another, since it usually takes extraordinately long time and then becomes practically impossible to perform. To overcome these difficulties, a reversible scheme to generate a Markov chain, in terms of which the simulated density may be successful in rather general cases of practically avoiding being trapped in local humps, was suggested.     

Certain characterizations with linearity of regression in additive damage models

This paper investigates the characterizations of certain discrete distributions within the framework of a multivariate additive damage model. The univariate case for such a model appeared in an article by [2]. In this model a p-dimensional observation is subjected to damage according to a specified probability law represented by a joint survival distribution. Here, it is shown that the linearity of regression of the damaged part on the undamaged ones leads to the characterizations of the multivariate binomial, and multiple inverse hypergeometric distribution as survival distributions.     

非线性再生散度模型的Bayes估计

非线性再生散度模型是指数族非线性模型、广义线性模型和正态非线性回归模型的推广和发展,唐年胜等人研究了该模型参数的极大似然估计及其统计诊断。本文基于Gibbs抽样和MH抽样算法讨论非线性再生散度模型参数的Bayes估计。模拟研究和实例分析被用来说明该方法的有效性。     

A characterization of the multivariate normal distribution by using the hazard gradient

We give a general result to characterize a multivariate distribution from a relationship between the left truncated mean function and the hazard gradient function. This result allows us to obtain new characterizations of multivariate distributions. In particular, we show that, for the multivariate normal distribution, the simple relationship, obtained in standardized form by McGill (1992,Communications in Statistics. Theory Methods,21(11), 3053–3060), actually characterizes the multivariate normal distribution. Supported by Ministerio de Ciencia y Tecnologia under grant BFM2000-0362.     

Admissible minimax estimators of a mean vector of scale mixtures of multivariate normal distributions

The problem of estimating a mean vector of scale mixtures of multivariate normal distributions with the quadratic loss function is considered. For a certain class of these distributions, which includes at least multivariate-t distributions, admissible minimax estimators are given.     

Improved prediction for a multivariate normal distribution with unknown mean and variance

The prediction problem for a multivariate normal distribution is considered where both mean and variance are unknown. When the Kullback–Leibler loss is used, the Bayesian predictive density based on the right invariant prior, which turns out to be a density of a multivariate t-distribution, is the best invariant and minimax predictive density. In this paper, we introduce an improper shrinkage prior and show that the Bayesian predictive density against the shrinkage prior improves upon the best invariant predictive density when the dimension is greater than or equal to three.     

纵向数据下半参数联合均值协方差模型的贝叶斯分析

基于改进的Cholesky分解,研究分析了纵向数据下半参数联合均值协方差模型的贝叶斯估计和贝叶斯统计诊断,其中非参数部分采用B样条逼近.主要通过应用Gibbs抽样和Metropolis-Hastings算法相结合的混合算法获得模型中未知参数的贝叶斯估计和贝叶斯数据删除影响诊断统计量.并利用诊断统计量的大小来识别数据的异常点.模拟研究和实例分析都表明提出的贝叶斯估计和诊断方法是可行有效的.