Multivariate Gaussian Simulation Outside Arbitrary Ellipsoids

Methods for simulation from multivariate Gaussian distributions restricted to be from outside an arbitrary ellipsoidal region are often needed in applications. A standard rejection algorithm that draws a sample from a multivariate Gaussian distribution and accepts it if it is outside the ellipsoid is often employed; however, this is computationally inefficient if the probability of that ellipsoid under the multivariate normal distribution is substantial. We provide a two-stage rejection sampling scheme for drawing samples from such a truncated distribution. Experiments show that the added complexity of the two-stage approach results in the standard algorithm being more efficient for small ellipsoids (i.e., with small rejection probability). However, as the size of the ellipsoid increases, the efficiency of the two-stage approach relative to the standard algorithm increases indefinitely. The relative efficiency also increases as the number of dimensions increases, as the centers of the ellipsoid and the multivariate Gaussian distribution come closer, and as the shape of the ellipsoid becomes more spherical. We provide results of simulation experiments conducted to quantify the relative efficiency over a range of parameter settings.     

高维正态概率积分的降维算法与L_1逼近

高维正态概率积分计算一直是统计学家关注的课题．早期工作已由Gupta（1963）[1]评价，并给出大量的参考文献．近期工作则可参考Tong（1990）[2]的专著．虽然有关的文献很多，但是除了二、三维问题已有较好的算法外（例如见Zhana－Yana，1993[3]），更高维问题尚无公认的有效算法．在维数m＞3的高维情形，多数文章常假设积分域或相关阵有特殊形式，否则只有使用MonteCarlo方法［4］或拟MonteCarlo方法（亦称数论网格方法，例如见Fang－Wang，1994[5]）．但即使是被认为较好的拟MonteCarlo方法，其收敛阶仅为O（n－2／m），因此对于真…     

A New Algorithm for Simulating a Correlation Matrix Based on Parameter Expansion and Reparameterization

The correlation matrix (denoted by R) plays an important role in many statistical models. Unfortunately, sampling the correlation matrix in Markov chain Monte Carlo (MCMC) algorithms can be problematic. In addition to the positive definite constraint of covariance matrices, correlation matrices have diagonal elements fixed at one. In this article, we propose an efficient two-stage parameter expanded reparameterization and Metropolis-Hastings (PX-RPMH) algorithm for simulating R. Using this algorithm, we draw all elements of R simultaneously by first drawing a covariance matrix from an inverse Wishart distribution, and then translating it back to a correlation matrix through a reduction function and accepting it based on a Metropolis-Hastings acceptance probability. This algorithm is illustrated using multivariate probit (MVP) models and multivariate regression (MVR) models with a common correlation matrix across groups. Via both a simulation study and a real data example, the performance of the PX-RPMH algorithm is compared with those of other common algorithms. The results show that the PX-RPMH algorithm is more efficient than other methods for sampling a correlation matrix.     

Generation of Over-Dispersed and Under-Dispersed Binomial Variates

Abstract

This article proposes an algorithm for generating over-dispersed and under-dispersed binomial variates with specified mean and variance. The over-dispersed/under-dispersed distributions are derived from correlated binary variables with an underlying continuous multivariate distribution. Different multivariate distributions or different correlation matrices result in different over-dispersed (or under-dispersed) distributions. The over-dispersed binomial distributions that are generated from three different correlation matrices of a multivariate normal are compared with the beta-binomial distribution for various mean and over-dispersion parameters by quantile-quantile (Q-Q) plots. The two distributions appear to be similar. The under-dispersed binomial distribution is simulated to model an example data set that exhibits under-dispersed binomial variation.     

Properties of Prior and Posterior Distributions for Multivariate Categorical Response Data Models

In this article, we model multivariate categorical (binary and ordinal) response data using a very rich class of scale mixture of multivariate normal (SMMVN) link functions to accommodate heavy tailed distributions. We consider both noninformative as well as informative prior distributions for SMMVN-link models. The notation of informative prior elicitation is based on available similar historical studies. The main objectives of this article are (i) to derive theoretical properties of noninformative and informative priors as well as the resulting posteriors and (ii) to develop an efficient Markov chain Monte Carlo algorithm to sample from the resulting posterior distribution. A real data example from prostate cancer studies is used to illustrate the proposed methodologies.     

多元正态分布的VDR条件拟合优度检验

提出多元正态性χ2检验统计量.多元正态分布转换样本Yd=RVd服从PearsonII型分布,证明了R2服从贝塔分布.基于贝塔分布和单位球均匀分布,得到多元正态性检验统计量χ2的渐近卡方分布.功效模拟显示,χ2统计量优于已有主要多元正态性检验统计量.做iris数据多元正态性的拟合优度检验.     

Covariance Decompositions for Accurate Computation in Bayesian Scale-Usage Models

Analyses of multivariate ordinal probit models typically use data augmentation to link the observed (discrete) data to latent (continuous) data via a censoring mechanism defined by a collection of “cutpoints.” Most standard models, for which effective Markov chain Monte Carlo (MCMC) sampling algorithms have been developed, use a separate (and independent) set of cutpoints for each element of the multivariate response. Motivated by the analysis of ratings data, we describe a particular class of multivariate ordinal probit models where it is desirable to use a common set of cutpoints. While this approach is attractive from a data-analytic perspective, we show that the existing efficient MCMC algorithms can no longer be accurately applied. Moreover, we show that attempts to implement these algorithms by numerically approximating required multivariate normal integrals over high-dimensional rectangular regions can result in severely degraded estimates of the posterior distribution. We propose a new data augmentation that is based on a covariance decomposition and that admits a simple and accurate MCMC algorithm. Our data augmentation requires only that univariate normal integrals be evaluated, which can be done quickly and with high accuracy. We provide theoretical results that suggest optimal decompositions within this class of data augmentations, and, based on the theory, recommend default decompositions that we demonstrate work well in practice. This article has supplementary material online.     

A heuristic approach for the generation of multivariate random samples with specified marginal distributions and correlation matrix

Summary  This paper presents a heuristic approach for multivariate random number generation. Our aim is to generate multivariate samples with specified marginal distributions and correlation matrix, which can be incorporated into risk analysis models to conduct simulation studies. The proposed sampling approach involves two distinct steps: first a univariate random sample from each specified probability distribution is generated; then a heuristic combinatorial optimization procedure is used to rearrange the generated univariate samples, in order to obtain the desired correlations between them. The combinatorial optimization step is performed with a simulated annealing algorithm, which changes only the positions and not the values of the numbers generated in the first step. The proposed multivariate sampling approach can be used with any type of marginal distributions: continuous or discrete, parametric or non-parametric, etc.     

Efficient ML Estimation of the Multivariate Normal Distribution from Incomplete Data

It is well known that the maximum likelihood estimates (MLEs) of a multivariate normal distribution from incomplete data with a monotone pattern have closed-form expressions and that the MLEs from incomplete data with a general missing-data pattern can be obtained using the Expectation-Maximization (EM) algorithm. This article gives closed-form expressions, analogous to the extension of the Bartlett decomposition, for both the MLEs of the parameters and the associated Fisher information matrix from incomplete data with a monotone missing-data pattern. For MLEs of the parameters from incomplete data with a general missing-data pattern, we implement EM and Expectation-Constrained-Maximization-Either (ECME), by augmenting the observed data into a complete monotone sample. We also provide a numerical example, which shows that the monotone EM (MEM) and monotone ECME (MECME) algorithms converge much faster than the EM algorithm.     

Genetic algorithms with noisy fitness

The convergence properties of genetic algorithms with noisy fitness information are studied here. In the proposed scheme, hypothesis testing methods are used to compare sample fitness values. The “best” individual of each generation is kept and a greater-than-zero mutation rate is used so that every individual will be generated with positive probability in each generation. The convergence criterion is different from the frequently-used uniform population criterion; instead, the sequence of the “best” individual in each generation is considered, and the algorithm is regarded as convergent if the sequence of the “best” individuals converges with probability one to a point with optimal average fitness.     

Efficient Bayesian Inference for Multivariate Probit Models With Sparse Inverse Correlation Matrices

We propose a Bayesian approach for inference in the multivariate probit model, taking into account the association structure between binary observations. We model the association through the correlation matrix of the latent Gaussian variables. Conditional independence is imposed by setting some off-diagonal elements of the inverse correlation matrix to zero and this sparsity structure is modeled using a decomposable graphical model. We propose an efficient Markov chain Monte Carlo algorithm relying on a parameter expansion scheme to sample from the resulting posterior distribution. This algorithm updates the correlation matrix within a simple Gibbs sampling framework and allows us to infer the correlation structure from the data, generalizing methods used for inference in decomposable Gaussian graphical models to multivariate binary observations. We demonstrate the performance of this model and of the Markov chain Monte Carlo algorithm on simulated and real datasets. This article has online supplementary materials.     

Estimation of parameters in latent class models using fuzzy clustering algorithms

A mixture approach to clustering is an important technique in cluster analysis. A mixture of multivariate multinomial distributions is usually used to analyze categorical data with latent class model. The parameter estimation is an important step for a mixture distribution. Described here are four approaches to estimating the parameters of a mixture of multivariate multinomial distributions. The first approach is an extended maximum likelihood (ML) method. The second approach is based on the well-known expectation maximization (EM) algorithm. The third approach is the classification maximum likelihood (CML) algorithm. In this paper, we propose a new approach using the so-called fuzzy class model and then create the fuzzy classification maximum likelihood (FCML) approach for categorical data. The accuracy, robustness and effectiveness of these four types of algorithms for estimating the parameters of multivariate binomial mixtures are compared using real empirical data and samples drawn from the multivariate binomial mixtures of two classes. The results show that the proposed FCML algorithm presents better accuracy, robustness and effectiveness. Overall, the FCML algorithm has the superiority over the ML, EM and CML algorithms. Thus, we recommend FCML as another good tool for estimating the parameters of mixture multivariate multinomial models.     

Implementing random scan Gibbs samplers

Summary  The Gibbs sampler, being a popular routine amongst Markov chain Monte Carlo sampling methodologies, has revolutionized the application of Monte Carlo methods in statistical computing practice. The performance of the Gibbs sampler relies heavily on the choice of sweep strategy, that is, the means by which the components or blocks of the random vector X of interest are visited and updated. We develop an automated, adaptive algorithm for implementing the optimal sweep strategy as the Gibbs sampler traverses the sample space. The decision rules through which this strategy is chosen are based on convergence properties of the induced chain and precision of statistical inferences drawn from the generated Monte Carlo samples. As part of the development, we analytically derive closed form expressions for the decision criteria of interest and present computationally feasible implementations of the adaptive random scan Gibbs sampler via a Gaussian approximation to the target distribution. We illustrate the results and algorithms presented by using the adaptive random scan Gibbs sampler developed to sample multivariate Gaussian target distributions, and screening test and image data. Research by RL and ZY supported in part by a US National Science Foundation FRG grant 0139948 and a grant from Lawrence Livermore National Laboratory, Livermore, California, USA.     

The exact likelihood for a multivariate ARMA model

A number of algorithms are presented for calculating the exact likelihood of a multivariate ARMA model. There are two aspects to the algorithms. Firstly, the parameterization is in terms of AR parameters and autocovariances. This obviates difficulties with initial MA estimates. Secondly, the algorithms explicitly account for specification of the lag structure of the multivariate time series. Additionally, an algorithm is presented to deal with missing data. The algorithms are, of themselves, not new but they have not been applied to likelihood construction in the manner discussed here.     

A characterization of multivariate normal distribution and its application

Let X₁, X₂, …, X_n be i.i.d. d-dimensional random vectors with a continuous density. Let and . In this paper we find that the distribution of Z_k (or Y_k) can be used for characterizing multivariate normal distribution. This characterization can be employed for testing multivariate normality in terms of the so-called transformation method.     

Two noniterative algorithms for computing posteriors

In this paper, we first propose a noniterative sampling method to obtain an i.i.d. sample approximately from posteriors by combining the inverse Bayes formula, sampling/importance resampling and posterior mode estimates. We then propose a new exact algorithm to compute posteriors by improving the PMDA-Exact using the sampling-wise IBF. If the posterior mode is available from the EM algorithm, then these two algorithms compute posteriors well and eliminate the convergence problem of Markov Chain Monte Carlo methods. We show good performances of our methods by some examples.     

MM Algorithms for Some Discrete Multivariate Distributions

The MM (minorization–maximization) principle is a versatile tool for constructing optimization algorithms. Every EM algorithm is an MM algorithm but not vice versa. This article derives MM algorithms for maximum likelihood estimation with discrete multivariate distributions such as the Dirichlet-multinomial and Connor–Mosimann distributions, the Neerchal–Morel distribution, the negative-multinomial distribution, certain distributions on partitions, and zero-truncated and zero-inflated distributions. These MM algorithms increase the likelihood at each iteration and reliably converge to the maximum from well-chosen initial values. Because they involve no matrix inversion, the algorithms are especially pertinent to high-dimensional problems. To illustrate the performance of the MM algorithms, we compare them to Newton’s method on data used to classify handwritten digits.     

Heuristic algorithms for a multi-period multi-stop transportation planning problem

We consider a multi-period multi-stop transportation planning problem (MPMSTP) in a one-warehouse multi-retailer distribution system where a fleet of homogeneous vehicles delivers products from a warehouse to retailers. The objective of the MPMSTP is to minimize the total transportation distance for product delivery over the planning horizon while satisfying demands of the retailers. We suggest two heuristic algorithms based on the column generation method and the simulated annealing algorithm. Computational experiments on randomly generated test problems showed that the suggested algorithms gave better solutions than an algorithm currently used in practice and algorithms modified from existing algorithms for vehicle routing problems.     

高维空间L1范数约束的一类数据稀疏距离测度学习算法与应用

现有一类分类算法通常采用经典欧氏测度描述样本间相似关系,然而欧氏测度不能较好地反映一些数据集样本的内在分布结构,从而影响这些方法对数据的描述能力.提出一种用于改善一类分类器描述性能的高维空间一类数据距离测度学习算法,与已有距离测度学习算法相比,该算法只需提供目标类数据,通过引入样本先验分布正则化项和L1范数惩罚的距离测度稀疏性约束,能有效解决高维空间小样本情况下的一类数据距离测度学习问题,并通过采用分块协调下降算法高效的解决距离测度学习的优化问题.学习的距离测度能容易的嵌入到一类分类器中,仿真实验结果表明采用学习的距离测度能有效改善一类分类器的描述性能,特别能够改善SVDD的描述能力,从而使得一类分类器具有更强的推广能力.     

Local influence analysis of multivariate probit latent variable models

The multivariate probit model is very useful for analyzing correlated multivariate dichotomous data. Recently, this model has been generalized with a confirmatory factor analysis structure for accommodating more general covariance structure, and it is called the MPCFA model. The main purpose of this paper is to consider local influence analysis, which is a well-recognized important step of data analysis beyond the maximum likelihood estimation, of the MPCFA model. As the observed-data likelihood associated with the MPCFA model is intractable, the famous Cook's approach cannot be applied to achieve local influence measures. Hence, the local influence measures are developed via Zhu and Lee's [Local influence for incomplete data model, J. Roy. Statist. Soc. Ser. B 63 (2001) 111-126.] approach that is closely related to the EM algorithm. The diagnostic measures are derived from the conformal normal curvature of an appropriate function. The building blocks are computed via a sufficiently large random sample of the latent response strengths and latent variables that are generated by the Gibbs sampler. Some useful perturbation schemes are discussed. Results that are obtained from analyses of an artificial example and a real example are presented to illustrate the newly developed methodology.