Bayesian Synthetic Likelihood

Having the ability to work with complex models can be highly beneficial. However, complex models often have intractable likelihoods, so methods that involve evaluation of the likelihood function are infeasible. In these situations, the benefits of working with likelihood-free methods become apparent. Likelihood-free methods, such as parametric Bayesian indirect likelihood that uses the likelihood of an alternative parametric auxiliary model, have been explored throughout the literature as a viable alternative when the model of interest is complex. One of these methods is called the synthetic likelihood (SL), which uses a multivariate normal approximation of the distribution of a set of summary statistics. This article explores the accuracy and computational efficiency of the Bayesian version of the synthetic likelihood (BSL) approach in comparison to a competitor known as approximate Bayesian computation (ABC) and its sensitivity to its tuning parameters and assumptions. We relate BSL to pseudo-marginal methods and propose to use an alternative SL that uses an unbiased estimator of the SL, when the summary statistics have a multivariate normal distribution. Several applications of varying complexity are considered to illustrate the findings of this article. Supplemental materials are available online. Computer code for implementing the methods on all examples is available at https://github.com/cdrovandi/Bayesian-Synthetic-Likelihood.     

Hypothesis testing for the generalized multivariate modified Bessel model

In this paper, we consider hypothesis testing problems in which the involved samples are drawn from generalized multivariate modified Bessel populations. This is a much more general distribution that includes both the multivariate normal and multivariate-t distributions as special cases. We derive the distribution of the Hotelling's T²-statistic for both the one- and two-sample problems, as well as the distribution of the Scheffe's T²-statistic for the Behrens–Fisher problem. In all cases, the non-null distribution of the corresponding F-statistic follows a new distribution which we introduce as the non-central F-Bessel distribution. Some statistical properties of this distribution are studied. Furthermore, this distribution was utilized to perform some power calculations for tests of means for different models which are special cases of the generalized multivariate modified Bessel distribution, and the results compared with those obtained under the multivariate normal case. Under the null hypothesis, however, the non-central F-Bessel distribution reduces to the central F-distribution obtained under the classical normal model.     

Comparison of Methods for the Computation of Multivariate t Probabilities

This article compares methods for the numerical computation of multivariate t probabilities for hyper-rectangular integration regions. Methods based on acceptance-rejection, spherical-radial transformations, and separation-of-variables transformations are considered. Tests using randomly chosen problems show that the most efficient numerical methods use a transformation developed by Genz for multivariate normal probabilities. These methods allow moderately accurate multivariate t probabilities to be quickly computed for problems with as many as 20 variables. Methods for the noncentral multivariate t distribution are also described.     

Multivariate Gaussian Simulation Outside Arbitrary Ellipsoids

Methods for simulation from multivariate Gaussian distributions restricted to be from outside an arbitrary ellipsoidal region are often needed in applications. A standard rejection algorithm that draws a sample from a multivariate Gaussian distribution and accepts it if it is outside the ellipsoid is often employed; however, this is computationally inefficient if the probability of that ellipsoid under the multivariate normal distribution is substantial. We provide a two-stage rejection sampling scheme for drawing samples from such a truncated distribution. Experiments show that the added complexity of the two-stage approach results in the standard algorithm being more efficient for small ellipsoids (i.e., with small rejection probability). However, as the size of the ellipsoid increases, the efficiency of the two-stage approach relative to the standard algorithm increases indefinitely. The relative efficiency also increases as the number of dimensions increases, as the centers of the ellipsoid and the multivariate Gaussian distribution come closer, and as the shape of the ellipsoid becomes more spherical. We provide results of simulation experiments conducted to quantify the relative efficiency over a range of parameter settings.     

Regression models for multivariate ordered responses via the Plackett distribution

We investigate the properties of a class of discrete multivariate distributions whose univariate marginals have ordered categories, all the bivariate marginals, like in the Plackett distribution, have log-odds ratios which do not depend on cut points and all higher-order interactions are constrained to 0. We show that this class of distributions may be interpreted as a discretized version of a multivariate continuous distribution having univariate logistic marginals. Convenient features of this class relative to the class of ordered probit models (the discretized version of the multivariate normal) are highlighted. Relevant properties of this distribution like quadratic log-linear expansion, invariance to collapsing of adjacent categories, properties related to positive dependence, marginalization and conditioning are discussed briefly. When continuous explanatory variables are available, regression models may be fitted to relate the univariate logits (as in a proportional odds model) and the log-odds ratios to covariates.     

Maximum-likelihood estimation of the parameters of a multivariate normal distribution

This paper provides an exposition of alternative approaches for obtaining maximum- likelihood estimators (MLE) for the parameters of a multivariate normal distribution under different assumptions about the parameters. A central focus is on two general techniques, namely, matrix differentiation and matrix transformations. These are systematically applied to derive the MLE of the means under a rank constraint and of the covariances when there are missing observations. Derivations using induction and inequalities are also included to illustrate alternative methods. Other examples, such as a connection with an econometric model, are included. Although the paper is primarily expository, some of the proofs are new.     

Finite-sample inference with monotone incomplete multivariate normal data, I

We consider problems in finite-sample inference with two-step, monotone incomplete data drawn from , a multivariate normal population with mean and covariance matrix . We derive a stochastic representation for the exact distribution of , the maximum likelihood estimator of . We obtain ellipsoidal confidence regions for through T², a generalization of Hotelling’s statistic. We derive the asymptotic distribution of, and probability inequalities for, T² under various assumptions on the sizes of the complete and incomplete samples. Further, we establish an upper bound for the supremum distance between the probability density functions of and , a normal approximation to .     

Model testing for multivariate censored data

Summary. In the fields like Astronomy and Ecology, the need for proper statistical analysis of data that are censored is being increasingly recognized. Such data occur when, due to noise or other factors, instruments fail to detect low luminosities of celestial objects, or low concentrations of certain pollutants. For multivariate censored data sets there are very few distribution free methods available and researchers in the various fields often impose an assumption on the joint distribution, such as multivariate normality, and carry out parametric inferences. Under censoring, however, such parametric inferences are asymptotically wrong if the imposed assumption is incorrect. In this paper we propose a class of goodness-of-fit procedures for testing assumptions about the multivariate distribution under random censoring. The test procedures generalize Pearson's goodness-of-fit test in the sense that they are based on the concept of observed-minus-expected frequencies. The theory of the test statistic, however, differs from that for the classical Pearson test due to the accommodation of censored data. Received: 24 May 1994 / In revised form: 3 March 1996     

Convergence of consistent and inconsistent finite difference schemes and an acceleration technique

This paper states and generalizes in part some recent results on finite difference methods for Dirichlet problems in a bounded domain Ω which the author has obtained by himself or with coworkers. After stating a superconvergence property of finite difference solution for the case where the exact solution u belongs to , it is remarked that such a property does not hold in general if . Next, a convergence theorem is given for inconsistent schemes under some assumptions. Furthermore, it is shown that the accuracy of the approximate solution can be improved by a coordinate transformation. Numerical examples are also given.     

An accept-reject algorithm for the positive multivariate normal distribution

The need to simulate from a positive multivariate normal distribution arises in several settings, specifically in Bayesian analysis. A variety of algorithms can be used to sample from this distribution, but most of these algorithms involve Gibbs sampling. Since the sample is generated from a Markov chain, the user has to account for the fact that sequential draws in the sample depend on one another and that the sample generated only follows a positive multivariate normal distribution asymptotically. The user would not have to account for such issues if the sample generated was i.i.d. In this paper, an accept-reject algorithm is introduced in which variates from a positive multivariate normal distribution are proposed from a multivariate skew-normal distribution. This new algorithm generates an i.i.d. sample and is shown, under certain conditions, to be very efficient.     

Multivariate Polynomial Minimization and Its Application in Signal Processing

We make a conjecture that the number of isolated local minimum points of a 2n-degree or (2n+1)-degree r-variable polynomial is not greater than n ^r when n 2. We show that this conjecture is the minimal estimate, and is true in several cases. In particular, we show that a cubic polynomial of r variables may have at most one local minimum point though it may have 2^r critical points. We then study the global minimization problem of an even-degree multivariate polynomial whose leading order coefficient tensor is positive definite. We call such a multivariate polynomial a normal multivariate polynomial. By giving a one-variable polynomial majored below a normal multivariate polynomial, we show the existence of a global minimum of a normal multivariate polynomial, and give an upper bound of the norm of the global minimum and a lower bound of the global minimization value. We show that the quartic multivariate polynomial arising from broad-band antenna array signal processing, is a normal polynomial, and give a computable upper bound of the norm of the global minimum and a computable lower bound of the global minimization value of this normal quartic multivariate polynomial. We give some sufficient and necessary conditions for an even order tensor to be positive definite. Several challenging questions remain open.     

Asymptotic distributions of robust shape matrices and scales

It has been frequently observed in the literature that many multivariate statistical methods require the covariance or dispersion matrix Σ of an elliptical distribution only up to some scaling constant. If the topic of interest is not the scale but only the shape of the elliptical distribution, it is not meaningful to focus on the asymptotic distribution of an estimator for Σ or another matrix Γ∝Σ. In the present work, robust estimators for the shape matrix and the associated scale are investigated. Explicit expressions for their joint asymptotic distributions are derived. It turns out that if the joint asymptotic distribution is normal, the estimators presented are asymptotically independent for one and only one specific choice of the scale function. If it is non-normal (this holds for example if the estimators for the shape matrix and scale are based on the minimum volume ellipsoid estimator) only the scale function presented leads to asymptotically uncorrelated estimators. This is a generalization of a result obtained by Paindaveine [D. Paindaveine, A canonical definition of shape, Statistics and Probability Letters 78 (2008) 2240-2247] in the context of local asymptotic normality theory.     

On the Relation between Piecewise Polynomial and Rational Approximation in <Emphasis Type="Italic">L</Emphasis><Subscript><Emphasis Type="Italic">p</Emphasis></Subscript>(<Emphasis Type="Bold">R</Emphasis><Superscript>2</Superscript>)

In the univariate case there are certain equivalences between the nonlinear approximation methods that use piecewise polynomials and those that use rational functions. It is known that for certain parameters the respective approximation spaces are identical and may be described as Besov spaces. The characterization of the approximation spaces of the multivariate nonlinear approximation by piecewise polynomials and by rational functions is not known. In this work we compare between the two methods in the bivariate case. We show some relations between the approximation spaces of piecewise polynomials defined on n triangles and those of bivariate rational functions of total degree n which are described by n parameters. Thus we compare two classes of approximants with the same number Cn of parameters. We consider this the proper comparison between the two methods.     

Iteration methods for convexly constrained ill-posed problems in hilbert space

Minimization problems in Hilbert space with quadratic objective function and closed convex constraint set C are considered. In case the minimum is not unique we are looking for the solution of minimal norm. If a problem is ill-posed, i.e. if the solution does not depend continuously on the data, and if the data are subject to errors then it has to be solved by means of regularization methods. The regularizing properties of some gradient projection methods—i.e. convergence for exact data, order of convergence under additional assumptions on the solution and stability for perturbed data—are the main issues of this paper.     

On generating multivariate Poisson data in management science applications

Generating multivariate Poisson random variables is essential in many applications, such as multi echelon supply chain systems, multi‐item/multi‐period pricing models, accident monitoring systems, etc. Current simulation methods suffer from limitations ranging from computational complexity to restrictions on the structure of the correlation matrix, and therefore are rarely used in management science. Instead, multivariate Poisson data are commonly approximated by either univariate Poisson or multivariate Normal data. However, these approximations are often not adequate in practice. In this paper, we propose a conceptually appealing correction for NORTA (NORmal To Anything) for generating multivariate Poisson data with a flexible correlation structure and rates. NORTA is based on simulating data from a multivariate Normal distribution and converting it into an arbitrary continuous distribution with a specific correlation matrix. We show that our method is both highly accurate and computationally efficient. We also show the managerial advantages of generating multivariate Poisson data over univariate Poisson or multivariate Normal data. Copyright © 2011 John Wiley & Sons, Ltd.     

Normal Linear Regression Models With Recursive Graphical Markov Structure

A multivariate normal statistical model defined by the Markov properties determined by an acyclic digraph admits a recursive factorization of its likelihood function (LF) into the product of conditional LFs, each factor having the form of a classical multivariate linear regression model (≡WMANOVA model). Here these models are extended in a natural way to normal linear regression models whose LFs continue to admit such recursive factorizations, from which maximum likelihood estimators and likelihood ratio (LR) test statistics can be derived by classical linear methods. The central distribution of the LR test statistic for testing one such multivariate normal linear regression model against another is derived, and the relation of these regression models to block-recursive normal linear systems is established. It is shown how a collection of nonnested dependent normal linear regression models (≡Wseemingly unrelated regressions) can be combined into a single multivariate normal linear regression model by imposing a parsimonious set of graphical Markov (≡Wconditional independence) restrictions.     

Asymptotic conditional distribution of exceedance counts: fragility index with different margins

We consider a random vector X, whose components are neither necessarily independent nor identically distributed. The fragility index (FI), if it exists, is defined as the limit of the expected number of exceedances among the components of X above a high threshold, given that there is at least one exceedance. It measures the asymptotic stability of the system of components. The system is called stable if the FI is one and fragile otherwise. In this paper, we show that the asymptotic conditional distribution of exceedance counts exists, if the copula of X is in the domain of attraction of a multivariate extreme value distribution, and if the marginal distribution functions satisfy an appropriate tail condition. This enables the computation of the FI corresponding to X and of the extended FI as well as of the asymptotic distribution of the exceedance cluster length also in that case, where the components of X are not identically distributed.     

Effects of Smoothing on Multivariate Statistical Properties of the Normal to a Rough Curve or Surface

Much of the practical interest attached to curves and surfaces derives from features of roughness, rather than smoothness. For example, considerable attention has been paid to fractal models of curves and surfaces, for which the notions of a normal and curvature are usually not well defined. Nevertheless, these quantities are sometimes measurable, because the device for recording a rough surface (such as a stylus or “compass”) adds its own intrinsic smoothness. In this paper we address the effect of such smoothing operations on the multivariate statistical properties of a normal to the surface. Particular attention is paid to the validity of commonly assumed unimodal approximations to the distribution of the normal. It is shown that the actual distribution may have more than one mode, although in a range of situations the unimodal approximation is valid.     

Two-sample inference for normal mean vectors based on monotone missing data

Inferential procedures for the difference between two multivariate normal mean vectors based on incomplete data matrices with different monotone patterns are developed. Assuming that the population covariance matrices are equal, a pivotal quantity, similar to the Hotelling T² statistic, is proposed, and its approximate distribution is derived. Hypothesis testing and confidence estimation of the difference between the mean vectors based on the approximate distribution are outlined. The validity of the approximation is investigated using Monte Carlo simulation. Monte Carlo studies indicate that the approximate method is very satisfactory even for small samples. A multiple comparison procedure is outlined and the proposed methods are illustrated using an example.     

Moment properties of multivariate infinitely divisible laws and criteria for multivariate self-decomposability

Ramachandran (1969) [9, Theorem 8] has shown that for any univariate infinitely divisible distribution and any positive real number α, an absolute moment of order α relative to the distribution exists (as a finite number) if and only if this is so for a certain truncated version of the corresponding Lévy measure. A generalized version of this result in the case of multivariate infinitely divisible distributions, involving the concept of g-moments, was given by Sato (1999) [6, Theorem 25.3]. We extend Ramachandran’s theorem to the multivariate case, keeping in mind the immediate requirements under appropriate assumptions of cumulant studies of the distributions referred to; the format of Sato’s theorem just referred to obviously varies from ours and seems to have a different agenda. Also, appealing to a further criterion based on the Lévy measure, we identify in a certain class of multivariate infinitely divisible distributions the distributions that are self-decomposable; this throws new light on structural aspects of certain multivariate distributions such as the multivariate generalized hyperbolic distributions studied by Barndorff-Nielsen (1977) [12] and others. Various points relevant to the study are also addressed through specific examples.