On Bernoulli decompositions for random variables, concentration bounds, and spectral localization

As was noted already by A. N. Kolmogorov, any random variable has a Bernoulli component. This observation provides a tool for the extension of results which are known for Bernoulli random variables to arbitrary distributions. Two applications are provided here: (i) an anti-concentration bound for a class of functions of independent random variables, where probabilistic bounds are extracted from combinatorial results, and (ii) a proof, based on the Bernoulli case, of spectral localization for random Schrödinger operators with arbitrary probability distributions for the single site coupling constants. For a general random variable, the Bernoulli component may be defined so that its conditional variance is uniformly positive. The natural maximization problem is an optimal transport question which is also addressed here.     

Comparison results for exchangeable credit risk portfolios

This paper is dedicated to risk analysis of credit portfolios. Assuming that default indicators form an exchangeable sequence of Bernoulli random variables and as a consequence of de Finetti’s theorem, default indicators are Binomial mixtures. We can characterize the supermodular order between two exchangeable Bernoulli random vectors in terms of the convex ordering of their corresponding mixture distributions. Thus we can proceed to some comparisons between stop-loss premiums, CDO tranche premiums and convex risk measures on aggregate losses. This methodology provides a unified analysis of dependence for a number of CDO pricing models based on factor copulas, multivariate Poisson and structural approaches.     

Mixture Modeling for Longitudinal Data

In this article, we propose an unbiased estimating equation approach for a two-component mixture model with correlated response data. We adapt the mixture-of-experts model and a generalized linear model for component distribution and mixing proportion, respectively. The new approach only requires marginal distributions of both component densities and latent variables. We use serial correlations from subjects’ subgroup memberships, which improves estimation efficiency and classification accuracy, and show that estimation consistency does not depend on the choice of the working correlation matrix. The proposed estimating equation is solved by an expectation-estimating-equation (EEE) algorithm. In the E-step of the EEE algorithm, we propose a joint imputation based on the conditional linear property for the multivariate Bernoulli distribution. In addition, we establish asymptotic properties for the proposed estimators and the convergence property using the EEE algorithm. Our method is compared to an existing competitive mixture model approach in both simulation studies and an election data application. Supplementary materials for this article are available online.     

The multivariate Behrens-Fisher distribution

The main purpose of this paper is the study of the multivariate Behrens-Fisher distribution. It is defined as the convolution of two independent multivariate Student t distributions. Some representations of this distribution as the mixture of known distributions are shown. An important result presented in the paper is the elliptical condition of this distribution in the special case of proportional scale matrices of the Student t distributions in the defining convolution. For the bivariate Behrens-Fisher problem, the authors propose a non-informative prior distribution leading to highest posterior density (H.P.D.) regions for the difference of the mean vectors whose coverage probability matches the frequentist coverage probability more accurately than that obtained using the independence-Jeffreys prior distribution, even with small samples.     

On a Class of Shrinkage Priors for Covariance Matrix Estimation

We propose a flexible class of models based on scale mixture of uniform distributions to construct shrinkage priors for covariance matrix estimation. This new class of priors enjoys a number of advantages over the traditional scale mixture of normal priors, including its simplicity and flexibility in characterizing the prior density. We also exhibit a simple, easy to implement Gibbs sampler for posterior simulation, which leads to efficient estimation in high-dimensional problems. We first discuss the theory and computational details of this new approach and then extend the basic model to a new class of multivariate conditional autoregressive models for analyzing multivariate areal data. The proposed spatial model flexibly characterizes both the spatial and the outcome correlation structures at an appealing computational cost. Examples consisting of both synthetic and real-world data show the utility of this new framework in terms of robust estimation as well as improved predictive performance. Supplementary materials are available online.     

Improving density forecast by modeling asymmetric features: An application to S&P500 returns

A density forecast is an estimate of the probability distribution of the possible future values of a random variable. From the current literature, an economic time series may have three types of asymmetry: asymmetry in unconditional distribution, asymmetry in conditional distribution, volatility asymmetry. In this paper, we propose three density forecasting methods under two-piece normal assumption to capture these asymmetric features. A GARCH model with two-piece normal distribution is developed to capture asymmetries in the conditional distributions. In this approach, we first estimate parameters of a GARCH model by assuming normal innovations, and then fit a two-piece normal distribution to the empirical residuals. Block bootstrap procedure, and moving average method with two-piece normal distribution are presented for volatility asymmetry and asymmetry in the conditional distributions. Application of the developed methods to the weekly S&P500 returns illustrates that forecast quality can be significantly improved by modeling these asymmetric features.     

Multivariate analysis of variance with fewer observations than the dimension

In this article, we consider the problem of testing a linear hypothesis in a multivariate linear regression model which includes the case of testing the equality of mean vectors of several multivariate normal populations with common covariance matrix Σ, the so-called multivariate analysis of variance or MANOVA problem. However, we have fewer observations than the dimension of the random vectors. Two tests are proposed and their asymptotic distributions under the hypothesis as well as under the alternatives are given under some mild conditions. A theoretical comparison of these powers is made.     

A test for the mean vector with fewer observations than the dimension

In this paper, we consider a test for the mean vector of independent and identically distributed multivariate normal random vectors where the dimension p is larger than or equal to the number of observations N. This test is invariant under scalar transformations of each component of the random vector. Theories and simulation results show that the proposed test is superior to other two tests available in the literature. Interest in such significance test for high-dimensional data is motivated by DNA microarrays. However, the methodology is valid for any application which involves high-dimensional data.     

Nonparametric Bayesian Modeling for Multivariate Ordinal Data

This article proposes a probability model for k-dimensional ordinal outcomes, that is, it considers inference for data recorded in k-dimensional contingency tables with ordinal factors. The proposed approach is based on full posterior inference, assuming a flexible underlying prior probability model for the contingency table cell probabilities. We use a variation of the traditional multivariate probit model, with latent scores that determine the observed data. In our model, a mixture of normals prior replaces the usual single multivariate normal model for the latent variables. By augmenting the prior model to a mixture of normals we generalize inference in two important ways. First, we allow for varying local dependence structure across the contingency table. Second, inference in ordinal multivariate probit models is plagued by problems related to the choice and resampling of cutoffs defined for these latent variables. We show how the proposed mixture model approach entirely removes these problems. We illustrate the methodology with two examples, one simulated dataset and one dataset of interrater agreement.     

On Pearson-Kotz Dirichlet distributions

In this paper, we discuss some basic distributional and asymptotic properties of the Pearson-Kotz Dirichlet multivariate distributions. These distributions, which appear as the limit of conditional Dirichlet random vectors, possess many appealing properties and are interesting from theoretical as well as applied points of view. We illustrate an application concerning the approximation of the joint conditional excess distribution of elliptically symmetric random vectors.     

Dependence structures of multivariate Bernoulli random vectors

In some situations, it is difficult and tedious to check notions of dependence properties and dependence orders for multivariate distributions supported on a finite lattice. The purpose of this paper is to utilize a newly developed tool, majorization with respect to weighted trees, to lay out some general results that can be used to identify some dependence properties and dependence orders for multivariate Bernoulli random vectors. Such a study gives us some new insight into the relations between the concepts of dependence.     

A new data envelopment analysis model with fuzzy random inputs and outputs

This paper first presents several formulas for mean chance distributions of triangular fuzzy random variables and their functions, then develops a new class of fuzzy random data envelopment analysis (FRDEA) models with mean chance constraints, in which the inputs and outputs are assumed to be characterized by fuzzy random variables with known possibility and probability distributions. According to the established formulas for the mean chance distributions, we can turn the mean chance constraints into their equivalent stochastic ones. On the other hand, since the objective in the FRDEA model is the expectation about the ratio of the weighted sum of outputs and the weighted sum of inputs for a target decision-making unite (DMU), for general fuzzy random inputs and outputs, we suggest an approximation method to evaluate the objective; and for triangular fuzzy random inputs and outputs, we propose a method to reduce the objective to its equivalent stochastic one. As a consequence, under the assumption that the inputs and the outputs are triangular fuzzy random vectors, the proposed FRDEA model can be reduced to its equivalent stochastic programming one, in which the constraints contain the standard normal distribution function, and the objective is the expectation for a function of the normal random variable. To solve the equivalent stochastic programming model, we design a hybrid algorithm by integrating stochastic simulation and genetic algorithm (GA). Finally, one numerical example is presented to demonstrate the proposed FRDEA modeling idea and the effectiveness of the designed hybrid algorithm.     

Multivariate Discrete Distributions with a Product-Type Dependence

A discrete multivariate probability distribution for dependent random variables, which contains the Poisson and Geometric conditionals distributions as particular cases, is characterized by means of conditional expectations of arbitrary one-to-one functions. Independence of the random variables is also characterized in terms of these conditional expectations. For certain exchangeable and partially exchangeable random variables with a joint distribution of this form it is shown that maximum likelihood estimates coincide with the simple method of moments estimates, suggesting that these models offer a pragmatic way to analyze certain dependent data.     

New multivariate orderings based on conditional distributions

In this paper, we propose and study new multivariate extensions of the dispersive, right‐spread, decreasing mean residual life and new better than used in expectation univariate orders. These new orders are based on the comparison of univariate marginal distributions conditional on survival data for the rest of the components. Relationships among multivariate orders and applications to some multivariate random vectors are also provided. Copyright © 2011 John Wiley & Sons, Ltd.     

Optimizing random scan Gibbs samplers

The Gibbs sampler is a popular Markov chain Monte Carlo routine for generating random variates from distributions otherwise difficult to sample. A number of implementations are available for running a Gibbs sampler varying in the order through which the full conditional distributions used by the Gibbs sampler are cycled or visited. A common, and in fact the original, implementation is the random scan strategy, whereby the full conditional distributions are updated in a randomly selected order each iteration. In this paper, we introduce a random scan Gibbs sampler which adaptively updates the selection probabilities or “learns” from all previous random variates generated during the Gibbs sampling. In the process, we outline a number of variations on the random scan Gibbs sampler which allows the practitioner many choices for setting the selection probabilities and prove convergence of the induced (Markov) chain to the stationary distribution of interest. Though we emphasize flexibility in user choice and specification of these random scan algorithms, we present a minimax random scan which determines the selection probabilities through decision theoretic considerations on the precision of estimators of interest. We illustrate and apply the results presented by using the adaptive random scan Gibbs sampler developed to sample from multivariate Gaussian target distributions, to automate samplers for posterior simulation under Dirichlet process mixture models, and to fit mixtures of distributions.     

A new class of bivariate distributions and its mixture

A new class of bivariate distributions is presented in this paper. The procedure used in this paper is based on a latent random variable with exponential distribution. The model introduced here is of Marshall-Olkin type. A mixture of the proposed bivariate distributions is also discussed. The results obtained here generalize those of the bivariate exponential distribution present in the literature.     

Estimation and control for linear,partially observable systems with non-gaussian initial distribution

The nonlinear filtering problem of estimating the state of a linear stochastic system from noisy observations is solved for a broad class of probability distributions of the initial state. It is shown that the conditional density of the present state, given the past observations, is a mixture of Gaussian distributions, and is parametrically determined by two sets of sufficient statistics which satisfy stochastic DEs; this result leads to a generalization of the Kalman–Bucy filter to a structure with a conditional mean vector, and additional sufficient statistics that obey nonlinear equations, and determine a generalized (random) Kalman gain. The theory is used to solve explicitly a control problem with quadratic running and terminal costs, and bounded controls.     

Runs,scans and URN model distributions: A unified Markov chain approach

This paper presents a unified approach for the study of the exact distribution (probability mass function, mean, generating functions) of three types of random variables: (a) variables related to success runs in a sequence of Bernoulli trials (b) scan statistics, i.e. variables enumerating the moving windows in a linearly ordered sequence of binary outcomes (success or failure) which contain prescribed number of successes and (c) success run statistics related to several well known urn models. Our approach is based on a Markov chain imbedding which permits the construction of probability vectors satisfying triangular recurrence relations. The results presented here cover not only the case of identical and independently distributed Bernoulli variables, but the non-identical case as well. An extension to models exhibiting Markov dependence among the successive trials is also discussed in brief.     

Bayesian Estimation of Discrete Multivariate Latent Structure Models With Structural Zeros

In multivariate categorical data, models based on conditional independence assumptions, such as latent class models, offer efficient estimation of complex dependencies. However, Bayesian versions of latent structure models for categorical data typically do not appropriately handle impossible combinations of variables, also known as structural zeros. Allowing nonzero probability for impossible combinations results in inaccurate estimates of joint and conditional probabilities, even for feasible combinations. We present an approach for estimating posterior distributions in Bayesian latent structure models with potentially many structural zeros. The basic idea is to treat the observed data as a truncated sample from an augmented dataset, thereby allowing us to exploit the conditional independence assumptions for computational expediency. As part of the approach, we develop an algorithm for collapsing a large set of structural zero combinations into a much smaller set of disjoint marginal conditions, which speeds up computation. We apply the approach to sample from a semiparametric version of the latent class model with structural zeros in the context of a key issue faced by national statistical agencies seeking to disseminate confidential data to the public: estimating the number of records in a sample that are unique in the population on a set of publicly available categorical variables. The latent class model offers remarkably accurate estimates of population uniqueness, even in the presence of a large number of structural zeros.     

Simulation of certain multivariate generalized Pareto distributions

The investigation of multivariate generalized Pareto distributions (GPDs) has begun only recently. For further progress with these distributions simulation methods are an important part. We describe several methods of simulating GPDs, beginning with an efficient method for the logistic GPD. The algorithm is based on the Shi transformation, which was already used for the simulation of multivariate extreme value distributions (EVDs) of logistic type. In the sequel another algorithm is presented simulating a broader class of GPDs. Due to its numerical complexity it is only practicably applicable in low dimensions. A method is given to generate unconditional GPD random vectors from conditionally GPD distributed random vectors. A short application of the simulation methods in the analysis of a real hydrological data set concludes the article. The simulation algorithms are available on the author’s home page .