Generating a Random Collection of Discrete Joint Probability Distributions Subject to Partial Information

In this paper, we develop a practical and flexible methodology for generating a random collection of discrete joint probability distributions, subject to a specified information set, which can be expressed as a set of linear constraints (e.g., marginal assessments, moments, or pairwise correlations). Our approach begins with the construction of a polytope using this set of linear constraints. This polytope defines the set of all joint distributions that match the given information; we refer to this set as the “truth set.” We then implement a Monte Carlo procedure, the Hit-and-Run algorithm, to sample points uniformly from the truth set. Each sampled point is a joint distribution that matches the specified information. We provide guidelines to determine the quality of this sampled collection. The sampled points can be used to solve optimization models and to simulate systems under different uncertainty scenarios.     

Choosing joint distributions so that the variance of the sum is small

The paper considers how to choose the joint distribution of several random variables each with a given marginal distribution so that their sum has a variance as small as possible. A theorem is given that allows the solution of this and of related problems for normal random variables. Several specific applications are given. Additional results are provided for radially symmetric joint distributions of three random variables when the sum is identically zero.     

Combining marginal probability distributions via minimization of weighted sum of Kullback-Leibler divergences

This paper deals with the problem of combining marginal probability distributions as a means for aggregating pieces of expert information. A novel approach, which takes the combining problem as an analogy of statistical estimation, is proposed and discussed. The combined distribution is then searched as a minimizer of a weighted sum of Kullback-Leibler divergences of the given marginal distributions and corresponding marginals of the searched one. Necessary and sufficient conditions for a distribution to be a minimizer are stated. For discrete random variables an iterative algorithm for approximate solution of the minimization problem is proposed and its convergence is proved.     

A Characterization of Joint Distribution of Two-Valued Random Variables and Its Applications

We obtain an explicit representation for joint distribution of two-valued random variables with given marginals and for a copula corresponding to such random variables. The results are applied to prove a characterization of r-independent two-valued random variables in terms of their mixed first moments. The characterization is used to obtain an exact estimate for the number of almost independent random variables that can be defined on a discrete probability space and necessary conditions for a sequence of r-independent random variables to be stationary.     

Normal correlation coefficient of non-normal variables using piece-wise linear approximation

The correlation coefficient of non-normal variables is expressed as a function of the correlation coefficient of normal variables using piece-wise linear approximation of each univariate transform of normal to anything, and the second order moments of a multiply truncated bivariate normal distribution. For the inverse problem, an algorithm iterates this analytic function in order to assign a normal correlation coefficient to two non-normal variables. The algorithm is applied for the generation of randomized bivariate samples with given correlation coefficient and marginal distributions and used in a randomization test for bivariate nonlinearity. The test correctly does not reject the null hypothesis of linear correlation if the nonlinearity is plausible and due to the sample transform alone.     

Mixture Modeling for Longitudinal Data

In this article, we propose an unbiased estimating equation approach for a two-component mixture model with correlated response data. We adapt the mixture-of-experts model and a generalized linear model for component distribution and mixing proportion, respectively. The new approach only requires marginal distributions of both component densities and latent variables. We use serial correlations from subjects’ subgroup memberships, which improves estimation efficiency and classification accuracy, and show that estimation consistency does not depend on the choice of the working correlation matrix. The proposed estimating equation is solved by an expectation-estimating-equation (EEE) algorithm. In the E-step of the EEE algorithm, we propose a joint imputation based on the conditional linear property for the multivariate Bernoulli distribution. In addition, we establish asymptotic properties for the proposed estimators and the convergence property using the EEE algorithm. Our method is compared to an existing competitive mixture model approach in both simulation studies and an election data application. Supplementary materials for this article are available online.     

Multivariate Discrete Distributions with a Product-Type Dependence

A discrete multivariate probability distribution for dependent random variables, which contains the Poisson and Geometric conditionals distributions as particular cases, is characterized by means of conditional expectations of arbitrary one-to-one functions. Independence of the random variables is also characterized in terms of these conditional expectations. For certain exchangeable and partially exchangeable random variables with a joint distribution of this form it is shown that maximum likelihood estimates coincide with the simple method of moments estimates, suggesting that these models offer a pragmatic way to analyze certain dependent data.     

Bivariate Markov chain embeddable variables of polynomial type

The primary aim of the present article is to provide a general framework for investigating the joint distribution of run length accumulating/enumerating variables by the aid of a Markov chain embedding technique. To achieve that we introduce first a class of bivariate discrete random variables whose joint distribution can be described by the aid of a Markov chain and develop formulae for their joint probability mass function, generating functions and moments. The results are then exploited for the derivation of the distribution of a bivariate run-related statistic. Finally, some interesting uses of our results in reliability theory and educational psychology are highlighted. Research supported by General Secretary of Research and Technology of Greece under grand PENED 2001.     

Using Spherical–Radial Quadrature to Fit Generalized Linear Mixed Effects Models

Although generalized linear mixed effects models have received much attention in the statistical literature, there is still no computationally efficient algorithm for computing maximum likelihood estimates for such models when there are a moderate number of random effects. Existing algorithms are either computationally intensive or they compute estimates from an approximate likelihood. Here we propose an algorithm—the spherical–radial algorithm—that is computationally efficient and computes maximum likelihood estimates. Although we concentrate on two-level, generalized linear mixed effects models, the same algorithm can be applied to many other models as well, including nonlinear mixed effects models and frailty models. The computational difficulty for estimation in these models is in integrating the joint distribution of the data and the random effects to obtain the marginal distribution of the data. Our algorithm uses a multidimensional quadrature rule developed in earlier literature to integrate the joint density. This article discusses how this rule may be combined with an optimization algorithm to efficiently compute maximum likelihood estimates. Because of stratification and other aspects of the quadrature rule, the resulting integral estimator has significantly less variance than can be obtained through simple Monte Carlo integration. Computational efficiency is achieved, in part, because relatively few evaluations of the joint density may be required in the numerical integration.     

Two-stage stochastic linear programs with incomplete information on uncertainty

Two-stage stochastic linear programming is a classical model in operations research. The usual approach to this model requires detailed information on distribution of the random variables involved. In this paper, we only assume the availability of the first and second moments information of the random variables. By using duality of semi-infinite programming and adopting a linear decision rule, we show that a deterministic equivalence of the two-stage problem can be reformulated as a second-order cone optimization problem. Preliminary numerical experiments are presented to demonstrate the computational advantage of this approach.     

Ascending Runs in Dependent Uniformly Distributed Random Variables: Application to Wireless Networks

We analyze in this paper the longest increasing contiguous sequence or maximal ascending run of random variables with common uniform distribution but not independent. Their dependence is characterized by the fact that two successive random variables cannot take the same value. Using a Markov chain approach, we study the distribution of the maximal ascending run and we develop an algorithm to compute it. This problem comes from the analysis of several self-organizing protocols designed for large-scale wireless sensor networks, and we show how our results apply to this domain.     

Multivariate extreme value distributions for stationary Gaussian sequences

Under weak regularity conditions of the covariance sequence, it is shown that the joint limiting distribution of the maxima on each coordinate of a stationary Gaussian multivariate sequence is that of independent random variables with marginal Gumbel distributions.     

Joint cumulative distribution functions for Dempster–Shafer belief structures using copulas

We first introduce the Dempster–Shafer belief structure and highlight its role in the representation of information about a random variable for which our knowledge of the probabilities is interval-valued. We investigate the formation of the cumulative distribution function (CDF) for these types of variables. It is noted that this is also interval-valued and is expressible in terms of plausibility and belief measures. The class of aggregation operators known as copulas are introduced and a number of their properties are provided. We discuss Sklar’s theorem, which provides for the use of copulas in the formulation of joint CDFs from the marginal CDFs of classic random variables. We then look to extend these ideas to the case of joining the marginal CDFs associated with Dempster–Shafer belief structures. Finally we look at the formulation CDFs obtained from functions of multiple D–S belief structures.     

Regular multigraphs and their application to the Monte Carlo evaluation of moments of non-linear functions of Gaussian random variables

This paper expands on the multigraph method for expressing moments of non-linear functions of Gaussian random variables. In particular, it includes a list of regular multigraphs that is needed for the computation of some of these moments. The multigraph method is then used to evaluate numerically the moments of non-Gaussian self-similar processes. These self-similar processes are of interest in various applications and the numerical value of their marginal moments yield qualitative information about the behavior of the probability tails of their marginal distributions.     

Simulating Multivariate Extreme Value Distributions of Logistic Type

Methods are given for simulating from symmetric and asymmetric versions of the multivariate logistic distribution, and from other multivariate extreme value distributions based on the well known logistic model. We consider two general approaches. The first approach uses transformations to derive random variables with a joint distribution function from which it is easy to simulate. The second approach derives from a specification of conditionally independent marginal components, conditioning on positive stable random variables. This specification extends to models of nested or hierarchical type and leads to an efficient way of incorporating marginal censoring. The algorithms presented in Sections 2 and 3 are available on request from the author. They are also included in the R (Ihaka and Gentleman, 1996) package evd (Stephenson, 2002), which is available from http://www.maths.lancs.ac.uk/~stephena/.     

A practical non-parametric copula algorithm for system reliability with correlations

System reliability analysis involving correlated random variables is challenging because the failure probability cannot be uniquely determined under the given probability information. This paper proposes a system reliability evaluation method based on non-parametric copulas. The approximated joint probability distribution satisfying the constraints specified by correlations has the maximal relative entropy with respect to the joint probability distribution of independent random variables. Thus the reliability evaluation is unbiased from the perspective of information theory. The estimation of the non-parametric copula parameters from Pearson linear correlation, Spearman rank correlation, and Kendall rank correlation are provided, respectively. The approximated maximum entropy distribution is then integrated with the first and second order system reliability method. Four examples are adopted to illustrate the accuracy and efficiency of the proposed method. It is found that traditional system reliability method encodes excessive dependence information for correlated random variables and the estimated failure probability can be significantly biased.     

A general estimate in the invariance principle

We obtain estimates for the accuracy with which a random broken line constructed from sums of independent nonidentically distributed random variables can be approximated by a Wiener process. All estimates depend explicitly on the moments of the random variables; meanwhile, these moments can be of a rather general form. In the case of identically distributed random variables we succeed for the first time in constructing an estimate depending explicitly on the common distribution of the summands and directly implying all results of the famous articles by Komlós, Major, and Tusnády which are devoted to estimates in the invariance principle.     

Joint Distributions for Movements of Elements in Sattolo's and the Fisher-Yates Algorithm

Sattolo's algorithm creates a random cyclic permutation by interchanging pairs of elements in an appropriate manner; the Fisher-Yates algorithm produces random (not necessarily cyclic) permutations in a very similar way. The distributions of the movements of the elements in these two algorithms have already been treated quite extensively in past works. In this paper, we are interested in the joint distribution of two elements j and k; we are able to compute the bivariate generating functions explicitly, although it is quite involved. From it, moments and limiting distributions can be deduced. Furthermore, we compute the probability that elements i and j ever change places in both algorithms.     

有限域上随机变量联合分布及二阶矩的分解与应用

本文给出了有限域上随机变量联合概率和二阶矩的分解公式，给出了有限域上随机变量相互独立的谱刻划，应用上述结果，建立了在进行频次分析时，对有限域上随机向量构造的Χ平方统计量与该随机向量坐标函数的非零线性组合的Χ平方统计量之间的内在联系，给出了有限域上相关免疫函数谱特征的新证明，建立了有限域上多输出函数的差分分布与其广义Chrestenson循环谱之间的内在联系，建立了多输出函数的平衡性其差分分布之间的内在联系。     

Statistical dependence through common risk factors: With applications in uncertainty analysis

A model for building statistical dependence between marginal distribution with bounded support is discussed. The model is geared towards elicitation of dependence parameters through expert judgment. The resulting joint distribution may be useful in uncertainty analyses where dependence between random variables with a bounded support is present due to common risk factors, such as, e.g., in the classical Project Evaluation and Review Technique.