Canonical representation of conditionally specified multivariate discrete distributions

Most work on conditionally specified distributions has focused on approaches that operate on the probability space, and the constraints on the probability space often make the study of their properties challenging. We propose decomposing both the joint and conditional discrete distributions into characterizing sets of canonical interactions, and we prove that certain interactions of a joint distribution are shared with its conditional distributions. This invariance opens the door for checking the compatibility between conditional distributions involving the same set of variables. We formulate necessary and sufficient conditions for the existence and uniqueness of discrete conditional models, and we show how a joint distribution can be easily computed from the pool of interactions collected from the conditional distributions. Hence, the methods can be used to calculate the exact distribution of a Gibbs sampler. Furthermore, issues such as how near compatibility can be reconciled are also discussed. Using mixed parametrization, we show that the proposed approach is based on the canonical parameters, while the conventional approaches are based on the mean parameters. Our advantage is partly due to the invariance that holds only for the canonical parameters.     

Functional Compatibility,Markov Chains,and Gibbs Sampling with Improper Posteriors

Abstract

The members of a set of conditional probability density functions are called compatible if there exists a joint probability density function that generates them. We generalize this concept by calling the conditionals functionally compatible if there exists a non-negative function that behaves like a joint density as far as generating the conditionals according to the probability calculus, but whose integral over the whole space is not necessarily finite. A necessary and sufficient condition for functional compatibility is given that provides a method of calculating this function, if it exists. A Markov transition function is then constructed using a set of functionally compatible conditional densities and it is shown, using the compatibility results, that the associated Markov chain is positive recurrent if and only if the conditionals are compatible. A Gibbs Markov chain, constructed via “Gibbs conditionals” from a hierarchical model with an improper posterior, is a special case. Therefore, the results of this article can be used to evaluate the consequences of applying the Gibbs sampler when the posterior's impropriety is unknown to the user. Our results cannot, however, be used to detect improper posteriors. Monte Carlo approximations based on Gibbs chains are shown to have undesirable limiting behavior when the posterior is improper. The results are applied to a Bayesian hierarchical one-way random effects model with an improper posterior distribution. The model is simple, but also quite similar to some models with improper posteriors that have been used in conjunction with the Gibbs sampler in the literature.     

The Construction of Multivariate Distributions from Markov Random Fields

We address the problem of constructing and identifying a valid joint probability density function from a set of specified conditional densities. The approach taken is based on the development of relations between the joint and the conditional densities using Markov random fields (MRFs). We give a necessary and sufficient condition on the support sets of the random variables to allow these relations to be developed. This condition, which we call the Markov random field support condition, supercedes a common assumption known generally as the positivity condition. We show how these relations may be used in reverse order to construct a valid model from specification of conditional densities alone. The constructive process and the role of conditions needed for its application are illustrated with several examples, including MRFs with multiway dependence and a spatial beta process.     

Multiway Dependence in Exponential Family Conditional Distributions

Conditionally specified statistical models are frequently constructed from one-parameter exponential family conditional distributions. One way to formulate such a model is to specify the dependence structure among random variables through the use of a Markov random field (MRF). A common assumption on the Gibbsian form of the MRF model is that dependence is expressed only through pairs of random variables, which we refer to as the “pairwise-only dependence” assumption. Based on this assumption, J. Besag (1974, J. Roy. Statist. Soc. Ser. B36, 192–225) formulated exponential family “auto-models” and showed the form that one-parameter exponential family conditional densities must take in such models. We extend these results by relaxing the pairwise-only dependence assumption, and we give a necessary form that one-parameter exponential family conditional densities must take under more general conditions of multiway dependence. Data on the spatial distribution of the European corn borer larvae are fitted using a model with Bernoulli conditional distributions and several dependence structures, including pairwise-only, three-way, and four-way dependencies.     

Compatibility of discrete conditional distributions with structural zeros

A general algorithm is provided for determining the compatibility among full conditionals of discrete random variables with structural zeros. The algorithm is scalable and it can be implemented in a fairly straightforward manner. A MATLAB program is included in the Appendix and therefore, it is now feasible to check the compatibility of multi-dimensional conditional distributions with constrained supports. Rather than the linear equations in the restricted domain of Arnold et al. (2002) [11] Tian et al. (2009) [16], the approach is odds-oriented and it is a discrete adaptation of the compatibility check of Besag (1994) [17]. The method naturally leads to the calculation of a compatible joint distribution or, in the absence of compatibility, a nearly compatible joint distribution. Besag’s [5] factorization of a joint density in terms of conditional densities is used to justify the algorithm.     

Some conditional results for conditionally strong mixing sequences of random variables

The relation between strong mixing and conditionally strong mixing is answered by examples, that is, the strong mixing property of random variables does not imply the conditionally strong mixing property, and the opposite implication is also not true. Some equivalent definitions and basic properties of conditional strong mixing random variables are derived, and several conditional covariance inequalities are obtained. By means of these properties and conditional covariance inequalities, a conditional central limit theorem stated in terms of conditional characteristic functions is established, which is a conditional version of the earlier result under non-conditional case.     

Bayesian Estimation of Discrete Multivariate Latent Structure Models With Structural Zeros

In multivariate categorical data, models based on conditional independence assumptions, such as latent class models, offer efficient estimation of complex dependencies. However, Bayesian versions of latent structure models for categorical data typically do not appropriately handle impossible combinations of variables, also known as structural zeros. Allowing nonzero probability for impossible combinations results in inaccurate estimates of joint and conditional probabilities, even for feasible combinations. We present an approach for estimating posterior distributions in Bayesian latent structure models with potentially many structural zeros. The basic idea is to treat the observed data as a truncated sample from an augmented dataset, thereby allowing us to exploit the conditional independence assumptions for computational expediency. As part of the approach, we develop an algorithm for collapsing a large set of structural zero combinations into a much smaller set of disjoint marginal conditions, which speeds up computation. We apply the approach to sample from a semiparametric version of the latent class model with structural zeros in the context of a key issue faced by national statistical agencies seeking to disseminate confidential data to the public: estimating the number of records in a sample that are unique in the population on a set of publicly available categorical variables. The latent class model offers remarkably accurate estimates of population uniqueness, even in the presence of a large number of structural zeros.     

Some bivariate gamma distributions

We introduce two new bivariate gamma distributions based on a characterizing property involving products of gamma and beta random variables. We derive various representations for their joint densities, product moments, conditional densities and conditional moments. Some of these representations involve special functions such as the complementary incomplete gamma and Whittaker functions. We also discuss ways to construct multivariate generalizations.     

Conditional independence,conditional mixing and conditional association

Some properties of conditionally independent random variables are studied. Conditional versions of generalized Borel-Cantelli lemma, generalized Kolmogorov’s inequality and generalized Hájek-Rényi inequality are proved. As applications, a conditional version of the strong law of large numbers for conditionally independent random variables and a conditional version of the Kolmogorov’s strong law of large numbers for conditionally independent random variables with identical conditional distributions are obtained. The notions of conditional strong mixing and conditional association for a sequence of random variables are introduced. Some covariance inequalities and a central limit theorem for such sequences are mentioned.     

Characterizations of bivariate models using dynamic Kullback-Leibler discrimination measures

In this paper, the residual Kullback-Leibler discrimination information measure is extended to conditionally specified models. The extension is used to characterize some bivariate distributions. These distributions are also characterized in terms of proportional hazard rate models and weighted distributions. Moreover, we also obtain some bounds for this dynamic discrimination function by using the likelihood ratio order and some preceding results.     

Conditional limit theorems for conditionally linearly negative quadrant dependent random variables

From the ordinary notion of linearly negative quadrant dependence for a sequence of random variables, a new concept called conditionally linearly negative quadrant dependence is introduced. The relation between the two kinds of dependence is answered by examples, that is, the linearly negative quadrant dependence does not imply the conditionally linearly negative quadrant dependence, and vice versa. The fundamental properties of conditionally linearly negative quadrant dependence are developed, which extend the corresponding ones under the non-conditioning setup. By means of these properties, some conditional exponential inequalities, conditionally complete convergence results and a conditional central limit theorem stated in terms of conditional characteristic functions are established.     

A penalty method for numerically handling dispersive equations with incompatible initial and boundary data

This article is the numerical counterpart of a theoretical work in progress Qin and Teman, Applicable Anal (2011), 1–19, related to the approximation of evolution hyperbolic equations with incompatible data. The Korteweg‐de Vries and Schrödinger equations with incompatible initial and boundary data are considered here. For hyperbolic equations, the lack of regularity (compatibility) is known to produce large numerical errors which propagate throughout the spatial domain, destroying convergence. In this article, we numerically test the effectiveness of the penalty‐based method proposed in Qin and Teman, Applicable Anal (2011), 1–19, which replaces the hyperbolic equations with incompatible data by a system with compatible data. We observe that convergence is increased. As explained in the text, in the case of the Schrödinger equation, the impact of incompatible (nonregular) data is most severe, and the authors are not aware of any other method that can handle such severe incompatible data. © 2011 Wiley Periodicals, Inc. Numer Methods Partial Differential Eq 2011     

Bayesian Semiparametric Density Deconvolution in the Presence of Conditionally Heteroscedastic Measurement Errors

We consider the problem of estimating the density of a random variable when precise measurements on the variable are not available, but replicated proxies contaminated with measurement error are available for sufficiently many subjects. Under the assumption of additive measurement errors this reduces to a problem of deconvolution of densities. Deconvolution methods often make restrictive and unrealistic assumptions about the density of interest and the distribution of measurement errors, for example, normality and homoscedasticity and thus independence from the variable of interest. This article relaxes these assumptions and introduces novel Bayesian semiparametric methodology based on Dirichlet process mixture models for robust deconvolution of densities in the presence of conditionally heteroscedastic measurement errors. In particular, the models can adapt to asymmetry, heavy tails, and multimodality. In simulation experiments, we show that our methods vastly outperform a recent Bayesian approach based on estimating the densities via mixtures of splines. We apply our methods to data from nutritional epidemiology. Even in the special case when the measurement errors are homoscedastic, our methodology is novel and dominates other methods that have been proposed previously. Additional simulation results, instructions on getting access to the dataset and R programs implementing our methods are included as part of online supplementary materials.     

Multivariate Discrete Distributions with a Product-Type Dependence

A discrete multivariate probability distribution for dependent random variables, which contains the Poisson and Geometric conditionals distributions as particular cases, is characterized by means of conditional expectations of arbitrary one-to-one functions. Independence of the random variables is also characterized in terms of these conditional expectations. For certain exchangeable and partially exchangeable random variables with a joint distribution of this form it is shown that maximum likelihood estimates coincide with the simple method of moments estimates, suggesting that these models offer a pragmatic way to analyze certain dependent data.     

Latent class model with conditional dependency per modes to cluster categorical data

We propose a parsimonious extension of the classical latent class model to cluster categorical data by relaxing the conditional independence assumption. Under this new mixture model, named conditional modes model (CMM), variables are grouped into conditionally independent blocks. Each block follows a parsimonious multinomial distribution where the few free parameters model the probabilities of the most likely levels, while the remaining probability mass is uniformly spread over the other levels of the block. Thus, when the conditional independence assumption holds, this model defines parsimonious versions of the standard latent class model. Moreover, when this assumption is violated, the proposed model brings out the main intra-class dependencies between variables, summarizing thus each class with relatively few characteristic levels. The model selection is carried out by an hybrid MCMC algorithm that does not require preliminary parameter estimation. Then, the maximum likelihood estimation is performed via an EM algorithm only for the best model. The model properties are illustrated on simulated data and on three real data sets by using the associated R package CoModes. The results show that this model allows to reduce biases involved by the conditional independence assumption while providing meaningful parameters.     

Conditional computability of real functions with respect to a class of operators

For any class of operators which transform unary total functions in the set of natural numbers into functions of the same kind, we define what it means for a real function to be uniformly computable or conditionally computable with respect to this class. These two computability notions are natural generalizations of certain notions introduced in a previous paper co-authored by Andreas Weiermann and in another previous paper by the same authors, respectively. Under certain weak assumptions about the class in question, we show that conditional computability is preserved by substitution, that all conditionally computable real functions are locally uniformly computable, and that the ones with compact domains are uniformly computable. The introduced notions have some similarity with the uniform computability and its non-uniform extension considered by Katrin Tent and Martin Ziegler, however, there are also essential differences between the conditional computability and the non-uniform computability in question.     

Bayesian analysis of quantile regression for censored dynamic panel data

This paper develops a Bayesian approach to analyzing quantile regression models for censored dynamic panel data. We employ a likelihood-based approach using the asymmetric Laplace error distribution and introduce lagged observed responses into the conditional quantile function. We also deal with the initial conditions problem in dynamic panel data models by introducing correlated random effects into the model. For posterior inference, we propose a Gibbs sampling algorithm based on a location-scale mixture representation of the asymmetric Laplace distribution. It is shown that the mixture representation provides fully tractable conditional posterior densities and considerably simplifies existing estimation procedures for quantile regression models. In addition, we explain how the proposed Gibbs sampler can be utilized for the calculation of marginal likelihood and the modal estimation. Our approach is illustrated with real data on medical expenditures.     

Clustering Effect for Stationary Points of Discrepancy Functionals Associated with Conditionally Well-Posed Inverse Problems

In a Hilbert space we consider a class of conditionally well-posed inverse problems for which a Hölder-type estimate of conditional stability on a closed convex bounded subset holds. We investigate the Ivanov quasi-solution method and its finite-dimensional version associated with minimization of a multi-extremal discrepancy functional over a conditional stability set or over a finite-dimensional section of this set, respectively. For these optimization problems, we prove that each of their stationary points that is located not too far from the desired solution of the original inverse problem belongs to a small neighborhood of the solution. Estimates for the diameter of this neighborhood in terms of error levels in input data are also given.     

Controlling the GI/M/1 queue by conditional acceptance of customers

In this paper we consider the problem of controlling the arrival of customers into a GI/M/1 service station. It is known that when the decisions controlling the system are made only at arrival epochs, the optimal acceptance strategy is of a control-limit type, i.e., an arrival is accepted if and only if fewer than n customers are present in the system. The question is whether exercising conditional acceptance can further increase the expected long run average profit of a firm which operates the system. To reveal the relevance of conditional acceptance we consider an extension of the control-limit rule in which the nth customer is conditionally admitted to the queue. This customer may later be rejected if neither service completion nor arrival has occurred within a given time period since the last arrival epoch. We model the system as a semi-Markov decision process, and develop conditions under which such a policy is preferable to the simple control-limit rule.     

Copula-Type Estimators for Flexible Multivariate Density Modeling Using Mixtures

Copulas are popular as models for multivariate dependence because they allow the marginal densities and the joint dependence to be modeled separately. However, they usually require that the transformation from uniform marginals to the marginals of the joint dependence structure is known. This can only be done for a restricted set of copulas, for example, a normal copula. Our article introduces copula-type estimators for flexible multivariate density estimation which also allow the marginal densities to be modeled separately from the joint dependence, as in copula modeling, but overcomes the lack of flexibility of most popular copula estimators. An iterative scheme is proposed for estimating copula-type estimators and its usefulness is demonstrated through simulation and real examples. The joint dependence is modeled by mixture of normals and mixture of normal factor analyzer models, and mixture of t and mixture of t-factor analyzer models. We develop efficient variational Bayes algorithms for fitting these in which model selection is performed automatically. Based on these mixture models, we construct four classes of copula-type densities which are far more flexible than current popular copula densities, and outperform them in a simulated dataset and several real datasets. Supplementary material for this article is available online.