Inference for the Number of Topics in the Latent Dirichlet Allocation Model via Bayesian Mixture Modeling

In latent Dirichlet allocation, the number of topics, T, is a hyperparameter of the model that must be specified before one can fit the model. The need to specify T in advance is restrictive. One way of dealing with this problem is to put a prior on T, but unfortunately the distribution on the latent variables of the model is then a mixture of distributions on spaces of different dimensions, and estimating this mixture distribution by Markov chain Monte Carlo is very difficult. We present a variant of the Metropolis–Hastings algorithm that can be used to estimate this mixture distribution, and in particular the posterior distribution of the number of topics. We evaluate our methodology on synthetic data and compare it with procedures that are currently used in the machine learning literature. We also give an illustration on two collections of articles from Wikipedia. Supplemental materials for this article are available online.     

Bayesian Inference in the Noncentral Student-t Model

This article takes up Bayesian inference in linear models with disturbances from a noncentral Student-t distribution. The distribution is useful when both long tails and asymmetry are features of the data. The distribution can be expressed as a location-scale mixture of normals with inverse weights distributed according to a chi-square distribution. The computations are performed using Gibbs sampling with data augmentation. An empirical application to Standard and Poor's stock returns indicates that posterior odds strongly favor a noncentral Student-t specification over its symmetric counterpart.     

An Algorithm for Robust Inference for the Cox Model with Frailties

Abstract

This article proposes a robust method of statistical inference for the Cox's proportional hazards model with frailties. We use the Metropolis—Hastings algorithm and the bootstrap method. We present a computationally efficient algorithm with a customized data structure to implement this method and demonstrate this technique with real data.     

Likelihood Inference for Large Scale Stochastic Blockmodels With Covariates Based on a Divide-and-Conquer Parallelizable Algorithm With Communication

We consider a stochastic blockmodel equipped with node covariate information, that is, helpful in analyzing social network data. The key objective is to obtain maximum likelihood estimates of the model parameters. For this task, we devise a fast, scalable Monte Carlo EM type algorithm based on case-control approximation of the log-likelihood coupled with a subsampling approach. A key feature of the proposed algorithm is its parallelizability, by processing portions of the data on several cores, while leveraging communication of key statistics across the cores during each iteration of the algorithm. The performance of the algorithm is evaluated on synthetic datasets and compared with competing methods for blockmodel parameter estimation. We also illustrate the model on data from a Facebook derived social network enhanced with node covariate information. Supplemental materials for this article are available online.     

Bayesian Inference on a Mixture Model With Spatial Dependence

We introduce a new technique to select the number of components of a mixture model with spatial dependence. The method consists of an estimation of the integrated completed likelihood based on a Laplace’s approximation and a new technique to deal with the normalizing constant intractability of the hidden Potts model. Our proposal is applied to a real satellite image. Supplementary materials are available online.     

Fast and Accurate Approximation of the Full Conditional for Gamma Shape Parameters

The gamma distribution arises frequently in Bayesian models, but there is not an easy-to-use conjugate prior for the shape parameter of a gamma. This inconvenience is usually dealt with by using either Metropolis–Hastings moves, rejection sampling methods, or numerical integration. However, in models with a large number of shape parameters, these existing methods are slower or more complicated than one would like, making them burdensome in practice. It turns out that the full conditional distribution of the gamma shape parameter is well approximated by a gamma distribution, even for small sample sizes, when the prior on the shape parameter is also a gamma distribution. This article introduces a quick and easy algorithm for finding a gamma distribution that approximates the full conditional distribution of the shape parameter. We empirically demonstrate the speed and accuracy of the approximation across a wide range of conditions. If exactness is required, the approximation can be used as a proposal distribution for Metropolis–Hastings. Supplementary material for this article is available online.     

An Approximation Method for Improving Dynamic Network Model Fitting

There has been a great deal of interest recently in the modeling and simulation of dynamic networks, that is, networks that change over time. One promising model is the separable temporal exponential-family random graph model (ERGM) of Krivitsky and Handcock, which treats the formation and dissolution of ties in parallel at each time step as independent ERGMs. However, the computational cost of fitting these models can be substantial, particularly for large, sparse networks. Fitting cross-sectional models for observations of a network at a single point in time, while still a nonnegligible computational burden, is much easier. This article examines model fitting when the available data consist of independent measures of cross-sectional network structure and the duration of relationships under the assumption of stationarity. We introduce a simple approximation to the dynamic parameters for sparse networks with relationships of moderate or long duration and show that the approximation method works best in precisely those cases where parameter estimation is most likely to fail—networks with very little change at each time step. We consider a variety of cases: Bernoulli formation and dissolution of ties, independent-tie formation and Bernoulli dissolution, independent-tie formation and dissolution, and dependent-tie formation models.     

Bayesian Inference for Extremes: Accounting for the Three Extremal Types

The Extremal Types Theorem identifies three distinct types of extremal behaviour. Two different strategies for statistical inference for extreme values have been developed to exploit this asymptotic representation. One strategy uses a model for which the three types are combined into a unified parametric family with the shape parameter of the family determining the type: positive (Fréchet), zero (Gumbel), and negative (negative Weibull). This form of approach never selects the Gumbel type as that type is reduced to a single point in a continuous parameter space. The other strategy first selects the extremal type, based on hypothesis tests, and then estimates the best fitting model within the selected type. Such approaches ignore the uncertainty of the choice of extremal type on the subsequent inference. We overcome these deficiencies by applying the Bayesian inferential framework to an extended model which explicitly allocates a non-zero probability to the Gumbel type. Application of our procedure suggests that the effect of incorporating the knowledge of the Extremal Types Theorem into the inference for extreme values is to reduce uncertainty, with the degree of reduction depending on the shape parameter of the true extremal distribution and the prior weight given to the Gumbel type.     

Continuous Contour Monte Carlo for Marginal Density Estimation With an Application to a Spatial Statistical Model

The problem of marginal density estimation for a multivariate density function f(x) can be generally stated as a problem of density function estimation for a random vector λ(x) of dimension lower than that of x. In this article, we propose a technique, the so-called continuous Contour Monte Carlo (CCMC) algorithm, for solving this problem. CCMC can be viewed as a continuous version of the contour Monte Carlo (CMC) algorithm recently proposed in the literature. CCMC abandons the use of sample space partitioning and incorporates the techniques of kernel density estimation into its simulations. CCMC is more general than other marginal density estimation algorithms. First, it works for any density functions, even for those having a rugged or unbalanced energy landscape. Second, it works for any transformation λ(x) regardless of the availability of the analytical form of the inverse transformation. In this article, CCMC is applied to estimate the unknown normalizing constant function for a spatial autologistic model, and the estimate is then used in a Bayesian analysis for the spatial autologistic model in place of the true normalizing constant function. Numerical results on the U.S. cancer mortality data indicate that the Bayesian method can produce much more accurate estimates than the MPLE and MCMLE methods for the parameters of the spatial autologistic model.     

Model selection of a switching mechanism for financial time series

The threshold autoregressive model with generalized autoregressive conditionally heteroskedastic (GARCH) specification is a popular nonlinear model that captures the well‐known asymmetric phenomena in financial market data. The switching mechanisms of hysteretic autoregressive GARCH models are different from threshold autoregressive model with GARCH as regime switching may be delayed when the hysteresis variable lies in a hysteresis zone. This paper conducts a Bayesian model comparison among competing models by designing an adaptive Markov chain Monte Carlo sampling scheme. We illustrate the performance of three kinds of criteria by comparing models with fat‐tailed and/or skewed errors: deviance information criteria, Bayesian predictive information, and an asymptotic version of Bayesian predictive information. A simulation study highlights the properties of the three Bayesian criteria and the accuracy as well as their favorable performance as model selection tools. We demonstrate the proposed method in an empirical study of 12 international stock markets, providing evidence to strongly support for both models with skew fat‐tailed innovations. Copyright © 2016 John Wiley & Sons, Ltd.     

Geometric Ergodicity of Gibbs and Block Gibbs Samplers for a Hierarchical Random Effects Model

We consider fixed scan Gibbs and block Gibbs samplers for a Bayesian hierarchical random effects model with proper conjugate priors. A drift condition given in Meyn and Tweedie (1993, Chapter 15) is used to show that these Markov chains are geometrically ergodic. Showing that a Gibbs sampler is geometrically ergodic is the first step toward establishing central limit theorems, which can be used to approximate the error associated with Monte Carlo estimates of posterior quantities of interest. Thus, our results will be of practical interest to researchers using these Gibbs samplers for Bayesian data analysis.     

M-G-P型复杂网络中有寿命节点的进入退出机制

现实中复杂网络结构复杂,形式多样,处在高度动态变化的过程.为了更好地理解真实网络的演化,基于复杂网络的特性进行分析,建立了Poissotn连续时间增长节点具有寿命的M-G-P型复杂网络模型,模型中包括:新节点加入、节点老化和老节点退出等,基于齐次马尔可夫链对模型的度分布进行计算,得出M-G-P型网络的度分布符合幂律分布,模型和BA模型一样能产生指数γ=3的无标度网络,验证了导致无标度网络度分布特征起关键性作用的是链接的偏好特性.