Monte Carlo EM加速算法

EM算法是近年来常用的求后验众数的估计的一种数据增广算法, 但由于求出其E步中积分的显示表达式有时很困难, 甚至不可能, 限制了其应用的广泛性. 而Monte Carlo EM算法很好地解决了这个问题, 将EM算法中E步的积分用Monte Carlo模拟来有效实现, 使其适用性大大增强. 但无论是EM算法, 还是Monte Carlo EM算法, 其收敛速度都是线性的, 被缺损信息的倒数所控制, 当缺损数据的比例很高时, 收敛速度就非常缓慢. 而Newton-Raphson算法在后验众数的附近具有二次收敛速率. 本文提出Monte Carlo EM加速算法, 将Monte Carlo EM算法与Newton-Raphson算法结合, 既使得EM算法中的E步用Monte Carlo模拟得以实现, 又证明了该算法在后验众数附近具有二次收敛速度. 从而使其保留了Monte Carlo EM算法的优点, 并改进了Monte Carlo EM算法的收敛速度. 本文通过数值例子, 将Monte Carlo EM加速算法的结果与EM算法、Monte Carlo EM算法的结果进行比较, 进一步说明了Monte Carlo EM加速算法的优良性.     

Bayesian Variable Selection via Particle Stochastic Search

We focus on Bayesian variable selection in regression models. One challenge is to search the huge model space adequately, while identifying high posterior probability regions. In the past decades, the main focus has been on the use of Markov chain Monte Carlo (MCMC) algorithms for these purposes. In this article, we propose a new computational approach based on sequential Monte Carlo (SMC), which we refer to as particle stochastic search (PSS). We illustrate PSS through applications to linear regression and probit models.     

Two noniterative algorithms for computing posteriors

In this paper, we first propose a noniterative sampling method to obtain an i.i.d. sample approximately from posteriors by combining the inverse Bayes formula, sampling/importance resampling and posterior mode estimates. We then propose a new exact algorithm to compute posteriors by improving the PMDA-Exact using the sampling-wise IBF. If the posterior mode is available from the EM algorithm, then these two algorithms compute posteriors well and eliminate the convergence problem of Markov Chain Monte Carlo methods. We show good performances of our methods by some examples.     

基于贝叶斯推断的TAR模型的门限非线性检验

在正态分布的假定下,变点问题按照均值和方差的变化有四种情形.本文把TAR模型门限非线性的检验问题,看作是对应均值变化,方差不变情形下的变点问题.然后利用可逆跳马尔可夫蒙特卡罗模拟(RJMCMC)方法计算两个比较模型(AR和TAR模型)的后验概率.后验概率的结果支持TAR模型表明门限非线性的存在.模拟实验的结果说明基于贝叶斯推断的检验方法可以很好的区分AR和TAR模型.     

Computational aspects of sequential Monte Carlo filter and smoother

Progress in information technologies has enabled to apply computer-intensive methods to statistical analysis. In time series modeling, sequential Monte Carlo method was developed for general nonlinear non-Gaussian state-space models and it enables to consider very complex nonlinear non-Gaussian models for real-world problems. In this paper, we consider several computational problems associated with sequential Monte Carlo filter and smoother, such as the use of a huge number of particles, two-filter formula for smoothing, and parallel computation. The posterior mean smoother and the Gaussian-sum smoother are also considered.     

Laplace Variational Approximation for Semiparametric Regression in the Presence of Heteroscedastic Errors

Variational approximations provide fast, deterministic alternatives to Markov chain Monte Carlo for Bayesian inference on the parameters of complex, hierarchical models. Variational approximations are often limited in practicality in the absence of conjugate posterior distributions. Recent work has focused on the application of variational methods to models with only partial conjugacy, such as in semiparametric regression with heteroscedastic errors. Here, both the mean and log variance functions are modeled as smooth functions of covariates. For this problem, we derive a mean field variational approximation with an embedded Laplace approximation to account for the nonconjugate structure. Empirical results with simulated and real data show that our approximate method has significant computational advantages over traditional Markov chain Monte Carlo; in this case, a delayed rejection adaptive Metropolis algorithm. The variational approximation is much faster and eliminates the need for tuning parameter selection, achieves good fits for both the mean and log variance functions, and reasonably reflects the posterior uncertainty. We apply the methods to log-intensity data from a small angle X-ray scattering experiment, in which properly accounting for the smooth heteroscedasticity leads to significant improvements in posterior inference for key physical characteristics of an organic molecule.     

A central limit theorem and improved error bounds for a hybrid-Monte Carlo sequence with applications in computational finance

In problems of moderate dimensions, the quasi-Monte Carlo method usually provides better estimates than the Monte Carlo method. However, as the dimension of the problem increases, the advantages of the quasi-Monte Carlo method diminish quickly. A remedy for this problem is to use hybrid sequences; sequences that combine pseudorandom and low-discrepancy vectors. In this paper we discuss a particular hybrid sequence called the mixed sequence. We will provide improved discrepancy bounds for this sequence and prove a central limit theorem for the corresponding estimator. We will also provide numerical results that compare the mixed sequence with the Monte Carlo and randomized quasi-Monte Carlo methods.     

Bayesian joint quantile regression for mixed effects models with censoring and errors in covariates

In this paper, we discuss Bayesian joint quantile regression of mixed effects models with censored responses and errors in covariates simultaneously using Markov Chain Monte Carlo method. Under the assumption of asymmetric Laplace error distribution, we establish a Bayesian hierarchical model and derive the posterior distributions of all unknown parameters based on Gibbs sampling algorithm. Three cases including multivariate normal distribution and other two heavy-tailed distributions are considered for fitting random effects of the mixed effects models. Finally, some Monte Carlo simulations are performed and the proposed procedure is illustrated by analyzing a group of AIDS clinical data set.     

电力系统可靠性的蒙特卡洛方法

本文首先对中国科学技术大学管理科研楼电力系统可靠度评估建立了线性传感器模型。由于线性传感器可靠度评估是一个#P问题,没有多项式时间的算法。所以本文运用了蒙特卡罗方法,考虑到未加改进的蒙特卡洛方法对于解决本身可靠度很高的系统时的效率非常低,本文使用了广泛应用于网络可靠性的RVR(Recursive Variance Reduction)方法,给出了可靠度的测算结果。     

Slice Sampling σ-Stable Poisson-Kingman Mixture Models

The article is concerned with the use of Markov chain Monte Carlo methods for posterior sampling in Bayesian nonparametric mixture models.In particular, we consider the problem of slice sampling mixture models for a large class of mixing measures generalizing the celebrated Dirichlet process. Such a class of measures, known in the literature as σ-stable Poisson-Kingman models, includes as special cases most of the discrete priors currently known in Bayesian nonparametrics, for example, the two-parameter Poisson-Dirichlet process and the normalized generalized Gamma process. The proposed approach is illustrated on some simulated data examples. This article has online supplementary material.     

资产跳跃下CM策略多期收入保证价格模拟

恒定混合策略(CM策略)多期收入保证价格是保本基金发行方采取设置止损的CM\linebreak策略作为投资策略时收取保 本费的理论依据, 其中标的资产由复合泊松过程和维纳过程共同驱动, 这一定价问题内嵌奇异期权, 蒙特卡罗模拟方法擅长处理这种高维数量金融问题. 基于风险中性测度推导出多期收入保证价格的现值表达式, 用条件蒙特卡罗推导出这一现值表达式的模拟公式. 在给定参数下分别用普通蒙特卡罗和条件蒙特卡罗计算CM策略多期收入保证价格的数值解, 结果显示两种蒙特卡罗方法均能有效计算其数值解, 之后通过给定显著性水平下的置信区间长度评价两种方法的精确度, 结果显示条件蒙特卡罗比普通蒙特卡罗有很大改进. 接着运用条件蒙特卡罗模拟研究多期收入保证价格对不同参数范围的变化情况.     

A Monte Carlo method for an objective Bayesian procedure

This paper describes a method for an objective selection of the optimal prior distribution, or for adjusting its hyper-parameter, among the competing priors for a variety of Bayesian models. In order to implement this method, the integration of very high dimensional functions is required to get the normalizing constants of the posterior and even of the prior distribution. The logarithm of the high dimensional integral is reduced to the one-dimensional integration of a cerain function with respect to the scalar parameter over the range of the unit interval. Having decided the prior, the Bayes estimate or the posterior mean is used mainly here in addition to the posterior mode. All of these are based on the simulation of Gibbs distributions such as Metropolis' Monte Carlo algorithm. The improvement of the integration's accuracy is substantial in comparison with the conventional crude Monte Carlo integration. In the present method, we have essentially no practical restrictions in modeling the prior and the likelihood. Illustrative artificial data of the lattice system are given to show the practicability of the present procedure.     

A real delivery problem dealt with Monte Carlo Techniques

In this paper we use Monte Carlo Techniques to deal with a real world delivery problem of a food company in Valencia (Spain). The problem is modeled as a set of 11 instances of the well known Vehicle Routing Problem, VRP, with additional time constraints. Given that VRP is a NP-hard problem, a heuristic algorithm, based on Monte Carlo techniques, is implemented. The solution proposed by this heuristic algorithm reaches distance and money savings of about 20% and 5% respectively. This work has been partially supported by thePlan de Incentivo a la Investigación/98 of the Universidad Politécnica de Valencia, under the project “Técnicas Monte Carlo aplicadas a Problemas de Rutas de Vehículos”.     

Simultaneous Posterior Probability Statements From Monte Carlo Output

This article considers the problem of making simultaneous probability statements in multivariate inferential problems based on samples from a posterior distribution. The calculation of simultaneous credible bands is reviewed and—as an alternative—contour probabilities are proposed. These are defined as 1 minus the content of the highest posterior density region which just covers a certain point of interest. We discuss a Monte Carlo method to estimate contour probabilities and distinguish whether or not the functional form of the posterior density is available. In the latter case, an approach based on Rao-Blackwellization is proposed. We highlight that this new estimate has an important invariance property. We illustrate the performance of the different methods in three applications.     

The exponential matrix function and linear differential equations

Monte Carlo optimization has been shown to be useful in solving multivariate optimization problems. In most Monte Carlo simulations, to find the optimal solution of an integer programming problem, the computer gets close to the true optimal solution, but does not find the exact optimal. This can be a problem at times but it can be overcome by multi‐stage Monte Carlo integer programs that find a preliminary nearly optimal solution and use this as a focal point and springboard for finding the true optimal.     

Variational Markov chain Monte Carlo for Bayesian smoothing of non-linear diffusions

In this paper we develop set of novel Markov chain Monte Carlo algorithms for Bayesian smoothing of partially observed non-linear diffusion processes. The sampling algorithms developed herein use a deterministic approximation to the posterior distribution over paths as the proposal distribution for a mixture of an independence and a random walk sampler. The approximating distribution is sampled by simulating an optimized time-dependent linear diffusion process derived from the recently developed variational Gaussian process approximation method. The novel diffusion bridge proposal derived from the variational approximation allows the use of a flexible blocking strategy that further improves mixing, and thus the efficiency, of the sampling algorithms. The algorithms are tested on two diffusion processes: one with double-well potential drift and another with SINE drift. The new algorithm’s accuracy and efficiency is compared with state-of-the-art hybrid Monte Carlo based path sampling. It is shown that in practical, finite sample applications the algorithm is accurate except in the presence of large observation errors and low observation densities, which lead to a multi-modal structure in the posterior distribution over paths. More importantly, the variational approximation assisted sampling algorithm outperforms hybrid Monte Carlo in terms of computational efficiency, except when the diffusion process is densely observed with small errors in which case both algorithms are equally efficient.     

Multivariate versus univariate Kriging metamodels for multi-response simulation models

To analyze the input/output behavior of simulation models with multiple responses, we may apply either univariate or multivariate Kriging (Gaussian process) metamodels. In multivariate Kriging we face a major problem: the covariance matrix of all responses should remain positive-definite; we therefore use the recently proposed “nonseparable dependence” model. To evaluate the performance of univariate and multivariate Kriging, we perform several Monte Carlo experiments that simulate Gaussian processes. These Monte Carlo results suggest that the simpler univariate Kriging gives smaller mean square error.     

Local consistency of Markov chain Monte Carlo methods

In this paper, we introduce the notion of efficiency (consistency) and examine some asymptotic properties of Markov chain Monte Carlo methods. We apply these results to the data augmentation (DA) procedure for independent and identically distributed observations. More precisely, we show that if both the sample size and the running time of the DA procedure tend to infinity, the empirical distribution of the DA procedure tends to the posterior distribution. This is a local property of the DA procedure, which may be, in some cases, more helpful than the global properties to describe its behavior. The advantages of using the local properties are the simplicity and the generality of the results. The local properties provide useful insight into the problem of how to construct efficient algorithms.     

Optimal Estimation and Cramér-Rao Bounds for Partial Non-Gaussian State Space Models

Partial non-Gaussian state-space models include many models of interest while keeping a convenient analytical structure. In this paper, two problems related to partial non-Gaussian models are addressed. First, we present an efficient sequential Monte Carlo method to perform Bayesian inference. Second, we derive simple recursions to compute posterior Cramér-Rao bounds (PCRB). An application to jump Markov linear systems (JMLS) is given.     

Participating life insurance policies: an accurate and efficient parallel software for COTS clusters

In this paper we discuss the development of a parallel software for the numerical simulation of Participating Life Insurance Policies in distributed environments. The main computational kernels in the mathematical models for the solution of the problem are multidimensional integrals and stochastic differential equations. The former is solved by means of Monte Carlo method combined with the Antithetic Variates variance reduction technique, while differential equations are approximated via a fully implicit, positivity-preserving, Euler method. The parallelization strategy we adopted relies on the parallelization of Monte Carlo algorithm. We implemented and tested the software on a PC Linux cluster.