An Autoregressive Ordered Probit Model With Application to High-Frequency Financial Data

This article introduces a model that can be considered as an autoregressive extension of the ordered probit model. For parameter estimation we first develop a standard Gibbs sampler which however exhibits bad convergence properties. Using a special transformation group on the sample space we develop a grouped move multigrid Monte Carlo (GM-MGMC) Gibbs sampler and illustrate its fundamental superiority in convergence compared to the standard sampler. To be able to compare the autoregressive ordered probit (AOP) model to other models we further provide an estimation procedure for the marginal likelihood which enables us to compute Bayes factors. We apply the new model to absolute price changes of the IBM stock traded on December 4, 2000, at the New York Stock Exchange. To detect whether the data contain an autoregressive structure we then fit the AOP model as well as the common ordered probit (OP) model to the data. By estimating the corresponding Bayes factor we show that the AOP model fits the data decisively better than the common OP model.     

Perturbation bounds for Monte Carlo within Metropolis via restricted approximations

The Monte Carlo within Metropolis (MCwM) algorithm, interpreted as a perturbed Metropolis–Hastings (MH) algorithm, provides an approach for approximate sampling when the target distribution is intractable. Assuming the unperturbed Markov chain is geometrically ergodic, we show explicit estimates of the difference between the $n$th step distributions of the perturbed MCwM and the unperturbed MH chains. These bounds are based on novel perturbation results for Markov chains which are of interest beyond the MCwM setting. To apply the bounds, we need to control the difference between the transition probabilities of the two chains and to verify stability of the perturbed chain.     

Adaptive Bayesian Nonstationary Modeling for Large Spatial Datasets Using Covariance Approximations

Gaussian process models have been widely used in spatial statistics but face tremendous modeling and computational challenges for very large nonstationary spatial datasets. To address these challenges, we develop a Bayesian modeling approach using a nonstationary covariance function constructed based on adaptively selected partitions. The partitioned nonstationary class allows one to knit together local covariance parameters into a valid global nonstationary covariance for prediction, where the local covariance parameters are allowed to be estimated within each partition to reduce computational cost. To further facilitate the computations in local covariance estimation and global prediction, we use the full-scale covariance approximation (FSA) approach for the Bayesian inference of our model. One of our contributions is to model the partitions stochastically by embedding a modified treed partitioning process into the hierarchical models that leads to automated partitioning and substantial computational benefits. We illustrate the utility of our method with simulation studies and the global Total Ozone Matrix Spectrometer (TOMS) data. Supplementary materials for this article are available online.     

基于贝叶斯统计方法的两总体基因表达数据分类

在疾病的诊断过程中,对疾病的精确分类是提高诊断准确率和疾病治愈率至 关重要的一个环节,DNA芯片技术的出现使得我们从微观的层次获得与疾病分类及诊断 密切相关的基因功能信息．但是DNA芯片技术得到的基因的表达模式数据具有多变量小 样本特点,使得分类过程极不稳定,因此我们首先筛选出表达模式发生显著性变化的基因 作为特征基因集合以减少变量个数,然后再根据此特征基因集合建立分类器对样本进行分 类．本文运用似然比检验筛选出特征基因,然后基于贝叶斯方法建立了统计分类模型,并 应用马尔科夫链蒙特卡罗(MCMC)抽样方法计算样本归类后验概率．最后我们将此模型 应用到两组真实的DNA芯片数据上,并将样本成功分类．     

Empirical Bayes analysis of unreplicated microarray data

Because of the high costs of microarray experiments and the availability of only limited biological materials, microarray experiments are often performed with a small number of replicates. Investigators, therefore, often have to perform their experiments with low replication or without replication. However, the heterogeneous error variability observed in microarray experiments increases the difficulty in analyzing microarray data without replication. No current analysis techniques are practically applicable to such microarray data analysis. We here introduce a statistical method, the so-called unreplicated heterogeneous error model (UHEM) for the microarray data analysis without replication. This method is possible by utilizing many adjacent-intensity genes for estimating local error variance after nonparametric elimination of differentially expressed genes between different biological conditions. We compared the performance of UHEM with three empirical Bayes prior specification methods: between-condition local pooled error, pseudo standard error, or adaptive standard error-based HEM. We found that our unreplicated HEM method is effective for the microarray data analysis when replication of an array experiment is impractical or prohibited.     

Approximate Predetermined Convergence Properties of the Gibbs Sampler

This article aims to provide a method for approximately predetermining convergence properties of the Gibbs sampler. This is to be done by first finding an approximate rate of convergence for a normal approximation of the target distribution. The rates of convergence for different implementation strategies of the Gibbs sampler are compared to find the best one. In general, the limiting convergence properties of the Gibbs sampler on a sequence of target distributions (approaching a limit) are not the same as the convergence properties of the Gibbs sampler on the limiting target distribution. Theoretical results are given in this article to justify that under conditions, the convergence properties of the Gibbs sampler can be approximated as well. A number of practical examples are given for illustration.     

Hybrid parallel tempering and simulated annealing method

In this paper, we propose a new hybrid scheme of parallel tempering and simulated annealing (hybrid PT/SA). Within the hybrid PT/SA scheme, a composite system with multiple conformations is evolving in parallel on a temperature ladder with various transition step sizes. The simulated annealing (SA) process uses a cooling scheme to decrease the temperature values in the temperature ladder to the target temperature. The parallel tempering (PT) scheme is employed to reduce the equilibration relaxation time of the composite system at a particular temperature ladder configuration in the SA process. The hybrid PT/SA method reduces the waiting time in deep local minima and thus leads to a more efficient sampling capability on high-dimensional complicated objective function landscapes. Compared to the approaches PT and parallel SA with the same temperature ladder, transition step sizes, and cooling scheme (parallel SA) configurations, our preliminary results obtained with the hybrid PT/SA method confirm the expected improvements in simulations of several test objective functions, including the Rosenbrock’s function and the “rugged” funnel-like function, and several instantiations of the traveling salesman problem. The hybrid PT/SA may have slower convergence than genetic algorithms (GA) with good crossover heuristics, but it has the advantage of tolerating “bad” initial values and displaying robust sampling capability, even in the absence of additional information. Moreover, the hybrid PT/SA has natural parallelization potential.     

散度偏大计数数据回归模型的变量选择与模型比较

讨论了具有散度偏大特征计数数据的建模与拟合问题.针对导致数据散度偏大的原因和常用的几类候选模型的结构,分别给出了关于嵌套模型的模型与变量同时选择的Bayes方法和关于非嵌套模型的模型检验与比较方法,并在此基础上进一步完善,提出了较为系统完整的模型与变量选择方法.实际例子说明了方法的具体实现过程和有效性.     

Path coupling without contraction

Path coupling is a useful technique for simplifying the analysis of a coupling of a Markov chain. Rather than defining and analysing the coupling on every pair in Ω×Ω, where Ω is the state space of the Markov chain, analysis is done on a smaller set SΩ×Ω. If the coefficient of contraction β is strictly less than one, no further analysis is needed in order to show rapid mixing. However, if β=1 then analysis (of the variance) is still required for all pairs in Ω×Ω. In this paper we present a new approach which shows rapid mixing in the case β=1 with a further condition which only needs to be checked for pairs in S, greatly simplifying the work involved. We also present a technique applicable when β=1 and our condition is not met.     

Optimal Scaling for Random Walk Metropolis on Spherically Constrained Target Densities

We consider the problem of optimal scaling of the proposal variance for multidimensional random walk Metropolis algorithms. It is well known, for a wide range of continuous target densities, that the optimal scaling of the proposal variance leads to an average acceptance rate of 0.234. Therefore a natural question is, do similar results hold for target densities which have discontinuities? In the current work, we answer in the affirmative for a class of spherically constrained target densities. Even though the acceptance probability is more complicated than for continuous target densities, the optimal scaling of the proposal variance again leads to an average acceptance rate of 0.234.     

Statistically Efficient Thinning of a Markov Chain Sampler

It is common to subsample Markov chain output to reduce the storage burden. Geyer shows that discarding k ? 1 out of every k observations will not improve statistical efficiency, as quantified through variance in a given computational budget. That observation is often taken to mean that thinning Markov chain Monte Carlo (MCMC) output cannot improve statistical efficiency. Here, we suppose that it costs one unit of time to advance a Markov chain and then θ > 0 units of time to compute a sampled quantity of interest. For a thinned process, that cost θ is incurred less often, so it can be advanced through more stages. Here, we provide examples to show that thinning will improve statistical efficiency if θ is large and the sample autocorrelations decay slowly enough. If the lag ? ? 1 autocorrelations of a scalar measurement satisfy ρ_? > ρ_{? + 1} > 0, then there is always a θ < ∞ at which thinning becomes more efficient for averages of that scalar. Many sample autocorrelation functions resemble first order AR(1) processes with ρ_? = ρ^|?| for some ? 1 < ρ < 1. For an AR(1) process, it is possible to compute the most efficient subsampling frequency k. The optimal k grows rapidly as ρ increases toward 1. The resulting efficiency gain depends primarily on θ, not ρ. Taking k = 1 (no thinning) is optimal when ρ ? 0. For ρ > 0, it is optimal if and only if θ ? (1 ? ρ)²/(2ρ). This efficiency gain never exceeds 1 + θ. This article also gives efficiency bounds for autocorrelations bounded between those of two AR(1) processes. Supplementary materials for this article are available online.     

两部雷达最佳部署问题研究

本文研究两部雷达联合发现目标概率,采用Monte Carlo法解决了最佳部署问题。     

Noisy Hamiltonian Monte Carlo for Doubly Intractable Distributions

Hamiltonian Monte Carlo (HMC) has been progressively incorporated within the statistician’s toolbox as an alternative sampling method in settings when standard Metropolis–Hastings is inefficient. HMC generates a Markov chain on an augmented state space with transitions based on a deterministic differential flow derived from Hamiltonian mechanics. In practice, the evolution of Hamiltonian systems cannot be solved analytically, requiring numerical integration schemes. Under numerical integration, the resulting approximate solution no longer preserves the measure of the target distribution, therefore an accept–reject step is used to correct the bias. For doubly intractable distributions—such as posterior distributions based on Gibbs random fields—HMC suffers from some computational difficulties: computation of gradients in the differential flow and computation of the accept–reject proposals poses difficulty. In this article, we study the behavior of HMC when these quantities are replaced by Monte Carlo estimates. Supplemental codes for implementing methods used in the article are available online.     

Bayesian Variable Selection via Particle Stochastic Search

We focus on Bayesian variable selection in regression models. One challenge is to search the huge model space adequately, while identifying high posterior probability regions. In the past decades, the main focus has been on the use of Markov chain Monte Carlo (MCMC) algorithms for these purposes. In this article, we propose a new computational approach based on sequential Monte Carlo (SMC), which we refer to as particle stochastic search (PSS). We illustrate PSS through applications to linear regression and probit models.     

Towards a Faster Implementation of Density Estimation With Logistic Gaussian Process Priors

A novel method is proposed to compute the Bayes estimate for a logistic Gaussian process prior for density estimation. The method gains speed by drawing samples from the posterior of a finite-dimensional surrogate prior, which is obtained by imputation of the underlying Gaussian process. We establish that imputation results in quite accurate computation. Simulation studies show that accuracy and high speed can be combined. This fact, along with known flexibility of the logistic Gaussian priors for modeling smoothness and recent results on their large support, makes these priors and the resulting density estimate very attractive.     

Conditioning in Markov Chain Monte Carlo

Abstract

The so-called “Rao-Blackwellized” estimators proposed by Gelfand and Smith do not always reduce variance in Markov chain Monte Carlo when the dependence in the Markov chain is taken into account. An illustrative example is given, and a theorem characterizing the necessary and sufficient condition for such an estimator to always reduce variance is proved.     

General Methods for Monitoring Convergence of Iterative Simulations

Abstract

We generalize the method proposed by Gelman and Rubin (1992a) for monitoring the convergence of iterative simulations by comparing between and within variances of multiple chains, in order to obtain a family of tests for convergence. We review methods of inference from simulations in order to develop convergence-monitoring summaries that are relevant for the purposes for which the simulations are used. We recommend applying a battery of tests for mixing based on the comparison of inferences from individual sequences and from the mixture of sequences. Finally, we discuss multivariate analogues, for assessing convergence of several parameters simultaneously.     

Stopping Rules for the Stochastic Nested Partitions Method

We have recently developed a global optimization methodology for solving combinatorial problems with either deterministic or stochastic performance functions. This method, the Nested Partitions (NP) method has been shown to generate a Markov chain and with probability one to converge to a global optimum. In this paper, we study the rate of convergence of the method through the use of Markov Chain Monte Carlo (MCMC) methods, and use this to derive stopping rules that can be applied during simulation-based optimization. A numerical example serves to illustrate the feasibility of our approach.     

From metropolis to diffusions: Gibbs states and optimal scaling

This paper investigates the behaviour of the random walk Metropolis algorithm in high-dimensional problems. Here we concentrate on the case where the components in the target density is a spatially homogeneous Gibbs distribution with finite range. The performance of the algorithm is strongly linked to the presence or absence of phase transition for the Gibbs distribution; the convergence time being approximately linear in dimension for problems where phase transition is not present. Related to this, there is an optimal way to scale the variance of the proposal distribution in order to maximise the speed of convergence of the algorithm. This turns out to involve scaling the variance of the proposal as the reciprocal of dimension (at least in the phase transition-free case). Moreover, the actual optimal scaling can be characterised in terms of the overall acceptance rate of the algorithm, the maximising value being 0.234, the value as predicted by studies on simpler classes of target density. The results are proved in the framework of a weak convergence result, which shows that the algorithm actually behaves like an infinite-dimensional diffusion process in high dimensions.     

A Pivotal Allocation-Based Algorithm for Solving the Label-Switching Problem in Bayesian Mixture Models

In Bayesian analysis of mixture models, the label-switching problem occurs as a result of the posterior distribution being invariant to any permutation of cluster indices under symmetric priors. To solve this problem, we propose a novel relabeling algorithm and its variants by investigating an approximate posterior distribution of the latent allocation variables instead of dealing with the component parameters directly. We demonstrate that our relabeling algorithm can be formulated in a rigorous framework based on information theory. Under some circumstances, it is shown to resemble the classical Kullback-Leibler relabeling algorithm and include the recently proposed equivalence classes representatives relabeling algorithm as a special case. Using simulation studies and real data examples, we illustrate the efficiency of our algorithm in dealing with various label-switching phenomena. Supplemental materials for this article are available online.