Stability of Partially Implicit Langevin Schemes and Their MCMC Variants

A broad class of implicit or partially implicit time discretizations for the Langevin diffusion are considered and used as proposals for the Metropolis–Hastings algorithm. Ergodic properties of our proposed schemes are studied. We show that introducing implicitness in the discretization leads to a process that often inherits the convergence rate of the continuous time process. These contrast with the behavior of the naive or Euler–Maruyama discretization, which can behave badly even in simple cases. We also show that our proposed chains, when used as proposals for the Metropolis–Hastings algorithm, preserve geometric ergodicity of their implicit Langevin schemes and thus behave better than the local linearization of the Langevin diffusion. We illustrate the behavior of our proposed schemes with examples. Our results are described in detail in one dimension only, although extensions to higher dimensions are also described and illustrated.     

Automatically tuned general-purpose MCMC via new adaptive diagnostics

Adaptive Markov Chain Monte Carlo (MCMC) algorithms attempt to ‘learn’ from the results of past iterations so the Markov chain can converge quicker. Unfortunately, adaptive MCMC algorithms are no longer Markovian, so their convergence is difficult to guarantee. In this paper, we develop new diagnostics to determine whether the adaption is still improving the convergence. We present an algorithm which automatically stops adapting once it determines further adaption will not increase the convergence speed. Our algorithm allows the computer to tune a ‘good’ Markov chain through multiple phases of adaption, and then run conventional non-adaptive MCMC. In this way, the efficiency gains of adaptive MCMC can be obtained while still ensuring convergence to the target distribution.     

On some algorithms for limiting average Markov decision processes

We consider limiting average Markov decision processes (MDP) with finite state and action spaces. We propose some algorithms to determine optimal strategies for deterministic and general MDPs. These algorithms are based on graph theory and the construction of levels in some aggregated MDP.     

On compound Poisson processes arising in change-point type statistical models as limiting likelihood ratios

Different change-point type models encountered in statistical inference for stochastic processes give rise to different limiting likelihood ratio processes. In a previous paper of one of the authors it was established that one of these likelihood ratios, which is an exponential functional of a two-sided Poisson process driven by some parameter, can be approximated (for sufficiently small values of the parameter) by another one, which is an exponential functional of a two-sided Brownian motion. In this paper we consider yet another likelihood ratio, which is the exponent of a two-sided compound Poisson process driven by some parameter. We establish, that similarly to the Poisson type one, the compound Poisson type likelihood ratio can be approximated by the Brownian type one for sufficiently small values of the parameter. We equally discuss the asymptotics for large values of the parameter and illustrate the results by numerical simulations.     

A limiting weak type estimate for capacitary maximal function

A capacitary analogue of the limiting weak type estimate of P. Janakiraman for the Hardy–Littlewood maximal function of an L¹(Rⁿ)$L^1({\mathbb{R}}^n)$-function (cf.  and ) is discovered.     

On confined McKean Langevin processes satisfying the mean no-permeability boundary condition

We construct a confined Langevin type process aimed to satisfy a mean no-permeability condition at the boundary. This Langevin process lies in the class of conditional McKean Lagrangian stochastic models studied by Bossy, Jabir and Talay (2010) [5]. The confined process considered here is a first construction of solutions to the class of Lagrangian stochastic equations with boundary condition issued by the so-called PDF methods for Computational Fluid Dynamics. We prove the well-posedness of the confined system when the state space of the Langevin process is a half-space.     

Subset selection for vector autoregressive processes via adaptive Lasso

Subset selection is a critical component of vector autoregressive (VAR) modeling. This paper proposes simple and hybrid subset selection procedures for VAR models via the adaptive Lasso. By a proper choice of tuning parameters, one can identify the correct subset and obtain the asymptotic normality of the nonzero parameters with probability tending to one. Simulation results show that for small samples, a particular hybrid procedure has the best performance in terms of prediction mean squared errors, estimation errors and subset selection accuracy under various settings. The proposed method is also applied to modeling the IS-LM data for illustration.     

Optimal strategies for a class of adaptive search processes

Optimal strategies are investigated for a class of one-dimensional search processes in which the objective is to find a point which is near, but not beyond, a boundary of uncertain location. Problems of this type are encountered in the analysis of mining operations. Upper and lower bounds for the optimal expected payoff are derived, and the optimal search strategies are described explicitly for a large subclass of these processes. Results are obtained by formulating the search as a multistage decision process and using a dynamic programming approach.     

Sobolev type embeddings in the limiting case

We use interpolation methods to prove a new version of the limiting case of the Sobolev embedding theorem, which includes the result of Hansson and Brezis-Wainger for W _n ^k/k as a special case. We deal with generalized Sobolev spaces W _A ^k , where instead of requiring the functions and their derivatives to be in L_n/k, they are required to be in a rearrangement invariant space A which belongs to a certain class of spaces “close” to L_n/k. We also show that the embeddings given by our theorem are optimal, i.e., the target spaces into which the above Sobolev spaces are shown to embed cannot be replaced by smaller rearrangement invariant spaces. This slightly sharpens and generalizes an, earlier optimality result obtained by Hansson with respect to the Riesz potential operator. In memory of Gene Fabes. Acknowledgements and Notes This research was supported by Technion V.P.R. Fund-M. and C. Papo Research Fund.     

A computational successive improvement scheme for adaptive optimal control processes

To computationally solve an adaptive optimal control problem by means of conventional dynamic programming, a backward recursion must be used with an optimal value and optimal control determined for all conceivable prior information patterns and prior control histories. Consequently, almost all problems are beyond the capability of even large computers.As an alternative, we develop in this paper a computational successive improvement scheme which involves choosing a nominal control policy and then improving it at each iteration. Each improvement involves considerable computation, but much less than the straightforward dynamic programming algorithm. As in any local-improvement procedure, the scheme may converge to something which is inferior to the absolutely optimal control.This paper has been supported by the National Science Foundation under Grant No. GP-25081.     

Almost self-optimizing strategies for the adaptive control of diffusion processes

The ergodic control of a multidimensional diffusion process described by a stochastic differential equation that has some unknown parameters appearing in the drift is investigated. The invariant measure of the diffusion process is shown to be a continuous function of the unknown parameters. For the optimal ergodic cost for the known system, an almost optimal adaptive control is constructed for the unknown system.This research was partially supported by NSF Grants ECS-87-18026, ECS-91-02714, and ECS-91-13029.     

Componentwise adaptation for high dimensional MCMC

Summary  We introduce a new adaptive MCMC algorithm, based on the traditional single component Metropolis-Hastings algorithm and on our earlier adaptive Metropolis algorithm (AM). In the new algorithm the adaption is performed component by component. The chain is no more Markovian, but it remains ergodic. The algorithm is demonstrated to work well in varying test cases up to 1000 dimensions.     

Quadrature processes for integrals of Cauchy type

We study questions relating to convergence of the process $$\int_{ - 1}^{ + 1} \rho (t)\frac{{f(t)}}{{t - x}}dt \approx \sum\nolimits_{k = 1}^n {\alpha _{k,n} } (x)f(x_k^{(n)} )( - 1< x1)$$ wherein the singular integral is taken in the principal value sense. General conditions for convergence in the class of continuously differentiable functionsf are formulated. In the case of the weight function ρ(t)=(√1-t²)^?1, we investigate, under various assumptions onf, the convergence of a specific quadrature process.     

Bayesian ergodic adaptive control of diffusion processes

A Bayesian adaptive control problem for ergodic diffusions is considered. A nearly self-optimising strategy is constructed on the basis of discrete-time observations of the state process. In this derivation a key role is played by an embedded Markov chain with an associated Bellman-type equation which appears to be of independent interest     

MCMC Control Spreadsheets for Exponential Mixture Estimation

Abstract

This article presents Bayesian inference for exponential mixtures, including the choice of a noninformative prior based on a location-scale reparameterization of the mixture. Adapted control sheets are proposed for studying the convergence of the associated Gibbs sampler. They exhibit a strong lack of stability in the allocations of the observations to the different components of the mixture. The setup is extended to the case when the number of components in the mixture is unknown and a reversible jump MCMC technique is implemented. The results are illustrated on simulations and a real dataset.     

Movies for the Visualization of MCMC Output

Past work on visualization methods for Markov chain Monte Carlo analyses leaves many open questions. Particular difficulties arise when the researcher wishes to monitor several aspects of the chain's behavior jointly. We propose a movie, which is an extension of the basic trace plot used for a single parameter, as a dynamic way of assessing the progress of a chain and monitoring the joint posterior distribution. This idea is demonstrated on several examples, including a multiple changepoint problem with data from a corbelled tomb (tholos) in Stylos, Crete.