Posterior contraction rates for the Bayesian approach to linear ill-posed inverse problems

We consider a Bayesian nonparametric approach to a family of linear inverse problems in a separable Hilbert space setting with Gaussian noise. We assume Gaussian priors, which are conjugate to the model, and present a method of identifying the posterior using its precision operator. Working with the unbounded precision operator enables us to use partial differential equations (PDE) methodology to obtain rates of contraction of the posterior distribution to a Dirac measure centered on the true solution. Our methods assume a relatively weak relation between the prior covariance, noise covariance and forward operator, allowing for a wide range of applications.     

Sampling Schemes for Bayesian Variable Selection in Generalized Linear Models

Bayesian approaches to prediction and the assessment of predictive uncertainty in generalized linear models are often based on averaging predictions over different models, and this requires methods for accounting for model uncertainty. When there are linear dependencies among potential predictor variables in a generalized linear model, existing Markov chain Monte Carlo algorithms for sampling from the posterior distribution on the model and parameter space in Bayesian variable selection problems may not work well. This article describes a sampling algorithm based on the Swendsen-Wang algorithm for the Ising model, and which works well when the predictors are far from orthogonality. In problems of variable selection for generalized linear models we can index different models by a binary parameter vector, where each binary variable indicates whether or not a given predictor variable is included in the model. The posterior distribution on the model is a distribution on this collection of binary strings, and by thinking of this posterior distribution as a binary spatial field we apply a sampling scheme inspired by the Swendsen-Wang algorithm for the Ising model in order to sample from the model posterior distribution. The algorithm we describe extends a similar algorithm for variable selection problems in linear models. The benefits of the algorithm are demonstrated for both real and simulated data.     

AN ACCELERATION STRATEGY FOR RANDOMIZE-THEN-OPTIMIZE SAMPLING VIA DEEP NEURAL NETWORKS

Randomize-then-optimize (RTO) is widely used for sampling from posterior distribu-tions in Bayesian inverse problems.However,RTO can be computationally intensive for complexity problems due to repetitive evaluations of the expensive forward model and its gradient.In this work,we present a novel goal-oriented deep neural networks (DNN) sur-rogate approach to substantially reduce the computation burden of RTO.In particular,we propose to drawn the training points for the DNN-surrogate from a local approximated posterior distribution-yielding a flexible and efficient sampling algorithm that converges to the direct RTO approach.We present a Bayesian inverse problem governed by elliptic PDEs to demonstrate the computational accuracy and efficiency of our DNN-RTO ap-proach,which shows that DNN-RTO can significantly outperform the traditional RTO.     

Arc refractor methods for adaptive importance sampling on large Bayesian networks under evidential reasoning

Approximate Bayesian inference by importance sampling derives probabilistic statements from a Bayesian network, an essential part of evidential reasoning with the network and an important aspect of many Bayesian methods. A critical problem in importance sampling on Bayesian networks is the selection of a good importance function to sample a network’s prior and posterior probability distribution. The initially optimal importance functions eventually start deviating from the optimal function when sampling a network’s posterior distribution given evidence, even when adaptive methods are used that adjust an importance function to the evidence by learning. In this article we propose a new family of Refractor Importance Sampling (RIS) algorithms for adaptive importance sampling under evidential reasoning. RIS applies “arc refractors” to a Bayesian network by adding new arcs and refining the conditional probability tables. The goal of RIS is to optimize the importance function for the posterior distribution and reduce the error variance of sampling. Our experimental results show a significant improvement of RIS over state-of-the-art adaptive importance sampling algorithms.     

Constructive Solution of Inverse Parametric Linear/Quadratic Programming Problems

Parametric convex programming has received a lot of attention, since it has many applications in chemical engineering, control engineering, signal processing, etc. Further, inverse optimality plays an important role in many contexts, e.g., image processing, motion planning. This paper introduces a constructive solution of the inverse optimality problem for the class of continuous piecewise affine functions. The main idea is based on the convex lifting concept. Accordingly, an algorithm to construct convex liftings of a given convexly liftable partition will be put forward. Following this idea, an important result will be presented in this article: Any continuous piecewise affine function defined over a polytopic partition is the solution of a parametric linear/quadratic programming problem. Regarding linear optimal control, it will be shown that any continuous piecewise affine control law can be obtained via a linear optimal control problem with the control horizon at most equal to 2 prediction steps.     

二阶广义分布参数系统的谱分布

应用泛函分析算子理论的方法研究了Hilbert空间中二阶广义分布参数系统的谱分布问题,利用有界线性算子的广义逆给出了所讨论问题的解及解的构造性表达式。这对研究二阶广义分布参数系统的镇定及渐进稳定性问题都有重要的理论价值。     

基于服装舒适性的纺织材料设计反问题

纺织材料设计反问题是数学物理反问题的一个新领域,也被称为应用数学与计算数学的一个分支.综述纺织材料设计反问题的来源、数学归结,并基于服装的热湿舒适性、压力舒适性提出了设计反问题,给出了反问题解的定义,综述了求解纺织材料设计反问题的数值算法,列举了若干具有挑战的研究课题.     

A Bayesian inference approach to identify a Robin coefficient in one-dimensional parabolic problems

This paper investigates a nonlinear inverse problem associated with the heat conduction problem of identifying a Robin coefficient from boundary temperature measurement. A Bayesian inference approach is presented for the solution of this problem. The prior modeling is achieved via the Markov random field (MRF). The use of a hierarchical Bayesian method for automatic selection of the regularization parameter in the function estimation inverse problem is discussed. The Markov chain Monte Carlo (MCMC) algorithm is used to explore the posterior state space. Numerical results indicate that MRF provides an effective prior regularization, and the Bayesian inference approach can provide accurate estimates as well as uncertainty quantification to the solution of the inverse problem.     

A data assimilation process for linear ill‐posed problems

In this paper, an iteration process is considered to solve linear ill‐posed problems. Based on the randomness of the involved variables, this kind of problems is regarded as simulation problems of the posterior distribution of the unknown variable given the noise data. We construct a new ensemble Kalman filter‐based method to seek the posterior target distribution. Despite the ensemble Kalman filter method having widespread applications, there has been little analysis of its theoretical properties, especially in the field of inverse problems. This paper analyzes the propagation of the error with the iteration step for the proposed algorithm. The theoretical analysis shows that the proposed algorithm is convergence. We compare the numerical effect with the Bayesian inversion approach by two numerical examples: backward heat conduction problem and the first kind of integral equation. The numerical tests show that the proposed algorithm is effective and competitive with the Bayesian method. Copyright © 2017 John Wiley & Sons, Ltd.     

A Stochastic Algorithm for Fault Inverse Problems in Elastic Half Space with Proof of Convergence

A general stochastic algorithm for solving mixed linear and nonlinear problems was introduced in [11]. We show in this paper how it can be used to solve the fault inverse problem, where a planar fault in elastic half-space and a slip on that fault have to be reconstructed from noisy surface displacement measurements. With the parameter giving the plane containing the fault denoted by $m$ and the regularization parameter for the linear part of the inverse problem denoted by $C$,both modeled as random variables, we derive a formula for the posterior marginal of $m.$ Modeling $C$ as a random variable allows to sweep through a wide range of possible values which was shown to be superior to selecting a fixed value [11]. We prove that this posterior marginal of $m$ is convergent as the number of measurement points and the dimension of the space for discretizing slips increase. Simply put, our proof only assumes that the regularized discrete error functional for processing measurements relates to an order 1 quadrature rule and that the union of the finite-dimensional spaces for discretizing slips is dense. Our proof relies on trace class operator theory to show that an adequate sequence of determinants is uniformly bounded. We also explain how our proof can be extended to a whole class of inverse problems, as long as some basic requirements are met. Finally, we show numerical simulations that illustrate the numerical convergence of our algorithm.     

Forward-Douglas–Rachford splitting and forward-partial inverse method for solving monotone inclusions

We provide two weakly convergent algorithms for finding a zero of the sum of a maximally monotone operator, a cocoercive operator, and the normal cone to a closed vector subspace of a real Hilbert space. The methods exploit the intrinsic structure of the problem by activating explicitly the cocoercive operator in the first step, and taking advantage of a vector space decomposition in the second step. The second step of the first method is a Douglas–Rachford iteration involving the maximally monotone operator and the normal cone. In the second method, it is a proximal step involving the partial inverse of the maximally monotone operator with respect to the vector subspace. Connections between the proposed methods and other methods in the literature are provided. Applications to monotone inclusions with finitely many maximally monotone operators and optimization problems are examined.     

Posterior consistency via precision operators for Bayesian nonparametric drift estimation in SDEs

We study a Bayesian approach to nonparametric estimation of the periodic drift function of a one-dimensional diffusion from continuous-time data. Rewriting the likelihood in terms of local time of the process, and specifying a Gaussian prior with precision operator of differential form, we show that the posterior is also Gaussian with the precision operator also of differential form. The resulting expressions are explicit and lead to algorithms which are readily implementable. Using new functional limit theorems for the local time of diffusions on the circle, we bound the rate at which the posterior contracts around the true drift function.     

Algorithms for the Split Variational Inequality Problem

We propose a prototypical Split Inverse Problem (SIP) and a new variational problem, called the Split Variational Inequality Problem (SVIP), which is a SIP. It entails finding a solution of one inverse problem (e.g., a Variational Inequality Problem (VIP)), the image of which under a given bounded linear transformation is a solution of another inverse problem such as a VIP. We construct iterative algorithms that solve such problems, under reasonable conditions, in Hilbert space and then discuss special cases, some of which are new even in Euclidean space.     

Estimation and Uncertainty Quantification for Piecewise Smooth Signal Recovery

This paper presents an application of the sparse Bayesian learning (SBL) algorithm to linear inverse problems with a high order total variation (HOTV) sparsity prior. For the problem of sparse signal recovery, SBL often produces more accurate estimates than maximum a posterioriestimates, including those that use $\ell_1$ regularization. Moreover, rather than a single signal estimate, SBL yields a full posterior density estimate which can be used for uncertainty quantification. However, SBL is only immediately applicable to problems having a directsparsity prior, or to those that can be formed via synthesis. This paper demonstrates how a problem with an HOTV sparsity prior can be formulated via synthesis, and then utilizes SBL. This expands the class of problems available to Bayesian learning to include, e.g., inverse problems dealing with the recovery of piecewise smooth functions or signals from data. Numerical examples are provided to demonstrate how this new technique is effectively employed.     

Score tests for inverse Gaussian mixtures

The mixed inverse Gaussian given by Whitmore (biScand. J. Statist., 13 , 1986, 211–220) provides a convenient way for testing the goodness‐of‐fit of a pure inverse Gaussian distribution. The test is a one‐sided score test with the null hypothesis being the pure inverse Gaussian (i.e. the mixing parameter is zero) and the alternative a mixture. We devise a simple score test and study its finite sample properties. Monte Carlo results show that it compares favourably with the smooth test of Ducharme ( Test , 10 , 2001, 271‐290). In practical applications, when the pure inverse Gaussian distribution is rejected, one is interested in making inference about the general values of the mixing parameter. However, as it is well known that the inverse Gaussian mixture is a defective distribution; hence, the standard likelihood inference cannot be applied. We propose several alternatives and provide score tests for the mixing parameter. Finite sample properties of these tests are examined by Monte Carlo simulation. Copyright © 2010 John Wiley & Sons, Ltd.     

Reparameterized and Marginalized Posterior and Predictive Sampling for Complex Bayesian Geostatistical Models

This article proposes a four-pronged approach to efficient Bayesian estimation and prediction for complex Bayesian hierarchical Gaussian models for spatial and spatiotemporal data. The method involves reparameterizing the covariance structure of the model, reformulating the means structure, marginalizing the joint posterior distribution, and applying a simplex-based slice sampling algorithm. The approach permits fusion of point-source data and areal data measured at different resolutions and accommodates nonspatial correlation and variance heterogeneity as well as spatial and/or temporal correlation. The method produces Markov chain Monte Carlo samplers with low autocorrelation in the output, so that fewer iterations are needed for Bayesian inference than would be the case with other sampling algorithms. Supplemental materials are available online.     

Variational Markov chain Monte Carlo for Bayesian smoothing of non-linear diffusions

In this paper we develop set of novel Markov chain Monte Carlo algorithms for Bayesian smoothing of partially observed non-linear diffusion processes. The sampling algorithms developed herein use a deterministic approximation to the posterior distribution over paths as the proposal distribution for a mixture of an independence and a random walk sampler. The approximating distribution is sampled by simulating an optimized time-dependent linear diffusion process derived from the recently developed variational Gaussian process approximation method. The novel diffusion bridge proposal derived from the variational approximation allows the use of a flexible blocking strategy that further improves mixing, and thus the efficiency, of the sampling algorithms. The algorithms are tested on two diffusion processes: one with double-well potential drift and another with SINE drift. The new algorithm’s accuracy and efficiency is compared with state-of-the-art hybrid Monte Carlo based path sampling. It is shown that in practical, finite sample applications the algorithm is accurate except in the presence of large observation errors and low observation densities, which lead to a multi-modal structure in the posterior distribution over paths. More importantly, the variational approximation assisted sampling algorithm outperforms hybrid Monte Carlo in terms of computational efficiency, except when the diffusion process is densely observed with small errors in which case both algorithms are equally efficient.     

A Marginal Sampler for σ-Stable Poisson–Kingman Mixture Models

We investigate the class of σ-stable Poisson–Kingman random probability measures (RPMs) in the context of Bayesian nonparametric mixture modeling. This is a large class of discrete RPMs, which encompasses most of the popular discrete RPMs used in Bayesian nonparametrics, such as the Dirichlet process, Pitman–Yor process, the normalized inverse Gaussian process, and the normalized generalized Gamma process. We show how certain sampling properties and marginal characterizations of σ-stable Poisson–Kingman RPMs can be usefully exploited for devising a Markov chain Monte Carlo (MCMC) algorithm for performing posterior inference with a Bayesian nonparametric mixture model. Specifically, we introduce a novel and efficient MCMC sampling scheme in an augmented space that has a small number of auxiliary variables per iteration. We apply our sampling scheme to a density estimation and clustering tasks with unidimensional and multidimensional datasets, and compare it against competing MCMC sampling schemes. Supplementary materials for this article are available online.     

Decomposition and Bayesian analysis of invariant normal linear models

Using the theory of linear group representations, we analyse the normal linear model with known sampling covariance structure invariant under a symmetry group, and sampling mean structure equivariant under the same group. In particular, assuming an invariant normal prior distribution on the parameter space, the problem of Bayesian inference is shown to decompose naturally into several independent subproblems. Within any such subproblem, if additional irreducibility conditions hold, it is shown that the posterior expectation of any parameter is a fixed scalar multiple of its unique unbiased estimator, and similarly the posterior covariance of any two parameters is a fixed scalar multiple of the prior covariance. The theoretical framework is illustrated with reference to experimental designs.     

Reconstruction of the Dirac-Type Integro-Differential Operator From Nodal Data

Abstract

The inverse nodal problem for Dirac type integro-differential operator with the spectral parameter in the boundary conditions is studied. We prove that dense subset of the nodal points determines the coefficients of differential part of operator and gives partial information for integral part of it.