Isoseparation and robustness in parametric Bayesian inference

This paper introduces a new family of local density separations for assessing robustness of finite-dimensional Bayesian posterior inferences with respect to their priors. Unlike for their global equivalents, under these novel separations posterior robustness is recovered even when the functioning posterior converges to a defective distribution, irrespectively of whether the prior densities are grossly misspecified and of the form and the validity of the assumed data sampling distribution. For exponential family models, the local density separations are shown to form the basis of a weak topology closely linked to the Euclidean metric on the natural parameters. In general, the local separations are shown to measure relative roughness of the prior distribution with respect to its corresponding posterior and provide explicit bounds for the total variation distance between an approximating posterior density to a genuine posterior. We illustrate the application of these bounds for assessing robustness of the posterior inferences for a dynamic time series model of blood glucose concentration in diabetes mellitus patients with respect to alternative prior specifications.     

The play operator,the truncated variation and the generalisation of the Jordan decomposition

The play operator minimalizes the total variation on intervals, [0,T],T > 0, of functions approximating uniformly given regulated function with given accuracy and starting from a given point. In this article, we link the play operator with the so‐called truncated variation functionals, introduced recently by the second‐named author, and provide a semi‐explicit expression for the play operator in terms of these functionals. Generalisation for time‐dependent boundaries is also considered. This gives the best possible lower bounds for the total variation of the outputs of the play operator and its Jordan‐like decomposition. Copyright © 2014 John Wiley & Sons, Ltd.     

Computational Aspects of Bayesian Spectral Density Estimation

Gaussian time-series models are often specified through their spectral density. Such models present several computational challenges, in particular because of the nonsparse nature of the covariance matrix. We derive a fast approximation of the likelihood for such models. We propose to sample from the approximate posterior (i.e., the prior times the approximate likelihood), and then to recover the exact posterior through importance sampling. We show that the variance of the importance sampling weights vanishes as the sample size goes to infinity. We explain why the approximate posterior may typically be multimodal, and we derive a Sequential Monte Carlo sampler based on an annealing sequence to sample from that target distribution. Performance of the overall approach is evaluated on simulated and real datasets. In addition, for one real-world dataset, we provide some numerical evidence that a Bayesian approach to semiparametric estimation of spectral density may provide more reasonable results than its frequentist counterparts. The article comes with supplementary materials, available online, that contain an Appendix with a proof of our main Theorem, a Python package that implements the proposed procedure, and the Ethernet dataset.     

Triply fuzzy function approximation for hierarchical Bayesian inference

We prove that three independent fuzzy systems can uniformly approximate Bayesian posterior probability density functions by approximating the prior and likelihood probability densities as well as the hyperprior probability densities that underly the priors. This triply fuzzy function approximation extends the recent theorem for uniformly approximating the posterior density by approximating just the prior and likelihood densities. This approximation allows users to state priors and hyper-priors in words or rules as well as to adapt them from sample data. A fuzzy system with just two rules can exactly represent common closed-form probability densities so long as they are bounded. The function approximators can also be neural networks or any other type of uniform function approximator. Iterative fuzzy Bayesian inference can lead to rule explosion. We prove that conjugacy in the if-part set functions for prior, hyperprior, and likelihood fuzzy approximators reduces rule explosion. We also prove that a type of semi-conjugacy of if-part set functions for those fuzzy approximators results in fewer parameters in the fuzzy posterior approximator.     

On a class of Bayesian nonparametric estimates: II. Hazard rate estimates

The Bayes estimation of hazard rates for a family of multiplicative point processes is considered. We study the model for which a hazard rate can be linearly parametrized by a freely varied measure. The weighted gamma process is assumed to be the prior distribution of this measure; the posterior distributions and the posterior means are given in explicit forms. Examples of the evaluation of posterior means are given.The research of this author is supported in part by NSF Grant MCS 81-02523-01.     

Berry-Esséen bounds and almost sure CLT for the quadratic variation of the sub-fractional Brownian motion

In this paper, we derive explicit bounds for the Kolmogorov distance in the CLT and we prove the almost sure CLT for the quadratic variation of the sub-fractional Brownian motion. We use recent results on the Stein method combined with the Malliavin calculus and an almost sure CLT for multiple integrals.     

Empty Simplices of Polytopes and Graded Betti Numbers

The conjecture of Kalai, Kleinschmidt, and Lee on the number of empty simplices of a simplicial polytope is established by relating it to the first graded Betti numbers of the polytope and applying a result of Migliore and the author. This approach allows us to derive explicit optimal bounds on the number of empty simplices of any given dimension. As a key result, we prove optimal bounds for the graded Betti numbers of any standard graded K-algebra in terms of its Hilbert function.     

Estimation and Uncertainty Quantification for Piecewise Smooth Signal Recovery

This paper presents an application of the sparse Bayesian learning (SBL) algorithm to linear inverse problems with a high order total variation (HOTV) sparsity prior. For the problem of sparse signal recovery, SBL often produces more accurate estimates than maximum a posterioriestimates, including those that use $\ell_1$ regularization. Moreover, rather than a single signal estimate, SBL yields a full posterior density estimate which can be used for uncertainty quantification. However, SBL is only immediately applicable to problems having a directsparsity prior, or to those that can be formed via synthesis. This paper demonstrates how a problem with an HOTV sparsity prior can be formulated via synthesis, and then utilizes SBL. This expands the class of problems available to Bayesian learning to include, e.g., inverse problems dealing with the recovery of piecewise smooth functions or signals from data. Numerical examples are provided to demonstrate how this new technique is effectively employed.     

Optimal Estimation and Cramér-Rao Bounds for Partial Non-Gaussian State Space Models

Partial non-Gaussian state-space models include many models of interest while keeping a convenient analytical structure. In this paper, two problems related to partial non-Gaussian models are addressed. First, we present an efficient sequential Monte Carlo method to perform Bayesian inference. Second, we derive simple recursions to compute posterior Cramér-Rao bounds (PCRB). An application to jump Markov linear systems (JMLS) is given.     

不同先验分布下的后验分布确定土力学参数

岩土工程中各土层参数的取值是根据现场及室内试验数据,采用经典统计学方法进行确定的,但这往往忽略了先验信息的作用.与经典统计学方法不同的是,Bayes法能从考虑先验分布的角度结合样本分布去推导后验分布,为岩土参数的取值提供一种新的分析方法.岩土工程勘察可视为对总体地层的随机抽样,当抽样完成时,样本分布密度函数是确定的,故Bayes法中的后验分布取决于先验分布,因此推导出两套不同的先验分布:利用先验信息确定先验分布及共轭先验分布.通过对先验及后验分布中超参数的计算,当样本总体符合N(μ,σ²)正态分布时,对所要研究的未知参数μ和σ展开分析,综合对比不同先验分布下后验分布的区间长度,给出岩土参数Bayes推断中最佳后验分布所要选择的先验分布.结果表明:共轭情况下的后验分布总是比无信息情况下的后验区间短,概率密度函数分布更集中,取值更方便.在正态总体情形下,根据未知参数μ和σ的联合后验分布求极值方法,确定样本总体中最大概率均值μ_max和方差σ_max作为工程设计采用值,为岩土参数取值方法提供了一条新的路径,有较好的工程意义.     

基于局部多项式展开的多元非参数模型贝叶斯带宽选择

在多元非参数模型中带宽和阶的选择对局部多项式估计量的表现十分重要。本文基于交叉验证准则提出一个自适应贝叶斯带宽选择方法。在给定的误差密度函数下,该方法可推导出对应的似然函数,并构造带宽参数的后验密度函数。随后,通过带宽的后验期望可同时获得阶和带宽的估计。数值模拟的结果表明,该方法不仅比大拇指准则方法精确,且比交叉验证方法耗时更少。与此同时,与Nadaraya-Watson估计相比,所提带宽选择方法对多元非参数模型的适应性要更好。最后,本文通过一组实际数据说明有限样本下所提贝叶斯带宽选择的表现很好。     

Piecewise polynomial Taylor methods for initial value problems

A class of explicit Taylor-type methods for numerically solving first-order ordinary differential equations is presented. The basic idea is that of generating a piecewise polynomial approximating function, with a given order of differentiability, by repeated Taylor expansion. Sharp error bounds for the approximation and its derivatives are given along with a stability analysis.This work was supported by the United States Atomic Energy Commission.     

Weighted frames of exponentials and stable recovery of multidimensional functions from nonuniform Fourier samples

In this paper, we consider the problem of recovering a compactly supported multivariate function from a collection of pointwise samples of its Fourier transform taken nonuniformly. We do this by using the concept of weighted Fourier frames. A seminal result of Beurling shows that sampling points give rise to a classical Fourier frame provided they are relatively separated and of sufficient density. However, this result does not allow for arbitrary clustering of sampling points, as is often the case in practice. Whilst keeping the density condition sharp and dimension independent, our first result removes the separation condition and shows that density alone suffices. However, this result does not lead to estimates for the frame bounds. A known result of Gröchenig provides explicit estimates, but only subject to a density condition that deteriorates linearly with dimension. In our second result we improve these bounds by reducing the dimension dependence. In particular, we provide explicit frame bounds which are dimensionless for functions having compact support contained in a sphere. Next, we demonstrate how our two main results give new insight into a reconstruction algorithm—based on the existing generalized sampling framework—that allows for stable and quasi-optimal reconstruction in any particular basis from a finite collection of samples. Finally, we construct sufficiently dense sampling schemes that are often used in practice—jittered, radial and spiral sampling schemes—and provide several examples illustrating the effectiveness of our approach when tested on these schemes.     

Regularized Kernel-Based Reconstruction in Generalized Besov Spaces

We present a theoretical framework for reproducing kernel-based reconstruction methods in certain generalized Besov spaces based on positive, essentially self-adjoint operators. An explicit representation of the reproducing kernel is given in terms of an infinite series. We provide stability estimates for the kernel, including inverse Bernstein-type estimates for kernel-based trial spaces, and we give condition estimates for the interpolation matrix. Then, a deterministic error analysis for regularized reconstruction schemes is presented by means of sampling inequalities. In particular, we provide error bounds for a regularized reconstruction scheme based on a numerically feasible approximation of the kernel. This allows us to derive explicit coupling relations between the series truncation, the regularization parameters and the data set.     

The influence of sequential extremal processes on the partial sum process

In this paper we derive general upper bounds for the total variation distance between the distributions of a partial sum process in row-wise independent, non-negative triangular arrays and the sum of a fixed number of corresponding extremal processes. As a special case we receive bounds for the supremum distance between the distribution functions of a partial sum and the sum of corresponding upper extremes which improve upon existing results. The outcome may be interpreted as the influence of large insurance claims on the total loss. Moreover, under an additional infinitesimal condition we also prove explicit bounds for limits of the above quantities. Thereby we give a didactic and elementary proof of the Ferguson–Klass representation of Lévy processes on ?_?≥?0 which reflects the influence of extremal processes in insurance.     

On the posterior median estimators of possibly sparse sequences

We adopt the Bayesian paradigm and discuss certain properties of posterior median estimators of possibly sparse sequences. The prior distribution considered is a mixture of an atom of probability at zero and a symmetric unimodal distribution, and the noise distribution is taken as another symmetric unimodal distribution. We derive an explicit form of the corresponding posterior median and show that it is an antisymmetric function and, under some conditions, a shrinkage and a thresholding rule. Furthermore we show that, as long as the tails of the nonzero part of the prior distribution are heavier than the tails of the noise distribution, the posterior median, under some constraints on the involved parameters, has the bounded shrinkage property, extending thus recent results to larger families of prior and noise distributions. Expressions of posterior distributions and posterior medians in particular cases of interest are obtained. The asymptotes of the derived posterior medians, which provide valuable information of how the corresponding estimators treat large coefficients, are also given. These results could be particularly useful for studying frequentist optimality properties and developing statistical techniques of the resulting posterior median estimators of possibly sparse sequences for a wider set of prior and noise distributions.     

Quantitative Approximation to the Ordered Dirichlet Distribution under Varying Basic Probability Spaces

An approximate expansion of a sequence of ordered Dirichlet densities is given under the set-up with varying dimensions of the relating basic probability spaces. The problem is handled as the approximation to the joint distribution of an increasing number of selected order statistics based on the random sample drawn from the uniform distribution U(0, 1). Some inverse factorial series to the expansion of logarithmic function enable us to give quantitative error evaluations to our problem. With the help of them the relating modified K-L information number, which is defined on an approximate main domain and different from the usual ones, is accurately evaluated. Further, the proof of the approximate joint normality of the selected order statistics is more systematically presented than those given in existing works. Concerning the approximate normality the modified affinity and the half variation distance are also evaluated.     

New prior near-ignorance models on the simplex

The aim of this paper is to derive new near-ignorance models on the probability simplex, which do not directly involve the Dirichlet distribution and, thus, are alternative to the Imprecise Dirichlet Model (IDM). We focus our investigation on a particular class of distributions on the simplex which is known as the class of Normalized Infinitely Divisible (NID) distributions; it includes the Dirichlet distribution as a particular case. For this class it is possible to derive general formulae for prior and posterior predictive inferences, by exploiting the Lévy–Khintchine representation theorem. This allows us to generally characterize the near-ignorance properties of the NID class. After deriving these general properties, we focus our attention on three members of this class. We will show that one of these near-ignorance models satisfies the representation invariance principle and, for a given value of the prior strength, always provides inferences that encompass those of the IDM. The other two models do not satisfy this principle, but their imprecision depends linearly or almost linearly on the number of observed categories; we argue that this is sometimes a desirable property for a predictive model.     

Analysis of a three node queueing network

The three node Jackson queueing network is the simplest acyclic network in which in equilibrium the sojourn times of a customer at each of the nodes are dependent. We show that assuming the individual sojourn times are independent provides a good approximation to the total sojourn time. This is done by simulating the network and showing that the sojourn times generally pass a Kolmogorov-Smirnov test as having come from the approximating distribution. Since the sum of dependent random variables may have the same distribution as the sum of independent random variables with the same marginal distributions, it is conceivable that our approximation is exact. However, we numerically compute upper and lower bounds for the distribution of the total sojourn time; these bounds are so close that the approximating distribution lies outside of the bounds. Thus, the bounds are accurate enough to distinguish between the two distributions even though the Kolmogorov-Smirnov test generally cannot.     

Higher-order total variation bounds for expectations of periodic functions and simple integer recourse approximations

We derive bounds on the expectation of a class of periodic functions using the total variations of higher-order derivatives of the underlying probability density function. These bounds are a strict improvement over those of Romeijnders et al. (Math Program 157:3–46, 2016b), and we use them to derive error bounds for convex approximations of simple integer recourse models. In fact, we obtain a hierarchy of error bounds that become tighter if the total variations of additional higher-order derivatives are taken into account. Moreover, each error bound decreases if these total variations become smaller. The improved bounds may be used to derive tighter error bounds for convex approximations of more general recourse models involving integer decision variables.