Deep Composition of Tensor-Trains Using Squared Inverse Rosenblatt Transports

Characterising intractable high-dimensional random variables is one of the fundamental challenges in stochastic computation. The recent surge of transport maps offers a mathematical foundation and new insights for tackling this challenge by coupling intractable random variables with tractable reference random variables. This paper generalises the functional tensor-train approximation of the inverse Rosenblatt transport recently developed by Dolgov et al. (Stat Comput 30:603–625, 2020) to a wide class of high-dimensional non-negative functions, such as unnormalised probability density functions. First, we extend the inverse Rosenblatt transform to enable the transport to general reference measures other than the uniform measure. We develop an efficient procedure to compute this transport from a squared tensor-train decomposition which preserves the monotonicity. More crucially, we integrate the proposed order-preserving functional tensor-train transport into a nested variable transformation framework inspired by the layered structure of deep neural networks. The resulting deep inverse Rosenblatt transport significantly expands the capability of tensor approximations and transport maps to random variables with complicated nonlinear interactions and concentrated density functions. We demonstrate the efficiency of the proposed approach on a range of applications in statistical learning and uncertainty quantification, including parameter estimation for dynamical systems and inverse problems constrained by partial differential equations.

     

A simple algorithm for tighter exact upper confidence bounds with rare attributes in finite universes

When attributes are rare and few or none are observed in the selected sample from a finite universe, sampling statisticians are increasingly being challenged to use whatever methods are available to declare with high probability or confidence that the universe is near or completely attribute-free. This is especially true when the attribute is undesirable. Approximations such as those based on normal theory are frequently inadequate with rare attributes. For simple random sampling without replacement, an appropriate probability distribution for statistical inference is the hypergeometric distribution. But even with the hypergeometric distribution, the investigator is limited from making claims of attribute-free with high confidence unless the sample size is quite large using nonrandomized techniques. For students in statistical theory, this short article seeks to revive the question of the relevance of randomized methods. When comparing methods for construction of confidence bounds in discrete settings, randomization methods are useful in fixing like confidence levels and hence facilitating the comparisons. Under simple random sampling, this article defines and presents a simple algorithm for the construction of exact “randomized” upper confidence bounds which permit one to possibly report tighter bounds than those exact bounds obtained using “nonrandomized” methods. A general theory for exact randomized confidence bounds is presented in Lehmann (1959, p. 81), but Lehmann's development requires more mathematical development than is required in this application. Not only is the development of these “randomized” bounds in this paper elementary, but their desirable properties and their link with the usual nonrandomized bounds are easy to see with the presented approach which leads to the same results as would be obtained using the method of Lehmann.     

The Kolmogorov–Obukhov Statistical Theory of Turbulence

In 1941 Kolmogorov and Obukhov postulated the existence of a statistical theory of turbulence, which allows the computation of statistical quantities that can be simulated and measured in a turbulent system. These are quantities such as the moments, the structure functions and the probability density functions (PDFs) of the turbulent velocity field. In this paper we will outline how to construct this statistical theory from the stochastic Navier–Stokes equation. The additive noise in the stochastic Navier–Stokes equation is generic noise given by the central limit theorem and the large deviation principle. The multiplicative noise consists of jumps multiplying the velocity, modeling jumps in the velocity gradient. We first estimate the structure functions of turbulence and establish the Kolmogorov–Obukhov 1962 scaling hypothesis with the She–Leveque intermittency corrections. Then we compute the invariant measure of turbulence, writing the stochastic Navier–Stokes equation as an infinite-dimensional Ito process, and solving the linear Kolmogorov–Hopf functional differential equation for the invariant measure. Finally we project the invariant measure onto the PDF. The PDFs turn out to be the normalized inverse Gaussian (NIG) distributions of Barndorff-Nilsen, and compare well with PDFs from simulations and experiments.     

Iterative Approximation of Statistical Distributions and Relation to Information Geometry

The optimal control of stochastic processes through sensor estimation of probability density functions is given a geometric setting via information theory and the information metric. Information theory identifies the exponential distribution as the maximum entropy distribution if only the mean is known and the Γ distribution if also the mean logarithm is known. The surface representing Γ models has a natural Riemannian information metric. The exponential distributions form a one-dimensional subspace of the two-dimensional space of all Γ distributions, so we have an isometric embedding of the random model as a subspace of the Γ models. This geometry provides an appropriate structure on which to represent the dynamics of a process and algorithms to control it. This short paper presents a comparative study on the parameter estimation performance between the geodesic equation and the B-spline function approximations when they are used to optimize the parameters of the Γ family distributions. In this case, the B-spline functions are first used to approximate the Γ probability density function on a fixed length interval; then the coefficients of the approximation are related, through mean and variance calculations, to the two parameters (i.e. μ and β) in Γ distributions. A gradient based parameter tuning method has been used to produce the trajectories for (μ, β) when B-spline functions are used, and desired results have been obtained which are comparable to the trajectories obtained from the geodesic equation. This revised version was published online in August 2006 with corrections to the Cover Date.     

A general control variate method for option pricing under Lévy processes

We present a general control variate method for simulating path dependent options under Lévy processes. It is based on fast numerical inversion of the cumulative distribution functions and exploits the strong correlation of the payoff of the original option and the payoff of a similar option under geometric Brownian motion. The method is applicable for all types of Lévy processes for which the probability density function of the increments is available in closed form. Numerical experiments confirm that our method achieves considerable variance reduction for different options and Lévy processes. We present the applications of our general approach for Asian, lookback and barrier options under variance gamma, normal inverse Gaussian, generalized hyperbolic and Meixner processes.     

Two sample Bayesian prediction intervals for order statistics based on the inverse exponential-type distributions using right censored sample

In this paper, two sample Bayesian prediction intervals for order statistics (OS) are obtained. This prediction is based on a certain class of the inverse exponential-type distributions using a right censored sample. A general class of prior density functions is used and the predictive cumulative function is obtained in the two samples case. The class of the inverse exponential-type distributions includes several important distributions such the inverse Weibull distribution, the inverse Burr distribution, the loglogistic distribution, the inverse Pareto distribution and the inverse paralogistic distribution. Special cases of the inverse Weibull model such as the inverse exponential model and the inverse Rayleigh model are considered.     

Certain estimation problems for multivariate hypergeometric models

Certain estimation problems associated with the multivariate hypergeometric models: the property of completeness, maximum likelihood estimates of the parameters of multivariate negative hypergeometric, multivariate negative inverse hypergeometric, Bayesian estimation of the parameters of multivariate hypergeometric and multivariate inverse hypergeometrics are discussed in this paper. A two stage approach for generating the prior distribution, first by setting up a parametric super population and then choosing a prior distribution is followed. Posterior expectations and variances of certain functions of the parameters of the finite population are provided in cases of direct and inverse sampling procedures. It is shown that under extreme diffuseness of prior knowledge the posterior distribution of the finite population mean has an approximate mean and variance (N-n)S ²/Nn, providing a Bayesian interpretation for the classical unbiased estimates in traditional sample survey theory.     

随机估计和VDR检验

众所周知统计推断有三种理论:普遍承认的Neyman理论(频率学派),Bayes推断和信仰推断(Fiducial)。Bayes推断基于后验分布,由先验分布和样本分布求得。信仰推断是基于信仰分布(Confidence Distribution,简称CD),直接利用样本求得。两者推断方式一致,都是用分布函数作推断,称为分布推断。从分析传统的参数估计、假设检验特性来看,经典统计推断也可以视为分布推断。通常将置信上限看做置信度的函数。其反函数,即置信度是置信上界的函数,恰是分布函数,该分布恰是近年来引起许多学者兴趣的CD。在本文中,基于随机化估计(其分布是一CD)的概率密度函数,提出VDR检验。常见正态分布期望或方差的检验,多元正态分布期望的Hoteling检验等是其特例。VDR(vertical density representation)检验适合于多元分布参数检验,实现了非正态的多元线性变换分布族的参数检验。VDR构造的参数的置信域有最小Lebesgue测度。     

Convolution kernels implementation of cardinalized probability hypothesis density filter

The probability hypothesis density （PHD） propagates the posterior intensity in place of the poste- rior probability density of the multi-target state. The cardinalized PHD （CPHD） recursion is a generalization of PHD recursion, which jointly propagates the posterior intensity function and posterior cardinality distribution. A number of sequential Monte Carlo （SMC） implementations of PHD and CPHD filters （also known as SMC- PHD and SMC-CPHD filters, respectively） for general non-linear non-Gaussian models have been proposed. However, these approaches encounter the limitations when the observation variable is analytically unknown or the observation noise is null or too small. In this paper, we propose a convolution kernel approach in the SMC-CPHD filter. The simuIation results show the performance of the proposed filter on several simulated case studies when compared to the SMC-CPHD filter.     

Inverse problems in queueing theory and Internet probing

Queueing theory is typically concerned with the solution of direct problems, where the trajectory of the queueing system, and laws thereof, are derived based on a complete specification of the system, its inputs and initial conditions. In this paper we point out the importance of inverse problems in queueing theory, which aim to deduce unknown parameters of the system based on partially observed trajectories. We focus on the class of problems stemming from probing based methods for packet switched telecommunications networks, which have become a central tool in the measurement of the structure and performance of the Internet. We provide a general definition of the inverse problems in this class and map out the key variants: the analytical methods, the statistical methods and the design of experiments. We also contribute to the theory in each of these subdomains. Accordingly, a particular inverse problem based on product-form queueing network theory is tackled in detail, and a number of other examples are given. We also show how this inverse problem viewpoint translates to the design of concrete Internet probing applications.     

不同先验分布下的后验分布确定土力学参数

岩土工程中各土层参数的取值是根据现场及室内试验数据,采用经典统计学方法进行确定的,但这往往忽略了先验信息的作用.与经典统计学方法不同的是,Bayes法能从考虑先验分布的角度结合样本分布去推导后验分布,为岩土参数的取值提供一种新的分析方法.岩土工程勘察可视为对总体地层的随机抽样,当抽样完成时,样本分布密度函数是确定的,故Bayes法中的后验分布取决于先验分布,因此推导出两套不同的先验分布:利用先验信息确定先验分布及共轭先验分布.通过对先验及后验分布中超参数的计算,当样本总体符合N(μ,σ²)正态分布时,对所要研究的未知参数μ和σ展开分析,综合对比不同先验分布下后验分布的区间长度,给出岩土参数Bayes推断中最佳后验分布所要选择的先验分布.结果表明:共轭情况下的后验分布总是比无信息情况下的后验区间短,概率密度函数分布更集中,取值更方便.在正态总体情形下,根据未知参数μ和σ的联合后验分布求极值方法,确定样本总体中最大概率均值μ_max和方差σ_max作为工程设计采用值,为岩土参数取值方法提供了一条新的路径,有较好的工程意义.     

Adaptive Methods for Spatial Scan Analysis via Semiparametric Mixture Models

Spatial scan density (SSD) estimation via mixture models is an important problem in the field of spatial statistical analysis and has wide applications in image analysis. The “borrowed strength” density estimation (BSDE) method via mixture models enables one to estimate the local probability density function in a random field wherein potential similarities between the density functions for the subregions are exploited. This article proposes an efficient methods for SSD estimation by integrating the borrowed strength technique into the alternative EM framework which combines the statistical basis of the BSDE approach with the stability and improved convergence rate of the alternative EM methods. In addition, we propose adaptive SSD estimation methods that extend the aforementioned approach by eliminating the need to find the posterior probability of membership of the component densities afresh in each subregion. Simulation results and an application to the detection and identification of man-made regions of interest in an unmanned aerial vehicle imagery experiment show that the adaptive methods significantly outperform the BSDE method. Other applications include automatic target recognition, mammographic image analysis, and minefield detection.     

Bayesian Inversion of Piecewise Affine Operators in a Gaussian Framework

Piecewise affine inverse problems form a general class of nonlinear inverse problems. In particular inverse problems obeying certain variational structures, such as Fermat's principle in travel time tomography, are of this type. In a piecewise affine inverse problem a parameter is to be reconstructed when its mapping through a piecewise affine operator is observed, possibly with errors. A piecewise affine operator is defined by partitioning the parameter space and assigning a specific affine operator to each part. A Bayesian approach with a Gaussian random field prior on the parameter space is used. Both problems with a discrete finite partition and a continuous partition of the parameter space are considered.

The main result is that the posterior distribution is decomposed into a mixture of truncated Gaussian distributions, and the expression for the mixing distribution is partially analytically tractable. The general framework has, to the authors' knowledge, not previously been published, although the result for the finite partition is generally known.

Inverse problems are currently of large interest in many fields. The Bayesian approach is popular and most often highly computer intensive. The posterior distribution is frequently concentrated close to high-dimensional nonlinear spaces, resulting in slow mixing for generic sampling algorithms. Inverse problems are, however, often highly structured. In order to develop efficient sampling algorithms for a problem at hand, the problem structure must be exploited.

The decomposition of the posterior distribution that is derived in the current work can be used to develop specialized sampling algorithms. The article contains examples of such sampling algorithms. The proposed algorithms are applicable also for problems with exact observations. This is a case for which generic sampling algorithms tend to fail.     

Random approximations to some measures of accuracy in nonparametric curve estimation

This paper deals with a quite general nonparametric statistical curve estimation setting. Special cases include estimation or probability density functions, regression functions, and hazard functions. The class of “fractional delta sequence estimators” is defined and treated here. This class includes the familiar kernel, orthogonal series, and histogram methods. It is seen that, under some mild assumptions, both the average square error and integrated square error provide reasonable (random) approximations to the mean integrated square error. This is important for two reasons. First, it provides theoretical backing to a practice that has been employed in several simulation studies. Second, it provides a vital tool for proving theorems about selecting smoothing parameters for several different nonparametric curve estimators.     

On the normality a posteriori for exponential distributions,using the Bayesian estimation

Summary The Bayesian estimation problem for the parameter θ of an exponential probability distribution is considered, when it is assumed that θ has a natural conjugate prior density and a loss-function depending on the squared error is used. It is shown that, with probability one, the posterior density of the Bayesian—centered and scaled parameter converges pointwise to the normal probability density. The weak convergence of the posterior distributions to the normal distribution follows directly. Both correct and incorrect models are studied and the asymptotic normality is stated respectively.     

Second order Subexponential Distributions with Finite Mean and Their Applications to Subordinated Distributions

Consider a probability distribution subordinate to a subexponential distribution with finite mean. In this paper, we discuss the second order tail behavior of the subordinated distribution within a rather general framework in which we do not require the existence of density functions. For this aim, the so-called second order subexponential distribution is proposed and some of its related properties are established. Our results unify and improve some classical results.     

Asymptotic expansions of posterior expectations,distributions and densities for stochastic processes

Asymptotic expansions are derived for Bayesian posterior expectations, distribution functions and density functions. The observations constitute a general stochastic process in discrete or continuous time.     

Univariate and multivariate versions of the negative binomial-inverse Gaussian distributions with applications

In this paper we propose a new compound negative binomial distribution by mixing the p negative binomial parameter with an inverse Gaussian distribution and where we consider the reparameterization p=exp(−λ). This new formulation provides a tractable model with attractive properties which make it suitable for application not only in the insurance setting but also in other fields where overdispersion is observed. Basic properties of the new distribution are studied. A recurrence for the probabilities of the new distribution and an integral equation for the probability density function of the compound version, when the claim severities are absolutely continuous, are derived. A multivariate version of the new distribution is proposed. For this multivariate version, we provide marginal distributions, the means vector, the covariance matrix and a simple formula for computing multivariate probabilities. Estimation methods are discussed. Finally, examples of application for both univariate and bivariate cases are given.     

Some asymptotic results for the combination of evidence problem

The combination of evidence problem is treated here as the construction of a posterior possibility function (or probability function, as a special case) describing an unknown state parameter vector of interest. This function exhibits the appropriate components contributing to knowledge of the parameter, including conditions or inference rules, relating the parameter with observable characteristics or attributes, and errors or confidences of observed or reported data. Multivalued logic operators - in particular, disjunction, conjunction, and implication operators, where needed – are used to connect these components and structure the posterior function. Typically, these operators are well-defined for only a finite number of arguments. Yet, often in the problem at hand, a number of observable attributes represent probabilistic concepts in the form of probability density functions. This occur, for example, for attributes representing ordinary numerical measurements- as opposed to those attributes representing linguistic-based information, where non-probabilistic possibility functions are used. Thus the problem of discretization of probabilistic attributes arises, where p.d.f.'s are truncated and discretized to probability functions. As the discretization process becomes finer and finer, intuitively the posterior function should better and better represent the information available. Hence, the basic question that arises is: what is the limiting behavior of the resulting posterior functions when the level of discretization becomes infinitely fine, and, in effect, the entire p.d.f.'s are used?It is shown in this paper that under mild analytic conditions placed upon the relevant functions and operators involved, nontrivial limits in the above sense do exist and involve monotone transforms of statistical expectations of functions of random variable corresponding to the p.d.f.'s for the probabilistic attributes.     

Inclusion–exclusion principle for belief functions

The inclusion–exclusion principle is a well-known property in probability theory, and is instrumental in some computational problems such as the evaluation of system reliability or the calculation of the probability of a Boolean formula in diagnosis. However, in the setting of uncertainty theories more general than probability theory, this principle no longer holds in general. It is therefore useful to know for which families of events it continues to hold. This paper investigates this question in the setting of belief functions. After exhibiting original sufficient and necessary conditions for the principle to hold, we illustrate its use on the uncertainty analysis of Boolean and non-Boolean systems in reliability.