Integral Operator Approach to Learning Theory with Unbounded Sampling

This paper mainly focuses on the least square regularized regression learning algorithm in a setting of unbounded sampling. Our task is to establish learning rates by means of integral operators. By imposing a moment hypothesis on the unbounded sampling outputs and a function space condition associated with marginal distribution ?? _X , we derive learning rates which are consistent with those in the bounded sampling setting.     

Multiple sampling and interpolation in Bergman spaces

We study multiple sampling and interpolation problems with unbounded multiplicities in the weighted Bergman space, both in the hilbertian case and the uniform case.     

Concentration estimates for learning with unbounded sampling

The least-square regression problem is considered by regularization schemes in reproducing kernel Hilbert spaces. The learning algorithm is implemented with samples drawn from unbounded sampling processes. The purpose of this paper is to present concentration estimates for the error based on ?²-empirical covering numbers, which improves learning rates in the literature.     

Support vector machines regression with unbounded sampling

Uniform boundedness of output variables is a standard assumption in most theoretical analysis of regression algorithms. This standard assumption has recently been weaken to a moment hypothesis in least square regression (LSR) setting. Although there has been a large literature on error analysis for LSR under the moment hypothesis, very little is known about the statistical properties of support vector machines regression with unbounded sampling. In this paper, we fill the gap in the literature. Without any restriction on the boundedness of the output sampling, we establish an ad hoc convergence analysis for support vector machines regression under very mild conditions.     

Stochastic Accelerated Alternating Direction Method of Multipliers with Importance Sampling

In this paper, we incorporate importance sampling strategy into accelerated framework of stochastic alternating direction method of multipliers for solving a class of stochastic composite problems with linear equality constraint. The rates of convergence for primal residual and feasibility violation are established. Moreover, the estimation of variance of stochastic gradient is improved due to the use of important sampling. The proposed algorithm is capable of dealing with the situation, where the feasible set is unbounded. The experimental results indicate the effectiveness of the proposed method.     

ERM learning with unbounded sampling

The learning approach of empirical risk minimization (ERM) is taken for the regression problem in the least square framework. A standard assumption for the error analysis in the literature is the uniform boundedness of the output sampling process. In this paper we abandon this boundedness assumption and conduct error analysis for the ERM learning algorithm with unbounded sampling processes satisfying an increment condition for the moments of the output. The key novelty of our analysis is a covering number argument for estimating the sample error.     

Revisiting Landauʼs density theorems for Paley–Wiener spaces

We present a surprisingly simple approach to Landau?s density theorems for sampling and interpolation, which provides stronger versions of these results. In particular, we extend the interpolation theorem to unbounded spectra.     

The Generalized Multiset Sampler

The multiset sampler, an MCMC algorithm recently proposed by Leman and coauthors, is an easy-to-implement algorithm which is especially well-suited to drawing samples from a multimodal distribution. We generalize the algorithm by redefining the multiset sampler with an explicit link between target distribution and sampling distribution. The generalized formulation replaces the multiset with a K-tuple, which allows us to use the algorithm on unbounded parameter spaces, improves estimation, and sets up further extensions to adaptive MCMC techniques. Theoretical properties of the algorithm are provided and guidance is given on its implementation. Examples, both simulated and real, confirm that the generalized multiset sampler provides a simple, general and effective approach to sampling from multimodal distributions. Supplementary materials for this article are available online.     

Worst-Case Violation of Sampled Convex Programs for Optimization with Uncertainty

A deterministic approach called robust optimization has been recently proposed to deal with optimization problems including inexact data, i.e., uncertainty. The basic idea of robust optimization is to seek a solution that is guaranteed to perform well in terms of feasibility and near-optimality for all possible realizations of the uncertain input data. To solve robust optimization problems, Calafiore and Campi have proposed a randomized approach based on sampling of constraints, where the number of samples is determined so that only a small portion of the original constraints is violated by the randomized solution. Our main concern is not only the probability of violation, but also the degree of violation, i.e., the worst-case violation. We derive an upper bound of the worst-case violation for the sampled convex programs and consider the relation between the probability of violation and the worst-case violation. The probability of violation and the degree of violation are simultaneously bounded by a prescribed value when the number of random samples is large enough. In addition, a confidence interval of the optimal value is obtained when the objective function includes uncertainty. Our method is applicable to not only a bounded uncertainty set but also an unbounded one. Hence, the scope of our method includes random sampling following an unbounded distribution such as the normal distribution.     

无界抽样情形下不定核的系数正则化回归

本文讨论样本依赖空间中无界抽样情形下最小二乘损失函数的系数正则化问题. 这里的学习准则与之前再生核Hilbert空间的准则有着本质差异: 核除了满足连续性和有界性之外, 不需要再满足对称性和正定性; 正则化子是函数关于样本展开系数的l²-范数; 样本输出是无界的. 上述差异给误差分析增加了额外难度. 本文的目的是在样本输出不满足一致有界的情形下, 通过l²-经验覆盖数给出误差的集中估计(concentration estimates). 通过引入一个恰当的Hilbert空间以及l²-经验覆盖数的技巧, 得到了与假设空间的容量以及与回归函数的正则性有关的较满意的学习速率.     

Learning with coefficient-based regularization and ℓ1 −penalty

The least-square regression problem is considered by coefficient-based regularization schemes with ?¹??penalty. The learning algorithm is analyzed with samples drawn from unbounded sampling processes. The purpose of this paper is to present an elaborate concentration estimate for the algorithms by means of a novel stepping stone technique. The learning rates derived from our analysis can be achieved in a more general setting. Our refined analysis will lead to satisfactory learning rates even for non-smooth kernels.     

Constrained semi-Markov decision processes with ratio and time expected average criteria in Polish spaces

This paper deals with the ratio and time expected average criteria for constrained semi-Markov decision processes (SMDPs). The state and action spaces are Polish spaces, the rewards and costs are unbounded from above and from below, and the mean holding times are allowed to be unbounded from above. First, under general conditions we prove the existence of constrained-optimal policies for the ratio expected average criterion by developing a technique of occupation measures including the mean holding times for SMDPs, which are the generalizations of those for the standard discrete-time and continuous-time MDPs. Then, we give suitable conditions under which we establish the equivalence of the two average criteria by the optional sampling theorem, and thus we show the existence of constrained-optimal policies for the time expected average criterion. Finally, we illustrate the application of our main results with a controlled linear system, for which an exact optimal policy is obtained.     

Infinite horizon autonomous systems with unbounded cost

We discuss control systems defined on an infinite horizon, where typically all the associated costs become unbounded as the time grows indefinitely. It is proved, under certain lower semicontinuity and controllability assumptions, that a linear time function can be subtracted from the cost, resulting in a modified cost, which is bounded on the infinite time interval. The cost evaluated over one sampling interval has a simple representation in terms of the initial and final states. Applying this representation we obtain an optimality result for control systems represented by ordinary differential equations whose cost integrand contains a discounting factor.     

蒙特卡罗计算点通量的二次碰撞概率方法和重要抽样方法

1.引言 应用蒙特卡罗方法计算点通量,在粒子输运问题中占有非常重要的地位。首先,是由于点通量的计算问题在实际问题中经常遇到;其次,是由于任何局部通量计算问题均可通过点通量的计算实现;最后,是由于用其他数值方法计算点通量存在一点困难,尤其是对于那些几何以及其他因素复杂的问题更是如此。     

Bounded mean oscillation and bandlimited interpolation in the presence of noise

We study some problems related to the effect of bounded, additive sample noise in the bandlimited interpolation given by the Whittaker-Shannon-Kotelnikov (WSK) sampling formula. We establish a generalized form of the WSK series that allows us to consider the bandlimited interpolation of any bounded sequence at the zeros of a sine-type function. The main result of the paper is that if the samples in this series consist of independent, uniformly distributed random variables, then the resulting bandlimited interpolation almost surely has a bounded global average. In this context, we also explore the related notion of a bandlimited function with bounded mean oscillation. We prove some properties of such functions, and in particular, we show that they are either bounded or have unbounded samples at any positive sampling rate. We also discuss a few concrete examples of functions that demonstrate these properties.     

Cauchy-type integral formula and Plemelj formula of hypermonogenic functions on unbounded domains

In this paper, we discuss the Cauchy-type integral formula of hypermonogenic functions on unbounded domains in real Clifford analysis, then we extend the Plemelj formula and Cauchy–Pompeiu formula of hypermonogenic functions on bounded domains to unbounded domains. We also deal with the Green-type formula on unbounded domains and get several important corollaries.     

Dirichlet外问题的概率数值方法

针对定解区域是无界区域的Dirichlet外问题,提出了一种新的有效的概率数值方法,它是从解的随机表达式出发,将无界区域上的问题转化成区域边界上的问题.此时,只要在边界上进行剖分,将问题离散化,然后在无界区域外的有界区域内构作一个辅助球,并且利用布朗运动、漂移布朗运动从球外一点出发,首中球面的位置和时间的分布等,就可以获得Dirichlet外问题的数值解.     

On the solution of the Fokker-Planck equation on infinite domains using problem-specific orthonormal basis functions in a Galerkin-type method

Abstract: The dynamical behavior of a mechanical system under random excitation can be described by its probability density function (PDF). The PDF can be determined by random sampling or directly computed as the solution of the corresponding FOKKER-PLANCK equation (FPE). Since the class of FPE with known exact solution is very small, the solution is mostly approximated by numerical methods including finite elements, finite differences or other types of local discretization methods that are best suited for finite domains. Whilst this restriction can be desired for some models, the domain needs to be unbounded in the general case. The authors present a method of calculating a semi-analytical approximate solution for the stationary FPE over an unbounded domain using a GALERKIN-type method. The utilization of a problem-specific orthonormal basis for the ansatz-space ensures that only a small number of improper integrals needs to be computed. In fact, these computations are inexpensive and can be performed prior to the more costly algorithm that constructs the system of linear equations. The computation of the specific basis and its properties are discussed in detail. In order to demonstrate the method and its application on technical systems, the PDF for a nonlinear bistable magneto-piezo-electric energy harvester is calculated. (© 2014 Wiley-VCH Verlag GmbH & Co. KGaA, Weinheim)     

RAZUMIKHIN-TYPE THEOREM FOR NEUTRAL STOCHASTIC FUNCTIONAL DIFFERENTIAL EQUATIONS WITH UNBOUNDED DELAY

This paper establishes the Razumikhin-type theorem on stability for neutral stochastic functional differential equations with unbounded delay. To overcome difficulties from unbounded delay, we develop several different techniques to investigate stability. To show our idea clearly, we examine neutral stochastic delay differential equations with unbounded delay and linear neutral stochastic Volterra unbounded-delay-integro-differential equations.     

Quantile regression with ℓ 1—regularization and Gaussian kernels

The quantile regression problem is considered by learning schemes based on ? ₁—regularization and Gaussian kernels. The purpose of this paper is to present concentration estimates for the algorithms. Our analysis shows that the convergence behavior of ? ₁—quantile regression with Gaussian kernels is almost the same as that of the RKHS-based learning schemes. Furthermore, the previous analysis for kernel-based quantile regression usually requires that the output sample values are uniformly bounded, which excludes the common case with Gaussian noise. Our error analysis presented in this paper can give satisfactory convergence rates even for unbounded sampling processes. Besides, numerical experiments are given which support the theoretical results.