On Uniform Laws of Large Numbers for Ergodic Diffusions and Consistency of Estimators

Consider a regular diffusion process X with finite speed measure m. Denote the normalized speed measure by μ. We prove that the uniform law of large numbers holds if the class has an envelope function that is μ-integrable, or if is bounded in L _p(μ) for some p>1. In contrast with uniform laws of large numbers for i.i.d. random variables, we do not need conditions on the ‘size’ of the class in terms of bracketing or covering numbers. The result is a consequence of a number of asymptotic properties of diffusion local time that we derive. We apply our abstract results to improve consistency results for the local time estimator (LTE) and to prove consistency for a class of simple M-estimators. This revised version was published online in June 2006 with corrections to the Cover Date.     

Glivenko-Cantelli type theorems: An application of the convergence theory of stochastic suprema

The uniform convergence of empirical processes on certain classes of sets follows from the convergence theory for random lower semicontinuous functions studies in the context of stochastic optimization. In the process, a richer class of sets for which one can prove this type of result is exhibited.Research supported in part by grants from Ministero Publica Istruzione and the National Science Foundation.     

Reserve-dependent benefits and costs in life and health insurance contracts

Premiums and benefits associated with traditional life insurance contracts are usually specified as fixed amounts in policy conditions. However, reserve-dependent surrender values and reserve-dependent expenses are common in insurance practice. The famous Cantelli theorem in life insurance ensures that under appropriate assumptions surrendering can be ignored in reserve calculations provided that the surrender payment equals the accumulated reserve. In this paper, more complex reserve-dependent payment patterns are considered, in line with insurance practice. Explicit formulas are derived for the corresponding reserve.     

Upper and Lower Bounds for Probabilities in the Conditional Borel–Cantelli Lemma

We obtain an inequality connected with a conditional version of the generalized Borel–Cantelli lemma.     

Game-theoretic versions of strong law of large numbers for unbounded variables

We consider strong law of large numbers (SLLN) in the framework of game-theoretic probability of Shafer and Vovk (Shafer, G. and Vovk, V. 2001, Probability and Finance: It's Only a Game! (New York: Wiley)). We prove several versions of SLLN for the case that Reality's moves are unbounded. Our game-theoretic versions of SLLN largely correspond to standard measure-theoretic results. However game-theoretic proofs are different from measure-theoretic ones in the explicit consideration of various hedges. In measure-theoretic proofs existence of moments is assumed, whereas in our game-theoretic proofs we assume availability of various hedges to Skeptic for finite prices.     

Learning theory: stability is sufficient for generalization and necessary and sufficient for consistency of empirical risk minimization

Solutions of learning problems by Empirical Risk Minimization (ERM) – and almost-ERM when the minimizer does not exist – need to be consistent, so that they may be predictive. They also need to be well-posed in the sense of being stable, so that they might be used robustly. We propose a statistical form of stability, defined as leave-one-out (LOO) stability. We prove that for bounded loss classes LOO stability is (a) sufficient for generalization, that is convergence in probability of the empirical error to the expected error, for any algorithm satisfying it and, (b) necessary and sufficient for consistency of ERM. Thus LOO stability is a weak form of stability that represents a sufficient condition for generalization for symmetric learning algorithms while subsuming the classical conditions for consistency of ERM. In particular, we conclude that a certain form of well-posedness and consistency are equivalent for ERM. Dedicated to Charles A. Micchelli on his 60th birthday Mathematics subject classifications (2000) 68T05, 68T10, 68Q32, 62M20. Tomaso Poggio: Corresponding author.     

Convergence of Empirical Processes for Interacting Particle Systems with Applications to Nonlinear Filtering

In this paper, we investigate the convergence of empirical processes for a class of interacting particle numerical schemes arising in biology, genetic algorithms and advanced signal processing. The Glivenko–Cantelli and Donsker theorems presented in this work extend the corresponding statements in the classical theory and apply to a class of genetic type particle numerical schemes of the nonlinear filtering equation.     

On the use of the Borel–Cantelli lemma in Markov chains

In the present paper, we propose technical generalizations of the Borel–Cantelli lemma. These generalizations can be further used to derive strong limit results for Markov chains. In our work, we obtain some strong limit results.     

Upper and lower bounds of Borel–Cantelli Lemma in a general measure space

It is known that the Borel–Cantelli Lemma plays an important role in probability theory. Many attempts were made to generalize its second part. In this article, we investigate the upper and lower bounds of Borel–Cantelli Lemma for the nonnegative functions in a general measure space. Our results extend the corresponding results obtained in a probability space. Some examples including dependent random variables are illustrated to our results.     

ℓ-norm and its Application to Learning Theory

We investigate connections between an important parameter in the theory of Banach spaces called the -norm, and two properties of classes of functions which are essential in Learning Theory – the uniform law of large numbers and the Vapnik–Chervonenkis (VC) dimension. We show that if the -norm of a set of functions is bounded in some sense, then the set satisfies the uniform law of large numbers. Applying this result, we show that if X is a Banach space which has a nontrivial type, then the unit ball of its dual satisfies the uniform law of large numbers. Next, we estimate the -norm of a set of {0,1}-functions in terms of its VC dimension. Finally, we present a `Gelfand number' like estimate of certain classes of functions. We use this estimate to formulate a learning rule, which may be used to approximate functions from the unit balls of several Banach spaces.