Design Adaptive Nearest Neighbor Regression Estimation

This paper deals with nonparametric regression estimation under arbitrary sampling with an unknown distribution. The effect of the distribution of the design, which is a nuisance parameter, can be eliminated by conditioning. An upper bound for the conditional mean squared error of k−NN estimates leads us to consider an optimal number of neighbors, which is a random function of the sampling. The corresponding estimate can be used for nonasymptotic inference and is also consistent under a minimal recurrence condition. Some deterministic equivalents are found for the random rate of convergence of this optimal estimate, for deterministic and random designs with vanishing or diverging densities. The proposed estimate is rate optimal for standard designs.     

Stochastic Convexity of the Poisson Mixture Model

This paper is devoted to the study of the compound Poisson mixture model in an actuarial framework. Using the s-convex stochastic orderings and stochastic s-convexity, several problems involving an unknown mixing parameter with given moments are examined; namely, the specification of the number of support points in a finite mixture model, and the derivation of extremal mixture distributions. The theory is enhanced with the derivation of theoretical and numerical bounds on several quantities of actuarial interest.     

Étude de la consistance d'un algorithme stochastique dans un cadre mélangeant

This Note is devoted to the study of the consistency of a stochastic algorithm having for object the estimates of a parameter under strong mixing assumption. To cite this article: A. Ait Saidi, A. Dahmani, C. R. Acad. Sci. Paris, Ser. I 340 (2005).     

A sparsity preserving stochastic gradient methods for sparse regression

We propose a new stochastic first-order algorithm for solving sparse regression problems. In each iteration, our algorithm utilizes a stochastic oracle of the subgradient of the objective function. Our algorithm is based on a stochastic version of the estimate sequence technique introduced by Nesterov (Introductory lectures on convex optimization: a basic course, Kluwer, Amsterdam, 2003). The convergence rate of our algorithm depends continuously on the noise level of the gradient. In particular, in the limiting case of noiseless gradient, the convergence rate of our algorithm is the same as that of optimal deterministic gradient algorithms. We also establish some large deviation properties of our algorithm. Unlike existing stochastic gradient methods with optimal convergence rates, our algorithm has the advantage of readily enforcing sparsity at all iterations, which is a critical property for applications of sparse regressions.     

On adaptive wavelet estimation of a quadratic functional from a deconvolution problem

We observe a stochastic process where a convolution product of an unknown function and a known function is corrupted by Gaussian noise. We wish to estimate the squared \mathbbL²{\mathbb{L}^2} -norm of the unknown function from the observations. To reach this goal, we develop adaptive estimators based on wavelet and thresholding. We prove that they achieve (near) optimal rates of convergence under the mean squared error over a wide range of smoothness classes.     

Strong consistency of recursive identification by no use of persistent excitation condition

For the discrete-time stochastic system without monitoring, the strong consistency of the estimate given by the stochastic gradient algorithm is considered when the persistent excitation condition is possibly not fulfilled. In addition, the convergence rate is given for a specific class of system noises, while in the adaptive tracking case the convergence rate for the parameter estimates as well as the tracking error are obtained when the reference signal is disturbed by a dither.Institute of Systems Science, Academia Sinica     

Fixed design regression for time series: Asymptotic normality

Consider the fixed regression model with general weights, and suppose that the error random variables are coming from a strictly stationary stochastic process, satisfying the strong mixing condition. The asymptotic normality of the proposed estimate is established under weak conditions. The applicability of the results obtained is demonstrated by way of two existing estimates, the Gasser-Müller estimate and that of Priestley and Chao. The asymptotic normality of these estimates is further illustrated by means of a concrete example from the class of autoregressive processes.     

Efficiency of the estimate refinement method for polyhedral approximation of multidimensional balls

The estimate refinement method for the polyhedral approximation of convex compact bodies is analyzed. When applied to convex bodies with a smooth boundary, this method is known to generate polytopes with an optimal order of growth of the number of vertices and facets depending on the approximation error. In previous studies, for the approximation of a multidimensional ball, the convergence rates of the method were estimated in terms of the number of faces of all dimensions and the cardinality of the facial structure (the norm of the f-vector) of the constructed polytope was shown to have an optimal rate of growth. In this paper, the asymptotic convergence rate of the method with respect to faces of all dimensions is compared with the convergence rate of best approximation polytopes. Explicit expressions are obtained for the asymptotic efficiency, including the case of low dimensions. Theoretical estimates are compared with numerical results.     

Minimum distance regression-type estimates with rates under weak dependence

Under weak dependence, a minimum distance estimate is obtained for a smooth function and its derivatives in a regression-type framework. The upper bound of the risk depends on the Kolmogorov entropy of the underlying space and the mixing coefficient. It is shown that the proposed estimates have the same rate of convergence, in the L ₁-norm sense, as in the independent case.This work was partially supported by a research grant from the Natural Sciences and Engineering Research Council of Canada.     

Approximation of some stochastic differential equations by the splitting up method

In this paper we deal with the convergence of some iterative schemes suggested by Lie-Trotter product formulas for stochastic differential equations of parabolic type. The stochastic equation is split into two problems which are simpler for numerical computations, as already shown, for example, for the Zakaï equation. An estimate of the approximation error is given in a particular case.The work of A. Bensoussan and R. Glowinski was supported by the U.S. Army Research Office under Contract DAAL03-86-K-0138. Additional support was given by NSF via Grant INT-8612680.     

An Approximation for the Zakai Equation

In this article we consider a polygonal approximation to the unnormalized conditional measure of a filtering problem, which is the solution of the Zakai stochastic differential equation on measure space. An estimate of the convergence rate based on a distance which is equivalent to the weak convergence topology is derived. We also study the density of the unnormalized conditional measure, which is the solution of the Zakai stochastic partial differential equation. An estimate of the convergence rate is also given in this case. 60F25, 60H10.} Accepted 23 April 2001. Online publication 14 August 2001.     

Semiparametric mixtures of nonparametric regressions

In this article, we propose and study a new class of semiparametric mixture of regression models, where the mixing proportions and variances are constants, but the component regression functions are smooth functions of a covariate. A one-step backfitting estimate and two EM-type algorithms have been proposed to achieve the optimal convergence rate for both the global parameters and the nonparametric regression functions. We derive the asymptotic property of the proposed estimates and show that both the proposed EM-type algorithms preserve the asymptotic ascent property. A generalized likelihood ratio test is proposed for semiparametric inferences. We prove that the test follows an asymptotic \(\chi ^2\)-distribution under the null hypothesis, which is independent of the nuisance parameters. A simulation study and two real data examples have been conducted to demonstrate the finite sample performance of the proposed model.     

Algorithm for the continuous estimation of a disturbance in a stochastic differential equation

Based on the approach of the theory of dynamic inversion, the problem of continuous estimation of an unknown deterministic disturbance in an Ito stochastic differential equation is investigated with the use of inaccurate measurements of the current phase state. An auxiliary model equation with a control approximating the unknown input is derived. The suggested solution algorithm is constructive; an estimate for its convergence rate is written explicitly.     

On Monte Carlo methods for Bayesian multivariate regression models with heavy-tailed errors

We consider Bayesian analysis of data from multivariate linear regression models whose errors have a distribution that is a scale mixture of normals. Such models are used to analyze data on financial returns, which are notoriously heavy-tailed. Let π denote the intractable posterior density that results when this regression model is combined with the standard non-informative prior on the unknown regression coefficients and scale matrix of the errors. Roughly speaking, the posterior is proper if and only if n≥d+k, where n is the sample size, d is the dimension of the response, and k is number of covariates. We provide a method of making exact draws from π in the special case where n=d+k, and we study Markov chain Monte Carlo (MCMC) algorithms that can be used to explore π when n>d+k. In particular, we show how the Haar PX-DA technology studied in Hobert and Marchev (2008) [11] can be used to improve upon Liu’s (1996) [7] data augmentation (DA) algorithm. Indeed, the new algorithm that we introduce is theoretically superior to the DA algorithm, yet equivalent to DA in terms of computational complexity. Moreover, we analyze the convergence rates of these MCMC algorithms in the important special case where the regression errors have a Student’s t distribution. We prove that, under conditions on n, d, k, and the degrees of freedom of the t distribution, both algorithms converge at a geometric rate. These convergence rate results are important from a practical standpoint because geometric ergodicity guarantees the existence of central limit theorems which are essential for the calculation of valid asymptotic standard errors for MCMC based estimates.     

Constants in the estimates of the convergence rate in the Birkhoff ergodic theorem with continuous time

For the case of continuous time, we prove inequalities that make it possible, in the presence of power-type estimates of the convergence rate in the von Neumann ergodic theorem, to obtain estimates for the convergence rate in the Birkhoff ergodic theorem. These results have obvious exact analogs in the class of wide-sense stationary stochastic processes.     

Error Estimates for a Stochastic Impulse Control Problem

We obtain error bounds for monotone approximation schemes of a stochastic impulse control problem. This is an extension of the theory for error estimates for the Hamilton-Jacobi-Bellman equation. We obtain almost the same estimate on the rate of convergence as in the equation without impulsions [2], [3].     

Stochastic optimization with averaging of trajectories

A recursive stochastic optimization procedure under dependent disturbances is studied. It is based on the Polyak-Ruppert algorithm with trajectory averaging. Almost sure convergence of the algorithm is proved as well as asymptotic normality of the delivered estimates. It is shown that the presented algorithm attains the highest possible asymptotic convergence rate for stochastic approximation algorithms     

Convergence rate of numerical solutions to SFDEs with jumps

In this paper, we are interested in numerical solutions of stochastic functional differential equations with jumps. Under a global Lipschitz condition, we show that the pth-moment convergence of Euler-Maruyama numerical solutions to stochastic functional differential equations with jumps has order 1/p for any p≥2. This is significantly different from the case of stochastic functional differential equations without jumps, where the order is 1/2 for any p≥2. It is therefore best to use the mean-square convergence for stochastic functional differential equations with jumps. Moreover, under a local Lipschitz condition, we reveal that the order of mean-square convergence is close to 1/2, provided that local Lipschitz constants, valid on balls of radius j, do not grow faster than logj.     

多级评分及其Bayes估计

对多级评分的测验题型 ,给出了其Bayes模型 ,在无信息先验分布或先验分布是Dirichlet分布情形下求出了参数的Bayes估计 ,并对后者在不同样本条件下给出了先验分布超参数的估计