Estimation of the density of regression errors by pointwise model selection

This paper presents two results: a density estimator and an estimator of regression error density. We first propose a density estimator constructed by model selection, which is adaptive for the quadratic risk at a given point. Then we apply this result to estimate the error density in a homoscedastic regression framework Y _i = b(X _i ) + ε _i from which we observe a sample (X _i , Y _i ). Given an adaptive estimator $ \hat b $ \hat b of the regression function, we apply the density estimation procedure to the residuals $ \hat \varepsilon _i = Y_i - \hat b(X_i ) $ \hat \varepsilon _i = Y_i - \hat b(X_i ) . We get an estimator of the density of ε _i whose rate of convergence for the quadratic pointwise risk is the maximum of two rates: the minimax rate we would get if the errors were directly observed and the minimax rate of convergence of $ \hat b $ \hat b for the quadratic integrated risk.     

Prediction error criterion for selecting variables in a linear regression model

Several criteria, such as CV, C _p , AIC, CAIC, and MAIC, are used for selecting variables in linear regression models. It might be noted that C _p has been proposed as an estimator of the expected standardized prediction error, although the target risk function of CV might be regarded as the expected prediction error R _PE. On the other hand, the target risk function of AIC, CAIC, and MAIC is the expected log-predictive likelihood. In this paper, we propose a prediction error criterion, PE, which is an estimator of the expected prediction error R _PE. Consequently, it is also a competitor of CV. Results of this study show that PE is an unbiased estimator when the true model is contained in the full model. The property is shown without the assumption of normality. In fact, PE is demonstrated as more faithful for its risk function than CV. The prediction error criterion PE is extended to the multivariate case. Furthermore, using simulations, we examine some peculiarities of all these criteria.     

A Strong Representation of the Product-Limit Estimator for Left Truncated and Right Censored Data

In this paper we consider the TJW product-limit estimatorF_n(x) of an unknown distribution functionFwhen the data are subject to random left truncation and right censorship. An almost sure representation of PL-estimatorF_n(x) is derived with an improved error bound under some weaker assumptions. We obtain the strong approximation ofF_n(x)−F(x) by Gaussian processes and the functional law of the iterated logarithm is proved for maximal derivation of the product-limit estimator toF. A sharp rate of convergence theorem concerning the smoothed TJW product-limit estimator is obtained. Asymptotic properties of kernel estimators of density function based on TJW product-limit estimator is given.     

Simultaneous estimation of location parameters of the distribution with finite support

Summary LetX _i ,i=1,..., p be theith component of thep×1 vectorX=(X ₁,X ₂,...,X _p )′. Suppose thatX ₁,X ₂,...,X _p are independent and thatX _i has a probability density which is positive on a finite interval, is symmetric about θ _i and has the same variance. In estimation of the location vector θ=(θ₁, θ₂,...,θ _p )′ under the squared error loss function explicit estimators which dominateX are obtained by using integration by parts to evaluate the risk function. Further, explicit dominating estimators are given when the distributions ofX _i ′s are mixture of two uniform distributions. For the loss function such an estimator is also given when the distributions ofX _i ′s are uniform distributions.     

Integrated squared error of kernel-type estimator of distribution function

Let X ₁ ,...,X _n be a random sample drawn from distribution function F(x) with density function f(x) and suppose we want to estimate X(x). It is already shown that kernel estimator of F(x) is better than usual empirical distribution function in the sense of mean integrated squared error. In this paper we derive integrated squared error of kernel estimator and compare the error with that of the empirical distribution function. It is shown that the superiority of kernel estimators is not necessarily true in the sense of integrated squared error.     

The L2 norm of the approximation error for Bernstein polynomials

Wassily Hoeffding (J. Approximation Theory 4 (1971), 347–356) obtained a convergence rate for the L₁ norm of the approximation error, using Bernstein polynomials for a wide class of functions. Here, by a different method of proof, a similar result is obtained for the L₂ norm.     

Polyharmonic Approximation on the Sphere

The purpose of this article is to provide new error estimates for a popular type of spherical basis function (SBF) approximation on the sphere: approximating by linear combinations of Green’s functions of polyharmonic differential operators. We show that the L _p approximation order for this kind of approximation is σ for functions having L _p smoothness σ (for σ up to the order of the underlying differential operator, just as in univariate spline theory). This improves previous error estimates, which penalized the approximation order when measuring error in L _p , p>2 and held only in a restrictive setting when measuring error in L _p , p<2.     

Modelling and analysis of a software system with improvements in the testing phase

In this paper, we consider the error detection phenomena in the testing phase when modifications or improvements can be made to the software in the testing phase. The occurrence of improvements is described by a homogeneous Poisson process with intensity rate denoted by λ. The error detection phenomena is assumed to follow a nonhomogeneous Poisson process (NHPP) with the mean value function being denoted by m(t). Two models are presented and in one of the models, we have discussed an optimal release policy for the software taking into account the occurrences of errors and improvements. Finally, we discuss the possibility of an improvement removing k errors with probability p_k, k ≥ 0 in the software and develop a NHPP model for the error detection phenomena in this situation.     

Mean integrated squared error of kernel estimators when the density and its derivative are not necessarily continuous

Summary Asymptotic properties of the mean integrated squared error (MISE) of kernel estimators of a density function, based on a sampleX ₁, …,X _n, were obtained by Rosenblatt [4] and Epanechnikov [1] for the case when the densityf and its derivativef′ are continuous. They found, under certain additional regularity conditions, that the optimal choiceh _n0 for the scale factorh _n=Kn^−α is given byh _n0=K₀n^−1/5 withK ₀ depending onf and the kernel; they also showed that MISE(h _n0)=O(n^−4/5) and Epanechnikov [1] found the optimal kernel. In this paper we investigate the robustness of these results to departures from the assumptions concerning the smoothness of the density function. In particular it is shown, under certain regularity conditions, that whenf is continuous but its derivativef′ is not, the optimal value of α in the scale factor becomes 1/4 and MISE(h _n0)=O(n^−3/4); for the case whenf is not continuous the optimal value of α becomes 1/2 and MISE(h _n0)=O(n^−1/2). For this last case the optimal kernel is shown to be the double exponential density. Supported by the Natural Sciences and Engineering Research Council of Canada under Grant Nr. A 3114 and by the Gouvernement du Québec, Programme de formation de chercheurs et d'action concertée.     

Deconvolving a Density from Partially Contaminated Observations

We consider the problem of estimating a continuous bounded probability density function when independent data X₁, ..., X_n from the density are partially contaminated by measurement error. In particular, the observations Y₁, ..., Y_n are such that P(Y_j = X_j) = p and P(Y_j = X_j + ε_j) = 1 − p, where the errors ε_j are independent (of each other and of the X_j) and identically distributed from a known distribution. When p = 0 it is well known that deconvolution via kernel density estimators suffers from notoriously slow rates of convergence. For normally distributed ε_j the best possible rates are of logarithmic order pointwise and in mean square error. In this paper we demonstrate that for merely partially(0 < p <1) contaminated observations (where of course it is unknown which observations are contaminated and which are not) under mild conditions almost sure rates of order O(((log h⁻¹)/nh)^1/2) with h = h(n) = const(log n/n)^1/5 are achieved for convergence in L_∞-norm. This is equal to the optimal rate available in ordinary density estimation from direct uncontaminated observations (p = 1). A corresponding result is obtained for the mean integrated squared error.     

An algorithm for selecting a good value for the parameter c in radial basis function interpolation

The accuracy of many schemes for interpolating scattered data with radial basis functions depends on a shape parameter c of the radial basis function. In this paper we study the effect of c on the quality of fit of the multiquadric, inverse multiquadric and Gaussian interpolants. We show, numerically, that the value of the optimal c (the value of c that minimizes the interpolation error) depends on the number and distribution of data points, on the data vector, and on the precision of the computation. We present an algorithm for selecting a good value for c that implicitly takes all the above considerations into account. The algorithm selects c by minimizing a cost function that imitates the error between the radial interpolant and the (unknown) function from which the data vector was sampled. The cost function is defined by taking some norm of the error vector E = (E ₁, ... , E_N)^T where E _k = E_k = f_k - S^k x_k) and S _k is the interpolant to a reduced data set obtained by removing the point x _k and the corresponding data value f _k from the original data set. The cost function can be defined for any radial basis function and any dimension. We present the results of many numerical experiments involving interpolation of two dimensional data sets by the multiquadric, inverse multiquadric and Gaussian interpolants and we show that our algorithm consistently produces good values for the parameter c. This revised version was published online in June 2006 with corrections to the Cover Date.     

Asymptotic Behavior of the Perturbed Empirical Distribution-Functions Evaluated at a Random Point for Absolutely Regular Sequences

Let _n be an estimator obtained by integrating a kernel type density estimator based on a random sample of size n from smooth distribution function F. A central limit theorem is established for the target statistic _n(U_n) where the underlying random variable form an absolutely regular stationary process and where {U_n} is a sequence of U-statistics. The result obtained generalizes Puri and Ralescu (1986, J. Multivariate Anal.19, 273-279) under the iid set-up.     

On a class of renewal risk model with random income

In this paper, we consider a renewal risk process with random premium income based on a Poisson process. Generating function for the discounted penalty function is obtained. We show that the discounted penalty function satisfies a defective renewal equation and the corresponding explicit expression can be obtained via a compound geometric tail. Finally, we consider the Laplace transform of the time to ruin, and derive the closed‐form expression for it when the claims have a discrete K_m distribution (i.e. the generating function of the distribution function is a ratio of two polynomials of order m∈?⁺). Copyright © 2008 John Wiley & Sons, Ltd.     

Estimating a density and its derivatives via the minimum distance method

Summary This paper deals with minimum distance (MD) estimators and minimum penalized distance (MPD) estimators which are based on the L _p distance. Rates of strong consistency of MPD density estimators are established within the family of density functions which have a bounded m-th derivative. For the case p=2, it is also proved that the MPD density estimator achieves the optimum rate of decrease of the mean integrated square error and the L ₁ error. Estimation of derivatives of the density is considered as well.In a class parametrized by entire functions, it is proved that the rate of convergence of the MD density estimator (and its derivatives) to the unknown density (its derivatives) is of order in expected L ₁ and L ₂ distances. In the same class of distributions, MD estimators of unknown density and its derivatives are proved to achieve an extraordinary rate (log log n/n)^1/2 of strong consistency.     

Asymptotic expansions of the Robbins-Monro process

Assume that the function values f(x) of an unknown regression function f: ℝ → ℝ can be observed with some random error V. To estimate the zero ϑ of f, Robbins and Monro suggested to run the recursion X _n+1 = X _n − a/n Y _n with Y _n = f(X _n ) − V _n . Under regularity assumptions, the normalized Robbins-Monro process, given by (X _n+1 − ϑ)/√Var(X _n+1, is asymptotically standard normal. In this paper Edgeworth expansions are presented which provide approximations of the distribution function up to an error of order o(1/√n) or even o(1/n). As corollaries asymptotic confidence intervals for the unknown parameter ϑ are obtained with coverage probability errors of order O(1/n). Further results concern Cornish-Fisher expansions of the quantile function, an Edgeworth correction of the distribution function and a stochastic expansion in terms of a bivariate polynomial in 1/√n and a standard normal random variable. The proofs of this paper heavily rely on recently published results on Edgeworth expansions for approximations of the Robbins-Monro process.      

On the error of quadrature formulae for Cauchy principal value integrals based on piecewise interpolation

We consider the computation of the Cauchy principal value integral by quadrature formulae Q _n ^F [f] of compound type, which are obtained by replacing f by a piecewise defined function F_n[f]. The behaviour of the constants k_{i, n} in the estimates [R _n ^F [f]] |⩽K _i,n ‖f ⁽ⁱ⁾‖_∞ (where R _n ^F [f] is the quadrature error) is determined for fixed i and n→∞, which means that not only the order, but also the coefficient of the main term of k_{i, n} is determined. The behaviour of these error constants k_{i, n} is compared with the corresponding ones obtained for the method of subtraction of the singularity. As it turns out, these error constants have, in general, the same asymptotic behaviour.     

A Chebyshev Quadrature Rule for One Sided Finite Part Integrals

This paper is concerned with a Chebyshev quadrature rule for approximating one sided finite part integrals with smooth density functions. Our quadrature rule is based on the Chebyshev interpolation polynomial with the zeros of the Chebyshev polynomial T_N+1(τ)−T_N−1(t). We analyze the stability and the convergence for the quadrature rule with a differentiable function. Also we show that the quadrature rule has an exponential convergence when the density function is analytic.     

Sharp Estimates for the Deviation of the Mean Value of a Periodic Function in Terms of Moduli of Continuity of Higher Order

Estimates from above for the uniform deviation of the mean value of a periodic function and the best approximation by constants are obtained on some classes of functions defined by moduli of continuity of even order. Similar results are established for approximations in the space L ₂ and for the error of rectangular formula. Bibliography: 10 titles.     

Nonasymptotic Bounds on the L 2 Error of Neural Network Regression Estimates

The estimation of multivariate regression functions from bounded i.i.d. data is considered. The L ₂ error with integration with respect to the design measure is used as an error criterion. The distribution of the design is assumed to be concentrated on a finite set. Neural network estimates are defined by minimizing the empirical L ₂ risk over various sets of feedforward neural networks. Nonasymptotic bounds on the L ₂ error of these estimates are presented. The results imply that neural networks are able to adapt to additive regression functions and to regression functions which are a sum of ridge functions, and hence are able to circumvent the curse of dimensionality in these cases.     

Asymptotic properties of some goodness-of-fit tests based on the L 1-norm

Some goodness-of-fit tests based on the L ₁-norm are considered. The asymptotic distribution of each statistic under the null hypothesis is the distribution of the L ₁-norm of the standard Wiener process on [0,1]. The distribution function, the density function and a table of some percentage points of the distribution are given. A result for the asymptotic tail probability of the L ₁-norm of a Gaussian process is also obtained. The result is useful for giving the approximate Bahadur efficiency of the test statistics whose asymptotic distributions are represented as the L ₁-norms of Gaussian processes.