Efficient Estimation of Spectral Functionals for Continuous-Time Stationary Models

The paper considers a problem of construction of asymptotically efficient estimators for functionals defined on a class of spectral densities, and bounding the minimax mean square risks. We define the concepts of H- and IK-efficiency of estimators, based on the variants of Hájek-Ibragimov-Khas’minskii convolution theorem and Hájek-Le Cam local asymptotic minimax theorem, respectively, and show that the simple “plug-in” statistic Φ(I _T ), where I _T =I _T (λ) is the periodogram of the underlying stationary Gaussian process X(t) with an unknown spectral density θ(λ), λ∈ℝ, is H- and IK-asymptotically efficient estimator for a linear functional Φ(θ), while for a nonlinear smooth functional Φ(θ) an H- and IK-asymptotically efficient estimator is the statistic F([^(q)]_T)\Phi(\widehat{\theta}_{T}), where [^(q)]_T\widehat{\theta}_{T} is a suitable sequence of the so-called “undersmoothed” kernel estimators of the unknown spectral density θ(λ). Exact asymptotic bounds for minimax mean square risks of estimators of linear functionals are also obtained.     

Estimating and Visualizing Conditional Densities

Abstract

We consider the kernel estimator of conditional density and derive its asymptotic bias, variance, and mean-square error. Optimal bandwidths (with respect to integrated mean-square error) are found and it is shown that the convergence rate of the density estimator is order n ^–2/3. We also note that the conditional mean function obtained from the estimator is equivalent to a kernel smoother. Given the undesirable bias properties of kernel smoothers, we seek a modified conditional density estimator that has mean equivalent to some other nonparametric regression smoother with better bias properties. It is also shown that our modified estimator has smaller mean square error than the standard estimator in some commonly occurring situations. Finally, three graphical methods for visualizing conditional density estimators are discussed and applied to a data set consisting of maximum daily temperatures in Melbourne, Australia.     

On the role of the shrinkage parameter in local linear smoothing

Summary. It has been shown that local linear smoothing possesses a variety of very attractive properties, not least being its mean square performance. However, such results typically refer only to asymptotic mean squared error, meaning the mean squared error of the asymptotic distribution, and in fact, the actual mean squared error is often infinite. See Seifert and Gasser (1996). This difficulty may be overcome by shrinking the local linear estimator towards another estimator with bounded mean square. However, that approach requires information about the size of the shrinkage parameter. From at least a theoretical viewpoint, very little is known about the effects of shrinkage. In particular, it is not clear how small the shrinkage parameter may be chosen without affecting first-order properties, or whether infinitely supported kernels such as the Gaussian require shrinkage in order to achieve first-order optimal performance. In the present paper we provide concise and definitive answers to such questions, in the context of general ridged and shrunken local linear estimators. We produce necessary and sufficient conditions on the size of the shrinkage parameter that ensure the traditional mean squared error formula. We show that a wide variety of infinitely-supported kernels, with tails even lighter than those of the Gaussian kernel, do not require any shrinkage at all in order to achieve traditional first-order optimal mean square performance. Received: 22 May 1995 / In revised form: 23 January 1997     

On one approach to the problem of robustness of density estimators

In this paper, we characterize the robustness of a density estimator by the square error increment caused by the “contamination” of a sample by a preassigned value ε. The dependence of this characteristic on ε is presented. A model example is considered. Translated fromStatisticheskie Metody Otsenivaniya i Proverki Gipotez, pp. 28–33, Perm, 1993.     

On a measure of integral square deviation with generalized weight for the Rosenblatt–Parzen probability density estimator

The limit distribution of an integral square deviation with weight in the form of “delta”-functions for the Rosenblatt–Parzen probability density estimator is determined. In addition, the limit power of the goodness-of-fit test constructed by using this deviation is investigated.     

Sharp adaptive estimation of quadratic functionals

Estimation of a quadratic functional of a function observed in the Gaussian white noise model is considered. A data-dependent method for choosing the amount of smoothing is given. The method is based on comparing certain quadratic estimators with each other. It is shown that the method is asymptotically sharp or nearly sharp adaptive simultaneously for the “regular” and “irregular” region. We consider l_p bodies and construct bounds for the risk of the estimator which show that for p=4 the estimator is exactly optimal and for example when p ∈[3,100], then the upper bound is at most 1.055 times larger than the lower bound. We show the connection of the estimator to the theory of optimal recovery. The estimator is a calibration of an estimator which is nearly minimax optimal among quadratic estimators. Writing of this article was financed by Deutsche Forschungsgemeinschaft under project MA1026/6-2, CIES, France, and Jenny and AnttiWihuri Foundation.     

Density Estimation with Replicate Heteroscedastic Measurements

We present a deconvolution estimator for the density function of a random variable from a set of independent replicate measurements. We assume that measurements are made with normally distributed errors having unknown and possibly heterogeneous variances. The estimator generalizes well-known deconvoluting kernel density estimators, with error variances estimated from the replicate observations. We derive expressions for the integrated mean squared error and examine its rate of convergence as n → ∞ and the number of replicates is fixed. We investigate the finite-sample performance of the estimator through a simulation study and an application to real data.     

Asymptotic bounds for the expected L error of a multivariate kernel density estimator

The kernel estimator of a multivariate probability density function is studied. An asymptotic upper bound for the expected L¹ error of the estimator is derived. An asymptotic lower bound result and a formula for the exact asymptotic error are also given. The goodness of the smoothing parameter value derived by minimizing an explicit upper bound is examined in numerical simulations that consist of two different experiments. First, the L¹ error is estimated using numerical integration and, second, the effect of the choice of the smoothing parameter in discrimination tasks is studied.     

On universal estimators in learning theory

This paper addresses the problem of constructing and analyzing estimators for the regression problem in supervised learning. Recently, there has been great interest in studying universal estimators. The term “universal” means that, on the one hand, the estimator does not depend on the a priori assumption that the regression function f _ρ belongs to some class F from a collection of classes F and, on the other hand, the estimation error for f _ρ is close to the optimal error for the class F. This paper is an illustration of how the general technique of constructing universal estimators, developed in the author’s previous paper, can be applied in concrete situations. The setting of the problem studied in the paper has been motivated by a recent paper by Smale and Zhou. The starting point for us is a kernel K(x, u) defined on X × Ω. On the base of this kernel, we build an estimator that is universal for classes defined in terms of nonlinear approximations with regard to the system {K(·, u)} _uεΩ. To construct an easily implementable estimator, we apply the relaxed greedy algorithm. Published in Russian in Trudy Matematicheskogo Instituta imeni V.A. Steklova, 2006, Vol. 255, pp. 256–272.     

Contribution to the bandwidth choice for kernel density estimates

In the present paper we focus on the problem of the bandwidth choice for the kernel density estimates. The problem of finding the optimal bandwidth belongs to the crucial problems of the kernel estimates. As a criterion of quality of the estimates the L ₂ type measure is used. A special iterative method based on a relevant estimation of mean integrated square error given in papers Müller and Wang (Prob Theor Relat Fields 85:523–538, 1990), Jones et al. (Ann Stat 19:1919–1932, 1991) is suggested. Moreover the idea of maximal smoothing principle (Terrell in J Am Stat Assoc 85:470–477, 1990) is extended to the higher order kernels. A simulation study brings a comparison of the proposed method and the cross-validation method. Research supported by the GACR:402/04/1308.     

Empirical Bayes Estimation in Regression Model

This paper considers the empirical Bayes （EB） estimation problem for the parameter β of the linear regression model y = Xβ＋ ε with ε- N（0, σ^2I） given β. Based on Pitman closeness （PC） criterion and mean square error matrix （MSEM） criterion, we prove the superiority of the EB estimator over the ordinary least square estimator （OLSE）.     

Estimation of the criticality parameters of branching processes by the Monte Carlo method

Monte Carlo algorithms designed for the estimation of the criticality parameters of multiplying particle transport processes (actually, these are inhomogeneous branching processes) are described and examined. The effective multiplication factor and the time multiplication constant are used as the basic criticality parameters. Algorithms for the direct simulation of “trees” of trajectories are considered as algorithms for the statistical modeling of the iterations of an integral operator with the kernel equal to the substochastic density of the transition to the next generation of fission events in the corresponding phase space. These algorithms provide a basis for constructing effective statistical estimates of the criticality parameters (with regard to the sequence of generations with different indexes) and for the analysis of the corresponding error.     

A graphical tool for selecting the number of slices and the dimension of the model in SIR and SAVE approaches

Sliced inverse regression (SIR) and related methods were introduced in order to reduce the dimensionality of regression problems. In general semiparametric regression framework, these methods determine linear combinations of a set of explanatory variables X related to the response variable Y, without losing information on the conditional distribution of Y given X. They are based on a “slicing step” in the population and sample versions. They are sensitive to the choice of the number H of slices, and this is particularly true for SIR-II and SAVE methods. At the moment there are no theoretical results nor practical techniques which allows the user to choose an appropriate number of slices. In this paper, we propose an approach based on the quality of the estimation of the effective dimension reduction (EDR) space: the square trace correlation between the true EDR space and its estimate can be used as goodness of estimation. We introduce a na?ve bootstrap estimation of the square trace correlation criterion to allow selection of an “optimal” number of slices. Moreover, this criterion can also simultaneously select the corresponding suitable dimension K (number of the linear combination of X). From a practical point of view, the choice of these two parameters H and K is essential. We propose a 3D-graphical tool, implemented in R, which can be useful to select the suitable couple (H, K). An R package named “edrGraphicalTools” has been developed. In this article, we focus on the SIR-I, SIR-II and SAVE methods. Moreover the proposed criterion can be use to determine which method seems to be efficient to recover the EDR space, that is the structure between Y and X. We indicate how the proposed criterion can be used in practice. A simulation study is performed to illustrate the behavior of this approach and the need for selecting properly the number H of slices and the dimension K. A short real-data example is also provided.     

Estimating the probability density function of a shift parameter

The kernel estimators of a prior density function of a shift parameter are proposed. The upper and lower bounds for the mean square error of these estimates are evaluated. Translated fromStatisticheskie Metody Otsenivaniya i Proverki Gipotez, pp. 39–46, Perm, 1991.     

The Poisson Kernel for Hardy Algebras

This note contributes to a circle of ideas that we have been developing recently in which we view certain abstract operator algebras H^∞(E), which we call Hardy algebras, and which are noncommutative generalizations of classical H^∞, as spaces of functions defined on their spaces of representations. We define a generalization of the Poisson kernel, which “reproduces” the values, on , of the “functions” coming from H^∞(E). We present results that are natural generalizations of the Poisson integral formula. They also are easily seen to be generalizations of formulas that Popescu developed. We relate our Poisson kernel to the idea of a characteristic operator function and show how the Poisson kernel identifies the “model space” for the canonical model that can be attached to a point in the disc . We also connect our Poisson kernel to various “point evaluations” and to the idea of curvature. The first named author was supported in part by grants from the National Science Foundation and from the U.S.-Israel Binational Science Foundation. The second named author was supported in part by the U.S.-Israel Binational Science Foundation and by the B. and G. Greenberg Research Fund (Ottawa).     

Regression estimation for bivariate normal distributions

Summary In estimating the mean μ _y of one variable in a bivariate normal distribution, the experimenter can use the other variable,x, as an auxiliary variable to increase precision. In particular, if μ _x is known, he can use the regression estimator. When μ _x is unknown, a preliminary test can be performed and the estimator will be made to depend on the result of the preliminary test. The bias and mean square error of the preliminary test estimator are obtained and the relative efficiency is are discussed.     

Integrated squared error of kernel-type estimator of distribution function

Let X ₁ ,...,X _n be a random sample drawn from distribution function F(x) with density function f(x) and suppose we want to estimate X(x). It is already shown that kernel estimator of F(x) is better than usual empirical distribution function in the sense of mean integrated squared error. In this paper we derive integrated squared error of kernel estimator and compare the error with that of the empirical distribution function. It is shown that the superiority of kernel estimators is not necessarily true in the sense of integrated squared error.     

Prediction Error for Continuous-Time Stationary Processes with Singular Spectral Densities

The paper considers the mean square linear prediction problem for some classes of continuous-time stationary Gaussian processes with spectral densities possessing singularities. Specifically, we are interested in estimating the rate of decrease to zero of the relative prediction error of a future value of the process using the finite past, compared with the whole past, provided that the underlying process is nondeterministic and is “close” to white noise. We obtain explicit expressions and asymptotic formulae for relative prediction error in the cases where the spectral density possess either zeros (the underlying model is an anti-persistent process), or poles (the model is a long memory processes). Our approach to the problem is based on the Krein’s theory of continual analogs of orthogonal polynomials and the continual analogs of Szeg? theorem on Toeplitz determinants. A key fact is that the relative prediction error can be represented explicitly by means of the so-called “parameter function” which is a continual analog of the Verblunsky coefficients (or reflection parameters) associated with orthogonal polynomials on the unit circle. To this end first we discuss some properties of Krein’s functions, state continual analogs of Szeg? “weak” theorem, and obtain formulae for the resolvents and Fredholm determinants of the corresponding Wiener-Hopf truncated operators.     

Minimax estimates of a linear parameter function in a regression model under restrictions on the parameters and variance-covariance matrix

Minimax nonhomogeneous linear estimators of scalar linear parameter functions are studied in the paper under restrictions on the parameters and variance-covariance matrix. The variance-covariance matrix of the linear model under consideration is assumed to be unknown but from a specific set R of nonnegativedefinite matrices. It is shown under this assumption that, without any restriction on the parameters, minimax estimators correspond to the least-squares estimators of the parameter functions for the “worst” variance-covariance matrix. Then the minimax mean-square error of the estimator is derived using the Bayes approach, and finally the exact formulas are derived for the calculation of minimax estimators under elliptical restrictions on the parameter space and for two special classes of possible variance-covariance matrices R. For example, it is shown that a special choice of a constant q ₀ and a matrixW ₀ defining one of the above classes R leads to the well known Kuks—Olman admissible estimator (see [16]) with a known variance-covariance matrixW ₀. Bibliography:32 titles. Translated fromObchyslyuval'na ta Prykladna Matematyka, No. 81, 1997, pp. 79–92.     

Tail index estimation for heavy tails: accommodation of bias in the excesses over a high threshold

In statistics of extremes, inference is often based on the excesses over a high random threshold. Those excesses are approximately distributed as the set of order statistics associated to a sample from a generalized Pareto model. We then get the so-called “maximum likelihood” estimators of the tail index γ. In this paper, we are interested in the derivation of the asymptotic distributional properties of a similar “maximum likelihood” estimator of a positive tail index γ, based also on the excesses over a high random threshold, but with a trial of accommodation of bias in the Pareto model underlying those excesses. We next proceed to an asymptotic comparison of the two estimators at their optimal levels. An illustration of the finite sample behaviour of the estimators is provided through a small-scale Monte Carlo simulation study. Research partially supported by FCT/POCTI and POCI/FEDER.