Asymptotic minimax risk for sup-norm loss: Solution via optimal recovery

Summary We study the problem of estimating an unknown function on the unit interval (or itsk-th derivative), with supremum norm loss, when the function is observed in Gaussian white noise and the unknown function is known only to obey Lipschitz- smoothness, >k0. We discuss an optimization problem associated with the theory ofoptimal recovery. Although optimal recovery is concerned with deterministic noise chosen by a clever opponent, the solution of this problem furnishes the kernel of the minimax linear estimate for Gaussian white noise. Moreover, this minimax linear estimator is asymptotically minimax among all estimates. We sketch also applications to higher dimensions and to indirect measurement (e.g. deconvolution) problems.Dedicated to R.Z. Khas'minskii for his 60th birthday     

Optimal prediction for linear regression with infinitely many parameters

The problem of optimal prediction in the stochastic linear regression model with infinitely many parameters is considered. We suggest a prediction method that outperforms asymptotically the ordinary least squares predictor. Moreover, if the random errors are Gaussian, the method is asymptotically minimax over ellipsoids in ?₂. The method is based on a regularized least squares estimator with weights of the Pinsker filter. We also consider the case of dynamic linear regression, which is important in the context of transfer function modeling.     

Thresholding projection estimators in functional linear models

We consider the problem of estimating the regression function in functional linear regression models by proposing a new type of projection estimators which combine dimension reduction and thresholding. The introduction of a threshold rule allows us to get consistency under broad assumptions as well as minimax rates of convergence under additional regularity hypotheses. We also consider the particular case of Sobolev spaces generated by the trigonometric basis which permits us to get easily mean squared error of prediction as well as estimators of the derivatives of the regression function. We prove that these estimators are minimax and rates of convergence are given for some particular cases.     

Exact asymptotic minimax constants for the estimation of analytical functions in L p

The L _p minimax risks (1≤p<∞) are studied for statistical estimation in the Gaussian white noise model. The asymptotic rate and constants are given, and the optimal estimator is proposed. This, together with the work of Golubev, Levit and Tsybakov (1996) establishes the classification of the L _p minimax constants on the classes of analytical functions. Received: 10 December 1996 / Revised version: 14 December 1997     

Sharp adaptive estimation of quadratic functionals

Estimation of a quadratic functional of a function observed in the Gaussian white noise model is considered. A data-dependent method for choosing the amount of smoothing is given. The method is based on comparing certain quadratic estimators with each other. It is shown that the method is asymptotically sharp or nearly sharp adaptive simultaneously for the “regular” and “irregular” region. We consider l_p bodies and construct bounds for the risk of the estimator which show that for p=4 the estimator is exactly optimal and for example when p ∈[3,100], then the upper bound is at most 1.055 times larger than the lower bound. We show the connection of the estimator to the theory of optimal recovery. The estimator is a calibration of an estimator which is nearly minimax optimal among quadratic estimators. Writing of this article was financed by Deutsche Forschungsgemeinschaft under project MA1026/6-2, CIES, France, and Jenny and AnttiWihuri Foundation.     

Statistical approach to some ill-posed problems for linear partial differential equations

For linear partial differential equations, some inverse source problems are treated statistically based on nonparametric estimation ideas. By observing the solution in a small Gaussian white noise, the kernel type of estimators is used to estimate the unknown source function and its partial derivatives.. It is proved that such estimators are consistent as the noise intensity tends to zero. Depending on the principal part of the differential operator, the optimal asymptotic rate of convergence is ascertained within a wide class of risk functions in a minimax sense. Received: 5 May 1997 / Revised version: 18 June 1998     

Variance function estimation in multivariate nonparametric regression with fixed design

Variance function estimation in multivariate nonparametric regression is considered and the minimax rate of convergence is established in the iid Gaussian case. Our work uses the approach that generalizes the one used in [A. Munk, Bissantz, T. Wagner, G. Freitag, On difference based variance estimation in nonparametric regression when the covariate is high dimensional, J. R. Stat. Soc. B 67 (Part 1) (2005) 19-41] for the constant variance case. As is the case when the number of dimensions d=1, and very much contrary to standard thinking, it is often not desirable to base the estimator of the variance function on the residuals from an optimal estimator of the mean. Instead it is desirable to use estimators of the mean with minimal bias. Another important conclusion is that the first order difference based estimator that achieves minimax rate of convergence in the one-dimensional case does not do the same in the high dimensional case. Instead, the optimal order of differences depends on the number of dimensions.     

Asymptotic equivalence for nonparametric generalized linear models

Summary. We establish that a non-Gaussian nonparametric regression model is asymptotically equivalent to a regression model with Gaussian noise. The approximation is in the sense of Le Cam's deficiency distance Δ; the models are then asymptotically equivalent for all purposes of statistical decision with bounded loss. Our result concerns a sequence of independent but not identically distributed observations with each distribution in the same real-indexed exponential family. The canonical parameter is a value f(t _i ) of a regression function f at a grid point t _i (nonparametric GLM). When f is in a H?lder ball with exponent we establish global asymptotic equivalence to observations of a signal Γ(f(t)) in Gaussian white noise, where Γ is related to a variance stabilizing transformation in the exponential family. The result is a regression analog of the recently established Gaussian approximation for the i.i.d. model. The proof is based on a functional version of the Hungarian construction for the partial sum process. Received: 4 February 1997     

Sharp adaptation for spherical inverse problems with applications to medical imaging

This paper examines the estimation of an indirect signal embedded in white noise for the spherical case. It is found that the sharp minimax bound is determined by the degree to which the indirect signal is embedded in the linear operator. Thus, when the linear operator has polynomial decay, recovery of the signal is polynomial, whereas if the linear operator has exponential decay, recovery of the signal is logarithmic. The constants are determined for these classes as well. Adaptive sharp estimation is also carried out. In the polynomial case a blockwise shrinkage estimator is needed while in the exponential case, a straight projection estimator will suffice. The framework of this paper include applications to medical imaging, in particular, to cone beam image reconstruction and to diffusion magnetic resonance imaging. Discussion of these applications are included.     

Spline-based sieve maximum likelihood estimation in the partly linear model under monotonicity constraints

We study a spline-based likelihood method for the partly linear model with monotonicity constraints. We use monotone B-splines to approximate the monotone nonparametric function and apply the generalized Rosen algorithm to compute the estimators jointly. We show that the spline estimator of the nonparametric component achieves the possible optimal rate of convergence under the smooth assumption and that the estimator of the regression parameter is asymptotically normal and efficient. Moreover, a spline-based semiparametric likelihood ratio test is established to make inference of the regression parameter. Also an observed profile information method to consistently estimate the standard error of the spline estimator of the regression parameter is proposed. A simulation study is conducted to evaluate the finite sample performance of the proposed method. The method is illustrated by an air pollution study.     

Testing model assumptions in functional regression models

In the functional regression model where the responses are curves, new tests for the functional form of the regression and the variance function are proposed, which are based on a stochastic process estimating L²-distances. Our approach avoids the explicit estimation of the functional regression and it is shown that normalized versions of the proposed test statistics converge weakly. The finite sample properties of the tests are illustrated by means of a small simulation study. It is also demonstrated that for small samples, bootstrap versions of the tests improve the quality of the approximation of the nominal level.     

Robustness of Stein-type estimators under a non-scalar error covariance structure

The Stein-rule (SR) and positive-part Stein-rule (PSR) estimators are two popular shrinkage techniques used in linear regression, yet very little is known about the robustness of these estimators to the disturbances’ deviation from the white noise assumption. Recent studies have shown that the OLS estimator is quite robust, but whether this is so for the SR and PSR estimators is less clear as these estimators also depend on the F statistic which is highly susceptible to covariance misspecification. This study attempts to evaluate the effects of misspecifying the disturbances as white noise on the SR and PSR estimators by a sensitivity analysis. Sensitivity statistics of the SR and PSR estimators are derived and their properties are analyzed. We find that the sensitivity statistics of these estimators exhibit very similar properties and both estimators are extremely robust to MA(1) disturbances and reasonably robust to AR(1) disturbances except for the cases of severe autocorrelation. The results are useful in light of the rising interest of the SR and PSR techniques in the applied literature.     

Adaptive minimax testing in the discrete regression scheme

We consider the problem of testing hypotheses on the regression function from n observations on the regular grid on [0,1]. We wish to test the null hypothesis that the regression function belongs to a given functional class (parametric or even nonparametric) against a composite nonparametric alternative. The functions under the alternative are separated in the L₂-norm from any function in the null hypothesis. We assume that the regression function belongs to a wide range of Hölder classes but as the smoothness parameter of the regression function is unknown, an adaptive approach is considered. It leads to an optimal and unavoidable loss of order Open image in new window in the minimax rate of testing compared with the non-adaptive setting. We propose a smoothness-free test that achieves the optimal rate, and finally we prove the lower bound showing that no test can be consistent if in the distance between the functions under the null hypothesis and those in the alternative, the loss is of order smaller than the optimal loss.     

A lack-of-fit test in Tobit errors-in-variables regression models

The problem of fitting a parametric model in Tobit errors-in-variables regression models is discussed in this paper. The proposed test is based on the supremum of the Khmaladze type transformation of a certain partial sum process of calibrated residuals. This framework covers the usual error-free Tobit model as a special case. The asymptotic null distribution of this transformed process is shown to be the same as that of a time transformed standard Brownian motion. Consistency against some fixed alternatives and asymptotic power under some local nonparametric alternatives of this test are also discussed. Simulation studies are conducted to assess the finite sample performance of the proposed test.     

Estimating the error distribution in nonparametric multiple regression with applications to model testing

In this paper we consider the estimation of the error distribution in a heteroscedastic nonparametric regression model with multivariate covariates. As estimator we consider the empirical distribution function of residuals, which are obtained from multivariate local polynomial fits of the regression and variance functions, respectively. Weak convergence of the empirical residual process to a Gaussian process is proved. We also consider various applications for testing model assumptions in nonparametric multiple regression. The model tests obtained are able to detect local alternatives that converge to zero at an n^−1/2-rate, independent of the covariate dimension. We consider in detail a test for additivity of the regression function.     

Bayesian predictive densities based on superharmonic priors for the 2-dimensional Wishart model

Bayesian predictive densities for the 2-dimensional Wishart model are investigated. The performance of predictive densities is evaluated by using the Kullback–Leibler divergence. It is proved that a Bayesian predictive density based on a prior exactly dominates that based on the Jeffreys prior if the prior density satisfies some geometric conditions. An orthogonally invariant prior is introduced and it is shown that the Bayesian predictive density based on the prior is minimax and dominates that based on the right invariant prior with respect to the triangular group.     

Minimax adjustment technique in a parameter restricted linear model

We consider an approach yielding a minimax estimator in the linear regression model with a priori information on the parameter vector, e.g., ellipsoidal restrictions. This estimator is computed directly from the loss function and can be motivated by the general Pitman nearness criterion. It turns out that this approach coincides with the projection estimator which is obtained by projecting an initial arbitrary estimate on the subset defined by the restrictions.     

Empirical Bayes predictive densities for high-dimensional normal models

This paper addresses the problem of estimating the density of a future outcome from a multivariate normal model. We propose a class of empirical Bayes predictive densities and evaluate their performances under the Kullback–Leibler (KL) divergence. We show that these empirical Bayes predictive densities dominate the Bayesian predictive density under the uniform prior and thus are minimax under some general conditions. We also establish the asymptotic optimality of these empirical Bayes predictive densities in infinite-dimensional parameter spaces through an oracle inequality.     

On rates of convergence in functional linear regression

This paper investigates the rate of convergence of estimating the regression weight function in a functional linear regression model. It is assumed that the predictor as well as the weight function are smooth and periodic in the sense that the derivatives are equal at the boundary points. Assuming that the functional data are observed at discrete points with measurement error, the complex Fourier basis is adopted in estimating the true data and the regression weight function based on the penalized least-squares criterion. The rate of convergence is then derived for both estimators. A simulation study is also provided to illustrate the numerical performance of our approach, and to make a comparison with the principal component regression approach.     

Depth notions for orthogonal regression

Global depth, tangent depth and simplicial depths for classical and orthogonal regression are compared in examples, and properties that are useful for calculations are derived. The robustness of the maximum simplicial depth estimates is shown in examples. Algorithms for the calculation of depths for orthogonal regression are proposed, and tests for multiple regression are transferred to orthogonal regression. These tests are distribution free in the case of bivariate observations. For a particular test problem, the powers of tests that are based on simplicial depth and tangent depth are compared by simulations.