Maxisets for Model Selection

We address the statistical issue of determining the maximal spaces (maxisets) where model selection procedures attain a given rate of convergence. By considering first general dictionaries, then orthonormal bases, we characterize these maxisets in terms of approximation spaces. These results are illustrated by classical choices of wavelet model collections. For each of them, the maxisets are described in terms of functional spaces. We give special attention to the issue of calculability and measure the induced loss of performance in terms of maxisets.     

On the block thresholding wavelet estimators with censored data

We consider block thresholding wavelet-based density estimators with randomly right-censored data and investigate their asymptotic convergence rates. Unlike for the complete data case, the empirical wavelet coefficients are constructed through the Kaplan-Meier estimators of the distribution functions in the censored data case. On the basis of a result of Stute [W. Stute, The central limit theorem under random censorship, Ann. Statist. 23 (1995) 422-439] that approximates the Kaplan-Meier integrals as averages of i.i.d. random variables with a certain rate in probability, we can show that these wavelet empirical coefficients can be approximated by averages of i.i.d. random variables with a certain error rate in L². Therefore we can show that these estimators, based on block thresholding of empirical wavelet coefficients, achieve optimal convergence rates over a large range of Besov function classes , p≥2, q≥1 and nearly optimal convergence rates when 1≤p<2. We also show that these estimators achieve optimal convergence rates over a large class of functions that involve many irregularities of a wide variety of types, including chirp and Doppler functions, and jump discontinuities. Therefore, in the presence of random censoring, wavelet estimators still provide extensive adaptivity to many irregularities of large function classes. The performance of the estimators is tested via a modest simulation study.     

On the posterior median estimators of possibly sparse sequences

We adopt the Bayesian paradigm and discuss certain properties of posterior median estimators of possibly sparse sequences. The prior distribution considered is a mixture of an atom of probability at zero and a symmetric unimodal distribution, and the noise distribution is taken as another symmetric unimodal distribution. We derive an explicit form of the corresponding posterior median and show that it is an antisymmetric function and, under some conditions, a shrinkage and a thresholding rule. Furthermore we show that, as long as the tails of the nonzero part of the prior distribution are heavier than the tails of the noise distribution, the posterior median, under some constraints on the involved parameters, has the bounded shrinkage property, extending thus recent results to larger families of prior and noise distributions. Expressions of posterior distributions and posterior medians in particular cases of interest are obtained. The asymptotes of the derived posterior medians, which provide valuable information of how the corresponding estimators treat large coefficients, are also given. These results could be particularly useful for studying frequentist optimality properties and developing statistical techniques of the resulting posterior median estimators of possibly sparse sequences for a wider set of prior and noise distributions.     

Estimating cumulative incidence functions when the life distributions are constrained

In competing risks studies, the Kaplan-Meier estimators of the distribution functions (DFs) of lifetimes and the corresponding estimators of cumulative incidence functions (CIFs) are used widely when no prior information is available for these distributions. In some cases better estimators of the DFs of lifetimes are available when they obey some inequality constraints, e.g., if two lifetimes are stochastically or uniformly stochastically ordered, or some functional of a DF obeys an inequality in an empirical likelihood estimation procedure. If the restricted estimator of a lifetime differs from the unrestricted one, then the usual estimators of the CIFs will not add up to the lifetime estimator. In this paper we show how to estimate the CIFs in this case. These estimators are shown to be strongly uniformly consistent. In all cases we consider, when the inequality constraints are strict the asymptotic properties of the restricted and the unrestricted estimators are the same, thus providing the asymptotic properties of the restricted estimators essentially “free of charge”. We give an example to illustrate our procedure.     

Nonparametric regression on random fields with random design using wavelet method

We consider non-linear wavelet-based estimators of spatial regression functions with (known) random design on strictly stationary random fields, which are indexed by the integer lattice points in the \(N\)-dimensional Euclidean space and are assumed to satisfy some mixing conditions. We investigate their asymptotic rates of convergence based on thresholding of empirical wavelet coefficients and show that these estimators achieve nearly optimal convergence rates within a logarithmic term over a large range of Besov function classes \(B^{s}_{p,q}\). Therefore, wavelet estimators still achieve nearly optimal convergence rates for random fields and provide explicitly the extraordinary local adaptability.     

Thresholding projection estimators in functional linear models

We consider the problem of estimating the regression function in functional linear regression models by proposing a new type of projection estimators which combine dimension reduction and thresholding. The introduction of a threshold rule allows us to get consistency under broad assumptions as well as minimax rates of convergence under additional regularity hypotheses. We also consider the particular case of Sobolev spaces generated by the trigonometric basis which permits us to get easily mean squared error of prediction as well as estimators of the derivatives of the regression function. We prove that these estimators are minimax and rates of convergence are given for some particular cases.     

Use of prior information in the consistent estimation of regression coefficients in measurement error models

A multivariate ultrastructural measurement error model is considered and it is assumed that some prior information is available in the form of exact linear restrictions on regression coefficients. Using the prior information along with the additional knowledge of covariance matrix of measurement errors associated with explanatory vector and reliability matrix, we have proposed three methodologies to construct the consistent estimators which also satisfy the given linear restrictions. Asymptotic distribution of these estimators is derived when measurement errors and random error component are not necessarily normally distributed. Dominance conditions for the superiority of one estimator over the other under the criterion of Löwner ordering are obtained for each case of the additional information. Some conditions are also proposed under which the use of a particular type of information will give a more efficient estimator.     

Fourier transform methods for pathwise covariance estimation in the presence of jumps

We provide a new non-parametric Fourier procedure to estimate the trajectory of the instantaneous covariance process (from discrete observations of a multidimensional price process) in the presence of jumps extending the seminal work of Malliavin and Mancino (2002, 2009). Our approach relies on a modification of (classical) jump-robust estimators of integrated realized covariance to estimate the Fourier coefficients of the covariance trajectory. Using Fourier–Féjer inversion we reconstruct the path of the instantaneous covariance. We prove consistency and a central limit theorem (CLT) and in particular that the asymptotic estimator variance is smaller by a factor 2/3 in comparison to classical local estimators.     

On rate-optimal nonparametric wavelet regression with long memory moving average errors

We consider the wavelet-based estimators of mean regression function with long memory moving average errors and investigate their asymptotic rates of convergence based on thresholding of empirical wavelet coefficients. We show that these estimators achieve nearly optimal minimax convergence rates within a logarithmic term over a large range of Besov function classes $B^{s}_{p,q}$ B p , q s . Therefore, in the presence of long memory non-Gaussian moving average noise, wavelet estimators still achieve nearly optimal convergence rates and provide explicitly the extraordinary local adaptability. The theory is illustrated with some numerical examples.     

Iterated Barron density estimators

Barron estimators are modified histograms depending on some reference probability density. They have been proved to have good consistency properties according to several information theoretical criteria. In this Note, Barron estimators are viewed as dynamical systems in a functional space with the reference density as initial state. Properties of stationary densities of these systems are investigated. It is then proved, under some assumptions, that the trajectories become almost surely stationary after a finite number of iterations.     

On information pooling, adaptability and superefficiency in nonparametric function estimation

The connections between information pooling and adaptability as well as superefficiency are considered. Separable rules, which figure prominently in wavelet and other orthogonal series methods, are shown to lack adaptability; they are necessarily not rate-adaptive. A sharp lower bound on the cost of adaptation for separable rules is obtained. We show that adaptability is achieved through information pooling. A tight lower bound on the amount of information pooling required for achieving rate-optimal adaptation is given. Furthermore, in a sharp contrast to the separable rules, it is shown that adaptive non-separable estimators can be superefficient at every point in the parameter spaces. The results demonstrate that information pooling is the key to increasing estimation precision as well as achieving adaptability and even superefficiency.     

On stochastic orderings of the Wilcoxon Rank Sum test statistic—With applications to reproducibility probability estimation testing

Recently, the possibility of testing statistical hypotheses through the estimate of the reproducibility probability (i.e. the estimate of the power of the statistical test) in a general parametric framework has been introduced. In this paper, we provide some results on the stochastic orderings of the Wilcoxon Rank Sum (WRS) statistic, implying, for example, that the related test is strictly unbiased. Moreover, under some regularity conditions, we show that it is possible to define a continuous and strictly monotone power function of the WRS test. This last result is useful in order to obtain a point estimator and lower bounds for the power of the WRS test. In analogy with the parametric setting, we show that these power estimators, alias reproducibility probability estimators, can be used as test statistic, i.e. it is possible to refer directly to the estimate of the reproducibility probability to perform the WRS test. Some reproducibility probability estimators based on asymptotic approximations of the power are provided. A brief simulation shows a very high agreement between the approximated reproducibility probability based tests and the classical one.     

Estimation of Time Varying Linear Systems

Based on kernel and wavelet estimators of the evolutionary spectrum and cross-spectrum we propose nonlinear wavelet estimators of the time varying coefficients of a linear system, whose input and output are locally stationary processes, in the sense of Dahlhaus (1997). We obtain large sample properties of these estimators, present some simulated examples and derive results on the L ₂-risk for the wavelet threshold estimators, assuming that the coefficients belong to some smoothness class. This revised version was published online in June 2006 with corrections to the Cover Date.     

Approximation in Learning Theory

This paper addresses some problems of supervised learning in the setting formulated by Cucker and Smale. Supervised learning, or learning-from-examples, refers to a process that builds on the base of available data of inputs x_i and outputs y_i, i = 1,...,m, a function that best represents the relation between the inputs x ∈ X and the corresponding outputs y ∈ Y. The goal is to find an estimator f_z on the base of given data z := ((x₁,y₁),...,(x_m,y_m)) that approximates well the regression function f_ρ (or its projection) of an unknown Borel probability measure ρ defined on Z = X × Y. We assume that (x_i,y_i), i = 1,...,m, are independent and distributed according to ρ. We discuss the following two problems: I. the projection learning problem (improper function learning problem); II. universal (adaptive) estimators in the proper function learning problem. In the first problem we do not impose any restrictions on a Borel measure ρ except our standard assumption that |y|≤ M a.e. with respect to ρ. In this case we use the data z to estimate (approximate) the L₂(ρ_X) projection (f_ρ)_W of f_ρ onto a function class W of our choice. Here, ρ_X is the marginal probability measure. In [KT1,2] this problem has been studied for W satisfying the decay condition ε_n(W,B) ≤ Dn^-r of the entropy numbers ε_n(W,B) of W in a Banach space B in the case B = C(X) or B = L₂(\rho_X). In this paper we obtain the upper estimates in the case ε_n(W,L₁(ρ_X)) ≤ Dn^-r with an extra assumption that W is convex. In the second problem we assume that an unknown measure ρ satisfies some conditions. Following the standard way from nonparametric statistics we formulate these conditions of the form f_ρ ∈ Θ. Next, we assume that the only a priori information available is that f_ρ belongs to a class Θ (unknown) from a known collection {Θ} of classes. We want to build an estimator that provides approximation of f_ρ close to the optimal for the class Θ. Along with standard penalized least squares estimators we consider a new method of construction of universal estimators. This method is based on a combination of two powerful ideas in building universal estimators. The first one is the use of penalized least squares estimators. This idea works well in the case of general setting with rather abstract methods of approximation. The second one is the idea of thresholding that works very well when we use wavelets expansions as an approximation tool. A new estimator that we call the big jump estimator uses the least squares estimators and chooses a right model by a thresholding criteria instead of the penalization. In this paper we illustrate how ideas and methods of approximation theory can be used in learning theory both in formulating a problem and in solving it.     

The FEXP estimator for potentially non-stationary linear time series

We consider semiparametric fractional exponential (FEXP) estimators of the memory parameter d for a potentially non-stationary linear long-memory time series with additive polynomial trend. We use differencing to annihilate the polynomial trend, followed by tapering to handle the potential non-invertibility of the differenced series. We propose a method of pooling the tapered periodogram which leads to more efficient estimators of d than existing pooled, tapered estimators. We establish asymptotic normality of the tapered FEXP estimator in the Gaussian case with or without pooling. We establish asymptotic normality of the estimator in the linear case if pooling is used. Finally, we consider minimax rate-optimality and feasible nearly rate-optimal estimators in the Gaussian case.     

Consistent learning by composite proximal thresholding

We investigate the modeling and the numerical solution of machine learning problems with prediction functions which are linear combinations of elements of a possibly infinite dictionary of functions. We propose a novel flexible composite regularization model, which makes it possible to incorporate various priors on the coefficients of the prediction function, including sparsity and hard constraints. We show that the estimators obtained by minimizing the regularized empirical risk are consistent in a statistical sense, and we design an error-tolerant composite proximal thresholding algorithm for computing such estimators. New results on the asymptotic behavior of the proximal forward–backward splitting method are derived and exploited to establish the convergence properties of the proposed algorithm. In particular, our method features a o(1 / m) convergence rate in objective values.     

增长曲线模型回归系数线性估计的可容许性

该文讨论增长曲线模型中回归系数线性估计的可容许性, 在矩阵损失(d(Y)-KBL)(d(Y)-KBL)' 下, 分别给出了回归系数的齐次与非齐次线性估计是可容许的充要条件, 推广了以往文献的相关结论.     

Statistically based multiwavelet denoising

In this work, we consider a statistically based multiwavelet thresholding method which acts on the empirical wavelet coefficients in groups, rather than individually, in order to obtain an edge-preserving image denoising technique. Our strategy allows us to exploit the dependencies between neighboring coefficients to make a simultaneous thresholding decision, so that estimation accuracy is increased.

By interpreting the multiwavelet analysis in a statistical context, we propose a new weighted multiwavelet matrix thresholding rule, based on the statistical modeling of empirical coefficients. This allows the thresholding decision to be adapted to the local structure of the underlying image, hence producing edge-preserving denoising. Extensive numerical results are presented showing the performance of our denoising procedure.     

Optimally thresholded realized power variations for Lévy jump diffusion models

Thresholded Realized Power Variations (TPVs) are one of the most popular nonparametric estimators for general continuous-time processes with a wide range of applications. In spite of their popularity, a common drawback lies in the necessity of choosing a suitable threshold for the estimator, an issue which so far has mostly been addressed by heuristic selection methods. To address this important issue, we propose an objective selection method based on desirable optimality properties of the estimators. Concretely, we develop a well-posed optimization problem which, for a fixed sample size and time horizon, selects a threshold that minimizes the expected total number of jump misclassifications committed by the thresholding mechanism associated with these estimators. We analytically solve the optimization problem under mild regularity conditions on the density of the underlying jump distribution, allowing us to provide an explicit infill asymptotic characterization of the resulting optimal thresholding sequence at a fixed time horizon. The leading term of the optimal threshold sequence is shown to be proportional to Lévy’s modulus of continuity of the underlying Brownian motion, hence theoretically justifying and sharpening selection methods previously proposed in the literature based on power functions or multiple testing procedures. Furthermore, building on the aforementioned asymptotic characterization, we develop an estimation algorithm, which allows for a feasible implementation of the newfound optimal sequence. Simulations demonstrate the improved finite sample performance offered by optimal TPV estimators in comparison to other popular non-optimal alternatives.     

Wavelets-based estimation of nonlinear canonical analysis

Using a wavelets-based estimator of the bivariate density, we introduce an estimation method for nonlinear canonical analysis. Consistency of the resulting estimators of the canonical coefficients and the canonical functions is established. Under some conditions, asymptotic normality results for these estimators are obtained. Then it is shown how to compute in practice these estimators by usingmatrix computations, and the finite-sample performance of the proposed method is evaluated through simulations.