Risk bounds for model selection via penalization

Performance bounds for criteria for model selection are developed using recent theory for sieves. The model selection criteria are based on an empirical loss or contrast function with an added penalty term motivated by empirical process theory and roughly proportional to the number of parameters needed to describe the model divided by the number of observations. Most of our examples involve density or regression estimation settings and we focus on the problem of estimating the unknown density or regression function. We show that the quadratic risk of the minimum penalized empirical contrast estimator is bounded by an index of the accuracy of the sieve. This accuracy index quantifies the trade-off among the candidate models between the approximation error and parameter dimension relative to sample size. If we choose a list of models which exhibit good approximation properties with respect to different classes of smoothness, the estimator can be simultaneously minimax rate optimal in each of those classes. This is what is usually called adaptation. The type of classes of smoothness in which one gets adaptation depends heavily on the list of models. If too many models are involved in order to get accurate approximation of many wide classes of functions simultaneously, it may happen that the estimator is only approximately adaptive (typically up to a slowly varying function of the sample size). We shall provide various illustrations of our method such as penalized maximum likelihood, projection or least squares estimation. The models will involve commonly used finite dimensional expansions such as piecewise polynomials with fixed or variable knots, trigonometric polynomials, wavelets, neural nets and related nonlinear expansions defined by superposition of ridge functions. Received: 7 July 1995 / Revised version: 1 November 1997     

关于奇异积分三次复样条近似的误差阶估计

讨论曲线上柯西型奇异积分利用三次复样条进行近似计算的误差估计,对于相关函数类给出了这类逼近的误差阶.     

Convergence Rates of Adaptive Methods,Besov Spaces,and Multilevel Approximation

This paper concerns characterizations of approximation classes associated with adaptive finite element methods with isotropic h-refinements. It is known from the seminal work of Binev, Dahmen, DeVore and Petrushev that such classes are related to Besov spaces. The range of parameters for which the inverse embedding results hold is rather limited, and recently, Gaspoz and Morin have shown, among other things, that this limitation disappears if we replace Besov spaces by suitable approximation spaces associated with finite element approximation from uniformly refined triangulations. We call the latter spaces multievel approximation spaces and argue that these spaces are placed naturally halfway between adaptive approximation classes and Besov spaces, in the sense that it is more natural to relate multilevel approximation spaces with either Besov spaces or adaptive approximation classes, than to go directly from adaptive approximation classes to Besov spaces. In particular, we prove embeddings of multilevel approximation spaces into adaptive approximation classes, complementing the inverse embedding theorems of Gaspoz and Morin. Furthermore, in the present paper, we initiate a theoretical study of adaptive approximation classes that are defined using a modified notion of error, the so-called total error, which is the energy error plus an oscillation term. Such approximation classes have recently been shown to arise naturally in the analysis of adaptive algorithms. We first develop a sufficiently general approximation theory framework to handle such modifications, and then apply the abstract theory to second-order elliptic problems discretized by Lagrange finite elements, resulting in characterizations of modified approximation classes in terms of memberships of the problem solution and data into certain approximation spaces, which are in turn related to Besov spaces. Finally, it should be noted that throughout the paper we paid equal attention to both conforming and non-conforming triangulations.     

Histogram selection for possibly censored data

We propose in this article a unified approach to functional estimation problems based on possibly censored data. The general framework that we define allows, for instance, to handle density and hazard rate estimation based on randomly right-censored data, or regression. Given a collection of histograms, our estimation procedure consists in selecting the best histogram among that collection from the data, by minimizing a penalized least-squares type criterion. For a general collection of histograms, we obtain nonasymptotic oracle-type inequalities. Then, we consider the collection of histograms built on partitions into dyadic intervals, a choice inspired by an approximation result due to DeVore and Yu. In that case, our estimator is also adaptive in the minimax sense over a wide range of smoothness classes that contain functions of inhomogeneous smoothness. Besides, its computational complexity is only linear in the size of the sample.     

Nonlinear Approximation with Dictionaries I. Direct Estimates

We study various approximation classes associated with m-term approximation by elements from a (possibly redundant) dictionary in a Banach space. The standard approximation class associated with the best m-term approximation is compared to new classes defined by considering m-term approximation with algorithmic constraints: thresholding and Chebychev approximation classes are studied, respectively. We consider embeddings of the Jackson type (direct estimates) of sparsity spaces into the mentioned approximation classes. General direct estimates are based on the geometry of the Banach space, and we prove that assuming a certain structure of the dictionary is sufficient and (almost) necessary to obtain stronger results. We give examples of classical dictionaries in L^p spaces and modulation spaces where our results recover some known Jackson type estimates, and discuss some new estimates they provide.     

Bounds for a class of linear functionals with applications to Hermite interpolation

A general estimation theorem is given for a class of linear functionals on Sobolev spaces. The functionals considered are those which annihilate certain classes of polynomials. An interpolation scheme of Hermite type is defined inN-dimensions and the accuracy in approximation is bounded by means of the above mentioned theorem. In one and two dimensions our schemes reduce to the usual ones, however our estimates in two dimensions are new in that they involve only the pure partial derivatives.This research was supported in part by the National Science Foundation under grant number N.S.F.-G.P.-9467.     

On the approximation of curves with polylines

The exact values of the estimation of the approximation error of parametrically defined curves by inscribed polylines in them-dimensional space R ^m for classes of functions defined by moduli of continuity are presented. The result is a sort of generalization of the results of B.N. Malozemov on the approximation of continuous functions with polylines. Also, the problem of finding the upper bounds of deviations of parametrically defined curves for this class is solved based on the assumption that these curves intersect at N (N ≥ 2) points of the partition of [0, L]. In the case of m = 2, from the obtained results follow the previous results on the approximation of plane curves with polylines in Euclidean, Hausdorff, and Hamming metrices.     

Parameter estimation of ordinary differential equations

This paper addresses the development of a new algorithm forparameter estimation of ordinary differential equations. Here,we show that (1) the simultaneous approach combined with orthogonalcyclic reduction can be used to reduce the estimation problemto an optimization problem subject to a fixed number of equalityconstraints without the need for structural information to devisea stable embedding in the case of non-trivial dichotomy and(2) the Newton approximation of the Hessian information of theLagrangian function of the estimation problem should be usedin cases where hypothesized models are incorrect or only a limitedamount of sample data is available. A new algorithm is proposedwhich includes the use of the sequential quadratic programming(SQP) Gauss–Newton approximation but also encompassesthe SQP Newton approximation along with tests of when to usethis approximation. This composite approach relaxes the restrictionson the SQP Gauss–Newton approximation that the hypothesizedmodel should be correct and the sample data set large enough.This new algorithm has been tested on two standard problems.     

积分波动率二阶变差估计量分析:鞍点算法

本文利用鞍点逼近方法对Black-Scholes模型的积分波动率的二阶变差估计量的估计误差进行分析,得到了相对于中心极限定理更为精细的结果,并且给出了逼近的鞍点算法。结果表明鞍点逼近是中心极限定理的纠正。模拟结果表明鞍点算法给出的估计误差分布相对于正态逼近更合理。该结果在对积分波动率进行统计假设检验时是有意义的。     

Approximation by entire functions on countable unions of segments of the real axis: 2. proof of the main theorem

In this study, we consider an approximation of entire functions of Hölder classes on a countable union of segments by entire functions of exponential type. It is essential that the approximation rate in the neighborhood of segment ends turns out to be higher in the scale that had first appeared in the theory of polynomial approximation by functions of Hölder classes on a segment and made it possible to harmonize the so-called “direct” and “inverse” theorems for that case, i.e., restore the Hölder smoothness by the rate of polynomial approximation in this scale. Approximations by entire functions on a countable union of segments have not been considered earlier. The first section of this paper presents several lemmas and formulates the main theorem. In this study, we prove this theorem on the basis of earlier given lemmas.     

用指数型整函数的最佳限制逼近

我们研究了由仅有实零点的代数多项式导出的微分算子确定的广义Sobolev类利用指数型整函数作为逼近工具的最佳限制逼近问题.利用Fourier变换和周期化等方法,得到在L_2(R)范数下的广义Sobolev光滑函数类的相对平均宽度和最佳限制逼近的精确常数,以及当0是这个代数多项式的一个至多2重的零点时,得到最佳限制逼近在L_1(R)范数和一致范数下的广义Sobolev类的精确到阶的结果.     

Simultaneous Approximation of Sobolev Classes by Piecewise Cubic Hermite Interpolation

For the approximation in $L_p$-norm, we determine the weakly asymptotic orders for the simultaneous approximation errors of Sobolev classes by piecewise cubic Hermite interpolation with equidistant knots. For $p = 1$, $∞$, we obtain its values. By these results we know that for the Sobolev classes, the approximation errors by piecewise cubic Hermite interpolation are weakly equivalent to the corresponding infinite-dimensional Kolmogorov widths. At the same time, the approximation errors of derivatives are weakly equivalent to the corresponding infinite-dimensional Kolmogorov widths.     

Degree of Approximation for Nonlinear Multivariate Sampling Kantorovich Operators on Some Functions Spaces

In this article, the problem of the order of approximation for the nonlinear multivariate sampling Kantorovich operators is investigated. The case of uniformly continuous and bounded functions belonging to Lipschitz classes is considered, as well as the case of functions in Orlicz spaces. In the latter setting, suitable Zygmung-type classes are introduced by using the modular functionals of the spaces. The results obtained show that the order of approximation depends on both the kernels of our operators and the engaged functions. Several examples of kernels are considered in special instances of Orlicz spaces, typically used in approximation theory and for applications to signal and image processing.     

Besov类上的贪婪算法

我们讨论了Besov类MBpr,θ上的相应于张量积小波词典Wd的最佳m-项 逼近问题,证明了其最佳m-项逼近的阶可以通过简单的贪婪算法得到.     

A random model approach for the LASSO

The least absolute selection and shrinkage operator (LASSO) is a method of estimation for linear models similar to ridge regression. It shrinks the effect estimates, potentially shrinking some to be identically zero. The amount of shrinkage is governed by a single parameter. Using a random model formulation of the LASSO, this parameter can be specified as the ratio of dispersion parameters. These parameters are estimated using an approximation to the marginal likelihood of the observed data. The observed score equations from the approximation are biased and hence are adjusted by subtracting an empirical estimate of the expected value. After estimation, the model effects can be tested (via simulation) as the distribution of the observed data given that all model effects are zero is known. Two related simulation studies are presented that show that dispersion parameter estimation results in effect estimates that are competitive with other estimation methods (including other LASSO methods).     

关于非线性共同逼近的强唯一性

本文首先指出文献［１］中的一个错误，举例说明弱拟凸集的最佳逼近未必具有广义强唯一性，进而讨论两类共同逼近的强唯一性，在空间是一致凸、逼近集是共同太阳集的条件下，证明了最佳共同逼近具有广义强唯一性     

在函数逼近论观点下半参数变系数非线性回归函数的估计

在函数逼近论的观点下研究了半参数变系数非线性回归函数的估计问题.采用总体之下L2多项式最佳逼近的方法与样本之下矩估计的方法,独立地分别作出非参数部分与变系数参数部分的解析函数形式的估计,最终得到回归函数的L2与强相合之联合收敛意义下的估计.     

Optimized Tensor-Product Approximation Spaces

This paper is concerned with the construction of optimized grids and approximation spaces for elliptic differential and integral equations. The main result is the analysis of the approximation of the embedding of the intersection of classes of functions with bounded mixed derivatives in standard Sobolev spaces. Based on the framework of tensor-product biorthogonal wavelet bases and stable subspace splittings, the problem is reduced to diagonal mappings between Hilbert sequence spaces. We construct operator adapted finite element subspaces with a lower dimension than the standard full-grid spaces. These new approximation spaces preserve the approximation order of the standard full-grid spaces, provided that certain additional regularity assumptions are fulfilled. The form of the approximation spaces is governed by the ratios of the smoothness exponents of the considered classes of functions. We show in which cases the so-called curse of dimensionality can be broken. The theory covers elliptic boundary value problems as well as boundary integral equations. September 17, 1998. Date revised: March 5, 1999. Date accepted: September 20, 1999.