Polynomial approximation on the sphere using scattered data

We consider the problem of approximately reconstructing a function f defined on the surface of the unit sphere in the Euclidean space ℝ^{q +1} by using samples of f at scattered sites. A central role is played by the construction of a new operator for polynomial approximation, which is a uniformly bounded quasi‐projection in the de la Vallée Poussin style, i.e. it reproduces spherical polynomials up to a certain degree and has uniformly bounded L^p operator norm for 1 ≤ p ≤ ∞. Using certain positive quadrature rules for scattered sites due to Mhaskar, Narcowich and Ward, we discretize this operator obtaining a polynomial approximation of the target function which can be computed from scattered data and provides the same approximation degree of the best polynomial approximation. To establish the error estimates we use Marcinkiewicz–Zygmund inequalities, which we derive from our continuous approximating operator. We give concrete bounds for all constants in the Marcinkiewicz–Zygmund inequalities as well as in the error estimates. (© 2008 WILEY‐VCH Verlag GmbH & Co. KGaA, Weinheim)     

PowerNet: Efficient Representations of Polynomials and Smooth Functions by Deep Neural Networks with Rectified Power Units

Deep neural network with rectified linear units (ReLU) is getting more and more popular recently. However, the derivatives of the function represented by a ReLU network are not continuous, which limit the usage of ReLU network to situations only when smoothness is not required. In this paper, we construct deep neural networks with rectified power units (RePU), which can give better approximations for smooth functions. Optimal algorithms are proposed to explicitly build neural networks with sparsely connected RePUs, which we call PowerNets, to represent polynomials with no approximation error. For general smooth functions, we first project the function to their polynomial approximations, then use the proposed algorithms to construct corresponding PowerNets. Thus, the error of best polynomial approximation provides an upper bound of the best RePU network approximation error. For smooth functions in higher dimensional Sobolev spaces, we use fast spectral transforms for tensor-product grid and sparse grid discretization to get polynomial approximations. Our constructive algorithms show clearly a close connection between spectral methods and deep neural networks: PowerNets with $n$ hidden layers can exactly represent polynomials up to degree $s^n$, where $s$ is the power of RePUs. The proposed PowerNets have potential applications in the situations where high-accuracy is desired or smoothness is required.     

ON THE GENERALIZATIONS OF KALANDIA''''S LEMMA

Based on Bernstein's Theorem, Kalandia's Lemma describes the error estimate and the smoothness of the remainder under the second part of Holder norm when a Holder function is approximated by its best polynomial approximation. In this paper, Kalandia's Lemma is generalized to the cases that the best polynomial is replaced by one of its four kinds of Chebyshev polynomial expansions, the error estimates of the remainder are given out under Holder norm or the weighted Holder norms.     

Local polynomial maximum likelihood estimation for Pareto-type distributions

We discuss the estimation of the tail index of a heavy-tailed distribution when covariate information is available. The approach followed here is based on the technique of local polynomial maximum likelihood estimation. The generalized Pareto distribution is fitted locally to exceedances over a high specified threshold. The method provides nonparametric estimates of the parameter functions and their derivatives up to the degree of the chosen polynomial. Consistency and asymptotic normality of the proposed estimators will be proven under suitable regularity conditions. This approach is motivated by the fact that in some applications the threshold should be allowed to change with the covariates due to significant effects on scale and location of the conditional distributions. Using the asymptotic results we are able to derive an expression for the asymptotic mean squared error, which can be used to guide the selection of the bandwidth and the threshold. The applicability of the method will be demonstrated with a few practical examples.     

Empirical bayesian test of the smoothness

In the context of adaptive nonparametric curve estimation a common assumption is that a function (signal) to estimate belongs to a nested family of functional classes. These classes are often parametrized by a quantity representing the smoothness of the signal. It has already been realized by many that the problem of estimating the smoothness is not sensible. What can then be inferred about the smoothness? The paper attempts to answer this question. We consider implications of our results to hypothesis testing about the smoothness and smoothness classification problem. The test statistic is based on the empirical Bayes approach, i.e., it is the marginalized maximum likelihood estimator of the smoothness parameter for an appropriate prior distribution on the unknown signal.     

Estimation of a probability density function and its derivatives

We investigate statistical estimates of a probability density distribution function and its derivatives. As the starting point of the investigation we take a priori assumptions about the degree of smoothness of the probability density to be estimated. By using these assumptions we can construct estimates of the probability density function itself and its derivatives which are distinguished by the high rate of decrease of the error in the estimate as the sample size increases.Translated from Matematicheskie Zametki, Vol. 12, No. 5, 621–626, November, 1972.     

关于Dini-Lipschitz型逼近定理的统一处理方法方法

This survey paper studies the approximation of (polynomial) processes for which the operator norms do not form a bounded sequence. In view of familiar direct estimates and quantitative uniform boundedness principles, a unified approach is given to results concerning the equivalence of Dini-Lipschitz-type conditions with (strong) convergence on (smoothness) classes. Emphasis is laid upon the necessity of these conditions, essential ingredients of the proofs are suitable modifications of the familiar gliding hump method. Apart from the classical results concerned with Fourier partial sums, explicit applications are treated for (trigonometric as well as algebralc) Lagrange interpolation, interpolatory quadrature rules based upon Jacobl knots, multipliers or strong convergence, and for Bochner-Riesz means of multivariate Fourier series for parameter values below the critical index.     

Weighted Moduli of Smoothness and Sign-Preserving Approximation

We consider a continuous function that changes its sign on an interval finitely many times and pose the problem of the approximation of this function by a polynomial that inherits its sign. For this approximation, we obtain (in the case where this is possible) Jackson-type estimates containing modified weighted moduli of smoothness of the Ditzian-Totik type. In some cases, constants in these estimates depend substantially on the location of points where the function changes its sign. We give examples of functions for which these constants are unimprovable. We also prove theorems that are analogous, in a certain sense, to inverse theorems of approximation without restrictions. __________ Translated from Ukrains'kyi Matematychnyi Zhurnal, Vol. 57, No. 3, pp. 400–420, March, 2005.     

Smooth solutions of an initial-value problem for some differential difference equations

The subject matter of this paper is an initial-value problem with an initial function for a linear differential difference equation of neutral type. The problem is to find an initial function such that the solution generated by this function has some given smoothness at the points multiple of the delay. The problem is solved using a method of polynomial quasisolutions, which is based on a representation of the unknown function in the form of a polynomial of some degree. Substituting this into the initial problem yields some incorrectness in the sense of degree of polynomials, which is compensated for by introducing some residual into the equation. For this residual, an exact analytical formula as a measure of disturbance of the initial-value problem is obtained. It is shown that if a polynomial quasisolution of degree N is chosen as an initial function for the initial-value problem in question, the solution generated will have smoothness not lower than N at the abutment points.     

Bayesian estimation for generalized exponential distribution based on progressive type-I interval censoring

In this study, we consider the Bayesian estimation of unknown parameters and reliability function of the generalized exponential distribution based on progressive type-I interval censoring. The Bayesian estimates of parameters and reliability function cannot be obtained as explicit forms by applying squared error loss and Linex loss functions, respectively; thus, we present the Lindley’s approximation to discuss these estimations. Then, the Bayesian estimates are compared with the maximum likelihood estimates by using the Monte Carlo simulations.     

A bayesian approach to the probability density estimation

Summary A Bayesian procedure for the probability density estimation is proposed. The procedure is based on the multinomial logit transformations of the parameters of a finely segmented histogram model. The smoothness of the estimated density is guaranteed by the introduction of a prior distribution of the parameters. The estimates of the parameters are defined as the mode of the posterior distribution. The prior distribution has several adjustable parameters (hyper-parameters), whose values are chosen so that ABIC (Akaike's Bayesian Information Criterion) is minimized. The basic procedure is developed under the assumption that the density is defined on a bounded interval. The handling of the general case where the support of the density function is not necessarily bounded is also discussed. The practical usefulness of the procedure is demonstrated by numerical examples. The Institute of Statistical Mathematics     

球面S~(d-1)上的广义Ul'yanov型不等式

本文对于单位球面上的经典连续模,给出了一个非常有用的广义Ul'yanov型不等式.该不等式在球面多项式逼近、球面嵌入理论以及球面上函数空间的插值理论等领域有着非常重要的应用.我们的证明基于球面调和多项式展开的新的估计,这些估计本身也具有独立的意义.     

ON THE GENERALIZATIONS OF KALANDIA＇S LEMMA

Based on Bernstein＇s Theorem, Kalandia＇s Lemma describes the error estimate and the smoothness of the remainder under the second part of Hoelder norm when a HSlder function is approximated by its best polynomial approximation. In this paper, Kalandia＇s Lemma is generalized to the cases that the best polynomial is replaced by one of its four kinds of Chebyshev polynomial expansions, the error estimates of the remainder are given out under Hoeder norm or the weighted HSlder norms.     

A new formulation of Bernstein-Bezier based smoothness conditions forpp functions

In this note, we establish a new formulation of smoothness conditions for piecewise polynomial (: =pp) functions in terms of the B-net representation in the general n-dimensional setting. It plays an important role for 2-dimensional setting in the constructive proof of the fact that the spaces of polynomial splines with smoothness r and total degree k≥3r+2 over arbitrary triangulations achieve the optimal approximation order with the approximation constant depending only on k and the smallest angle of the partition in [5].     

随机加权法在密度估计中的应用

本文给出了概率密度函数的椭机加权估计，证明了承机加权分布与密度估计的标准化估计量的分布的逼近精度可达到ｏ（１／√ｎｈ），并且构造了Ｅｆｎ（ｘ）的置信区间，其中ｆｎ（ｘ）为密度函数的核估计，ｈ＝ｈｎ炒估计的窗宽。     

ℋ︁‐matrices for the convection‐diffusion equation

Hierarchical matrices (ℋ︁‐matrices) provide a technique for the sparse approximation of large, fully populated matrices. This technique has been shown to be applicable to stiffness matrices arising in boundary element method applications where the kernel function displays certain smoothness properties. The error estimates for an approximation of the kernel function by a separable function can be carried over directly to error estimates for an approximation of the stiffness matrix by an ℋ︁‐matrix, using a certain standard partitioning and admissibility condition for matrix blocks. Similarly, ℋ︁‐matrix techniques can be applied in the finite element context where it is the inverse of the stiffness matrix that is fully populated. Here one needs a separable approximation of Green's function of the underlying boundary value problem in order to prove approximability by matrix blocks of low rank. Unfortunately, Green's function for the convection‐diffusion equation does not satisfy the required smoothness properties, hence prohibiting a straightforward generalization of the separable approximation through Taylor polynomials. We will use Green's function to motivate a modification in the (hierarchical) partitioning of the index set and as a consequence the resulting hierarchy of block partitionings as well as the admissibility condition. We will illustrate the effect of the proposed modifications by numerical results.     

On 3-monotone approximation by piecewise polynomials

We consider 3-monotone approximation by piecewise polynomials with prescribed knots. A general theorem is proved, which reduces the problem of 3-monotone uniform approximation of a 3-monotone function, to convex local L₁ approximation of the derivative of the function. As the corollary we obtain Jackson-type estimates on the degree of 3-monotone approximation by piecewise polynomials with prescribed knots. Such estimates are well known for monotone and convex approximation, and to the contrary, they in general are not valid for higher orders of monotonicity. Also we show that any convex piecewise polynomial can be modified to be, in addition, interpolatory, while still preserving the degree of the uniform approximation. Alternatively, we show that we may smooth the approximating piecewise polynomials to be twice continuously differentiable, while still being 3-monotone and still keeping the same degree of approximation.     

Multivariate Epi-splines and Evolving Function Identification Problems

The broad class of extended real-valued lower semicontinuous (lsc) functions on ? ⁿ captures nearly all functions of practical importance in equation solving, variational problems, fitting, and estimation. The paper develops piecewise polynomial functions, called epi-splines, that approximate any lsc function to an arbitrary level of accuracy. Epi-splines provide the foundation for the solution of a rich class of function identification problems that incorporate general constraints on the function to be identified including those derived from information about smoothness, shape, proximity to other functions, and so on. As such extrinsic information as well as observed function and subgradient values often evolve in applications, we establish conditions under which the computed epi-splines converge to the function we seek to identify. Numerical examples in response surface building and probability density estimation illustrate the framework.     

Risk bounds for model selection via penalization

Performance bounds for criteria for model selection are developed using recent theory for sieves. The model selection criteria are based on an empirical loss or contrast function with an added penalty term motivated by empirical process theory and roughly proportional to the number of parameters needed to describe the model divided by the number of observations. Most of our examples involve density or regression estimation settings and we focus on the problem of estimating the unknown density or regression function. We show that the quadratic risk of the minimum penalized empirical contrast estimator is bounded by an index of the accuracy of the sieve. This accuracy index quantifies the trade-off among the candidate models between the approximation error and parameter dimension relative to sample size. If we choose a list of models which exhibit good approximation properties with respect to different classes of smoothness, the estimator can be simultaneously minimax rate optimal in each of those classes. This is what is usually called adaptation. The type of classes of smoothness in which one gets adaptation depends heavily on the list of models. If too many models are involved in order to get accurate approximation of many wide classes of functions simultaneously, it may happen that the estimator is only approximately adaptive (typically up to a slowly varying function of the sample size). We shall provide various illustrations of our method such as penalized maximum likelihood, projection or least squares estimation. The models will involve commonly used finite dimensional expansions such as piecewise polynomials with fixed or variable knots, trigonometric polynomials, wavelets, neural nets and related nonlinear expansions defined by superposition of ridge functions. Received: 7 July 1995 / Revised version: 1 November 1997     

On Bivariate Smoothness Spaces Associated with Nonlinear Approximation

In recent years there have been various attempts at the representations of {\mbox multivariate} signals such as images, which outperform wavelets. As is well known, wavelets are not optimal in that they do not take full advantage of the geometrical regularities and singularities of the images. Thus these approaches have been based on tracing curves of singularities and applying bandlets, curvelets, ridgelets, etc., or allocating some weights to curves of singularities like the Mumford–Shah functional and its modifications. In the latter approach a function is approximated on subdomains where it is smoother but there is a penalty in the form of the total length (or other measurement) of the partitioning curves. We introduce a combined measure of smoothness of the function in several dimensions by augmenting its smoothness on subdomains by the smoothness of the partitioning curves. Also, it is known that classical smoothness spaces fail to characterize approximation spaces corresponding to multivariate piecewise polynomial nonlinear approximation. We show how the proposed notion of smoothness can almost characterize these spaces. The question whether the characterization proposed in this work can be further simplified remains open.