Approximation by Entire Functions on a Countable Union of Segments on the Real Axis: 3. Further Generalization

In this paper, an approximation of functions of extensive classes set on a countable unit of segments of a real axis using the entire functions of exponential type is considered. The higher the type of the approximating function is, the higher the rate of approximation near segment ends can be made, compared with their inner points. The general approximation scale, which is nonuniform over its segments, depending on the type of the entire function, is similar to the scale set out for the first time in the study of the approximation of the function by polynomials. For cases with one segment and its approximation by polynomials, this scale has allowed us to connect the so-called direct theorems, which state a possible rate of smooth function approximation by polynomials, and the inverse theorems, which give the smoothness of a function approximated by polynomials at a given rate. The approximations by entire functions on a countable unit of segments for the case of Hölder spaces have been studied by the authors in two preceding papers. This paper significantly expands the class of spaces for the functions, which are used to plot an approximation that engages the entire functions with the required properties.     

Approximation by entire functions on countable unions of segments of the real axis: 2. proof of the main theorem

In this study, we consider an approximation of entire functions of Hölder classes on a countable union of segments by entire functions of exponential type. It is essential that the approximation rate in the neighborhood of segment ends turns out to be higher in the scale that had first appeared in the theory of polynomial approximation by functions of Hölder classes on a segment and made it possible to harmonize the so-called “direct” and “inverse” theorems for that case, i.e., restore the Hölder smoothness by the rate of polynomial approximation in this scale. Approximations by entire functions on a countable union of segments have not been considered earlier. The first section of this paper presents several lemmas and formulates the main theorem. In this study, we prove this theorem on the basis of earlier given lemmas.     

Restricted Nonlinear Approximation

We introduce a new form of nonlinear approximation called restricted approximation . It is a generalization of n -term wavelet approximation in which a weight function is used to control the terms in the wavelet expansion of the approximant. This form of approximation occurs in statistical estimation and in the characterization of interpolation spaces for certain pairs of L _p and Besov spaces. We characterize, both in terms of their wavelet coefficients and also in terms of their smoothness, the functions which are approximated with a specified rate by restricted approximation. We also show the relation of this form of approximation with certain types of thresholding of wavelet coefficients. March 31, 1998. Date accepted: January 28, 1999.     

A new formulation of Bernstein-Bezier based smoothness conditions forpp functions

In this note, we establish a new formulation of smoothness conditions for piecewise polynomial (: =pp) functions in terms of the B-net representation in the general n-dimensional setting. It plays an important role for 2-dimensional setting in the constructive proof of the fact that the spaces of polynomial splines with smoothness r and total degree k≥3r+2 over arbitrary triangulations achieve the optimal approximation order with the approximation constant depending only on k and the smallest angle of the partition in [5].     

PowerNet: Efficient Representations of Polynomials and Smooth Functions by Deep Neural Networks with Rectified Power Units

Deep neural network with rectified linear units (ReLU) is getting more and more popular recently. However, the derivatives of the function represented by a ReLU network are not continuous, which limit the usage of ReLU network to situations only when smoothness is not required. In this paper, we construct deep neural networks with rectified power units (RePU), which can give better approximations for smooth functions. Optimal algorithms are proposed to explicitly build neural networks with sparsely connected RePUs, which we call PowerNets, to represent polynomials with no approximation error. For general smooth functions, we first project the function to their polynomial approximations, then use the proposed algorithms to construct corresponding PowerNets. Thus, the error of best polynomial approximation provides an upper bound of the best RePU network approximation error. For smooth functions in higher dimensional Sobolev spaces, we use fast spectral transforms for tensor-product grid and sparse grid discretization to get polynomial approximations. Our constructive algorithms show clearly a close connection between spectral methods and deep neural networks: PowerNets with $n$ hidden layers can exactly represent polynomials up to degree $s^n$, where $s$ is the power of RePUs. The proposed PowerNets have potential applications in the situations where high-accuracy is desired or smoothness is required.     

Optimal control in nonlinear infinite-dimensional systems with nondifferentiability of two types

We consider the optimal control problem for systems described by nonlinear equations of elliptic type. If the nonlinear term in the equation is smooth and the nonlinearity increases at a comparatively low rate of growth, then necessary conditions for optimality can be obtained by well-known methods. For small values of the nonlinearity exponent in the smooth case, we propose to approximate the state operator by a certain differentiable operator. We show that the solution of the approximate problem obtained by standard methods ensures that the optimality criterion for the initial problem is close to its minimal value. For sufficiently large values of the nonlinearity exponent, the dependence of the state function on the control is nondifferentiable even under smoothness conditions for the operator. But this dependence becomes differentiable in a certain extended sense, which is sufficient for obtaining necessary conditions for optimality. Finally, if there is no smoothness and no restrictions are imposed on the nonlinearity exponent of the equation, then a smooth approximation of the state operator is possible. Next, we obtain necessary conditions for optimality of the approximate problem using the notion of extended differentiability of the solution of the equation approximated with respect to the control, and then we show that the optimal control of the approximated extremum problem minimizes the original functional with arbitrary accuracy.     

On approximation by spherical reproducing kernel Hilbert spaces

The spherical approximation between two nested reproducing kernels Hilbert spaces generated from different smooth kernels is investigated. It is shown that the functions of a space can be approximated by that of the subspace with better smoothness. Furthermore, the upper bound of approximation error is given.     

Approximating smooth and sparse functions by deep neural networks: Optimal approximation rates and saturation

Constructing neural networks for function approximation is a classical and longstanding topic in approximation theory. In this paper, we aim at constructing deep neural networks with three hidden layers using a sigmoidal activation function to approximate smooth and sparse functions. Specifically, we prove that the constructed deep nets with controllable magnitude of free parameters can reach the optimal approximation rate in approximating both smooth and sparse functions. In particular, we prove that neural networks with three hidden layers can avoid the phenomenon of saturation, i.e., the phenomenon that for some neural network architectures, the approximation rate stops improving for functions of very high smoothness.     

Semialgebraic complexity of functions

In this paper we study the rate of the best approximation of a given function by semialgebraic functions of a prescribed “combinatorial complexity”. We call this rate a “Semialgebraic Complexity” of the approximated function. By the classical Approximation Theory, the rate of a polynomial approximation is determined by the regularity of the approximated function (the number of its continuous derivatives, the domain of analyticity, etc.). In contrast, semialgebraic complexity (being always bounded from above in terms of regularity) may be small for functions not regular in the usual sense. We give various natural examples of functions of low semialgebraic complexity, including maxima of smooth families, compositions, series of a special form, etc. We show that certain important characteristics of the functions, in particular, the geometry of their critical values (Morse–Sard Theorem) are determined by their semialgebraic complexity, and not by their regularity.     

Estimates of efficiency for two methods of stable numerical summation of smooth functions

We consider a classical ill-posed problem of reconstruction of continuous functions from their noisy Fourier coefficients. We study the case of functions of two variables that has been much less investigated. The smoothness of reconstructed functions is measured in terms of the Sobolev classes as well as the classes of functions with dominated mixed derivatives. We investigate two summation methods, that are based on ideas of the rectangle and the hyperbolic cross, respectively. For both of these methods we establish the estimates of the accuracy on the classes that are considered as well the estimates of computational costs. Moreover, we made the comparison of their efficiency based on obtained estimates. A somehow surprising outcome of our study is that for both types of the considered smoothness classes one should employ hyperbolic cross approximation that is not typical for the functions under consideration.     

Greedy-Type Approximation in Banach Spaces and Applications

We continue to study the efficiency of approximation and convergence of greedy-type algorithms in uniformly smooth Banach spaces. Two greedy-type approximation methods, the Weak Chebyshev Greedy Algorithm (WCGA) and the Weak Relaxed Greedy Algorithm (WRGA), have been introduced and studied in [24]. These methods (WCGA and WRGA) are very general approximation methods that work well in an arbitrary uniformly smooth Banach space $X$ for any dictionary ${\Cal D}$. It turns out that these general approximation methods are also very good for specific dictionaries. It has been observed in [7] that the WCGA and WRGA provide constructive methods in $m$-term trigonometric approximation in $L_p$, $p\in[2,\infty)$, which realize an optimal rate of $m$-term approximation for different function classes. In [25] the WCGA and WRGA have been used in constructing deterministic cubature formulas for a wide variety of function classes with error estimates similar to those for the Monte Carlo Method. The WCGA and WRGA can be considered as a constructive deterministic alternative to (or substitute for) some powerful probabilistic methods. This observation encourages us to continue a thorough study of the WCGA and WRGA. In this paper we study modifications of the WCGA and WRGA that are motivated by numerical applications. In these modifications we are able to perform steps of the WCGA (or WRGA) approximately with some controlled errors. We prove that the modified versions of the {\it WCGA and WRGA perform as well as the WCGA and WRGA}. We give two applications of greedy-type algorithms. First, we use them to provide a constructive proof of optimal estimates for best $m$-term trigonometric approximation in the uniform norm. Second, we use them to construct deterministic sets of points $\{\xi^1,\ldots,\xi^m\} \subset [0,1]^d$ with the $L_p$ discrepancy less than $Cp^{1/2}m^{-1/2}$, $C$ is an effective absolute constant.     

THE SMOOTHNESS AND DIMENSION OF FRACTAL INTERPOLATION FUNCTIONS

In this paper, we investigate the smoothness of non-equidistant fractal interpolation functions We obtain the Holder exponents of such fractal interpolation functions by using the technique of operator approximation. At last, We discuss the series expressiong of these functions and give a Box-counting dimension estimation of “critical” fractal interpohltion functions by using our smoothness results.     

On the order of local approximation of functions by trigonometric polynomials that are partial sums of averaging operators

We study the order of polynomial approximations of periodic functions on intervals which are internal with respect to the main interval of periodicity and on which these functions are sufficiently smooth. The estimates obtained contain parameters which characterize the smoothness and alternation of signs of nuclear functions and parameters that determine classes of approximated functions. Translated from Ukrainskii Matematicheskii Zhurnal, Vol. 49, No. 5, pp.706–714, May, 1997.     

Multidimensional conjugation operators and deformations of the classesZ(ω (2);C(T m ))

The smoothness of conjugate functions of several variables is studied in terms of the second moduli of smoothness. It is proved that in a sufficiently general case the invariance of the classes considered is violated in just the same way as for classes specified by moduli of continuity.Translated fromMatematicheskie Zametki, Vol. 63, No. 6, pp. 853–861, June, 1998.     

Multivariate Jackson-type inequality for a new type neural network approximation

In this paper, we introduce a new type neural networks by superpositions of a sigmoidal function and study its approximation capability. We investigate the multivariate quantitative constructive approximation of real continuous multivariate functions on a cube by such type neural networks. This approximation is derived by establishing multivariate Jackson-type inequalities involving the multivariate modulus of smoothness of the target function. Our networks require no training in the traditional sense.     

Local smoothing of uniformly smooth maps

We solve the problem on the uniform approximation of uniformly continuous (smooth) maps by maps having the maximum possible local and uniform smoothness. In particular, we prove that each uniformly continuous map of the Hilbert space l ₂ into itself can be approximated by locally infinitely differentiable maps having a Lipschitz derivative.