Parameter redundancy in neural networks: an application of Chebyshev polynomials

This paper deals with feedforward neural networks containing a single hidden layer and with sigmoid/logistic activation function. Training such a network is equivalent to implementing nonlinear regression using a flexible functional form, but the functional form in question is not easy to deal with. The Chebyshev polynomials are suggested as a way forward, providing an approximation to the network which is superior to Taylor series expansions. Application of these approximations suggests that the network is liable to a ‘naturally occurring’ parameter redundancy, which has implications for the training process as well as certain statistical implications. On the other hand, parameter redundancy does not appear to damage the fundamental property of universal approximation.      

On a smoothness problem in ridge function representation

In this paper we prove that if a function of a certain smoothness class is represented by a sum of r arbitrarily behaved ridge functions, then, under suitable conditions, it can be represented by a sum of ridge functions of the same smoothness class and some polynomial of a certain degree.     

The construction and approximation of some neural networks operators

In this paper,the technique of approximate partition of unity is used to construct a class of neural networks operators with sigmoidal functions.Using the modulus of continuity of function as a metric,...     

Smoothness and uniqueness in ridge function representation

In this note we consider problems of uniqueness, smoothness and representation of linear combinations of a finite number of ridge functions with fixed directions.     

Approximation by ridge function fields over compact sets

We study the approximation of a continuous function field over a compact set T by a continuous field of ridge approximants over T, named ridge function fields. We first give general density results about function fields and show how they apply to ridge function fields. We next discuss the parameterization of sets of ridge function fields and give additional density results for a class of continuous ridge function fields that admits a weak parameterization. Finally, we discuss the construction of the elements in that class.     

Approximation by ridge functions and neural networks with a bounded number of neurons

In this paper, we consider the problem of approximation of continuous multivariate functions by neural networks with a bounded number of neurons in hidden layers. We prove the existence of single-hidden-layer networks with bounded number of neurons, which have approximation capabilities not worse than those of networks with arbitrarily many neurons. Our analysis is based on the properties of ridge functions.     

The estimate for approximation error of spherical neural networks

Compared with planar hyperplane, fitting data on the sphere has been an important and an active issue in geoscience, metrology, brain imaging, and so on. In this paper, with the help of the Jackson‐type theorem of polynomial approximation on the sphere, we construct spherical feed‐forward neural networks to approximate the continuous function defined on the sphere. As a metric, the modulus of smoothness of spherical function is used to measure the error of the approximation, and a Jackson‐type theorem on the approximation is established. Copyright © 2011 John Wiley & Sons, Ltd.     

Approximation of Sobolev classes by polynomials and ridge functions

Let be the usual Sobolev class of functions on the unit ball in , and be the subclass of all radial functions in . We show that for the classes and , the orders of best approximation by polynomials in coincide. We also obtain exact orders of best approximation in of the classes by ridge functions and, as an immediate consequence, we obtain the same orders in for the usual Sobolev classes .     

The construction and approximation of feedforward neural network with hyp erb olic tangent function

In this paper, we discuss some analytic properties of hyperbolic tangent function and estimate some approximation errors of neural network operators with the hyperbolic tangent activation functionFirstly, an equation of partitions of unity for the hyperbolic tangent function is givenThen, two kinds of quasi-interpolation type neural network operators are constructed to approximate univariate and bivariate functions, respectivelyAlso, the errors of the approximation are estimated by means of the modulus of continuity of functionMoreover, for approximated functions with high order derivatives, the approximation errors of the constructed operators are estimated.     

Pollaczek polynomials and hypergeometric representation

The Pfaff-Euler Transform for hypergeometric ₂ F ₁-series is applied to provide a direct and elementary proof that the hypergeometric representation with algebraic parameters of Pollaczek polynomials are indeed polynomials. Dedicated to Richard Askey on the occasion of his 70th birthday. 2000 Mathematics Subject Classification Primary—33C45; Secondary—33C05     

Some properties and an application of multivariate exponential polynomials

In the paper, the authors introduce a notion “multivariate exponential polynomials” which generalize exponential numbers and polynomials, establish explicit formulas, inversion formulas, and recurrence relations for multivariate exponential polynomials in terms of the Stirling numbers of the first and second kinds with the help of the Faà di Bruno formula, two identities for the Bell polynomials of the second kind, and the inversion theorem for the Stirling numbers of the first and second kinds, construct some determinantal inequalities and product inequalities for multivariate exponential polynomials with the aid of some properties of completely monotonic functions and other known results, derive the logarithmic convexity and logarithmic concavity for multivariate exponential polynomials, and finally find an application of multivariate exponential polynomials to white noise distribution theory by confirming that multivariate exponential polynomials satisfy conditions for sequences required in white noise distribution theory.     

Independence polynomials of circulants with an application to music

The independence polynomial of a graph G is the generating function I(G,x)=∑_k≥0i_kx^k, where i_k is the number of independent sets of cardinality k in G. We show that the problem of evaluating the independence polynomial of a graph at any fixed non-zero number is intractable, even when restricted to circulants. We provide a formula for the independence polynomial of a certain family of circulants, and its complement. As an application, we derive a formula for the number of chords in an n-tet musical system (one where the ratio of frequencies in a semitone is 2^1/n) without ‘close’ pitch classes.     

Approximation by radial bases and neural networks

In this paper, we study approximation by radial basis functions including Gaussian, multiquadric, and thin plate spline functions, and derive order of approximation under certain conditions. Moreover, neural networks are also constructed by wavelet recovery formula and wavelet frames.     

The rate of approximation of Gaussian radial basis neural networks in continuous function space

There have been many studies on the dense theorem of approximation by radial basis feedforword neural networks, and some approximation problems by Gaussian radial basis feedforward neural networks(GRBFNs)in some special function space have also been investigated. This paper considers the approximation by the GRBFNs in continuous function space. It is proved that the rate of approximation by GRNFNs with n~d neurons to any continuous function f defined on a compact subset K(R~d)can be controlled by ω(f, n~(-1/2)), where ω(f, t)is the modulus of continuity of the function f .     

Neural networks for of continuous optimal approximation functions in R^d

Using some regular matrices we present a method to express any multivariate algebraic polynomial of total order n in a normal form. Consequently, we prove constructively that, to approximate continuous target functions defined on some compact set of ? ^d , neural networks are at least as good as algebraic polynomials.     

基于散乱阈值结点的周期神经网络的逼近阶

§1Introduction Inrecentyearstherehasbeengrowinginterestintheproblemofneuralnetworkand relatedapproximation,manyimportantresultsareobtained.Becauseofitsabilityof parallelcomputationinlargescaleandofperfectself-adaptingandapproximation,the neuralnetworkhasbeenwidelyapplied.Theapproximationabilityoftheneuralnetwork dependsonitstopologicalstructure.LetRsbeans-dimensionalEuclidSpaceand(x)isa realfunctiondefinedonRs.When(x)isanexcitationfunctionandx∈Rsisaninput vector,thesimpleneuralnetwork…     

The penetration function and its application to microscale problems

The penetration function measures the effect of the boundary data on the energy of the solution of a second order linear elliptic PDE taken over an interior subdomain. Here the coefficients of the PDE are functions of position and often represent the material properties of non homogeneous media with microstructure. The penetration function is used to assess the accuracy of global-local approaches for recovering local solution features from coarse grained solutions such as those delivered by homogenization theory. AMS subject classification (2000)  65N15, 78M40     

The capability of approximation for neural networks interpolant on the sphere

Compared with planar hyperplane, fitting data on the sphere has been an important and active issue in geoscience, metrology, brain imaging, and so on. In this paper, using a functional approach, we rigorously prove that for given distinct samples on the unit sphere there exists a feed‐forward neural network with single hidden layer which can interpolate the samples, and simultaneously near best approximate the target function in continuous function space. Also, by using the relation between spherical positive definite radial basis functions and the basis function on the Euclidean space ?^{d + 1}, a similar result in a spherical Sobolev space is established. Copyright © 2010 John Wiley & Sons, Ltd.     

Chebyshev approximation by discrete superposition. Application to neural networks

In this paper, we develop two algorithms for Chebyshev approximation of continuous functions on [0, 1] ⁿ using the modulus of continuity and the maximum norm estimated by a given finite data system. The algorithms are based on constructive versions of Kolmogorov's superposition theorem. One of the algorithms we apply to neural networks.