Parameter redundancy in neural networks: an application of Chebyshev polynomials

This paper deals with feedforward neural networks containing a single hidden layer and with sigmoid/logistic activation function. Training such a network is equivalent to implementing nonlinear regression using a flexible functional form, but the functional form in question is not easy to deal with. The Chebyshev polynomials are suggested as a way forward, providing an approximation to the network which is superior to Taylor series expansions. Application of these approximations suggests that the network is liable to a ‘naturally occurring’ parameter redundancy, which has implications for the training process as well as certain statistical implications. On the other hand, parameter redundancy does not appear to damage the fundamental property of universal approximation.      

On a smoothness problem in ridge function representation

In this paper we prove that if a function of a certain smoothness class is represented by a sum of r arbitrarily behaved ridge functions, then, under suitable conditions, it can be represented by a sum of ridge functions of the same smoothness class and some polynomial of a certain degree.     

The construction and approximation of some neural networks operators

In this paper,the technique of approximate partition of unity is used to construct a class of neural networks operators with sigmoidal functions.Using the modulus of continuity of function as a metric,...     

Smoothness and uniqueness in ridge function representation

In this note we consider problems of uniqueness, smoothness and representation of linear combinations of a finite number of ridge functions with fixed directions.     

Approximation by ridge function fields over compact sets

We study the approximation of a continuous function field over a compact set T by a continuous field of ridge approximants over T, named ridge function fields. We first give general density results about function fields and show how they apply to ridge function fields. We next discuss the parameterization of sets of ridge function fields and give additional density results for a class of continuous ridge function fields that admits a weak parameterization. Finally, we discuss the construction of the elements in that class.     

Approximation of Sobolev classes by polynomials and ridge functions

Let be the usual Sobolev class of functions on the unit ball in , and be the subclass of all radial functions in . We show that for the classes and , the orders of best approximation by polynomials in coincide. We also obtain exact orders of best approximation in of the classes by ridge functions and, as an immediate consequence, we obtain the same orders in for the usual Sobolev classes .     

Pollaczek polynomials and hypergeometric representation

The Pfaff-Euler Transform for hypergeometric ₂ F ₁-series is applied to provide a direct and elementary proof that the hypergeometric representation with algebraic parameters of Pollaczek polynomials are indeed polynomials. Dedicated to Richard Askey on the occasion of his 70th birthday. 2000 Mathematics Subject Classification Primary—33C45; Secondary—33C05     

The construction and approximation of feedforward neural network with hyp erb olic tangent function

In this paper, we discuss some analytic properties of hyperbolic tangent function and estimate some approximation errors of neural network operators with the hyperbolic tangent activation functionFirstly, an equation of partitions of unity for the hyperbolic tangent function is givenThen, two kinds of quasi-interpolation type neural network operators are constructed to approximate univariate and bivariate functions, respectivelyAlso, the errors of the approximation are estimated by means of the modulus of continuity of functionMoreover, for approximated functions with high order derivatives, the approximation errors of the constructed operators are estimated.     

Approximation by radial bases and neural networks

In this paper, we study approximation by radial basis functions including Gaussian, multiquadric, and thin plate spline functions, and derive order of approximation under certain conditions. Moreover, neural networks are also constructed by wavelet recovery formula and wavelet frames.     

Independence polynomials of circulants with an application to music

The independence polynomial of a graph G is the generating function I(G,x)=∑_k≥0i_kx^k, where i_k is the number of independent sets of cardinality k in G. We show that the problem of evaluating the independence polynomial of a graph at any fixed non-zero number is intractable, even when restricted to circulants. We provide a formula for the independence polynomial of a certain family of circulants, and its complement. As an application, we derive a formula for the number of chords in an n-tet musical system (one where the ratio of frequencies in a semitone is 2^1/n) without ‘close’ pitch classes.     

The rate of approximation of Gaussian radial basis neural networks in continuous function space

There have been many studies on the dense theorem of approximation by radial basis feedforword neural networks, and some approximation problems by Gaussian radial basis feedforward neural networks(GRBFNs)in some special function space have also been investigated. This paper considers the approximation by the GRBFNs in continuous function space. It is proved that the rate of approximation by GRNFNs with n~d neurons to any continuous function f defined on a compact subset K(R~d)can be controlled by ω(f, n~(-1/2)), where ω(f, t)is the modulus of continuity of the function f .     

基于散乱阈值结点的周期神经网络的逼近阶

§1Introduction Inrecentyearstherehasbeengrowinginterestintheproblemofneuralnetworkand relatedapproximation,manyimportantresultsareobtained.Becauseofitsabilityof parallelcomputationinlargescaleandofperfectself-adaptingandapproximation,the neuralnetworkhasbeenwidelyapplied.Theapproximationabilityoftheneuralnetwork dependsonitstopologicalstructure.LetRsbeans-dimensionalEuclidSpaceand(x)isa realfunctiondefinedonRs.When(x)isanexcitationfunctionandx∈Rsisaninput vector,thesimpleneuralnetwork…     

Neural networks for of continuous optimal approximation functions in R^d

Using some regular matrices we present a method to express any multivariate algebraic polynomial of total order n in a normal form. Consequently, we prove constructively that, to approximate continuous target functions defined on some compact set of ? ^d , neural networks are at least as good as algebraic polynomials.     

The penetration function and its application to microscale problems

The penetration function measures the effect of the boundary data on the energy of the solution of a second order linear elliptic PDE taken over an interior subdomain. Here the coefficients of the PDE are functions of position and often represent the material properties of non homogeneous media with microstructure. The penetration function is used to assess the accuracy of global-local approaches for recovering local solution features from coarse grained solutions such as those delivered by homogenization theory. AMS subject classification (2000)  65N15, 78M40     

Chebyshev approximation by discrete superposition. Application to neural networks

In this paper, we develop two algorithms for Chebyshev approximation of continuous functions on [0, 1] ⁿ using the modulus of continuity and the maximum norm estimated by a given finite data system. The algorithms are based on constructive versions of Kolmogorov's superposition theorem. One of the algorithms we apply to neural networks.     

Approximation by neural networks with sigmoidal functions

In this paper, we introduce a type of approximation operators of neural networks with sigmodal functions on compact intervals, and obtain the pointwise and uniform estimates of the ap- proximation. To improve the approximation rate, we further introduce a type of combinations of neurM networks. Moreover, we show that the derivatives of functions can also be simultaneously approximated by the derivatives of the combinations. We also apply our method to construct approximation operators of neural networks with sigmodal functions on infinite intervals.     

Some inequalities for symmetric functions and an application to orthogonal polynomials

We present some sharp inequalities for symmetric functions and give an application to orthogonal polynomials.     

Fuzzy polynomial regression with fuzzy neural networks

Recently, fuzzy linear regression is considered by Mosleh et al. [1]. In this paper, a novel hybrid method based on fuzzy neural network for approximate fuzzy coefficients (parameters) of fuzzy polynomial regression models with fuzzy output and crisp inputs, is presented. Here a neural network is considered as a part of a large field called neural computing or soft computing. Moreover, in order to find the approximate parameters, a simple algorithm from the cost function of the fuzzy neural network is proposed. Finally, we illustrate our approach by some numerical examples.     

Representation of multivariate functions by sums of ridge functions

In this paper, we find geometric means of deciding if any continuous multivariate function can be represented by sums of two continuous ridge functions.     

A family of univalent polynomials in an angular domain

We study the properties of symmetric polynomials of a particular form and use them to find a maximal angular domain of univalence for some family of polynomials.