Neural networks and finite-order approximations

Email: Curry{at}Cardiff.ac.uk This paper investigates the approximation properties of standardfeedforward neural networks (NNs) through the application ofmultivanate Thylor-series expansions. The capacity to approximatearbitrary functional forms is central to the NN philosophy,but is usually proved by allowing the number of hidden nodesto increase to infinity. The Thylor-series approach does notdepend on such limiting cases, lie paper shows how the seriesapproximation depends on individual network weights. The roleof the bias term is taken as an example. We are also able tocompare the sigmoid and hyperbolic-tangent activation functions,with particular emphasis on their impact on the bias term. Thepaper concludes by discussing the potential importance of ourresults for NN modelling: of particular importance is the trainingprocess.     

Neural networks and flexible approximations

Email: cuny{at}cardiff.ac.uk This paper supports the view that neural networks are best seenas devices that can approximate a wide range of functions. Theauthors argue the need to consider the precise details of howthe approximations operate in practice. The paper shows howstandard network models can be regarded as polynomial functions,obtained from expanding exponential terms. Multivariate Taylor-seriesexpansions, obtained through the MAPLE software package, areused for this purpose. The expansions serve to cast light onthe role of the hidden nodes. Also considered is the relativedifficulty of fitting different types of function. Quadraticfunctions are compared with Gaussian shapes.     

一类神经网络全局渐近稳定性分析

在这篇文章里,我们研究了一类时滞神经网络的平衡点的存在,唯一性,及其全局渐近稳定性(GA S).我们的主要思想方法是同胚映射,李雅谱诺夫泛函方法,我们在很弱的条件下解决一类时滞神经网络的GA S.     

Neural networks for of continuous optimal approximation functions in R^d

Using some regular matrices we present a method to express any multivariate algebraic polynomial of total order n in a normal form. Consequently, we prove constructively that, to approximate continuous target functions defined on some compact set of ? ^d , neural networks are at least as good as algebraic polynomials.     

Neural networks with an infinite number of cells

We give existence and uniqueness results for the equations describing the dynamics of some neural networks for which there are infinitely many cells.     

Neural networks and reaction-diffusion systems with prescribed dynamics

Reaction-diffusion systems and neural networks are considered. We prove that they can produce any structurally stable inertial dynamics.     

Neural networks approach for determining total claim amounts in insurance

In this study, we present an approach based on neural networks, as an alternative to the ordinary least squares method, to describe the relation between the dependent and independent variables. It has been suggested to construct a model to describe the relation between dependent and independent variables as an alternative to the ordinary least squares method. A new model, which contains the month and number of payments, is proposed based on real data to determine total claim amounts in insurance as an alternative to the model suggested by Rousseeuw et al. (1984) [Rousseeuw, P., Daniels, B., Leroy, A., 1984. Applying robust regression to insurance. Insurance: Math. Econom. 3, 67–72] in view of an insurer.     

Neural networks,linear functions and neglected non-linearity

The multiplicity of approximation theorems for Neural Networks do not relate to approximation of linear functions per se. The problem for the network is to construct a linear function by superpositions of non-linear activation functions such as the sigmoid function. This issue is important for applications of NNs in statistical tests for neglected nonlinearity, where it is common practice to include a linear function through skip-layer connections. Our theoretical analysis and evidence point in a similar direction, suggesting that the network can in fact provide linear approximations without additional assistance. Our paper suggests that skip-layer connections are unnecessary, and if employed could lead to misleading results.Received: August 2002, Revised: March 2003, AMS Classification: 82c32The authors are grateful to Prof. Mick Silver and to GFK Marketing for help with the provision of data.     

Neural networks and seasonality: Some technical considerations

Debate continues regarding the capacity of feedforward neural networks (NNs) to deal with seasonality without pre-processing. The purpose of this paper is to provide, with examples, some theoretical perspective for the debate. In the first instance it considers possible specification errors arising through use of autoregressive forms. Secondly, it examines seasonal variation in the context of the so-called ‘universal approximation’ capabilities of NNs, finding that a short (bounded) sinusoidal series is easy for the network but that a series with many turning points becomes progressively more difficult. This follows from results contained in one of the seminal papers on NN approximation. It is confirmed in examples which also show that, to model seasonality with NNs, very large numbers of hidden nodes may be required.     

Neural and delay based heuristics for the Steiner problem in networks

High speed networks such as the B-ISDN must be adequately equipped to handle multipoint communication in a fast and economical manner. Multicast applications include desktop video conferencing, distance learning, distributed database applications, etc. In networks employing the asynchronous transfer mode (ATM) technology, routing a multicast is achieved by constructing a tree that spans the source and all the destinations. For the purpose of routing, the network is modeled as a weighted, undirected graph. The graph-theoretic solution is to find a minimum Steiner tree for the graph given a set of destinations. This formulation suffices for building multicast trees with a single optimization constraint as would be the xcase for best effort transport. For real-time traffic, however, it is necessary to ensure that the delay between the sender and each of the receivers is bounded. In this case the network is modeled as an undirected graph, where the edges have both a cost and a delay associated with them. The graph-theoretic solution is then to find a constrained minimum Steiner tree such that the delay between the source and each of the destinations does not violate the specified bound. Both of these problems are NP-complete. In this paper we review prior work on the multipoint routing problem and discuss the formulation of the unconstrained and constrained Steiner problems. We use the random neural network (RNN) to significantly improve the quality of trees found by the two existing best heuristics for finding Steiner trees - the minimum spanning tree heuristic and the average distance heuristic. We also develop a new heuristic for finding delay constrained Steiner trees. Experimental results are presented which show that the new heuristics improve significantly over existing ones.     

Neural networks for a class of bi-level variational inequalities

In this paper, a class of bi-level variational inequalities for describing some practical equilibrium problems, which especially arise from engineering, management and economics, is presented, and a neural network approach for solving the bi-level variational inequalities is proposed. The energy function and neural dynamics of the proposed neural network are defined in this paper, and then the existence of the solution and the asymptotic stability of the neural network are shown. The simulation algorithm is presented and the performance of the proposed neural network approach is demonstrated by some numerical examples.     

Neural networks and organizational systems: Modeling non-linear relationships

For decades, organizational researchers have employed standard statistical methods to uncover relationships among variables and constructs. However, in complex organization systems, the prevalence of non-linearity and outliers is to be expected. Under such circumstances, the use of standard statistical methods becomes unreliable and, correspondingly, results in degraded predictions of the relationships within the organizational systems. We describe the use of neural network analyses to model team effectiveness so as to provide more accurate predictions for managers.     

Neural networks for solving second-order cone constrained variational inequality problem

In this paper, we consider using the neural networks to efficiently solve the second-order cone constrained variational inequality (SOCCVI) problem. More specifically, two kinds of neural networks are proposed to deal with the Karush-Kuhn-Tucker (KKT) conditions of the SOCCVI problem. The first neural network uses the Fischer-Burmeister (FB) function to achieve an unconstrained minimization which is a merit function of the Karush-Kuhn-Tucker equation. We show that the merit function is a Lyapunov function and this neural network is asymptotically stable. The second neural network is introduced for solving a projection formulation whose solutions coincide with the KKT triples of SOCCVI problem. Its Lyapunov stability and global convergence are proved under some conditions. Simulations are provided to show effectiveness of the proposed neural networks.     

Neural networks analysis of free laminar convection heat transfer in a partitioned enclosure

The feasibility of using neural networks (NNs) to predict the complete thermal and flow variables throughout a complicated domain, due to free convection, is demonstrated. Attention is focused on steady, laminar, two-dimensional, natural convective flow within a partitioned cavity. The objective is to use NN (trained on a database generated by a CFD analysis of the problem of a partitioned enclosure) to predict new cases; thus saving effort and computation time. Three types of NN are evaluated, namely General Regression NNs, Polynomial NNs, and a versatile design of Backpropagation neural networks. An important aspect of the study was optimizing network architecture in order to achieve best performance. For each of the three different NN architectures evaluated, parametric studies were performed to determine network parameters that best predict the flow variables.A CFD simulation software was used to generate a database that covered the range of Rayleigh number Ra = 10⁴–5 × 10⁶. The software was used to calculate the temperature, the pressure, and the horizontal and vertical components of flow speed. The results of the CFD were used for training and testing the neural networks (NN). The robustness of the trained NNs was tested by applying them to a “production” data set (1500 patterns for Ra = 8 × 10⁴ and 1500 patterns for Ra = 3 × 10⁶), which the networks have never been “seen” before. The results of applying the technique on the “production” data set show excellent prediction when the NNs are properly designed. The success of the NN in accurately predicting free convection in partitioned enclosures should help reduce analysis-time and effort. Neural networks could potentially help solve some cases in which CFD fails to solve because of numerical instability.     

Neural networks with discrete and distributed time-varying delays: A general stability analysis

In this paper, the global asymptotic and exponential stability are investigated for a class of neural networks with both the discrete and distributed time-varying delays. By using appropriate Lyapunov–Krasovskii functional and linear matrix inequality (LMI) technique, several delay-dependent sufficient conditions are obtained to guarantee the global asymptotic and exponential stability of the addressed neural networks. These conditions are expressed in terms of LMIs, and are dependent on both the discrete and distributed time delays. Therefore, the stability of the neural networks can be checked readily by resorting to the Matlab LMI toolbox. In addition, the proposed stability criteria do not require the monotonicity of the activation functions and the differentiability of the discrete and distributed time-varying delays, which means that our results generalize and further improve those in the earlier publications. A simulation example is given to show the effectiveness and less conservatism of the obtained conditions.     

Neural networks as set-valued dynamical systems and the universality of the windowed Fourier transform

Summary The visual pathway and other brain structures consist of a large number of layers of neurons. At each point of a three-dimensional laminated structure there exists a direction that is perpendicular to the layers. Assuming an information flow from top to bottom, the perceptive field of the neurons grows as one moves in the direction . This enables the system to perform a multiscale analysis.Suppose that the density of the connections between adjacent layers is distributed by a Gaussian function and the autocorrelationQ of the input is of the formQ(x,x) =Q(¦x-x¦) (i.e., shift invariant). Then it is shown that the laminated system does indeed converge to some universal attractor. Under certain conditions, the universal attractor takes the form of the Gabor filter (windowed Fourier transform). This enables the net to combine multiscale resolution with spectral analysis over a small portion of the global receptive field. Under more general conditions the transition from layeri to layeri + 1 is given by a set-valued dynamical system, and partial results on its global behavior are given.Supported by the GFAT Academic Lectureship-France.     

Exponential stability and periodic solution for fuzzy BAM Neural networks with time varying delays

In this paper, a class of fuzzy BAM neural networks with time varying delays is discussed. By using the properties of M-matrix, Linear Matrix Inequality（LMI） approach and general Lyapunov-Krasovskii functional, some new sufficient conditions are derived to ensure the existence of periodic solutions and the global exponential stability of the fuzzy BAM neural networks with time varying delays. These results have important significance in the design of global exponential stable BAM networks with delays. Moreover, an example is given to illustrate that the conditions of the results in the paper are feasible.