Self averaging and the space of interactions in neural networks

We prove (through a precise exponential inequality) that the logarithm of the size of the intersection of M random half spaces with the unit sphere of ℝ^N (resp., the discrete cube {−1, 1}^N) is, as N→∞, a self averaging quantity. This provides justification for one of the first steps of a famous computation by E. Gardner [J. Phys. A 21 (1988), 257–270]. ©1999 John Wiley & Sons, Inc. Random Struct. Alg., 14, 199–213, 1999     

Prediction of bond ratings using statistical and neural networks techniques

Quantitative models have been developed to predict bond ratings of firms in the Indian manufacturing sector, using financial leverage, profitability, asset management ability, stability and market sensitivity of the firm, which totally involved 16 variables. These 16 independent variables are first reduced to seven orthogonal variables using principal component analysis. Then these variables are used to build three types of models, namely Multiple Discriminant Analysis (MDA), Multinomial Logistic Regression (MLR) and Artificial Neural Networks (ANN). Based on both in-sample classification and out-sample validation it is found that both MLR and ANN models are superior to MDA, with little difference in performance between themselves. (© 2008 WILEY-VCH Verlag GmbH & Co. KGaA, Weinheim)     

Classification by ensembles of neural networks

We introduce a new procedure for training of artificial neural networks by using the approximation of an objective function by arithmetic mean of an ensemble of selected randomly generated neural networks, and apply this procedure to the classification (or pattern recognition) problem. This approach differs from the standard one based on the optimization theory. In particular, any neural network from the mentioned ensemble may not be an approximation of the objective function.     

Chebyshev approximation by discrete superposition. Application to neural networks

In this paper, we develop two algorithms for Chebyshev approximation of continuous functions on [0, 1] ⁿ using the modulus of continuity and the maximum norm estimated by a given finite data system. The algorithms are based on constructive versions of Kolmogorov's superposition theorem. One of the algorithms we apply to neural networks.     

Approximation by series of sigmoidal functions with applications to neural networks

Annali di Matematica Pura ed Applicata (1923 -) - In this paper, we develop a constructive theory for approximating absolutely continuous functions by series of certain sigmoidal functions....     

The rate of approximation of Gaussian radial basis neural networks in continuous function space

There have been many studies on the dense theorem of approximation by radial basis feedforword neural networks, and some approximation problems by Gaussian radial basis feedforward neural networks(GRBFNs)in some special function space have also been investigated. This paper considers the approximation by the GRBFNs in continuous function space. It is proved that the rate of approximation by GRNFNs with n~d neurons to any continuous function f defined on a compact subset K(R~d)can be controlled by ω(f, n~(-1/2)), where ω(f, t)is the modulus of continuity of the function f .     

Using artificial neural networks to solve the orienteering problem

In the orienteering problem, we are given a transportation network in which a start point and an end point are specified. Other points have associated scores. Given a fixed amount of time, the goal is to determine a path from start to end through a subset of locations in order to maximize the total path score. This problem has received a considerable amount of attention in the last ten years. The traveling salesman problem is a variant of the orienteering problem. This paper applies a modified, continuous Hopfield neural network to attack this NP-hard optimization problem. In it, we design an effective energy function and learning algorithm. Unlike some applications of neural networks to optimization problems, this approach is shown to perform quite well.     

Structural evaluation of materials by artificial neural networks

It is shown that due to the complexity of interaction of the excitation field with a material in determining its physical characteristics, as well as sophisticated correlation relationships between the physical characteristics and structure of a real material, in many cases, relization of the structural evaluation of materials on the basis of idealized mathematical models of the underlying physical processes is of limited use. As an alternative, it is proposed to use an artificial neural network for the extraction of quantitative information of interest from measurements of the physical characteristics. A general overview of artificial neural networks is given. A methodology of the use of a multilayer perceptron for determining various structural parameters from the dielectric spectra is described. As an example, determination of the moisture content and density of sawdust from the dielectric modulusis considered by the neural-network technique. The noise performance of the neural network is analyzed by applying an additive and multiplicative noise to the input data.Institute of Polymer Mechanics, University of Latvia, Riga, LV-1006 Latvia. Published in Mekhanika Kompozitnykh Materialov, Vol. 35, No. 1, pp. 109–124, January–February, 1999.     

On the dynamics of hypervelocity liquid jet impact on a flat rigid surface

The phenomena associated with the impact of a right circular fluid cylinder with a flat rigid surface are studied with the aid of a two-dimensional axisymmetric finite difference code. Both sub-and supersonic initial conditions are investigated. It is shown that, contrary to earlier reports, the maximum pressure sustained is precisely the one-dimensional maximum, i.e., the shock Hugoniot. The relaxation time corresponds to the time for the edge rarefaction, initiating at the impact corner, to traverse the jet radius. The maximum lateral speed, at the impact corner, was found to be nearly twice that of impact, slightly higher for very low Mach number and slightly lower for supersonic impact.     

Using neural networks to forecast the systematic risk of stocks

This paper discusses different methods of predicting a stock's systematic risk, using the financial statements of 67 German corporations from the period 1967 to 1986. We show that the most precise forecasts are given by neural networks, whose topology has been optimized by a genetic algorithm. In addition we analyze and visualize the dependencies that influence the forecasts of a stock's systematic risk.     

Numerical solution of the nonlinear Schrodinger equation by feedforward neural networks

We present a method to solve boundary value problems using artificial neural networks (ANN). A trial solution of the differential equation is written as a feed-forward neural network containing adjustable parameters (the weights and biases). From the differential equation and its boundary conditions we prepare the energy function which is used in the back-propagation method with momentum term to update the network parameters. We improved energy function of ANN which is derived from Schrodinger equation and the boundary conditions. With this improvement of energy function we can use unsupervised training method in the ANN for solving the equation. Unsupervised training aims to minimize a non-negative energy function. We used the ANN method to solve Schrodinger equation for few quantum systems. Eigenfunctions and energy eigenvalues are calculated. Our numerical results are in agreement with their corresponding analytical solution and show the efficiency of ANN method for solving eigenvalue problems.     

An approach to the solution of recognition problems using neural networks

Conditions are determined under which, for pattern recognition problems with standard information (Ω-regular problems), a correct algorithm and a six-level spatial neural network reproducing the calculations performed by the correct algorithm can be constructed. The proposed approach to constructing the neural network is not related to the traditional approach based on minimizing a functional.     

Learning of Chua's circuit attractors by locally recurrent neural networks

Many practical applications of neural networks require the identification of strongly non-linear (e.g., chaotic) systems. In this paper, locally recurrent neural networks (LRNNs) are used to learn the attractors of Chua's circuit, a paradigm for studying chaos. LRNNs are characterized by a feed-forward structure whose synapses between adjacent layers have taps and feedback connections. In general, the learning procedures of LRNNs are computationally simpler than those of globally recurrent networks. Results show that LRNNs can be trained to identify the underlying link among Chua's circuit state variables, and exhibit chaotic attractors under autonomous working conditions.     

On the near optimality of the stochastic approximation of smooth functions by neural networks

We consider the problem of approximating the Sobolev class of functions by neural networks with a single hidden layer, establishing both upper and lower bounds. The upper bound uses a probabilistic approach, based on the Radon and wavelet transforms, and yields similar rates to those derived recently under more restrictive conditions on the activation function. Moreover, the construction using the Radon and wavelet transforms seems very natural to the problem. Additionally, geometrical arguments are used to establish lower bounds for two types of commonly used activation functions. The results demonstrate the tightness of the bounds, up to a factor logarithmic in the number of nodes of the neural network. This revised version was published online in June 2006 with corrections to the Cover Date.     

Techniques for the mathematical analysis of neural networks

This expository paper covers the following topics: (1) a very brief introduction to neural networks for those unfamiliar with the basic concepts; (2) an equality brief survey of various mathematical approaches to neural systems with an emphasis on approximation theory; (3) an algorithmic approach to the analysis of networks developed by this author using the tools of numerical linear algebra. This approach is novel and was first proposed by the author in (1990).

A detailed analysis of one popular algorithm (the delta rule) will be given, indicating why one implementation leads to a stable numerical process, whereas an initially attractive variant (essentially a form of steepest descent) does not. Similar considerations apply to the backpropagation algorithm. The effect of filtering and other preprocessing of the input data will also be discussed systematically, with a new result on the effect of linear filtering on the rate of convergence of the delta rule.     

On the complexity of loading shallow neural networks

We formalize a notion of loading information into connectionist networks that characterizes the training of feed-forward neural networks. This problem is NP-complete, so we look for tractable subcases of the problem by placing constraints on the network architecture. The focus of these constraints is on various families of “shallow” architectures which are defined to have bounded depth and un-bounded width. We introduce a perspective on shallow networks, called the Support Cone Interaction (SCI) graph, which is helpful in distinguishing tractable from intractable subcases: When the SCI graph is a tree or is of limited bandwidth, loading can be accomplished in polynomial time; when its bandwidth is not limited we find the problem NP-complete even if the SCI graph is a simple 2-dimensional planar grid.     

A characteristic of the space of orbital complex-valued functions on a compactum

A characteristic of the spaces of continuous orbital complex-valued functions is given on a Hausdorff compactum in the category of complex Banach spaces.