Solving regression problem with complementary neural networks and an adjusted averaging technique

This research deals with complementary neural networks (CMTNN) for the regression problem. Complementary neural networks consist of a pair of neural networks called truth neural network and falsity neural network, which are trained to predict truth and falsity outputs, respectively. In this paper, a novel adjusted averaging technique is proposed in order to enhance the result obtained from the basic CMTNN. We test our proposed technique based on the classical benchmark problems including housing, concrete compressive strength, and computer hardware data sets from the UCI machine learning repository. We also realize our technique to the porosity prediction problem based on well log data set obtained from practical field data in the oil and gas industry. We found that our proposed technique provides better performance when compared to the traditional CMTNN, backpropagation neural network, and support vector regression with linear, polynomial, and radial basis function kernels.     

Cauchy problem and averaging in Fock space

We investigate a Cauchy problem in the Fock space for a system consisting of a two-level atom, a quantum field, and a classical field. A solution estimate is obtained for the Cauchy problem with initial data from a special class. This class is invariant with respect to the dynamic semigroup of the system. We propose an averaging method for solving the Cauchy problem in the case where the Hamiltonian parameters differ greatly in the order of magnitude. An estimate of the averaging error is obtained. Translated from Teoreticheskaya i Matematicheskaya Fizika, Vol. 117, No. 1, pp. 92–106, October, 1998.     

Programmed interactions in higher-order neural networks: The outer-product algorithm

Recent results on the memory storage capacity of the outer-product algorithm indicate that the algorithm stores of the order of n/log n memories in a network of n fully interconnected linear threshold elements when it is required that each memory be exactly recovered from a probe which is close enough to it. In this paper a rigourous analysis is presented of generalizations of the outer-product algorithm to higher-order networks of densely interconnected polynomial thresh-old units of degree d. Precise notions of memory storage capacity are formulated, and it is demonstrated that both static and dynamic storage capacities of all variants of the outer-product algorithm of degree d are of the order of n^d/log n.     

Prediction of hypervelocity orbital debris impact damage to the space station by neural networks

Prediction of damage to orbiting space craft due to collisions with hypervelocity space debris is an important issue in the design of Space Station Freedom. Space station wall structures are designed to absorb impact energy during a collision. A proposed wall structure consists of a multilayer insulation (MLI) directly covering the pressure wall, and a bumper layer placed 100mm from the pressure wall. In experiments at The Marshall Space Flight Center, 2.5–12.7 mm projectiles have been fired at this wall structure at speeds of 2–8 km/s. In this paper, three-layer backpropagation networks are trained with two sets of impact damage data. The input parameters for training are pressure wall thickness, bumper plate thickness, projectile diameter, impact angle, and the projectile velocity. Output from the network consists of hole dimensions for the bumper and the pressure wall in the minor and major axis directions, and damage to the MLI. To evaluate network generalization, networks are tested with experimental data points that are not used for training. Network performance is compared with that of other damage prediction methods. Network determination of qualitative damage estimation is suggested as a new direction for research. Preliminary testing of qualitative prediction of pressure wall damage is presented. The results are promising, and suggest several areas for further study.     

Feed forward neural networks modeling for K–P interactions

Artificial intelligence techniques involving neural networks became vital modeling tools where model dynamics are difficult to track with conventional techniques. The paper make use of the feed forward neural networks (FFNN) to model the charged multiplicity distribution of K–P interactions at high energies. The FFNN was trained using experimental data for the multiplicity distributions at different lab momenta. Results of the FFNN model were compared to that generated using the parton two fireball model and the experimental data. The proposed FFNN model results showed good fitting to the experimental data. The neural network model performance was also tested at non-trained space and was found to be in good agreement with the experimental data.     

The rate of approximation of Gaussian radial basis neural networks in continuous function space

There have been many studies on the dense theorem of approximation by radial basis feedforword neural networks, and some approximation problems by Gaussian radial basis feedforward neural networks(GRBFNs)in some special function space have also been investigated. This paper considers the approximation by the GRBFNs in continuous function space. It is proved that the rate of approximation by GRNFNs with n~d neurons to any continuous function f defined on a compact subset K(R~d)can be controlled by ω(f, n~(-1/2)), where ω(f, t)is the modulus of continuity of the function f .     

Stochastic optimization in supply chain networks: averaging robust solutions

We propose a novel robust optimization approach to analyze and optimize the expected performance of supply chain networks. We model uncertainty in the dema     

Activation clustering in neural and social networks

Questions related to the evolution of the structure of networks have received recently a lot of attention in the literature. But what is the state of the network given its structure? For example, there is the question of how the structures of neural networks make them behave? Or, in the case of a network of humans, the question could be related to the states of humans in general, given the structure of the social network. The models based on stochastic processes developed in this article, do not attempt to capture the fine details of social or neural dynamics. Rather they aim to describe the general relationship between the variables describing the network and the aggregate behavior of the network. A number of nontrivial results are obtained using computer simulations. © 2005 Wiley Periodicals, Inc. Complexity 10: 42–50, 2005     

Interacting neural networks and the emergence of social structure

The article describes a computational model for the simulation of the emergence of social structure or social order, respectively. The model is theoretically based on the theory of social typifying by Berger and Luckmann. It consists of interacting artificial actors (agents), which are represented by two neural networks, an action net, and a perception net. By mutually adjusting of their actions, the agents are able to constitute a self‐organized social order in dependency of their personal characteristics and certain features of their environment. A fictitious example demonstrates the applicability of the model to problems of extra‐terrestrial robotics. © 2007 Wiley Periodicals, Inc. Complexity 12: 41–52, 2007     

Frequency transitions in synchronized neural networks

Temporal organization of events can emerge in complex systems, like neural networks. Here, random graph and cellular automaton are used to represent coupled neural structures, in order to investigate the occurrence of synchronization. The connectivity pattern of this toy model of neural system is of Newman–Watts type, formed from a regular lattice with additional random connections. Two networks with this coupling topology are connected by extra random links and an impulse stimulus is either constantly or periodically applied to a unique neuron. Numerical simulations reveal that this model can exhibit a variety of dynamic behaviors. Usually, the whole system achieves synchronization; however, the oscillation frequencies of the stimulus and of each network can be different. The dynamics is evaluated in function of the network size, the amount of the randomly added edges and the number of time steps in which a neuron can remain firing. The biological relevance of these results is discussed.     

Kohonen neural networks and genetic classification

We discuss the property of a.e. and in mean convergence of the Kohonen algorithm considered as a stochastic process. The various conditions ensuring a.e. convergence are described and the connection with the rate decay of the learning parameter is analyzed. The rate of convergence is discussed for different choices of learning parameters. We prove rigorously that the rate of decay of the learning parameter which is most used in the applications is a sufficient condition for a.e. convergence and we check it numerically. The aim of the paper is also to clarify the state of the art on the convergence property of the algorithm in view of the growing number of applications of the Kohonen neural networks. We apply our theorem and considerations to the case of genetic classification which is a rapidly developing field.     

Universal computation in fluid neural networks

Fluid neural networks can be used as a theoretical framework for a wide range of complex systems as social insects. In this article we show that collective logical gates can be built in such a way that complex computation can be possible by means of the interplay between local interactions and the collective creation of a global field. This is exemplified by a NOR gate. Some general implications for ant societies are outlined. © 1996 John Wiley & Sons, Inc.     

Critical branching processes in neural networks

Self-organized criticality generates complex behavior in systems of simple elements. It is observed in various biological neural systems and has been analyzed in simplified model systems. Branching processes often considered to be a mean-field approximation to the dynamics of critical systems. Here we study the validity of such an approximation for the case of a neural network. (© 2008 WILEY-VCH Verlag GmbH & Co. KGaA, Weinheim)     

Ehresmann connections and feedforward neural networks

In this paper, we describe how Ehresmann connections can be used to study certain properties of feedforward neural networks. Essentially, we calculate a Lie group approximation to the structure of the inverse image set above a certain point in the output space and this structure can then be locally transported to the inverse image above a neighbouring point in the output space by means of an Ehresmann connection. This enables us to find a continuous approximation to the underlying topological structure of the data from discrete data pairs (input/output pairs).     

Advances in verification of ReLU neural networks

Journal of Global Optimization - We consider the problem of verifying linear properties of neural networks. Despite their success in many classification and prediction tasks, neural networks may...     

Techniques for the mathematical analysis of neural networks

This expository paper covers the following topics: (1) a very brief introduction to neural networks for those unfamiliar with the basic concepts; (2) an equality brief survey of various mathematical approaches to neural systems with an emphasis on approximation theory; (3) an algorithmic approach to the analysis of networks developed by this author using the tools of numerical linear algebra. This approach is novel and was first proposed by the author in (1990).

A detailed analysis of one popular algorithm (the delta rule) will be given, indicating why one implementation leads to a stable numerical process, whereas an initially attractive variant (essentially a form of steepest descent) does not. Similar considerations apply to the backpropagation algorithm. The effect of filtering and other preprocessing of the input data will also be discussed systematically, with a new result on the effect of linear filtering on the rate of convergence of the delta rule.     

On the complexity of loading shallow neural networks

We formalize a notion of loading information into connectionist networks that characterizes the training of feed-forward neural networks. This problem is NP-complete, so we look for tractable subcases of the problem by placing constraints on the network architecture. The focus of these constraints is on various families of “shallow” architectures which are defined to have bounded depth and un-bounded width. We introduce a perspective on shallow networks, called the Support Cone Interaction (SCI) graph, which is helpful in distinguishing tractable from intractable subcases: When the SCI graph is a tree or is of limited bandwidth, loading can be accomplished in polynomial time; when its bandwidth is not limited we find the problem NP-complete even if the SCI graph is a simple 2-dimensional planar grid.     

State space collapse and stability of queueing networks

We study the stability of subcritical multi-class queueing networks with feedback allowed and a work-conserving head-of-the-line service discipline. Assuming that the fluid limit model associated to the queueing network satisfies a state space collapse condition, we show that the queueing network is stable provided that any solution of an associated linear Skorokhod problem is attracted to the origin in finite time. We also give sufficient conditions ensuring this attraction in terms of the reflection matrix of the Skorokhod problem, by using an adequate Lyapunov function. State space collapse establishes that the fluid limit of the queue process can be expressed in terms of the fluid limit of the workload process by means of a lifting matrix.