Collective behavior of interacting locally synchronized oscillations in neuronal networks

Local circuits in the cortex and hippocampus are endowed with resonant, oscillatory firing properties which underlie oscillations in various frequency ranges (e.g. gamma range) frequently observed in the local field potentials, and in electroencephalography. Synchronized oscillations are thought to play important roles in information binding in the brain. This paper addresses the collective behavior of interacting locally synchronized oscillations in realistic neural networks. A network of five neurons is proposed in order to produce locally synchronized oscillations. The neuron models are Hindmarsh–Rose type with electrical and/or chemical couplings. We construct large-scale models using networks of such units which capture the essential features of the dynamics of cells and their connectivity patterns. The profile of the spike synchronization is then investigated considering different model parameters such as strength and ratio of excitatory/inhibitory connections. We also show that transmission time-delay might enhance the spike synchrony. The influence of spike-timing-dependence-plasticity is also studies on the spike synchronization.     

Backfitting neural networks

Summary  Regression and classification problems can be viewed as special cases of the problem of function estimation. It is rather well known that a two-layer perceptron with sigmoidal transformation functions can approximate any continuous function on the compact subsets ofRP if there are sufficient number of hidden nodes. In this paper, we present an algorithm for fitting perceptron models, which is quite different from the usual backpropagation or Levenberg-Marquardt algorithm. This new algorithm based on backfitting ensures a better convergence than backpropagation. We have also used resampling techniques to select an ideal number of hidden nodes automatically using the training data itself. This resampling technique helps to avoid the problem of overfitting that one faces for the usual perceptron learning algorithms without any model selection scheme. Case studies and simulation results are presented to illustrate the performance of this proposed algorithm.     

Modeling sustainability transitions on complex networks

There has been renewed interest in sociotechnical systems in the context of transitioning to a more sustainable society. While gains have been made in the qualitative understanding of sustainable transitions and sociotechnical systems, these approaches have not been well‐operationalized. Given the importance of meeting future energy and environmental policy targets, there is need to develop predictive techniques and more robust methods to quantify and analyze sociotechnical systems undergoing rapid change and uncertainty due to sustainability pressures. Sustainability transitions depend on large‐scale diffusion of technological and behavioral innovations on physical and virtual networked systems. Transitions can therefore be viewed as a subclass of diffusion phenomenon and subject to a range of mathematical and computational methods. We review, categorize, and critically assess methodological and theoretical approaches that integrate different aspects of sustainability, innovation, and complexity. We argue that these approaches should be adapted to improve our understanding of the behavior and dynamics of a broad range of sociotechnical systems to meet sustainability objectives. We therefore also make the conceptual link between complexity and sustainability as complimentary fields of research to inform policy and decision making to achieve more sustainable outcomes. © 2014 Wiley Periodicals, Inc. Complexity 19: 8–22, 2014     

High-symmetry Hopfield-type neural networks

We study the set of fixed points of a Hopfield-type neural network with a connection matrix constructed from a high-symmetry set of memorized patterns using the Hebb rule. The memorized patterns depending on an external parameter are interpreted as distorted copies of a vector standard to be learned by the network. The dependence of the fixed-point set of the network on the distortion parameter is described analytically. The investigation results are interpreted in terms of neural networks and the Ising model. Translated from Teoreticheskaya i Matematicheskaya Fizika, Vol. 118, No. 1, pp. 133–158, January, 1999.     

Universal computation in fluid neural networks

Fluid neural networks can be used as a theoretical framework for a wide range of complex systems as social insects. In this article we show that collective logical gates can be built in such a way that complex computation can be possible by means of the interplay between local interactions and the collective creation of a global field. This is exemplified by a NOR gate. Some general implications for ant societies are outlined. © 1996 John Wiley & Sons, Inc.     

One- and two-cluster synchronized dynamics of non-diffusively coupled Tchebycheff map networks

We use the master stability formalism to discuss one- and two-cluster synchronization of coupled Tchebycheff map networks. For diffusively coupled map systems, the one-cluster synchronized dynamics is given by the behaviour of the individual maps, and the coupling only determines the stability of the coherent state. For the case of non-diffusive coupling and for two-cluster synchronization, the synchronized dynamics on networks is different from the behaviour of the single individual map. Depending on the coupling, we study numerically the characteristics of various forms of the resulting synchronized dynamics. The stability properties of the respective one-cluster synchronized states are discussed for arbitrary network structures. For the case of two-cluster synchronization on bipartite networks we also present analytical expressions for fixed points and zig-zag patterns, and explicitly determine the linear stability of these orbits for the special case of ring-networks.     

Post-training on RBF neural networks

Radial basis function neural networks are the most widely used networks due to their rapid training, generality, and simplicity. The nature of these networks necessitates some types of errors which can never be removed by traditional training algorithms. This paper is an attempt to introduce the natural error sources of neural networks such as bias error, iteration-restricted error, and Gibbs error. Moreover, a new method is introduced, called post-training, to reduce these errors as far as desired.     

Global stability analysis in hopfield neural networks

In this paper, one approach is employed to investigate the existence and uniqueness of the equilibrium and the global attractivity of Hopfield neural network models. Without assuming the boundedness, monotonicity, and differentiability of the activation functions, by using M-matrix theory, Liapunov functions are constructed and employed to establish sufficient conditions for global asymptotic stability.     

Advances in verification of ReLU neural networks

Journal of Global Optimization - We consider the problem of verifying linear properties of neural networks. Despite their success in many classification and prediction tasks, neural networks may...     

Spatial complexity in multi-layer cellular neural networks

This study investigates the complexity of the global set of output patterns for one-dimensional multi-layer cellular neural networks with input. Applying labeling to the output space produces a sofic shift space. Two invariants, namely spatial entropy and dynamical zeta function, can be exactly computed by studying the induced sofic shift space. This study gives sofic shift a realization through a realistic model. Furthermore, a new phenomenon, the broken of symmetry of entropy, is discovered in multi-layer cellular neural networks with input.     

Wave propagation in RTD-based cellular neural networks

This work investigates the existence of monotonic traveling wave and standing wave solutions of RTD-based cellular neural networks in the one-dimensional integer lattice . For nonzero wave speed c, applying the monotone iteration method with the aid of real roots of the corresponding characteristic function of the profile equation, we can partition the parameter space (γ,δ)-plane into four regions such that all the admissible monotonic traveling wave solutions connecting two neighboring equilibria can be classified completely. For the case of c=0, a discrete version of the monotone iteration scheme is established for proving the existence of monotonic standing wave solutions. Furthermore, if γ or δ is zero then the profile equation for the standing waves can be viewed as an one-dimensional iteration map and we then prove the multiplicity results of monotonic standing waves by using the techniques of dynamical systems for maps. Some numerical results of the monotone iteration scheme for traveling wave solutions are also presented.     

Fuzzy regression analysis using neural networks

In this paper, we propose simple but powerful methods for fuzzy regression analysis using neural networks. Since neural networks have high capability as an approximator of nonlinear mappings, the proposed methods can be applied to more complex systems than the existing LP based methods. First we propose learning algorithms of neural networks for determining a nonlinear interval model from the given input-output patterns. A nonlinear interval model whose outputs approximately include all the given patterns can be determined by two neural networks. Next we show two methods for deriving nonlinear fuzzy models from the interval model determined by the proposed algorithms. Nonlinear fuzzy models whose h-level sets approximately include all the given patterns can be derived. Last we show an application of the proposed methods to a real problem.     

Kohonen neural networks and genetic classification

We discuss the property of a.e. and in mean convergence of the Kohonen algorithm considered as a stochastic process. The various conditions ensuring a.e. convergence are described and the connection with the rate decay of the learning parameter is analyzed. The rate of convergence is discussed for different choices of learning parameters. We prove rigorously that the rate of decay of the learning parameter which is most used in the applications is a sufficient condition for a.e. convergence and we check it numerically. The aim of the paper is also to clarify the state of the art on the convergence property of the algorithm in view of the growing number of applications of the Kohonen neural networks. We apply our theorem and considerations to the case of genetic classification which is a rapidly developing field.     

Constructive approximate interpolation by neural networks

We present a type of single-hidden layer feedforward neural networks with sigmoidal nondecreasing activation function. We call them ai-nets. They can approximately interpolate, with arbitrary precision, any set of distinct data in one or several dimensions. They can uniformly approximate any continuous function of one variable and can be used for constructing uniform approximants of continuous functions of several variables. All these capabilities are based on a closed expression of the networks.     

Stability of Hopfield-type neural networks II

The global asympatotic stability and asymptotic stability for Hopfield-type neural networks with delays are investigated. Project supported by the National Natural Science Foundation of China.     

Stability of delayed cellular neural networks

Global asymptotic stability and exponential stability of delayed cellular neural networks is considered in this paper. Based on the Lyapunov stability theorem as well as a fact about the elemental inequality, some new sufficient conditions are given for global asymptotic stability and exponential stability of delayed cellular neural networks. The results are less conservative than those established in the earlier references. Three examples are given to illustrate the applicability of these conditions.     

Ehresmann connections and feedforward neural networks

In this paper, we describe how Ehresmann connections can be used to study certain properties of feedforward neural networks. Essentially, we calculate a Lie group approximation to the structure of the inverse image set above a certain point in the output space and this structure can then be locally transported to the inverse image above a neighbouring point in the output space by means of an Ehresmann connection. This enables us to find a continuous approximation to the underlying topological structure of the data from discrete data pairs (input/output pairs).     

Classification by ensembles of neural networks

We introduce a new procedure for training of artificial neural networks by using the approximation of an objective function by arithmetic mean of an ensemble of selected randomly generated neural networks, and apply this procedure to the classification (or pattern recognition) problem. This approach differs from the standard one based on the optimization theory. In particular, any neural network from the mentioned ensemble may not be an approximation of the objective function.     

Global analysis of planar neural networks

In this paper, the global qualitative analysis of cubic dynamical systems is established. These systems are used as learning models of planar neural networks.     

Hopfield neural networks in large-scale linear optimization problems

Hopfield neural networks and affine scaling interior point methods are combined in a hybrid approach for solving linear optimization problems. The Hopfield networks perform the early stages of the optimization procedures, providing enhanced feasible starting points for both primal and dual affine scaling interior point methods, thus facilitating the steps towards optimality. The hybrid approach is applied to a set of real world linear programming problems. The results show the potential of the integrated approach, indicating that the combination of neural networks and affine scaling interior point methods can be a good alternative to obtain solutions for large-scale optimization problems.