Random recurrent neural networks dynamics

This paper is a review dealing with the study of large size random recurrent neural networks. The connection weights are varying according to a probability law and it is possible to predict the network dynamics at a macroscopic scale using an averaging principle. After a first introductory section, the section 2 reviews the various models from the points of view of the single neuron dynamics and of the global network dynamics. A summary of notations is presented, which is quite helpful for the sequel. In section 3, mean-field dynamics is developed. The probability distribution characterizing global dynamics is computed. In section 4, some applications of mean-field theory to the prediction of chaotic regime for Analog Formal Random Recurrent Neural Networks (AFRRNN) are displayed. The case of AFRRNN with an homogeneous population of neurons is studied in section 4.1. Then, a two-population model is studied in section 4.2. The occurrence of a cyclo-stationary chaos is displayed using the results of [16]. In section 5, an insight of the application of mean-field theory to IF networks is given using the results of [9].     

Transient dynamics in a small ensemble of synaptically coupled Morris-Lecar neurons

We propose a new model of synaptic transmission on the basis of a FitzHugh–Nagumo system with nonlinear recovery. It is shown that the Morrice–Lecar neuron ensemble formed by such couplings shows various structurally stable regimes of transient dynamics in the form of sequential transitions between various metastable oscillatory states of the system.     

Relating neural dynamics to neural coding

We demonstrate that two key theoretical objects used widely in computational neuroscience, the phase-resetting curve (PRC) from dynamics and the spike triggered average (STA) from statistical analysis, are closely related when neurons fire in a nearly regular manner and the stimulus is sufficiently small. We prove that the STA due to injected noisy current is proportional to the derivative of the PRC. We compare these analytic results with numerical calculations for the Hodgkin-Huxley neuron and we apply the method to neurons in the olfactory bulb of mice. This observation allows us to relate the stimulus-response properties of a neuron to its dynamics, bridging the gap between dynamical and information theoretic approaches to understanding brain computations and facilitating the interpretation of changes in channels and other cellular properties as influencing the representation of stimuli.     

Random fields and quantum dynamics

We shall discuss quantum mechanical operators depending on the time or on the manifold in space. There is a similarity to the case of stochastic processes or random fields, where the innovation approach is one of the powerful tools to investigate their probabilistic structure. Having had some review of the innovation, similar attempt is made for some cases in quantum dynamics.     

Synaptic plasticity modulates autonomous transitions between waking and sleep states: insights from a Morris-Lecar model

The transitions between waking and sleep states are characterized by considerable changes in neuronal firing. During waking, neurons fire tonically at irregular intervals and a desynchronized activity is observed at the electroencephalogram. This activity becomes synchronized with slow wave sleep onset when neurons start to oscillate between periods of firing (up-states) and periods of silence (down-states). Recently, it has been proposed that the connections between neurons undergo potentiation during waking, whereas they weaken during slow wave sleep. Here, we propose a dynamical model to describe basic features of the autonomous transitions between such states. We consider a network of coupled neurons in which the strength of the interactions is modulated by synaptic long term potentiation and depression, according to the spike time-dependent plasticity rule (STDP). The model shows that the enhancement of synaptic strength between neurons occurring in waking increases the propensity of the network to synchronize and, conversely, desynchronization appears when the strength of the connections become weaker. Both transitions appear spontaneously, but the transition from sleep to waking required a slight modification of the STDP rule with the introduction of a mechanism which becomes active during sleep and changes the proportion between potentiation and depression in accordance with biological data. At the neuron level, transitions from desynchronization to synchronization and vice versa can be described as a bifurcation between two different states, whose dynamical regime is modulated by synaptic strengths, thus suggesting that transition from a state to an another can be determined by quantitative differences between potentiation and depression.     

Designing the dynamics of spiking neural networks

Precise timing of spikes and temporal locking are key elements of neural computation. Here we demonstrate how even strongly heterogeneous, deterministic neural networks with delayed interactions and complex topology can exhibit periodic patterns of spikes that are precisely timed. We develop an analytical method to find the set of all networks exhibiting a predefined pattern dynamics. Such patterns may be arbitrarily long and of complicated temporal structure. We point out that the same pattern can exist in very different networks and have different stability properties.     

Chaos and neural dynamics

Small neural networks (central pattern generators), which control rythmic behavior of animals, are considered. The mechanisms responsible for regularity, reliability, and plasticity of such behavior (swimming, running, walking, etc.) are analyzed. New data on the origin of regular cooperative behavior of neurons are obtained by using methods of nonlinear dynamics.Translated from Izvestiya Vysshikh Uchebnykh Zavedenii, Radiofizika, Vol. 39, No. 6, pp. 757–770, June, 1996.     

Random walk representations of the Heisenberg model

We develop random walk representations for the spin-S Heisenberg ferromagnet with nearest neighbor interactions. We show that the spin-S Heisenberg model is a diffusion with local times controlled by the spin-S Ising model. As a consequence, expectations for the Heisenberg model conditioned on zero diffusion are shown to be Ising expectations.     

Chaotic dynamics of high-order neural networks

The dynamics of an extremely diluted neural network with high-order synapses acting as corrections to the Hopfield model is investigated. The learning rules for the high-order connections contain mixing of memories, different from all the previous generalizations of the Hopfield model. The dynamics may display fixed points or periodic and chaotic orbits, depending on the weight of the high-order connections , the noise levelT, and the network load, defined as the ratio between the number of stored patterns and the mean connectivity per neuron, =P/C. As in the related fully connected case, there is an optimal value of the weight that improves the storage capacity of the system (the capacity diverges).     

Random dispersion approximation for the Hubbard model

We use the Random Dispersion Approximation (RDA) to study the Mott-Hubbard transition in the Hubbard model at half band filling. The RDA becomes exact for the Hubbard model in infinite dimensions. We implement the RDA on finite chains and employ the Lanczos exact diagonalization method in real space to calculate the ground-state energy, the average double occupancy, the charge gap, the momentum distribution, and the quasi-particle weight. We find a satisfactory agreement with perturbative results in the weak- and strong-coupling limits. A straightforward extrapolation of the RDA data for L ≤ 14 lattice results in a continuous Mott-Hubbard transition at U_c≈W. We discuss the significance of a possible signature of a coexistence region between insulating and metallic ground states in the RDA that would correspond to the scenario of a discontinuous Mott-Hubbard transition as found in numerical investigations of the Dynamical Mean-Field Theory for the Hubbard model.     

Random walk model of impact phenomena

A model of the permanent distortion of an elastic material due to high velocity projectile impact is described using a random walk model, with unusual temporal statistics, of the transport of dislocations. The biased motion of dislocations in a stress field and also in a random environment is considered. A temperature dependence for certain scaling exponents is derived. The experimentally observed scaling of the total integrated momentum as well as the scaling of the penetration and strength of the shock wave with time are obtained with this model.     

Unlearning dynamics in Hopfield neural network

The dynamics of a convergent iterative unlearning algorithm proposed earlier [7, 8] is examined. A self-consistent system of equations of the spectral dynamics of a synaptic matrix is obtained at the thermodynamic limit. The unlearning intensity (which varies during the iteration process) that optimizes the algorithm's rate of convergence on the projector matrix is found. The synaptic-matrix spectrum dynamics for optimal unlearning is determined.Institute of Physicotechnical Problems, Moscow. Translated from Izvestiya Vysshikh Uchebnykh Zavedenii, Radiofizika, Vol. 37, No. 9, pp. 1104–1115, September, 1994.     

Adaptation dynamics of the quasispecies model

We study the adaptation dynamics of an initially maladapted population evolving via the elementary processes of mutation and selection. The evolution occurs on rugged fitness landscapes which are defined on the multi-dimensional genotypic space and have many local peaks separated by low fitness valleys. We mainly focus on the Eigen’s model that describes the deterministic dynamics of an infinite number of self-replicating molecules. In the stationary state, for small mutation rates such a population forms a quasispecies which consists of the fittest genotype and its closely related mutants. The quasispecies dynamics on rugged fitness landscape follow a punctuated (or steplike) pattern in which a population jumps from a low fitness peak to a higher one, stays there for a considerable time before shifting the peak again and eventually reaches the global maximum of the fitness landscape. We calculate exactly several properties of this dynamical process within a simplified version of the quasispecies model.      

Glauber dynamics of the Little-Hopfield model

The Glauber dynamics of the Little-Hopfield model is studied analytically with the help of path-integrals recently introduced by Sommers. The results of Amit, Gutfreund and Sompolinsky concerning the statics of the Little-Hopfield model near saturation are dynamically verified. Their replica-symmetric solution corresponds exactly to the self-consistency equations of the long-time limits of some time-dependent functions emerging in our dynamical treatment. We also found their generalized Almeida-Thouless line where the solution becomes unstable. The time-dependent evolution of the macroscopic overlaps for a finite number of stored patterns is investigated and the problems arising for an infinite number of them are indicated.Work performed within the research program of the Sonderforschungsbereich 125, Aachen-Jülich-Köln     

Statistical dynamics of the Lorenz model

The dynamics of the Lorenz model in the turbulent regime (r>r _T is investigated by applying methods for treating many-body systems. Symmetry properties are used to derive relations between correlation functions. The basic ones are evaluated numerically and discussed for several values of the parameterr. A theory for the spectra of the two independent relaxation functions is presented using a dispersion relation representation in terms of relaxation kernels and characteristic frequencies. Their role in the dynamics of the system is discussed and it is shown that their numerical values increase in proportion to r. The approximation of the relaxation kernels that represent nonlinear coupling between the variables by a relaxation time expression and a simple mode coupling approximation, respectively, is shown to explain the two different fluctuation spectra. The coupling strength for the modes is determined by a Kubo relation imposing selfconsistency. Comparison with the experimental spectra is made for three values ofr.