From topology to dynamics in biochemical networks

Abstract formulations of the regulation of gene expression as random Boolean switching networks have been studied extensively over the past three decades. These models have been developed to make statistical predictions of the types of dynamics observed in biological networks based on network topology and interaction bias, p. For values of mean connectivity chosen to correspond to real biological networks, these models predict disordered dynamics. However, chaotic dynamics seems to be absent from the functioning of a normal cell. While these models use a fixed number of inputs for each element in the network, recent experimental evidence suggests that several biological networks have distributions in connectivity. We therefore study randomly constructed Boolean networks with distributions in the number of inputs, K, to each element. We study three distributions: delta function, Poisson, and power law (scale free). We analytically show that the critical value of the interaction bias parameter, p, above which steady state behavior is observed, is independent of the distribution in the limit of the number of elements N--> infinity. We also study these networks numerically. Using three different measures (types of attractors, fraction of elements that are active, and length of period), we show that finite, scale-free networks are more ordered than either the Poisson or delta function networks below the critical point. Thus the topology of scale-free biochemical networks, characterized by a wide distribution in the number of inputs per element, may provide a source of order in living cells. (c) 2001 American Institute of Physics.     

Random recurrent neural networks dynamics

This paper is a review dealing with the study of large size random recurrent neural networks. The connection weights are varying according to a probability law and it is possible to predict the network dynamics at a macroscopic scale using an averaging principle. After a first introductory section, the section 2 reviews the various models from the points of view of the single neuron dynamics and of the global network dynamics. A summary of notations is presented, which is quite helpful for the sequel. In section 3, mean-field dynamics is developed. The probability distribution characterizing global dynamics is computed. In section 4, some applications of mean-field theory to the prediction of chaotic regime for Analog Formal Random Recurrent Neural Networks (AFRRNN) are displayed. The case of AFRRNN with an homogeneous population of neurons is studied in section 4.1. Then, a two-population model is studied in section 4.2. The occurrence of a cyclo-stationary chaos is displayed using the results of [16]. In section 5, an insight of the application of mean-field theory to IF networks is given using the results of [9].     

Chaotic dynamics of high-order neural networks

The dynamics of an extremely diluted neural network with high-order synapses acting as corrections to the Hopfield model is investigated. The learning rules for the high-order connections contain mixing of memories, different from all the previous generalizations of the Hopfield model. The dynamics may display fixed points or periodic and chaotic orbits, depending on the weight of the high-order connections , the noise levelT, and the network load, defined as the ratio between the number of stored patterns and the mean connectivity per neuron, =P/C. As in the related fully connected case, there is an optimal value of the weight that improves the storage capacity of the system (the capacity diverges).     

Designing the dynamics of spiking neural networks

Precise timing of spikes and temporal locking are key elements of neural computation. Here we demonstrate how even strongly heterogeneous, deterministic neural networks with delayed interactions and complex topology can exhibit periodic patterns of spikes that are precisely timed. We develop an analytical method to find the set of all networks exhibiting a predefined pattern dynamics. Such patterns may be arbitrarily long and of complicated temporal structure. We point out that the same pattern can exist in very different networks and have different stability properties.     

Stable irregular dynamics in complex neural networks

Irregular dynamics in multidimensional systems is commonly associated with chaos. For infinitely large sparse networks of spiking neurons, mean field theory shows that a balanced state of highly irregular activity arises under various conditions. Here we analytically investigate the microscopic irregular dynamics in finite networks of arbitrary connectivity, keeping track of all individual spike times. For delayed, purely inhibitory interactions we demonstrate that any irregular dynamics that characterizes the balanced state is not chaotic but rather stable and convergent towards periodic orbits. These results highlight that chaotic and stable dynamics may be equally irregular.     

Complex brain networks: From topological communities to clustered dynamics

Recent research has revealed a rich and complicated network topology in the cortical connectivity of mammalian brains. A challenging task is to understand the implications of such network structures on the functional organisation of the brain activities. We investigate synchronisation dynamics on the corticocortical network of the cat by modelling each node of the network (cortical area) with a subnetwork of interacting excitable neurons. We find that this network of networks displays clustered synchronisation behaviour and the dynamical clusters closely coincide with the topological community structures observed in the anatomical network. The correlation between the firing rate of the areas and the areal intensity is additionally examined. Our results provide insights into the relationship between the global organisation and the functional specialisation of the brain cortex.      

Relating neural dynamics to neural coding

We demonstrate that two key theoretical objects used widely in computational neuroscience, the phase-resetting curve (PRC) from dynamics and the spike triggered average (STA) from statistical analysis, are closely related when neurons fire in a nearly regular manner and the stimulus is sufficiently small. We prove that the STA due to injected noisy current is proportional to the derivative of the PRC. We compare these analytic results with numerical calculations for the Hodgkin-Huxley neuron and we apply the method to neurons in the olfactory bulb of mice. This observation allows us to relate the stimulus-response properties of a neuron to its dynamics, bridging the gap between dynamical and information theoretic approaches to understanding brain computations and facilitating the interpretation of changes in channels and other cellular properties as influencing the representation of stimuli.     

On the parallel dynamics of theQ-state Potts andQ-Ising neural networks

Using a probabilistic approach, the parallel dynamics of theQ-state Potts andQ-Ising neural networks are studied at zero and at nonzero temperatures. Evolution equations are derived for the first time step and arbitraryQ. These formulas constitute recursion relations for the exact parallel dynamics of the extremely diluted asymmetric versions of these networks. An explicit analysis, including dynamical capacity-temperature diagrams and the temperature dependence of the overlap, is carried out forQ=3. Both types of models are compared.On leave of absence from the Laboratory of Theoretical Physics, Joint Institute for Nuclear Research, Dubna 141980, Russia.     

Symmetry-adapted graph neural networks for constructing molecular dynamics force fields

Molecular dynamics is a powerful simulation tool to explore material properties. Most realistic material systems are too large to be simulated using first-principles molecular dynamics. Classical molecular dynamics has a lower computational cost but requires accurate force fields to achieve chemical accuracy. In this work, we develop a symmetry-adapted graph neural network framework called the molecular dynamics graph neural network(MDGNN) to construct force fields automatically for molecular dynamics simulations for both molecules and crystals. This architecture consistently preserves translation, rotation, and permutation invariance in the simulations. We also propose a new feature engineering method that includes high-order terms of interatomic distances and demonstrate that the MDGNN accurately reproduces the results of both classical and first-principles molecular dynamics. In addition, we demonstrate that force fields constructed by the proposed model have good transferability.The MDGNN is thus an efficient and promising option for performing molecular dynamics simulations of large-scale systems with high accuracy.     

Itinerant memory dynamics and global bifurcations in chaotic neural networks

We have considered itinerant memory dynamics in a chaotic neural network composed of four chaotic neurons with synaptic connections determined by two orthogonal stored patterns as a simple example of a chaotic itinerant phenomenon in dynamical associative memory. We have analyzed a mechanism of generating the itinerant memory dynamics with respect to intersection of a pair of alpha branches of periodic points and collapse of a periodic in-phase attracting set. The intersection of invariant sets is numerically verified by a novel method proposed in this paper.     

Freezing transition in asymmetric random neural networks with deterministic dynamics

A network of binary elements (spins, neurons) which are completely connected by random couplings is investigated for a deterministic dynamics. For general values of the symmetry parameter extensive numerical simulations have been performed. With increasing symetry a sharp transition from a chaotic motion to a frozen state is found.     

Stochastic analyses of the dynamics of generalized Little-Hopfield-Hemmen type neural networks

Stochastic analyses are conducted of model neural networks of the generalized Little-Hopfield-Hemmen type, in which the synaptic connections with linearly embeddedp sets of patterns are free of symmetric ones, and a Glauber dynamics of a Markovian type is assumed. Two kinds of approaches are taken to study the stochastic dynamical behavior of the network system. First, by developing the method of the nonlinear master equation in the thermodynamic limitN, an exact self-consistent equation is derived for the time evolultion of the pattern overlaps which play the role of the order parameters of the system. The self-consistent equation is shown to describe almost completely the macroscopic dynamical behavior of the network system. Second, conducting the system-size expansion of the master equation for theN-body probability distribution of the Glauber dynamics makes it possible to analyze the fluctuations. In the course of the analysis, the self-consistent equation for the pattern overlaps is derived again. The main result of the rigorous fluctuation analysis is that as far as the fluctuations are concerned, the time course of the pattern overlap fluctuations behaves independently of the fluctuations in the remaining modes of the system's macrovariables, in accordance with the self-determining property of the macroscopic motion of the pattern overlaps for neural networks with linear synaptic couplings.     

Topology and dynamics of attractor neural networks: The role of loopiness

We derive an exact representation of the topological effect on the dynamics of sequence processing neural networks within signal-to-noise analysis. A new network structure parameter, loopiness coefficient, is introduced to quantitatively study the loop effect on network dynamics. A large loopiness coefficient means a high probability of finding loops in the networks. We develop recursive equations for the overlap parameters of neural networks in terms of their loopiness. It was found that a large loopiness increases the correlation among the network states at different times and eventually reduces the performance of neural networks. The theory is applied to several network topological structures, including fully-connected, densely-connected random, densely-connected regular and densely-connected small-world, where encouraging results are obtained.     

More relaxed condition for dynamics of discrete time delayed Hopfield neural networks

The dynamics of discrete time delayed Hopfield neural networks is investigated. By using a difference inequality combining with the linear matrix inequality, a sufficient condition ensuring global exponential stability of the unique equilibrium point of the networks is found. The result obtained holds not only for constant delay but also for time-varying delays.     

Transient dynamics of sparsely connected Hopfield neural networks with arbitrary degree distributions

Using probabilistic approach, the transient dynamics of sparsely connected Hopfield neural networks is studied for arbitrary degree distributions. A recursive scheme is developed to determine the time evolution of overlap parameters. As illustrative examples, the explicit calculations of dynamics for networks with binomial, power-law, and uniform degree distribution are performed. The results are good agreement with the extensive numerical simulations. It indicates that with the same average degree, there is a gradual improvement of network performance with increasing sharpness of its degree distribution, and the most efficient degree distribution for global storage of patterns is the delta function.     

From regular to growing small-world networks

We propose a growing model which interpolates between one-dimensional regular lattice and small-world networks. The model undergoes an interesting phase transition from large to small worlds. We investigate the structural properties by both theoretical predictions and numerical simulations. Our growing model is a complementarity for the important static Watts-Strogatz network model.