From topology to dynamics in biochemical networks

Abstract formulations of the regulation of gene expression as random Boolean switching networks have been studied extensively over the past three decades. These models have been developed to make statistical predictions of the types of dynamics observed in biological networks based on network topology and interaction bias, p. For values of mean connectivity chosen to correspond to real biological networks, these models predict disordered dynamics. However, chaotic dynamics seems to be absent from the functioning of a normal cell. While these models use a fixed number of inputs for each element in the network, recent experimental evidence suggests that several biological networks have distributions in connectivity. We therefore study randomly constructed Boolean networks with distributions in the number of inputs, K, to each element. We study three distributions: delta function, Poisson, and power law (scale free). We analytically show that the critical value of the interaction bias parameter, p, above which steady state behavior is observed, is independent of the distribution in the limit of the number of elements N--> infinity. We also study these networks numerically. Using three different measures (types of attractors, fraction of elements that are active, and length of period), we show that finite, scale-free networks are more ordered than either the Poisson or delta function networks below the critical point. Thus the topology of scale-free biochemical networks, characterized by a wide distribution in the number of inputs per element, may provide a source of order in living cells. (c) 2001 American Institute of Physics.     

Random recurrent neural networks dynamics

This paper is a review dealing with the study of large size random recurrent neural networks. The connection weights are varying according to a probability law and it is possible to predict the network dynamics at a macroscopic scale using an averaging principle. After a first introductory section, the section 2 reviews the various models from the points of view of the single neuron dynamics and of the global network dynamics. A summary of notations is presented, which is quite helpful for the sequel. In section 3, mean-field dynamics is developed. The probability distribution characterizing global dynamics is computed. In section 4, some applications of mean-field theory to the prediction of chaotic regime for Analog Formal Random Recurrent Neural Networks (AFRRNN) are displayed. The case of AFRRNN with an homogeneous population of neurons is studied in section 4.1. Then, a two-population model is studied in section 4.2. The occurrence of a cyclo-stationary chaos is displayed using the results of [16]. In section 5, an insight of the application of mean-field theory to IF networks is given using the results of [9].     

Chaotic dynamics of high-order neural networks

The dynamics of an extremely diluted neural network with high-order synapses acting as corrections to the Hopfield model is investigated. The learning rules for the high-order connections contain mixing of memories, different from all the previous generalizations of the Hopfield model. The dynamics may display fixed points or periodic and chaotic orbits, depending on the weight of the high-order connections , the noise levelT, and the network load, defined as the ratio between the number of stored patterns and the mean connectivity per neuron, =P/C. As in the related fully connected case, there is an optimal value of the weight that improves the storage capacity of the system (the capacity diverges).     

神经元传输函数对人工神经网络动力学特性的影响

以神经元局域场分布为基础，重新研究了连续神经元传输函数对具有联想记忆的人工神经网络功能的影响.与以往的认识不同的是，研究发现连续传输函数与硬极限传输函数相比并不存在明显的优越性，相反，连续传输函数对网络的某些功能，如最大存储率具有负面影响.研究表明神经网络的特性主要决定于网络的动力学结构(具体体现为网络吸引子对应的神经元局域场分布)，网络的动力学结构可以通过选择合适的设计规则进行有效控制，不同的传输函数虽然也能影响到网络的动力学结构，但是它所带来的影响是被动的，可控性很差. 关键词： 联想记忆 神经网络 吸引子 局域场分布     

Designing the dynamics of spiking neural networks

Precise timing of spikes and temporal locking are key elements of neural computation. Here we demonstrate how even strongly heterogeneous, deterministic neural networks with delayed interactions and complex topology can exhibit periodic patterns of spikes that are precisely timed. We develop an analytical method to find the set of all networks exhibiting a predefined pattern dynamics. Such patterns may be arbitrarily long and of complicated temporal structure. We point out that the same pattern can exist in very different networks and have different stability properties.     

Stable irregular dynamics in complex neural networks

Irregular dynamics in multidimensional systems is commonly associated with chaos. For infinitely large sparse networks of spiking neurons, mean field theory shows that a balanced state of highly irregular activity arises under various conditions. Here we analytically investigate the microscopic irregular dynamics in finite networks of arbitrary connectivity, keeping track of all individual spike times. For delayed, purely inhibitory interactions we demonstrate that any irregular dynamics that characterizes the balanced state is not chaotic but rather stable and convergent towards periodic orbits. These results highlight that chaotic and stable dynamics may be equally irregular.     

Complex brain networks: From topological communities to clustered dynamics

Recent research has revealed a rich and complicated network topology in the cortical connectivity of mammalian brains. A challenging task is to understand the implications of such network structures on the functional organisation of the brain activities. We investigate synchronisation dynamics on the corticocortical network of the cat by modelling each node of the network (cortical area) with a subnetwork of interacting excitable neurons. We find that this network of networks displays clustered synchronisation behaviour and the dynamical clusters closely coincide with the topological community structures observed in the anatomical network. The correlation between the firing rate of the areas and the areal intensity is additionally examined. Our results provide insights into the relationship between the global organisation and the functional specialisation of the brain cortex.      

Relating neural dynamics to neural coding

We demonstrate that two key theoretical objects used widely in computational neuroscience, the phase-resetting curve (PRC) from dynamics and the spike triggered average (STA) from statistical analysis, are closely related when neurons fire in a nearly regular manner and the stimulus is sufficiently small. We prove that the STA due to injected noisy current is proportional to the derivative of the PRC. We compare these analytic results with numerical calculations for the Hodgkin-Huxley neuron and we apply the method to neurons in the olfactory bulb of mice. This observation allows us to relate the stimulus-response properties of a neuron to its dynamics, bridging the gap between dynamical and information theoretic approaches to understanding brain computations and facilitating the interpretation of changes in channels and other cellular properties as influencing the representation of stimuli.     

Physics-informed recurrent neural networks for linear and nonlinear flame dynamics

This paper demonstrates the ability of recurrent neural networks (RNNs) to predict the linear and the nonlinear response of a premixed laminar flame to incoming velocity perturbations. We develop data-driven models, which require the velocity and heat release rate fluctuations as input data. Both time series are obtained from Direct Numerical Simulations (DNS) of a laminar flame. The length of the signals, and, hence, the cost of the simulation, is comparable to those used in the linear framework of System Identification. A more robust type of RNNs, namely long short term memory (LSTM), is employed to reduce the dependency on large datasets. The LSTM framework is modeled as a time series regression problem and four models are trained with decreasing data set lengths. All purely data-driven models accurately predict the unsteady time series of the heat release rate and, hence, the Flame Transfer Functions (FTFs). We further improve the model accuracy by incorporating a physical constraint, namely the low-frequency limit for perfectly-premixed flames, into the LSTM model. This step reduces the required data length compared to the purely data-driven approach. The proposed model, called PI-LSTM, is able to reproduce the linear and the nonlinear FTFs for amplitudes up to 50% of the laminar flame based on one numerical simulation, where the length of the time series is 100 ms.     

Freezing transition in asymmetric random neural networks with deterministic dynamics

A network of binary elements (spins, neurons) which are completely connected by random couplings is investigated for a deterministic dynamics. For general values of the symmetry parameter extensive numerical simulations have been performed. With increasing symetry a sharp transition from a chaotic motion to a frozen state is found.     

Itinerant memory dynamics and global bifurcations in chaotic neural networks

We have considered itinerant memory dynamics in a chaotic neural network composed of four chaotic neurons with synaptic connections determined by two orthogonal stored patterns as a simple example of a chaotic itinerant phenomenon in dynamical associative memory. We have analyzed a mechanism of generating the itinerant memory dynamics with respect to intersection of a pair of alpha branches of periodic points and collapse of a periodic in-phase attracting set. The intersection of invariant sets is numerically verified by a novel method proposed in this paper.     

Symmetry-adapted graph neural networks for constructing molecular dynamics force fields

Molecular dynamics is a powerful simulation tool to explore material properties. Most realistic material systems are too large to be simulated using first-principles molecular dynamics. Classical molecular dynamics has a lower computational cost but requires accurate force fields to achieve chemical accuracy. In this work, we develop a symmetry-adapted graph neural network framework called the molecular dynamics graph neural network(MDGNN) to construct force fields automatically for molecular dynamics simulations for both molecules and crystals. This architecture consistently preserves translation, rotation, and permutation invariance in the simulations. We also propose a new feature engineering method that includes high-order terms of interatomic distances and demonstrate that the MDGNN accurately reproduces the results of both classical and first-principles molecular dynamics. In addition, we demonstrate that force fields constructed by the proposed model have good transferability.The MDGNN is thus an efficient and promising option for performing molecular dynamics simulations of large-scale systems with high accuracy.     

On the parallel dynamics of theQ-state Potts andQ-Ising neural networks

Using a probabilistic approach, the parallel dynamics of theQ-state Potts andQ-Ising neural networks are studied at zero and at nonzero temperatures. Evolution equations are derived for the first time step and arbitraryQ. These formulas constitute recursion relations for the exact parallel dynamics of the extremely diluted asymmetric versions of these networks. An explicit analysis, including dynamical capacity-temperature diagrams and the temperature dependence of the overlap, is carried out forQ=3. Both types of models are compared.On leave of absence from the Laboratory of Theoretical Physics, Joint Institute for Nuclear Research, Dubna 141980, Russia.     

The characteristics of nonlinear chaotic dynamics in quantum cellular neural networks

With the polarization of quantum-dot cell and quantum phase serving as state variables, this paper does both theoretical analysis and simulation for the complex nonlinear dynamical behaviour of a three-cell-coupled Quantum Cellular Neural Network （QCNN）, including equilibrium points, bifurcation and chaotic behaviour. Different phenomena, such as quasi-periodic, chaotic and hyper-chaotic states as well as bifurcations are revealed. The system＇s bifurcation and chaotic behaviour under the influence of the different coupling parameters are analysed. And it finds that the unbalanced cells coupled QCNN is easy to cause chaotic oscillation and the system response enters into chaotic state from quasi-periodic state by quasi-period bifurcation; however, the balanced cells coupled QCNN also can be chaotic when coupling parameters is in some region. Additionally, both the unbalanced and balanced cells coupled QCNNs can possess hyper-chaotic behaviour. It provides valuable information about QCNNs for future application in high-parallel signal processing and novel ultra-small chaotic generators.     

Stochastic analyses of the dynamics of generalized Little-Hopfield-Hemmen type neural networks

Stochastic analyses are conducted of model neural networks of the generalized Little-Hopfield-Hemmen type, in which the synaptic connections with linearly embeddedp sets of patterns are free of symmetric ones, and a Glauber dynamics of a Markovian type is assumed. Two kinds of approaches are taken to study the stochastic dynamical behavior of the network system. First, by developing the method of the nonlinear master equation in the thermodynamic limitN, an exact self-consistent equation is derived for the time evolultion of the pattern overlaps which play the role of the order parameters of the system. The self-consistent equation is shown to describe almost completely the macroscopic dynamical behavior of the network system. Second, conducting the system-size expansion of the master equation for theN-body probability distribution of the Glauber dynamics makes it possible to analyze the fluctuations. In the course of the analysis, the self-consistent equation for the pattern overlaps is derived again. The main result of the rigorous fluctuation analysis is that as far as the fluctuations are concerned, the time course of the pattern overlap fluctuations behaves independently of the fluctuations in the remaining modes of the system's macrovariables, in accordance with the self-determining property of the macroscopic motion of the pattern overlaps for neural networks with linear synaptic couplings.     

More relaxed condition for dynamics of discrete time delayed Hopfield neural networks

The dynamics of discrete time delayed Hopfield neural networks is investigated. By using a difference inequality combining with the linear matrix inequality, a sufficient condition ensuring global exponential stability of the unique equilibrium point of the networks is found. The result obtained holds not only for constant delay but also for time-varying delays.     

具有时滞的一阶细胞神经网络动态行为研究

研究了具有时滞的一阶细胞神经网络的复杂动力学行为-证明了Hopf分叉的存在性，指出了 若选择适当的参数，则该网络中可以产生混沌-计算机仿真实验表明所得结论的正确性-由于 该网络的构成较简单，因此实际中可以作为混沌发生器加以应用- 关键词： 细胞神经网络 时滞 Hopf分叉 混沌