Coherent periodic activity in excitatory Erdös-Renyi neural networks: the role of network connectivity

In this article, we investigate the role of connectivity in promoting coherent activity in excitatory neural networks. In particular, we would like to understand if the onset of collective oscillations can be related to a minimal average connectivity and how this critical connectivity depends on the number of neurons in the networks. For these purposes, we consider an excitatory random network of leaky integrate-and-fire pulse coupled neurons. The neurons are connected as in a directed Erdo?s-Renyi graph with average connectivity scaling as a power law with the number of neurons in the network. The scaling is controlled by a parameter γ, which allows to pass from massively connected to sparse networks and therefore to modify the topology of the system. At a macroscopic level, we observe two distinct dynamical phases: an asynchronous state corresponding to a desynchronized dynamics of the neurons and a regime of partial synchronization (PS) associated with a coherent periodic activity of the network. At low connectivity, the system is in an asynchronous state, while PS emerges above a certain critical average connectivity (c). For sufficiently large networks, (c) saturates to a constant value suggesting that a minimal average connectivity is sufficient to observe coherent activity in systems of any size irrespectively of the kind of considered network: sparse or massively connected. However, this value depends on the nature of the synapses: reliable or unreliable. For unreliable synapses, the critical value required to observe the onset of macroscopic behaviors is noticeably smaller than for reliable synaptic transmission. Due to the disorder present in the system, for finite number of neurons we have inhomogeneities in the neuronal behaviors, inducing a weak form of chaos, which vanishes in the thermodynamic limit. In such a limit, the disordered systems exhibit regular (non chaotic) dynamics and their properties correspond to that of a homogeneous fully connected network for any γ-value. Apart for the peculiar exception of sparse networks, which remain intrinsically inhomogeneous at any system size.     

Emergent criticality through adaptive information processing in boolean networks

We study information processing in populations of boolean networks with evolving connectivity and systematically explore the interplay between the learning capability, robustness, the network topology, and the task complexity. We solve a long-standing open question and find computationally that, for large system sizes N, adaptive information processing drives the networks to a critical connectivity K(c)=2. For finite size networks, the connectivity approaches the critical value with a power law of the system size N. We show that network learning and generalization are optimized near criticality, given that the task complexity and the amount of information provided surpass threshold values. Both random and evolved networks exhibit maximal topological diversity near K(c). We hypothesize that this diversity supports efficient exploration and robustness of solutions. Also reflected in our observation is that the variance of the fitness values is maximal in critical network populations. Finally, we discuss implications of our results for determining the optimal topology of adaptive dynamical networks that solve computational tasks.     

Ensemble averageability in network spectra

The extreme eigenvalues of connectivity matrices govern the influence of the network structure on a number of network dynamical processes. A fundamental open question is whether the eigenvalues of large networks are well represented by ensemble averages. Here we investigate this question explicitly and validate the concept of ensemble averageability in random scale-free networks by showing that the ensemble distributions of extreme eigenvalues converge to peaked distributions as the system size increases. We discuss the significance of this result using synchronization and epidemic spreading as example processes.     

Spatially Extended Networks with Singular Multi-scale Connectivity Patterns

The cortex is a very large network characterized by a complex connectivity including at least two scales: a microscopic scale at which the interconnections are non-specific and very dense, while macroscopic connectivity patterns connecting different regions of the brain at larger scale are extremely sparse. This motivates to analyze the behavior of networks with multiscale coupling, in which a neuron is connected to its \(v(N)\) nearest-neighbors where \(v(N)=o(N)\) , and in which the probability of macroscopic connection between two neurons vanishes. These are called singular multi-scale connectivity patterns. We introduce a class of such networks and derive their continuum limit. We show convergence in law and propagation of chaos in the thermodynamic limit. The limit equation obtained is an intricate non-local McKean–Vlasov equation with delays which is universal with respect to the type of micro-circuits and macro-circuits involved.     

Epidemic outbreaks in complex heterogeneous networks

We present a detailed analytical and numerical study for the spreading of infections with acquired immunity in complex population networks. We show that the large connectivity fluctuations usually found in these networks strengthen considerably the incidence of epidemic outbreaks. Scale-free networks, which are characterized by diverging connectivity fluctuations in the limit of a very large number of nodes, exhibit the lack of an epidemic threshold and always show a finite fraction of infected individuals. This particular weakness, observed also in models without immunity, defines a new epidemiological framework characterized by a highly heterogeneous response of the system to the introduction of infected individuals with different connectivity. The understanding of epidemics in complex networks might deliver new insights in the spread of information and diseases in biological and technological networks that often appear to be characterized by complex heterogeneous architectures. Received 20 September 2001 and Received in final form 4 February 2002     

Large Deviations, Dynamics and Phase Transitions in Large Stochastic and Disordered Neural Networks

Neuronal networks are characterized by highly heterogeneous connectivity, and this disorder was recently related experimentally to qualitative properties of the network. The motivation of this paper is to mathematically analyze the role of these disordered connectivities on the large-scale properties of neuronal networks. To this end, we analyze here large-scale limit behaviors of neural networks including, for biological relevance, multiple populations, random connectivities and interaction delays. Due to the randomness of the connectivity, usual mean-field methods (e.g. coupling) cannot be applied, but, similarly to studies developed for spin glasses, we will show that the sequences of empirical measures satisfy a large deviation principle, and converge towards a self-consistent non-Markovian process. From a mathematical viewpoint, the proof differs from previous works in that we are working in infinite-dimensional spaces (interaction delays) and consider multiple cell types. The limit obtained formally characterizes the macroscopic behavior of the network. We propose a dynamical systems approach in order to address the qualitative nature of the solutions of these very complex equations, and apply this methodology to three instances in order to show how non-centered coefficients, interaction delays and multiple populations networks are affected by disorder levels. We identify a number of phase transitions in such systems upon changes in delays, connectivity patterns and dispersion, and particularly focus on the emergence of non-equilibrium states involving synchronized oscillations.     

非线性网络的动力学复杂性研究

综述了非线性网络的动力学复杂性研究在网络理论、实证和应用方面所取得的主要进展和重要成果;深刻揭示了复杂网络的若干复杂性特征与基本定量规律;提出和建立了网络科学的统一混合理论体系(三部曲)和网络金字塔,并引入一类广义Farey组织的网络家族,阐明网络的复杂性-简单性与多样性-普适性之间转变关系;揭示了网络的拓扑结构特征与网络的动态特性之间关系;建立具有长程连接的规则网络的部分同步理论并应用于随机耦合的时空非线性系统的同步;提出复杂网络的动力学同步与控制多种方法;提出若干提高同步能力的模型、方法和途径,如同步最优和同步优先模型、同步与网络特征量关系、权重作用、叶子节点影响等;提出复杂混沌网络的多目标控制及具有小世界和无标度拓扑的束流输运网络的束晕-混沌控制方法;提出集群系统的自适应同步模型及蜂拥控制方法;探讨网络上拥塞与路由控制、资源博弈及不同类型网络上传播的若干规律;揭示含权经济科学家合作网及其演化特点;实证研究并揭示了多层次的高科技企业网和若干社会网络的特点;提出一种复杂网络的非平衡统计方法,把宏观网络推进到微观量子网络。     

Epidemic spreading in scale-free networks

The Internet has a very complex connectivity recently modeled by the class of scale-free networks. This feature, which appears to be very efficient for a communications network, favors at the same time the spreading of computer viruses. We analyze real data from computer virus infections and find the average lifetime and persistence of viral strains on the Internet. We define a dynamical model for the spreading of infections on scale-free networks, finding the absence of an epidemic threshold and its associated critical behavior. This new epidemiological framework rationalizes data of computer viruses and could help in the understanding of other spreading phenomena on communication and social networks.     

Dynamical decoherence of a qubit coupled to a quantum dot or the SYK black hole

We study the dynamical decoherence of a qubit weakly coupled to a two-body random interaction model (TBRIM) describing a quantum dot of interacting fermions or the Sachdev–Ye–Kitaev (SYK) black hole model. We determine the rates of qubit relaxation and dephasing for regimes of dynamical thermalization of the quantum dot or of quantum chaos in the SYK model. These rates are found to correspond to the Fermi golden rule and quantum Zeno regimes depending on the qubit–fermion coupling strength. An unusual regime is found where these rates are practically independent of TBRIM parameters. We push forward an analogy between TBRIM and quantum small-world networks with an explosive spreading over exponentially large number of states in a finite time being similar to six degrees of separation in small-world social networks. We find that the SYK model has approximately two–three degrees of separation.     

Random Incidence Matrices: Moments of the Spectral Density

We study numerically and analytically the spectrum of incidence matrices of random labeled graphs on N vertices: any pair of vertices is connected by an edge with probability p. We give two algorithms to compute the moments of the eigenvalue distribution as explicit polynomials in N and p. For large N and fixed p the spectrum contains a large eigenvalue at Np and a semicircle of small eigenvalues. For large N and fixed average connectivity pN (dilute or sparse random matrices limit) we show that the spectrum always contains a discrete component. An anomaly in the spectrum near eigenvalue 0 for connectivity close to e is observed. We develop recursion relations to compute the moments as explicit polynomials in pN. Their growth is slow enough so that they determine the spectrum. The extension of our methods to the Laplacian matrix is given in Appendix.     

Grand Canonical Ensembles of Sparse Networks and Bayesian Inference

Maximum entropy network ensembles have been very successful in modelling sparse network topologies and in solving challenging inference problems. However the sparse maximum entropy network models proposed so far have fixed number of nodes and are typically not exchangeable. Here we consider hierarchical models for exchangeable networks in the sparse limit, i.e., with the total number of links scaling linearly with the total number of nodes. The approach is grand canonical, i.e., the number of nodes of the network is not fixed a priori: it is finite but can be arbitrarily large. In this way the grand canonical network ensembles circumvent the difficulties in treating infinite sparse exchangeable networks which according to the Aldous-Hoover theorem must vanish. The approach can treat networks with given degree distribution or networks with given distribution of latent variables. When only a subgraph induced by a subset of nodes is known, this model allows a Bayesian estimation of the network size and the degree sequence (or the sequence of latent variables) of the entire network which can be used for network reconstruction.     

Proof of a Conjecture on the Infinite Dimension Limit of a Unifying Model for Random Matrix Theory

Journal of Statistical Physics - We study the large N limit of a sparse random block matrix ensemble. It depends on two parameters: the average connectivity Z and the size of the blocks d, which is...     

Damage Spreading in the Ising Model with a Special Metropolis Dynamics Approach

The time evolution of the Hamming distance (damage spreading) for the S=1/2 and S=1 Ising models on the square lattice is performed with a special metropolis dynamics algorithm. Two distinct regimes are observed according to the temperature range for both models: a low-temperature one where the distance in the long-time limit is finite and seems not to depend on the initial distance and the system size; a high-temperature one where the distance vanishes in the long-time limit. Using the finite size scaling method, the dynamical phase transition (damage spreading transition) temperature is obtained as T_c≌1.675±0.025 for the S=1 Ising model.     

From topology to dynamics in biochemical networks

Abstract formulations of the regulation of gene expression as random Boolean switching networks have been studied extensively over the past three decades. These models have been developed to make statistical predictions of the types of dynamics observed in biological networks based on network topology and interaction bias, p. For values of mean connectivity chosen to correspond to real biological networks, these models predict disordered dynamics. However, chaotic dynamics seems to be absent from the functioning of a normal cell. While these models use a fixed number of inputs for each element in the network, recent experimental evidence suggests that several biological networks have distributions in connectivity. We therefore study randomly constructed Boolean networks with distributions in the number of inputs, K, to each element. We study three distributions: delta function, Poisson, and power law (scale free). We analytically show that the critical value of the interaction bias parameter, p, above which steady state behavior is observed, is independent of the distribution in the limit of the number of elements N--> infinity. We also study these networks numerically. Using three different measures (types of attractors, fraction of elements that are active, and length of period), we show that finite, scale-free networks are more ordered than either the Poisson or delta function networks below the critical point. Thus the topology of scale-free biochemical networks, characterized by a wide distribution in the number of inputs per element, may provide a source of order in living cells. (c) 2001 American Institute of Physics.     

Stable irregular dynamics in complex neural networks

Irregular dynamics in multidimensional systems is commonly associated with chaos. For infinitely large sparse networks of spiking neurons, mean field theory shows that a balanced state of highly irregular activity arises under various conditions. Here we analytically investigate the microscopic irregular dynamics in finite networks of arbitrary connectivity, keeping track of all individual spike times. For delayed, purely inhibitory interactions we demonstrate that any irregular dynamics that characterizes the balanced state is not chaotic but rather stable and convergent towards periodic orbits. These results highlight that chaotic and stable dynamics may be equally irregular.     

Synchronization structure of evolving epileptic networks using cross-entropy

In this paper we present connectivity patterns of evolving large scale epileptic networks. We employed a cross-entropy measure in the frequency domain on EEG signals to infer the networks, before and during episodes of epileptic seizures. This measure allowed us to make a richer portrait about the node interactions on the graph and to identify emergent structures associated with the synchronization of brain activity. Our results points to a more complex scenario of network organization than the synchronized/unsynchronized dichotomy, with two main results: first, showing regions with unsynchronized (or independent) behavior, even during absence seizures, contradicting the concept of hypersynchrony. Furthermore, we explore the cross-entropy fluctuations along the seizure: a group of nodes became more similar over time while another group became more different, showing a complementary behaviour and different local brain activities. These results bring new questions about the spreading and the sustenance of the epileptic seizures and others synchronization phenomena in living systems.     

Excitable elements controlled by noise and network structure

We study collective dynamics of complex networks of stochastic excitable elements, active rotators. In the thermodynamic limit of infinite number of elements, we apply a mean-field theory for the network and then use a Gaussian approximation to obtain a closed set of deterministic differential equations. These equations govern the order parameters of the network. We find that a uniform decrease in the number of connections per element in a homogeneous network merely shifts the bifurcation thresholds without producing qualitative changes in the network dynamics. In contrast, heterogeneity in the number of connections leads to bifurcations in the excitable regime. In particular we show that a critical value of noise intensity for the saddle-node bifurcation decreases with growing connectivity variance. The corresponding critical values for the onset of global oscillations (Hopf bifurcation) show a non-monotone dependency on the structural heterogeneity, displaying a minimum at moderate connectivity variances.     

Scaling of directed dynamical small-world networks with random responses

A dynamical model of small-world networks, with directed links which describe various correlations in social and natural phenomena, is presented. Random responses of sites to the input message are introduced to simulate real systems. The interplay of these ingredients results in the collective dynamical evolution of a spinlike variable S(t) of the whole network. The global average spreading length (s) and average spreading time (s) are found to scale as p(-alpha)ln(N with different exponents. Meanwhile, S(t) behaves in a duple scaling form for N>N(*): S approximately f(p(-beta)q(gamma)t), where p and q are rewiring and external parameters, alpha, beta, and gamma are scaling exponents, and f(t) is a universal function. Possible applications of the model are discussed.     

On the Role of the Excitation/Inhibition Balance of Homeostatic Artificial Neural Networks

Homeostatic models of artificial neural networks have been developed to explain the self-organization of a stable dynamical connectivity between the neurons of the net. These models are typically two-population models, with excitatory and inhibitory cells. In these models, connectivity is a means to regulate cell activity, and in consequence, intracellular calcium levels towards a desired target level. The excitation/inhibition (E/I) balance is usually set to 80:20, a value characteristic for cortical cell distributions. We study the behavior of these homeostatic models outside of the physiological range of the E/I balance, and we find a pronounced bifurcation at about the physiological value of this balance. Lower inhibition values lead to sparsely connected networks. At a certain threshold value, the neurons develop a reasonably connected network that can fulfill the homeostasis criteria in a stable way. Beyond the threshold, the behavior of the artificial neural network changes drastically, with failing homeostasis and in consequence with an exploding number of connections. While the exact value of the balance at the bifurcation point is subject to the parameters of the model, the existence of this bifurcation might explain the stability of a certain E/I balance across a wide range of biological neural networks. Assuming that this class of models describes the self-organization of biological network connectivity reasonably realistically, the omnipresent physiological balance might represent a case of self-organized criticality in order to obtain a good connectivity while allowing for a stable intracellular calcium homeostasis.