Random recurrent neural networks dynamics

This paper is a review dealing with the study of large size random recurrent neural networks. The connection weights are varying according to a probability law and it is possible to predict the network dynamics at a macroscopic scale using an averaging principle. After a first introductory section, the section 2 reviews the various models from the points of view of the single neuron dynamics and of the global network dynamics. A summary of notations is presented, which is quite helpful for the sequel. In section 3, mean-field dynamics is developed. The probability distribution characterizing global dynamics is computed. In section 4, some applications of mean-field theory to the prediction of chaotic regime for Analog Formal Random Recurrent Neural Networks (AFRRNN) are displayed. The case of AFRRNN with an homogeneous population of neurons is studied in section 4.1. Then, a two-population model is studied in section 4.2. The occurrence of a cyclo-stationary chaos is displayed using the results of [16]. In section 5, an insight of the application of mean-field theory to IF networks is given using the results of [9].     

Inaccessibility in online learning of recurrent neural networks

We apply nonlinear dynamical system techniques to recurrent neural networks. In particular, we numerically analyze the dynamical system characteristics of the online learning process. By introducing the notion of inaccessibility, we show that the learning process is well characterized by strong nonhyperbolicity and inaccessibility, which is a greater uncertainty than chaotic unpredictability. These results are clearly contrasted with a gradient descent dynamics, or ordinary chaos.     

A recurrent perceptron learning algorithm for cellular neural networks

A supervised learning algorithm for obtaining the template coefficients in completely stable Cellular Neural Networks (CNNs) is analysed in the paper. The considered algorithm resembles the well-known perceptron learning algorithm and hence called as Recurrent Perceptron Learning Algorithm (RPLA) when applied to a dynamical network. The RPLA learns pointwise defined algebraic mappings from initial-state and input spaces into steady-state output space; despite learning whole trajectories through desired equilibrium points. The RPLA has been used for training CNNs to perform some image processing tasks and found to be successful in binary image processing. The edge detection templates found by RPLA have performances comparable to those of Canny's edge detector for binary images.     

不确定混沌系统的回归神经网络同步

将高阶连接的神经元融合到分布式回归神经网络，研究了出现非模型动态性时不确定混沌系统的辨识和同步问题。采用李雅谱诺夫稳定理论对高阶神经网络回归模型的权值进行学习更新，同时，获取整个系统稳定特性的分析结果，而且通过李雅谱诺夫方法设计出消除不确定混沌系统的同步误差的自适应控制律。最后将所提出的方法应用到不确定Rossler混沌系统的建模与同步     

Physics-informed recurrent neural networks for linear and nonlinear flame dynamics

This paper demonstrates the ability of recurrent neural networks (RNNs) to predict the linear and the nonlinear response of a premixed laminar flame to incoming velocity perturbations. We develop data-driven models, which require the velocity and heat release rate fluctuations as input data. Both time series are obtained from Direct Numerical Simulations (DNS) of a laminar flame. The length of the signals, and, hence, the cost of the simulation, is comparable to those used in the linear framework of System Identification. A more robust type of RNNs, namely long short term memory (LSTM), is employed to reduce the dependency on large datasets. The LSTM framework is modeled as a time series regression problem and four models are trained with decreasing data set lengths. All purely data-driven models accurately predict the unsteady time series of the heat release rate and, hence, the Flame Transfer Functions (FTFs). We further improve the model accuracy by incorporating a physical constraint, namely the low-frequency limit for perfectly-premixed flames, into the LSTM model. This step reduces the required data length compared to the purely data-driven approach. The proposed model, called PI-LSTM, is able to reproduce the linear and the nonlinear FTFs for amplitudes up to 50% of the laminar flame based on one numerical simulation, where the length of the time series is 100 ms.     

Exponentially convergent state estimation for delayed switched recurrent neural networks

This paper deals with the delay-dependent exponentially convergent state estimation problem for delayed switched neural networks. A set of delay-dependent criteria is derived under which the resulting estimation error system is exponentially stable. It is shown that the gain matrix of the proposed state estimator is characterised in terms of the solution to a set of linear matrix inequalities (LMIs), which can be checked readily by using some standard numerical packages. An illustrative example is given to demonstrate the effectiveness of the proposed state estimator.     

Feed-forward chains of recurrent attractor neural networks with finite dilution near saturation

A stationary state replica analysis for a dual neural network model that interpolates between a fully recurrent symmetric attractor network and a strictly feed-forward layered network, studied by Coolen and Viana, is extended in this work to account for finite dilution of the recurrent Hebbian interactions between binary Ising units within each layer. Gradual dilution is found to suppress part of the phase transitions that arise from the competition between recurrent and feed-forward operation modes of the network. Despite that, a long chain of layers still exhibits a relatively good performance under finite dilution for a balanced ratio between inter-layer and intra-layer interactions.     

Multistability of delayed complex-valued recurrent neural networks with discontinuous real-imaginarytype activation functions

In this paper, the multistability issue is discussed for delayed complex-valued recurrent neural networks with discontinuous real-imaginary-type activation functions. Based on a fixed theorem and stability definition, sufficient criteria are established for the existence and stability of multiple equilibria of complex-valued recurrent neural networks. The number of stable equilibria is larger than that of real-valued recurrent neural networks, which can be used to achieve high-capacity associative memories. One numerical example is provided to show the effectiveness and superiority of the presented results.     

Stochastic exponential stability of the delayed reaction-diffusion recurrent neural networks with Markovian jumping parameters

Some criteria for the global stochastic exponential stability of the delayed reaction-diffusion recurrent neural networks with Markovian jumping parameters are presented. The jumping parameters considered here are generated from a continuous-time discrete-state homogeneous Markov process, which are governed by a Markov process with discrete and finite state space. By employing a new Lyapunov-Krasovskii functional, a linear matrix inequality (LMI) approach is developed to establish some easy-to-test criteria of global exponential stability in the mean square for the stochastic neural networks. The criteria are computationally efficient, since they are in the forms of some linear matrix inequalities.     

Stability theory by Lyapunov’s first method and recurrent neural networks

In the present paper, we give some relationship between Lyapunov’s exponents and the recurrent neural network model described by the system of delay-differential equations. We investigate the dynamic properties of the specific class of nonlinear delay-differential equations by studying the asymptotic behavior of their solutions by means of Lyapunov’s exponents. The text was submitted by the author in English.     

Topological analysis of chaos in neural spike train bursts

We show how a topological model which describes the stretching and squeezing mechanisms responsible for creating chaotic behavior can be extracted from the neural spike train data. The mechanism we have identified is the same one ("gateau roule," or jelly-roll) which has previously been identified in the Duffing oscillator [Gilmore and McCallum, Phys. Rev. E 51, 935 (1995)] and in a YAG laser [Boulant et al., Phys. Rev. E 55, 5082 (1997)]. (c) 1999 American Institute of Physics.     

Some new results for recurrent neural networks with varying-time coefficients and delays

In this Letter, we consider the recurrent neural networks with varying-time coefficients and delays. By constructing new Lyapunov functional, introducing ingeniously many real parameters and applying the technique of Young inequality, we establish a series of criteria on the boundedness, global exponential stability and the existence of periodic solutions. In these criteria, we do not require that the response functions are differentiable, bounded and monotone nondecreasing. Some previous works are improved and extended.     

Novel delay-distribution-dependent stability analysis for continuous-time recurrent neural networks with stochastic delay

<正>In this paper,the problem of delay-distribution-dependent stability is investigated for continuous-time recurrent neural networks(CRNNs) with stochastic delay.Different from the common assumptions on time delays,it is assumed that the probability distribution of the delay taking values in some intervals is known a priori.By making full use of the information concerning the probability distribution of the delay and by using a tighter bounding technique(the reciprocally convex combination method),less conservative asymptotic mean-square stable sufficient conditions are derived in terms of linear matrix inequalities(LMIs).Two numerical examples show that our results are better than the existing ones.     

Probing the attractors in neural networks

I propose tools to probe the nature of the retrieval attractors in neural networks. These include the activity distribution, the evolutions of the state damage, activity damage and temporal correlation damage. They enable us to demonstrate that the retrieval attractors in dillute asymmetric neutral networks are not clouds of attractors, but consists of a single chaotic attractor for each stored pattern. Furthermore, they facilitate the devise of effective freezing procedures, which significantly improve the quality of retrieval in dilute asymmetric neural networks.     

Synchronization and propagation of bursts in networks of coupled map neurons

The present paper studies regular and complex spatiotemporal behaviors in networks of coupled map-based bursting oscillators. In-phase and antiphase synchronization of bursts are studied, explaining their underlying mechanisms in order to determine how network parameters separate them. Conditions for emergent bursting in the coupled system are derived from our analysis. In the region of emergence, patterns of chaotic transitions between synchronization and propagation of bursts are found. We show that they consist of transient standing and rotating waves induced by symmetry-breaking bifurcations, and can be viewed as a manifestation of the phenomenon of chaotic itinerancy.     

Robust stability analysis of Takagi-Sugeno uncertain stochastic fuzzy recurrent neural networks with mixed time-varying delays

In this paper,the global stability of Takagi-Sugeno (TS) uncertain stochastic fuzzy recurrent neural networks with discrete and distributed time-varying delays (TSUSFRNNs) is considered.A novel LMI-based stability criterion is obtained by using Lyapunov functional theory to guarantee the asymptotic stability of TSUSFRNNs.The proposed stability conditions are demonstrated through numerical examples.Furthermore,the supplementary requirement that the time derivative of time-varying delays must be smaller than one is removed.Comparison results are demonstrated to show that the proposed method is more able to guarantee the widest stability region than the other methods available in the existing literature.     

Improved delay-dependent globally asymptotic stability of delayed uncertain recurrent neural networks with Markovian jumping parameters

In this paper, we have improved delay-dependent stability criteria for recurrent neural networks with a delay varying over a range and Markovian jumping parameters. The criteria improve over some previous ones in that they have fewer matrix variables yet less conservatism. In addition, a numerical example is provided to illustrate the applicability of the result using the linear matrix inequality toolbox in MATLAB.