Nonlinear Time Series Prediction Using Chaotic Neural Networks

A nonlinear feedback term is introduced into the evaluation equation of weights of the backpropagation algorithm for neural network,the network becomes a chaotic one.For the purpose of that we can investigate how the different feedback terms affect the process of learning and forecasting,we use the model to forecast the nonlinear time series which is produced by Makey-Glass equation.By selecting the suitable feedback term,the system can escape from the local minima and converge to the global minimum or its approximate solutions,and the forecasting results are better than those of backpropagation algorithm.     

Phase Space Prediction Model Using Neural Networks

A new nonlinear prediction technique is proposed by feedforward neural network, the learning algorithmfor network is a chaotic one. A timc-delay embedding is used to reconstruct the underlying attractor, the predictionmodel is based on the time evolution of the topological neighboring in the phase space, the spatial neighbors are chosenby the rate of exponential divergence of close trajectory. The model is tested for the Mackey-Glass delay equation andLorentz equations, good results are obtained for the prediction.     

Dynamics of Transiently Chaotic Neural Network and Its Application to Optimization

Through adding a nonlinear self-feedback term in the evolution equations of nerual network,we introduced a transiently chaotic neural network model.In order to utilize the transiently chaotic dynamics mechanism in optimization problem efficiently,we have analyzed the dynamical pocedure of the transiently chaotic neural network model and studied the function of the crucial bifurcation parameter which governs the chaotic behavior of the system.Based on the dynamical analysis of the transiently chaotic neural network model,Chaotic annealing algorithm is also examined and improved.As an example,we applied chaotic annealing method to the traveling salesman problem and obtained good results.     

Nonlinear Time Series Forecast Using Radial Basis Function Neural Networks

In the research of using Radial Basis Function Neural Network (RBF NN) forecasting nonlinear timeseries, we investigate how the different clusterings affect the process of learning and forecasting. We find that k-meansclustering is very suitable. In order to increase the precision we introduce a nonlinear feedback term to escape from thelocal minima of energy, then we use the model to forecast the nonlinear time series which are produced by Mackey-Glassequation and stocks. By selecting the k-means clustering and the suitable feedback term, much better forecasting resultsare obtained.     

Nonlinear Time Series Forecast Using Radial Basis Function Neural Networks

In the research of using Radial Basis Function Neural Network (RBF NN) forecasting nonlinear time series, we investigate how the different clusterings affect the process of learning and forecasting. We find that k-means clustering is very suitable. In order to increase the precision we introduce a nonlinear feedback term to escape from the local minima of energy, then we use the model to forecast the nonlinear time series which are produced by Mackey-Glass equation and stocks. By selecting the k-means clustering and the suitable feedback term, much better forecasting results are obtained.     

Discontinuity and Complex Dynamics of Neural Networks

We focus on the discontinuity of a neural network model with diluted and clipped synaptic connections (±l only). The exact evolution rule of the average firing rate becomes a discontinuous piece-wise nonlinear map when very simple functions of dynamical threshold are introduced into the network. Complex dynamics is observed.     

Time Series Prediction Based on Chaotic Attractor

A new prediction technique is proposed for chaotic time series. The usefulness of the technique is thatit can kick off some false neighbor points which are not suitable for the local estimation of the dynamics systems. Atime-delayed embedding is used to reconstruct the underlying attractor, and the prediction model is based on the timeevolution of the topological neighboring in the phase space. We use a feedforward neural network to approximate thelocal dominant Lyapunov exponent, and choose the spatial neighbors by the Lyapunov exponent. The model is testedfor the Mackey-Glass equation and the convection amplitude of lorenz systems. The results indicate that this predictiontechnique can improve the prediction of chaotic time series.     

Time Series Prediction Based on Chaotic Attractor

A new prediction technique is proposed for chaotic time series. The usefulness of the technique is that it can kick off some false neighbor points which are not suitable for the local estimation of the dynamics systems. A time-delayed embedding is used to reconstruct the underlying attractor, and the prediction model is based on the time evolution of the topological neighboring in the phase space. We use a feedforward neural network to approximate the local dominant Lyapunov exponent, and choose the spatial neighbors by the Lyapunov exponent. The model is tested for the Mackey-Glass equation and the convection amplitude of lorenz systems. The results indicate that this prediction technique can improve the prediction of chaotic time series.     

Entanglement-Structured LSTM Boosts Chaotic Time Series Forecasting

Traditional machine-learning methods are inefficient in capturing chaos in nonlinear dynamical systems, especially when the time difference Δt between consecutive steps is so large that the extracted time series looks apparently random. Here, we introduce a new long-short-term-memory (LSTM)-based recurrent architecture by tensorizing the cell-state-to-state propagation therein, maintaining the long-term memory feature of LSTM, while simultaneously enhancing the learning of short-term nonlinear complexity. We stress that the global minima of training can be most efficiently reached by our tensor structure where all nonlinear terms, up to some polynomial order, are treated explicitly and weighted equally. The efficiency and generality of our architecture are systematically investigated and tested through theoretical analysis and experimental examinations. In our design, we have explicitly used two different many-body entanglement structures—matrix product states (MPS) and the multiscale entanglement renormalization ansatz (MERA)—as physics-inspired tensor decomposition techniques, from which we find that MERA generally performs better than MPS, hence conjecturing that the learnability of chaos is determined not only by the number of free parameters but also the tensor complexity—recognized as how entanglement entropy scales with varying matricization of the tensor.     

基于模糊边界模块化神经网络的混沌时间序列预测

提出一种模糊边界模块化神经网络（FBMNN）的混沌时间序列预测方法,该方法先对混沌时间序列观测点重构的相空间进行模块化划分,划分点的选取由遗传算法自动寻优.然后定义一个模糊隶属度函数,在划分边界一侧按照一定的模糊隶属度设定模糊边界带,通过模糊化处理,解决了各模块划分点附近预测结果的跳跃问题.最后每一模块,及其模糊边界的样本点都对应一个递归神经网络进行训练,通过预测合成模块输出结果.该方法对三个混沌时间序列基准数据集Mackey-Glass,Lorenz,Henon进行实验,结果表明该方法有效地提高了混沌时间序列预测效果. 关键词： 模糊边界 模块化神经网络 混沌时间序列 预测     

A Method for Estimating the Entropy of Time Series Using Artificial Neural Networks

Measuring the predictability and complexity of time series using entropy is essential tool designing and controlling a nonlinear system. However, the existing methods have some drawbacks related to the strong dependence of entropy on the parameters of the methods. To overcome these difficulties, this study proposes a new method for estimating the entropy of a time series using the LogNNet neural network model. The LogNNet reservoir matrix is filled with time series elements according to our algorithm. The accuracy of the classification of images from the MNIST-10 database is considered as the entropy measure and denoted by NNetEn. The novelty of entropy calculation is that the time series is involved in mixing the input information in the reservoir. Greater complexity in the time series leads to a higher classification accuracy and higher NNetEn values. We introduce a new time series characteristic called time series learning inertia that determines the learning rate of the neural network. The robustness and efficiency of the method is verified on chaotic, periodic, random, binary, and constant time series. The comparison of NNetEn with other methods of entropy estimation demonstrates that our method is more robust and accurate and can be widely used in practice.     

Neural Volterra filter for chaotic time series prediction

A new second-order neural Volterra filter （SONVF） with conjugate gradient （CG） algorithm is proposed to predict chaotic time series based on phase space delay-coordinate reconstruction of chaotic dynamics system in this paper, where the neuron activation functions are introduced to constraint Volterra series terms for improving the nonlinear approximation of second-order Volterra filter （SOVF）. The SONVF with CG algorithm improves the accuracy of prediction without increasing the computation complexity. Meanwhile, the difficulty of neuron number determination does not exist here. Experimental results show that the proposed filter can predict chaotic time series effectively, and one-step and multi-step prediction performances are obviously superior to those of SOVF, which demonstrate that the proposed SONVF is feasible and effective.     

Prediction of Time Series Gene Expression and Structural Analysis of Gene Regulatory Networks Using Recurrent Neural Networks

Methods for time series prediction and classification of gene regulatory networks (GRNs) from gene expression data have been treated separately so far. The recent emergence of attention-based recurrent neural network (RNN) models boosted the interpretability of RNN parameters, making them appealing for the understanding of gene interactions. In this work, we generated synthetic time series gene expression data from a range of archetypal GRNs and we relied on a dual attention RNN to predict the gene temporal dynamics. We show that the prediction is extremely accurate for GRNs with different architectures. Next, we focused on the attention mechanism of the RNN and, using tools from graph theory, we found that its graph properties allow one to hierarchically distinguish different architectures of the GRN. We show that the GRN responded differently to the addition of noise in the prediction by the RNN and we related the noise response to the analysis of the attention mechanism. In conclusion, this work provides a way to understand and exploit the attention mechanism of RNNs and it paves the way to RNN-based methods for time series prediction and inference of GRNs from gene expression data.     

基于多层神经网络的非线性图像分割

提出了一种用多层神经网络对图像进行非线性分割的方法。讨论了所用多层神经网络的学习速度的改进与训练样本的选择方法。实验表明，该多层神经网络系统可用于实时图像分割，并能获得很好的结果。     

基于在线最小二乘支持向量机回归的混沌时间序列预测

提出了一种基于在线最小二乘支持向量机（LS-SVM）回归的混沌时间序列的预测方法.与离线支持向量机相比，在线最小二乘支持向量机预测方法即使当混沌系统的参数随时间变化时仍然有效.以Chen's混沌系统、Rssler混沌系统、Hénon映射及脑电(EEG)信号四种混沌时 间序列为例评估本文提出的预测方法，结果验证了其混沌时间序列预测的有效性. 关键词： 混沌时间序列 预测 在线学习 支持向量机     

History Marginalization Improves Forecasting in Variational Recurrent Neural Networks

Deep probabilistic time series forecasting models have become an integral part of machine learning. While several powerful generative models have been proposed, we provide evidence that their associated inference models are oftentimes too limited and cause the generative model to predict mode-averaged dynamics. Mode-averaging is problematic since many real-world sequences are highly multi-modal, and their averaged dynamics are unphysical (e.g., predicted taxi trajectories might run through buildings on the street map). To better capture multi-modality, we develop variational dynamic mixtures (VDM): a new variational family to infer sequential latent variables. The VDM approximate posterior at each time step is a mixture density network, whose parameters come from propagating multiple samples through a recurrent architecture. This results in an expressive multi-modal posterior approximation. In an empirical study, we show that VDM outperforms competing approaches on highly multi-modal datasets from different domains.     

Effects of Coupling Distance on Synchronization and Coherence in Chaotic Neural Networks

Effects of coupling distance on synchronization and coherence of chaotic neurons in complex networks are numerically investigated. We find that it is not beneficial to neurons synchronization if confining the coupling distance of random edges to a limit dmax, but help to improve their coherence. Moreover, there is an optimal value of dmax at which the coherence is maximum.     

Phase Space Prediction of Chaotic Time Series with Nu-Support Vector Machine Regression

A new class of support vector machine, nu-support vector machine, is discussed which can handle both classification and regression. We focus on nu-support vector machine regression and use it for phase space prediction of chaotic time series. The effectiveness of the method is demonstrated by applying it to the Hénon map. This study also compares nu-support vector machine with back propagation (BP) networks in order to better evaluate the performance of the proposed methods. The experimental results show that the nu-support vector machine regression obtains lower root mean squared error than the BP networks and provides an accurate chaotic time series prediction. These results can be attributable to the fact that nu-support vector machine implements the structural risk minimization principle and this leads to better generalization than the BP networks.     

具有增加删除机制的正则化极端学习机的混沌时间序列预测

针对正则化极端学习机的隐层具有随机选择的特性,提出了一种增加删除机制来自适应地确定正则化极端学习机的隐层节点数. 这种机制以对优化目标函数影响的大小作为评价隐层节点优劣的标准,从而淘汰那些比较“差”的节点,将那些比较“优”的节点保留下来,起到一个优化正则化极端学习机隐层节点数的目的. 与已有的只具有增加隐层节点数的机制相比较,本文提出的增加删除机制在减少正则化极端学习机隐层节点数、增强其泛化性能、提高其实时性等方面具有一定的优势. 典型混沌时间序列的实例证明了具有增加删除机制的正则化极端学习机的有效性和可行性. 关键词： 混沌时间序列 人工神经网络 极端学习机     

Coupling between Blood Pressure and Subarachnoid Space Width Oscillations during Slow Breathing

The precise mechanisms connecting the cardiovascular system and the cerebrospinal fluid (CSF) are not well understood in detail. This paper investigates the couplings between the cardiac and respiratory components, as extracted from blood pressure (BP) signals and oscillations of the subarachnoid space width (SAS), collected during slow ventilation and ventilation against inspiration resistance. The experiment was performed on a group of 20 healthy volunteers (12 females and 8 males; BMI =22.1±3.2 kg/m2; age 25.3±7.9 years). We analysed the recorded signals with a wavelet transform. For the first time, a method based on dynamical Bayesian inference was used to detect the effective phase connectivity and the underlying coupling functions between the SAS and BP signals. There are several new findings. Slow breathing with or without resistance increases the strength of the coupling between the respiratory and cardiac components of both measured signals. We also observed increases in the strength of the coupling between the respiratory component of the BP and the cardiac component of the SAS and vice versa. Slow breathing synchronises the SAS oscillations, between the brain hemispheres. It also diminishes the similarity of the coupling between all analysed pairs of oscillators, while inspiratory resistance partially reverses this phenomenon. BP–SAS and SAS–BP interactions may reflect changes in the overall biomechanical characteristics of the brain.