The effect of distributed time-delays on the synchronization of neuronal networks

Here we investigate the synchronization of networks of FitzHugh-Nagumo neurons coupled in scale-free, small-world and random topologies, in the presence of distributed time delays in the coupling of neurons. We explore how the synchronization transition is affected when the time delays in the interactions between pairs of interacting neurons are non-uniform. We find that the presence of distributed time-delays does not change the behavior of the synchronization transition significantly, vis-a-vis networks with constant time-delay, where the value of the constant time-delay is the mean of the distributed delays. We also notice that a normal distribution of delays gives rise to a transition at marginally lower coupling strengths, vis-a-vis uniformly distributed delays. These trends hold across classes of networks and for varying standard deviations of the delay distribution, indicating the generality of these results. So we conclude that distributed delays, which may be typically expected in real-world situations, do not have a notable effect on synchronization. This allows results obtained with constant delays to remain relevant even in the case of randomly distributed delays.     

From Continuous-Time Chaotic Systems to Pseudo Random Number Generators: Analysis and Generalized Methodology

The use of chaotic systems in electronics, such as Pseudo-Random Number Generators (PRNGs), is very appealing. Among them, continuous-time ones are used less because, in addition to having strong temporal correlations, they require further computations to obtain the discrete solutions. Here, the time step and discretization method selection are first studied by conducting a detailed analysis of their effect on the systems’ statistical and chaotic behavior. We employ an approach based on interpreting the time step as a parameter of the new “maps”. From our analysis, it follows that to use them as PRNGs, two actions should be achieved (i) to keep the chaotic oscillation and (ii) to destroy the inner and temporal correlations. We then propose a simple methodology to achieve chaos-based PRNGs with good statistical characteristics and high throughput, which can be applied to any continuous-time chaotic system. We analyze the generated sequences by means of quantifiers based on information theory (permutation entropy, permutation complexity, and causal entropy × complexity plane). We show that the proposed PRNG generates sequences that successfully pass Marsaglia Diehard and NIST (National Institute of Standards and Technology) tests. Finally, we show that its hardware implementation requires very few resources.     

Role of the phase in the identification of delay time in semiconductor lasers with optical feedback

We consider a semiconductor laser with external optical feedback operating at a regime for which the delay time signature is extremely difficult to identify from the analysis of the intensity time series, using standard techniques. We show that such a delay signature can be successfully retrieved by computing the same quantifiers from the phase, the real or the imaginary part of the field, even in the presence of noise. Therefore, the choice of the observable is the determinant for parameter identification.     

A Rolling Bearing Fault Classification Scheme Based on k-Optimized Adaptive Local Iterative Filtering and Improved Multiscale Permutation Entropy

The health condition of the rolling bearing seriously affects the operation of the whole mechanical system. When the rolling bearing parts fail, the time series collected in the field generally shows strong nonlinearity and non-stationarity. To obtain the faulty characteristics of mechanical equipment accurately, a rolling bearing fault detection technique based on k-optimized adaptive local iterative filtering (ALIF), improved multiscale permutation entropy (improved MPE), and BP neural network was proposed. In the ALIF algorithm, a k-optimized ALIF method based on permutation entropy (PE) is presented to select the number of ALIF decomposition layers adaptively. The completely average coarse-graining method was proposed to excavate more hidden information. The performance analysis of the simulation signal shows that the improved MPE can more accurately dig out the depth information of the time series, and the entropy value obtained is more consistent and stable. In the research application, rolling bearing time series are decomposed by k-optimized ALIF to obtain a certain number of intrinsic mode functions (IMFs). Then the improved MPE value of effective IMF is calculated and input into backpropagation (BP) neural network as the feature vector for automatic fault identification. The comparative analysis of simulation signals shows that this method can extract fault information effectively. At the same time, the experimental part shows that this scheme not only effectively extracts the fault features, but also realizes the classification and identification of different fault modes and faults of different degrees, which has a certain application prospect in the research and application direction of rolling bearing fault identification.     

Estimating Permutation Entropy Variability via Surrogate Time Series

In the last decade permutation entropy (PE) has become a popular tool to analyze the degree of randomness within a time series. In typical applications, changes in the dynamics of a source are inferred by observing changes of PE computed on different time series generated by that source. However, most works neglect the crucial question related to the statistical significance of these changes. The main reason probably lies in the difficulty of assessing, out of a single time series, not only the PE value, but also its uncertainty. In this paper we propose a method to overcome this issue by using generation of surrogate time series. The analysis conducted on both synthetic and experimental time series shows the reliability of the approach, which can be promptly implemented by means of widely available numerical tools. The method is computationally affordable for a broad range of users.     

利用排列熵检测近40年华北地区气温突变的研究

运用一种新的动力学突变检测方法——排列熵(permutation entropy，PE)算法，计算并分析了中国华北地区52个站点1960年—2000年逐日平均气温资料的排列熵演化情况，发现中国华北地区气温在20世纪70年代中期、80年代初均发生了较大突变；进一步用经验模态分解(empirical mode decomposition，EMD)方法对排列熵序列进行逐级平稳化处理，结果发现这一地区的气温突变与准10年这一年代际时间尺度的周期变率密切相关，其原因与太阳黑子活动有着密切联系. 关键词： 华北 突变 排列熵算法 经验模态分解     

Characteristics and synchronization of time-delay systems driven by a common noise

We investigate the characteristics of time-delay systems in the presence of Gaussian noise. We show that the delay time embedded in the time series of time-delay system with constant delay cannot be estimated in the presence noise for appropriate values of noise intensity thereby forbidding any possibility of phase space reconstruction. We also demonstrate the existence of complete synchronization between two independent identical time-delay systems driven by a common noise without explicitly establishing any external coupling between them.     

Cross-Scale Causality and Information Transfer in Simulated Epileptic Seizures

An information-theoretic approach for detecting causality and information transfer was applied to phases and amplitudes of oscillatory components related to different time scales and obtained using the wavelet transform from a time series generated by the Epileptor model. Three main time scales and their causal interactions were identified in the simulated epileptic seizures, in agreement with the interactions of the model variables. An approach consisting of wavelet transform, conditional mutual information estimation, and surrogate data testing applied to a single time series generated by the model was demonstrated to be successful in the identification of all directional (causal) interactions between the three different time scales described in the model. Thus, the methodology was prepared for the identification of causal cross-frequency phase–phase and phase–amplitude interactions in experimental and clinical neural data.     

Unsupervised Anomaly Detection with Distillated Teacher-Student Network Ensemble

We address the problem of unsupervised anomaly detection for multivariate data. Traditional machine learning based anomaly detection algorithms rely on specific assumptions of normal patterns and fail to model complex feature interactions and relations. Recently, existing deep learning based methods are promising for extracting representations from complex features. These methods train an auxiliary task, e.g., reconstruction and prediction, on normal samples. They further assume that anomalies fail to perform well on the auxiliary task since they are never trained during the model optimization. However, the assumption does not always hold in practice. Deep models may also perform the auxiliary task well on anomalous samples, leading to the failure detection of anomalies. To effectively detect anomalies for multivariate data, this paper introduces a teacher-student distillation based framework Distillated Teacher-Student Network Ensemble (DTSNE). The paradigm of the teacher-student distillation is able to deal with high-dimensional complex features. In addition, an ensemble of student networks provides a better capability to avoid generalizing the auxiliary task performance on anomalous samples. To validate the effectiveness of our model, we conduct extensive experiments on real-world datasets. Experimental results show superior performance of DTSNE over competing methods. Analysis and discussion towards the behavior of our model are also provided in the experiment section.     

Efficient use of correlation entropy for analysing time series data

The correlation dimension D ₂ and correlation entropy K ₂ are both important quantifiers in nonlinear time series analysis. However, use of D ₂ has been more common compared to K ₂ as a discriminating measure. One reason for this is that D ₂ is a static measure and can be easily evaluated from a time series. However, in many cases, especially those involving coloured noise, K ₂ is regarded as a more useful measure. Here we present an efficient algorithmic scheme to compute K ₂ directly from a time series data and show that K ₂ can be used as a more effective measure compared to D ₂ for analysing practical time series involving coloured noise.      

The Impact of Linear Filter Preprocessing in the Interpretation of Permutation Entropy

Permutation Entropy (PE) is a powerful tool for measuring the amount of information contained within a time series. However, this technique is rarely applied directly on raw signals. Instead, a preprocessing step, such as linear filtering, is applied in order to remove noise or to isolate specific frequency bands. In the current work, we aimed at outlining the effect of linear filter preprocessing in the final PE values. By means of the Wiener–Khinchin theorem, we theoretically characterize the linear filter’s intrinsic PE and separated its contribution from the signal’s ordinal information. We tested these results by means of simulated signals, subject to a variety of linear filters such as the moving average, Butterworth, and Chebyshev type I. The PE results from simulations closely resembled our predicted results for all tested filters, which validated our theoretical propositions. More importantly, when we applied linear filters to signals with inner correlations, we were able to theoretically decouple the signal-specific contribution from that induced by the linear filter. Therefore, by providing a proper framework of PE linear filter characterization, we improved the PE interpretation by identifying possible artifact information introduced by the preprocessing steps.     

Optimal instruments and models for noisy chaos

Analysis of finite, noisy time series data leads to modern statistical inference methods. Here we adapt Bayesian inference for applied symbolic dynamics. We show that reconciling Kolmogorov's maximum-entropy partition with the methods of Bayesian model selection requires the use of two separate optimizations. First, instrument design produces a maximum-entropy symbolic representation of time series data. Second, Bayesian model comparison with a uniform prior selects a minimum-entropy model, with respect to the considered Markov chain orders, of the symbolic data. We illustrate these steps using a binary partition of time series data from the logistic and Henon maps as well as the R?ssler and Lorenz attractors with dynamical noise. In each case we demonstrate the inference of effectively generating partitions and kth-order Markov chain models.     

Determination of parameters of elements and coupling architecture in ensembles of coupled time-delay systems from their time series

We propose a method for reconstructing model differential equations with time delay for ensembles of coupled time-delay systems from their time series. The method has made it possible to recover the parameters of elements of the ensemble as well as the architecture and strength of couplings in ensembles of nonidentical systems with delay with an arbitrary number of unidirectional and bidirectional couplings between them. The effectiveness of the method is demonstrated for chaotic and periodic time series of model equations for ensembles of diffusively coupled systems with time delay in the presence of noise, as well as experimental time series for resistively coupled radiotechnical oscillators with delayed feedback.     

Use of random matrix theory in the discovery of resting state brain networks

Connectomics identifies brain networks in vivo in resting state functional MRI. However, the presence of noise produces spurious identification of brain networks, which have low test-retest reliability. A Network Based Statistics approach to network identification has been previously proposed that affords much better statistical power relative to Bonferroni method but nevertheless provides a sufficiently conservative, family-wise control for false positives. We propose the use of Random Matrix Theory (RMT) to discover brain networks and to associate those networks with demographic and clinical variables. We parcellated the brain into cortical and subcortical regions using either an anatomical or a functional brain atlas. We applied RMT to study functional connectivity across brain regions by first computing the correlation matrix for time courses in those brain regions and then identifying eigenvalues that deviate from the theoretical random distribution that RMT predicts, on the assumption that real brain networks would produce eigenvalues that differ significantly from the random distribution. We assessed the specificity and test-retest reliability of identified networks through application of this RMT-based approach to (1) synthetic data generated under the null-hypothesis, (2) resting state functional MRI data from 4 real-world cohorts of patients and healthy controls, and (3) synthetic data generated by the addition of increasing amounts of noise to real-world datasets. Our findings showed that RMT method was robust to the atlas used for parcellating the brain and did not discover a brain network in synthetic data when in fact a network was not present (i.e., specificity was high); RMT-identified networks in the real-world dataset had high test-retest reliability; and RMT-based method consistently discovered the same network in the presence of increasing noise in the real-world dataset.     

Regular nonlinear response of the driven Duffing oscillator to chaotic time series

Nonlinear response of the driven Duffng oscillator to periodic or quasi-periodic signals has been well studied.In this paper,we investigate the nonlinear response of the driven Duffng oscillator to non-periodic,more specifically,chaotic time series.Through numerical simulations,we find that the driven Duffng oscillator can also show regular nonlinear response to the chaotic time series with different degree of chaos as generated by the same chaotic series generating model,and there exists a relationship between the state of the driven Duffng oscillator and the chaoticity of the input signal of the driven Duffng oscillator.One real-world and two artificial chaotic time series are used to verify the new feature of Duffng oscillator.A potential application of the new feature of Duffng oscillator is also indicated.     

Minimum Message Length in Hybrid ARMA and LSTM Model Forecasting

Modeling and analysis of time series are important in applications including economics, engineering, environmental science and social science. Selecting the best time series model with accurate parameters in forecasting is a challenging objective for scientists and academic researchers. Hybrid models combining neural networks and traditional Autoregressive Moving Average (ARMA) models are being used to improve the accuracy of modeling and forecasting time series. Most of the existing time series models are selected by information-theoretic approaches, such as AIC, BIC, and HQ. This paper revisits a model selection technique based on Minimum Message Length (MML) and investigates its use in hybrid time series analysis. MML is a Bayesian information-theoretic approach and has been used in selecting the best ARMA model. We utilize the long short-term memory (LSTM) approach to construct a hybrid ARMA-LSTM model and show that MML performs better than AIC, BIC, and HQ in selecting the model—both in the traditional ARMA models (without LSTM) and with hybrid ARMA-LSTM models. These results held on simulated data and both real-world datasets that we considered.We also develop a simple MML ARIMA model.     

The Refined Composite Downsampling Permutation Entropy Is a Relevant Tool in the Muscle Fatigue Study Using sEMG Signals

Surface electromyography (sEMG) is a valuable technique that helps provide functional and structural information about the electric activity of muscles. As sEMG measures output of complex living systems characterized by multiscale and nonlinear behaviors, Multiscale Permutation Entropy (MPE) is a suitable tool for capturing useful information from the ordinal patterns of sEMG time series. In a previous work, a theoretical comparison in terms of bias and variance of two MPE variants—namely, the refined composite MPE (rcMPE) and the refined composite downsampling (rcDPE), was addressed. In the current paper, we assess the superiority of rcDPE over MPE and rcMPE, when applied to real sEMG signals. Moreover, we demonstrate the capacity of rcDPE in quantifying fatigue levels by using sEMG data recorded during a fatiguing exercise. The processing of four consecutive temporal segments, during biceps brachii exercise maintained at 70% of maximal voluntary contraction until exhaustion, shows that the 10th-scale of rcDPE was capable of better differentiation of the fatigue segments. This scale actually brings the raw sEMG data, initially sampled at 10 kHz, to the specific 0–500 Hz sEMG spectral band of interest, which finally reveals the inner complexity of the data. This study promotes good practices in the use of MPE complexity measures on real data.     

Quantum quantifiers of Raman photon pairs with relativistic motion

In this paper, we develop a model for four-level double Raman pairs by exploiting the required optimal conditions for this system that are feasible with real experimental realization. We investigate qualitatively the entanglement, statistical properties, and geometric phase for the pair of Stokes and anti-Stokes photons in the presence of the relativistic motion. We show that these quantifiers are very sensitive to the change of the Rabi frequency under relativistic motion, exhibiting substantial phenomena that depend on this kind of the coupling between the atom and photons. Finally, we explore the relationship between the quantum quantifiers for constant and time-dependent coupling.     

History Marginalization Improves Forecasting in Variational Recurrent Neural Networks

Deep probabilistic time series forecasting models have become an integral part of machine learning. While several powerful generative models have been proposed, we provide evidence that their associated inference models are oftentimes too limited and cause the generative model to predict mode-averaged dynamics. Mode-averaging is problematic since many real-world sequences are highly multi-modal, and their averaged dynamics are unphysical (e.g., predicted taxi trajectories might run through buildings on the street map). To better capture multi-modality, we develop variational dynamic mixtures (VDM): a new variational family to infer sequential latent variables. The VDM approximate posterior at each time step is a mixture density network, whose parameters come from propagating multiple samples through a recurrent architecture. This results in an expressive multi-modal posterior approximation. In an empirical study, we show that VDM outperforms competing approaches on highly multi-modal datasets from different domains.