Minimum Message Length in Hybrid ARMA and LSTM Model Forecasting

Modeling and analysis of time series are important in applications including economics, engineering, environmental science and social science. Selecting the best time series model with accurate parameters in forecasting is a challenging objective for scientists and academic researchers. Hybrid models combining neural networks and traditional Autoregressive Moving Average (ARMA) models are being used to improve the accuracy of modeling and forecasting time series. Most of the existing time series models are selected by information-theoretic approaches, such as AIC, BIC, and HQ. This paper revisits a model selection technique based on Minimum Message Length (MML) and investigates its use in hybrid time series analysis. MML is a Bayesian information-theoretic approach and has been used in selecting the best ARMA model. We utilize the long short-term memory (LSTM) approach to construct a hybrid ARMA-LSTM model and show that MML performs better than AIC, BIC, and HQ in selecting the model—both in the traditional ARMA models (without LSTM) and with hybrid ARMA-LSTM models. These results held on simulated data and both real-world datasets that we considered.We also develop a simple MML ARIMA model.     

Granger Causality on forward and Reversed Time Series

In this study, the information flow time arrow is investigated for stochastic data defined by vector autoregressive models. The time series are analyzed forward and backward by different Granger causality detection methods. Besides the normal distribution, which is usually required for the validity of Granger causality analysis, several other distributions of predictive errors are considered. A clear effect of a change in the order of cause and effect on the time-reversed series of unidirectionally connected variables was detected with standard Granger causality test (GC), when the product of the connection strength and the ratio of the predictive errors of the driver and the recipient was below a certain level, otherwise bidirectional causal connection was detected. On the other hand, opposite causal link was detected unconditionally by the methods based on the time reversal testing, but they were not able to detect correct bidirectional connection. The usefulness of the backward analysis is manifested in cases where falsely detected unidirectional connections can be rejected by applying the result obtained after the time reversal, and in cases of uncorrelated causally independent variables, where the absence of a causal link detected by GC on the original series should be confirmed on the time-reversed series.     

Granger causality and the inverse Ising problem

The inference of the couplings of an Ising model with given means and correlations is called the inverse Ising problem. This approach has received a lot of attention as a tool to analyze neural data. We show that autoregressive methods may be used to learn the couplings of an Ising model, also in the case of asymmetric connections and for multispin interactions. We find that, for each link, the linear Granger causality is two times the corresponding transfer entropy (i.e., the information flow on that link) in the weak coupling limit. For sparse connections and a low number of samples, the ?₁ regularized least squares method is used to detect the interacting pairs of spins. Nonlinear Granger causality is related to multispin interactions.     

Cross-Scale Causality and Information Transfer in Simulated Epileptic Seizures

An information-theoretic approach for detecting causality and information transfer was applied to phases and amplitudes of oscillatory components related to different time scales and obtained using the wavelet transform from a time series generated by the Epileptor model. Three main time scales and their causal interactions were identified in the simulated epileptic seizures, in agreement with the interactions of the model variables. An approach consisting of wavelet transform, conditional mutual information estimation, and surrogate data testing applied to a single time series generated by the model was demonstrated to be successful in the identification of all directional (causal) interactions between the three different time scales described in the model. Thus, the methodology was prepared for the identification of causal cross-frequency phase–phase and phase–amplitude interactions in experimental and clinical neural data.     

Transfer Entropy Granger Causality between News Indices and Stock Markets in U.S. and Latin America during the COVID-19 Pandemic

The relationship between three different groups of COVID-19 news series and stock market volatility for several Latin American countries and the U.S. are analyzed. To confirm the relationship between these series, a maximal overlap discrete wavelet transform (MODWT) was applied to determine the specific periods wherein each pair of series is significantly correlated. To determine if the news series cause Latin American stock markets’ volatility, a one-sided Granger causality test based on transfer entropy (GC-TE) was applied. The results confirm that the U.S. and Latin American stock markets react differently to COVID-19 news. Some of the most statistically significant results were obtained from the reporting case index (RCI), A-COVID index, and uncertainty index, in that order, which are statistically significant for the majority of Latin American stock markets. Altogether, the results suggest these COVID-19 news indices could be used to forecast stock market volatility in the U.S. and Latin America.     

Granger Causality among Graphs and Application to Functional Brain Connectivity in Autism Spectrum Disorder

Graphs/networks have become a powerful analytical approach for data modeling. Besides, with the advances in sensor technology, dynamic time-evolving data have become more common. In this context, one point of interest is a better understanding of the information flow within and between networks. Thus, we aim to infer Granger causality (G-causality) between networks’ time series. In this case, the straightforward application of the well-established vector autoregressive model is not feasible. Consequently, we require a theoretical framework for modeling time-varying graphs. One possibility would be to consider a mathematical graph model with time-varying parameters (assumed to be random variables) that generates the network. Suppose we identify G-causality between the graph models’ parameters. In that case, we could use it to define a G-causality between graphs. Here, we show that even if the model is unknown, the spectral radius is a reasonable estimate of some random graph model parameters. We illustrate our proposal’s application to study the relationship between brain hemispheres of controls and children diagnosed with Autism Spectrum Disorder (ASD). We show that the G-causality intensity from the brain’s right to the left hemisphere is different between ASD and controls.     

Simple Derivation of Minimum Length, Minimum Dipole Moment and Lack of Space–Time Continuity

The principle of maximum power makes it possible to summarize special relativity, quantum theory and general relativity in one fundamental limit principle each. Special relativity contains an upper limit to speed; following Bohr, quantum theory is based on a lower limit to action; recently, a maximum power given by c ⁵/4G was shown to be equivalent to the full field equations of general relativity. Taken together, these three fundamental principles imply a limit value for every physical observable, from acceleration to size. The new, precise limit values differ from the usual Planck values by numerical prefactors of order unity. Among others, minimum length and time intervals appear. The limits imply that elementary particles are not point-like and suggest a lower limit on electric dipole values. The minimum intervals also imply that the non-continuity of space–time is an inevitable result of the unification of quantum theory and relativity, independently of the approach used. PACS numbers: 04.20.Cv; 13.40.Em; 04.60.-m.     

Local Lead–Lag Relationships and Nonlinear Granger Causality: An Empirical Analysis

The Granger causality test is essential for detecting lead–lag relationships between time series. Traditionally, one uses a linear version of the test, essentially based on a linear time series regression, itself being based on autocorrelations and cross-correlations of the series. In the present paper, we employ a local Gaussian approach in an empirical investigation of lead–lag and causality relations. The study is carried out for monthly recorded financial indices for ten countries in Europe, North America, Asia and Australia. The local Gaussian approach makes it possible to examine lead–lag relations locally and separately in the tails and in the center of the return distributions of the series. It is shown that this results in a new and much more detailed picture of these relationships. Typically, the dependence is much stronger in the tails than in the center of the return distributions. It is shown that the ensuing nonlinear Granger causality tests may detect causality where traditional linear tests fail.     

一套新型基于物理放大原理的微小长度测量工具的设计

设计了一套利用圆盘放大和普通长度测量工具相结合的测量微小长度的工具,分别利用该设计装置和螺旋测微器对钢丝的直径和普通A4纸张的厚度进行了测量。结果显示了新设计的可行性、创新性和数据的高精度性、准确性。     

A Linguistic Analysis of News Coverage of E-Healthcare in China with a Heterogeneous Graphical Model

E-healthcare has been envisaged as a major component of the infrastructure of modern healthcare, and has been developing rapidly in China. For healthcare, news media can play an important role in raising public interest and utilization of a particular service and complicating (and, perhaps clouding) debate on public health policy issues. We conducted a linguistic analysis of news reports from January 2015 to June 2021 related to E-healthcare in mainland China, using a heterogeneous graphical modeling approach. This approach can simultaneously cluster the datasets and estimate the conditional dependence relationships of keywords. It was found that there were eight phases of media coverage. The focuses and main topics of media coverage were extracted based on the network hub and module detection. The temporal patterns of media reports were found to be mostly consistent with the policy trend. Specifically, in the policy embryonic period (2015–2016), two phases were obtained, industry management was the main topic, and policy and regulation were the focuses of media coverage. In the policy development period (2017–2019), four phases were discovered. All the four main topics, namely industry development, health care, financial market, and industry management, were present. In 2017 Q3–2017 Q4, the major focuses of media coverage included social security, healthcare and reform, and others. In 2018 Q1, industry regulation and finance became the focuses. In the policy outbreak period (2020–), two phases were discovered. Financial market and industry management were the main topics. Medical insurance and healthcare for the elderly became the focuses. This analysis can offer insights into how the media responds to public policy for E-healthcare, which can be valuable for the government, public health practitioners, health care industry investors, and others.     

Information Theoretic Causality Detection between Financial and Sentiment Data

The interaction between the flow of sentiment expressed on blogs and media and the dynamics of the stock market prices are analyzed through an information-theoretic measure, the transfer entropy, to quantify causality relations. We analyzed daily stock price and daily social media sentiment for the top 50 companies in the Standard & Poor (S&P) index during the period from November 2018 to November 2020. We also analyzed news mentioning these companies during the same period. We found that there is a causal flux of information that links those companies. The largest fraction of significant causal links is between prices and between sentiments, but there is also significant causal information which goes both ways from sentiment to prices and from prices to sentiment. We observe that the strongest causal signal between sentiment and prices is associated with the Tech sector.     

Message Passing-Based Inference for Time-Varying Autoregressive Models

Time-varying autoregressive (TVAR) models are widely used for modeling of non-stationary signals. Unfortunately, online joint adaptation of both states and parameters in these models remains a challenge. In this paper, we represent the TVAR model by a factor graph and solve the inference problem by automated message passing-based inference for states and parameters. We derive structured variational update rules for a composite “AR node” with probabilistic observations that can be used as a plug-in module in hierarchical models, for example, to model the time-varying behavior of the hyper-parameters of a time-varying AR model. Our method includes tracking of variational free energy (FE) as a Bayesian measure of TVAR model performance. The proposed methods are verified on a synthetic data set and validated on real-world data from temperature modeling and speech enhancement tasks.     

Robust Spike-Based Continual Meta-Learning Improved by Restricted Minimum Error Entropy Criterion

The spiking neural network (SNN) is regarded as a promising candidate to deal with the great challenges presented by current machine learning techniques, including the high energy consumption induced by deep neural networks. However, there is still a great gap between SNNs and the online meta-learning performance of artificial neural networks. Importantly, existing spike-based online meta-learning models do not target the robust learning based on spatio-temporal dynamics and superior machine learning theory. In this invited article, we propose a novel spike-based framework with minimum error entropy, called MeMEE, using the entropy theory to establish the gradient-based online meta-learning scheme in a recurrent SNN architecture. We examine the performance based on various types of tasks, including autonomous navigation and the working memory test. The experimental results show that the proposed MeMEE model can effectively improve the accuracy and the robustness of the spike-based meta-learning performance. More importantly, the proposed MeMEE model emphasizes the application of the modern information theoretic learning approach on the state-of-the-art spike-based learning algorithms. Therefore, in this invited paper, we provide new perspectives for further integration of advanced information theory in machine learning to improve the learning performance of SNNs, which could be of great merit to applied developments with spike-based neuromorphic systems.     

Graphical Models in Reconstructability Analysis and Bayesian Networks

Reconstructability Analysis (RA) and Bayesian Networks (BN) are both probabilistic graphical modeling methodologies used in machine learning and artificial intelligence. There are RA models that are statistically equivalent to BN models and there are also models unique to RA and models unique to BN. The primary goal of this paper is to unify these two methodologies via a lattice of structures that offers an expanded set of models to represent complex systems more accurately or more simply. The conceptualization of this lattice also offers a framework for additional innovations beyond what is presented here. Specifically, this paper integrates RA and BN by developing and visualizing: (1) a BN neutral system lattice of general and specific graphs, (2) a joint RA-BN neutral system lattice of general and specific graphs, (3) an augmented RA directed system lattice of prediction graphs, and (4) a BN directed system lattice of prediction graphs. Additionally, it (5) extends RA notation to encompass BN graphs and (6) offers an algorithm to search the joint RA-BN neutral system lattice to find the best representation of system structure from underlying system variables. All lattices shown in this paper are for four variables, but the theory and methodology presented in this paper are general and apply to any number of variables. These methodological innovations are contributions to machine learning and artificial intelligence and more generally to complex systems analysis. The paper also reviews some relevant prior work of others so that the innovations offered here can be understood in a self-contained way within the context of this paper.     

Dynamic Analyses of Contagion Risk and Module Evolution on the SSE A-Shares Market Based on Minimum Information Entropy

The interactive effect is significant in the Chinese stock market, exacerbating the abnormal market volatilities and risk contagion. Based on daily stock returns in the Shanghai Stock Exchange (SSE) A-shares, this paper divides the period between 2005 and 2018 into eight bull and bear market stages to investigate interactive patterns in the Chinese financial market. We employ the Least Absolute Shrinkage and Selection Operator (LASSO) method to construct the stock network, compare the heterogeneity of bull and bear markets, and further use the Map Equation method to analyse the evolution of modules in the SSE A-shares market. Empirical results show that (1) the connected effect is more significant in bear markets than bull markets and gives rise to abnormal volatilities in the stock market; (2) a system module can be found in the network during the first four stages, and the industry aggregation effect leads to module differentiation in the last four stages; (3) some stocks have leading effects on others throughout eight periods, and medium- and small-cap stocks with poor financial conditions are more likely to become risk sources, especially in bear markets. Our conclusions are beneficial to improving investment strategies and making regulatory policies.