An Exploratory Study on the Complexity and Machine Learning Predictability of Stock Market Data

This paper shows if and how the predictability and complexity of stock market data changed over the last half-century and what influence the M1 money supply has. We use three different machine learning algorithms, i.e., a stochastic gradient descent linear regression, a lasso regression, and an XGBoost tree regression, to test the predictability of two stock market indices, the Dow Jones Industrial Average and the NASDAQ (National Association of Securities Dealers Automated Quotations) Composite. In addition, all data under study are discussed in the context of a variety of measures of signal complexity. The results of this complexity analysis are then linked with the machine learning results to discover trends and correlations between predictability and complexity. Our results show a decrease in predictability and an increase in complexity for more recent years. We find a correlation between approximate entropy, sample entropy, and the predictability of the employed machine learning algorithms on the data under study. This link between the predictability of machine learning algorithms and the mentioned entropy measures has not been shown before. It should be considered when analyzing and predicting complex time series data, e.g., stock market data, to e.g., identify regions of increased predictability.     

Evolution of Entropy in Art Painting Based on the Wavelet Transform

Quantitative studies of art and aesthetics are representative of interdisciplinary research. In this work, we conducted a large-scale quantitative study of 36,000 paintings covering both Eastern and Western paintings. The information entropy and wavelet entropy of the images were calculated based on their complexity and energy. Wavelet energy entropy is a feature that can characterize rich information in images, and this is the first study to introduce this feature into aesthetic analysis of art paintings. This study shows that the process of entropy change coincides with the development process of art painting. Further, the experimental results demonstrate an important change in the evolution of art painting, and since the rise of modern art in the twentieth century, the entropy values in painting have started to become diverse. In comparison with Western paintings, Eastern paintings have distinct low entropy characteristics in which the wavelet entropy feature of the images has better results in the machine learning classification task of Eastern and Western paintings (i.e., the F1 score can reach 97%). Our study can be the basis for future quantitative analysis and comparative research in the context of Western and Eastern art aesthetics.     

Intelligent Fault Diagnosis of Rolling-Element Bearings Using a Self-Adaptive Hierarchical Multiscale Fuzzy Entropy

The fuzzy-entropy-based complexity metric approach has achieved fruitful results in bearing fault diagnosis. However, traditional hierarchical fuzzy entropy (HFE) and multiscale fuzzy entropy (MFE) only excavate bearing fault information on different levels or scales, but do not consider bearing fault information on both multiple layers and multiple scales at the same time, thus easily resulting in incomplete fault information extraction and low-rise identification accuracy. Besides, the key parameters of most existing entropy-based complexity metric methods are selected based on specialist experience, which indicates that they lack self-adaptation. To address these problems, this paper proposes a new intelligent bearing fault diagnosis method based on self-adaptive hierarchical multiscale fuzzy entropy. On the one hand, by integrating the merits of HFE and MFE, a novel complexity metric method, named hierarchical multiscale fuzzy entropy (HMFE), is presented to extract a multidimensional feature matrix of the original bearing vibration signal, where the important parameters of HMFE are automatically determined by using the bird swarm algorithm (BSA). On the other hand, a nonlinear feature matrix classifier with strong robustness, known as support matrix machine (SMM), is introduced for learning the discriminant fault information directly from the extracted multidimensional feature matrix and automatically identifying different bearing health conditions. Two experimental results on bearing fault diagnosis show that the proposed method can obtain average identification accuracies of 99.92% and 99.83%, respectively, which are higher those of several representative entropies reported by this paper. Moreover, in the two experiments, the standard deviations of identification accuracy of the proposed method were, respectively, 0.1687 and 0.2705, which are also greater than those of the comparison methods mentioned in this paper. The effectiveness and superiority of the proposed method are verified by the experimental results.     

Increased Entropic Brain Dynamics during DeepDream-Induced Altered Perceptual Phenomenology

In recent years, the use of psychedelic drugs to study brain dynamics has flourished due to the unique opportunity they offer to investigate the neural mechanisms of conscious perception. Unfortunately, there are many difficulties to conduct experiments on pharmacologically-induced hallucinations, especially regarding ethical and legal issues. In addition, it is difficult to isolate the neural effects of psychedelic states from other physiological effects elicited by the drug ingestion. Here, we used the DeepDream algorithm to create visual stimuli that mimic the perception of hallucinatory states. Participants were first exposed to a regular video, followed by its modified version, while recording electroencephalography (EEG). Results showed that the frontal region’s activity was characterized by a higher entropy and lower complexity during the modified video, with respect to the regular one, at different time scales. Moreover, we found an increased undirected connectivity and a greater level of entropy in functional connectivity networks elicited by the modified video. These findings suggest that DeepDream and psychedelic drugs induced similar altered brain patterns and demonstrate the potential of adopting this method to study altered perceptual phenomenology in neuroimaging research.     

Complexity for Extended Dynamical Systems

We consider dynamical systems for which the spatial extension plays an important role. For these systems, the notions of attractor, ϵ-entropy and topological entropy per unit time and volume have been introduced previously. In this paper we use the notion of Kolmogorov complexity to introduce, for extended dynamical systems, a notion of complexity per unit time and volume which plays the same role as the metric entropy for classical dynamical systems. We introduce this notion as an almost sure limit on orbits of the system. Moreover we prove a kind of variational principle for this complexity.     

Cross-Subject Emotion Recognition Using Fused Entropy Features of EEG

Emotion recognition based on electroencephalography (EEG) has attracted high interest in fields such as health care, user experience evaluation, and human–computer interaction (HCI), as it plays an important role in human daily life. Although various approaches have been proposed to detect emotion states in previous studies, there is still a need to further study the dynamic changes of EEG in different emotions to detect emotion states accurately. Entropy-based features have been proved to be effective in mining the complexity information in EEG in many areas. However, different entropy features vary in revealing the implicit information of EEG. To improve system reliability, in this paper, we propose a framework for EEG-based cross-subject emotion recognition using fused entropy features and a Bidirectional Long Short-term Memory (BiLSTM) network. Features including approximate entropy (AE), fuzzy entropy (FE), Rényi entropy (RE), differential entropy (DE), and multi-scale entropy (MSE) are first calculated to study dynamic emotional information. Then, we train a BiLSTM classifier with the inputs of entropy features to identify different emotions. Our results show that MSE of EEG is more efficient than other single-entropy features in recognizing emotions. The performance of BiLSTM is further improved with an accuracy of 70.05% using fused entropy features compared with that of single-type feature.     

Weakly Regular Quantum Grammars and Asynchronous Quantum Automata

In this paper, we define weakly regular quantum grammars (WRQG), regular quantum grammars (RQG), asynchronous quantum automata (AQA) and synchronous quantum automata (SQA). Moreover, we investigate the relationships between quantum languages generated by weakly quantum regular grammars and by asynchronous quantum automata. At the mean time, we discuss the relationships between regular quantum grammars and synchronous quantum automata. This work is supported by National Science Foundation of China (Grant No. 10571112) and 973 Program of China (No. 2002CB312200).     

Machine Learning Models and Statistical Complexity to Analyze the Effects of Posture on Cerebral Hemodynamics

The mechanism of cerebral blood flow autoregulation can be of great importance in diagnosing and controlling a diversity of cerebrovascular pathologies such as vascular dementia, brain injury, and neurodegenerative diseases. To assess it, there are several methods that use changing postures, such as sit-stand or squat-stand maneuvers. However, the evaluation of the dynamic cerebral blood flow autoregulation (dCA) in these postures has not been adequately studied using more complex models, such as non-linear ones. Moreover, dCA can be considered part of a more complex mechanism called cerebral hemodynamics, where others (CO₂ reactivity and neurovascular-coupling) that affect cerebral blood flow (BF) are included. In this work, we analyzed postural influences using non-linear machine learning models of dCA and studied characteristics of cerebral hemodynamics under statistical complexity using eighteen young adult subjects, aged 27 ± 6.29 years, who took the systemic or arterial blood pressure (BP) and cerebral blood flow velocity (BFV) for five minutes in three different postures: stand, sit, and lay. With models of a Support Vector Machine (SVM) through time, we used an AutoRegulatory Index (ARI) to compare the dCA in different postures. Using wavelet entropy, we estimated the statistical complexity of BFV for three postures. Repeated measures ANOVA showed that only the complexity of lay-sit had significant differences.     

功能磁共振成像的基本尺度熵分析

人体大脑活动的复杂度随年龄变化而变化, 并且和性别有一定的联系, 通过对功能磁共振成像复杂度的分析有助于发现人脑活动和性别年龄之间关系的规律. 本文提出需要根据年龄段的变化对基本尺度熵的参数做适当的调整, 以便获得良好的信号区分效果. 本文研究了人脑活动和性别年龄之间存在的关系. 结果证明, 同龄男女的基本尺度熵值存在一定的差异, 并且随年龄段的不同发生相应的变化, 另外基本尺度熵中的参数在数据分析中也随年龄变化存在一定规律的变化. 通过对fMRI数据的分析表明, 基本尺度熵能够有效地区分不同人群fMRI数据特征, 为进一步信号复杂度分析提供方便. 关键词： 功能磁共振成像 基本尺度熵 复杂度     

Entropy-Based Classification of Elementary Cellular Automata under Asynchronous Updating: An Experimental Study

Classification of asynchronous elementary cellular automata (AECAs) was explored in the first place by Fates et al. (Complex Systems, 2004) who employed the asymptotic density of cells as a key metric to measure their robustness to stochastic transitions. Unfortunately, the asymptotic density seems unable to distinguish the robustnesses of all AECAs. In this paper, we put forward a method that goes one step further via adopting a metric entropy (Martin, Complex Systems, 2000), with the aim of measuring the asymptotic mean entropy of local pattern distribution in the cell space of any AECA. Numerical experiments demonstrate that such an entropy-based measure can actually facilitate a complete classification of the robustnesses of all AECA models, even when all local patterns are restricted to length 1. To gain more insights into the complexity concerning the forward evolution of all AECAs, we consider another entropy defined in the form of Kolmogorov–Sinai entropy and conduct preliminary experiments on classifying their uncertainties measured in terms of the proposed entropy. The results reveal that AECAs with low uncertainty tend to converge remarkably faster than models with high uncertainty.     

Practical considerations of permutation entropy

More than ten years ago Bandt and Pompe introduced a new measure to quantify complexity in measured time series. During these ten years, this measure has been modified and extended. In this review we will give a brief introduction to permutation entropy, explore the different fields of utilization where permutation entropy has been applied and provide a guide on how to choose appropriate parameters for different applications of permutation entropy.     

基于改进的符号转移熵的心脑电信号耦合研究

试图探究动力系统中的耦合关系一直以来都是国内外众多学者关注的热点,传统的时间序列符号化分析方法会使研究结果受序列非平稳性的严重影响,本文在原有转移熵的研究基础上,应用粗粒化提取,经过理论与实验的分析,发现心脑电信号耦合研究中的转移熵值在不同提取情况下对应不同的分布趋势,并选择效果最好的信号数据提取方法用在其后的应用分析中. 此外,对时间序列符号化方法提出改进,采用动态的自适应分割方法. 实验结果表明,无论清醒期还是睡眠期,改进的符号转移熵算法观测分析到的心脑电信号耦合作用更显著,能更好的捕捉到信号中的动态信息、系统动力学复杂性的改变,更利于医学临床实践应用中的检测,在分析非平稳的时间序列上具有更好的效果. 关键词： 心脑电信号 粗粒化 符号转移熵 基本尺度     

A Tsallis’ statistics based neural network model for novel word learning

We invoke the Tsallis entropy formalism, a nonextensive entropy measure, to include some degree of non-locality in a neural network that is used for simulation of novel word learning in adults. A generalization of the gradient descent dynamics, realized via nonextensive cost functions, is used as a learning rule in a simple perceptron. The model is first investigated for general properties, and then tested against the empirical data, gathered from simple memorization experiments involving two populations of linguistically different subjects. Numerical solutions of the model equations corresponded to the measured performance states of human learners. In particular, we found that the memorization tasks were executed with rather small but population-specific amounts of nonextensivity, quantified by the entropic index q. Our findings raise the possibility of using entropic nonextensivity as a means of characterizing the degree of complexity of learning in both natural and artificial systems.     

Relative Entropy of Correct Proximal Policy Optimization Algorithms with Modified Penalty Factor in Complex Environment

In the field of reinforcement learning, we propose a Correct Proximal Policy Optimization (CPPO) algorithm based on the modified penalty factor β and relative entropy in order to solve the robustness and stationarity of traditional algorithms. Firstly, In the process of reinforcement learning, this paper establishes a strategy evaluation mechanism through the policy distribution function. Secondly, the state space function is quantified by introducing entropy, whereby the approximation policy is used to approximate the real policy distribution, and the kernel function estimation and calculation of relative entropy is used to fit the reward function based on complex problem. Finally, through the comparative analysis on the classic test cases, we demonstrated that our proposed algorithm is effective, has a faster convergence speed and better performance than the traditional PPO algorithm, and the measure of the relative entropy can show the differences. In addition, it can more efficiently use the information of complex environment to learn policies. At the same time, not only can our paper explain the rationality of the policy distribution theory, the proposed framework can also balance between iteration steps, computational complexity and convergence speed, and we also introduced an effective measure of performance using the relative entropy concept.     

两种熵测度在量化射击运动员短时心率变异性信号复杂度上的一致性

利用两种基于熵的非线性复杂度测度：近似熵和样本熵,研究了专业射击运动员两种不同状态下(休息和练习赛)心率变异性信号的复杂度.计算结果表明：射击运动员休息时其心率变异性信号的熵值大于射击比赛时信号的熵值,这意味着运动员一旦进行射击比赛时,其心率变异性信号复杂度降低了,心跳变得更为规则了.为了更好地应用这两种基于熵的方法,进一步分析了算法中的两个重要影响因素：矢量匹配容差r和序列长度N对算法性能的影响.分析结果表明：只要参数选择在合适的范围内,近似熵和样本熵都能够正确地区分出两种不 关键词： 近似熵 样本熵 复杂度 射击     

Taming the Chaos in Neural Network Time Series Predictions

Machine learning methods, such as Long Short-Term Memory (LSTM) neural networks can predict real-life time series data. Here, we present a new approach to predict time series data combining interpolation techniques, randomly parameterized LSTM neural networks and measures of signal complexity, which we will refer to as complexity measures throughout this research. First, we interpolate the time series data under study. Next, we predict the time series data using an ensemble of randomly parameterized LSTM neural networks. Finally, we filter the ensemble prediction based on the original data complexity to improve the predictability, i.e., we keep only predictions with a complexity close to that of the training data. We test the proposed approach on five different univariate time series data. We use linear and fractal interpolation to increase the amount of data. We tested five different complexity measures for the ensemble filters for time series data, i.e., the Hurst exponent, Shannon’s entropy, Fisher’s information, SVD entropy, and the spectrum of Lyapunov exponents. Our results show that the interpolated predictions consistently outperformed the non-interpolated ones. The best ensemble predictions always beat a baseline prediction based on a neural network with only a single hidden LSTM, gated recurrent unit (GRU) or simple recurrent neural network (RNN) layer. The complexity filters can reduce the error of a random ensemble prediction by a factor of 10. Further, because we use randomly parameterized neural networks, no hyperparameter tuning is required. We prove this method useful for real-time time series prediction because the optimization of hyperparameters, which is usually very costly and time-intensive, can be circumvented with the presented approach.     

Randomness in denoised stock returns: The case of Moroccan family business companies

In this paper, we scrutinize entropy in family business stocks listed on Casablanca stock exchange and market index to assess randomness in their returns. For this purpose, we adopt a novel approach based on combination of stationary wavelet transform and Tsallis entropy for empirical analysis of the return series. The obtained empirical results show strong evidence that their respective entropy functions are characterized by opposite dynamics. Indeed, the information contents of their respective dynamics are statistically and significantly different. Obviously, information on regular events carried by family business returns is more certain, whilst that carried by market returns is uncertain. Such results are definitively useful to understand the nonlinear dynamics on returns on family business companies and those of the market. Without a doubt, they could be helpful for quantitative portfolio managers and investors.     

基于复杂度的自适应门限弱小目标检测方法

针对红外弱小目标检测问题,提出了一种基于图像复杂度的自适应门限目标检测方法.讨论了天空中四类不同区域的图像信息熵.图像信息熵虽然较好地表达了图像的平均信息量,但对图像的突变点不敏感.将它改进得到图像方差加权信息熵,其较好地反映了图像的复杂度特征.将图像方差加权信息熵作为图像复杂度的定量描述,用两种特定的分析模板对图像复杂度进行分析.在目标区域中,两种分析模板得到的复杂度差异较大,而非目标区域的两种复杂度则基本没有差异.算法获取两种分析模板下的复杂度图像,再对两种复杂度图像做差,得到复杂度差值图像.对差值图像建立指数模型得到自适应分割门限完成目标检测.实验结果表明,该方法对低信杂比的红外云层背景弱小目标图像具有良好的检测效果.     

An Application of Inverse Reinforcement Learning to Estimate Interference in Drone Swarms

Despite the increasing applications, demands, and capabilities of drones, in practice they have only limited autonomy for accomplishing complex missions, resulting in slow and vulnerable operations and difficulty adapting to dynamic environments. To lessen these weaknesses, we present a computational framework for deducing the original intent of drone swarms by monitoring their movements. We focus on interference, a phenomenon that is not initially anticipated by drones but results in complicated operations due to its significant impact on performance and its challenging nature. We infer interference from predictability by first applying various machine learning methods, including deep learning, and then computing entropy to compare against interference. Our computational framework begins by building a set of computational models called double transition models from the drone movements and revealing reward distributions using inverse reinforcement learning. These reward distributions are then used to compute the entropy and interference across a variety of drone scenarios specified by combining multiple combat strategies and command styles. Our analysis confirmed that drone scenarios experienced more interference, higher performance, and higher entropy as they became more heterogeneous. However, the direction of interference (positive vs. negative) was more dependent on combinations of combat strategies and command styles than homogeneity.     

Identifying Important Nodes in Complex Networks Based on Node Propagation Entropy

In recent years, the identification of the essential nodes in complex networks has attracted significant attention because of their theoretical and practical significance in many applications, such as preventing and controlling epidemic diseases and discovering essential proteins. Several importance measures have been proposed from diverse perspectives to identify crucial nodes more accurately. In this paper, we propose a novel importance metric called node propagation entropy, which uses a combination of the clustering coefficients of nodes and the influence of the first- and second-order neighbor numbers on node importance to identify essential nodes from an entropy perspective while considering the local and global information of the network. Furthermore, the susceptible–infected–removed and susceptible–infected–removed–susceptible epidemic models along with the Kendall coefficient are used to reveal the relevant correlations among the various importance measures. The results of experiments conducted on several real networks from different domains show that the proposed metric is more accurate and stable in identifying significant nodes than many existing techniques, including degree centrality, betweenness centrality, closeness centrality, eigenvector centrality, and H-index.