The Systematic Bias of Entropy Calculation in the Multi-Scale Entropy Algorithm

Entropy indicates irregularity or randomness of a dynamic system. Over the decades, entropy calculated at different scales of the system through subsampling or coarse graining has been used as a surrogate measure of system complexity. One popular multi-scale entropy analysis is the multi-scale sample entropy (MSE), which calculates entropy through the sample entropy (SampEn) formula at each time scale. SampEn is defined by the “logarithmic likelihood” that a small section (within a window of a length m) of the data “matches” with other sections will still “match” the others if the section window length increases by one. “Match” is defined by a threshold of r times standard deviation of the entire time series. A problem of current MSE algorithm is that SampEn calculations at different scales are based on the same matching threshold defined by the original time series but data standard deviation actually changes with the subsampling scales. Using a fixed threshold will automatically introduce systematic bias to the calculation results. The purpose of this paper is to mathematically present this systematic bias and to provide methods for correcting it. Our work will help the large MSE user community avoiding introducing the bias to their multi-scale SampEn calculation results.     

Effects of Tau and Sampling Frequency on the Regularity Analysis of ECG and EEG Signals Using ApEn and SampEn Entropy Estimators

Electrocardiography (ECG) and electroencephalography (EEG) signals provide clinical information relevant to determine a patient’s health status. The nonlinear analysis of ECG and EEG signals allows for discovering characteristics that could not be found with traditional methods based on amplitude and frequency. Approximate entropy (ApEn) and sampling entropy (SampEn) are nonlinear data analysis algorithms that measure the data’s regularity, and these are used to classify different electrophysiological signals as normal or pathological. Entropy calculation requires setting the parameters r (tolerance threshold), m (immersion dimension), and τ (time delay), with the last one being related to how the time series is downsampled. In this study, we showed the dependence of ApEn and SampEn on different values of τ, for ECG and EEG signals with different sampling frequencies (F_s), extracted from a digital repository. We considered four values of F_s (128, 256, 384, and 512 Hz for the ECG signals, and 160, 320, 480, and 640 Hz for the EEG signals) and five values of τ (from 1 to 5). We performed parametric and nonparametric statistical tests to confirm that the groups of normal and pathological ECG and EEG signals were significantly different (p < 0.05) for each F and τ value. The separation between the entropy values of regular and irregular signals was variable, demonstrating the dependence of ApEn and SampEn with F_s and τ. For ECG signals, the separation between the conditions was more robust when using SampEn, the lowest value of F_s, and τ larger than 1. For EEG signals, the separation between the conditions was more robust when using SampEn with large values of F_s and τ larger than 1. Therefore, adjusting τ may be convenient for signals that were acquired with different F_s to ensure a reliable clinical classification. Furthermore, it is useful to set τ to values larger than 1 to reduce the computational cost.     

Entropy Profiling: A Reduced—Parametric Measure of Kolmogorov—Sinai Entropy from Short-Term HRV Signal

Entropy profiling is a recently introduced approach that reduces parametric dependence in traditional Kolmogorov-Sinai (KS) entropy measurement algorithms. The choice of the threshold parameter r of vector distances in traditional entropy computations is crucial in deciding the accuracy of signal irregularity information retrieved by these methods. In addition to making parametric choices completely data-driven, entropy profiling generates a complete profile of entropy information as against a single entropy estimate (seen in traditional algorithms). The benefits of using “profiling” instead of “estimation” are: (a) precursory methods such as approximate and sample entropy that have had the limitation of handling short-term signals (less than 1000 samples) are now made capable of the same; (b) the entropy measure can capture complexity information from short and long-term signals without multi-scaling; and (c) this new approach facilitates enhanced information retrieval from short-term HRV signals. The novel concept of entropy profiling has greatly equipped traditional algorithms to overcome existing limitations and broaden applicability in the field of short-term signal analysis. In this work, we present a review of KS-entropy methods and their limitations in the context of short-term heart rate variability analysis and elucidate the benefits of using entropy profiling as an alternative for the same.     

基于模糊近似熵的抑郁症患者静息态功能磁共振成像信号复杂度分析

提出采用模糊近似熵的方法对功能磁共振成像(functional magnetic resonance imaging,fMRI)复杂度量化分析,并与样本熵进行比较.采用的22个成年抑郁症患者中,11位男性,年龄在18—65岁之间.我们期望测量的静息态fMRI信号复杂度与Goldberger/Lipsitz模型一致,越健康、越稳健其生理表现的复杂度越大,且复杂度随年龄的增大而降低.全脑平均模糊近似熵与年龄之间差异性显著(r=-0.512,p0.001).相比之下,样本熵与年龄之间差异性不显著(r=-0.102,p=0.482).模糊近似熵同样与年龄相关脑区(额叶、顶叶、边缘系统、颞叶、小脑顶叶)之间差异性显著(p0.05),样本熵与年龄相关脑区之间差异性不显著性.这些结果与Goldberger/Lipsitz模型一致,说明采用模糊近似熵分析fMRI数据复杂度是一个有效的新方法.     

The Impact of the COVID-19 Pandemic on the Unpredictable Dynamics of the Cryptocurrency Market

The global economy is under great shock again in 2020 due to the COVID-19 pandemic; it has not been long since the global financial crisis in 2008. Therefore, we investigate the evolution of the complexity of the cryptocurrency market and analyze the characteristics from the past bull market in 2017 to the present the COVID-19 pandemic. To confirm the evolutionary complexity of the cryptocurrency market, three general complexity analyses based on nonlinear measures were used: approximate entropy (ApEn), sample entropy (SampEn), and Lempel-Ziv complexity (LZ). We analyzed the market complexity/unpredictability for 43 cryptocurrency prices that have been trading until recently. In addition, three non-parametric tests suitable for non-normal distribution comparison were used to cross-check quantitatively. Finally, using the sliding time window analysis, we observed the change in the complexity of the cryptocurrency market according to events such as the COVID-19 pandemic and vaccination. This study is the first to confirm the complexity/unpredictability of the cryptocurrency market from the bull market to the COVID-19 pandemic outbreak. We find that ApEn, SampEn, and LZ complexity metrics of all markets could not generalize the COVID-19 effect of the complexity due to different patterns. However, market unpredictability is increasing by the ongoing health crisis.     

Novel Features for Binary Time Series Based on Branch Length Similarity Entropy

Branch length similarity (BLS) entropy is defined in a network consisting of a single node and branches. In this study, we mapped the binary time-series signal to the circumference of the time circle so that the BLS entropy can be calculated for the binary time-series. We obtained the BLS entropy values for “1” signals on the time circle. The set of values are the BLS entropy profile. We selected the local maximum (minimum) point, slope, and inflection point of the entropy profile as the characteristic features of the binary time-series and investigated and explored their significance. The local maximum (minimum) point indicates the time at which the rate of change in the signal density becomes zero. The slope and inflection points correspond to the degree of change in the signal density and the time at which the signal density changes occur, respectively. Moreover, we show that the characteristic features can be widely used in binary time-series analysis by characterizing the movement trajectory of Caenorhabditis elegans. We also mention the problems that need to be explored mathematically in relation to the features and propose candidates for additional features based on the BLS entropy profile.     

Analysis on the Development of Digital Economy in Guangdong Province Based on Improved Entropy Method and Multivariate Statistical Analysis

The lack of adequate indicators in the research of digital economy may lead to the shortage of data support on decision making for governments. To solve this problem, first we establish a digital economy indicator evaluation system by dividing the digital economy into four types: “basic type”, “technology type”, “integration type” and “service type” and select 5 indicators for each type. On this basis, the weight of each indicator is calculated to find the deficiencies in the development of some digital economic fields by the improved entropy method. By drawing on the empowerment idea of Analytic Hierarchy Process, the improved entropy method firstly compares the difference coefficient of indicators in pairs and maps the comparison results to the scales 1–9. Then, the judgment matrix is constructed based on the information entropy, which can solve as much as possible the problem that the difference among the weight of each indicator is too large in traditional entropy method. The results indicate that: the development of digital economy in Guangdong Province was relatively balanced from 2015 to 2018 and will be better in the future while the development of rural e-commerce in Guangdong Province is relatively backward, and there is an obvious digital gap between urban and rural areas. Next we extract two new variables respectively to replace the 20 indicators we select through principal component analysis and factor analysis methods in multivariate statistical analysis, which can retain the original information to the greatest extent and provide convenience for further research in the future. Finally, we and provide constructive comments of digital economy in Guangdong Province from 2015 to 2018.     

Bell Diagonal and Werner State Generation: Entanglement,Non-Locality,Steering and Discord on the IBM Quantum Computer

We propose the first correct special-purpose quantum circuits for preparation of Bell diagonal states (BDS), and implement them on the IBM Quantum computer, characterizing and testing complex aspects of their quantum correlations in the full parameter space. Among the circuits proposed, one involves only two quantum bits but requires adapted quantum tomography routines handling classical bits in parallel. The entire class of Bell diagonal states is generated, and several characteristic indicators, namely entanglement of formation and concurrence, CHSH non-locality, steering and discord, are experimentally evaluated over the full parameter space and compared with theory. As a by-product of this work, we also find a remarkable general inequality between “quantum discord” and “asymmetric relative entropy of discord”: the former never exceeds the latter. We also prove that for all BDS the two coincide.     

Information and Self-Organization II: Steady State and Phase Transition

This paper starts from Schrödinger’s famous question “what is life” and elucidates answers that invoke, in particular, Friston’s free energy principle and its relation to the method of Bayesian inference and to Synergetics 2nd foundation that utilizes Jaynes’ maximum entropy principle. Our presentation reflects the shift from the emphasis on physical principles to principles of information theory and Synergetics. In view of the expected general audience of this issue, we have chosen a somewhat tutorial style that does not require special knowledge on physics but familiarizes the reader with concepts rooted in information theory and Synergetics.     

Using the modified sample entropy to detect determinism

A modified sample entropy (mSampEn), based on the nonlinear continuous and convex function, has been proposed and proven to be superior to the standard sample entropy (SampEn) in several aspects. In this Letter, we empirically investigate the ability of the mSampEn statistic combined with surrogate data method to detect determinism. The effects of the datasets length and noise on the proposed method to differentiate between deterministic and stochastic dynamics are tested on several benchmark time series. The noise performance of the mSampEn statistic is also compared with the singular value decomposition (SVD) and symplectic geometry spectrum (SGS) based methods. The results indicate that the mSampEn statistic is a robust index for detecting determinism in short and noisy time series.     

Approximate Entropy of Brain Network in the Study of Hemispheric Differences

Human brain, a dynamic complex system, can be studied with different approaches, including linear and nonlinear ones. One of the nonlinear approaches widely used in electroencephalographic (EEG) analyses is the entropy, the measurement of disorder in a system. The present study investigates brain networks applying approximate entropy (ApEn) measure for assessing the hemispheric EEG differences; reproducibility and stability of ApEn data across separate recording sessions were evaluated. Twenty healthy adult volunteers were submitted to eyes-closed resting EEG recordings, for 80 recordings. Significant differences in the occipital region, with higher values of entropy in the left hemisphere than in the right one, show that the hemispheres become active with different intensities according to the performed function. Besides, the present methodology proved to be reproducible and stable, when carried out on relatively brief EEG epochs but also at a 1-week distance in a group of 36 subjects. Nonlinear approaches represent an interesting probe to study the dynamics of brain networks. ApEn technique might provide more insight into the pathophysiological processes underlying age-related brain disconnection as well as for monitoring the impact of pharmacological and rehabilitation treatments.     

Generalized Ordinal Patterns and the KS-Entropy

Ordinal patterns classifying real vectors according to the order relations between their components are an interesting basic concept for determining the complexity of a measure-preserving dynamical system. In particular, as shown by C. Bandt, G. Keller and B. Pompe, the permutation entropy based on the probability distributions of such patterns is equal to Kolmogorov–Sinai entropy in simple one-dimensional systems. The general reason for this is that, roughly speaking, the system of ordinal patterns obtained for a real-valued “measuring arrangement” has high potential for separating orbits. Starting from a slightly different approach of A. Antoniouk, K. Keller and S. Maksymenko, we discuss the generalizations of ordinal patterns providing enough separation to determine the Kolmogorov–Sinai entropy. For defining these generalized ordinal patterns, the idea is to substitute the basic binary relation ≤ on the real numbers by another binary relation. Generalizing the former results of I. Stolz and K. Keller, we establish conditions that the binary relation and the dynamical system have to fulfill so that the obtained generalized ordinal patterns can be used for estimating the Kolmogorov–Sinai entropy.     

From Rényi Entropy Power to Information Scan of Quantum States

In this paper, we generalize the notion of Shannon’s entropy power to the Rényi-entropy setting. With this, we propose generalizations of the de Bruijn identity, isoperimetric inequality, or Stam inequality. This framework not only allows for finding new estimation inequalities, but it also provides a convenient technical framework for the derivation of a one-parameter family of Rényi-entropy-power-based quantum-mechanical uncertainty relations. To illustrate the usefulness of the Rényi entropy power obtained, we show how the information probability distribution associated with a quantum state can be reconstructed in a process that is akin to quantum-state tomography. We illustrate the inner workings of this with the so-called “cat states”, which are of fundamental interest and practical use in schemes such as quantum metrology. Salient issues, including the extension of the notion of entropy power to Tsallis entropy and ensuing implications in estimation theory, are also briefly discussed.     

Effects of back extensor strengthening exercises on postural alignment,physical function and performance,self-efficacy,and quality of life in Japanese community-dwelling older adults: A controlled clinical trial

Objective: To examine the changes in postural alignment and kyphosis-correlated factors after 6 months of back extensor strengthening exercise in a group of community-dwelling older adults aged ≥65 years. Methods: We quasi-randomized 29 subjects into an intervention group treated with a back extensor strengthening program and a control group treated with a full-body exercise program. These groups completed 20-30 minutes of exercise directed by a physical therapist one or more times per week and were instructed to exercise at home as well. The participants were assessed prior to and after the intervention using the following criteria: postural alignment of “usual” and “best” posture, physical function, physical performance, self-efficacy, and quality of life. The differences between two factors (group and period) were compared for each of the measurement variables. Results: Subjects who adequately completed the exercises were analyzed. A reduced knee flexion angle was noted in the “best” posture of both groups, as were improved physical function and performance with the exception of one-leg standing time. Verifying the effect size in the post-hoc analysis, the body parts that showed changes to postural alignment after the intervention differed between groups. Conclusions: Back extensor strengthening exercises improved physical function and performance, but did not improve spinal alignment. The changes due to these interventions were not significantly different from changes observed in the full-body exercise group. However, post-hoc analysis revealed that the effect size of posture change was different, possible indicating that the two groups experienced different changes in the postural alignment.     

Influence of Ectopic Beats on Heart Rate Variability Analysis

The analysis of heart rate variability (HRV) plays a dominant role in the study of physiological signal variability. HRV reflects the information of the adjustment of sympathetic and parasympathetic nerves on the cardiovascular system and, thus, is widely used to evaluate the functional status of the cardiovascular system. Ectopic beats may affect the analysis of HRV. However, the quantitative relationship between the burden of ectopic beats and HRV indices, including entropy measures, has not yet been investigated in depth. In this work, we analyzed the effects of different numbers of ectopic beats on several widely accepted HRV parameters in time-domain (SDNN), frequency-domain (LF/HF), as well as non-linear features (SampEn and Pt-SampEn (physical threshold-based SampEn)). The results showed that all four indices were influenced by ectopic beats, and the degree of influence was roughly increased with the increase of the number of ectopic beats. Ectopic beats had the greatest impact on the frequency domain index LF/HF, whereas the Pt-SampEn was minimally accepted by ectopic beats. These results also indicated that, compared with the other three indices, Pt-SampEn had better robustness for ectopic beats.     

A New Family of Continuous Probability Distributions

In this paper, a new parametric compound G family of continuous probability distributions called the Poisson generalized exponential G (PGEG) family is derived and studied. Relevant mathematical properties are derived. Some new bivariate G families using the theorems of “Farlie-Gumbel-Morgenstern copula”, “the modified Farlie-Gumbel-Morgenstern copula”, “the Clayton copula”, and “the Renyi’s entropy copula” are presented. Many special members are derived, and a special attention is devoted to the exponential and the one parameter Pareto type II model. The maximum likelihood method is used to estimate the model parameters. A graphical simulation is performed to assess the finite sample behavior of the estimators of the maximum likelihood method. Two real-life data applications are proposed to illustrate the importance of the new family.     

From Continuous-Time Chaotic Systems to Pseudo Random Number Generators: Analysis and Generalized Methodology

The use of chaotic systems in electronics, such as Pseudo-Random Number Generators (PRNGs), is very appealing. Among them, continuous-time ones are used less because, in addition to having strong temporal correlations, they require further computations to obtain the discrete solutions. Here, the time step and discretization method selection are first studied by conducting a detailed analysis of their effect on the systems’ statistical and chaotic behavior. We employ an approach based on interpreting the time step as a parameter of the new “maps”. From our analysis, it follows that to use them as PRNGs, two actions should be achieved (i) to keep the chaotic oscillation and (ii) to destroy the inner and temporal correlations. We then propose a simple methodology to achieve chaos-based PRNGs with good statistical characteristics and high throughput, which can be applied to any continuous-time chaotic system. We analyze the generated sequences by means of quantifiers based on information theory (permutation entropy, permutation complexity, and causal entropy × complexity plane). We show that the proposed PRNG generates sequences that successfully pass Marsaglia Diehard and NIST (National Institute of Standards and Technology) tests. Finally, we show that its hardware implementation requires very few resources.     

基于近似熵测度的铝合金P-MIG焊亚射流过渡自适应控制研究

铝合金熔化极惰性气体保护焊(MIG)的亚射流过渡方式具有弧长调节作用强、焊缝成形美观的优点.本文从非线性动力学近似熵测度的角度对铝合金脉冲MIG焊的电弧电压信号进行分析,进而评价自适应控制模式针对亚射流过渡方式的适用性;讨论在自适应控制模式下,改变工艺参数和熔滴过渡方式时,近似熵值的变化;并对比自适应控制与普通脉冲控制在射滴过渡方式下的近似熵值大小.结果表明近似熵越小,自适应控制越成功,控制越有效. 关键词： 自适应控制 亚射流过渡 熔化极惰性气体保护焊(MIG) 近似熵     

Order-Stability in Complex Biological,Social, and AI-Systems from Quantum Information Theory

This paper is our attempt, on the basis of physical theory, to bring more clarification on the question “What is life?” formulated in the well-known book of Schrödinger in 1944. According to Schrödinger, the main distinguishing feature of a biosystem’s functioning is the ability to preserve its order structure or, in mathematical terms, to prevent increasing of entropy. However, Schrödinger’s analysis shows that the classical theory is not able to adequately describe the order-stability in a biosystem. Schrödinger also appealed to the ambiguous notion of negative entropy. We apply quantum theory. As is well-known, behaviour of the quantum von Neumann entropy crucially differs from behaviour of classical entropy. We consider a complex biosystem S composed of many subsystems, say proteins, cells, or neural networks in the brain, that is, S=(Si). We study the following problem: whether the compound system S can maintain “global order” in the situation of an increase of local disorder and if S can preserve the low entropy while other Si increase their entropies (may be essentially). We show that the entropy of a system as a whole can be constant, while the entropies of its parts rising. For classical systems, this is impossible, because the entropy of S cannot be less than the entropy of its subsystem Si. And if a subsystems’s entropy increases, then a system’s entropy should also increase, by at least the same amount. However, within the quantum information theory, the answer is positive. The significant role is played by the entanglement of a subsystems’ states. In the absence of entanglement, the increasing of local disorder implies an increasing disorder in the compound system S (as in the classical regime). In this note, we proceed within a quantum-like approach to mathematical modeling of information processing by biosystems—respecting the quantum laws need not be based on genuine quantum physical processes in biosystems. Recently, such modeling found numerous applications in molecular biology, genetics, evolution theory, cognition, psychology and decision making. The quantum-like model of order stability can be applied not only in biology, but also in social science and artificial intelligence.