Bivariate Entropy Analysis of Electrocardiographic RR–QT Time Series

QT interval variability (QTV) and heart rate variability (HRV) are both accepted biomarkers for cardiovascular events. QTV characterizes the variations in ventricular depolarization and repolarization. It is a predominant element of HRV. However, QTV is also believed to accept direct inputs from upstream control system. How QTV varies along with HRV is yet to be elucidated. We studied the dynamic relationship of QTV and HRV during different physiological conditions from resting, to cycling, and to recovering. We applied several entropy-based measures to examine their bivariate relationships, including cross sample entropy (XSampEn), cross fuzzy entropy (XFuzzyEn), cross conditional entropy (XCE), and joint distribution entropy (JDistEn). Results showed no statistically significant differences in XSampEn, XFuzzyEn, and XCE across different physiological states. Interestingly, JDistEn demonstrated significant decreases during cycling as compared with that during the resting state. Besides, JDistEn also showed a progressively recovering trend from cycling to the first 3 min during recovering, and further to the second 3 min during recovering. It appeared to be fully recovered to its level in the resting state during the second 3 min during the recovering phase. The results suggest that there is certain nonlinear temporal relationship between QTV and HRV, and that the JDistEn could help unravel this nuanced property.     

Joint Early Stopping Criterions for Protograph LDPC Codes-Based JSCC System in Images Transmission

In this paper, a joint early stopping criterion based on cross entropy (CE), named joint CE criterion, is presented for double-protograph low-density parity-check (DP-LDPC) codes-based joint source-channel coding (JSCC) systems in images transmission to reduce the decoding complexity and decrease the decoding delay. The proposed early stopping criterion adopts the CE from the output likelihood ratios (LLRs) of the joint decoder. Moreover, a special phenomenon named asymmetry oscillation-like convergence (AOLC) in the changing process of CE is uncovered in the source decoder and channel decoder of this system meanwhile, and the proposed joint CE criterion can reduce the impact from the AOLC phenomenon. Comparing to the counterparts, the results show that the joint CE criterion can perform well in the decoding complexity and decoding latency in the low–moderate signal-to-noise ratio (SNR) region and achieve performance improvement in the high SNR region with appropriate parameters, which also demonstrates that this system with joint CE is a low-latency and low-power system.     

Assessing the Europe 2020 Strategy Implementation Using Interval Entropy and Cluster Analysis for Interrelation between Two Groups of Headline Indicators

The research analyzes the progress of Member States in the implementation of Europe 2020 strategy targets and goals in 2016–2018. Multiple criteria decision-making approaches applied for this task. The set of headline indicators was divided into two logically explained groups. Interval entropy is proposed as an effective tool to make prioritization of headline indicators in separate groups. The sensitivity of the interval entropy is its advantage over classical entropy. Indicator weights were calculated by applying the WEBIRA (weight-balancing indicator ranks accordance) method. The WEBIRA method allows the best harmonization of ranking results according to different criteria groups—this is its advantage over other multiple-criteria methods. Final assessing and ranking of the 28 European Union countries (EU-28) was implemented through the α-cut approach. A k-means clustering procedure was applied to the EU-28 countries by summarizing the ranking results in 2016–2018. Investigation revealed the countries–leaders and countries–outsiders of the Europe 2020 strategy implementation process. It turned out that Sweden, Finland, Denmark, and Austria during the three-year period were the countries that exhibited the greatest progress according to two headline indicator groups’ interrelation. Cluster analysis results are mainly consistent with the EU-28 countries’ categorizations set by other authors.     

Detection of Coronary Artery Disease Using Multi-Domain Feature Fusion of Multi-Channel Heart Sound Signals

Heart sound signals reflect valuable information about heart condition. Previous studies have suggested that the information contained in single-channel heart sound signals can be used to detect coronary artery disease (CAD). But accuracy based on single-channel heart sound signal is not satisfactory. This paper proposed a method based on multi-domain feature fusion of multi-channel heart sound signals, in which entropy features and cross entropy features are also included. A total of 36 subjects enrolled in the data collection, including 21 CAD patients and 15 non-CAD subjects. For each subject, five-channel heart sound signals were recorded synchronously for 5 min. After data segmentation and quality evaluation, 553 samples were left in the CAD group and 438 samples in the non-CAD group. The time-domain, frequency-domain, entropy, and cross entropy features were extracted. After feature selection, the optimal feature set was fed into the support vector machine for classification. The results showed that from single-channel to multi-channel, the classification accuracy has increased from 78.75% to 86.70%. After adding entropy features and cross entropy features, the classification accuracy continued to increase to 90.92%. The study indicated that the method based on multi-domain feature fusion of multi-channel heart sound signals could provide more information for CAD detection, and entropy features and cross entropy features played an important role in it.     

The Value of Heart Rhythm Complexity in Identifying High-Risk Pulmonary Hypertension Patients

Pulmonary hypertension (PH) is a fatal disease—even with state-of-the-art medical treatment. Non-invasive clinical tools for risk stratification are still lacking. The aim of this study was to investigate the clinical utility of heart rhythm complexity in risk stratification for PH patients. We prospectively enrolled 54 PH patients, including 20 high-risk patients (group A; defined as WHO functional class IV or class III with severely compromised hemodynamics), and 34 low-risk patients (group B). Both linear and non-linear heart rate variability (HRV) variables, including detrended fluctuation analysis (DFA) and multiscale entropy (MSE), were analyzed. In linear and non-linear HRV analysis, low frequency and high frequency ratio, DFAα1, MSE slope 5, scale 5, and area 6–20 were significantly lower in group A. Among all HRV variables, MSE scale 5 (AUC: 0.758) had the best predictive power to discriminate the two groups. In multivariable analysis, MSE scale 5 (p = 0.010) was the only significantly predictor of severe PH in all HRV variables. In conclusion, the patients with severe PH had worse heart rhythm complexity. MSE parameters, especially scale 5, can help to identify high-risk PH patients.     

Estimating Phase Amplitude Coupling between Neural Oscillations Based on Permutation and Entropy

Cross-frequency phase–amplitude coupling (PAC) plays an important role in neuronal oscillations network, reflecting the interaction between the phase of low-frequency oscillation (LFO) and amplitude of the high-frequency oscillations (HFO). Thus, we applied four methods based on permutation analysis to measure PAC, including multiscale permutation mutual information (MPMI), permutation conditional mutual information (PCMI), symbolic joint entropy (SJE), and weighted-permutation mutual information (WPMI). To verify the ability of these four algorithms, a performance test including the effects of coupling strength, signal-to-noise ratios (SNRs), and data length was evaluated by using simulation data. It was shown that the performance of SJE was similar to that of other approaches when measuring PAC strength, but the computational efficiency of SJE was the highest among all these four methods. Moreover, SJE can also accurately identify the PAC frequency range under the interference of spike noise. All in all, the results demonstrate that SJE is better for evaluating PAC between neural oscillations.     

Entropy-Based Behavioural Efficiency of the Financial Market

The most known and used abstract model of the financial market is based on the concept of the informational efficiency (EMH) of that market. The paper proposes an alternative which could be named the behavioural efficiency of the financial market, which is based on the behavioural entropy instead of the informational entropy. More specifically, the paper supports the idea that, in the financial market, the only measure (if any) of the entropy is the available behaviours indicated by the implicit information. Therefore, the behavioural entropy is linked to the concept of behavioural efficiency. The paper argues that, in fact, in the financial markets, there is not a (real) informational efficiency, but there exists a behavioural efficiency instead. The proposal is based both on a new typology of information in the financial market (which provides the concept of implicit information—that is, that information ”translated” by the economic agents from observing the actual behaviours) and on a non-linear (more exactly, a logistic) curve linking the behavioural entropy to the behavioural efficiency of the financial markets. Finally, the paper proposes a synergic overcoming of both EMH and AMH based on the new concept of behavioural entropy in the financial market.     

Multiscale Entropy Analysis of Short Signals: The Robustness of Fuzzy Entropy-Based Variants Compared to Full-Length Long Signals

Multiscale entropy (MSE) analysis is a fundamental approach to access the complexity of a time series by estimating its information creation over a range of temporal scales. However, MSE may not be accurate or valid for short time series. This is why previous studies applied different kinds of algorithm derivations to short-term time series. However, no study has systematically analyzed and compared their reliabilities. This study compares the MSE algorithm variations adapted to short time series on both human and rat heart rate variability (HRV) time series using long-term MSE as reference. The most used variations of MSE are studied: composite MSE (CMSE), refined composite MSE (RCMSE), modified MSE (MMSE), and their fuzzy versions. We also analyze the errors in MSE estimations for a range of incorporated fuzzy exponents. The results show that fuzzy MSE versions—as a function of time series length—present minimal errors compared to the non-fuzzy algorithms. The traditional multiscale entropy algorithm with fuzzy counting (MFE) has similar accuracy to alternative algorithms with better computing performance. For the best accuracy, the findings suggest different fuzzy exponents according to the time series length.     

Entropy,Information, and the Updating of Probabilities

This paper is a review of a particular approach to the method of maximum entropy as a general framework for inference. The discussion emphasizes pragmatic elements in the derivation. An epistemic notion of information is defined in terms of its relation to the Bayesian beliefs of ideally rational agents. The method of updating from a prior to posterior probability distribution is designed through an eliminative induction process. The logarithmic relative entropy is singled out as a unique tool for updating (a) that is of universal applicability, (b) that recognizes the value of prior information, and (c) that recognizes the privileged role played by the notion of independence in science. The resulting framework—the ME method—can handle arbitrary priors and arbitrary constraints. It includes the MaxEnt and Bayes’ rules as special cases and, therefore, unifies entropic and Bayesian methods into a single general inference scheme. The ME method goes beyond the mere selection of a single posterior, and also addresses the question of how much less probable other distributions might be, which provides a direct bridge to the theories of fluctuations and large deviations.     

A Novel Blind Signal Detector Based on the Entropy of the Power Spectrum Subband Energy Ratio

In this paper, we present a novel blind signal detector based on the entropy of the power spectrum subband energy ratio (PSER), the detection performance of which is significantly better than that of the classical energy detector. This detector is a full power spectrum detection method, and does not require the noise variance or prior information about the signal to be detected. According to the analysis of the statistical characteristics of the power spectrum subband energy ratio, this paper proposes concepts such as interval probability, interval entropy, sample entropy, joint interval entropy, PSER entropy, and sample entropy variance. Based on the multinomial distribution, in this paper the formulas for calculating the PSER entropy and the variance of sample entropy in the case of pure noise are derived. Based on the mixture multinomial distribution, the formulas for calculating the PSER entropy and the variance of sample entropy in the case of the signals mixed with noise are also derived. Under the constant false alarm strategy, the detector based on the entropy of the power spectrum subband energy ratio is derived. The experimental results for the primary signal detection are consistent with the theoretical calculation results, which proves that the detection method is correct.     

Examining the Research on Business Information-Entropy Correlation in the Accounting Process of Organizations

Open business organizations, where information flows, is shared, and exchanged, are more prepared to adapt and survive chaos, uncertainty, and entropy, so they will be more predisposed to change management. The aim of this study is to analyze research trends at the international level on business information–entropy correlation in the accounting process of organizations. Mathematical and statistical techniques were applied to 980 articles during the period 1974–2020, obtaining results on the scientific productivity of the driving agents of this topic: authors, research institutions, countries/territories, and journals. Five lines of research were identified during the period analyzed, which mainly study information theory, maximum entropy, information entropy, decision-making, and enthalpy. Future research should focus on analyzing the evolution of this topic, which forms new thematic axes related to bitcoin market efficiency, business hierarchy information, business model evaluation systems, catastrophic economic collapse, corporate diversification, CSR reports affecting accounting conservatism, economic income accounting, and information loss. Currently, the research presents an upward trend, which allows a growing interest in the subject to be deduced in the academic and scientific community worldwide.     

A Thermodynamic Approach to Measuring Entropy in a Few-Electron Nanodevice

The entropy of a system gives a powerful insight into its microscopic degrees of freedom; however, standard experimental ways of measuring entropy through heat capacity are hard to apply to nanoscale systems, as they require the measurement of increasingly small amounts of heat. Two alternative entropy measurement methods have been recently proposed for nanodevices: through charge balance measurements and transport properties. We describe a self-consistent thermodynamic framework for applying thermodynamic relations to few-electron nanodevices—small systems, where fluctuations in particle number are significant, whilst highlighting several ongoing misconceptions. We derive a relation (a consequence of a Maxwell relation for small systems), which describes both existing entropy measurement methods as special cases, while also allowing the experimentalist to probe the intermediate regime between them. Finally, we independently prove the applicability of our framework in systems with complex microscopic dynamics—those with many excited states of various degeneracies—from microscopic considerations.     

Stochastic and Self-Organisation Patterns in a 17-Year PM10 Time Series in Athens,Greece

This paper utilises statistical and entropy methods for the investigation of a 17-year PM₁₀ time series recorded from five stations in Athens, Greece, in order to delineate existing stochastic and self-organisation trends. Stochastic patterns are analysed via lumping and sliding, in windows of various lengths. Decreasing trends are found between Windows 1 and 3500–4000, for all stations. Self-organisation is studied through Boltzmann and Tsallis entropy via sliding and symbolic dynamics in selected parts. Several values are below −2 (Boltzmann entropy) and 1.18 (Tsallis entropy) over the Boltzmann constant. A published method is utilised to locate areas for which the PM₁₀ system is out of stochastic behaviour and, simultaneously, exhibits critical self-organised tendencies. Sixty-six two-month windows are found for various dates. From these, nine are common to at least three different stations. Combining previous publications, two areas are non-stochastic and exhibit, simultaneously, fractal, long-memory and self-organisation patterns through a combination of 15 different fractal and SOC analysis techniques. In these areas, block-entropy (range 0.650–2.924) is significantly lower compared to the remaining areas of non-stochastic but self-organisation trends. It is the first time to utilise entropy analysis for PM₁₀ series and, importantly, in combination with results from previously published fractal methods.Data Set License: license under which the dataset is made available (CC0, CC-BY, CC-BY-SA, CC-BY-NC, etc.)     

Entropic Uncertainty for Two Coupled Dipole Spins Using Quantum Memory under the Dzyaloshinskii–Moriya Interaction

In the thermodynamic equilibrium of dipolar-coupled spin systems under the influence of a Dzyaloshinskii–Moriya (D–M) interaction along the z-axis, the current study explores the quantum-memory-assisted entropic uncertainty relation (QMA-EUR), entropy mixedness and the concurrence two-spin entanglement. Quantum entanglement is reduced at increased temperature values, but inflation uncertainty and mixedness are enhanced. The considered quantum effects are stabilized to their stationary values at high temperatures. The two-spin entanglement is entirely repressed if the D–M interaction is disregarded, and the entropic uncertainty and entropy mixedness reach their maximum values for equal coupling rates. Rather than the concurrence, the entropy mixedness can be a proper indicator of the nature of the entropic uncertainty. The effect of model parameters (D–M coupling and dipole–dipole spin) on the quantum dynamic effects in thermal environment temperature is explored. The results reveal that the model parameters cause significant variations in the predicted QMA-EUR.     

Generalized Ordinal Patterns and the KS-Entropy

Ordinal patterns classifying real vectors according to the order relations between their components are an interesting basic concept for determining the complexity of a measure-preserving dynamical system. In particular, as shown by C. Bandt, G. Keller and B. Pompe, the permutation entropy based on the probability distributions of such patterns is equal to Kolmogorov–Sinai entropy in simple one-dimensional systems. The general reason for this is that, roughly speaking, the system of ordinal patterns obtained for a real-valued “measuring arrangement” has high potential for separating orbits. Starting from a slightly different approach of A. Antoniouk, K. Keller and S. Maksymenko, we discuss the generalizations of ordinal patterns providing enough separation to determine the Kolmogorov–Sinai entropy. For defining these generalized ordinal patterns, the idea is to substitute the basic binary relation ≤ on the real numbers by another binary relation. Generalizing the former results of I. Stolz and K. Keller, we establish conditions that the binary relation and the dynamical system have to fulfill so that the obtained generalized ordinal patterns can be used for estimating the Kolmogorov–Sinai entropy.     

Working Memory Decline in Alzheimer’s Disease Is Detected by Complexity Analysis of Multimodal EEG-fNIRS

Alzheimer’s disease (AD) is characterized by working memory (WM) failures that can be assessed at early stages through administering clinical tests. Ecological neuroimaging, such as Electroencephalography (EEG) and functional Near Infrared Spectroscopy (fNIRS), may be employed during these tests to support AD early diagnosis within clinical settings. Multimodal EEG-fNIRS could measure brain activity along with neurovascular coupling (NC) and detect their modifications associated with AD. Data analysis procedures based on signal complexity are suitable to estimate electrical and hemodynamic brain activity or their mutual information (NC) during non-structured experimental paradigms. In this study, sample entropy of whole-head EEG and frontal/prefrontal cortex fNIRS was evaluated to assess brain activity in early AD and healthy controls (HC) during WM tasks (i.e., Rey–Osterrieth complex figure and Raven’s progressive matrices). Moreover, conditional entropy between EEG and fNIRS was evaluated as indicative of NC. The findings demonstrated the capability of complexity analysis of multimodal EEG-fNIRS to detect WM decline in AD. Furthermore, a multivariate data-driven analysis, performed on these entropy metrics and based on the General Linear Model, allowed classifying AD and HC with an AUC up to 0.88. EEG-fNIRS may represent a powerful tool for the clinical evaluation of WM decline in early AD.     

基于脉冲耦合神经网络和图像熵的各向异性扩散模型研究

在图像去噪过程中, 大部分基于偏微分方程的各向异性扩散模型均使用梯度信息检测边缘, 当边缘部分被噪声严重污染时, 这些方法不能有效检测出这些边缘, 因而无法保留边缘特征. 为了较完整的保留图像的区域信息, 用脉冲耦合神经网络(PCNN)能使具有相似输入的神经元同时产生脉冲的性质对噪声图像做处理, 得到图像熵序列, 并将图像熵序列作为边缘检测算子引入到扩散方程中, 不仅能克服仅用梯度作为边缘检测算子易受噪声影响的弊端, 而且能较完整地保留图像的区域信息. 然后, 用最小交叉熵准则搜索使去噪前后图像信息量差异最小的阈值, 设计最佳阈值控制扩散强度, 建立基于脉冲耦合神经网络与图像熵改进的各向异性扩散模型(PCNN-IEAD). 分析与仿真结果表明, 该模型与经典模型相比, 保留了更多的图像信息, 能够兼顾去除图像的噪声和保护图像的边缘纹理等细节信息, 较完整的保留了图像的区域信息, 性能指标同样也证实了新模型的优越性. 另外, 该模型的运行时间较经典模型的短, 因此, 该模型是一个理想的模型.     

Alternative Entropy Measures and Generalized Khinchin–Shannon Inequalities

The Khinchin–Shannon generalized inequalities for entropy measures in Information Theory, are a paradigm which can be used to test the Synergy of the distributions of probabilities of occurrence in physical systems. The rich algebraic structure associated with the introduction of escort probabilities seems to be essential for deriving these inequalities for the two-parameter Sharma–Mittal set of entropy measures. We also emphasize the derivation of these inequalities for the special cases of one-parameter Havrda–Charvat’s, Rényi’s and Landsberg–Vedral’s entropy measures.     

On Conditional Tsallis Entropy

There is no generally accepted definition for conditional Tsallis entropy. The standard definition of (unconditional) Tsallis entropy depends on a parameter α that converges to the Shannon entropy as α approaches 1. In this paper, we describe three proposed definitions of conditional Tsallis entropy suggested in the literature—their properties are studied and their values, as a function of α, are compared. We also consider another natural proposal for conditional Tsallis entropy and compare it with the existing ones. Lastly, we present an online tool to compute the four conditional Tsallis entropies, given the probability distributions and the value of the parameter α.     

Empirical Mode Decomposition-Derived Entropy Features Are Beneficial to Distinguish Elderly People with a Falling History on a Force Plate Signal

Fall risk prediction is an important issue for the elderly. A center of pressure signal, derived from a force plate, is useful for the estimation of body calibration. However, it is still difficult to distinguish elderly people’s fall history by using a force plate signal. In this study, older adults with and without a history of falls were recruited to stand still for 60 s on a force plate. Forces in the x, y and z directions (Fx, Fy, and Fz) and center of pressure in the anteroposterior (COPx) and mediolateral directions (COPy) were derived. There were 49 subjects in the non-fall group, with an average age of 71.67 (standard derivation: 6.56). There were also 27 subjects in the fall group, with an average age of 70.66 (standard derivation: 6.38). Five signal series—forces in x, y, z (Fx, Fy, Fz), COPX, and COPy directions—were used. These five signals were further decomposed with empirical mode decomposition (EMD) with seven intrinsic mode functions. Time domain features (mean, standard derivation and coefficient of variations) and entropy features (approximate entropy and sample entropy) of the original signals and EMD-derived signals were extracted. Results showed that features extracted from the raw COP data did not differ significantly between the fall and non-fall groups. There were 10 features extracted using EMD, with significant differences observed among fall and non-fall groups. These included four features from COPx and two features from COPy, Fx and Fz.