On generalized distributions and pathways

The scalar version of the pathway model of Mathai [A.M. Mathai, Linear Alg. Appl. 396 (2005) 317] is shown to be associated with a large number of probability models used in physics. Different families of densities are listed here, which are all connected through the pathway parameter α, generating a distributional pathway. The idea is to switch from one functional form to another through this parameter and it is shown that one can proceed from the generalized type-1 beta family to generalized type-2 beta family to generalized gamma family. It is also shown that the pathway model is available by maximizing a generalized measure of entropy, leading to an entropic pathway, covering the particularly interesting cases of Tsallis statistics [C. Tsallis, J. Stat. Phys. 52 (1988) 479] and superstatistics [C. Beck, E.G.D. Cohen, Physica A 322 (2003) 267].     

Pathway model,superstatistics, Tsallis statistics,and a generalized measure of entropy

The pathway model of Mathai [A pathway to matrix-variate gamma and normal densities, Linear Algebra Appl. 396 (2005) 317–328] is shown to be inferable from the maximization of a certain generalized entropy measure. This entropy is a variant of the generalized entropy of order α$\alpha$, considered in Mathai and Rathie [Basic Concepts in Information Theory and Statistics: Axiomatic Foundations and Applications, Wiley Halsted, New York and Wiley Eastern, New Delhi, 1975], and it is also associated with Shannon, Boltzmann–Gibbs, Rényi, Tsallis, and Havrda–Charvát entropies. The generalized entropy measure introduced here is also shown to have interesting statistical properties and it can be given probabilistic interpretations in terms of inaccuracy measure, expected value, and information content in a scheme. Particular cases of the pathway model are shown to be Tsallis statistics [C. Tsallis, Possible generalization of Boltzmann-Gibbs statistics, J. Stat. Phys. 52 (1988) 479–487] and superstatistics introduced by Beck and Cohen [Superstatistics, Physica A 322 (2003) 267–275]. The pathway model's connection to fractional calculus is illustrated by considering a fractional reaction equation.     

Uncertainty of Interval Type-2 Fuzzy Sets Based on Fuzzy Belief Entropy

Interval type-2 fuzzy sets (IT2 FS) play an important part in dealing with uncertain applications. However, how to measure the uncertainty of IT2 FS is still an open issue. The specific objective of this study is to present a new entropy named fuzzy belief entropy to solve the problem based on the relation among IT2 FS, belief structure, and Z-valuations. The interval of membership function can be transformed to interval BPA [Bel,Pl]. Then, Bel and Pl are put into the proposed entropy to calculate the uncertainty from the three aspects of fuzziness, discord, and nonspecificity, respectively, which makes the result more reasonable. Compared with other methods, fuzzy belief entropy is more reasonable because it can measure the uncertainty caused by multielement fuzzy subsets. Furthermore, when the membership function belongs to type-1 fuzzy sets, fuzzy belief entropy degenerates to Shannon entropy. Compared with other methods, several numerical examples are demonstrated that the proposed entropy is feasible and persuasive.     

Asymptotic Normality for Plug-In Estimators of Generalized Shannon’s Entropy

Shannon’s entropy is one of the building blocks of information theory and an essential aspect of Machine Learning (ML) methods (e.g., Random Forests). Yet, it is only finitely defined for distributions with fast decaying tails on a countable alphabet. The unboundedness of Shannon’s entropy over the general class of all distributions on an alphabet prevents its potential utility from being fully realized. To fill the void in the foundation of information theory, Zhang (2020) proposed generalized Shannon’s entropy, which is finitely defined everywhere. The plug-in estimator, adopted in almost all entropy-based ML method packages, is one of the most popular approaches to estimating Shannon’s entropy. The asymptotic distribution for Shannon’s entropy’s plug-in estimator was well studied in the existing literature. This paper studies the asymptotic properties for the plug-in estimator of generalized Shannon’s entropy on countable alphabets. The developed asymptotic properties require no assumptions on the original distribution. The proposed asymptotic properties allow for interval estimation and statistical tests with generalized Shannon’s entropy.     

Entropy-Regularized Optimal Transport on Multivariate Normal and q-normal Distributions

The distance and divergence of the probability measures play a central role in statistics, machine learning, and many other related fields. The Wasserstein distance has received much attention in recent years because of its distinctions from other distances or divergences. Although computing the Wasserstein distance is costly, entropy-regularized optimal transport was proposed to computationally efficiently approximate the Wasserstein distance. The purpose of this study is to understand the theoretical aspect of entropy-regularized optimal transport. In this paper, we focus on entropy-regularized optimal transport on multivariate normal distributions and q-normal distributions. We obtain the explicit form of the entropy-regularized optimal transport cost on multivariate normal and q-normal distributions; this provides a perspective to understand the effect of entropy regularization, which was previously known only experimentally. Furthermore, we obtain the entropy-regularized Kantorovich estimator for the probability measure that satisfies certain conditions. We also demonstrate how the Wasserstein distance, optimal coupling, geometric structure, and statistical efficiency are affected by entropy regularization in some experiments. In particular, our results about the explicit form of the optimal coupling of the Tsallis entropy-regularized optimal transport on multivariate q-normal distributions and the entropy-regularized Kantorovich estimator are novel and will become the first step towards the understanding of a more general setting.     

A Generalized Measure of Cumulative Residual Entropy

In this work, we introduce a generalized measure of cumulative residual entropy and study its properties. We show that several existing measures of entropy, such as cumulative residual entropy, weighted cumulative residual entropy and cumulative residual Tsallis entropy, are all special cases of this generalized cumulative residual entropy. We also propose a measure of generalized cumulative entropy, which includes cumulative entropy, weighted cumulative entropy and cumulative Tsallis entropy as special cases. We discuss a generating function approach, using which we derive different entropy measures. We provide residual and cumulative versions of Sharma–Taneja–Mittal entropy and obtain them as special cases this generalized measure of entropy. Finally, using the newly introduced entropy measures, we establish some relationships between entropy and extropy measures.     

Generalized Ordinal Patterns and the KS-Entropy

Ordinal patterns classifying real vectors according to the order relations between their components are an interesting basic concept for determining the complexity of a measure-preserving dynamical system. In particular, as shown by C. Bandt, G. Keller and B. Pompe, the permutation entropy based on the probability distributions of such patterns is equal to Kolmogorov–Sinai entropy in simple one-dimensional systems. The general reason for this is that, roughly speaking, the system of ordinal patterns obtained for a real-valued “measuring arrangement” has high potential for separating orbits. Starting from a slightly different approach of A. Antoniouk, K. Keller and S. Maksymenko, we discuss the generalizations of ordinal patterns providing enough separation to determine the Kolmogorov–Sinai entropy. For defining these generalized ordinal patterns, the idea is to substitute the basic binary relation ≤ on the real numbers by another binary relation. Generalizing the former results of I. Stolz and K. Keller, we establish conditions that the binary relation and the dynamical system have to fulfill so that the obtained generalized ordinal patterns can be used for estimating the Kolmogorov–Sinai entropy.     

A generalized entropy optimization and Maxwell–Boltzmann distribution

Based on the results of the diffusion entropy analysis of Super-Kamiokande solar neutrino data, a generalized entropy, introduced earlier by the first author is optimized under various conditions and it is shown that Maxwell–Boltzmann distribution, Raleigh distribution and other distributions can be obtained through such optimization procedures. Some properties of the entropy measure are examined and then Maxwell–Boltzmann and Raleigh densities are extended to multivariate cases. Connections to geometrical probability problems, isotropic random points, and spherically symmetric and elliptically contoured statistical distributions are pointed out.     

Differential Entropy and Dynamics of Uncertainty

We analyze the functioning of Gibbs-type entropy functionals in the time domain, with emphasis on Shannon and Kullback-Leibler entropies of time-dependent continuous probability distributions. The Shannon entropy validity is extended to probability distributions inferred from L ²(R ⁿ ) quantum wave packets. In contrast to the von Neumann entropy which simply vanishes on pure states, the differential entropy quantifies the degree of probability (de)localization and its time development. The associated dynamics of the Fisher information functional quantifies nontrivial power transfer processes in the mean, both in dissipative and quantum mechanical cases. PACS NUMBERS: 05.45.+b, 02.50.-r, 03.65.Ta, 03.67.-a     

A class of asymmetric pathway distributions and an entropy interpretation

Asymmetric distributions are widely used in probability modeling and statistical analysis. Recently, various asymmetric distributions are being developed by many researchers for modeling various data sets in real life contexts. In the present paper, we introduce a new class of q-type asymmetric distributions which include q-analogues of asymmetric Laplace, exponential power, Weibull etc. and corresponding standard distributions as special cases. Also we show that this pathway model can be obtained by optimizing Mathai’s generalized entropy with more general setup, which is a generalization of various entropy measures due to Shannon and others.     

Some Further Results on the Fractional Cumulative Entropy

In this paper, the fractional cumulative entropy is considered to get its further properties and also its developments to dynamic cases. The measure is used to characterize a family of symmetric distributions and also another location family of distributions. The links between the fractional cumulative entropy and the classical differential entropy and some reliability quantities are also unveiled. In addition, the connection the measure has with the standard deviation is also found. We provide some examples to establish the variability property of this measure.     

Estimation for Entropy and Parameters of Generalized Bilal Distribution under Adaptive Type II Progressive Hybrid Censoring Scheme

Entropy measures the uncertainty associated with a random variable. It has important applications in cybernetics, probability theory, astrophysics, life sciences and other fields. Recently, many authors focused on the estimation of entropy with different life distributions. However, the estimation of entropy for the generalized Bilal (GB) distribution has not yet been involved. In this paper, we consider the estimation of the entropy and the parameters with GB distribution based on adaptive Type-II progressive hybrid censored data. Maximum likelihood estimation of the entropy and the parameters are obtained using the Newton–Raphson iteration method. Bayesian estimations under different loss functions are provided with the help of Lindley’s approximation. The approximate confidence interval and the Bayesian credible interval of the parameters and entropy are obtained by using the delta and Markov chain Monte Carlo (MCMC) methods, respectively. Monte Carlo simulation studies are carried out to observe the performances of the different point and interval estimations. Finally, a real data set has been analyzed for illustrative purposes.     

Nonequilibrium Gas and Generalized Billiards

Generalized billiards describe nonequilibrium gas, consisting of finitely many particles, that move in a container, whose walls heat up or cool down. Generalized billiards can be considered both in the framework of the Newtonian mechanics and of the relativity theory. In the Newtonian case, a generalized billiard may possess an invariant measure; the Gibbs entropy with respect to this measure is constant. On the contrary, generalized relativistic billiards are always dissipative,and the Gibbs entropy with respect to the same measure grows under some natural conditions. In this article, we find the necessary and sufficient conditions for a generalized Newtonian billiard to possess a smooth invariant measure, which is independent of the boundary action: the corresponding classical billiard should have an additional first integral of special type. In particular,the generalized Sinai billiards do not possess a smooth invariant measure. We then consider generalized billiards inside a ball, which is one of the main examples of the Newtonian generalized billiards which does have an invariant measure. We construct explicitly the invariant measure, and find the conditions for the Gibbs entropy growth for the corresponding relativistic billiard both formonotone and periodic action of the boundary.     

Estimating Non-Gaussianity of a Quantum State by Measuring Orthogonal Quadratures

We derive the lower bounds for a non-Gaussianity measure based on quantum relative entropy (QRE). Our approach draws on the observation that the QRE-based non-Gaussianity measure of a single-mode quantum state is lower bounded by a function of the negentropies for quadrature distributions with maximum and minimum variances. We demonstrate that the lower bound can outperform the previously proposed bound by the negentropy of a quadrature distribution. Furthermore, we extend our method to establish lower bounds for the QRE-based non-Gaussianity measure of a multimode quantum state that can be measured by homodyne detection, with or without leveraging a Gaussian unitary operation. Finally, we explore how our lower bound finds application in non-Gaussian entanglement detection.     

Real-World Data Difficulty Estimation with the Use of Entropy

In the era of the Internet of Things and big data, we are faced with the management of a flood of information. The complexity and amount of data presented to the decision-maker are enormous, and existing methods often fail to derive nonredundant information quickly. Thus, the selection of the most satisfactory set of solutions is often a struggle. This article investigates the possibilities of using the entropy measure as an indicator of data difficulty. To do so, we focus on real-world data covering various fields related to markets (the real estate market and financial markets), sports data, fake news data, and more. The problem is twofold: First, since we deal with unprocessed, inconsistent data, it is necessary to perform additional preprocessing. Therefore, the second step of our research is using the entropy-based measure to capture the nonredundant, noncorrelated core information from the data. Research is conducted using well-known algorithms from the classification domain to investigate the quality of solutions derived based on initial preprocessing and the information indicated by the entropy measure. Eventually, the best 25% (in the sense of entropy measure) attributes are selected to perform the whole classification procedure once again, and the results are compared.     

Entropy Estimation Using a Linguistic Zipf–Mandelbrot–Li Model for Natural Sequences

Entropy estimation faces numerous challenges when applied to various real-world problems. Our interest is in divergence and entropy estimation algorithms which are capable of rapid estimation for natural sequence data such as human and synthetic languages. This typically requires a large amount of data; however, we propose a new approach which is based on a new rank-based analytic Zipf–Mandelbrot–Li probabilistic model. Unlike previous approaches, which do not consider the nature of the probability distribution in relation to language; here, we introduce a novel analytic Zipfian model which includes linguistic constraints. This provides more accurate distributions for natural sequences such as natural or synthetic emergent languages. Results are given which indicates the performance of the proposed ZML model. We derive an entropy estimation method which incorporates the linguistic constraint-based Zipf–Mandelbrot–Li into a new non-equiprobable coincidence counting algorithm which is shown to be effective for tasks such as entropy rate estimation with limited data.     

Generalized thermostatistics based on the Sharma-Mittal entropy and escort mean values

A generalized thermostatistics is developed for an entropy measure introduced by Sharma and Mittal. A maximum-entropy scheme involving the maximization of the Sharma and Mittal entropy under appropriate constraints expressed as escort mean values is advanced. Maximum-entropy distributions exhibiting a power law behavior in the asymptotic limit are obtained. Thus, results previously derived for the Renyi entropy and the Tsallis entropy are generalized. In addition, it is shown that for almost deterministic systems among all possible composable entropies with kernels that are described by power laws the Sharma-Mittal entropy is the only entropy measure that gives rise to a thermostatistics based on escort mean values and admitting of a partition function. Received 27 June 2002 Published online 31 December 2002     

Working Memory Decline in Alzheimer’s Disease Is Detected by Complexity Analysis of Multimodal EEG-fNIRS

Alzheimer’s disease (AD) is characterized by working memory (WM) failures that can be assessed at early stages through administering clinical tests. Ecological neuroimaging, such as Electroencephalography (EEG) and functional Near Infrared Spectroscopy (fNIRS), may be employed during these tests to support AD early diagnosis within clinical settings. Multimodal EEG-fNIRS could measure brain activity along with neurovascular coupling (NC) and detect their modifications associated with AD. Data analysis procedures based on signal complexity are suitable to estimate electrical and hemodynamic brain activity or their mutual information (NC) during non-structured experimental paradigms. In this study, sample entropy of whole-head EEG and frontal/prefrontal cortex fNIRS was evaluated to assess brain activity in early AD and healthy controls (HC) during WM tasks (i.e., Rey–Osterrieth complex figure and Raven’s progressive matrices). Moreover, conditional entropy between EEG and fNIRS was evaluated as indicative of NC. The findings demonstrated the capability of complexity analysis of multimodal EEG-fNIRS to detect WM decline in AD. Furthermore, a multivariate data-driven analysis, performed on these entropy metrics and based on the General Linear Model, allowed classifying AD and HC with an AUC up to 0.88. EEG-fNIRS may represent a powerful tool for the clinical evaluation of WM decline in early AD.     

Two direct methods to calculate fluctuation forces between rigid objects embedded in fluid membranes

The fluctuation-induced attractive interaction of rigid flat objects embedded in a fluid membrane is calculated for a pair of parallel strips and a pair of equal circular disks. Assuming flat boundary conditions, we derive the interaction from the entropy of the suppressed boundary angle fluctuation modes. Each mode entropy is computed in two ways: from the boundary angles themselves and from the mean-curvature mode functions. A formula for the entropy loss of suppressing one or more mean-curvature modes is developed and applied. For the pair of disks we recover the result of Goulian et al. and Golestanian et al. in a direct manner, avoiding any mappings by Hubbard-Stratonovitch transformations. The mode-by-mode agreement of the two computed entropies in both systems confirms an earlier claim that mean curvature is the natural measure of integration for fluid membranes. Received 15 December 2000     

Relative Entropy of Correct Proximal Policy Optimization Algorithms with Modified Penalty Factor in Complex Environment

In the field of reinforcement learning, we propose a Correct Proximal Policy Optimization (CPPO) algorithm based on the modified penalty factor β and relative entropy in order to solve the robustness and stationarity of traditional algorithms. Firstly, In the process of reinforcement learning, this paper establishes a strategy evaluation mechanism through the policy distribution function. Secondly, the state space function is quantified by introducing entropy, whereby the approximation policy is used to approximate the real policy distribution, and the kernel function estimation and calculation of relative entropy is used to fit the reward function based on complex problem. Finally, through the comparative analysis on the classic test cases, we demonstrated that our proposed algorithm is effective, has a faster convergence speed and better performance than the traditional PPO algorithm, and the measure of the relative entropy can show the differences. In addition, it can more efficiently use the information of complex environment to learn policies. At the same time, not only can our paper explain the rationality of the policy distribution theory, the proposed framework can also balance between iteration steps, computational complexity and convergence speed, and we also introduced an effective measure of performance using the relative entropy concept.