Real and spurious contributions for the Shannon, Rényi and Tsallis entropies

A two-parameter probability distribution is constructed by dilatation (or contraction) of the escort probability distribution. This transformation involves a physical probability distribution P associated with the system under study and an almost arbitrary reference probability distribution P^′. In contrast to the Shannon and Rényi entropies, the Tsallis entropy does not decompose as the sum of the physical contribution due to P and the reference or spurious part owing to P^′. For solving this problem, a slight modification to the relation between Tsallis and Rényi entropies must be introduced. The procedure in this paper gives rise to a nonconventional one-parameter Shannon entropy and to two-parameter Rényi and Tsallis entropies associated with P. It also contributes to clarify the meaning and role of the escort probabilities set.     

Tsallis entropy measure of noise-aided information transmission in a binary channel

Noise-aided information transmission via stochastic resonance is shown and analyzed in a binary channel by means of information measures based on the Tsallis entropy. The analysis extends the classic reference of binary information transmission based on the Shannon entropy, and also parallels a recent study based on the Rényi entropy. The conditions for a maximally pronounced stochastic resonance identify optimal Tsallis measures. The study involves a correspondence between Tsallis and Rényi information measures, specially relevant to the characterization of stochastic resonance, and establishing that for such effects identical properties are shared in common by both Tsallis and Rényi measures.     

On Rényi Permutation Entropy

Among various modifications of the permutation entropy defined as the Shannon entropy of the ordinal pattern distribution underlying a system, a variant based on Rényi entropies was considered in a few papers. This paper discusses the relatively new concept of Rényi permutation entropies in dependence of non-negative real number q parameterizing the family of Rényi entropies and providing the Shannon entropy for q=1. Its relationship to Kolmogorov–Sinai entropy and, for q=2, to the recently introduced symbolic correlation integral are touched.     

Instabilities of Rényi entropies

We show that for systems with a large number of microstates Rényi entropies do not represent experimentally observable quantities except the Rényi entropy that coincides with the Shannon entropy.Work supported by the DFG (1978); author is recipient of a Feodor-Lynen grant from the Alexander von Humboldt Stiftung.     

Rényi’s information transfer between financial time series

In this paper, we quantify the statistical coherence between financial time series by means of the Rényi entropy. With the help of Campbell’s coding theorem, we show that the Rényi entropy selectively emphasizes only certain sectors of the underlying empirical distribution while strongly suppressing others. This accentuation is controlled with Rényi’s parameter q$q$. To tackle the issue of the information flow between time series, we formulate the concept of Rényi’s transfer entropy as a measure of information that is transferred only between certain parts of underlying distributions. This is particularly pertinent in financial time series, where the knowledge of marginal events such as spikes or sudden jumps is of a crucial importance. We apply the Rényian information flow to stock market time series from 11 world stock indices as sampled at a daily rate in the time period 02.01.1990–31.12.2009. Corresponding heat maps and net information flows are represented graphically. A detailed discussion of the transfer entropy between the DAX and S&P500 indices based on minute tick data gathered in the period 02.04.2008–11.09.2009 is also provided. Our analysis shows that the bivariate information flow between world markets is strongly asymmetric with a distinct information surplus flowing from the Asia–Pacific region to both European and US markets. An important yet less dramatic excess of information also flows from Europe to the US. This is particularly clearly seen from a careful analysis of Rényi information flow between the DAX and S&P500 indices.     

Rényi entropy and quantum phase transition in the Dicke model

The Rényi entropies of the Dicke model are presented. This quantum-optical model describes a single-mode bosonic field interacting with an ensemble of N two-level atoms. There is a quantum phase transition in the N→∞ limit. It is shown that there is an abrupt change in the Rényi entropy of order β at the transition point. Around the critical value of the coupling strength λc the Rényi entropy is proportional to the logarithm of the characteristic length and diverges as ln|λc−λ| for any order β. The pseudocapacity defined here in analogy with the heat capacity exhibits the phase transition. The critical exponent for the Dicke model is found to be 1 for any value of the parameter β.     

Role of information theoretic uncertainty relations in quantum theory

Uncertainty relations based on information theory for both discrete and continuous distribution functions are briefly reviewed. We extend these results to account for (differential) Rényi entropy and its related entropy power. This allows us to find a new class of information-theoretic uncertainty relations (ITURs). The potency of such uncertainty relations in quantum mechanics is illustrated with a simple two-energy-level model where they outperform both the usual Robertson–Schrödinger uncertainty relation and Shannon entropy based uncertainty relation. In the continuous case the ensuing entropy power uncertainty relations are discussed in the context of heavy tailed wave functions and Schrödinger cat states. Again, improvement over both the Robertson–Schrödinger uncertainty principle and Shannon ITUR is demonstrated in these cases. Further salient issues such as the proof of a generalized entropy power inequality and a geometric picture of information-theoretic uncertainty relations are also discussed.     

Source coding with escort distributions and Rényi entropy bounds

We discuss the interest of escort distributions and Rényi entropy in the context of source coding. We first recall a source coding theorem by Campbell relating a generalized measure of length to the Rényi-Tsallis entropy. We show that the associated optimal codes can be obtained using considerations on escort-distributions. We propose a new family of measure of length involving escort-distributions and we show that these generalized lengths are also bounded below by the Rényi entropy. Furthermore, we obtain that the standard Shannon codes lengths are optimum for the new generalized lengths measures, whatever the entropic index. Finally, we show that there exists in this setting an interplay between standard and escort distributions.     

Comment on ‘multifractal diffusion entropy analysis on stock volatility in financial markets’ [Physica A 391 (2012) 5739–5745]

In their recent article ‘multifractal diffusion entropy analysis on stock volatility in financial markets’ Huang, Shang and Zhao (2012) [6] suggested a generalization of the diffusion entropy analysis method with the main goal of being able to reveal scaling exponents for multifractal times series. The main idea seems to be replacing the Shannon entropy by the Rényi entropy, which is a one-parametric family of entropies. The authors claim that based on their method they are able to separate long range and short correlations of financial market multifractal time series. In this comment I show that the suggested new method does not bring much valuable information in obtaining the correct scaling for a multifractal/mono-fractal process beyond the original diffusion entropy analysis method. I also argue that the mathematical properties of the multifractal diffusion entropy analysis should be carefully explored to avoid possible numerical artefacts when implementing the method in analysis of real sequences of data.     

Pathway model,superstatistics, Tsallis statistics,and a generalized measure of entropy

The pathway model of Mathai [A pathway to matrix-variate gamma and normal densities, Linear Algebra Appl. 396 (2005) 317–328] is shown to be inferable from the maximization of a certain generalized entropy measure. This entropy is a variant of the generalized entropy of order α$\alpha$, considered in Mathai and Rathie [Basic Concepts in Information Theory and Statistics: Axiomatic Foundations and Applications, Wiley Halsted, New York and Wiley Eastern, New Delhi, 1975], and it is also associated with Shannon, Boltzmann–Gibbs, Rényi, Tsallis, and Havrda–Charvát entropies. The generalized entropy measure introduced here is also shown to have interesting statistical properties and it can be given probabilistic interpretations in terms of inaccuracy measure, expected value, and information content in a scheme. Particular cases of the pathway model are shown to be Tsallis statistics [C. Tsallis, Possible generalization of Boltzmann-Gibbs statistics, J. Stat. Phys. 52 (1988) 479–487] and superstatistics introduced by Beck and Cohen [Superstatistics, Physica A 322 (2003) 267–275]. The pathway model's connection to fractional calculus is illustrated by considering a fractional reaction equation.     

Tsallis distribution as a standard maximum entropy solution with ‘tail’ constraint

We show that Tsallis' distributions can be derived from the standard (Shannon) maximum entropy setting, by incorporating a constraint on the divergence between the distribution and another distribution imagined as its tail. In this setting, we find an underlying entropy which is the Rényi entropy. Furthermore, escort distributions and generalized means appear as a direct consequence of the construction. Finally, the “maximum entropy tail distribution” is identified as a Generalized Pareto Distribution.     

Fisher information,Rényi entropy power and quantum phase transition in the Dicke model

Fisher information, Rényi entropy power and Fisher–Rényi information product are presented for the Dicke model. There is a quantum phase transition in this quantum optical model. It is pointed out that there is an abrupt change in the Fisher information, Rényi entropy power, the Fisher, Shannon and Rényi lengths at the transition point. It is found that these quantities diverge as the characteristic length: |_λc−λ|^−1/4${|{\lambda }_c-\lambda |}^{-1/4}$ around the critical value of the coupling strength _λc${\lambda }_c$ for any value of the parameter β$\beta$.     

On the way towards a generalized entropy maximization procedure

We propose a generalized entropy maximization procedure, which takes into account the generalized averaging procedures and information gain definitions underlying the generalized entropies. This novel generalized procedure is then applied to Rényi and Tsallis entropies. The generalized entropy maximization procedure for Rényi entropies results in the exponential stationary distribution asymptotically for q∈(0,1] in contrast to the stationary distribution of the inverse power law obtained through the ordinary entropy maximization procedure. Another result of the generalized entropy maximization procedure is that one can naturally obtain all the possible stationary distributions associated with the Tsallis entropies by employing either ordinary or q-generalized Fourier transforms in the averaging procedure.     

Fisher-Rényi entropy product and information plane

Connection between Fisher information and Rényi entropy has been established. This link allows us to define the Fisher-Rényi information plane and an entropic product in terms of these quantities. New Rényi uncertainty relations are obtained for single particle densities of many particle systems in position-momentum conjugate spaces.     

Multifractal diffusion entropy analysis on stock volatility in financial markets

This paper introduces a generalized diffusion entropy analysis method to analyze long-range correlation then applies this method to stock volatility series. The method uses the techniques of the diffusion process and Rényi entropy to focus on the scaling behaviors of regular volatility and extreme volatility respectively in developed and emerging markets. It successfully distinguishes their differences where regular volatility exhibits long-range persistence while extreme volatility reveals anti-persistence.     

Nonextensive random-matrix theory based on Kaniadakis entropy

The joint eigenvalue distributions of random-matrix ensembles are derived by applying the principle maximum entropy to the Rényi, Abe and Kaniadakis entropies. While the Rényi entropy produces essentially the same matrix-element distributions as the previously obtained expression by using the Tsallis entropy, and the Abe entropy does not lead to a closed form expression, the Kaniadakis entropy leads to a new generalized form of the Wigner surmise that describes a transition of the spacing distribution from chaos to order. This expression is compared with the corresponding expression obtained by assuming Tsallis' entropy as well as the results of a previous numerical experiment.     

Rényi information of atoms

Position and momentum space Rényi information of order α has been studied within a Hartree-Fock framework for 103 neutral atoms, 54 singly charged cations and 43 anions in their ground state. The values of α?1 (α?1) stress the shell structure for position-space (momentum-space) Rényi information. The relationship between the complexity and Rényi information is also studied.     

Critique of multinomial coefficients method for evaluating Tsallis and Rényi entropies

Oikonomou [Th. Oikonomou, Physica A 386 (2007) 119] has published a calculation which purports to show that the Tsallis and Rényi entropies can be obtained from the generalized multinomial coefficients. In this paper, we prove that the method of generalized multinomial coefficients failed to determine the Tsallis entropy at equilibrium. Moreover, it is shown that Oikonomou’s analysis contains mistakes which led to misleading statements related to the Jaynes principle of maximum entropy, the Tsallis and the Rényi statistics.     

On classes of non-Gaussian asymptotic minimizers in entropic uncertainty principles

In this paper we revisit the Bialynicki-Birula and Mycielski uncertainty principle and its cases of equality. This Shannon entropic version of the well-known Heisenberg uncertainty principle can be used when dealing with variables that admit no variance. In this paper, we extend this uncertainty principle to Rényi entropies. We recall that in both Shannon and Rényi cases, and for a given dimension n$n$, the only case of equality occurs for Gaussian random vectors. We show that as n$n$ grows, however, the bound is also asymptotically attained in the cases of n$n$-dimensional Student-t$t$ and Student-r$r$ distributions. A complete analytical study is performed in a special case of a Student-t$t$ distribution. We also show numerically that this effect exists for the particular case of a n$n$-dimensional Cauchy variable, whatever the Rényi entropy considered, extending the results of Abe and illustrating the analytical asymptotic study of the Student-t$t$ case. In the Student-r$r$ case, we show numerically that the same behavior occurs for uniformly distributed vectors. These particular cases and other ones investigated in this paper are interesting since they show that this asymptotic behavior cannot be considered as a “Gaussianization” of the vector when the dimension increases.     

Tsallis, Rényi and nonextensive Gaussian entropy derived from the respective multinomial coefficients

We explore the generalization of the ordinary multinomial coefficient, based on the deformed q-multiplication and q-division. Aim of this study is to construct the appropriate multinomial coefficients, from which one can obtain the Tsallis, Rényi and nonextensive Gaussian entropy, respectively. We show that for all three above entropies there are two possible ways to define the generalized multinomial coefficient. Its consequence is discussed.