Comment on “Critique of multinomial coefficient method for evaluating Tsallis and Rényi entropies” by A.S. Parvan

Parvan [A.S. Parvan, Critique of multinomial coefficients method for evaluating Tsallis and Rényi entropies, Physica A 389 (2010) 5645] has recently presented some calculations in order to demonstrate the incorrectness of the results obtained from the generalized multinomial coefficients (GMC) presented in Oikonomou (2007) [1]. According to Parvan, the aforementioned approach of studying maximum entropy probability distributions is erroneous. In this comment I demonstrate that Parvan’s arguments do not hold true and that the obtained results from GMC do not present either mathematical or physical discrepancies.     

Tsallis, Rényi and nonextensive Gaussian entropy derived from the respective multinomial coefficients

We explore the generalization of the ordinary multinomial coefficient, based on the deformed q-multiplication and q-division. Aim of this study is to construct the appropriate multinomial coefficients, from which one can obtain the Tsallis, Rényi and nonextensive Gaussian entropy, respectively. We show that for all three above entropies there are two possible ways to define the generalized multinomial coefficient. Its consequence is discussed.     

On the way towards a generalized entropy maximization procedure

We propose a generalized entropy maximization procedure, which takes into account the generalized averaging procedures and information gain definitions underlying the generalized entropies. This novel generalized procedure is then applied to Rényi and Tsallis entropies. The generalized entropy maximization procedure for Rényi entropies results in the exponential stationary distribution asymptotically for q∈(0,1] in contrast to the stationary distribution of the inverse power law obtained through the ordinary entropy maximization procedure. Another result of the generalized entropy maximization procedure is that one can naturally obtain all the possible stationary distributions associated with the Tsallis entropies by employing either ordinary or q-generalized Fourier transforms in the averaging procedure.     

Tsallis entropy measure of noise-aided information transmission in a binary channel

Noise-aided information transmission via stochastic resonance is shown and analyzed in a binary channel by means of information measures based on the Tsallis entropy. The analysis extends the classic reference of binary information transmission based on the Shannon entropy, and also parallels a recent study based on the Rényi entropy. The conditions for a maximally pronounced stochastic resonance identify optimal Tsallis measures. The study involves a correspondence between Tsallis and Rényi information measures, specially relevant to the characterization of stochastic resonance, and establishing that for such effects identical properties are shared in common by both Tsallis and Rényi measures.     

Nonextensive random-matrix theory based on Kaniadakis entropy

The joint eigenvalue distributions of random-matrix ensembles are derived by applying the principle maximum entropy to the Rényi, Abe and Kaniadakis entropies. While the Rényi entropy produces essentially the same matrix-element distributions as the previously obtained expression by using the Tsallis entropy, and the Abe entropy does not lead to a closed form expression, the Kaniadakis entropy leads to a new generalized form of the Wigner surmise that describes a transition of the spacing distribution from chaos to order. This expression is compared with the corresponding expression obtained by assuming Tsallis' entropy as well as the results of a previous numerical experiment.     

Real and spurious contributions for the Shannon, Rényi and Tsallis entropies

A two-parameter probability distribution is constructed by dilatation (or contraction) of the escort probability distribution. This transformation involves a physical probability distribution P associated with the system under study and an almost arbitrary reference probability distribution P^′. In contrast to the Shannon and Rényi entropies, the Tsallis entropy does not decompose as the sum of the physical contribution due to P and the reference or spurious part owing to P^′. For solving this problem, a slight modification to the relation between Tsallis and Rényi entropies must be introduced. The procedure in this paper gives rise to a nonconventional one-parameter Shannon entropy and to two-parameter Rényi and Tsallis entropies associated with P. It also contributes to clarify the meaning and role of the escort probabilities set.     

Source coding with escort distributions and Rényi entropy bounds

We discuss the interest of escort distributions and Rényi entropy in the context of source coding. We first recall a source coding theorem by Campbell relating a generalized measure of length to the Rényi-Tsallis entropy. We show that the associated optimal codes can be obtained using considerations on escort-distributions. We propose a new family of measure of length involving escort-distributions and we show that these generalized lengths are also bounded below by the Rényi entropy. Furthermore, we obtain that the standard Shannon codes lengths are optimum for the new generalized lengths measures, whatever the entropic index. Finally, we show that there exists in this setting an interplay between standard and escort distributions.     

A generalized thermodynamics for power-law statistics

We show that there exists a natural way to define a condition of generalized thermal equilibrium between systems governed by Tsallis thermostatistics, under the hypotheses that (i) the coupling between the systems is weak, (ii) the structure functions of the systems have a power-law dependence on the energy. It is found that the q values of two such systems at equilibrium must satisfy a relationship involving the respective numbers of degrees of freedom. The physical properties of a Tsallis distribution can be conveniently characterized by a new parameter η which can vary between 0 and +∞, these limits corresponding, respectively, to the two opposite situations of a microcanonical distribution and of a distribution with a predominant power-tail at high energies. We prove that the statistical expression of the thermodynamic functions is univocally determined by the requirements that (a) systems at thermal equilibrium have the same temperature, (b) the definitions of temperature and entropy are consistent with the second law of thermodynamics. We find that, for systems satisfying the hypotheses (i) and (ii) specified above, the thermodynamic entropy is given by Rényi entropy.     

The superdiffusion entropy production paradox in the space-fractional case for extended entropies

Contrary to intuition, entropy production rates grow as reversible, wave-like behavior is approached. This paradox was discovered in time-fractional diffusion equations. It was found to persist for extended entropies and for space-fractional diffusion as well. This paper completes the possibilities by showing that the paradox persists for Tsallis and Rényi entropies in the space-fractional case. Complications arising due to the heavy tail solutions of space-fractional diffusion equations are discussed in detail.     

Pathway model,superstatistics, Tsallis statistics,and a generalized measure of entropy

The pathway model of Mathai [A pathway to matrix-variate gamma and normal densities, Linear Algebra Appl. 396 (2005) 317–328] is shown to be inferable from the maximization of a certain generalized entropy measure. This entropy is a variant of the generalized entropy of order α$\alpha$, considered in Mathai and Rathie [Basic Concepts in Information Theory and Statistics: Axiomatic Foundations and Applications, Wiley Halsted, New York and Wiley Eastern, New Delhi, 1975], and it is also associated with Shannon, Boltzmann–Gibbs, Rényi, Tsallis, and Havrda–Charvát entropies. The generalized entropy measure introduced here is also shown to have interesting statistical properties and it can be given probabilistic interpretations in terms of inaccuracy measure, expected value, and information content in a scheme. Particular cases of the pathway model are shown to be Tsallis statistics [C. Tsallis, Possible generalization of Boltzmann-Gibbs statistics, J. Stat. Phys. 52 (1988) 479–487] and superstatistics introduced by Beck and Cohen [Superstatistics, Physica A 322 (2003) 267–275]. The pathway model's connection to fractional calculus is illustrated by considering a fractional reaction equation.     

Escort entropies and divergences and related canonical distribution

We discuss two families of two-parameter entropies and divergences, derived from the standard Rényi and Tsallis entropies and divergences. These divergences and entropies are found as divergences or entropies of escort distributions. Exploiting the nonnegativity of the divergences, we derive the expression of the canonical distribution associated to the new entropies and a observable given as an escort-mean value. We show that this canonical distribution extends, and smoothly connects, the results obtained in nonextensive thermodynamics for the standard and generalized mean value constraints.     

Instabilities of Rényi entropies

We show that for systems with a large number of microstates Rényi entropies do not represent experimentally observable quantities except the Rényi entropy that coincides with the Shannon entropy.Work supported by the DFG (1978); author is recipient of a Feodor-Lynen grant from the Alexander von Humboldt Stiftung.     

Fisher-Rényi entropy product and information plane

Connection between Fisher information and Rényi entropy has been established. This link allows us to define the Fisher-Rényi information plane and an entropic product in terms of these quantities. New Rényi uncertainty relations are obtained for single particle densities of many particle systems in position-momentum conjugate spaces.     

Tsallis distribution as a standard maximum entropy solution with ‘tail’ constraint

We show that Tsallis' distributions can be derived from the standard (Shannon) maximum entropy setting, by incorporating a constraint on the divergence between the distribution and another distribution imagined as its tail. In this setting, we find an underlying entropy which is the Rényi entropy. Furthermore, escort distributions and generalized means appear as a direct consequence of the construction. Finally, the “maximum entropy tail distribution” is identified as a Generalized Pareto Distribution.     

Extensive Rényi statistics from non-extensive entropy

We show that starting with either the non-extensive Tsallis entropy in Wang's formalism or the extensive Rényi entropy, it is possible to construct equilibrium non-Gibbs canonical distribution functions which satisfy the fundamental equations of thermodynamics. The statistical mechanics with Tsallis entropy does not satisfy the zeroth law of thermodynamics at dynamical and statistical independence request, whereas the extensive Rényi statistics fulfills all requirements of equilibrium thermodynamics in the microcanonical ensemble. Transformation formulas between Tsallis statistics in Wang representation and Rényi statistics are presented. The one-particle distribution function in Rényi statistics for a classical ideal gas at finite particle number has a power-law tail for large momenta.     

Multifractal diffusion entropy analysis on stock volatility in financial markets

This paper introduces a generalized diffusion entropy analysis method to analyze long-range correlation then applies this method to stock volatility series. The method uses the techniques of the diffusion process and Rényi entropy to focus on the scaling behaviors of regular volatility and extreme volatility respectively in developed and emerging markets. It successfully distinguishes their differences where regular volatility exhibits long-range persistence while extreme volatility reveals anti-persistence.     

Rényi’s information transfer between financial time series

In this paper, we quantify the statistical coherence between financial time series by means of the Rényi entropy. With the help of Campbell’s coding theorem, we show that the Rényi entropy selectively emphasizes only certain sectors of the underlying empirical distribution while strongly suppressing others. This accentuation is controlled with Rényi’s parameter q$q$. To tackle the issue of the information flow between time series, we formulate the concept of Rényi’s transfer entropy as a measure of information that is transferred only between certain parts of underlying distributions. This is particularly pertinent in financial time series, where the knowledge of marginal events such as spikes or sudden jumps is of a crucial importance. We apply the Rényian information flow to stock market time series from 11 world stock indices as sampled at a daily rate in the time period 02.01.1990–31.12.2009. Corresponding heat maps and net information flows are represented graphically. A detailed discussion of the transfer entropy between the DAX and S&P500 indices based on minute tick data gathered in the period 02.04.2008–11.09.2009 is also provided. Our analysis shows that the bivariate information flow between world markets is strongly asymmetric with a distinct information surplus flowing from the Asia–Pacific region to both European and US markets. An important yet less dramatic excess of information also flows from Europe to the US. This is particularly clearly seen from a careful analysis of Rényi information flow between the DAX and S&P500 indices.     

Acoustic classification of Australian anurans based on hybrid spectral-entropy approach

A new hybrid method for automated frog sound identification, using spectral centroid, Shannon entropy and Rényi entropy is proposed. The advantage of using entropy based information theoretic approach for analyzing complexity of bioacoustics signals in animal vocalization is discussed. Sound samples from nine species of Microhylidae frogs are first segmented into syllables. Fourier spectral centroid, Shannon entropy and Rényi entropy of the syllables are then determined. Finally, nonparametric k-th nearest neighbour (k-NN) classifier is used to recognize the frog species based on these three extracted features. Result shows that the k-NN classifier based on these selected features is capable to identify the species of the frogs with an average accuracy of 98%. It is found that the accuracy reduces significantly only when the noise levels higher than −20 dB.     

From Rényi Entropy Power to Information Scan of Quantum States

In this paper, we generalize the notion of Shannon’s entropy power to the Rényi-entropy setting. With this, we propose generalizations of the de Bruijn identity, isoperimetric inequality, or Stam inequality. This framework not only allows for finding new estimation inequalities, but it also provides a convenient technical framework for the derivation of a one-parameter family of Rényi-entropy-power-based quantum-mechanical uncertainty relations. To illustrate the usefulness of the Rényi entropy power obtained, we show how the information probability distribution associated with a quantum state can be reconstructed in a process that is akin to quantum-state tomography. We illustrate the inner workings of this with the so-called “cat states”, which are of fundamental interest and practical use in schemes such as quantum metrology. Salient issues, including the extension of the notion of entropy power to Tsallis entropy and ensuing implications in estimation theory, are also briefly discussed.     

Rényi entropy and quantum phase transition in the Dicke model

The Rényi entropies of the Dicke model are presented. This quantum-optical model describes a single-mode bosonic field interacting with an ensemble of N two-level atoms. There is a quantum phase transition in the N→∞ limit. It is shown that there is an abrupt change in the Rényi entropy of order β at the transition point. Around the critical value of the coupling strength λc the Rényi entropy is proportional to the logarithm of the characteristic length and diverges as ln|λc−λ| for any order β. The pseudocapacity defined here in analogy with the heat capacity exhibits the phase transition. The critical exponent for the Dicke model is found to be 1 for any value of the parameter β.