Tsallis, Rényi and nonextensive Gaussian entropy derived from the respective multinomial coefficients

We explore the generalization of the ordinary multinomial coefficient, based on the deformed q-multiplication and q-division. Aim of this study is to construct the appropriate multinomial coefficients, from which one can obtain the Tsallis, Rényi and nonextensive Gaussian entropy, respectively. We show that for all three above entropies there are two possible ways to define the generalized multinomial coefficient. Its consequence is discussed.     

Real and spurious contributions for the Shannon, Rényi and Tsallis entropies

A two-parameter probability distribution is constructed by dilatation (or contraction) of the escort probability distribution. This transformation involves a physical probability distribution P associated with the system under study and an almost arbitrary reference probability distribution P^′. In contrast to the Shannon and Rényi entropies, the Tsallis entropy does not decompose as the sum of the physical contribution due to P and the reference or spurious part owing to P^′. For solving this problem, a slight modification to the relation between Tsallis and Rényi entropies must be introduced. The procedure in this paper gives rise to a nonconventional one-parameter Shannon entropy and to two-parameter Rényi and Tsallis entropies associated with P. It also contributes to clarify the meaning and role of the escort probabilities set.     

Critique of multinomial coefficients method for evaluating Tsallis and Rényi entropies

Oikonomou [Th. Oikonomou, Physica A 386 (2007) 119] has published a calculation which purports to show that the Tsallis and Rényi entropies can be obtained from the generalized multinomial coefficients. In this paper, we prove that the method of generalized multinomial coefficients failed to determine the Tsallis entropy at equilibrium. Moreover, it is shown that Oikonomou’s analysis contains mistakes which led to misleading statements related to the Jaynes principle of maximum entropy, the Tsallis and the Rényi statistics.     

Nonextensive random-matrix theory based on Kaniadakis entropy

The joint eigenvalue distributions of random-matrix ensembles are derived by applying the principle maximum entropy to the Rényi, Abe and Kaniadakis entropies. While the Rényi entropy produces essentially the same matrix-element distributions as the previously obtained expression by using the Tsallis entropy, and the Abe entropy does not lead to a closed form expression, the Kaniadakis entropy leads to a new generalized form of the Wigner surmise that describes a transition of the spacing distribution from chaos to order. This expression is compared with the corresponding expression obtained by assuming Tsallis' entropy as well as the results of a previous numerical experiment.     

Rényi and Tsallis entropies for incomplete or overcomplete systems of events

In this article, Shannon, Rényi and Tsallis entropies are considered for a system of events characterized by an arbitrary probability distribution P that can be incomplete, complete or overcomplete. After a suitable transformation that leads to the escort probabilities of P, these can be written as the canonical probability distribution for a set of pseudo-energies (Hartley information, E_n=−lnP_n) and a dimensionless parameter q that plays the role of thermodynamics β. Several relations between the entropies are presented, including the analysis of compound systems. The method is illustrated with an example.     

Pathway model,superstatistics, Tsallis statistics,and a generalized measure of entropy

The pathway model of Mathai [A pathway to matrix-variate gamma and normal densities, Linear Algebra Appl. 396 (2005) 317–328] is shown to be inferable from the maximization of a certain generalized entropy measure. This entropy is a variant of the generalized entropy of order α$\alpha$, considered in Mathai and Rathie [Basic Concepts in Information Theory and Statistics: Axiomatic Foundations and Applications, Wiley Halsted, New York and Wiley Eastern, New Delhi, 1975], and it is also associated with Shannon, Boltzmann–Gibbs, Rényi, Tsallis, and Havrda–Charvát entropies. The generalized entropy measure introduced here is also shown to have interesting statistical properties and it can be given probabilistic interpretations in terms of inaccuracy measure, expected value, and information content in a scheme. Particular cases of the pathway model are shown to be Tsallis statistics [C. Tsallis, Possible generalization of Boltzmann-Gibbs statistics, J. Stat. Phys. 52 (1988) 479–487] and superstatistics introduced by Beck and Cohen [Superstatistics, Physica A 322 (2003) 267–275]. The pathway model's connection to fractional calculus is illustrated by considering a fractional reaction equation.     

A generalized thermodynamics for power-law statistics

We show that there exists a natural way to define a condition of generalized thermal equilibrium between systems governed by Tsallis thermostatistics, under the hypotheses that (i) the coupling between the systems is weak, (ii) the structure functions of the systems have a power-law dependence on the energy. It is found that the q values of two such systems at equilibrium must satisfy a relationship involving the respective numbers of degrees of freedom. The physical properties of a Tsallis distribution can be conveniently characterized by a new parameter η which can vary between 0 and +∞, these limits corresponding, respectively, to the two opposite situations of a microcanonical distribution and of a distribution with a predominant power-tail at high energies. We prove that the statistical expression of the thermodynamic functions is univocally determined by the requirements that (a) systems at thermal equilibrium have the same temperature, (b) the definitions of temperature and entropy are consistent with the second law of thermodynamics. We find that, for systems satisfying the hypotheses (i) and (ii) specified above, the thermodynamic entropy is given by Rényi entropy.     

The superdiffusion entropy production paradox in the space-fractional case for extended entropies

Contrary to intuition, entropy production rates grow as reversible, wave-like behavior is approached. This paradox was discovered in time-fractional diffusion equations. It was found to persist for extended entropies and for space-fractional diffusion as well. This paper completes the possibilities by showing that the paradox persists for Tsallis and Rényi entropies in the space-fractional case. Complications arising due to the heavy tail solutions of space-fractional diffusion equations are discussed in detail.     

Escort entropies and divergences and related canonical distribution

We discuss two families of two-parameter entropies and divergences, derived from the standard Rényi and Tsallis entropies and divergences. These divergences and entropies are found as divergences or entropies of escort distributions. Exploiting the nonnegativity of the divergences, we derive the expression of the canonical distribution associated to the new entropies and a observable given as an escort-mean value. We show that this canonical distribution extends, and smoothly connects, the results obtained in nonextensive thermodynamics for the standard and generalized mean value constraints.     

Tsallis entropy measure of noise-aided information transmission in a binary channel

Noise-aided information transmission via stochastic resonance is shown and analyzed in a binary channel by means of information measures based on the Tsallis entropy. The analysis extends the classic reference of binary information transmission based on the Shannon entropy, and also parallels a recent study based on the Rényi entropy. The conditions for a maximally pronounced stochastic resonance identify optimal Tsallis measures. The study involves a correspondence between Tsallis and Rényi information measures, specially relevant to the characterization of stochastic resonance, and establishing that for such effects identical properties are shared in common by both Tsallis and Rényi measures.     

On Rényi Permutation Entropy

Among various modifications of the permutation entropy defined as the Shannon entropy of the ordinal pattern distribution underlying a system, a variant based on Rényi entropies was considered in a few papers. This paper discusses the relatively new concept of Rényi permutation entropies in dependence of non-negative real number q parameterizing the family of Rényi entropies and providing the Shannon entropy for q=1. Its relationship to Kolmogorov–Sinai entropy and, for q=2, to the recently introduced symbolic correlation integral are touched.     

Non-Gaussian effects on quantum entropies

A deduction of generalized quantum entropies within the non-Gaussian frameworks, Tsallis and Kaniadakis, is derived using a generalized combinatorial method and the so-called q and κ calculus. In agreement with previous results, we also show that for the Tsallis formulation the q-quantum entropy is well-defined for values of the nonextensive parameter q lying in the interval [0,2].     

The maximization of Tsallis entropy with complete deformed functions and the problem of constraints

By only requiring the q deformed logarithms (q exponentials) to possess arguments chosen from the entire set of positive real numbers (all real numbers), we show that the q-logarithm (q exponential) can be written in such a way that its argument varies between 0 and 1 (among negative real numbers) for 1?q<2, while the interval 0<q?1 corresponds to any real argument greater than 1 (positive real numbers). These two distinct intervals of the nonextensivity index q, also the expressions of the deformed functions associated with them, are related to one another through the relation (2−q), which is so far used to obtain the ordinary stationary distributions from the corresponding escort distributions, and vice versa in an almost ad hoc manner. This shows that the escort distributions are only a means of extending the interval of validity of the deformed functions to the one of ordinary, undeformed ones. Moreover, we show that, since the Tsallis entropy is written in terms of the q-logarithm and its argument, being the inverse of microstate probabilities, takes values equal to or greater than 1, the resulting stationary solution is uniquely described by the one obtained from the ordinary constraint. Finally, we observe that even the escort stationary distributions can be obtained through the use of the ordinary averaging procedure if the argument of the q-exponential lies in (−∞,0]. However, this case corresponds to, although related, a different entropy expression than the Tsallis entropy.     

A family of nonextensive entropies

A generalized nonextensive two-parameter entropy is developed, along lines which unify current nonextensive frameworks. It recovers, as particular cases, the Tsallis and symmetric entropies, as well as the Boltzmann-Gibbs entropy. The properties of the new (q, q′)-entropy are analysed.     

Rényi entropy and quantum phase transition in the Dicke model

The Rényi entropies of the Dicke model are presented. This quantum-optical model describes a single-mode bosonic field interacting with an ensemble of N two-level atoms. There is a quantum phase transition in the N→∞ limit. It is shown that there is an abrupt change in the Rényi entropy of order β at the transition point. Around the critical value of the coupling strength λc the Rényi entropy is proportional to the logarithm of the characteristic length and diverges as ln|λc−λ| for any order β. The pseudocapacity defined here in analogy with the heat capacity exhibits the phase transition. The critical exponent for the Dicke model is found to be 1 for any value of the parameter β.     

Comment on “Critique of multinomial coefficient method for evaluating Tsallis and Rényi entropies” by A.S. Parvan

Parvan [A.S. Parvan, Critique of multinomial coefficients method for evaluating Tsallis and Rényi entropies, Physica A 389 (2010) 5645] has recently presented some calculations in order to demonstrate the incorrectness of the results obtained from the generalized multinomial coefficients (GMC) presented in Oikonomou (2007) [1]. According to Parvan, the aforementioned approach of studying maximum entropy probability distributions is erroneous. In this comment I demonstrate that Parvan’s arguments do not hold true and that the obtained results from GMC do not present either mathematical or physical discrepancies.     

Rényi’s information transfer between financial time series

In this paper, we quantify the statistical coherence between financial time series by means of the Rényi entropy. With the help of Campbell’s coding theorem, we show that the Rényi entropy selectively emphasizes only certain sectors of the underlying empirical distribution while strongly suppressing others. This accentuation is controlled with Rényi’s parameter q$q$. To tackle the issue of the information flow between time series, we formulate the concept of Rényi’s transfer entropy as a measure of information that is transferred only between certain parts of underlying distributions. This is particularly pertinent in financial time series, where the knowledge of marginal events such as spikes or sudden jumps is of a crucial importance. We apply the Rényian information flow to stock market time series from 11 world stock indices as sampled at a daily rate in the time period 02.01.1990–31.12.2009. Corresponding heat maps and net information flows are represented graphically. A detailed discussion of the transfer entropy between the DAX and S&P500 indices based on minute tick data gathered in the period 02.04.2008–11.09.2009 is also provided. Our analysis shows that the bivariate information flow between world markets is strongly asymmetric with a distinct information surplus flowing from the Asia–Pacific region to both European and US markets. An important yet less dramatic excess of information also flows from Europe to the US. This is particularly clearly seen from a careful analysis of Rényi information flow between the DAX and S&P500 indices.     

A generalized Kolmogorov-Sinai-like entropy under Markov shifts in symbolic dynamics

In this paper, a generalized Kolmogorov-Sinai-like entropy ( entropy) in the sense of Tsallis is proposed with a nonextensive parameter q under Markov shifts, which contains the classical Kolmogorov-Sinai (KS) entropy and the Rényi entropy as well as Bernoulli shifts as special cases. To verify the formula of this entropy, a one-dimensional iterative system is chosen as an example of Markov shifts, and its entropy is evaluated by a new refinement method of symbolic dynamics called symbolic refinement which differs from the conventional numerical method. The numerical results show that this entropy is monotonically decreasing as q increases.     

On the construction of complex networks with optimal Tsallis entropy

In this work, we first formulate the Tsallis entropy in the context of complex networks. We then propose a network construction whose topology maximizes the Tsallis entropy. The growing network model has two main ingredients: copy process and random attachment mechanism (C-R model). We show that the resulting degree distribution exactly agrees with the required degree distribution that maximizes the Tsallis entropy. We also provide another example of network model using a combination of preferential and random attachment mechanisms (P-R model) and compare it with the distribution of the Tsallis entropy. In this case, we show that by adequately identifying the exponent factor q, the degree distribution can also be written in the q-exponential form. Taken together, our findings suggest that both mechanisms, copy process and preferential attachment, play a key role for the realization of networks with maximum Tsallis entropy. Finally, we discuss the interpretation of q parameter of the Tsallis entropy in the context of complex networks.     

Instabilities of Rényi entropies

We show that for systems with a large number of microstates Rényi entropies do not represent experimentally observable quantities except the Rényi entropy that coincides with the Shannon entropy.Work supported by the DFG (1978); author is recipient of a Feodor-Lynen grant from the Alexander von Humboldt Stiftung.