Rényi and Tsallis entropies for incomplete or overcomplete systems of events

In this article, Shannon, Rényi and Tsallis entropies are considered for a system of events characterized by an arbitrary probability distribution P that can be incomplete, complete or overcomplete. After a suitable transformation that leads to the escort probabilities of P, these can be written as the canonical probability distribution for a set of pseudo-energies (Hartley information, E_n=−lnP_n) and a dimensionless parameter q that plays the role of thermodynamics β. Several relations between the entropies are presented, including the analysis of compound systems. The method is illustrated with an example.     

Electron density and Fisher information of Dirac-Fock atoms

Numerical calculations on the gradient and Laplacian forms of the position space Fisher information measure are reported using the 1-normalised Dirac-Fock densities (shape function), σ(r), for atoms H-Lr. It is shown that the difference in effective electrostatic potentials, corresponding to the gradient and the Laplacian form of Fishers' information, is completely defined by the shape function (the density per particle) at the nucleus, σ(r=0). The influence of relativistic effects on the Fisher information is recovered for the first time.     

Source coding with escort distributions and Rényi entropy bounds

We discuss the interest of escort distributions and Rényi entropy in the context of source coding. We first recall a source coding theorem by Campbell relating a generalized measure of length to the Rényi-Tsallis entropy. We show that the associated optimal codes can be obtained using considerations on escort-distributions. We propose a new family of measure of length involving escort-distributions and we show that these generalized lengths are also bounded below by the Rényi entropy. Furthermore, we obtain that the standard Shannon codes lengths are optimum for the new generalized lengths measures, whatever the entropic index. Finally, we show that there exists in this setting an interplay between standard and escort distributions.     

A simple mathematical model for anomalous diffusion via Fisher's information theory

Starting with the relative entropy based on a previously proposed entropy function , we find the corresponding Fisher's information measure. After function redefinition we then maximize the Fisher information measure with respect to the new function and obtain a differential operator that reduces to a space coordinate second derivative in the q→1 limit. We then propose a simple differential equation for anomalous diffusion and show that its solutions are a generalization of the functions in the Barenblatt-Pattle solution. We find that the mean squared displacement, up to a q-dependent constant, has a time dependence according to 〈x²〉∼K1/qt1/q, where the parameter q takes values (superdiffusion) and (subdiffusion), ∀n?1.     

The application of the transfer entropy to gappy time series

The application of the transfer entropy to gappy symbolic time series is discussed. Although the transfer entropy can fail to correctly identify the drive-response relationship, it is able to robustly detect phase relationships. Hence, it might still be of use in applications requiring the detection of changes in these relationships.     

Effects of time dependency and efficiency on information flow in financial markets

We investigated financial market data to determine which factors affect information flow between stocks. Two factors, the time dependency and the degree of efficiency, were considered in the analysis of Korean, the Japanese, the Taiwanese, the Canadian, and US market data. We found that the frequency of the significant information decreases as the time interval increases. However, no significant information flow was observed in the time series from which the temporal time correlation was removed. These results indicated that the information flow between stocks evidences time-dependency properties. Furthermore, we discovered that the difference in the degree of efficiency performs a crucial function in determining the direction of the significant information flow.     

Phase transitions in Edwards-Anderson model by means of information theory

A numeric method to characterize phase transitions is presented, explained and applied to a two-dimensional disordered system that can be thought of as a diluted ferromagnet or an Edwards-Anderson model near the ferromagnetic limit. A computer simulation is implemented to define a time series for order parameters; a file stores the time evolution of each parameter for different dilution concentrations and for a series of temperatures. These files are compressed and they reach a maximum size for temperatures in agreement with critical temperatures for the ferromagnetic/paramagnetic transition obtained by other methods. Site order parameter gives optimum results for this method based on data compression. Data compression procedures are invoked to give a qualitative explanation of this phenomenon. The advantages of this method are discussed by comparing results and procedures with two established methods: the crossing of Binder cumulants and the crossing of time autocorrelation functions. Other possible applications and extensions of this method are also mentioned.     

Generalized relative entropy in functional magnetic resonance imaging

The generalized Kullback-Leibler distance D_q (q is the Tsallis parameter) is shown to be an useful measure for analysis of functional magnetic resonance imaging (fMRI) data series. This generalized form of entropy is used to evaluate the “distance” between the probability functions p₁ and p₂ of the signal levels related to periods of stimulus and non-stimulus in event-related fMRI experiments. The probability densities of the mean distance (averaged over the N epochs of the entire experiment) are obtained through numerical simulations for different values of signal-to-noise ratio (SNR) and found to be fitted very well by Gamma distributions (χ²<0.0008) for small values of N (N<30). These distributions allow us to determine the sensitivity and specificity of the method by construction of the receiver operating characteristic (ROC) curves. The performance of the method is also investigated in terms of the parameters q and L (number of signal levels) and our results indicate that the optimum choice is q=0.8 and L=3. The entropic index q is found to exert control on both sensitivity and specificity of the method. As q (q>0) is raised, sensitivity increases but specificity decreases. Finally, the method is applied in the analysis of a real event-related fMRI motor stimulus experiment and the resulting maps show activation in primary and secondary motor brain areas.     

Tsallis distribution as a standard maximum entropy solution with ‘tail’ constraint

We show that Tsallis' distributions can be derived from the standard (Shannon) maximum entropy setting, by incorporating a constraint on the divergence between the distribution and another distribution imagined as its tail. In this setting, we find an underlying entropy which is the Rényi entropy. Furthermore, escort distributions and generalized means appear as a direct consequence of the construction. Finally, the “maximum entropy tail distribution” is identified as a Generalized Pareto Distribution.     

Linear Entropy and Entanglement in Two-Charge-Qubit Circuits

We consider a model that contains two coupled superconducting charge qubits by sharing a large Josephson junction. We examine the dynamical properties of the linear entropy of two qubits and the probability of both qubits being in an excited state. The results show that the initial mean photon number, the initial phase of the field and the relative phase of the two qubits＇ levels play an important role in the evolution of the linear entropy of the two qubits and the probability.     

Generalized statistics framework for rate distortion theory

Variational principles for the rate distortion (RD) theory in lossy compression are formulated within the ambit of the generalized nonextensive statistics of Tsallis, for values of the nonextensivity parameter satisfying 0<q<1 and q>1. Alternating minimization numerical schemes to evaluate the nonextensive RD function, are derived. Numerical simulations demonstrate the efficacy of generalized statistics RD models.     

The world according to Rényi: thermodynamics of multifractal systems

We discuss basic statistical properties of systems with multifractal structure. This is possible by extending the notion of the usual Gibbs-Shannon entropy into more general framework—Rényi’s information entropy. We address the renormalization issue for Rényi’s entropy on (multi)fractal sets and consequently show how Rényi’s parameter is connected with multifractal singularity spectrum. The maximal entropy approach then provides a passage between Rényi’s information entropy and thermodynamics of multifractals. Important issues such as Rényi’s entropy versus Tsallis-Havrda-Charvat entropy and PDF reconstruction theorem are also studied. Finally, some further speculations on a possible relevance of our approach to cosmology are discussed.     

Information erasure in quantum systems

The physical cost of information erasure is considered within a new approach that regards erasure as loss of correlation between the state of an erasable quantum system and that of an enduring “referent” system holding classical information. A physical model of information erasure built on this referential picture is described in detail, and lower bounds on entropic and energetic costs are obtained from quantum dynamics and entropic inequalities alone.     

Has the world economy reached its globalization limit?

The economy globalization measure problem is discussed. Four macroeconomic indices of twenty among the “richest” countries are examined. Four types of “distances” are calculated. Two types of networks are next constructed for each distance measure definition. It is shown that the globalization process can be best characterised by an entropy measure, based on entropy Manhattan distance. It is observed that a globalization maximum was reached during the interval 1970-2000. More recently a deglobalization process has been observed.     

Complexity and neutron star structure

We apply the statistical measure of complexity introduced by López-Ruiz, Mancini and Calbet (1995) [1] to neutron star structure. We continue the recent application of Sañudo and Pacheco (2009) [2] to white dwarfs. The interplay of gravity, the short-range nuclear force and the very short-range weak interaction shows that neutron stars, under the current theoretical framework, are ordered (low complexity) systems.     

Bandt-Pompe-Tsallis quantifier and quantum-classical transition

We concern ourselves with statistical quantifiers of semiclassical time-evolutions and their classical limit. The system of interest represents the interaction between matter and a given field. Our tool here is the so-called Permutation Entropy, evaluated by recourse to the so-called Bandt-Pompe technique, within a Tsallis scenario. We encounter that the most salient details of the quantum-classical transition are well-described, indeed, in a better fashion than that of previous approaches.     

On the way towards a generalized entropy maximization procedure

We propose a generalized entropy maximization procedure, which takes into account the generalized averaging procedures and information gain definitions underlying the generalized entropies. This novel generalized procedure is then applied to Rényi and Tsallis entropies. The generalized entropy maximization procedure for Rényi entropies results in the exponential stationary distribution asymptotically for q∈(0,1] in contrast to the stationary distribution of the inverse power law obtained through the ordinary entropy maximization procedure. Another result of the generalized entropy maximization procedure is that one can naturally obtain all the possible stationary distributions associated with the Tsallis entropies by employing either ordinary or q-generalized Fourier transforms in the averaging procedure.     

On entropy, financial markets and minority games

The paper builds upon an earlier statistical analysis of financial time series with Shannon information entropy, published in [L. Molgedey, W. Ebeling, Local order, entropy and predictability of financial time series, European Physical Journal B—Condensed Matter and Complex Systems 15/4 (2000) 733-737]. A novel generic procedure is proposed for making multistep-ahead predictions of time series by building a statistical model of entropy. The approach is first demonstrated on the chaotic Mackey-Glass time series and later applied to Japanese Yen/US dollar intraday currency data. The paper also reinterprets Minority Games [E. Moro, The minority game: An introductory guide, Advances in Condensed Matter and Statistical Physics (2004)] within the context of physical entropy, and uses models derived from minority game theory as a tool for measuring the entropy of a model in response to time series. This entropy conditional upon a model is subsequently used in place of information-theoretic entropy in the proposed multistep prediction algorithm.     

Fisher information,Rényi entropy power and quantum phase transition in the Dicke model

Fisher information, Rényi entropy power and Fisher–Rényi information product are presented for the Dicke model. There is a quantum phase transition in this quantum optical model. It is pointed out that there is an abrupt change in the Fisher information, Rényi entropy power, the Fisher, Shannon and Rényi lengths at the transition point. It is found that these quantities diverge as the characteristic length: |_λc−λ|^−1/4${|{\lambda }_c-\lambda |}^{-1/4}$ around the critical value of the coupling strength _λc${\lambda }_c$ for any value of the parameter β$\beta$.