Rényi Entropy and Free Energy

The Rényi entropy is a generalization of the usual concept of entropy which depends on a parameter q. In fact, Rényi entropy is closely related to free energy. Suppose we start with a system in thermal equilibrium and then suddenly divide the temperature by q. Then the maximum amount of work the system can perform as it moves to equilibrium at the new temperature divided by the change in temperature equals the system’s Rényi entropy in its original state. This result applies to both classical and quantum systems. Mathematically, we can express this result as follows: the Rényi entropy of a system in thermal equilibrium is without the ‘q−1-derivative’ of its free energy with respect to the temperature. This shows that Rényi entropy is a q-deformation of the usual concept of entropy.     

Non-Hermitian Generalization of Rényi Entropy

From their conception to present times, different concepts and definitions of entropy take key roles in a variety of areas from thermodynamics to information science, and they can be applied to both classical and quantum systems. Among them is the Rényi entropy. It is able to characterize various properties of classical information with a unified concise form. We focus on the quantum counterpart, which unifies the von Neumann entropy, max- and min-entropy, collision entropy, etc. It can only be directly applied to Hermitian systems because it usually requires that the density matrices is normalized. For a non-Hermitian system, the evolved density matrix may not be normalized; i.e., the trace can be larger or less than one as the time evolution. However, it is not well-defined for the Rényi entropy with a non-normalized probability distribution relevant to the density matrix of a non-Hermitian system, especially when the trace of the non-normalized density matrix is larger than one. In this work, we investigate how to describe the Rényi entropy for non-Hermitian systems more appropriately. We obtain a concisely and generalized form of α-Rényi entropy, which we extend the unified order-α from finite positive real numbers to zero and infinity. Our generalized α-Rényi entropy can be directly calculated using both of the normalized and non-normalized density matrices so that it is able to describe non-Hermitian entropy dynamics. We illustrate the necessity of our generalization by showing the differences between ours and the conventional Rényi entropy for non-Hermitian detuning two-level systems.     

Boltzmann Configurational Entropy Revisited in the Framework of Generalized Statistical Mechanics

As known, a method to introduce non-conventional statistics may be realized by modifying the number of possible combinations to put particles in a collection of single-particle states. In this paper, we assume that the weight factor of the possible configurations of a system of interacting particles can be obtained by generalizing opportunely the combinatorics, according to a certain analytical function f{π}(n) of the actual number of particles present in every energy level. Following this approach, the configurational Boltzmann entropy is revisited in a very general manner starting from a continuous deformation of the multinomial coefficients depending on a set of deformation parameters {π}. It is shown that, when f{π}(n) is related to the solutions of a simple linear difference–differential equation, the emerging entropy is a scaled version, in the occupational number representation, of the entropy of degree (κ,r) known, in the framework of the information theory, as Sharma–Taneja–Mittal entropic form.     

First-Principle Derivation of Single-Photon Entropy and Maxwell–Jüttner Velocity Distribution

This work is devoted to deriving the entropy of a single photon in a beam of light from first principles. Based on the quantum processes of light–matter interaction, we find that, if the light is not in equilibrium, there are two different ways, depending on whether the photon is being added or being removed from the light, of defining the single-photon entropy of this light. However, when the light is in equilibrium at temperature T, the two definitions are equivalent and the photon entropy of this light is hν/T. From first principles, we also re-derive the Jüttner velocity distribution showing that, even without interatomic collisions, two-level atoms will relax to the state satisfying the Maxwell–Jüttner velocity distribution when they are moving in blackbody radiation fields.     

On Rényi Permutation Entropy

Among various modifications of the permutation entropy defined as the Shannon entropy of the ordinal pattern distribution underlying a system, a variant based on Rényi entropies was considered in a few papers. This paper discusses the relatively new concept of Rényi permutation entropies in dependence of non-negative real number q parameterizing the family of Rényi entropies and providing the Shannon entropy for q=1. Its relationship to Kolmogorov–Sinai entropy and, for q=2, to the recently introduced symbolic correlation integral are touched.     

Folding Free Energy Determination of an RNA Three-Way Junction Using Fluctuation Theorems

Nonequilibrium work relations and fluctuation theorems permit us to extract equilibrium information from nonequilibrium measurements. They find application in single-molecule pulling experiments where molecular free energies can be determined from irreversible work measurements by using unidirectional (e.g., Jarzynski’s equality) and bidirectional (e.g., Crooks fluctuation theorem and Bennet’s acceptance ratio (BAR)) methods. However, irreversibility and the finite number of pulls limit their applicability: the higher the dissipation, the larger the number of pulls necessary to estimate ΔG within a few kBT. Here, we revisit pulling experiments on an RNA three-way junction (3WJ) that exhibits significant dissipation and work-distribution long tails upon mechanical unfolding. While bidirectional methods are more predictive, unidirectional methods are strongly biased. We also consider a cyclic protocol that combines the forward and reverse work values to increase the statistics of the measurements. For a fixed total experimental time, faster pulling rates permit us to efficiently sample rare events and reduce the bias, compensating for the increased dissipation. This analysis provides a more stringent test of the fluctuation theorem in the large irreversibility regime.     

Causal Inference in Time Series in Terms of Rényi Transfer Entropy

Uncovering causal interdependencies from observational data is one of the great challenges of a nonlinear time series analysis. In this paper, we discuss this topic with the help of an information-theoretic concept known as Rényi’s information measure. In particular, we tackle the directional information flow between bivariate time series in terms of Rényi’s transfer entropy. We show that by choosing Rényi’s parameter α, we can appropriately control information that is transferred only between selected parts of the underlying distributions. This, in turn, is a particularly potent tool for quantifying causal interdependencies in time series, where the knowledge of “black swan” events, such as spikes or sudden jumps, are of key importance. In this connection, we first prove that for Gaussian variables, Granger causality and Rényi transfer entropy are entirely equivalent. Moreover, we also partially extend these results to heavy-tailed α-Gaussian variables. These results allow establishing a connection between autoregressive and Rényi entropy-based information-theoretic approaches to data-driven causal inference. To aid our intuition, we employed the Leonenko et al. entropy estimator and analyzed Rényi’s information flow between bivariate time series generated from two unidirectionally coupled Rössler systems. Notably, we find that Rényi’s transfer entropy not only allows us to detect a threshold of synchronization but it also provides non-trivial insight into the structure of a transient regime that exists between the region of chaotic correlations and synchronization threshold. In addition, from Rényi’s transfer entropy, we could reliably infer the direction of coupling and, hence, causality, only for coupling strengths smaller than the onset value of the transient regime, i.e., when two Rössler systems are coupled but have not yet entered synchronization.     

A Generalization of the Concavity of Rényi Entropy Power

Recently, Savaré-Toscani proved that the Rényi entropy power of general probability densities solving the p-nonlinear heat equation in Rn is a concave function of time under certain conditions of three parameters n,p,μ , which extends Costa’s concavity inequality for Shannon’s entropy power to the Rényi entropy power. In this paper, we give a condition Φ(n,p,μ) of n,p,μ under which the concavity of the Rényi entropy power is valid. The condition Φ(n,p,μ) contains Savaré-Toscani’s condition as a special case and much more cases. Precisely, the points (n,p,μ) satisfying Savaré-Toscani’s condition consist of a two-dimensional subset of R3 , and the points satisfying the condition Φ(n,p,μ) consist a three-dimensional subset of R3 . Furthermore, Φ(n,p,μ) gives the necessary and sufficient condition in a certain sense. Finally, the conditions are obtained with a systematic approach.     

Which Physical Quantity Deserves the Name “Quantity of Heat”?

“What is heat?” was the title of a 1954 article by Freeman J. Dyson, published in Scientific American. Apparently, it was appropriate to ask this question at that time. The answer is given in the very first sentence of the article: heat is disordered energy. We will ask the same question again, but with a different expectation for its answer. Let us imagine that all the thermodynamic knowledge is already available: both the theory of phenomenological thermodynamics and that of statistical thermodynamics, including quantum statistics, but that the term “heat” has not yet been attributed to any of the variables of the theory. With the question “What is heat?” we now mean: which of the physical quantities deserves this name? There are several candidates: the quantities Q, H, Etherm and S. We can then formulate a desideratum, or a profile: What properties should such a measure of the quantity or amount of heat ideally have? Then, we evaluate all the candidates for their suitability. It turns out that the winner is the quantity S, which we know by the name of entropy. In the second part of the paper, we examine why entropy has not succeeded in establishing itself as a measure for the amount of heat, and we show that there is a real chance today to make up for what was missed.     

A Quantitative Comparison between Shannon and Tsallis–Havrda–Charvat Entropies Applied to Cancer Outcome Prediction

In this paper, we propose to quantitatively compare loss functions based on parameterized Tsallis–Havrda–Charvat entropy and classical Shannon entropy for the training of a deep network in the case of small datasets which are usually encountered in medical applications. Shannon cross-entropy is widely used as a loss function for most neural networks applied to the segmentation, classification and detection of images. Shannon entropy is a particular case of Tsallis–Havrda–Charvat entropy. In this work, we compare these two entropies through a medical application for predicting recurrence in patients with head–neck and lung cancers after treatment. Based on both CT images and patient information, a multitask deep neural network is proposed to perform a recurrence prediction task using cross-entropy as a loss function and an image reconstruction task. Tsallis–Havrda–Charvat cross-entropy is a parameterized cross-entropy with the parameter α . Shannon entropy is a particular case of Tsallis–Havrda–Charvat entropy for α=1 . The influence of this parameter on the final prediction results is studied. In this paper, the experiments are conducted on two datasets including in total 580 patients, of whom 434 suffered from head–neck cancers and 146 from lung cancers. The results show that Tsallis–Havrda–Charvat entropy can achieve better performance in terms of prediction accuracy with some values of α .     

Steady-State Thermodynamics of a Cascaded Collision Model

We study the steady-state thermodynamics of a cascaded collision model where two subsystems S1 and S2 collide successively with an environment R in the cascaded fashion. We first formulate general expressions of thermodynamics quantities and identify the nonlocal forms of work and heat that result from cascaded interactions of the system with the common environment. Focusing on a concrete system of two qubits, we then show that, to be able to unidirectionally influence the thermodynamics of S2, the former interaction of S1−R should not be energy conserving. We finally demonstrate that the steady-state coherence generated in the cascaded model is a kind of useful resource in extracting work, quantified by ergotropy, from the system. Our results provide a comprehensive understanding on the thermodynamics of the cascaded model and a possible way to achieve the unidirectional control on the thermodynamics process in the steady-state regime.     

Relationship between Continuum of Hurst Exponents of Noise-like Time Series and the Cantor Set

In this paper, we have modified the Detrended Fluctuation Analysis (DFA) using the ternary Cantor set. We propose a modification of the DFA algorithm, Cantor DFA (CDFA), which uses the Cantor set theory of base 3 as a scale for segment sizes in the DFA algorithm. An investigation of the phenomena generated from the proof using real-world time series based on the theory of the Cantor set is also conducted. This new approach helps reduce the overestimation problem of the Hurst exponent of DFA by comparing it with its inverse relationship with α of the Truncated Lévy Flight (TLF). CDFA is also able to correctly predict the memory behavior of time series.     

Hypercontractive Inequalities for the Second Norm of Highly Concentrated Functions,and Mrs. Gerber’s-Type Inequalities for the Second Rényi Entropy

Let Tϵ, 0≤ϵ≤1/2, be the noise operator acting on functions on the boolean cube {0,1}n. Let f be a distribution on {0,1}n and let q>1. We prove tight Mrs. Gerber-type results for the second Rényi entropy of Tϵf which take into account the value of the qth Rényi entropy of f. For a general function f on {0,1}n we prove tight hypercontractive inequalities for the ℓ2 norm of Tϵf which take into account the ratio between ℓq and ℓ1 norms of f.     

The Longitudinal Plasma Modes of κ-Deformed Kaniadakis Distributed Plasmas Carrying Orbital Angular Momentum

Based on plasma kinetic theory, the dispersion and Landau damping of Langmuir and ion-acoustic waves carrying finite orbital angular momentum (OAM) were investigated in the κ-deformed Kaniadakis distributed plasma system. The results showed that the peculiarities of the investigated subjects relied on the deformation parameter κ and OAM parameter η. For both Langmuir and ion-acoustic waves, dispersion was enhanced with increased κ, while the Landau damping was suppressed. Conversely, both the dispersion and Landau damping were depressed by OAM. Moreover, the results coincided with the straight propagating plane waves in a Maxwellian plasma system when κ=0 and η→∞. It was expected that the present results would give more insight into the trapping and transportation of plasma particles and energy.     

ϕ-Informational Measures: Some Results and Interrelations

In this paper, we focus on extended informational measures based on a convex function ϕ: entropies, extended Fisher information, and generalized moments. Both the generalization of the Fisher information and the moments rely on the definition of an escort distribution linked to the (entropic) functional ϕ. We revisit the usual maximum entropy principle—more precisely its inverse problem, starting from the distribution and constraints, which leads to the introduction of state-dependent ϕ-entropies. Then, we examine interrelations between the extended informational measures and generalize relationships such the Cramér–Rao inequality and the de Bruijn identity in this broader context. In this particular framework, the maximum entropy distributions play a central role. Of course, all the results derived in the paper include the usual ones as special cases.