Rényi Entropies of Multidimensional Oscillator and Hydrogenic Systems with Applications to Highly Excited Rydberg States

The various facets of the internal disorder of quantum systems can be described by means of the Rényi entropies of their single-particle probability density according to modern density functional theory and quantum information techniques. In this work, we first show the lower and upper bounds for the Rényi entropies of general and central-potential quantum systems, as well as the associated entropic uncertainty relations. Then, the Rényi entropies of multidimensional oscillator and hydrogenic-like systems are reviewed and explicitly determined for all bound stationary position and momentum states from first principles (i.e., in terms of the potential strength, the space dimensionality and the states’s hyperquantum numbers). This is possible because the associated wavefunctions can be expressed by means of hypergeometric orthogonal polynomials. Emphasis is placed on the most extreme, non-trivial cases corresponding to the highly excited Rydberg states, where the Rényi entropies can be amazingly obtained in a simple, compact, and transparent form. Powerful asymptotic approaches of approximation theory have been used when the polynomial’s degree or the weight-function parameter(s) of the Hermite, Laguerre, and Gegenbauer polynomials have large values. At present, these special states are being shown of increasing potential interest in quantum information and the associated quantum technologies, such as e.g., quantum key distribution, quantum computation, and quantum metrology.     

Causal Inference in Time Series in Terms of Rényi Transfer Entropy

Uncovering causal interdependencies from observational data is one of the great challenges of a nonlinear time series analysis. In this paper, we discuss this topic with the help of an information-theoretic concept known as Rényi’s information measure. In particular, we tackle the directional information flow between bivariate time series in terms of Rényi’s transfer entropy. We show that by choosing Rényi’s parameter α, we can appropriately control information that is transferred only between selected parts of the underlying distributions. This, in turn, is a particularly potent tool for quantifying causal interdependencies in time series, where the knowledge of “black swan” events, such as spikes or sudden jumps, are of key importance. In this connection, we first prove that for Gaussian variables, Granger causality and Rényi transfer entropy are entirely equivalent. Moreover, we also partially extend these results to heavy-tailed α-Gaussian variables. These results allow establishing a connection between autoregressive and Rényi entropy-based information-theoretic approaches to data-driven causal inference. To aid our intuition, we employed the Leonenko et al. entropy estimator and analyzed Rényi’s information flow between bivariate time series generated from two unidirectionally coupled Rössler systems. Notably, we find that Rényi’s transfer entropy not only allows us to detect a threshold of synchronization but it also provides non-trivial insight into the structure of a transient regime that exists between the region of chaotic correlations and synchronization threshold. In addition, from Rényi’s transfer entropy, we could reliably infer the direction of coupling and, hence, causality, only for coupling strengths smaller than the onset value of the transient regime, i.e., when two Rössler systems are coupled but have not yet entered synchronization.     

Non-Hermitian Generalization of Rényi Entropy

From their conception to present times, different concepts and definitions of entropy take key roles in a variety of areas from thermodynamics to information science, and they can be applied to both classical and quantum systems. Among them is the Rényi entropy. It is able to characterize various properties of classical information with a unified concise form. We focus on the quantum counterpart, which unifies the von Neumann entropy, max- and min-entropy, collision entropy, etc. It can only be directly applied to Hermitian systems because it usually requires that the density matrices is normalized. For a non-Hermitian system, the evolved density matrix may not be normalized; i.e., the trace can be larger or less than one as the time evolution. However, it is not well-defined for the Rényi entropy with a non-normalized probability distribution relevant to the density matrix of a non-Hermitian system, especially when the trace of the non-normalized density matrix is larger than one. In this work, we investigate how to describe the Rényi entropy for non-Hermitian systems more appropriately. We obtain a concisely and generalized form of α-Rényi entropy, which we extend the unified order-α from finite positive real numbers to zero and infinity. Our generalized α-Rényi entropy can be directly calculated using both of the normalized and non-normalized density matrices so that it is able to describe non-Hermitian entropy dynamics. We illustrate the necessity of our generalization by showing the differences between ours and the conventional Rényi entropy for non-Hermitian detuning two-level systems.     

From Rényi Entropy Power to Information Scan of Quantum States

In this paper, we generalize the notion of Shannon’s entropy power to the Rényi-entropy setting. With this, we propose generalizations of the de Bruijn identity, isoperimetric inequality, or Stam inequality. This framework not only allows for finding new estimation inequalities, but it also provides a convenient technical framework for the derivation of a one-parameter family of Rényi-entropy-power-based quantum-mechanical uncertainty relations. To illustrate the usefulness of the Rényi entropy power obtained, we show how the information probability distribution associated with a quantum state can be reconstructed in a process that is akin to quantum-state tomography. We illustrate the inner workings of this with the so-called “cat states”, which are of fundamental interest and practical use in schemes such as quantum metrology. Salient issues, including the extension of the notion of entropy power to Tsallis entropy and ensuing implications in estimation theory, are also briefly discussed.     

Rényi Entropy and Free Energy

The Rényi entropy is a generalization of the usual concept of entropy which depends on a parameter q. In fact, Rényi entropy is closely related to free energy. Suppose we start with a system in thermal equilibrium and then suddenly divide the temperature by q. Then the maximum amount of work the system can perform as it moves to equilibrium at the new temperature divided by the change in temperature equals the system’s Rényi entropy in its original state. This result applies to both classical and quantum systems. Mathematically, we can express this result as follows: the Rényi entropy of a system in thermal equilibrium is without the ‘q−1-derivative’ of its free energy with respect to the temperature. This shows that Rényi entropy is a q-deformation of the usual concept of entropy.     

Belavkin–Staszewski Relative Entropy,Conditional Entropy,and Mutual Information

Belavkin–Staszewski relative entropy can naturally characterize the effects of the possible noncommutativity of quantum states. In this paper, two new conditional entropy terms and four new mutual information terms are first defined by replacing quantum relative entropy with Belavkin–Staszewski relative entropy. Next, their basic properties are investigated, especially in classical-quantum settings. In particular, we show the weak concavity of the Belavkin–Staszewski conditional entropy and obtain the chain rule for the Belavkin–Staszewski mutual information. Finally, the subadditivity of the Belavkin–Staszewski relative entropy is established, i.e., the Belavkin–Staszewski relative entropy of a joint system is less than the sum of that of its corresponding subsystems with the help of some multiplicative and additive factors. Meanwhile, we also provide a certain subadditivity of the geometric Rényi relative entropy.     

A Generalization of the Concavity of Rényi Entropy Power

Recently, Savaré-Toscani proved that the Rényi entropy power of general probability densities solving the p-nonlinear heat equation in Rn is a concave function of time under certain conditions of three parameters n,p,μ , which extends Costa’s concavity inequality for Shannon’s entropy power to the Rényi entropy power. In this paper, we give a condition Φ(n,p,μ) of n,p,μ under which the concavity of the Rényi entropy power is valid. The condition Φ(n,p,μ) contains Savaré-Toscani’s condition as a special case and much more cases. Precisely, the points (n,p,μ) satisfying Savaré-Toscani’s condition consist of a two-dimensional subset of R3 , and the points satisfying the condition Φ(n,p,μ) consist a three-dimensional subset of R3 . Furthermore, Φ(n,p,μ) gives the necessary and sufficient condition in a certain sense. Finally, the conditions are obtained with a systematic approach.     

On Rényi Permutation Entropy

Among various modifications of the permutation entropy defined as the Shannon entropy of the ordinal pattern distribution underlying a system, a variant based on Rényi entropies was considered in a few papers. This paper discusses the relatively new concept of Rényi permutation entropies in dependence of non-negative real number q parameterizing the family of Rényi entropies and providing the Shannon entropy for q=1. Its relationship to Kolmogorov–Sinai entropy and, for q=2, to the recently introduced symbolic correlation integral are touched.     

Rényi Entropy,Signed Probabilities,and the Qubit

The states of the qubit, the basic unit of quantum information, are 2 × 2 positive semi-definite Hermitian matrices with trace 1. We contribute to the program to axiomatize quantum mechanics by characterizing these states in terms of an entropic uncertainty principle formulated on an eight-point phase space. We do this by employing Rényi entropy (a generalization of Shannon entropy) suitably defined for the signed phase-space probability distributions that arise in representing quantum states.     

Hypercontractive Inequalities for the Second Norm of Highly Concentrated Functions,and Mrs. Gerber’s-Type Inequalities for the Second Rényi Entropy

Let Tϵ, 0≤ϵ≤1/2, be the noise operator acting on functions on the boolean cube {0,1}n. Let f be a distribution on {0,1}n and let q>1. We prove tight Mrs. Gerber-type results for the second Rényi entropy of Tϵf which take into account the value of the qth Rényi entropy of f. For a general function f on {0,1}n we prove tight hypercontractive inequalities for the ℓ2 norm of Tϵf which take into account the ratio between ℓq and ℓ1 norms of f.     

Better Heisenberg Limits,Coherence Bounds,and Energy-Time Tradeoffs via Quantum Rényi Information

An uncertainty relation for the Rényi entropies of conjugate quantum observables is used to obtain a strong Heisenberg limit of the form RMSE≥f(α)/(〈N〉+12), bounding the root mean square error of any estimate of a random optical phase shift in terms of average photon number, where f(α) is maximised for non-Shannon entropies. Related simple yet strong uncertainty relations linking phase uncertainty to the photon number distribution, such as ΔΦ≥maxnpn, are also obtained. These results are significantly strengthened via upper and lower bounds on the Rényi mutual information of quantum communication channels, related to asymmetry and convolution, and applied to the estimation (with prior information) of unitary shift parameters such as rotation angle and time, and to obtain strong bounds on measures of coherence. Sharper Rényi entropic uncertainty relations are also obtained, including time-energy uncertainty relations for Hamiltonians with discrete spectra. In the latter case almost-periodic Rényi entropies are introduced for nonperiodic systems.     

A Two-Moment Inequality with Applications to Rényi Entropy and Mutual Information

This paper explores some applications of a two-moment inequality for the integral of the rth power of a function, where 0<r<1. The first contribution is an upper bound on the Rényi entropy of a random vector in terms of the two different moments. When one of the moments is the zeroth moment, these bounds recover previous results based on maximum entropy distributions under a single moment constraint. More generally, evaluation of the bound with two carefully chosen nonzero moments can lead to significant improvements with a modest increase in complexity. The second contribution is a method for upper bounding mutual information in terms of certain integrals with respect to the variance of the conditional density. The bounds have a number of useful properties arising from the connection with variance decompositions.     

Bayesian Analysis of Dynamic Cumulative Residual Entropy for Lindley Distribution

Dynamic cumulative residual (DCR) entropy is a valuable randomness metric that may be used in survival analysis. The Bayesian estimator of the DCR Rényi entropy (DCRRéE) for the Lindley distribution using the gamma prior is discussed in this article. Using a number of selective loss functions, the Bayesian estimator and the Bayesian credible interval are calculated. In order to compare the theoretical results, a Monte Carlo simulation experiment is proposed. Generally, we note that for a small true value of the DCRRéE, the Bayesian estimates under the linear exponential loss function are favorable compared to the others based on this simulation study. Furthermore, for large true values of the DCRRéE, the Bayesian estimate under the precautionary loss function is more suitable than the others. The Bayesian estimates of the DCRRéE work well when increasing the sample size. Real-world data is evaluated for further clarification, allowing the theoretical results to be validated.     

Rényi Entropy in Statistical Mechanics

Rényi entropy was originally introduced in the field of information theory as a parametric relaxation of Shannon (in physics, Boltzmann–Gibbs) entropy. This has also fuelled different attempts to generalise statistical mechanics, although mostly skipping the physical arguments behind this entropy and instead tending to introduce it artificially. However, as we will show, modifications to the theory of statistical mechanics are needless to see how Rényi entropy automatically arises as the average rate of change of free energy over an ensemble at different temperatures. Moreover, this notion is extended by considering distributions for isospectral, non-isothermal processes, resulting in relative versions of free energy, in which the Kullback–Leibler divergence or the relative version of Rényi entropy appear within the structure of the corrections to free energy. These generalisations of free energy recover the ordinary thermodynamic potential whenever isothermal processes are considered.     

Rényi Cross-Entropy Measures for Common Distributions and Processes with Memory

Two Rényi-type generalizations of the Shannon cross-entropy, the Rényi cross-entropy and the Natural Rényi cross-entropy, were recently used as loss functions for the improved design of deep learning generative adversarial networks. In this work, we derive the Rényi and Natural Rényi differential cross-entropy measures in closed form for a wide class of common continuous distributions belonging to the exponential family, and we tabulate the results for ease of reference. We also summarise the Rényi-type cross-entropy rates between stationary Gaussian processes and between finite-alphabet time-invariant Markov sources.     

Entropy Estimation Using a Linguistic Zipf–Mandelbrot–Li Model for Natural Sequences

Entropy estimation faces numerous challenges when applied to various real-world problems. Our interest is in divergence and entropy estimation algorithms which are capable of rapid estimation for natural sequence data such as human and synthetic languages. This typically requires a large amount of data; however, we propose a new approach which is based on a new rank-based analytic Zipf–Mandelbrot–Li probabilistic model. Unlike previous approaches, which do not consider the nature of the probability distribution in relation to language; here, we introduce a novel analytic Zipfian model which includes linguistic constraints. This provides more accurate distributions for natural sequences such as natural or synthetic emergent languages. Results are given which indicates the performance of the proposed ZML model. We derive an entropy estimation method which incorporates the linguistic constraint-based Zipf–Mandelbrot–Li into a new non-equiprobable coincidence counting algorithm which is shown to be effective for tasks such as entropy rate estimation with limited data.     

Some Useful Integral Representations for Information-Theoretic Analyses

This work is an extension of our earlier article, where a well-known integral representation of the logarithmic function was explored and was accompanied with demonstrations of its usefulness in obtaining compact, easily-calculable, exact formulas for quantities that involve expectations of the logarithm of a positive random variable. Here, in the same spirit, we derive an exact integral representation (in one or two dimensions) of the moment of a nonnegative random variable, or the sum of such independent random variables, where the moment order is a general positive non-integer real (also known as fractional moments). The proposed formula is applied to a variety of examples with an information-theoretic motivation, and it is shown how it facilitates their numerical evaluations. In particular, when applied to the calculation of a moment of the sum of a large number, n, of nonnegative random variables, it is clear that integration over one or two dimensions, as suggested by our proposed integral representation, is significantly easier than the alternative of integrating over n dimensions, as needed in the direct calculation of the desired moment.     

Error Exponents and α-Mutual Information

Over the last six decades, the representation of error exponent functions for data transmission through noisy channels at rates below capacity has seen three distinct approaches: (1) Through Gallager’s E0 functions (with and without cost constraints); (2) large deviations form, in terms of conditional relative entropy and mutual information; (3) through the α-mutual information and the Augustin–Csiszár mutual information of order α derived from the Rényi divergence. While a fairly complete picture has emerged in the absence of cost constraints, there have remained gaps in the interrelationships between the three approaches in the general case of cost-constrained encoding. Furthermore, no systematic approach has been proposed to solve the attendant optimization problems by exploiting the specific structure of the information functions. This paper closes those gaps and proposes a simple method to maximize Augustin–Csiszár mutual information of order α under cost constraints by means of the maximization of the α-mutual information subject to an exponential average constraint.