Structural Entropy of the Stochastic Block Models

With the rapid expansion of graphs and networks and the growing magnitude of data from all areas of science, effective treatment and compression schemes of context-dependent data is extremely desirable. A particularly interesting direction is to compress the data while keeping the “structural information” only and ignoring the concrete labelings. Under this direction, Choi and Szpankowski introduced the structures (unlabeled graphs) which allowed them to compute the structural entropy of the Erdős–Rényi random graph model. Moreover, they also provided an asymptotically optimal compression algorithm that (asymptotically) achieves this entropy limit and runs in expectation in linear time. In this paper, we consider the stochastic block models with an arbitrary number of parts. Indeed, we define a partitioned structural entropy for stochastic block models, which generalizes the structural entropy for unlabeled graphs and encodes the partition information as well. We then compute the partitioned structural entropy of the stochastic block models, and provide a compression scheme that asymptotically achieves this entropy limit.     

On Rényi Permutation Entropy

Among various modifications of the permutation entropy defined as the Shannon entropy of the ordinal pattern distribution underlying a system, a variant based on Rényi entropies was considered in a few papers. This paper discusses the relatively new concept of Rényi permutation entropies in dependence of non-negative real number q parameterizing the family of Rényi entropies and providing the Shannon entropy for q=1. Its relationship to Kolmogorov–Sinai entropy and, for q=2, to the recently introduced symbolic correlation integral are touched.     

Bayesian Analysis of Dynamic Cumulative Residual Entropy for Lindley Distribution

Dynamic cumulative residual (DCR) entropy is a valuable randomness metric that may be used in survival analysis. The Bayesian estimator of the DCR Rényi entropy (DCRRéE) for the Lindley distribution using the gamma prior is discussed in this article. Using a number of selective loss functions, the Bayesian estimator and the Bayesian credible interval are calculated. In order to compare the theoretical results, a Monte Carlo simulation experiment is proposed. Generally, we note that for a small true value of the DCRRéE, the Bayesian estimates under the linear exponential loss function are favorable compared to the others based on this simulation study. Furthermore, for large true values of the DCRRéE, the Bayesian estimate under the precautionary loss function is more suitable than the others. The Bayesian estimates of the DCRRéE work well when increasing the sample size. Real-world data is evaluated for further clarification, allowing the theoretical results to be validated.     

Bell Diagonal and Werner State Generation: Entanglement,Non-Locality,Steering and Discord on the IBM Quantum Computer

We propose the first correct special-purpose quantum circuits for preparation of Bell diagonal states (BDS), and implement them on the IBM Quantum computer, characterizing and testing complex aspects of their quantum correlations in the full parameter space. Among the circuits proposed, one involves only two quantum bits but requires adapted quantum tomography routines handling classical bits in parallel. The entire class of Bell diagonal states is generated, and several characteristic indicators, namely entanglement of formation and concurrence, CHSH non-locality, steering and discord, are experimentally evaluated over the full parameter space and compared with theory. As a by-product of this work, we also find a remarkable general inequality between “quantum discord” and “asymmetric relative entropy of discord”: the former never exceeds the latter. We also prove that for all BDS the two coincide.     

Order-Stability in Complex Biological,Social, and AI-Systems from Quantum Information Theory

This paper is our attempt, on the basis of physical theory, to bring more clarification on the question “What is life?” formulated in the well-known book of Schrödinger in 1944. According to Schrödinger, the main distinguishing feature of a biosystem’s functioning is the ability to preserve its order structure or, in mathematical terms, to prevent increasing of entropy. However, Schrödinger’s analysis shows that the classical theory is not able to adequately describe the order-stability in a biosystem. Schrödinger also appealed to the ambiguous notion of negative entropy. We apply quantum theory. As is well-known, behaviour of the quantum von Neumann entropy crucially differs from behaviour of classical entropy. We consider a complex biosystem S composed of many subsystems, say proteins, cells, or neural networks in the brain, that is, S=(Si). We study the following problem: whether the compound system S can maintain “global order” in the situation of an increase of local disorder and if S can preserve the low entropy while other Si increase their entropies (may be essentially). We show that the entropy of a system as a whole can be constant, while the entropies of its parts rising. For classical systems, this is impossible, because the entropy of S cannot be less than the entropy of its subsystem Si. And if a subsystems’s entropy increases, then a system’s entropy should also increase, by at least the same amount. However, within the quantum information theory, the answer is positive. The significant role is played by the entanglement of a subsystems’ states. In the absence of entanglement, the increasing of local disorder implies an increasing disorder in the compound system S (as in the classical regime). In this note, we proceed within a quantum-like approach to mathematical modeling of information processing by biosystems—respecting the quantum laws need not be based on genuine quantum physical processes in biosystems. Recently, such modeling found numerous applications in molecular biology, genetics, evolution theory, cognition, psychology and decision making. The quantum-like model of order stability can be applied not only in biology, but also in social science and artificial intelligence.     

Entropies and IPR as Markers for a Phase Transition in a Two-Level Model for Atom–Diatomic Molecule Coexistence

A quantum phase transition (QPT) in a simple model that describes the coexistence of atoms and diatomic molecules is studied. The model, which is briefly discussed, presents a second-order ground state phase transition in the thermodynamic (or large particle number) limit, changing from a molecular condensate in one phase to an equilibrium of diatomic molecules–atoms in coexistence in the other one. The usual markers for this phase transition are the ground state energy and the expected value of the number of atoms (alternatively, the number of molecules) in the ground state. In this work, other markers for the QPT, such as the inverse participation ratio (IPR), and particularly, the Rényi entropy, are analyzed and proposed as QPT markers. Both magnitudes present abrupt changes at the critical point of the QPT.     

Alternative Entropy Measures and Generalized Khinchin–Shannon Inequalities

The Khinchin–Shannon generalized inequalities for entropy measures in Information Theory, are a paradigm which can be used to test the Synergy of the distributions of probabilities of occurrence in physical systems. The rich algebraic structure associated with the introduction of escort probabilities seems to be essential for deriving these inequalities for the two-parameter Sharma–Mittal set of entropy measures. We also emphasize the derivation of these inequalities for the special cases of one-parameter Havrda–Charvat’s, Rényi’s and Landsberg–Vedral’s entropy measures.     

A Two-Moment Inequality with Applications to Rényi Entropy and Mutual Information

This paper explores some applications of a two-moment inequality for the integral of the rth power of a function, where 0<r<1. The first contribution is an upper bound on the Rényi entropy of a random vector in terms of the two different moments. When one of the moments is the zeroth moment, these bounds recover previous results based on maximum entropy distributions under a single moment constraint. More generally, evaluation of the bound with two carefully chosen nonzero moments can lead to significant improvements with a modest increase in complexity. The second contribution is a method for upper bounding mutual information in terms of certain integrals with respect to the variance of the conditional density. The bounds have a number of useful properties arising from the connection with variance decompositions.     

A Generalization of the Concavity of Rényi Entropy Power

Recently, Savaré-Toscani proved that the Rényi entropy power of general probability densities solving the p-nonlinear heat equation in Rn is a concave function of time under certain conditions of three parameters n,p,μ , which extends Costa’s concavity inequality for Shannon’s entropy power to the Rényi entropy power. In this paper, we give a condition Φ(n,p,μ) of n,p,μ under which the concavity of the Rényi entropy power is valid. The condition Φ(n,p,μ) contains Savaré-Toscani’s condition as a special case and much more cases. Precisely, the points (n,p,μ) satisfying Savaré-Toscani’s condition consist of a two-dimensional subset of R3 , and the points satisfying the condition Φ(n,p,μ) consist a three-dimensional subset of R3 . Furthermore, Φ(n,p,μ) gives the necessary and sufficient condition in a certain sense. Finally, the conditions are obtained with a systematic approach.     

Quantum Weak Invariants: Dynamical Evolution of Fluctuations and Correlations

Weak invariants are time-dependent observables with conserved expectation values. Their fluctuations, however, do not remain constant in time. On the assumption that time evolution of the state of an open quantum system is given in terms of a completely positive map, the fluctuations monotonically grow even if the map is not unital, in contrast to the fact that monotonic increases of both the von Neumann entropy and Rényi entropy require the map to be unital. In this way, the weak invariants describe temporal asymmetry in a manner different from the entropies. A formula is presented for time evolution of the covariance matrix associated with the weak invariants in cases where the system density matrix obeys the Gorini–Kossakowski–Lindblad–Sudarshan equation.     

Refined Young Inequality and Its Application to Divergences

We give bounds on the difference between the weighted arithmetic mean and the weighted geometric mean. These imply refined Young inequalities and the reverses of the Young inequality. We also studied some properties on the difference between the weighted arithmetic mean and the weighted geometric mean. Applying the newly obtained inequalities, we show some results on the Tsallis divergence, the Rényi divergence, the Jeffreys–Tsallis divergence and the Jensen–Shannon–Tsallis divergence.     

High Dimensional Atomic States of Hydrogenic Type: Heisenberg-like and Entropic Uncertainty Measures

High dimensional atomic states play a relevant role in a broad range of quantum fields, ranging from atomic and molecular physics to quantum technologies. The D-dimensional hydrogenic system (i.e., a negatively-charged particle moving around a positively charged core under a Coulomb-like potential) is the main prototype of the physics of multidimensional quantum systems. In this work, we review the leading terms of the Heisenberg-like (radial expectation values) and entropy-like (Rényi, Shannon) uncertainty measures of this system at the limit of high D. They are given in a simple compact way in terms of the space dimensionality, the Coulomb strength and the state’s hyperquantum numbers. The associated multidimensional position–momentum uncertainty relations are also revised and compared with those of other relevant systems.     

Role of information theoretic uncertainty relations in quantum theory

Uncertainty relations based on information theory for both discrete and continuous distribution functions are briefly reviewed. We extend these results to account for (differential) Rényi entropy and its related entropy power. This allows us to find a new class of information-theoretic uncertainty relations (ITURs). The potency of such uncertainty relations in quantum mechanics is illustrated with a simple two-energy-level model where they outperform both the usual Robertson–Schrödinger uncertainty relation and Shannon entropy based uncertainty relation. In the continuous case the ensuing entropy power uncertainty relations are discussed in the context of heavy tailed wave functions and Schrödinger cat states. Again, improvement over both the Robertson–Schrödinger uncertainty principle and Shannon ITUR is demonstrated in these cases. Further salient issues such as the proof of a generalized entropy power inequality and a geometric picture of information-theoretic uncertainty relations are also discussed.     

Information Processing in the Brain as Optimal Entropy Transport: A Theoretical Approach

We consider brain activity from an information theoretic perspective. We analyze the information processing in the brain, considering the optimality of Shannon entropy transport using the Monge–Kantorovich framework. It is proposed that some of these processes satisfy an optimal transport of informational entropy condition. This optimality condition allows us to derive an equation of the Monge–Ampère type for the information flow that accounts for the branching structure of neurons via the linearization of this equation. Based on this fact, we discuss a version of Murray’s law in this context.     

Some Parameterized Quantum Midpoint and Quantum Trapezoid Type Inequalities for Convex Functions with Applications

Quantum information theory, an interdisciplinary field that includes computer science, information theory, philosophy, cryptography, and entropy, has various applications for quantum calculus. Inequalities and entropy functions have a strong association with convex functions. In this study, we prove quantum midpoint type inequalities, quantum trapezoidal type inequalities, and the quantum Simpson’s type inequality for differentiable convex functions using a new parameterized q-integral equality. The newly formed inequalities are also proven to be generalizations of previously existing inequities. Finally, using the newly established inequalities, we present some applications for quadrature formulas.     

Entropic Dynamics on Gibbs Statistical Manifolds

Entropic dynamics is a framework in which the laws of dynamics are derived as an application of entropic methods of inference. Its successes include the derivation of quantum mechanics and quantum field theory from probabilistic principles. Here, we develop the entropic dynamics of a system, the state of which is described by a probability distribution. Thus, the dynamics unfolds on a statistical manifold that is automatically endowed by a metric structure provided by information geometry. The curvature of the manifold has a significant influence. We focus our dynamics on the statistical manifold of Gibbs distributions (also known as canonical distributions or the exponential family). The model includes an “entropic” notion of time that is tailored to the system under study; the system is its own clock. As one might expect that entropic time is intrinsically directional; there is a natural arrow of time that is led by entropic considerations. As illustrative examples, we discuss dynamics on a space of Gaussians and the discrete three-state system.     

QUANTUM MECHANICAL PROBABILITIES AND GENERAL PROBABILISTIC CONSTRAINTS FOR EINSTEIN–PODOLSKY–ROSEN–BOHM EXPERIMENTS

Relativistic causality, namely, the impossibility of signaling at superluminal speeds, restricts the kinds of correlations which can occur between different parts of a composite physical system. Here we establish the basic restrictions which relativistic causality imposes on the joint probabilities involved in an experiment of the Einstein–Podolsky–Rosen–Bohm type. Quantum mechanics, on the other hand, places further restrictions beyond those required by general considerations like causality and consistency. We illustrate this fact by considering the sum of correlations involved in the CHSH inequality. Within the general framework of the CHSH inequality, we also consider the nonlocality theorem derived by Hardy, and discuss the constraints that relativistic causality, on the one hand, and quantum mechanics, on the other hand, impose on it. Finally, we derive a simple inequality which can be used to test quantum mechanics against general probabilistic theories.     

Entropy: From Thermodynamics to Information Processing

Entropy is a concept that emerged in the 19th century. It used to be associated with heat harnessed by a thermal machine to perform work during the Industrial Revolution. However, there was an unprecedented scientific revolution in the 20th century due to one of its most essential innovations, i.e., the information theory, which also encompasses the concept of entropy. Therefore, the following question is naturally raised: “what is the difference, if any, between concepts of entropy in each field of knowledge?” There are misconceptions, as there have been multiple attempts to conciliate the entropy of thermodynamics with that of information theory. Entropy is most commonly defined as “disorder”, although it is not a good analogy since “order” is a subjective human concept, and “disorder” cannot always be obtained from entropy. Therefore, this paper presents a historical background on the evolution of the term “entropy”, and provides mathematical evidence and logical arguments regarding its interconnection in various scientific areas, with the objective of providing a theoretical review and reference material for a broad audience.     

Generalized Ordinal Patterns and the KS-Entropy

Ordinal patterns classifying real vectors according to the order relations between their components are an interesting basic concept for determining the complexity of a measure-preserving dynamical system. In particular, as shown by C. Bandt, G. Keller and B. Pompe, the permutation entropy based on the probability distributions of such patterns is equal to Kolmogorov–Sinai entropy in simple one-dimensional systems. The general reason for this is that, roughly speaking, the system of ordinal patterns obtained for a real-valued “measuring arrangement” has high potential for separating orbits. Starting from a slightly different approach of A. Antoniouk, K. Keller and S. Maksymenko, we discuss the generalizations of ordinal patterns providing enough separation to determine the Kolmogorov–Sinai entropy. For defining these generalized ordinal patterns, the idea is to substitute the basic binary relation ≤ on the real numbers by another binary relation. Generalizing the former results of I. Stolz and K. Keller, we establish conditions that the binary relation and the dynamical system have to fulfill so that the obtained generalized ordinal patterns can be used for estimating the Kolmogorov–Sinai entropy.