Physical information entropy and probability Shannon entropy

In a previous work by one of us (R. Urigu) concerning open quantum systems it was remarked that in processes of the type , when evaluating the information entropy of the environment as the Shannon entropy of the outcome probabilities in the channels , the total information entropy may decrease. We remark here that this decrease is easily excluded by requiring a condition of quantum modelizability of the environment even with respect to Shannon entropy (“cybernetic interpretability” of the environment). Further conditions on the quantum model of the environment are defined (“maximal observability” and “Boolean interpretability”), which are proved to be equivalent, and it turns out that, once satisfied in one model, they also are in any model with pure initial state; furthermore, these conditions turn out to be equivalent to the condition that the process consists of pure operations of the first kind. The relevance to the concept of macroscopicity and to the “von Neumann chain” is discussed.     

The time derivative of information gain as excess entropy production

An information gain depending on two nonequilibrium coarse-grained statistical operators is discussed. The relation between the time derivative of information gain and excess entropy production is derived. Prigogine's stability criterion is expressed by means of the information gain. It is shown in the domain of linear nonequilibrium thermodynamics that zero time derivative of information gain corresponds to a minimum of entropy production and K theorem can be formulated.     

信息熵、玻尔兹曼熵以及克劳修斯熵之间的关系--兼论玻尔兹曼熵和克劳修斯熵是否等价

论述了信息熵、玻尔兹曼熵以及克劳修斯熵之间的关系;由不涉及具体系统的方法从玻尔兹曼关系、信息熵推导出了克劳修斯熵的表达式;指出玻尔兹曼熵与克劳修斯熵不是等价关系,而是玻尔兹曼熵包含克劳修斯熵,信息熵又包含玻尔兹曼熵。     

Entropy, entropy flux and entropy rate of granular materials

The aim of this work is to analyze the entropy, entropy flux and entropy rate of granular materials within the frameworks of the Boltzmann equation and continuum thermodynamics. It is shown that the entropy inequality for a granular gas that follows from the Boltzmann equation differs from the one of a simple fluid due to the presence of a term which can be identified as the entropy density rate. From the knowledge of a non-equilibrium distribution function-valid for processes closed to equilibrium-it is obtained that the entropy density rate is proportional to the internal energy density rate divided by the temperature, while the entropy flux is equal to the heat flux vector divided by the temperature. A thermodynamic theory of a granular material is also developed whose objective is the determination of the basic fields of mass density, momentum density and internal energy density. The constitutive laws are restricted by the principle of material frame indifference and by the entropy principle. Through the exploitation of the entropy principle with Lagrange multipliers, it is shown that the results obtained from the kinetic theory for granular gases concerning the entropy density rate and entropy flux are valid in general for processes close to equilibrium of granular materials, where linearized constitutive equations hold.     

Irreversibility, entropy and incomplete information

The open system has been proved to be a system with perfect accessibility represented as a probability space in which is defined a PA-measure. But, the PA-measure is not yet known; consequently, it is difficult to develop the statistical thermodynamics for an irreversible system. Here its integral expression is obtained in order to its use in the statistical thermodynamic analysis of the complex and irreversible systems.     

The Tsallis entropy of natural information

Estimating the information contained in natural data, such as electroencephalography data, is unusually difficult because the relationship between the physical data and the information that it encodes is unknown. This unknown relationship is often called the encoding problem. The present work provides a solution to this problem by deriving a method to estimate the Tsallis entropy in natural data. The method is based on two findings. The first finding is that the physical instantiation of any information event, that is, the physical occurrence of a symbol of information, must begin and end at a discontinuity or critical point (maximum, minimum, or saddle point) in the data. The second finding is that, in certain data types such as the encephalogram (EEG), the variance within of an EEG waveform event is directly proportional to its probability of occurrence.These two outcomes yield two results. The first is the easy binning of data into separate information events. The second is the ability to estimate probabilities in two ways: frequency counting and computing the variance within of an EEG waveform. These results are used to derive a linear estimator of the Tsallis entropy functional, allowing it to be estimated without deducing the encoding.This method for estimating the Tsallis entropy is first used to estimate the information in simple signals. The amount of information estimated is highly accurate. The method is then applied to two problems in electroencephalography. The first is distinguishing normal aging from very early Alzheimer's disease (mild cognitive impairment), and the second is medication monitoring of Alzheimer's disease treatment. The former is done with an accuracy of 92% and the latter with an accuracy of 91%. This detection accuracy is the highest published accuracy in the literature, which suggests that this method for Tsallis entropy estimation is both accurate and useful.     

Symbolic transfer entropy rate is equal to transfer entropy rate for bivariate finite-alphabet stationary ergodic Markov processes

Transfer entropy is a measure of the magnitude and the direction of information flow between jointly distributed stochastic processes. In recent years, its permutation analogues are considered in the literature to estimate the transfer entropy by counting the number of occurrences of orderings of values, not the values themselves. It has been suggested that the method of permutation is easy to implement, computationally low cost and robust to noise when applying to real world time series data. In this paper, we initiate a theoretical treatment of the corresponding rates. In particular, we consider the transfer entropy rate and its permutation analogue, the symbolic transfer entropy rate, and show that they are equal for any bivariate finite-alphabet stationary ergodic Markov process. This result is an illustration of the duality method introduced in [T. Haruna, K. Nakajima, Physica D 240, 1370 (2011)]. We also discuss the relationship among the transfer entropy rate, the time-delayed mutual information rate and their permutation analogues.     

Stochastic operators,information, and entropy

For a stochastic operatorU on andL ₁-space, i.eU is linear, positive, and norm preserving on the positive cone ofL ₁, it is shown thatU decreases relative information between two nonnegativeL ₁-functions. Furthermore it is shown that the following properties ofU are closely related:U is energy decreasing (energy preserving),U isH-decreasing, whereH is Boltzmann'sH-functional, and the Maxwell distributions are fixed points ofU.     

On the notions of entropy and information

In this review article we look for some of the historical reasons for the “identification” of the information-theoretical and thermodynamic entropy concepts. We also discuss the Baron-Jauch entropy concept and explicitly show that, for classical systems in thermodynamic equilibrium, there exists a very simple connection between this general definition and the ordinary experimental entropy.     

On entropy and information of quantum states

Shannon entropy and information are applied to study the properties of quantum states of a system in the probability representation of quantum mechanics. Examples of spin states and mixed Gaussian states of the two-mode system are considered. The relationship between the new entropy and the von Neumann entropy is reviewed. Two tomographic maps are considered within the framework of the star-product quantization. The explicit expression of tomographic entropy associated with photon-number tomogram of the two-mode state of photons is obtained in terms of Hermite polynomials of four variables. Based on a contribution to the International Conference “New Trends in Quantum Mechanics. Fundamental Aspects and Applications” (Palermo, Italy, November 2005).     

Von Neumann entropy and entropy squeezing of a two-level atom and the superposition of squeezed displaced fock states

The work presents some aspects of studying the two-level atom in a Kerr media interacting with a single-mode electromagnetic field, which is the superposition of squeezed displaced Fock states. We discuss the effect of the new field on quantum entropy and the entanglement of the atom-field system in the two-photon process. The exact results are employed to perform a careful investigation of the temporal entropy evolution. The position and momentum entropy squeezing of the optical field are investigated. The general conclusions obtained are illustrated by numerical results.     

Minimum entropy production and logarithmic rate equations

A system interacting with a heat bath and radiation is considered. It is assumed that the steady state is exactly characterized by the principle of minimum entropy production. From this, the general form of the equations for the time rate of change of the probabilities of the states is derived and the rate equations are shown to be nonlinear and to involve the differences of the logarithms of the probabilities. Some properties of these equations are discussed and the specific cases of two- and three-state subsystems are considered and compared with results obtained from the usual linear rate equations.     

Von Neumann entropy in a Rashba-Dresselhaus nanodot; dynamical electronic spin-orbit entanglement

The main purpose of the present article is to report the characteristics of von Neumann entropy, thereby, the electronic hybrid entanglement, in the heterojunction of two semiconductors, with due attention to the Rashba and Dresselhaus spin-orbit interactions. To this end, we cast the von Neumann entropy in terms of spin polarization and compute its time evolution; with a vast span of applications. It is assumed that gate potentials are applied to the heterojunction, providing a two dimensional parabolic confining potential (forming an isotropic nanodot at the junction), as well as means of controlling the spin-orbit couplings. The spin degeneracy is also removed, even at electronic zero momentum, by the presence of an external magnetic field which, in turn, leads to the appearance of Landau states. We then proceed by computing the time evolution of the corresponding von Neumann entropy from a separable (spin-polarized) initial state. The von Neumann entropy, as we show, indicates that electronic hybrid entanglement does occur between spin and two-dimensional Landau levels. Our results also show that von Neumann entropy, as well as the degree of spin-orbit entanglement, periodically collapses and revives. The characteristics of such behavior; period, amplitude, etc., are shown to be determined from the controllable external agents. Moreover, it is demonstrated that the phenomenon of collapse-revivals’ in the behavior of von Neumann entropy, equivalently, electronic hybrid entanglement, is accompanied by plateaus (of great importance in quantum computation schemes) whose durations are, again, controlled by the external elements. Along these lines, we also make a comparison between effects of the two spin-orbit couplings on the entanglement (von Neumann entropy) characteristics. The finer details of the electronic hybrid entanglement, which may be easily verified through spin polarization measurements, are also accreted and discussed. The novel results of the present article, with potent applications in the field of quantum information processing, provide a deeper understanding of the electronic von Neumann entropy and hybrid entanglement that occurs in two-dimensional nanodots.     

Fisher-Rényi entropy product and information plane

Connection between Fisher information and Rényi entropy has been established. This link allows us to define the Fisher-Rényi information plane and an entropic product in terms of these quantities. New Rényi uncertainty relations are obtained for single particle densities of many particle systems in position-momentum conjugate spaces.     

Uncertainty relations for information entropy in wave mechanics

New uncertainty relations in quantum mechanics are derived. They express restrictions imposed by quantum theory on probability distributions of canonically conjugate variables in terms of corresponding information entropies. The Heisenberg uncertainty relation follows from those inequalities and so does the Gross-Nelson inequality.     

Maxwell's demon and the entropy cost of information

We present an analysis of Szilard's one-molecule Maxwell's demon, including a detailed entropy accounting, that suggests a general theory of the entropy cost of information. It is shown that the entropy of the demon increases during the expansion step, due to the decoupling of the molecule from the measurement information. It is also shown that there is an entropy symmetry between the measurement and erasure steps, whereby the two steps additivelv share a constant entropy change, but the proportion that occurs during each of the two steps is arbitrary. Therefore the measurement step may be accompanied by an entropy increase, a decrease, or no change at all, and likewise for the erasure step. Generalizing beyond the demon, decorrelation between a physical system and information about that system always causes an entropy increase in the joint system comprised of both the original system and the information. Decorrelation causes a net entropy increase in the universe unless, as in the Szilard demon, the information is used to decrease entropy elsewhere before the correlation is lost. Thus, information is thermodynamically costly precisely to the extent that it is not used to obtain work from the measured system.     

Variance estimators for the Lempel-Ziv entropy rate estimator

Lempel and Ziv's 1976 algorithm provides an easy-to-compute way to automatically estimate the entropy rate for symbolic time series, requiring no free parameters. Here we derive an analytical variance estimate for the Lempel-Ziv entropy rate estimator that is easily computable from observations with negligible extra effort beyond the entropy rate itself, and compare to another procedure, a time-series-based bootstrap method. These provide a justified "error bar" quantifying the size of expected fluctuations on the estimate itself, given by the single time series of symbols.     

Analysis of systems based on entropy and information characteristics

An approach is proposed for studying various systems based on the concept of entropy potentials of their parameters. In this approach, any state of a system can be described quantitatively using the values of entropy potentials of corresponding parameters. With such an approach, the states and the evolution processes in the systems being analyzed can be mapped onto the coordinate space of the “information field” of the system, which makes it possible to simplify the solution of a number of applied problems.