Thermodynamics of computing: Entropy of nonergodic systems

Landauer discussed the minimum energy necessary for computation and stated that erasure of information is accompanied with kT ln 2/ bit of heat generation. We reconsider this problem on the basis of Clausius's equation defining the thermodynamic entropy. We show that the erasing process, involving a transition from a nonergodic to an ergodic state, is irreversible and accompanied with k ln 2/bit of entropy generation, while the heat generation occurs in a writing process. The inverse of the erasing process corresponds to spontaneous symmetry breaking from an ergodic to a nonergodic state, which induces a decrease(!) in thermodynamic entropy. Our theory is examined by a simulation of a binary device described by a Langevin equation. We argue that the so-called residual entropy of symmetry broken states, such as in ice, is not a thermodynamic quantity, even if it might be called "information entropy." (c) 2001 American Institute of Physics.     

Macroscopic Time Evolution and MaxEnt Inference for Closed Systems with Hamiltonian Dynamics

MaxEnt inference algorithm and information theory are relevant for the time evolution of macroscopic systems considered as problem of incomplete information. Two different MaxEnt approaches are introduced in this work, both applied to prediction of time evolution for closed Hamiltonian systems. The first one is based on Liouville equation for the conditional probability distribution, introduced as a strict microscopic constraint on time evolution in phase space. The conditional probability distribution is defined for the set of microstates associated with the set of phase space paths determined by solutions of Hamilton’s equations. The MaxEnt inference algorithm with Shannon’s concept of the conditional information entropy is then applied to prediction, consistently with this strict microscopic constraint on time evolution in phase space. The second approach is based on the same concepts, with a difference that Liouville equation for the conditional probability distribution is introduced as a macroscopic constraint given by a phase space average. We consider the incomplete nature of our information about microscopic dynamics in a rational way that is consistent with Jaynes’ formulation of predictive statistical mechanics, and the concept of macroscopic reproducibility for time dependent processes. Maximization of the conditional information entropy subject to this macroscopic constraint leads to a loss of correlation between the initial phase space paths and final microstates. Information entropy is the theoretic upper bound on the conditional information entropy, with the upper bound attained only in case of the complete loss of correlation. In this alternative approach to prediction of macroscopic time evolution, maximization of the conditional information entropy is equivalent to the loss of statistical correlation, and leads to corresponding loss of information. In accordance with the original idea of Jaynes, irreversibility appears as a consequence of gradual loss of information about possible microstates of the system.     

An Order Model for Infinite Classical States

In 2002 Coecke and Martin (Research Report PRG-RR-02-07, Oxford University Computing Laboratory, 2002) created a model for the finite classical and quantum states in physics. This model is based on a type of ordered set which is standard in the study of information systems. It allows the information content of its elements to be compared and measured. Their work is extended to a model for the infinite classical states. These are the states which result when an observable is applied to a quantum system. When this extended order is restricted to a finite number of coordinates, the model of Coecke and Martin is obtained. The infinite model retains many desirable aspects of the finite model, such as pure states as maximal elements and expected behavior of thermodynamic entropy. But it looses some of the important domain theoretic aspects, such as having a least element and exactness. Shannon entropy is no longer defined over the entire model and both it and thermodynamic entropy cease to be a measurements in the sense of Martin.     

Time Evolution in Macroscopic Systems. II. The Entropy

The concept of entropy in nonequilibrium macroscopic systems is investigated in the light of an extended equation of motion for the density matrix obtained in a previous study. It is found that a time-dependent information entropy can be defined unambiguously, but it is the time derivative or entropy production that governs ongoing processes in these systems. The differences in physical interpretation and thermodynamic role of entropy in equilibrium and nonequilibrium systems is emphasized and the observable aspects of entropy production are noted. A basis for nonequilibrium thermodynamics is also outlined.     

Maxwell's demon and the entropy cost of information

We present an analysis of Szilard's one-molecule Maxwell's demon, including a detailed entropy accounting, that suggests a general theory of the entropy cost of information. It is shown that the entropy of the demon increases during the expansion step, due to the decoupling of the molecule from the measurement information. It is also shown that there is an entropy symmetry between the measurement and erasure steps, whereby the two steps additivelv share a constant entropy change, but the proportion that occurs during each of the two steps is arbitrary. Therefore the measurement step may be accompanied by an entropy increase, a decrease, or no change at all, and likewise for the erasure step. Generalizing beyond the demon, decorrelation between a physical system and information about that system always causes an entropy increase in the joint system comprised of both the original system and the information. Decorrelation causes a net entropy increase in the universe unless, as in the Szilard demon, the information is used to decrease entropy elsewhere before the correlation is lost. Thus, information is thermodynamically costly precisely to the extent that it is not used to obtain work from the measured system.     

Effect of Quantum Coherence on Landauer’s Principle

Landauer’s principle provides a fundamental lower bound for energy dissipation occurring with information erasure in the quantum regime. While most studies have related the entropy reduction incorporated with the erasure to the lower bound (entropic bound), recent efforts have also provided another lower bound associated with the thermal fluctuation of the dissipated energy (thermodynamic bound). The coexistence of the two bounds has stimulated comparative studies of their properties; however, these studies were performed for systems where the time-evolution of diagonal (population) and off-diagonal (coherence) elements of the density matrix are decoupled. In this paper, we aimed to broaden the comparative study to include the influence of quantum coherence induced by the tilted system–reservoir interaction direction. By examining their dependence on the initial state of the information-bearing system, we find that the following properties of the bounds are generically held regardless of whether the influence of the coherence is present or not: the entropic bound serves as the tighter bound for a sufficiently mixed initial state, while the thermodynamic bound is tighter when the purity of the initial state is sufficiently high. The exception is the case where the system dynamics involve only phase relaxation; in this case, the two bounds coincide when the initial coherence is zero; otherwise, the thermodynamic bound serves the tighter bound. We also find the quantum information erasure inevitably accompanies constant energy dissipation caused by the creation of system–reservoir correlation, which may cause an additional source of energetic cost for the erasure.     

Fluctuations When Driving Between Nonequilibrium Steady States

Maintained by environmental fluxes, biological systems are thermodynamic processes that operate far from equilibrium without detailed-balanced dynamics. Yet, they often exhibit well defined nonequilibrium steady states (NESSs). More importantly, critical thermodynamic functionality arises directly from transitions among their NESSs, driven by environmental switching. Here, we identify the constraints on excess heat and dissipated work necessary to control a system that is kept far from equilibrium by background, uncontrolled “housekeeping” forces. We do this by extending the Crooks fluctuation theorem to transitions among NESSs, without invoking an unphysical dual dynamics. This and corresponding integral fluctuation theorems determine how much work must be expended when controlling systems maintained far from equilibrium. This generalizes thermodynamic feedback control theory, showing that Maxwellian Demons can leverage mesoscopic-state information to take advantage of the excess energetics in NESS transitions. We also generalize an approach recently used to determine the work dissipated when driving between functionally relevant configurations of an active energy-consuming complex system. Altogether, these results highlight universal thermodynamic laws that apply to the accessible degrees of freedom within the effective dynamic at any emergent level of hierarchical organization. By way of illustration, we analyze a voltage-gated sodium ion channel whose molecular conformational dynamics play a critical functional role in propagating action potentials in mammalian neuronal membranes.     

Exergy flows as bases of constructal law

How do macroscopic systems react when imposed to external forces? A recent analysis of Carnot’s theorem has pointed out that the systems do not convert all the inflow energy to work and dissipation, but that some energy will be incorporated to internal processes. These exergy flows appear as the heat exchanged with a second thermal reservoir of a thermodynamic general engine. Flows of energy from surroundings are driven by internal forces to the systemic process. Also, when the system has evolved to reach a stationary state, internal flows manifest themselves as internal processes. Here, the entropy generation extrema theorem is used to prove how the macroscopic system will react upon forces that are imposed by the external surroundings. This provides the link between the entropy generation extrema approach and the constructal law.     

Information Theory Applied to Parameter Inference of Pake Doublet Spectra

Magnetic resonance absorption lineshapes can have subtle dependencies on the model parameters that specify the lineshape. To quantify how the model parameters influence the lineshape, it is useful to study simple model systems for which analytical expressions are available. We propose that information theory is a useful tool to quantify how well model parameters may be inferred from a noisy signal. Information theory also allows us to assess the importance of missing parameters from an incomplete model. We do this by monitoring the magnitude of a partition function determined from a suitably defined probability mass function as the model parameters are varied. The optimum parameter set makes the partition function a maximum, which establishes a computable criterion for determining the best model parameter set. Given the availability of a partition function, one may define thermodynamic functions such as the entropy. The optimum parameter set in this interpretation corresponds to the state of maximum entropy. In this work, we observe that at sufficiently low signal to noise ratio, the entropy landscape has no clear maximum, while a related quantity, the Fisher information, always has a clear minimum at the optimum parameter set. The qualitative information we are able to gather from the entropy landscapes is also difficult to assess when the parameters are far from their optimum values, at least for the model system studied here.     

Maxwell’s demon and the management of ignorance in stochastic thermodynamics

It is nearly 150 years since Maxwell challenged the validity of the second law of thermodynamics by imagining a tiny creature who could sort the molecules of a gas in such a way that would decrease entropy without exerting any work. The demon has been discussed largely using thought experiments, but it has recently become possible to exert control over nanoscale systems, just as Maxwell imagined, and the status of the second law has become a more practical matter, raising the issue of how measurements manage our ignorance in a way that can be exploited. The framework of stochastic thermodynamics extends macroscopic concepts such as heat, work, entropy and irreversibility to small systems and allows us explore the matter. Some arguments against a successful demon imply a second law that can be suspended indefinitely until we dissipate energy in order to remove the records of his operations. In contrast, under stochastic thermodynamics, the demon fails because on average, more work is performed upfront in making a measurement than can be extracted by exploiting the outcome. This requires us to exclude systems and a demon that evolve under what might be termed self-sorting dynamics, and we reflect on the constraints on control that this implies while still working within a thermodynamic framework.     

Hidden Dissipation and Irreversibility in Maxwell’s Demon

Maxwell’s demon is an entity in a 150-year-old thought experiment that paradoxically appears to violate the second law of thermodynamics by reducing entropy without doing work. It has increasingly practical implications as advances in nanomachinery produce devices that push the thermodynamic limits imposed by the second law. A well-known explanation claiming that information erasure restores second law compliance fails to resolve the paradox because it assumes the second law a priori, and does not predict irreversibility. Instead, a purely mechanical resolution that does not require information theory is presented. The transport fluxes of mass, momentum, and energy involved in the demon’s operation are analyzed and show that they imply “hidden” external work and dissipation. Computing the dissipation leads to a new lower bound on entropy production by the demon. It is strictly positive in all nontrivial cases, providing a more stringent limit than the second law and implying intrinsic thermodynamic irreversibility. The thermodynamic irreversibility is linked with mechanical irreversibility resulting from the spatial asymmetry of the demon’s speed selection criteria, indicating one mechanism by which macroscopic irreversibility may emerge from microscopic dynamics.     

A Thermodynamic Approach to Measuring Entropy in a Few-Electron Nanodevice

The entropy of a system gives a powerful insight into its microscopic degrees of freedom; however, standard experimental ways of measuring entropy through heat capacity are hard to apply to nanoscale systems, as they require the measurement of increasingly small amounts of heat. Two alternative entropy measurement methods have been recently proposed for nanodevices: through charge balance measurements and transport properties. We describe a self-consistent thermodynamic framework for applying thermodynamic relations to few-electron nanodevices—small systems, where fluctuations in particle number are significant, whilst highlighting several ongoing misconceptions. We derive a relation (a consequence of a Maxwell relation for small systems), which describes both existing entropy measurement methods as special cases, while also allowing the experimentalist to probe the intermediate regime between them. Finally, we independently prove the applicability of our framework in systems with complex microscopic dynamics—those with many excited states of various degeneracies—from microscopic considerations.     

Generalized measures of quantum correlations for mixed states

The exponential speedup achieved in certain quantum algorithms based on mixed states with negligible entanglement has renewed the interest on alternative measures of quantum correlations. Here we discuss a general measure of quantum correlations for composite systems based on generalized entropic functions, defined as the minimum information loss due to a local measurement. For pure states, the present measure becomes an entanglement entropy, i.e., it reduces to the generalized entropy of the reduced state. However, for mixed states it can be nonzero in separable states, vanishing just for states diagonal in a general product basis, like the quantum discord. Quadratic measures of quantum correlations can be derived as particular cases of the present formalism. The minimum information loss due to a joint local measurement is also considered. The evaluation of these measures in a simple yet relevant case is also discussed.     

(火积)理论在热功转换过程中的应用探讨

分析和讨论了(火积)理论在热功转换过程的应用及其局限性.对Carnot循环的分析表明,Carnot循环中系统的(火积)是平衡的,但(火积)和熵之间不存在dG=T2dS这样的联系.对于一般热力学过程,分析表明,在热量传递到内可逆循环中间接对外做功时,现有的(火积)理论可用于系统的分析.讨论了热功转换过程分析中(火积)理论与熵理论的不同.分析表明,两个理论的分析角度及优化输出功的前提条件是不同的.熵产从可用能损失的角度分析热功转换过程,而(火积)理论则从热量势能消耗的角度.当输入系统的可用能给定或者输入系统的热量及热量进、出系统的热力学力给定时,熵产最小化对应于输出功最大;对于(火积)理论,则当输入系统的热量及热量进、出系统的温度给定时,最大(火积)损失对应于最大输出功.同时,它们各自均有局限性.当相应的前提条件不满足时,最大(火积)损失或最小熵产可能不与最大输出功相对应.     

量子点接触对单电子量子态的量子测量

研究了用介观量子点接触(QPC)对单电子两态和多态系统的量子测量问题.发现,在任意测量电压下,该测量问题不能用标准的Lindblad量子主方程描述.考虑了测量仪器和被测系统之间的能量交换对细致平衡关系的影响,对该问题提供了一个恰当的理论描述,并对未来的固态量子测量和量子反馈控制可能产生一定影响. 关键词： 量子测量 量子比特 细致平衡 退局域化     

Statistical mechanics of two-dimensional Euler flows and minimum enstrophy states

A simplified thermodynamic approach of the incompressible 2D Euler equation is considered based on the conservation of energy, circulation and microscopic enstrophy. Statistical equilibrium states are obtained by maximizing the Miller-Robert-Sommeria (MRS) entropy under these sole constraints. We assume that these constraints are selected by properties of forcing and dissipation. We find that the vorticity fluctuations are Gaussian while the mean flow is characterized by a linear [`(w)]-y\overline{\omega}-\psi relationship. Furthermore, we prove that the maximization of entropy at fixed energy, circulation and microscopic enstrophy is equivalent to the minimization of macroscopic enstrophy at fixed energy and circulation. This provides a justification of the minimum enstrophy principle from statistical mechanics when only the microscopic enstrophy is conserved among the infinite class of Casimir constraints. Relaxation equations towards the statistical equilibrium state are derived. These equations can serve as numerical algorithms to determine maximum entropy or minimum enstrophy states. We use these relaxation equations to study geometry induced phase transitions in rectangular domains. In particular, we illustrate with the relaxation equations the transition between monopoles and dipoles predicted by Chavanis and Sommeria [J. Fluid Mech. 314, 267 (1996)]. We take into account stable as well as metastable states and show that metastable states are robust and have negative specific heats. This is the first evidence of negative specific heats in that context. We also argue that saddle points of entropy can be long-lived and play a role in the dynamics because the system may not spontaneously generate the perturbations that destabilize them.     

Thermodynamics based on the principle of least abbreviated action: Entropy production in a network of coupled oscillators

We present some novel thermodynamic ideas based on the Maupertuis principle. By considering Hamiltonians written in terms of appropriate action-angle variables we show that thermal states can be characterized by the action variables and by their evolution in time when the system is nonintegrable. We propose dynamical definitions for the equilibrium temperature and entropy as well as an expression for the nonequilibrium entropy valid for isolated systems with many degrees of freedom. This entropy is shown to increase in the relaxation to equilibrium of macroscopic systems with short-range interactions, which constitutes a dynamical justification of the Second Law of Thermodynamics. Several examples are worked out to show that this formalism yields the right microcanonical (equilibrium) quantities. The relevance of this approach to nonequilibrium situations is illustrated with an application to a network of coupled oscillators (Kuramoto model). We provide an expression for the entropy production in this system finding that its positive value is directly related to dissipation at the steady state in attaining order through synchronization.     

On the Entropy Production of Time Series with Unidirectional Linearity

There are non-Gaussian time series that admit a causal linear autoregressive moving average (ARMA) model when regressing the future on the past, but not when regressing the past on the future. The reason is that, in the latter case, the regression residuals are not statistically independent of the regressor. In previous work, we have experimentally verified that many empirical time series indeed show such a time inversion asymmetry. For various physical systems, it is known that time-inversion asymmetries are linked to the thermodynamic entropy production in non-equilibrium states. Here we argue that unidirectional linearity is also accompanied by entropy generation. To this end, we study the dynamical evolution of a physical toy system with linear coupling to an infinite environment and show that the linearity of the dynamics is inherited by the forward-time conditional probabilities, but not by the backward-time conditionals. The reason is that the environment permanently provides particles that are in a product state before they interact with the system, but show statistical dependence afterwards. From a coarse-grained perspective, the interaction thus generates entropy. We quantitatively relate the strength of the non-linearity of the backward process to the minimal amount of entropy generation. The paper thus shows that unidirectional linearity is an indirect implication of the thermodynamic arrow of time, given that the joint dynamics of the system and its environment is linear.     

Information-theoretic limits of control

Fundamental limits on the controllability of physical systems are discussed in the light of information theory. It is shown that the second law of thermodynamics, when generalized to include information, sets absolute limits to the minimum amount of dissipation required by open-loop control. In addition, an information-theoretic analysis of control systems shows feedback control to be a zero sum game: each bit of information gathered from a dynamical system by a control device can serve to decrease the entropy of that system by at most one bit additional to the reduction of entropy attainable without such information. Consequences for the control of discrete state systems and chaotic maps are discussed.     

A Scheme for Information Erasure in a Double-Well Potential

We design an experimental scheme to realize one-bit information erasure and restoring processes by considering an overdamped colloidal particle in a double-well optical trap, which is added by a controllable laser tweezer. Using the Monte Carlo method, we simulate numerically the Langevin equation to calculate the mean work spent during the entire process and validate the entropy production fluctuation theory. Our result shows that the distribution of entropy production becomes narrow with increasing temperature and becomes stationary, represents the diminishing extent of irreversibility.