漫谈熵

熵是物理中的一个既重要又微妙的概念。文章从物理学引入熵谈起,依次讨论熵与热力学第二定律、熵的统计力学定义、熵增与基础物理理论的矛盾,以及时间箭头与玻尔兹曼大脑,最后介绍著名的黑洞熵。     

Uncertainty relations expressed by Shannon-like entropies

Besides the well-known Shannon entropy, there is a set of Shannon-like entropies which have applications in statistical and quantum physics. These entropies are functions of certain parameters and converge toward Shannon entropy when these parameters approach the value 1. We describe briefly the most important Shannon-like entropies and present their graphical representations. Their graphs look almost identical, though by superimposing them it appears that they are distinct and characteristic of each Shannon-like entropy. We try to formulate the alternative entropic uncertainty relations by means of the Shannon-like entropies and show that all of them equally well express the uncertainty principle of quantum physics.     

熵的一种通俗教法

熵如力、能量和动量一样是物理学中一个重要概念,若能用一种通俗易懂的方法设计熵的教学,对文科物理的教学有重要意义.为此本文提出了一种通俗的熵的教法,这一教法不需要学生学习热力学第二定律也可以建立熵的概念.具体教学设计如下:通过日常生活例子引入熵的概念(也就是玻尔兹曼熵),设计两个例子让学生会计算熵,通过具体问题的讨论让学生充分理解熵的意义,通过一个实例由玻尔兹曼熵引入克劳修斯熵公式,设计一个演示实验强化教学效果,将熵与环境保护联系起来融入人文情怀,最后还强调了熵计算的不同层次.教学设计完全采用基于问题学习(PBL)的教学模式.     

On the connection of the formulas for entropy and stationary distribution

As is well known in statistical physics, the stationary distribution can be obtained by maximizing entropy. We show how one can reconstruct the formula for entropy knowing the formula for the stationary distribution. A general case is discussed and some concrete physical examples are considered.     

Energy–entropy competition and the effectiveness of stochastic gradient descent in machine learning

ABSTRACT

Finding parameters that minimise a loss function is at the core of many machine learning methods. The Stochastic Gradient Descent (SGD) algorithm is widely used and delivers state-of-the-art results for many problems. Nonetheless, SGD typically cannot find the global minimum, thus its empirical effectiveness is hitherto mysterious. We derive a correspondence between parameter inference and free energy minimisation in statistical physics. The degree of undersampling plays the role of temperature. Analogous to the energy–entropy competition in statistical physics, wide but shallow minima can be optimal if the system is undersampled, as is typical in many applications. Moreover, we show that the stochasticity in the algorithm has a non-trivial correlation structure which systematically biases it towards wide minima. We illustrate our argument with two prototypical models: image classification using deep learning and a linear neural network where we can analytically reveal the relationship between entropy and out-of-sample error.     

40 Gbit/s 相干偏振复用QPSK传输中基于Manakov方程的非线性补偿

 在40 Gbit/s相干偏振复用正交相移键控（QPSK）传输系统中,为了补偿由于光纤中非线性效应引起的传输信号损伤,采用了基于Manakov方程的反向传播非线性补偿算法.传统的基于标量非线性薛定谔方程（NLSE）的反向传播算法忽略了偏振模色散（PMD）的作用,因此在偏振复用系统中不能补偿由于PMD引起的信号损伤|而基于Manakov方程的数字信号处理方法能够对PMD与克尔非线性效应的耦合作用进行补偿.从仿真与实验两个方面对此方法在40 Gbit/s相干偏振复用QPSK传输系统中的补偿效果进行了验证.结果均表明,与NLSE相比,基于Manakov方程的反向传播算法在400 km长距离QPSK传输中显示出更好的性能.在光信噪比（OSNR）为18 dB时,基于Manakov方程的反向传播算法得到的Q值与NLSE相比提高约3dB.     

熵的两种关系式等价性的直接推证

基于热力学的平稳态对应于统计物理的最概然态，运用统计物理学方法，给出了直接推证熵的两种关系等价性的一种方法，从而加强了热力学与统计物理的连贯性。     

From space-time chaos to stochastic thermodynamics

We summarize the results of several experiments that show the evolution of some scientific interests and goals of the statistical and nonlinear physics community in the last 40 years. Specifically, we present how the ideas of extending concepts of equilibrium statistical physics to out-of-equilibrium physics have been developed to characterize various phenomena such as, for example, transition to space-time chaos and glass aging. We then discuss the applications of this out-of-equilibrium thermodynamics to microsystems driven out of equilibrium either by external forces or by temperature gradients. We show that in these systems thermal fluctuations play a role and that all thermodynamics quantities, such as work, heat, and entropy fluctuate. We recall general concepts such as fluctuation theorems and fluctuation dissipation relations used to characterize the statistical properties of these small systems. We describe experiments where all these concepts have been applied and tested with high accuracy. Finally, we show how these theoretical concepts and the experiments allowed us to improve our knowledge on the connection between information and thermodynamics.     

Maximum Kolmogorov-Sinai Entropy Versus Minimum Mixing Time in Markov Chains

We establish a link between the maximization of Kolmogorov Sinai entropy (KSE) and the minimization of the mixing time for general Markov chains. Since the maximisation of KSE is analytical and easier to compute in general than mixing time, this link provides a new faster method to approximate the minimum mixing time dynamics. It could be interesting in computer sciences and statistical physics, for computations that use random walks on graphs that can be represented as Markov chains.     

熵与绝热去磁制冷的物理原理

本文从熵的观点出发,利用热力学,统计物理与量子力学理论分别从宏观与微观的角度对磁制冷的物理原理进行了初等分析。     

Holographic Entanglement Entropy of the BTZ Black Hole

We investigate quantum entanglement of gravitational configurations in 3D AdS gravity using the AdS/CFT correspondence. We derive explicit formulas for the holographic entanglement entropy (EE) of the BTZ black hole, conical singularities and regularized AdS₃. The leading term in the large temperature expansion of the holographic EE of the BTZ black hole reproduces exactly its Bekenstein-Hawking entropy S _BH , whereas the subleading term behaves as ln S _BH . We also show that the leading term of the holographic EE for the BTZ black hole can be obtained from the large temperature expansion of the partition function of a broad class of 2D CFTs on the torus. This result indicates that black hole EE is not a fundamental feature of the underlying theory of quantum gravity but emerges when the semiclassical notion of spacetime geometry is used to describe the black hole.     

Quantum games entropy

We propose the study of quantum games from the point of view of quantum information theory and statistical mechanics. Every game can be described by a density operator, the von Neumann entropy and the quantum replicator dynamics. There exists a strong relationship between game theories, information theories and statistical physics. The density operator and entropy are the bonds between these theories. The analysis we propose is based on the properties of entropy, the amount of information that a player can obtain about his opponent and a maximum or minimum entropy criterion. The natural trend of a physical system is to its maximum entropy state. The minimum entropy state is a characteristic of a manipulated system, i.e., externally controlled or imposed. There exist tacit rules inside a system that do not need to be specified or clarified and search the system equilibrium based on the collective welfare principle. The other rules are imposed over the system when one or many of its members violate this principle and maximize its individual welfare at the expense of the group.     

Entropy of three-dimensional BTZ black holes

~~Entropy of three-dimensional BTZ black holes~~     

Complexity of Fracturing in Terms of Non-Extensive Statistical Physics: From Earthquake Faults to Arctic Sea Ice Fracturing

Fracturing processes within solid Earth materials are inherently a complex phenomenon so that the underlying physics that control fracture initiation and evolution still remain elusive. However, universal scaling relations seem to apply to the collective properties of fracturing phenomena. In this article we present a statistical physics approach to fracturing based on the framework of non-extensive statistical physics (NESP). Fracturing phenomena typically present intermittency, multifractality, long-range correlations and extreme fluctuations, properties that motivate the NESP approach. Initially we provide a brief review of the NESP approach to fracturing and earthquakes and then we analyze stress and stress direction time series within Arctic sea ice. We show that such time series present large fluctuations and probability distributions with “fat” tails, which can exactly be described with the q-Gaussian distribution derived in the framework of NESP. Overall, NESP provide a consistent theoretical framework, based on the principle of entropy, for deriving the collective properties of fracturing phenomena and earthquakes.     

On the multinomial logit model

We show that the Multinomial Logit model of bounded rational choice can be derived in the same way as the Gibbs–Boltzmann distribution in statistical physics. In particular, this model describes the behavior of a thermodynamic agent (which is an agent whose utility function depends on a very large number of variables) with respect to a small subset of variables “weakly interacting” with the others. We also show that the same model is obtained if entropic control costs or information costs are introduced, in which case the temperature like parameter can be considered as the price of (negative) entropy.     

An Inverse Problem in Quantum Statistical Physics

We address the following inverse problem in quantum statistical physics: does the quantum free energy (von Neumann entropy + kinetic energy) admit a unique minimizer among the density operators having a given local density n(x)? We give a positive answer to that question, in dimension one. This enables to define rigourously the notion of local quantum equilibrium, or quantum Maxwellian, which is at the basis of recently derived quantum hydrodynamic models and quantum drift-diffusion models. We also characterize this unique minimizer, which takes the form of a global thermodynamic equilibrium (canonical ensemble) with a quantum chemical potential.     

Stochastic resonance in perspective

Summary We outline the historical development of stochastic resonance (SR), a phenomenon in which the signal and/or the signal-to-noise ratio in a nonlinear system increase with increasing intensity of noise. We discuss basic theoretical ideas explaining and describing SR, and we review some revealing experimental data that place SR within the wider context of statistical physics. We emphasize the close relationship of SR to some effects that are well known in condensed-matter physics. Paper presented at the International Workshop ?Fluctuations in Physics and Biology: Stochastic Resonance, Signal Processing and Related Phenomena?, Elba, 5–10 June 1994.     

Testing methodologies for the nonlinear analysis of causal relationships in neurovascular coupling

We investigated the use and implementation of a nonlinear methodology for establishing which changes in neurophysiological signals cause changes in the blood oxygenation level-dependent (BOLD) contrast measured in functional magnetic resonance imaging. Unlike previous analytical approaches, which used linear correlation to establish covariations between neural activity and BOLD, we propose a directed information-theoretic measure, the transfer entropy, which can elucidate even highly nonlinear causal relationships between neural activity and BOLD signal. In this study we investigated the practicality of such an analysis given the limited data samples that can be collected experimentally due to the low temporal resolution of BOLD signals. We implemented several algorithms for the estimation of transfer entropy and we tested their effectiveness using simulated local field potentials (LFPs) and BOLD data constructed to match the main statistical properties of real LFP and BOLD signals measured simultaneously in monkey primary visual cortex. We found that using the advanced methods of entropy estimation implemented and described here, a transfer entropy analysis of neurovascular coupling based on experimentally attainable data sets is feasible.     

Information and Entropy Flow in the Kalman–Bucy Filter

We investigate the information theoretic properties of Kalman–Bucy filters in continuous time, developing notions of information supply, storage and dissipation. Introducing a concept of energy, we develop a physical analogy in which the unobserved signal describes a statistical mechanical system interacting with a heat bath. The abstract universe comprising the signal and the heat bath obeys a non-increase law of entropy; however, with the introduction of partial observations, this law can be violated. The Kalman–Bucy filter behaves like a Maxwellian demon in this analogy, returning signal energy to the heat bath without causing entropy increase. This is made possible by the steady supply of new information. In a second analogy the signal and filter interact, setting up a stationary non-equilibrium state, in which energy flows between the heat bath, the signal and the filter without causing any overall entropy increase. We introduce a rate of interactive entropy flow that isolates the statistical mechanics of this flow from marginal effects. Both analogies provide quantitative examples of Landauers Principle.