Monte Carlo Simulation of Stochastic Differential Equation to Study Information Geometry

Information Geometry is a useful tool to study and compare the solutions of a Stochastic Differential Equations (SDEs) for non-equilibrium systems. As an alternative method to solving the Fokker–Planck equation, we propose a new method to calculate time-dependent probability density functions (PDFs) and to study Information Geometry using Monte Carlo (MC) simulation of SDEs. Specifically, we develop a new MC SDE method to overcome the challenges in calculating a time-dependent PDF and information geometric diagnostics and to speed up simulations by utilizing GPU computing. Using MC SDE simulations, we reproduce Information Geometric scaling relations found from the Fokker–Planck method for the case of a stochastic process with linear and cubic damping terms. We showcase the advantage of MC SDE simulation over FPE solvers by calculating unequal time joint PDFs. For the linear process with a linear damping force, joint PDF is found to be a Gaussian. In contrast, for the cubic process with a cubic damping force, joint PDF exhibits a bimodal structure, even in a stationary state. This suggests a finite memory time induced by a nonlinear force. Furthermore, several power-law scalings in the characteristics of bimodal PDFs are identified and investigated.     

Extending Quantum Probability from Real Axis to Complex Plane

Probability is an important question in the ontological interpretation of quantum mechanics. It has been discussed in some trajectory interpretations such as Bohmian mechanics and stochastic mechanics. New questions arise when the probability domain extends to the complex space, including the generation of complex trajectory, the definition of the complex probability, and the relation of the complex probability to the quantum probability. The complex treatment proposed in this article applies the optimal quantum guidance law to derive the stochastic differential equation governing a particle’s random motion in the complex plane. The probability distribution ρc(t,x,y) of the particle’s position over the complex plane z=x+iy is formed by an ensemble of the complex quantum random trajectories, which are solved from the complex stochastic differential equation. Meanwhile, the probability distribution ρc(t,x,y) is verified by the solution of the complex Fokker–Planck equation. It is shown that quantum probability |Ψ|2 and classical probability can be integrated under the framework of complex probability ρc(t,x,y), such that they can both be derived from ρc(t,x,y) by different statistical ways of collecting spatial points.     

Arbitrary-Order Finite-Time Corrections for the Kramers–Moyal Operator

With the aim of improving the reconstruction of stochastic evolution equations from empirical time-series data, we derive a full representation of the generator of the Kramers–Moyal operator via a power-series expansion of the exponential operator. This expansion is necessary for deriving the different terms in a stochastic differential equation. With the full representation of this operator, we are able to separate finite-time corrections of the power-series expansion of arbitrary order into terms with and without derivatives of the Kramers–Moyal coefficients. We arrive at a closed-form solution expressed through conditional moments, which can be extracted directly from time-series data with a finite sampling intervals. We provide all finite-time correction terms for parametric and non-parametric estimation of the Kramers–Moyal coefficients for discontinuous processes which can be easily implemented—employing Bell polynomials—in time-series analyses of stochastic processes. With exemplary cases of insufficiently sampled diffusion and jump-diffusion processes, we demonstrate the advantages of our arbitrary-order finite-time corrections and their impact in distinguishing diffusion and jump-diffusion processes strictly from time-series data.     

Kinetic Theory and Memory Effects of Homogeneous Inelastic Granular Gases under Nonlinear Drag

We study a dilute granular gas immersed in a thermal bath made of smaller particles with masses not much smaller than the granular ones in this work. Granular particles are assumed to have inelastic and hard interactions, losing energy in collisions as accounted by a constant coefficient of normal restitution. The interaction with the thermal bath is modeled by a nonlinear drag force plus a white-noise stochastic force. The kinetic theory for this system is described by an Enskog–Fokker–Planck equation for the one-particle velocity distribution function. To get explicit results of the temperature aging and steady states, Maxwellian and first Sonine approximations are developed. The latter takes into account the coupling of the excess kurtosis with the temperature. Theoretical predictions are compared with direct simulation Monte Carlo and event-driven molecular dynamics simulations. While good results for the granular temperature are obtained from the Maxwellian approximation, a much better agreement, especially as inelasticity and drag nonlinearity increase, is found when using the first Sonine approximation. The latter approximation is, additionally, crucial to account for memory effects such as Mpemba and Kovacs-like ones.     

Medium Entropy Reduction and Instability in Stochastic Systems with Distributed Delay

Many natural and artificial systems are subject to some sort of delay, which can be in the form of a single discrete delay or distributed over a range of times. Here, we discuss the impact of this distribution on (thermo-)dynamical properties of time-delayed stochastic systems. To this end, we study a simple classical model with white and colored noise, and focus on the class of Gamma-distributed delays which includes a variety of distinct delay distributions typical for feedback experiments and biological systems. A physical application is a colloid subject to time-delayed feedback control, which is, in principle, experimentally realizable by co-moving optical traps. We uncover several unexpected phenomena in regard to the system’s linear stability and its thermodynamic properties. First, increasing the mean delay time can destabilize or stabilize the process, depending on the distribution of the delay. Second, for all considered distributions, the heat dissipated by the controlled system (e.g., the colloidal particle) can become negative, which implies that the delay force extracts energy and entropy of the bath. As we show here, this refrigerating effect is particularly pronounced for exponential delay. For a specific non-reciprocal realization of a control device, we find that the entropic costs, measured by the total entropy production of the system plus controller, are the lowest for exponential delay. The exponential delay further yields the largest stable parameter regions. In this sense, exponential delay represents the most effective and robust type of delayed feedback.     

Rate of Entropy Production in Stochastic Mechanical Systems

Entropy production in stochastic mechanical systems is examined here with strict bounds on its rate. Stochastic mechanical systems include pure diffusions in Euclidean space or on Lie groups, as well as systems evolving on phase space for which the fluctuation-dissipation theorem applies, i.e., return-to-equilibrium processes. Two separate ways for ensembles of such mechanical systems forced by noise to reach equilibrium are examined here. First, a restorative potential and damping can be applied, leading to a classical return-to-equilibrium process wherein energy taken out by damping can balance the energy going in from the noise. Second, the process evolves on a compact configuration space (such as random walks on spheres, torsion angles in chain molecules, and rotational Brownian motion) lead to long-time solutions that are constant over the configuration space, regardless of whether or not damping and random forcing balance. This is a kind of potential-free equilibrium distribution resulting from topological constraints. Inertial and noninertial (kinematic) systems are considered. These systems can consist of unconstrained particles or more complex systems with constraints, such as rigid-bodies or linkages. These more complicated systems evolve on Lie groups and model phenomena such as rotational Brownian motion and nonholonomic robotic systems. In all cases, it is shown that the rate of entropy production is closely related to the appropriate concept of Fisher information matrix of the probability density defined by the Fokker–Planck equation. Classical results from information theory are then repurposed to provide computable bounds on the rate of entropy production in stochastic mechanical systems.     

Stochastic Order and Generalized Weighted Mean Invariance

In this paper, we present order invariance theoretical results for weighted quasi-arithmetic means of a monotonic series of numbers. The quasi-arithmetic mean, or Kolmogorov–Nagumo mean, generalizes the classical mean and appears in many disciplines, from information theory to physics, from economics to traffic flow. Stochastic orders are defined on weights (or equivalently, discrete probability distributions). They were introduced to study risk in economics and decision theory, and recently have found utility in Monte Carlo techniques and in image processing. We show in this paper that, if two distributions of weights are ordered under first stochastic order, then for any monotonic series of numbers their weighted quasi-arithmetic means share the same order. This means for instance that arithmetic and harmonic mean for two different distributions of weights always have to be aligned if the weights are stochastically ordered, this is, either both means increase or both decrease. We explore the invariance properties when convex (concave) functions define both the quasi-arithmetic mean and the series of numbers, we show its relationship with increasing concave order and increasing convex order, and we observe the important role played by a new defined mirror property of stochastic orders. We also give some applications to entropy and cross-entropy and present an example of multiple importance sampling Monte Carlo technique that illustrates the usefulness and transversality of our approach. Invariance theorems are useful when a system is represented by a set of quasi-arithmetic means and we want to change the distribution of weights so that all means evolve in the same direction.     

Belavkin–Staszewski Relative Entropy,Conditional Entropy,and Mutual Information

Belavkin–Staszewski relative entropy can naturally characterize the effects of the possible noncommutativity of quantum states. In this paper, two new conditional entropy terms and four new mutual information terms are first defined by replacing quantum relative entropy with Belavkin–Staszewski relative entropy. Next, their basic properties are investigated, especially in classical-quantum settings. In particular, we show the weak concavity of the Belavkin–Staszewski conditional entropy and obtain the chain rule for the Belavkin–Staszewski mutual information. Finally, the subadditivity of the Belavkin–Staszewski relative entropy is established, i.e., the Belavkin–Staszewski relative entropy of a joint system is less than the sum of that of its corresponding subsystems with the help of some multiplicative and additive factors. Meanwhile, we also provide a certain subadditivity of the geometric Rényi relative entropy.     

On Rényi Permutation Entropy

Among various modifications of the permutation entropy defined as the Shannon entropy of the ordinal pattern distribution underlying a system, a variant based on Rényi entropies was considered in a few papers. This paper discusses the relatively new concept of Rényi permutation entropies in dependence of non-negative real number q parameterizing the family of Rényi entropies and providing the Shannon entropy for q=1. Its relationship to Kolmogorov–Sinai entropy and, for q=2, to the recently introduced symbolic correlation integral are touched.     

Clifford Wavelet Entropy for Fetal ECG Extraction

Analysis of the fetal heart rate during pregnancy is essential for monitoring the proper development of the fetus. Current fetal heart monitoring techniques lack the accuracy in fetal heart rate monitoring and features acquisition, resulting in diagnostic medical issues. The challenge lies in the extraction of the fetal ECG from the mother ECG during pregnancy. This approach has the advantage of being a reliable and non-invasive technique. In the present paper, a wavelet/multiwavelet method is proposed to perfectly extract the fetal ECG parameters from the abdominal mother ECG. In a first step, due to the wavelet/mutiwavelet processing, a denoising procedure is applied to separate the noised parts from the denoised ones. The denoised signal is assumed to be a mixture of both the MECG and the FECG. One of the well-known measures of accuracy in information processing is the concept of entropy. In the present work, a wavelet/multiwavelet Shannon-type entropy is constructed and applied to evaluate the order/disorder of the extracted FECG signal. The experimental results apply to a recent class of Clifford wavelets constructed in Arfaoui, et al. J. Math. Imaging Vis. 2020, 62, 73–97, and Arfaoui, et al. Acta Appl. Math. 2020, 170, 1–35. Additionally, classical Haar–Faber–Schauder wavelets are applied for the purpose of comparison. Two main well-known databases have been applied, the DAISY database and the CinC Challenge 2013 database. The achieved accuracy over the test databases resulted in Se = 100%, PPV = 100% for FECG extraction and peak detection.     

Generalised Geometric Brownian Motion: Theory and Applications to Option Pricing

Classical option pricing schemes assume that the value of a financial asset follows a geometric Brownian motion (GBM). However, a growing body of studies suggest that a simple GBM trajectory is not an adequate representation for asset dynamics, due to irregularities found when comparing its properties with empirical distributions. As a solution, we investigate a generalisation of GBM where the introduction of a memory kernel critically determines the behaviour of the stochastic process. We find the general expressions for the moments, log-moments, and the expectation of the periodic log returns, and then obtain the corresponding probability density functions using the subordination approach. Particularly, we consider subdiffusive GBM (sGBM), tempered sGBM, a mix of GBM and sGBM, and a mix of sGBMs. We utilise the resulting generalised GBM (gGBM) in order to examine the empirical performance of a selected group of kernels in the pricing of European call options. Our results indicate that the performance of a kernel ultimately depends on the maturity of the option and its moneyness.     

Entropy Estimation Using a Linguistic Zipf–Mandelbrot–Li Model for Natural Sequences

Entropy estimation faces numerous challenges when applied to various real-world problems. Our interest is in divergence and entropy estimation algorithms which are capable of rapid estimation for natural sequence data such as human and synthetic languages. This typically requires a large amount of data; however, we propose a new approach which is based on a new rank-based analytic Zipf–Mandelbrot–Li probabilistic model. Unlike previous approaches, which do not consider the nature of the probability distribution in relation to language; here, we introduce a novel analytic Zipfian model which includes linguistic constraints. This provides more accurate distributions for natural sequences such as natural or synthetic emergent languages. Results are given which indicates the performance of the proposed ZML model. We derive an entropy estimation method which incorporates the linguistic constraint-based Zipf–Mandelbrot–Li into a new non-equiprobable coincidence counting algorithm which is shown to be effective for tasks such as entropy rate estimation with limited data.     

Bivariate Entropy Analysis of Electrocardiographic RR–QT Time Series

QT interval variability (QTV) and heart rate variability (HRV) are both accepted biomarkers for cardiovascular events. QTV characterizes the variations in ventricular depolarization and repolarization. It is a predominant element of HRV. However, QTV is also believed to accept direct inputs from upstream control system. How QTV varies along with HRV is yet to be elucidated. We studied the dynamic relationship of QTV and HRV during different physiological conditions from resting, to cycling, and to recovering. We applied several entropy-based measures to examine their bivariate relationships, including cross sample entropy (XSampEn), cross fuzzy entropy (XFuzzyEn), cross conditional entropy (XCE), and joint distribution entropy (JDistEn). Results showed no statistically significant differences in XSampEn, XFuzzyEn, and XCE across different physiological states. Interestingly, JDistEn demonstrated significant decreases during cycling as compared with that during the resting state. Besides, JDistEn also showed a progressively recovering trend from cycling to the first 3 min during recovering, and further to the second 3 min during recovering. It appeared to be fully recovered to its level in the resting state during the second 3 min during the recovering phase. The results suggest that there is certain nonlinear temporal relationship between QTV and HRV, and that the JDistEn could help unravel this nuanced property.     

Paths Clustering and an Existence Result for Stochastic Vortex Systems

We prove, via a pathwise analysis, an existence result for stochastic differential equations with singular coefficients that govern stochastic vortex systems. The techniques are self-contained and rely on careful estimates on the displacements of particles, obtained by recursively identifying “vortex clusters“ whose mutual interactions can be controlled. This provides a non trivial extension of techniques of Marchioro and Pulvirenti⁽⁷⁾ for deterministic motion of vortices. AMS subject classification: 60H10, 60K35, 76B47     

Quantum Information and Entropy Spueezing of a Nonlinear Multiquantum JC Model

We investigate the entropy squeezing of the nonlinear k-quantum JC model. A definition of squeezing is presented for this system based on the quantum information theory. The utility of the definition is illustrated by examining squeezing in the information entropy of a nonlinear k-quantum two-level atom. The influence of the atomic coherence and the detuning parameter on the properties of the information entropy and squeezing of the atomic variables is examined.     

Simulations of Two-Step Maruyama Methods for Nonlinear Stochastic Delay Differential Equations

In this paper, we investigate the numerical performance of a family of P-stable two-step Maruyama schemes in mean-square sense for stochastic differential equations with time delay proposed in [8, 10] for a certain class of nonlinear stochastic delay differential equations with multiplicative white noises. We also test the convergence of one of the schemes for a time-delayed Burgers' equation with an additive white noise. Numerical results show that this family of two-step Maruyama methods exhibit similar stability for nonlinear equations as that for linear equations.     

Estimating Phase Amplitude Coupling between Neural Oscillations Based on Permutation and Entropy

Cross-frequency phase–amplitude coupling (PAC) plays an important role in neuronal oscillations network, reflecting the interaction between the phase of low-frequency oscillation (LFO) and amplitude of the high-frequency oscillations (HFO). Thus, we applied four methods based on permutation analysis to measure PAC, including multiscale permutation mutual information (MPMI), permutation conditional mutual information (PCMI), symbolic joint entropy (SJE), and weighted-permutation mutual information (WPMI). To verify the ability of these four algorithms, a performance test including the effects of coupling strength, signal-to-noise ratios (SNRs), and data length was evaluated by using simulation data. It was shown that the performance of SJE was similar to that of other approaches when measuring PAC strength, but the computational efficiency of SJE was the highest among all these four methods. Moreover, SJE can also accurately identify the PAC frequency range under the interference of spike noise. All in all, the results demonstrate that SJE is better for evaluating PAC between neural oscillations.     

Non-Equilibrium Entropy and Irreversibility in Generalized Stochastic Loewner Evolution from an Information-Theoretic Perspective

In this study, we theoretically investigated a generalized stochastic Loewner evolution (SLE) driven by reversible Langevin dynamics in the context of non-equilibrium statistical mechanics. Using the ability of Loewner evolution, which enables encoding of non-equilibrium systems into equilibrium systems, we formulated the encoding mechanism of the SLE by Gibbs entropy-based information-theoretic approaches to discuss its advantages as a means to better describe non-equilibrium systems. After deriving entropy production and flux for the 2D trajectories of the generalized SLE curves, we reformulated the system’s entropic properties in terms of the Kullback–Leibler (KL) divergence. We demonstrate that this operation leads to alternative expressions of the Jarzynski equality and the second law of thermodynamics, which are consistent with the previously suggested theory of information thermodynamics. The irreversibility of the 2D trajectories is similarly discussed by decomposing the entropy into additive and non-additive parts. We numerically verified the non-equilibrium property of our model by simulating the long-time behavior of the entropic measure suggested by our formulation, referred to as the relative Loewner entropy.     

Stochastic Hydrodynamics of Complex Fluids: Discretisation and Entropy Production

Many complex fluids can be described by continuum hydrodynamic field equations, to which noise must be added in order to capture thermal fluctuations. In almost all cases, the resulting coarse-grained stochastic partial differential equations carry a short-scale cutoff, which is also reflected in numerical discretisation schemes. We draw together our recent findings concerning the construction of such schemes and the interpretation of their continuum limits, focusing, for simplicity, on models with a purely diffusive scalar field, such as ‘Model B’ which describes phase separation in binary fluid mixtures. We address the requirement that the steady-state entropy production rate (EPR) must vanish for any stochastic hydrodynamic model in a thermal equilibrium. Only if this is achieved can the given discretisation scheme be relied upon to correctly calculate the nonvanishing EPR for ‘active field theories’ in which new terms are deliberately added to the fluctuating hydrodynamic equations that break detailed balance. To compute the correct probabilities of forward and time-reversed paths (whose ratio determines the EPR), we must make a careful treatment of so-called ‘spurious drift’ and other closely related terms that depend on the discretisation scheme. We show that such subtleties can arise not only in the temporal discretisation (as is well documented for stochastic ODEs with multiplicative noise) but also from spatial discretisation, even when noise is additive, as most active field theories assume. We then review how such noise can become multiplicative via off-diagonal couplings to additional fields that thermodynamically encode the underlying chemical processes responsible for activity. In this case, the spurious drift terms need careful accounting, not just to evaluate correctly the EPR but also to numerically implement the Langevin dynamics itself.