Discrete Versions of Jensen–Fisher,Fisher and Bayes–Fisher Information Measures of Finite Mixture Distributions

In this work, we first consider the discrete version of Fisher information measure and then propose Jensen–Fisher information, to develop some associated results. Next, we consider Fisher information and Bayes–Fisher information measures for mixing parameter vector of a finite mixture probability mass function and establish some results. We provide some connections between these measures with some known informational measures such as chi-square divergence, Shannon entropy, Kullback–Leibler, Jeffreys and Jensen–Shannon divergences.     

Refined Young Inequality and Its Application to Divergences

We give bounds on the difference between the weighted arithmetic mean and the weighted geometric mean. These imply refined Young inequalities and the reverses of the Young inequality. We also studied some properties on the difference between the weighted arithmetic mean and the weighted geometric mean. Applying the newly obtained inequalities, we show some results on the Tsallis divergence, the Rényi divergence, the Jeffreys–Tsallis divergence and the Jensen–Shannon–Tsallis divergence.     

Strongly Convex Divergences

We consider a sub-class of the f-divergences satisfying a stronger convexity property, which we refer to as strongly convex, or κ-convex divergences. We derive new and old relationships, based on convexity arguments, between popular f-divergences.     

Error Exponents and α-Mutual Information

Over the last six decades, the representation of error exponent functions for data transmission through noisy channels at rates below capacity has seen three distinct approaches: (1) Through Gallager’s E0 functions (with and without cost constraints); (2) large deviations form, in terms of conditional relative entropy and mutual information; (3) through the α-mutual information and the Augustin–Csiszár mutual information of order α derived from the Rényi divergence. While a fairly complete picture has emerged in the absence of cost constraints, there have remained gaps in the interrelationships between the three approaches in the general case of cost-constrained encoding. Furthermore, no systematic approach has been proposed to solve the attendant optimization problems by exploiting the specific structure of the information functions. This paper closes those gaps and proposes a simple method to maximize Augustin–Csiszár mutual information of order α under cost constraints by means of the maximization of the α-mutual information subject to an exponential average constraint.     

Information–Theoretic Aspects of Location Parameter Estimation under Skew–Normal Settings

In several applications, the assumption of normality is often violated in data with some level of skewness, so skewness affects the mean’s estimation. The class of skew–normal distributions is considered, given their flexibility for modeling data with asymmetry parameter. In this paper, we considered two location parameter (μ) estimation methods in the skew–normal setting, where the coefficient of variation and the skewness parameter are known. Specifically, the least square estimator (LSE) and the best unbiased estimator (BUE) for μ are considered. The properties for BUE (which dominates LSE) using classic theorems of information theory are explored, which provides a way to measure the uncertainty of location parameter estimations. Specifically, inequalities based on convexity property enable obtaining lower and upper bounds for differential entropy and Fisher information. Some simulations illustrate the behavior of differential entropy and Fisher information bounds.     

Quantum Thermodynamic Uncertainty Relations,Generalized Current Fluctuations and Nonequilibrium Fluctuation–Dissipation Inequalities

Thermodynamic uncertainty relations (TURs) represent one of the few broad-based and fundamental relations in our toolbox for tackling the thermodynamics of nonequilibrium systems. One form of TUR quantifies the minimal energetic cost of achieving a certain precision in determining a nonequilibrium current. In this initial stage of our research program, our goal is to provide the quantum theoretical basis of TURs using microphysics models of linear open quantum systems where it is possible to obtain exact solutions. In paper [Dong et al., Entropy 2022, 24, 870], we show how TURs are rooted in the quantum uncertainty principles and the fluctuation–dissipation inequalities (FDI) under fully nonequilibrium conditions. In this paper, we shift our attention from the quantum basis to the thermal manifests. Using a microscopic model for the bath’s spectral density in quantum Brownian motion studies, we formulate a “thermal” FDI in the quantum nonequilibrium dynamics which is valid at high temperatures. This brings the quantum TURs we derive here to the classical domain and can thus be compared with some popular forms of TURs. In the thermal-energy-dominated regimes, our FDIs provide better estimates on the uncertainty of thermodynamic quantities. Our treatment includes full back-action from the environment onto the system. As a concrete example of the generalized current, we examine the energy flux or power entering the Brownian particle and find an exact expression of the corresponding current–current correlations. In so doing, we show that the statistical properties of the bath and the causality of the system+bath interaction both enter into the TURs obeyed by the thermodynamic quantities.     

Some Parameterized Quantum Midpoint and Quantum Trapezoid Type Inequalities for Convex Functions with Applications

Quantum information theory, an interdisciplinary field that includes computer science, information theory, philosophy, cryptography, and entropy, has various applications for quantum calculus. Inequalities and entropy functions have a strong association with convex functions. In this study, we prove quantum midpoint type inequalities, quantum trapezoidal type inequalities, and the quantum Simpson’s type inequality for differentiable convex functions using a new parameterized q-integral equality. The newly formed inequalities are also proven to be generalizations of previously existing inequities. Finally, using the newly established inequalities, we present some applications for quadrature formulas.     

Belavkin–Staszewski Relative Entropy,Conditional Entropy,and Mutual Information

Belavkin–Staszewski relative entropy can naturally characterize the effects of the possible noncommutativity of quantum states. In this paper, two new conditional entropy terms and four new mutual information terms are first defined by replacing quantum relative entropy with Belavkin–Staszewski relative entropy. Next, their basic properties are investigated, especially in classical-quantum settings. In particular, we show the weak concavity of the Belavkin–Staszewski conditional entropy and obtain the chain rule for the Belavkin–Staszewski mutual information. Finally, the subadditivity of the Belavkin–Staszewski relative entropy is established, i.e., the Belavkin–Staszewski relative entropy of a joint system is less than the sum of that of its corresponding subsystems with the help of some multiplicative and additive factors. Meanwhile, we also provide a certain subadditivity of the geometric Rényi relative entropy.     

Computation of Kullback–Leibler Divergence in Bayesian Networks

Kullback–Leibler divergence KL(p,q) is the standard measure of error when we have a true probability distribution p which is approximate with probability distribution q. Its efficient computation is essential in many tasks, as in approximate computation or as a measure of error when learning a probability. In high dimensional probabilities, as the ones associated with Bayesian networks, a direct computation can be unfeasible. This paper considers the case of efficiently computing the Kullback–Leibler divergence of two probability distributions, each one of them coming from a different Bayesian network, which might have different structures. The paper is based on an auxiliary deletion algorithm to compute the necessary marginal distributions, but using a cache of operations with potentials in order to reuse past computations whenever they are necessary. The algorithms are tested with Bayesian networks from the bnlearn repository. Computer code in Python is provided taking as basis pgmpy, a library for working with probabilistic graphical models.     

Hypercontractive Inequalities for the Second Norm of Highly Concentrated Functions,and Mrs. Gerber’s-Type Inequalities for the Second Rényi Entropy

Let Tϵ, 0≤ϵ≤1/2, be the noise operator acting on functions on the boolean cube {0,1}n. Let f be a distribution on {0,1}n and let q>1. We prove tight Mrs. Gerber-type results for the second Rényi entropy of Tϵf which take into account the value of the qth Rényi entropy of f. For a general function f on {0,1}n we prove tight hypercontractive inequalities for the ℓ2 norm of Tϵf which take into account the ratio between ℓq and ℓ1 norms of f.     

An Objective Prior from a Scoring Rule

In this paper, we introduce a novel objective prior distribution levering on the connections between information, divergence and scoring rules. In particular, we do so from the starting point of convex functions representing information in density functions. This provides a natural route to proper local scoring rules using Bregman divergence. Specifically, we determine the prior which solves setting the score function to be a constant. Although in itself this provides motivation for an objective prior, the prior also minimizes a corresponding information criterion.     

Inequalities for the Casorati Curvature of Totally Real Spacelike Submanifolds in Statistical Manifolds of Type Para-Kähler Space Forms

The purpose of this article is to establish some inequalities concerning the normalized δ-Casorati curvatures (extrinsic invariants) and the scalar curvature (intrinsic invariant) of totally real spacelike submanifolds in statistical manifolds of the type para-Kähler space form. Moreover, this study is focused on the equality cases in these inequalities. Some examples are also provided.     

Entropies and IPR as Markers for a Phase Transition in a Two-Level Model for Atom–Diatomic Molecule Coexistence

A quantum phase transition (QPT) in a simple model that describes the coexistence of atoms and diatomic molecules is studied. The model, which is briefly discussed, presents a second-order ground state phase transition in the thermodynamic (or large particle number) limit, changing from a molecular condensate in one phase to an equilibrium of diatomic molecules–atoms in coexistence in the other one. The usual markers for this phase transition are the ground state energy and the expected value of the number of atoms (alternatively, the number of molecules) in the ground state. In this work, other markers for the QPT, such as the inverse participation ratio (IPR), and particularly, the Rényi entropy, are analyzed and proposed as QPT markers. Both magnitudes present abrupt changes at the critical point of the QPT.     

Du Bois–Reymond Type Lemma and Its Application to Dirichlet Problem with the p(t)–Laplacian on a Bounded Time Scale

This paper is devoted to study the existence of solutions and their regularity in the p(t)–Laplacian Dirichlet problem on a bounded time scale. First, we prove a lemma of du Bois–Reymond type in time-scale settings. Then, using direct variational methods and the mountain pass methodology, we present several sufficient conditions for the existence of solutions to the Dirichlet problem.     

Quantum Information of the Aharanov–Bohm Ring with Yukawa Interaction in the Presence of Disclination

We investigate quantum information by a theoretical measurement approach of an Aharanov–Bohm (AB) ring with Yukawa interaction in curved space with disclination. We obtained the so-called Shannon entropy through the eigenfunctions of the system. The quantum states considered come from Schrödinger theory with the AB field in the background of curved space. With this entropy, we can explore the quantum information at the position space and reciprocal space. Furthermore, we discussed how the magnetic field, the AB flux, and the topological defect influence the quantum states and the information entropy.     

Numerical Study on an RBF-FD Tangent Plane Based Method for Convection–Diffusion Equations on Anisotropic Evolving Surfaces

In this paper, we present a fully Lagrangian method based on the radial basis function (RBF) finite difference (FD) method for solving convection–diffusion partial differential equations (PDEs) on evolving surfaces. Surface differential operators are discretized by the tangent plane approach using Gaussian RBFs augmented with two-dimensional (2D) polynomials. The main advantage of our method is the simplicity of calculating differentiation weights. Additionally, we couple the method with anisotropic RBFs (ARBFs) to obtain more accurate numerical solutions for the anisotropic growth of surfaces. In the ARBF interpolation, the Euclidean distance is replaced with a suitable metric that matches the anisotropic surface geometry. Therefore, it will lead to a good result on the aspects of stability and accuracy of the RBF-FD method for this type of problem. The performance of this method is shown for various convection–diffusion equations on evolving surfaces, which include the anisotropic growth of surfaces and growth coupled with the solutions of PDEs.     

Rényi Cross-Entropy Measures for Common Distributions and Processes with Memory

Two Rényi-type generalizations of the Shannon cross-entropy, the Rényi cross-entropy and the Natural Rényi cross-entropy, were recently used as loss functions for the improved design of deep learning generative adversarial networks. In this work, we derive the Rényi and Natural Rényi differential cross-entropy measures in closed form for a wide class of common continuous distributions belonging to the exponential family, and we tabulate the results for ease of reference. We also summarise the Rényi-type cross-entropy rates between stationary Gaussian processes and between finite-alphabet time-invariant Markov sources.     

Revisiting Chernoff Information with Likelihood Ratio Exponential Families

The Chernoff information between two probability measures is a statistical divergence measuring their deviation defined as their maximally skewed Bhattacharyya distance. Although the Chernoff information was originally introduced for bounding the Bayes error in statistical hypothesis testing, the divergence found many other applications due to its empirical robustness property found in applications ranging from information fusion to quantum information. From the viewpoint of information theory, the Chernoff information can also be interpreted as a minmax symmetrization of the Kullback–Leibler divergence. In this paper, we first revisit the Chernoff information between two densities of a measurable Lebesgue space by considering the exponential families induced by their geometric mixtures: The so-called likelihood ratio exponential families. Second, we show how to (i) solve exactly the Chernoff information between any two univariate Gaussian distributions or get a closed-form formula using symbolic computing, (ii) report a closed-form formula of the Chernoff information of centered Gaussians with scaled covariance matrices and (iii) use a fast numerical scheme to approximate the Chernoff information between any two multivariate Gaussian distributions.     

A Two-Moment Inequality with Applications to Rényi Entropy and Mutual Information

This paper explores some applications of a two-moment inequality for the integral of the rth power of a function, where 0<r<1. The first contribution is an upper bound on the Rényi entropy of a random vector in terms of the two different moments. When one of the moments is the zeroth moment, these bounds recover previous results based on maximum entropy distributions under a single moment constraint. More generally, evaluation of the bound with two carefully chosen nonzero moments can lead to significant improvements with a modest increase in complexity. The second contribution is a method for upper bounding mutual information in terms of certain integrals with respect to the variance of the conditional density. The bounds have a number of useful properties arising from the connection with variance decompositions.