Discrete Versions of Jensen–Fisher,Fisher and Bayes–Fisher Information Measures of Finite Mixture Distributions

In this work, we first consider the discrete version of Fisher information measure and then propose Jensen–Fisher information, to develop some associated results. Next, we consider Fisher information and Bayes–Fisher information measures for mixing parameter vector of a finite mixture probability mass function and establish some results. We provide some connections between these measures with some known informational measures such as chi-square divergence, Shannon entropy, Kullback–Leibler, Jeffreys and Jensen–Shannon divergences.     

Noise-Robust Image Reconstruction Based on Minimizing Extended Class of Power-Divergence Measures

The problem of tomographic image reconstruction can be reduced to an optimization problem of finding unknown pixel values subject to minimizing the difference between the measured and forward projections. Iterative image reconstruction algorithms provide significant improvements over transform methods in computed tomography. In this paper, we present an extended class of power-divergence measures (PDMs), which includes a large set of distance and relative entropy measures, and propose an iterative reconstruction algorithm based on the extended PDM (EPDM) as an objective function for the optimization strategy. For this purpose, we introduce a system of nonlinear differential equations whose Lyapunov function is equivalent to the EPDM. Then, we derive an iterative formula by multiplicative discretization of the continuous-time system. Since the parameterized EPDM family includes the Kullback–Leibler divergence, the resulting iterative algorithm is a natural extension of the maximum-likelihood expectation-maximization (MLEM) method. We conducted image reconstruction experiments using noisy projection data and found that the proposed algorithm outperformed MLEM and could reconstruct high-quality images that were robust to measured noise by properly selecting parameters.     

Reliability Analysis of the New Exponential Inverted Topp–Leone Distribution with Applications

The inverted Topp–Leone distribution is a new, appealing model for reliability analysis. In this paper, a new distribution, named new exponential inverted Topp–Leone (NEITL) is presented, which adds an extra shape parameter to the inverted Topp–Leone distribution. The graphical representations of its density, survival, and hazard rate functions are provided. The following properties are explored: quantile function, mixture representation, entropies, moments, and stress–strength reliability. We plotted the skewness and kurtosis measures of the proposed model based on the quantiles. Three different estimation procedures are suggested to estimate the distribution parameters, reliability, and hazard rate functions, along with their confidence intervals. Additionally, stress–strength reliability estimators for the NEITL model were obtained. To illustrate the findings of the paper, two real datasets on engineering and medical fields have been analyzed.     

基于均值距离测度的医学图像配准

针对互信息测度在配准医学图像时易陷入局部极值、速度慢的缺点,提出了基于均值不等式的均值距离测度.首先根据均值不等式推导出5种均值距离测度：方根-算术均值距离(SAM)、方根-几何均值距离(SGM)、方根-调和均值距离(SHM)、算术-几何均值距离(AGM)、算术-调和均值距离(AHM).然后通过人体脑部CT/MR和MR-T1/PD图像的刚体配准实验,从函数曲线、配准精度、计算时间和收敛性能方面,对互信息与5种均值距离信息测度进行了比较与分析.实验结果表明,在不损失配准精度的前提下,AHM和SAM测度可以获得更快的配准速度,对噪声有很强的鲁棒性.     

Refined Young Inequality and Its Application to Divergences

We give bounds on the difference between the weighted arithmetic mean and the weighted geometric mean. These imply refined Young inequalities and the reverses of the Young inequality. We also studied some properties on the difference between the weighted arithmetic mean and the weighted geometric mean. Applying the newly obtained inequalities, we show some results on the Tsallis divergence, the Rényi divergence, the Jeffreys–Tsallis divergence and the Jensen–Shannon–Tsallis divergence.     

Automatic leukocyte nucleus segmentation by intuitionistic fuzzy divergence based thresholding

The paper proposes a robust approach to automatic segmentation of leukocyte's nucleus from microscopic blood smear images under normal as well as noisy environment by employing a new exponential intuitionistic fuzzy divergence based thresholding technique. The algorithm minimizes the divergence between the actual image and the ideally thresholded image to search for the final threshold. A new divergence formula based on exponential intuitionistic fuzzy entropy has been proposed. Further, to increase its noise handling capacity, a neighborhood-based membership function for the image pixels has been designed. The proposed scheme has been applied on 110 normal and 54 leukemia (chronic myelogenous leukemia) affected blood samples. The nucleus segmentation results have been validated by three expert hematologists. The algorithm achieves an average segmentation accuracy of 98.52% in noise-free environment. It beats the competitor algorithms in terms of several other metrics. The proposed scheme with neighborhood based membership function outperforms the competitor algorithms in terms of segmentation accuracy under noisy environment. It achieves 93.90% and 94.93% accuracies for Speckle and Gaussian noises, respectively. The average area under the ROC curves comes out to be 0.9514 in noisy conditions, which proves the robustness of the proposed algorithm.     

Economic Freedom: The Top,the Bottom,and the Reality. I. 1997–2007

We recall the historically admitted prerequisites of Economic Freedom (EF). We have examined 908 data points for the Economic Freedom of the World (EFW) index and 1884 points for the Index of Economic Freedom (IEF); the studied periods are 2000–2006 and 1997–2007, respectively, thereby following the Berlin wall collapse, and including 11 September 2001. After discussing EFW index and IEF, in order to compare the indices, one needs to study their overlap in time and space. That leaves 138 countries to be examined over a period extending from 2000 to 2006, thus 2 sets of 862 data points. The data analysis pertains to the rank-size law technique. It is examined whether the distributions obey an exponential or a power law. A correlation with the country’s Gross Domestic Product (GDP), an admittedly major determinant of EF, follows, distinguishing regional aspects, i.e., defining 6 continents. Semi-log plots show that the EFW-rank relationship is exponential for countries of high rank (≥20); overall the log–log plots point to a behaviour close to a power law. In contrast, for the IEF, the overall ranking has an exponential behaviour; but the log–log plots point to the existence of a transitional point between two different power laws, i.e., near rank 10. Moreover, log–log plots of the EFW index relationship to country GDP are characterised by a power law, with a rather stable exponent (γ≃0.674) as a function of time. In contrast, log–log plots of the IEF relationship with the country’s gross domestic product point to a downward evolutive power law as a function of time. Markedly the two studied indices provide different aspects of EF.     

新的Cauchy-Schwarz距离函数与多模态医学图像配准

信息论测度,特别是Shannon互信息是多模态图像配准的一种重要方法,但除了互信息之外,仍然存在其它的函数来实现这一任务。对互信息、Kullback-Leibler距离和Shannon不等式之间相互关系作了分析,根据这些关系和不等式理论,提出了新的Cauchy-Schwarz距离,并将这一距离测度用于多模态医学图像的配准处理。从计算速度、噪声容忍性、测度函数图形的特点和图像窗口大小影响等几个方面,通过MR和PET医学图像的实验分析,对新的Cauchy-Schwarz距离测度和典型的Shannon信息论测度进行了分析比较。实验结果表明,新的Cauchy-Schwarz距离测度函数同样可以用于多模态图像配准,而且有着更强的噪声容忍性和更为节省的计算量。     

Local Stability of McKean–Vlasov Equations Arising from Heterogeneous Gibbs Systems Using Limit of Relative Entropies

A family of heterogeneous mean-field systems with jumps is analyzed. These systems are constructed as a Gibbs measure on block graphs. When the total number of particles goes to infinity, the law of large numbers is shown to hold in a multi-class context, resulting in the weak convergence of the empirical vector towards the solution of a McKean–Vlasov system of equations. We then investigate the local stability of the limiting McKean–Vlasov system through the construction of a local Lyapunov function. We first compute the limit of adequately scaled relative entropy functions associated with the explicit stationary distribution of the N-particles system. Using a Laplace principle for empirical vectors, we show that the limit takes an explicit form. Then we demonstrate that this limit satisfies a descent property, which, combined with some mild assumptions shows that it is indeed a local Lyapunov function.     

F-Divergences and Cost Function Locality in Generative Modelling with Quantum Circuits

Generative modelling is an important unsupervised task in machine learning. In this work, we study a hybrid quantum-classical approach to this task, based on the use of a quantum circuit born machine. In particular, we consider training a quantum circuit born machine using f-divergences. We first discuss the adversarial framework for generative modelling, which enables the estimation of any f-divergence in the near term. Based on this capability, we introduce two heuristics which demonstrably improve the training of the born machine. The first is based on f-divergence switching during training. The second introduces locality to the divergence, a strategy which has proved important in similar applications in terms of mitigating barren plateaus. Finally, we discuss the long-term implications of quantum devices for computing f-divergences, including algorithms which provide quadratic speedups to their estimation. In particular, we generalise existing algorithms for estimating the Kullback–Leibler divergence and the total variation distance to obtain a fault-tolerant quantum algorithm for estimating another f-divergence, namely, the Pearson divergence.     

The Information Loss of a Stochastic Map

We provide a stochastic extension of the Baez–Fritz–Leinster characterization of the Shannon information loss associated with a measure-preserving function. This recovers the conditional entropy and a closely related information-theoretic measure that we call conditional information loss. Although not functorial, these information measures are semi-functorial, a concept we introduce that is definable in any Markov category. We also introduce the notion of an entropic Bayes’ rule for information measures, and we provide a characterization of conditional entropy in terms of this rule.     

Alternative Entropy Measures and Generalized Khinchin–Shannon Inequalities

The Khinchin–Shannon generalized inequalities for entropy measures in Information Theory, are a paradigm which can be used to test the Synergy of the distributions of probabilities of occurrence in physical systems. The rich algebraic structure associated with the introduction of escort probabilities seems to be essential for deriving these inequalities for the two-parameter Sharma–Mittal set of entropy measures. We also emphasize the derivation of these inequalities for the special cases of one-parameter Havrda–Charvat’s, Rényi’s and Landsberg–Vedral’s entropy measures.     

On a Variational Definition for the Jensen-Shannon Symmetrization of Distances Based on the Information Radius

We generalize the Jensen-Shannon divergence and the Jensen-Shannon diversity index by considering a variational definition with respect to a generic mean, thereby extending the notion of Sibson’s information radius. The variational definition applies to any arbitrary distance and yields a new way to define a Jensen-Shannon symmetrization of distances. When the variational optimization is further constrained to belong to prescribed families of probability measures, we get relative Jensen-Shannon divergences and their equivalent Jensen-Shannon symmetrizations of distances that generalize the concept of information projections. Finally, we touch upon applications of these variational Jensen-Shannon divergences and diversity indices to clustering and quantization tasks of probability measures, including statistical mixtures.     

Fast Approximations of the Jeffreys Divergence between Univariate Gaussian Mixtures via Mixture Conversions to Exponential-Polynomial Distributions

The Jeffreys divergence is a renown arithmetic symmetrization of the oriented Kullback–Leibler divergence broadly used in information sciences. Since the Jeffreys divergence between Gaussian mixture models is not available in closed-form, various techniques with advantages and disadvantages have been proposed in the literature to either estimate, approximate, or lower and upper bound this divergence. In this paper, we propose a simple yet fast heuristic to approximate the Jeffreys divergence between two univariate Gaussian mixtures with arbitrary number of components. Our heuristic relies on converting the mixtures into pairs of dually parameterized probability densities belonging to an exponential-polynomial family. To measure with a closed-form formula the goodness of fit between a Gaussian mixture and an exponential-polynomial density approximating it, we generalize the Hyvärinen divergence to α-Hyvärinen divergences. In particular, the 2-Hyvärinen divergence allows us to perform model selection by choosing the order of the exponential-polynomial densities used to approximate the mixtures. We experimentally demonstrate that our heuristic to approximate the Jeffreys divergence between mixtures improves over the computational time of stochastic Monte Carlo estimations by several orders of magnitude while approximating the Jeffreys divergence reasonably well, especially when the mixtures have a very small number of modes.     

Information Measures for Generalized Order Statistics and Their Concomitants under General Framework from Huang-Kotz FGM Bivariate Distribution

In this paper, we study the concomitants of dual generalized order statistics (and consequently generalized order statistics) when the parameters γ1,…,γn are assumed to be pairwise different from Huang–Kotz Farlie–Gumble–Morgenstern bivariate distribution. Some useful recurrence relations between single and product moments of concomitants are obtained. Moreover, Shannon’s entropy and the Fisher information number measures are derived. Finally, these measures are extensively studied for some well-known distributions such as exponential, Pareto and power distributions. The main motivation of the study of the concomitants of generalized order statistics (as an important practical kind to order the bivariate data) under this general framework is to enable researchers in different fields of statistics to use some of the important models contained in these generalized order statistics only under this general framework. These extended models are frequently used in the reliability theory, such as the progressive type-II censored order statistics.     

Entropy Estimation Using a Linguistic Zipf–Mandelbrot–Li Model for Natural Sequences

Entropy estimation faces numerous challenges when applied to various real-world problems. Our interest is in divergence and entropy estimation algorithms which are capable of rapid estimation for natural sequence data such as human and synthetic languages. This typically requires a large amount of data; however, we propose a new approach which is based on a new rank-based analytic Zipf–Mandelbrot–Li probabilistic model. Unlike previous approaches, which do not consider the nature of the probability distribution in relation to language; here, we introduce a novel analytic Zipfian model which includes linguistic constraints. This provides more accurate distributions for natural sequences such as natural or synthetic emergent languages. Results are given which indicates the performance of the proposed ZML model. We derive an entropy estimation method which incorporates the linguistic constraint-based Zipf–Mandelbrot–Li into a new non-equiprobable coincidence counting algorithm which is shown to be effective for tasks such as entropy rate estimation with limited data.     

Partial Exactness for the Penalty Function of Biconvex Programming

Biconvex programming (or inequality constrained biconvex optimization) is an important model in solving many engineering optimization problems in areas like machine learning and signal and information processing. In this paper, the partial exactness of the partial optimum for the penalty function of biconvex programming is studied. The penalty function is partially exact if the partial Karush–Kuhn–Tucker (KKT) condition is true. The sufficient and necessary partially local stability condition used to determine whether the penalty function is partially exact for a partial optimum solution is also proven. Based on the penalty function, an algorithm is presented for finding a partial optimum solution to an inequality constrained biconvex optimization, and its convergence is proven under some conditions.     

A Factor Analysis Perspective on Linear Regression in the ‘More Predictors than Samples’ Case

Linear regression (LR) is a core model in supervised machine learning performing a regression task. One can fit this model using either an analytic/closed-form formula or an iterative algorithm. Fitting it via the analytic formula becomes a problem when the number of predictors is greater than the number of samples because the closed-form solution contains a matrix inverse that is not defined when having more predictors than samples. The standard approach to solve this issue is using the Moore–Penrose inverse or the L2 regularization. We propose another solution starting from a machine learning model that, this time, is used in unsupervised learning performing a dimensionality reduction task or just a density estimation one—factor analysis (FA)—with one-dimensional latent space. The density estimation task represents our focus since, in this case, it can fit a Gaussian distribution even if the dimensionality of the data is greater than the number of samples; hence, we obtain this advantage when creating the supervised counterpart of factor analysis, which is linked to linear regression. We also create its semisupervised counterpart and then extend it to be usable with missing data. We prove an equivalence to linear regression and create experiments for each extension of the factor analysis model. The resulting algorithms are either a closed-form solution or an expectation–maximization (EM) algorithm. The latter is linked to information theory by optimizing a function containing a Kullback–Leibler (KL) divergence or the entropy of a random variable.     

CGHS Black Hole Analog Moving Mirror and Its Relativistic Quantum Information as Radiation Reaction

The Callan–Giddings–Harvey–Strominger black hole has a spectrum and temperature that correspond to an accelerated reflecting boundary condition in flat spacetime. The beta coefficients are identical to a moving mirror model, where the acceleration is exponential in laboratory time. The center of the black hole is modeled by the perfectly reflecting regularity condition that red-shifts the field modes, which is the source of the particle creation. In addition to computing the energy flux, we find the corresponding moving mirror parameter associated with the black hole mass and the cosmological constant in the gravitational analog system. Generalized to any mirror trajectory, we derive the self-force (Lorentz–Abraham–Dirac), consistently, expressing it and the Larmor power in connection with entanglement entropy, inviting an interpretation of acceleration radiation in terms of information flow. The mirror self-force and radiative power are applied to the particular CGHS black hole analog moving mirror, which reveals the physics of information at the horizon during asymptotic approach to thermal equilibrium.     

Development of hedge operator based fuzzy divergence measure and its application in segmentation of chronic myelogenous leukocytes from microscopic image of peripheral blood smear

This paper introduces a hedge operator based fuzzy divergence measure and its application in segmentation of leukocytes in case of chronic myelogenous leukemia using light microscopic images of peripheral blood smears. The concept of modified discrimination measure is applied to develop the measure of divergence based on Shannon exponential entropy and Yager's measure of entropy. These two measures of divergence are compared with the existing literatures and validated by ground truth images. Finally, it is found that hedge operator based divergence measure using Yager's entropy achieves better segmentation accuracy i.e., 98.29% for normal and 98.15% for chronic myelogenous leukocytes. Furthermore, Jaccard index has been performed to compare the segmented image with ground truth ones where it is found that that the proposed scheme leads to higher Jaccard index (0.39 for normal, 0.24 for chronic myelogenous leukemia).