F-Divergences and Cost Function Locality in Generative Modelling with Quantum Circuits

Generative modelling is an important unsupervised task in machine learning. In this work, we study a hybrid quantum-classical approach to this task, based on the use of a quantum circuit born machine. In particular, we consider training a quantum circuit born machine using f-divergences. We first discuss the adversarial framework for generative modelling, which enables the estimation of any f-divergence in the near term. Based on this capability, we introduce two heuristics which demonstrably improve the training of the born machine. The first is based on f-divergence switching during training. The second introduces locality to the divergence, a strategy which has proved important in similar applications in terms of mitigating barren plateaus. Finally, we discuss the long-term implications of quantum devices for computing f-divergences, including algorithms which provide quadratic speedups to their estimation. In particular, we generalise existing algorithms for estimating the Kullback–Leibler divergence and the total variation distance to obtain a fault-tolerant quantum algorithm for estimating another f-divergence, namely, the Pearson divergence.     

Computation of Kullback–Leibler Divergence in Bayesian Networks

Kullback–Leibler divergence KL(p,q) is the standard measure of error when we have a true probability distribution p which is approximate with probability distribution q. Its efficient computation is essential in many tasks, as in approximate computation or as a measure of error when learning a probability. In high dimensional probabilities, as the ones associated with Bayesian networks, a direct computation can be unfeasible. This paper considers the case of efficiently computing the Kullback–Leibler divergence of two probability distributions, each one of them coming from a different Bayesian network, which might have different structures. The paper is based on an auxiliary deletion algorithm to compute the necessary marginal distributions, but using a cache of operations with potentials in order to reuse past computations whenever they are necessary. The algorithms are tested with Bayesian networks from the bnlearn repository. Computer code in Python is provided taking as basis pgmpy, a library for working with probabilistic graphical models.     

Fast Approximations of the Jeffreys Divergence between Univariate Gaussian Mixtures via Mixture Conversions to Exponential-Polynomial Distributions

The Jeffreys divergence is a renown arithmetic symmetrization of the oriented Kullback–Leibler divergence broadly used in information sciences. Since the Jeffreys divergence between Gaussian mixture models is not available in closed-form, various techniques with advantages and disadvantages have been proposed in the literature to either estimate, approximate, or lower and upper bound this divergence. In this paper, we propose a simple yet fast heuristic to approximate the Jeffreys divergence between two univariate Gaussian mixtures with arbitrary number of components. Our heuristic relies on converting the mixtures into pairs of dually parameterized probability densities belonging to an exponential-polynomial family. To measure with a closed-form formula the goodness of fit between a Gaussian mixture and an exponential-polynomial density approximating it, we generalize the Hyvärinen divergence to α-Hyvärinen divergences. In particular, the 2-Hyvärinen divergence allows us to perform model selection by choosing the order of the exponential-polynomial densities used to approximate the mixtures. We experimentally demonstrate that our heuristic to approximate the Jeffreys divergence between mixtures improves over the computational time of stochastic Monte Carlo estimations by several orders of magnitude while approximating the Jeffreys divergence reasonably well, especially when the mixtures have a very small number of modes.     

Cumulative Residual q-Fisher Information and Jensen-Cumulative Residual χ2 Divergence Measures

In this work, we define cumulative residual q-Fisher (CRQF) information measures for the survival function (SF) of the underlying random variables as well as for the model parameter. We also propose q-hazard rate (QHR) function via q-logarithmic function as a new extension of hazard rate function. We show that CRQF information measure can be expressed in terms of the QHR function. We define further generalized cumulative residual χ2 divergence measures between two SFs. We then examine the cumulative residual q-Fisher information for two well-known mixture models, and the corresponding results reveal some interesting connections between the cumulative residual q-Fisher information and the generalized cumulative residual χ2 divergence measures. Further, we define Jensen-cumulative residual χ2 (JCR-χ2) measure and a parametric version of the Jensen-cumulative residual Fisher information measure and then discuss their properties and inter-connections. Finally, for illustrative purposes, we examine a real example of image processing and provide some numerical results in terms of the CRQF information measure.     

Statistical Divergences between Densities of Truncated Exponential Families with Nested Supports: Duo Bregman and Duo Jensen Divergences

By calculating the Kullback–Leibler divergence between two probability measures belonging to different exponential families dominated by the same measure, we obtain a formula that generalizes the ordinary Fenchel–Young divergence. Inspired by this formula, we define the duo Fenchel–Young divergence and report a majorization condition on its pair of strictly convex generators, which guarantees that this divergence is always non-negative. The duo Fenchel–Young divergence is also equivalent to a duo Bregman divergence. We show how to use these duo divergences by calculating the Kullback–Leibler divergence between densities of truncated exponential families with nested supports, and report a formula for the Kullback–Leibler divergence between truncated normal distributions. Finally, we prove that the skewed Bhattacharyya distances between truncated exponential families amount to equivalent skewed duo Jensen divergences.     

A Satellite Incipient Fault Detection Method Based on Decomposed Kullback–Leibler Divergence

Detection of faults at the incipient stage is critical to improving the availability and continuity of satellite services. The application of a local optimum projection vector and the Kullback–Leibler (KL) divergence can improve the detection rate of incipient faults. However, this suffers from the problem of high time complexity. We propose decomposing the KL divergence in the original optimization model and applying the property of the generalized Rayleigh quotient to reduce time complexity. Additionally, we establish two distribution models for subfunctions F1(w) and F3(w) to detect the slight anomalous behavior of the mean and covariance. The effectiveness of the proposed method was verified through a numerical simulation case and a real satellite fault case. The results demonstrate the advantages of low computational complexity and high sensitivity to incipient faults.     

Coupled VAE: Improved Accuracy and Robustness of a Variational Autoencoder

We present a coupled variational autoencoder (VAE) method, which improves the accuracy and robustness of the model representation of handwritten numeral images. The improvement is measured in both increasing the likelihood of the reconstructed images and in reducing divergence between the posterior and a prior latent distribution. The new method weighs outlier samples with a higher penalty by generalizing the original evidence lower bound function using a coupled entropy function based on the principles of nonlinear statistical coupling. We evaluated the performance of the coupled VAE model using the Modified National Institute of Standards and Technology (MNIST) dataset and its corrupted modification C-MNIST. Histograms of the likelihood that the reconstruction matches the original image show that the coupled VAE improves the reconstruction and this improvement is more substantial when seeded with corrupted images. All five corruptions evaluated showed improvement. For instance, with the Gaussian corruption seed the accuracy improves by 1014 (from 10−57.2 to 10−42.9) and robustness improves by 1022 (from 10−109.2 to 10−87.0). Furthermore, the divergence between the posterior and prior distribution of the latent distribution is reduced. Thus, in contrast to the β-VAE design, the coupled VAE algorithm improves model representation, rather than trading off the performance of the reconstruction and latent distribution divergence.     

On Representations of Divergence Measures and Related Quantities in Exponential Families

Within exponential families, which may consist of multi-parameter and multivariate distributions, a variety of divergence measures, such as the Kullback–Leibler divergence, the Cressie–Read divergence, the Rényi divergence, and the Hellinger metric, can be explicitly expressed in terms of the respective cumulant function and mean value function. Moreover, the same applies to related entropy and affinity measures. We compile representations scattered in the literature and present a unified approach to the derivation in exponential families. As a statistical application, we highlight their use in the construction of confidence regions in a multi-sample setup.     

On Selection Criteria for the Tuning Parameter in Robust Divergence

Although robust divergence, such as density power divergence and γ-divergence, is helpful for robust statistical inference in the presence of outliers, the tuning parameter that controls the degree of robustness is chosen in a rule-of-thumb, which may lead to an inefficient inference. We here propose a selection criterion based on an asymptotic approximation of the Hyvarinen score applied to an unnormalized model defined by robust divergence. The proposed selection criterion only requires first and second-order partial derivatives of an assumed density function with respect to observations, which can be easily computed regardless of the number of parameters. We demonstrate the usefulness of the proposed method via numerical studies using normal distributions and regularized linear regression.     

Bayesian Update with Importance Sampling: Required Sample Size

Importance sampling is used to approximate Bayes’ rule in many computational approaches to Bayesian inverse problems, data assimilation and machine learning. This paper reviews and further investigates the required sample size for importance sampling in terms of the χ2-divergence between target and proposal. We illustrate through examples the roles that dimension, noise-level and other model parameters play in approximating the Bayesian update with importance sampling. Our examples also facilitate a new direct comparison of standard and optimal proposals for particle filtering.     

Charged Shear-Free Fluids and Complexity in First Integrals

The equation yxx=f(x)y2+g(x)y3 is the charged generalization of the Emden-Fowler equation that is crucial in the study of spherically symmetric shear-free spacetimes. This version arises from the Einstein–Maxwell system for a charged shear-free matter distribution. We integrate this equation and find a new first integral. For this solution to exist, two integral equations arise as integrability conditions. The integrability conditions can be transformed to nonlinear differential equations, which give explicit forms for f(x) and g(x) in terms of elementary and special functions. The explicit forms and arise as repeated roots of a fourth order polynomial. This is a new solution to the Einstein-Maxwell equations. Our result complements earlier work in neutral and charged matter showing that the complexity of a charged self-gravitating fluid is connected to the existence of a first integral.     

A Generic Formula and Some Special Cases for the Kullback–Leibler Divergence between Central Multivariate Cauchy Distributions

This paper introduces a closed-form expression for the Kullback–Leibler divergence (KLD) between two central multivariate Cauchy distributions (MCDs) which have been recently used in different signal and image processing applications where non-Gaussian models are needed. In this overview, the MCDs are surveyed and some new results and properties are derived and discussed for the KLD. In addition, the KLD for MCDs is showed to be written as a function of Lauricella D-hypergeometric series FD(p). Finally, a comparison is made between the Monte Carlo sampling method to approximate the KLD and the numerical value of the closed-form expression of the latter. The approximation of the KLD by Monte Carlo sampling method are shown to converge to its theoretical value when the number of samples goes to the infinity.     

Antineutrino science by KamLAND

KamLAND measured the $\bar{\nu }$_e’s flux from distant nuclear reactors, and found fewer events than expected from standard assumptions about $\bar{\nu }$_e propagation at the 99.998% confidence level (C.L.). The observed energy spectrum disagrees with the expected spectral shape at 99.6% C.L., and prefers the distortion from neutrino oscillation effects. A two-flavor oscillation analysis of the data from KamLAND and solar neutrino experiments with CPT invariance, yields Δm2=7.9−0.5+0.6×10−5 eV² and tan2θ=0.40−0.07+0.10. All solutions to the solar neutrino problem except for the large mixing angle (LMA) region are excluded. KamLAND succeeded in detecting geoneutrinos produced by the decays of ²³⁸U and ²³²Th within the Earth. The total observed number of 4.5 to 54.2, assuming a Th/U mass concentration ratio of 3.9 is consistent with 19 predicted by geophysical models. This detection allows better estimation of the abundances and distributions of radioactive elements in the Earth, and of the Earth’s overall heat budget.     

Construction of a Family of Maximally Entangled Bases in ℂd ⊗ ℂd′

In this paper, we present a new method for the construction of maximally entangled states in Cd⊗Cd′ when d′≥2d. A systematic way of constructing a set of maximally entangled bases (MEBs) in Cd⊗Cd′ was established. Both cases when d′ is divisible by d and not divisible by d are discussed. We give two examples of maximally entangled bases in C2⊗C4, which are mutually unbiased bases. Finally, we found a new example of an unextendible maximally entangled basis (UMEB) in C2⊗C5.     

First Integrals of Shear-Free Fluids and Complexity

A single master equation governs the behaviour of shear-free neutral perfect fluid distributions arising in gravity theories. In this paper, we study the integrability of yxx=f(x)y2, find new solutions, and generate a new first integral. The first integral is subject to an integrability condition which is an integral equation which restricts the function f(x). We find that the integrability condition can be written as a third order differential equation whose solution can be expressed in terms of elementary functions and elliptic integrals. The solution of the integrability condition is generally given parametrically. A particular form of which corresponds to repeated roots of a cubic equation is given explicitly, which is a new result. Our investigation demonstrates that complexity of a self-gravitating shear-free fluid is related to the existence of a first integral, and this may be extendable to general matter distributions.     

Uncertainty relations in quantum optics. Is the photon intelligent?

The Robertson–Schrödinger, Heisenberg–Robertson and Trifonov uncertainty relations for arbitrary two functions f₁$f_1$ and f₂$f_2$ depending on the quantum phase and the number of photons respectively, are given. Intelligent states and states which minimize locally the product of uncertainties (Δf₁)²⋅(Δf₂)²${\left(\Delta f_1\right)}^2\cdot {\left(\Delta f_2\right)}^2$ or the sum (Δf₁)²+(Δf₂)²${\left(\Delta f_1\right)}^2+{\left(\Delta f_2\right)}^2$ are investigated for the cases f₁=?,exp(i?),exp(−i?),cos?,sin?$f_1=?,exp\left(i?\right),exp(-i?),cos?,sin?$ and f₂=n$f_2=n$.     

Discrete Versions of Jensen–Fisher,Fisher and Bayes–Fisher Information Measures of Finite Mixture Distributions

In this work, we first consider the discrete version of Fisher information measure and then propose Jensen–Fisher information, to develop some associated results. Next, we consider Fisher information and Bayes–Fisher information measures for mixing parameter vector of a finite mixture probability mass function and establish some results. We provide some connections between these measures with some known informational measures such as chi-square divergence, Shannon entropy, Kullback–Leibler, Jeffreys and Jensen–Shannon divergences.     

Confined Quantum Hard Spheres

We present computer simulation and theoretical results for a system of N Quantum Hard Spheres (QHS) particles of diameter σ and mass m at temperature T, confined between parallel hard walls separated by a distance Hσ, within the range 1≤H≤∞. Semiclassical Monte Carlo computer simulations were performed adapted to a confined space, considering effects in terms of the density of particles ρ*=N/V, where V is the accessible volume, the inverse length H−1 and the de Broglie’s thermal wavelength λB=h/2πmkT, where k and h are the Boltzmann’s and Planck’s constants, respectively. For the case of extreme and maximum confinement, 0.5<H−1<1 and H−1=1, respectively, analytical results can be given based on an extension for quantum systems of the Helmholtz free energies for the corresponding classical systems.