Inverse statistical problems: from the inverse Ising problem to data science

Inverse problems in statistical physics are motivated by the challenges of ‘big data’ in different fields, in particular high-throughput experiments in biology. In inverse problems, the usual procedure of statistical physics needs to be reversed: Instead of calculating observables on the basis of model parameters, we seek to infer parameters of a model based on observations. In this review, we focus on the inverse Ising problem and closely related problems, namely how to infer the coupling strengths between spins given observed spin correlations, magnetizations, or other data. We review applications of the inverse Ising problem, including the reconstruction of neural connections, protein structure determination, and the inference of gene regulatory networks. For the inverse Ising problem in equilibrium, a number of controlled and uncontrolled approximate solutions have been developed in the statistical mechanics community. A particularly strong method, pseudolikelihood, stems from statistics. We also review the inverse Ising problem in the non-equilibrium case, where the model parameters must be reconstructed based on non-equilibrium statistics.     

Statistical physics of inference: thresholds and algorithms

Many questions of fundamental interest in today's science can be formulated as inference problems: some partial, or noisy, observations are performed over a set of variables and the goal is to recover, or infer, the values of the variables based on the indirect information contained in the measurements. For such problems, the central scientific questions are: Under what conditions is the information contained in the measurements sufficient for a satisfactory inference to be possible? What are the most efficient algorithms for this task? A growing body of work has shown that often we can understand and locate these fundamental barriers by thinking of them as phase transitions in the sense of statistical physics. Moreover, it turned out that we can use the gained physical insight to develop new promising algorithms. The connection between inference and statistical physics is currently witnessing an impressive renaissance and we review here the current state-of-the-art, with a pedagogical focus on the Ising model which, formulated as an inference problem, we call the planted spin glass. In terms of applications we review two classes of problems: (i) inference of clusters on graphs and networks, with community detection as a special case and (ii) estimating a signal from its noisy linear measurements, with compressed sensing as a case of sparse estimation. Our goal is to provide a pedagogical review for researchers in physics and other fields interested in this fascinating topic.     

Prior-predictive value from fast growth simulations

Building on a variant of the Jarzynski equation we propose a new method to numerically determine the prior-predictive value in a Bayesian inference problem. The method generalizes thermodynamic integration and is not hampered by equilibration problems. We demonstrate its operation by applying it to two simple examples and elucidate its performance. In the case of multi-modal posterior distributions the performance is superior to thermodynamic integration.     

Entropy analysis of the dynamics of El Niño/Southern Oscillation during the Holocene

This study explores temporal changes in the dynamics of the Holocene ENSO proxy record of the Laguna Pallcacocha sedimentary data using two entropy quantifiers. In particular, we analyze the possible connections between changes in entropy and epochs of rapid climate change (RCC). Our results indicate that the dynamics of the ENSO proxy record during the RCC interval 9000-8000 BP displays very low entropy (high predictability) that is remarkably different from that of the other RCCs of the Holocene. Both entropy quantifiers point out to the existence of cycles with a period close to 2000 years during the mid-to-late Holocene. Within these cycles, we find a tendency for entropy to increase (predictability to decrease) during the two longer RCC periods (6000-5000 and 3500-2500 BP) which might be associated with the reported increased aridity of the low tropics.     

光学技术在运动人体科学中的应用

研究了光学技术在人体运动研究中潜在应用．光生物调节作用可以抑制细胞凋亡．色光疗法、低强度激光针灸和鼻腔内低强度激光照射可以调节植物神经功能和昼夜节律，它们具有防治运动性疲劳的潜在价值．蛋白质分子构型变化可以用园二色谱表征，组织和大脑血氧饱和度的变化可以用近红外光谱表征，尿液可以用傅里叶红外光谱进行代谢组学研究，它们都可以作为运动训练的有效表征工具．     

Integration of some examples of geodesic flows via solvable structures

Solvable structures are particularly useful in the integration by quadratures of ordinary differential equations. Nevertheless, for a given equation, it is not always possible to compute a solvable structure. In practice, the simplest solvable structures are those adapted to an already known system of symmetries. In this paper we propose a method of integration which uses solvable structures suitably adapted to both symmetries and first integrals. In the variational case, due to Noether theorem, this method is particularly effective as illustrated by some examples of integration of the geodesic flows.     

Establishment of a concept of visceral fat syndrome and discovery of adiponectin

Although obesity is a major background of life style-related diseases such as diabetes mellitus, lipid disorder, hypertension and cardiovascular disease, the extent of whole body fat accumulation does not necessarily the determinant for the occurrence of these diseases. We developed the method for body fat analysis using CT scan and established the concept of visceral fat obesity, in other word metabolic syndrome in which intra-abdominal visceral fat accumulation has an important role in the development of diabetes, lipid disorder, hypertension and atherosclerosis. In order to clarify the mechanism that visceral fat accumulation causes metabolic and cardiovascular diseases, we have analyzed gene expression profile in subcutaneous adipose tissue and visceral adipose tissue. From the analysis, we found that adipose tissue, especially visceral adipose tissue expressed abundantly the genes encoding bioactive substances such as cytokines, growth factors and complements. In addition to known bioactive substances, we found a novel collagen-like protein which we named adiponectin. Adiponectin is present in plasma at a very high concentration and is inversely associated with visceral fat accumulation. Adiponectin has anti-diabetic, anti-hypertensive and anti-atherogenic properties and recent studies revealed that this protein has an anti-inflammatory and anti-oncogenic function. Therefore hypoadiponectinemia induced by visceral fat accumulation should become a strong risk factor for metabolic and cardiovascular diseases and also some kinds of cancers.In this review article, I would like to discuss the mechanism of life style-related diseases by focusing on the dysregulation of adiponectin related to obesity, especially visceral obesity.     

Variational Message Passing and Local Constraint Manipulation in Factor Graphs

Accurate evaluation of Bayesian model evidence for a given data set is a fundamental problem in model development. Since evidence evaluations are usually intractable, in practice variational free energy (VFE) minimization provides an attractive alternative, as the VFE is an upper bound on negative model log-evidence (NLE). In order to improve tractability of the VFE, it is common to manipulate the constraints in the search space for the posterior distribution of the latent variables. Unfortunately, constraint manipulation may also lead to a less accurate estimate of the NLE. Thus, constraint manipulation implies an engineering trade-off between tractability and accuracy of model evidence estimation. In this paper, we develop a unifying account of constraint manipulation for variational inference in models that can be represented by a (Forney-style) factor graph, for which we identify the Bethe Free Energy as an approximation to the VFE. We derive well-known message passing algorithms from first principles, as the result of minimizing the constrained Bethe Free Energy (BFE). The proposed method supports evaluation of the BFE in factor graphs for model scoring and development of new message passing-based inference algorithms that potentially improve evidence estimation accuracy.     

Flexible and Efficient Inference with Particles for the Variational Gaussian Approximation

Variational inference is a powerful framework, used to approximate intractable posteriors through variational distributions. The de facto standard is to rely on Gaussian variational families, which come with numerous advantages: they are easy to sample from, simple to parametrize, and many expectations are known in closed-form or readily computed by quadrature. In this paper, we view the Gaussian variational approximation problem through the lens of gradient flows. We introduce a flexible and efficient algorithm based on a linear flow leading to a particle-based approximation. We prove that, with a sufficient number of particles, our algorithm converges linearly to the exact solution for Gaussian targets, and a low-rank approximation otherwise. In addition to the theoretical analysis, we show, on a set of synthetic and real-world high-dimensional problems, that our algorithm outperforms existing methods with Gaussian targets while performing on a par with non-Gaussian targets.     

Integration of enzyme kinetic models and isotopomer distribution analysis for studies of <Emphasis Type="Italic">in situ</Emphasis>cell operation

A current trend in neuroscience research is the use of stable isotope tracers in order to address metabolic processes in vivo. The tracers produce a huge number of metabolite forms that differ according to the number and position of labeled isotopes in the carbon skeleton (isotopomers) and such a large variety makes the analysis of isotopomer data highly complex. On the other hand, this multiplicity of forms does provide sufficient information to address cell operation in vivo. By the end of last millennium, a number of tools have been developed for estimation of metabolic flux profile from any possible isotopomer distribution data. However, although well elaborated, these tools were limited to steady state analysis, and the obtained set of fluxes remained disconnected from their biochemical context. In this review we focus on a new numerical analytical approach that integrates kinetic and metabolic flux analysis. The related computational algorithm estimates the dynamic flux based on the time-dependent distribution of all possible isotopomers of metabolic pathway intermediates that are generated from a labeled substrate. The new algorithm connects specific tracer data with enzyme kinetic characteristics, thereby extending the amount of data available for analysis: it uses enzyme kinetic data to estimate the flux profile, and vice versa, for the kinetic analysis it uses in vivo tracer data to reveal the biochemical basis of the estimated metabolic fluxes.     

基于可分解马尔科夫网的视频图像检测方法研究

研究了可分解马尔科夫网的概念、方法,分析了可分解马尔科夫网在序列图像数据挖掘中的作用与意义,并直接将马尔科夫网的结构作为决策或推理依据,应用于问题求解,拓广了可分解马尔科夫网应用的可能性;以真实交通违章的视频图像为例,以多种粒度(节点数)广泛研究建立视频图像间的可分解马尔科夫网并分析其对问题的适用性,通过网络结构分析来检测视频图像间的差异,从而发现某种有意义的模式(例如,交通违章);仿真结果表明所提方法具有实用价值和较好效果;研究结果表明可分解马尔科夫网可以很好地揭示数据间的抽象近邻关系,并且这种网络具有很好的知识表达和逻辑推理的作用,是重要的模式识别方法。     

On the Use of Space-Environmental Satellite Data for Global Magnetohydrodynamic Simulations

A global magnetohydrodynamic (MHD) model describes the solar-terrestrial system and the physical processes that live in it. Information obtained from satellites provides input to MHD model to compose a more realistic initial state for the equations and, therefore, more accurate simulations. However, the use of high resolution in time data can produce numerical instabilities that quickly interrupt the simulations. Moreover, satellite time series may have gaps which could be a problem in this context. In order to contribute to the overcoming of such challenges, we propose in this work a methodology based on a variant of the continuous wavelet transform to introduce environmental satellite data on the global resistive MHD model originally developed by Prof. Ogino at the University of Nagoya. Our methodology uses a simplified time-scale version of the original data that preserves the most important spectral features of the phenomena of interest. Then, we can do a long-term integration using this MHD model without any computational instability, while preserving the main time-scale features of the original data set and even overcome possible occurrence of gaps on the satellite data. This methodology also contributes to keeping more realistic physical results.     

Neural Dynamics under Active Inference: Plausibility and Efficiency of Information Processing

Active inference is a normative framework for explaining behaviour under the free energy principle—a theory of self-organisation originating in neuroscience. It specifies neuronal dynamics for state-estimation in terms of a descent on (variational) free energy—a measure of the fit between an internal (generative) model and sensory observations. The free energy gradient is a prediction error—plausibly encoded in the average membrane potentials of neuronal populations. Conversely, the expected probability of a state can be expressed in terms of neuronal firing rates. We show that this is consistent with current models of neuronal dynamics and establish face validity by synthesising plausible electrophysiological responses. We then show that these neuronal dynamics approximate natural gradient descent, a well-known optimisation algorithm from information geometry that follows the steepest descent of the objective in information space. We compare the information length of belief updating in both schemes, a measure of the distance travelled in information space that has a direct interpretation in terms of metabolic cost. We show that neural dynamics under active inference are metabolically efficient and suggest that neural representations in biological agents may evolve by approximating steepest descent in information space towards the point of optimal inference.     

Searching for gravitational waves from rotating neutron stars

Rotating neutron stars are one of the important sources of gravitational waves (GW) for the ground based as well as space based detectors. Since the waves are emitted continuously, the source is termed as a continuous gravitational wave (CGW) source. The expected weakness of the signal requires long integration times (∼year). The data analysis problem involves tracking the phase coherently over such large integration times, which makes it the most computationally intensive problem among all GW sources envisaged. In this article, the general problem of data analysis is discussed, and more so, in the context of searching for CGW sources orbiting another companion object. The problem is important because there are several pulsars, which could be deemed to be CGW sources orbiting another companion star. Differential geometric techniques for data analysis are described and used to obtain computational costs. These results are applied to known systems to assess whether such systems are detectable with current (or near future) computing resources.     

Probabilistic Models with Deep Neural Networks

Recent advances in statistical inference have significantly expanded the toolbox of probabilistic modeling. Historically, probabilistic modeling has been constrained to very restricted model classes, where exact or approximate probabilistic inference is feasible. However, developments in variational inference, a general form of approximate probabilistic inference that originated in statistical physics, have enabled probabilistic modeling to overcome these limitations: (i) Approximate probabilistic inference is now possible over a broad class of probabilistic models containing a large number of parameters, and (ii) scalable inference methods based on stochastic gradient descent and distributed computing engines allow probabilistic modeling to be applied to massive data sets. One important practical consequence of these advances is the possibility to include deep neural networks within probabilistic models, thereby capturing complex non-linear stochastic relationships between the random variables. These advances, in conjunction with the release of novel probabilistic modeling toolboxes, have greatly expanded the scope of applications of probabilistic models, and allowed the models to take advantage of the recent strides made by the deep learning community. In this paper, we provide an overview of the main concepts, methods, and tools needed to use deep neural networks within a probabilistic modeling framework.     

Analysis on the Development of Digital Economy in Guangdong Province Based on Improved Entropy Method and Multivariate Statistical Analysis

The lack of adequate indicators in the research of digital economy may lead to the shortage of data support on decision making for governments. To solve this problem, first we establish a digital economy indicator evaluation system by dividing the digital economy into four types: “basic type”, “technology type”, “integration type” and “service type” and select 5 indicators for each type. On this basis, the weight of each indicator is calculated to find the deficiencies in the development of some digital economic fields by the improved entropy method. By drawing on the empowerment idea of Analytic Hierarchy Process, the improved entropy method firstly compares the difference coefficient of indicators in pairs and maps the comparison results to the scales 1–9. Then, the judgment matrix is constructed based on the information entropy, which can solve as much as possible the problem that the difference among the weight of each indicator is too large in traditional entropy method. The results indicate that: the development of digital economy in Guangdong Province was relatively balanced from 2015 to 2018 and will be better in the future while the development of rural e-commerce in Guangdong Province is relatively backward, and there is an obvious digital gap between urban and rural areas. Next we extract two new variables respectively to replace the 20 indicators we select through principal component analysis and factor analysis methods in multivariate statistical analysis, which can retain the original information to the greatest extent and provide convenience for further research in the future. Finally, we and provide constructive comments of digital economy in Guangdong Province from 2015 to 2018.     

Message Passing-Based Inference for Time-Varying Autoregressive Models

Time-varying autoregressive (TVAR) models are widely used for modeling of non-stationary signals. Unfortunately, online joint adaptation of both states and parameters in these models remains a challenge. In this paper, we represent the TVAR model by a factor graph and solve the inference problem by automated message passing-based inference for states and parameters. We derive structured variational update rules for a composite “AR node” with probabilistic observations that can be used as a plug-in module in hierarchical models, for example, to model the time-varying behavior of the hyper-parameters of a time-varying AR model. Our method includes tracking of variational free energy (FE) as a Bayesian measure of TVAR model performance. The proposed methods are verified on a synthetic data set and validated on real-world data from temperature modeling and speech enhancement tasks.