Memory and Markov Blankets

In theoretical biology, we are often interested in random dynamical systems—like the brain—that appear to model their environments. This can be formalized by appealing to the existence of a (possibly non-equilibrium) steady state, whose density preserves a conditional independence between a biological entity and its surroundings. From this perspective, the conditioning set, or Markov blanket, induces a form of vicarious synchrony between creature and world—as if one were modelling the other. However, this results in an apparent paradox. If all conditional dependencies between a system and its surroundings depend upon the blanket, how do we account for the mnemonic capacity of living systems? It might appear that any shared dependence upon past blanket states violates the independence condition, as the variables on either side of the blanket now share information not available from the current blanket state. This paper aims to resolve this paradox, and to demonstrate that conditional independence does not preclude memory. Our argument rests upon drawing a distinction between the dependencies implied by a steady state density, and the density dynamics of the system conditioned upon its configuration at a previous time. The interesting question then becomes: What determines the length of time required for a stochastic system to ‘forget’ its initial conditions? We explore this question for an example system, whose steady state density possesses a Markov blanket, through simple numerical analyses. We conclude with a discussion of the relevance for memory in cognitive systems like us.     

Neural Dynamics under Active Inference: Plausibility and Efficiency of Information Processing

Active inference is a normative framework for explaining behaviour under the free energy principle—a theory of self-organisation originating in neuroscience. It specifies neuronal dynamics for state-estimation in terms of a descent on (variational) free energy—a measure of the fit between an internal (generative) model and sensory observations. The free energy gradient is a prediction error—plausibly encoded in the average membrane potentials of neuronal populations. Conversely, the expected probability of a state can be expressed in terms of neuronal firing rates. We show that this is consistent with current models of neuronal dynamics and establish face validity by synthesising plausible electrophysiological responses. We then show that these neuronal dynamics approximate natural gradient descent, a well-known optimisation algorithm from information geometry that follows the steepest descent of the objective in information space. We compare the information length of belief updating in both schemes, a measure of the distance travelled in information space that has a direct interpretation in terms of metabolic cost. We show that neural dynamics under active inference are metabolically efficient and suggest that neural representations in biological agents may evolve by approximating steepest descent in information space towards the point of optimal inference.     

Causality in Reversed Time Series: Reversed or Conserved?

The inference of causal relations between observable phenomena is paramount across scientific disciplines; however, the means for such enterprise without experimental manipulation are limited. A commonly applied principle is that of the cause preceding and predicting the effect, taking into account other circumstances. Intuitively, when the temporal order of events is reverted, one would expect the cause and effect to apparently switch roles. This was previously demonstrated in bivariate linear systems and used in design of improved causal inference scores, while such behaviour in linear systems has been put in contrast with nonlinear chaotic systems where the inferred causal direction appears unchanged under time reversal. The presented work explores the conditions under which the causal reversal happens—either perfectly, approximately, or not at all—using theoretical analysis, low-dimensional examples, and network simulations, focusing on the simplified yet illustrative linear vector autoregressive process of order one. We start with a theoretical analysis that demonstrates that a perfect coupling reversal under time reversal occurs only under very specific conditions, followed up by constructing low-dimensional examples where indeed the dominant causal direction is even conserved rather than reversed. Finally, simulations of random as well as realistically motivated network coupling patterns from brain and climate show that level of coupling reversal and conservation can be well predicted by asymmetry and anormality indices introduced based on the theoretical analysis of the problem. The consequences for causal inference are discussed.     

Probabilistic Inference with Polymerizing Biochemical Circuits

Probabilistic inference—the process of estimating the values of unobserved variables in probabilistic models—has been used to describe various cognitive phenomena related to learning and memory. While the study of biological realizations of inference has focused on animal nervous systems, single-celled organisms also show complex and potentially “predictive” behaviors in changing environments. Yet, it is unclear how the biochemical machinery found in cells might perform inference. Here, we show how inference in a simple Markov model can be approximately realized, in real-time, using polymerizing biochemical circuits. Our approach relies on assembling linear polymers that record the history of environmental changes, where the polymerization process produces molecular complexes that reflect posterior probabilities. We discuss the implications of realizing inference using biochemistry, and the potential of polymerization as a form of biological information-processing.     

Thermodynamic State Machine Network

We describe a model system—a thermodynamic state machine network—comprising a network of probabilistic, stateful automata that equilibrate according to Boltzmann statistics, exchange codes over unweighted bi-directional edges, update a state transition memory to learn transitions between network ground states, and minimize an action associated with fluctuation trajectories. The model is grounded in four postulates concerning self-organizing, open thermodynamic systems—transport-driven self-organization, scale-integration, input-functionalization, and active equilibration. After sufficient exposure to periodically changing inputs, a diffusive-to-mechanistic phase transition emerges in the network dynamics. The evolved networks show spatial and temporal structures that look much like spiking neural networks, although no such structures were incorporated into the model. Our main contribution is the articulation of the postulates, the development of a thermodynamically motivated methodology addressing them, and the resulting phase transition. As with other machine learning methods, the model is limited by its scalability, generality, and temporality. We use limitations to motivate the development of thermodynamic computers—engineered, thermodynamically self-organizing systems—and comment on efforts to realize them in the context of this work. We offer a different philosophical perspective, thermodynamicalism, addressing the limitations of the model and machine learning in general.     

Hierarchy Depth in Directed Networks

In this study, we explore the depth measures for flow hierarchy in directed networks. Two simple measures are defined—rooted depth and relative depth—and their properties are discussed. The method of loop collapse is introduced, allowing investigation of networks containing directed cycles. The behavior of the two depth measures is investigated in Erdös-Rényi random graphs, directed Barabási-Albert networks, and in Gnutella p2p share network. A clear distinction in the behavior between non-hierarchical and hierarchical networks is found, with random graphs featuring unimodal distribution of depths dependent on arc density, while for hierarchical systems the distributions are similar for different network densities. Relative depth shows the same behavior as existing trophic level measure for tree-like networks, but is only statistically correlated for more complex topologies, including acyclic directed graphs.     

A Drive towards Thermodynamic Efficiency for Dissipative Structures in Chemical Reaction Networks

Dissipative accounts of structure formation show that the self-organisation of complex structures is thermodynamically favoured, whenever these structures dissipate free energy that could not be accessed otherwise. These structures therefore open transition channels for the state of the universe to move from a frustrated, metastable state to another metastable state of higher entropy. However, these accounts apply as well to relatively simple, dissipative systems, such as convection cells, hurricanes, candle flames, lightning strikes, or mechanical cracks, as they do to complex biological systems. Conversely, interesting computational properties—that characterize complex biological systems, such as efficient, predictive representations of environmental dynamics—can be linked to the thermodynamic efficiency of underlying physical processes. However, the potential mechanisms that underwrite the selection of dissipative structures with thermodynamically efficient subprocesses is not completely understood. We address these mechanisms by explaining how bifurcation-based, work-harvesting processes—required to sustain complex dissipative structures—might be driven towards thermodynamic efficiency. We first demonstrate a simple mechanism that leads to self-selection of efficient dissipative structures in a stochastic chemical reaction network, when the dissipated driving chemical potential difference is decreased. We then discuss how such a drive can emerge naturally in a hierarchy of self-similar dissipative structures, each feeding on the dissipative structures of a previous level, when moving away from the initial, driving disequilibrium.     

Do Co-Worker Networks Increase or Decrease Productivity Differences?

Do labor mobility and co-worker networks contribute to convergence or divergence between regions? Based on the previous literature, labor mobility contributes to knowledge transfer between firms. Therefore, mobility may contribute to decreasing productivity differences, while limited mobility sustains higher differences. The effect of co-worker networks, however, can be two-fold in this process; they transmit information about potential jobs, which may enhance the mobility of workers—even between regions—and this enhanced mobility may contribute to levelling of differences. However, if mobility between regions involves movement costs, co-worker networks may concentrate locally—possibly contributing to the persistence of regional differences. In this paper, we build an agent-based model of labor mobility across firms and regions with knowledge spillovers that reflects key empirical observations on labor markets. We analyze the impact of network information provided about potential employers in this model and find that it contributes to increasing inter-regional mobility, and subsequently, to decreasing regional differences. We also find that both the density of coworker networks, as well as their regional concentrations, decrease if network information is available.     

Some Interesting Observations on the Free Energy Principle

Biehl et al. (2021) present some interesting observations on an early formulation of the free energy principle. We use these observations to scaffold a discussion of the technical arguments that underwrite the free energy principle. This discussion focuses on solenoidal coupling between various (subsets of) states in sparsely coupled systems that possess a Markov blanket—and the distinction between exact and approximate Bayesian inference, implied by the ensuing Bayesian mechanics.     

Fluctuation Relations for Dissipative Systems in Constant External Magnetic Field: Theory and Molecular Dynamics Simulations

We illustrate how, contrary to common belief, transient Fluctuation Relations (FRs) for systems in constant external magnetic field hold without the inversion of the field. Building on previous work providing generalized time-reversal symmetries for systems in parallel external magnetic and electric fields, we observe that the standard proof of these important nonequilibrium properties can be fully reinstated in the presence of net dissipation. This generalizes recent results for the FRs in orthogonal fields—an interesting but less commonly investigated geometry—and enables direct comparison with existing literature. We also present for the first time a numerical demonstration of the validity of the transient FRs with nonzero magnetic field via nonequilibrium molecular dynamics simulations of a realistic model of liquid NaCl.     

Testing Jump-Diffusion in Epileptic Brain Dynamics: Impact of Daily Rhythms

Stochastic approaches to complex dynamical systems have recently provided broader insights into spatial-temporal aspects of epileptic brain dynamics. Stochastic qualifiers based on higher-order Kramers-Moyal coefficients derived directly from time series data indicate improved differentiability between physiological and pathophysiological brain dynamics. It remains unclear, however, to what extent stochastic qualifiers of brain dynamics are affected by other endogenous and/or exogenous influencing factors. Addressing this issue, we investigate multi-day, multi-channel electroencephalographic recordings from a subject with epilepsy. We apply a recently proposed criterion to differentiate between Langevin-type and jump-diffusion processes and observe the type of process most qualified to describe brain dynamics to change with time. Stochastic qualifiers of brain dynamics are strongly affected by endogenous and exogenous rhythms acting on various time scales—ranging from hours to days. Such influences would need to be taken into account when constructing evolution equations for the epileptic brain or other complex dynamical systems subject to external forcings.     

Reconstructing Nonparametric Productivity Networks

Network models provide a general representation of inter-connected system dynamics. This ability to connect systems has led to a proliferation of network models for economic productivity analysis, primarily estimated non-parametrically using Data Envelopment Analysis (DEA). While network DEA models can be used to measure system performance, they lack a statistical framework for inference, due in part to the complex structure of network processes. We fill this gap by developing a general framework to infer the network structure in a Bayesian sense, in order to better understand the underlying relationships driving system performance. Our approach draws on recent advances in information science, machine learning and statistical inference from the physics of complex systems to estimate unobserved network linkages. To illustrate, we apply our framework to analyze the production of knowledge, via own and cross-disciplinary research, for a world-country panel of bibliometric data. We find significant interactions between related disciplinary research output, both in terms of quantity and quality. In the context of research productivity, our results on cross-disciplinary linkages could be used to better target research funding across disciplines and institutions. More generally, our framework for inferring the underlying network production technology could be applied to both public and private settings which entail spillovers, including intra- and inter-firm managerial decisions and public agency coordination. This framework also provides a systematic approach to model selection when the underlying network structure is unknown.     

Epistemic Communities under Active Inference

The spread of ideas is a fundamental concern of today’s news ecology. Understanding the dynamics of the spread of information and its co-option by interested parties is of critical importance. Research on this topic has shown that individuals tend to cluster in echo-chambers and are driven by confirmation bias. In this paper, we leverage the active inference framework to provide an in silico model of confirmation bias and its effect on echo-chamber formation. We build a model based on active inference, where agents tend to sample information in order to justify their own view of reality, which eventually leads to them to have a high degree of certainty about their own beliefs. We show that, once agents have reached a certain level of certainty about their beliefs, it becomes very difficult to get them to change their views. This system of self-confirming beliefs is upheld and reinforced by the evolving relationship between an agent’s beliefs and observations, which over time will continue to provide evidence for their ingrained ideas about the world. The epistemic communities that are consolidated by these shared beliefs, in turn, tend to produce perceptions of reality that reinforce those shared beliefs. We provide an active inference account of this community formation mechanism. We postulate that agents are driven by the epistemic value that they obtain from sampling or observing the behaviours of other agents. Inspired by digital social networks like Twitter, we build a generative model in which agents generate observable social claims or posts (e.g., ‘tweets’) while reading the socially observable claims of other agents that lend support to one of two mutually exclusive abstract topics. Agents can choose which other agent they pay attention to at each timestep, and crucially who they attend to and what they choose to read influences their beliefs about the world. Agents also assess their local network’s perspective, influencing which kinds of posts they expect to see other agents making. The model was built and simulated using the freely available Python package pymdp. The proposed active inference model can reproduce the formation of echo-chambers over social networks, and gives us insight into the cognitive processes that lead to this phenomenon.     

Entropy,Information, and the Updating of Probabilities

This paper is a review of a particular approach to the method of maximum entropy as a general framework for inference. The discussion emphasizes pragmatic elements in the derivation. An epistemic notion of information is defined in terms of its relation to the Bayesian beliefs of ideally rational agents. The method of updating from a prior to posterior probability distribution is designed through an eliminative induction process. The logarithmic relative entropy is singled out as a unique tool for updating (a) that is of universal applicability, (b) that recognizes the value of prior information, and (c) that recognizes the privileged role played by the notion of independence in science. The resulting framework—the ME method—can handle arbitrary priors and arbitrary constraints. It includes the MaxEnt and Bayes’ rules as special cases and, therefore, unifies entropic and Bayesian methods into a single general inference scheme. The ME method goes beyond the mere selection of a single posterior, and also addresses the question of how much less probable other distributions might be, which provides a direct bridge to the theories of fluctuations and large deviations.     

A Thermodynamic Approach to Measuring Entropy in a Few-Electron Nanodevice

The entropy of a system gives a powerful insight into its microscopic degrees of freedom; however, standard experimental ways of measuring entropy through heat capacity are hard to apply to nanoscale systems, as they require the measurement of increasingly small amounts of heat. Two alternative entropy measurement methods have been recently proposed for nanodevices: through charge balance measurements and transport properties. We describe a self-consistent thermodynamic framework for applying thermodynamic relations to few-electron nanodevices—small systems, where fluctuations in particle number are significant, whilst highlighting several ongoing misconceptions. We derive a relation (a consequence of a Maxwell relation for small systems), which describes both existing entropy measurement methods as special cases, while also allowing the experimentalist to probe the intermediate regime between them. Finally, we independently prove the applicability of our framework in systems with complex microscopic dynamics—those with many excited states of various degeneracies—from microscopic considerations.     

Temporal networks

A great variety of systems in nature, society and technology–from the web of sexual contacts to the Internet, from the nervous system to power grids–can be modeled as graphs of vertices coupled by edges. The network structure, describing how the graph is wired, helps us understand, predict and optimize the behavior of dynamical systems. In many cases, however, the edges are not continuously active. As an example, in networks of communication via e-mail, text messages, or phone calls, edges represent sequences of instantaneous or practically instantaneous contacts. In some cases, edges are active for non-negligible periods of time: e.g., the proximity patterns of inpatients at hospitals can be represented by a graph where an edge between two individuals is on throughout the time they are at the same ward. Like network topology, the temporal structure of edge activations can affect dynamics of systems interacting through the network, from disease contagion on the network of patients to information diffusion over an e-mail network. In this review, we present the emergent field of temporal networks, and discuss methods for analyzing topological and temporal structure and models for elucidating their relation to the behavior of dynamical systems. In the light of traditional network theory, one can see this framework as moving the information of when things happen from the dynamical system on the network, to the network itself. Since fundamental properties, such as the transitivity of edges, do not necessarily hold in temporal networks, many of these methods need to be quite different from those for static networks. The study of temporal networks is very interdisciplinary in nature. Reflecting this, even the object of study has many names—temporal graphs, evolving graphs, time-varying graphs, time-aggregated graphs, time-stamped graphs, dynamic networks, dynamic graphs, dynamical graphs, and so on. This review covers different fields where temporal graphs are considered, but does not attempt to unify related terminology—rather, we want to make papers readable across disciplines.     

Dynamic Expectation Maximization Algorithm for Estimation of Linear Systems with Colored Noise

The free energy principle from neuroscience has recently gained traction as one of the most prominent brain theories that can emulate the brain’s perception and action in a bio-inspired manner. This renders the theory with the potential to hold the key for general artificial intelligence. Leveraging this potential, this paper aims to bridge the gap between neuroscience and robotics by reformulating an FEP-based inference scheme—Dynamic Expectation Maximization—into an algorithm that can perform simultaneous state, input, parameter, and noise hyperparameter estimation of any stable linear state space system subjected to colored noises. The resulting estimator was proved to be of the form of an augmented coupled linear estimator. Using this mathematical formulation, we proved that the estimation steps have theoretical guarantees of convergence. The algorithm was rigorously tested in simulation on a wide variety of linear systems with colored noises. The paper concludes by demonstrating the superior performance of DEM for parameter estimation under colored noise in simulation, when compared to the state-of-the-art estimators like Sub Space method, Prediction Error Minimization (PEM), and Expectation Maximization (EM) algorithm. These results contribute to the applicability of DEM as a robust learning algorithm for safe robotic applications.     

Dynamics and thermodynamics of a simple model similar to self-gravitating systems: the HMF model

We discuss the dynamics and thermodynamics of the Hamiltonian Mean Field model (HMF) which is a prototypical system with long-range interactions. The HMF model can be seen as the one Fourier component of a one-dimensional self-gravitating system. Interestingly, it exhibits many features of real self-gravitating systems (violent relaxation, persistence of metaequilibrium states, slow collisional dynamics, phase transitions,...) while avoiding complicated problems posed by the singularity of the gravitational potential at short distances and by the absence of a large-scale confinement. We stress the deep analogy between the HMF model and self-gravitating systems by developing a complete parallel between these two systems. This allows us to apply many technics introduced in plasma physics and astrophysics to a new problem and to see how the results depend on the dimension of space and on the form of the potential of interaction. This comparative study brings new light in the statistical mechanics of self-gravitating systems. We also mention simple astrophysical applications of the HMF model in relation with the formation of bars in spiral galaxies.     

Critical Comparison of MaxCal and Other Stochastic Modeling Approaches in Analysis of Gene Networks

Learning the underlying details of a gene network with feedback is critical in designing new synthetic circuits. Yet, quantitative characterization of these circuits remains limited. This is due to the fact that experiments can only measure partial information from which the details of the circuit must be inferred. One potentially useful avenue is to harness hidden information from single-cell stochastic gene expression time trajectories measured for long periods of time—recorded at frequent intervals—over multiple cells. This raises the feasibility vs. accuracy dilemma while deciding between different models of mining these stochastic trajectories. We demonstrate that inference based on the Maximum Caliber (MaxCal) principle is the method of choice by critically evaluating its computational efficiency and accuracy against two other typical modeling approaches: (i) a detailed model (DM) with explicit consideration of multiple molecules including protein-promoter interaction, and (ii) a coarse-grain model (CGM) using Hill type functions to model feedback. MaxCal provides a reasonably accurate model while being significantly more computationally efficient than DM and CGM. Furthermore, MaxCal requires minimal assumptions since it is a top-down approach and allows systematic model improvement by including constraints of higher order, in contrast to traditional bottom-up approaches that require more parameters or ad hoc assumptions. Thus, based on efficiency, accuracy, and ability to build minimal models, we propose MaxCal as a superior alternative to traditional approaches (DM, CGM) when inferring underlying details of gene circuits with feedback from limited data.