Discovering Noncritical Organization: Statistical Mechanical,Information Theoretic,and Computational Views of Patterns in One-Dimensional Spin Systems

We compare and contrast three different, but complementary views of “structure” and “pattern” in spatial processes. For definiteness and analytical clarity, we apply all three approaches to the simplest class of spatial processes: one-dimensional Ising spin systems with finite-range interactions. These noncritical systems are well-suited for this study since the change in structure as a function of system parameters is more subtle than that found in critical systems where, at a phase transition, many observables diverge, thereby making the detection of change in structure obvious. This survey demonstrates that the measures of pattern from information theory and computational mechanics differ from known thermodynamic and statistical mechanical functions. Moreover, they capture important structural features that are otherwise missed. In particular, a type of mutual information called the excess entropy—an information theoretic measure of memory—serves to detect ordered, low entropy density patterns. It is superior in several respects to other functions used to probe structure, such as magnetization and structure factors. ϵ-Machines—the main objects of computational mechanics—are seen to be the most direct approach to revealing the (group and semigroup) symmetries possessed by the spatial patterns and to estimating the minimum amount of memory required to reproduce the configuration ensemble, a quantity known as the statistical complexity. Finally, we argue that the information theoretic and computational mechanical analyses of spatial patterns capture the intrinsic computational capabilities embedded in spin systems—how they store, transmit, and manipulate configurational information to produce spatial structure.     

Information Fragmentation,Encryption and Information Flow in Complex Biological Networks

Assessing where and how information is stored in biological networks (such as neuronal and genetic networks) is a central task both in neuroscience and in molecular genetics, but most available tools focus on the network’s structure as opposed to its function. Here, we introduce a new information-theoretic tool—information fragmentation analysis—that, given full phenotypic data, allows us to localize information in complex networks, determine how fragmented (across multiple nodes of the network) the information is, and assess the level of encryption of that information. Using information fragmentation matrices we can also create information flow graphs that illustrate how information propagates through these networks. We illustrate the use of this tool by analyzing how artificial brains that evolved in silico solve particular tasks, and show how information fragmentation analysis provides deeper insights into how these brains process information and “think”. The measures of information fragmentation and encryption that result from our methods also quantify complexity of information processing in these networks and how this processing complexity differs between primary exposure to sensory data (early in the lifetime) and later routine processing.     

Integrated Information in the Spiking–Bursting Stochastic Model

Integrated information has been recently suggested as a possible measure to identify a necessary condition for a system to display conscious features. Recently, we have shown that astrocytes contribute to the generation of integrated information through the complex behavior of neuron–astrocyte networks. Still, it remained unclear which underlying mechanisms governing the complex behavior of a neuron–astrocyte network are essential to generating positive integrated information. This study presents an analytic consideration of this question based on exact and asymptotic expressions for integrated information in terms of exactly known probability distributions for a reduced mathematical model (discrete-time, discrete-state stochastic model) reflecting the main features of the “spiking–bursting” dynamics of a neuron–astrocyte network. The analysis was performed in terms of the empirical “whole minus sum” version of integrated information in comparison to the “decoder based” version. The “whole minus sum” information may change sign, and an interpretation of this transition in terms of “net synergy” is available in the literature. This motivated our particular interest in the sign of the “whole minus sum” information in our analytical considerations. The behaviors of the “whole minus sum” and “decoder based” information measures are found to bear a lot of similarity—they have mutual asymptotic convergence as time-uncorrelated activity increases, and the sign transition of the “whole minus sum” information is associated with a rapid growth in the “decoder based” information. The study aims at creating a theoretical framework for using the spiking–bursting model as an analytically tractable reference point for applying integrated information concepts to systems exhibiting similar bursting behavior. The model can also be of interest as a new discrete-state test bench for different formulations of integrated information.     

Attractor-Specific and Common Expression Values in Random Boolean Network Models (with a Preliminary Look at Single-Cell Data)

Random Boolean Networks (RBNs for short) are strongly simplified models of gene regulatory networks (GRNs), which have also been widely studied as abstract models of complex systems and have been used to simulate different phenomena. We define the “common sea” (CS) as the set of nodes that take the same value in all the attractors of a given network realization, and the “specific part” (SP) as the set of all the other nodes, and we study their properties in different ensembles, generated with different parameter values. Both the CS and of the SP can be composed of one or more weakly connected components, which are emergent intermediate-level structures. We show that the study of these sets provides very important information about the behavior of the model. The distribution of distances between attractors is also examined. Moreover, we show how the notion of a “common sea” of genes can be used to analyze data from single-cell experiments.     

Computational Creativity and Aesthetics with Algorithmic Information Theory

We build an analysis based on the Algorithmic Information Theory of computational creativity and extend it to revisit computational aesthetics, thereby, improving on the existing efforts of its formulation. We discuss Kolmogorov complexity, models and randomness deficiency (which is a measure of how much a model falls short of capturing the regularities in an artifact) and show that the notions of typicality and novelty of a creative artifact follow naturally from such definitions. Other exciting formalizations of aesthetic measures include logical depth and sophistication with which we can define, respectively, the value and creator’s artistry present in a creative work. We then look at some related research that combines information theory and creativity and analyze them with the algorithmic tools that we develop throughout the paper. Finally, we assemble the ideas and their algorithmic counterparts to complete an algorithmic information theoretic recipe for computational creativity and aesthetics.     

Financial Return Distributions: Past,Present, and COVID-19

We analyze the price return distributions of currency exchange rates, cryptocurrencies, and contracts for differences (CFDs) representing stock indices, stock shares, and commodities. Based on recent data from the years 2017–2020, we model tails of the return distributions at different time scales by using power-law, stretched exponential, and q-Gaussian functions. We focus on the fitted function parameters and how they change over the years by comparing our results with those from earlier studies and find that, on the time horizons of up to a few minutes, the so-called “inverse-cubic power-law” still constitutes an appropriate global reference. However, we no longer observe the hypothesized universal constant acceleration of the market time flow that was manifested before in an ever faster convergence of empirical return distributions towards the normal distribution. Our results do not exclude such a scenario but, rather, suggest that some other short-term processes related to a current market situation alter market dynamics and may mask this scenario. Real market dynamics is associated with a continuous alternation of different regimes with different statistical properties. An example is the COVID-19 pandemic outburst, which had an enormous yet short-time impact on financial markets. We also point out that two factors—speed of the market time flow and the asset cross-correlation magnitude—while related (the larger the speed, the larger the cross-correlations on a given time scale), act in opposite directions with regard to the return distribution tails, which can affect the expected distribution convergence to the normal distribution.     

Accessing Active Inference Theory through Its Implicit and Deliberative Practice in Human Organizations

Active inference theory (AIT) is a corollary of the free-energy principle, which formalizes cognition of living system’s autopoietic organization. AIT comprises specialist terminology and mathematics used in theoretical neurobiology. Yet, active inference is common practice in human organizations, such as private companies, public institutions, and not-for-profits. Active inference encompasses three interrelated types of actions, which are carried out to minimize uncertainty about how organizations will survive. The three types of action are updating work beliefs, shifting work attention, and/or changing how work is performed. Accordingly, an alternative starting point for grasping active inference, rather than trying to understand AIT specialist terminology and mathematics, is to reflect upon lived experience. In other words, grasping active inference through autoethnographic research. In this short communication paper, accessing AIT through autoethnography is explained in terms of active inference in existing organizational practice (implicit active inference), new organizational methodologies that are informed by AIT (deliberative active inference), and combining implicit and deliberative active inference. In addition, these autoethnographic options for grasping AIT are related to generative learning.     

Conditional Action and Imperfect Erasure of Qubits

We consider state changes in quantum theory due to “conditional action” and relate these to the discussion of entropy decrease due to interventions of “intelligent beings” and the principles of Szilard and Landauer/Bennett. The mathematical theory of conditional actions is a special case of the theory of “instruments”, which describes changes of state due to general measurements and will therefore be briefly outlined in the present paper. As a detailed example, we consider the imperfect erasure of a qubit that can also be viewed as a conditional action and will be realized by the coupling of a spin to another small spin system in its ground state.     

Missing data in image and signal processing: The case of binary objects

I investigated which portions of the Fourier transform of binary signals, images and three-dimensional objects are necessary to correctly identify an object in the presence of noise. This is practically possible for very small binary data sets since the total number of possible objects is then very limited. There are for example 512 different binary images with 9 pixels. It is easy to see that this number soon becomes impractically large for bigger images or if one allows more than two possible pixel values. It turns out that even in the presence of large amounts of noise a relatively small portion of the Fourier transform is essential for deciding which of all possible binary objects the Fourier transform belongs to. These ‘decision experiments’ can be used as a standard for how well algorithms for retrieval of missing Fourier components perform. In another set of computer experiments I investigate the possibility of retrieving various missing Fourier components algorithmically. The main finding of this second set of computer experiments is that the simple retrieval algorithm (a limited form of ‘projection onto convex sets’) used falls very much short of what one might expect from the ‘decision experiments’.I conclude with a discussion what this discrepancy might be due to and some suggestions how to improve the performance of retrieval algorithms for binary objects.     

Probabilistic Inference with Polymerizing Biochemical Circuits

Probabilistic inference—the process of estimating the values of unobserved variables in probabilistic models—has been used to describe various cognitive phenomena related to learning and memory. While the study of biological realizations of inference has focused on animal nervous systems, single-celled organisms also show complex and potentially “predictive” behaviors in changing environments. Yet, it is unclear how the biochemical machinery found in cells might perform inference. Here, we show how inference in a simple Markov model can be approximately realized, in real-time, using polymerizing biochemical circuits. Our approach relies on assembling linear polymers that record the history of environmental changes, where the polymerization process produces molecular complexes that reflect posterior probabilities. We discuss the implications of realizing inference using biochemistry, and the potential of polymerization as a form of biological information-processing.     

Nature Has No Elementary Particles and Makes No Measurements or Predictions: Quantum Measurement and Quantum Theory,from Bohr to Bell and from Bell to Bohr

This article reconsiders the concept of physical reality in quantum theory and the concept of quantum measurement, following Bohr, whose analysis of quantum measurement led him to his concept of a (quantum) “phenomenon,” referring to “the observations obtained under the specified circumstances,” in the interaction between quantum objects and measuring instruments. This situation makes the terms “observation” and “measurement,” as conventionally understood, inapplicable. These terms are remnants of classical physics or still earlier history, from which classical physics inherited it. As defined here, a quantum measurement does not measure any preexisting property of the ultimate constitution of the reality responsible for quantum phenomena. An act of measurement establishes a quantum phenomenon by an interaction between the instrument and the quantum object or in the present view the ultimate constitution of the reality responsible for quantum phenomena and, at the time of measurement, also quantum objects. In the view advanced in this article, in contrast to that of Bohr, quantum objects, such as electrons or photons, are assumed to exist only at the time of measurement and not independently, a view that redefines the concept of quantum object as well. This redefinition becomes especially important in high-energy quantum regimes and quantum field theory and allows this article to define a new concept of quantum field. The article also considers, now following Bohr, the quantum measurement as the entanglement between quantum objects and measurement instruments. The argument of the article is grounded in the concept “reality without realism” (RWR), as underlying quantum measurement thus understood, and the view, the RWR view, of quantum theory defined by this concept. The RWR view places a stratum of physical reality thus designated, here the reality ultimately responsible for quantum phenomena, beyond representation or knowledge, or even conception, and defines the corresponding set of interpretations quantum mechanics or quantum field theory, such as the one assumed in this article, in which, again, not only quantum phenomena but also quantum objects are (idealizations) defined by measurement. As such, the article also offers a broadly conceived response to J. Bell’s argument “against ‘measurement’”.     

Functional Kernel Density Estimation: Point and Fourier Approaches to Time Series Anomaly Detection

We present an unsupervised method to detect anomalous time series among a collection of time series. To do so, we extend traditional Kernel Density Estimation for estimating probability distributions in Euclidean space to Hilbert spaces. The estimated probability densities we derive can be obtained formally through treating each series as a point in a Hilbert space, placing a kernel at those points, and summing the kernels (a “point approach”), or through using Kernel Density Estimation to approximate the distributions of Fourier mode coefficients to infer a probability density (a “Fourier approach”). We refer to these approaches as Functional Kernel Density Estimation for Anomaly Detection as they both yield functionals that can score a time series for how anomalous it is. Both methods naturally handle missing data and apply to a variety of settings, performing well when compared with an outlyingness score derived from a boxplot method for functional data, with a Principal Component Analysis approach for functional data, and with the Functional Isolation Forest method. We illustrate the use of the proposed methods with aviation safety report data from the International Air Transport Association (IATA).     

Coupled Criticality Analysis of Inflation and Unemployment

In this paper, we focus on the critical periods in the economy that are characterized by unusual and large fluctuations in macroeconomic indicators, like those measuring inflation and unemployment. We analyze U.S. data for 70 years from 1948 until 2018. To capture their fluctuation essence, we concentrate on the non-Gaussianity of their distributions. We investigate how the non-Gaussianity of these variables affects the coupling structure of them. We distinguish “regular” from “rare” events, in calculating the correlation coefficient, emphasizing that both cases might lead to a different response of the economy. Through the “multifractal random wall” model, one can see that the non-Gaussianity depends on time scales. The non-Gaussianity of unemployment is noticeable only for periods shorter than one year; for longer periods, the fluctuation distribution tends to a Gaussian behavior. In contrast, the non-Gaussianities of inflation fluctuations persist for all time scales. We observe through the “bivariate multifractal random walk” that despite the inflation features, the non-Gaussianity of the coupled structure is finite for scales less than one year, drops for periods larger than one year, and becomes small for scales greater than two years. This means that the footprint of the monetary policies intentionally influencing the inflation and unemployment couple is observed only for time horizons smaller than two years. Finally, to improve some understanding of the effect of rare events, we calculate high moments of the variables’ increments for various q orders and various time scales. The results show that coupling with high moments sharply increases during crises.     

Timelessness Strictly inside the Quantum Realm

Time is one of the undisputed foundations of our life in the real world. Here it is argued that inside small isolated quantum systems, time does not pass as we are used to, and it is primarily in this sense that quantum objects enjoy only limited reality. Quantum systems, which we know, are embedded in the everyday classical world. Their preparation as well as their measurement-phases leave durable records and traces in the entropy of the environment. The Landauer Principle then gives a quantitative threshold for irreversibility. With double slit experiments and tunneling as paradigmatic examples, it is proposed that a label of timelessness offers clues for rendering a Copenhagen-type interpretation of quantum physics more “realistic” and acceptable by providing a coarse but viable link from the fundamental quantum realm to the classical world which humans directly experience.     

From Quantum Probabilities to Quantum Amplitudes

The task of reconstructing the system’s state from the measurements results, known as the Pauli problem, usually requires repetition of two successive steps. Preparation in an initial state to be determined is followed by an accurate measurement of one of the several chosen operators in order to provide the necessary “Pauli data”. We consider a similar yet more general problem of recovering Feynman’s transition (path) amplitudes from the results of at least three consecutive measurements. The three-step histories of a pre- and post-selected quantum system are subjected to a type of interference not available to their two-step counterparts. We show that this interference can be exploited, and if the intermediate measurement is “fuzzy”, the path amplitudes can be successfully recovered. The simplest case of a two-level system is analysed in detail. The “weak measurement” limit and the usefulness of the path amplitudes are also discussed.     

Chess AI: Competing Paradigms for Machine Intelligence

Endgame studies have long served as a tool for testing human creativity and intelligence. We find that they can serve as a tool for testing machine ability as well. Two of the leading chess engines, Stockfish and Leela Chess Zero (LCZero), employ significantly different methods during play. We use Plaskett’s Puzzle, a famous endgame study from the late 1970s, to compare the two engines. Our experiments show that Stockfish outperforms LCZero on the puzzle. We examine the algorithmic differences between the engines and use our observations as a basis for carefully interpreting the test results. Drawing inspiration from how humans solve chess problems, we ask whether machines can possess a form of imagination. On the theoretical side, we describe how Bellman’s equation may be applied to optimize the probability of winning. To conclude, we discuss the implications of our work on artificial intelligence (AI) and artificial general intelligence (AGI), suggesting possible avenues for future research.     

What Should I Notice? Using Algorithmic Information Theory to Evaluate the Memorability of Events in Smart Homes

With the increasing number of connected devices, complex systems such as smart homes record a multitude of events of various types, magnitude and characteristics. Current systems struggle to identify which events can be considered more memorable than others. In contrast, humans are able to quickly categorize some events as being more “memorable” than others. They do so without relying on knowledge of the system’s inner working or large previous datasets. Having this ability would allow the system to: (i) identify and summarize a situation to the user by presenting only memorable events; (ii) suggest the most memorable events as possible hypotheses in an abductive inference process. Our proposal is to use Algorithmic Information Theory to define a “memorability” score by retrieving events using predicative filters. We use smart-home examples to illustrate how our theoretical approach can be implemented in practice.     

A Locust-Inspired Model of Collective Marching on Rings

We study the collective motion of autonomous mobile agents in a ringlike environment. The agents’ dynamics are inspired by known laboratory experiments on the dynamics of locust swarms. In these experiments, locusts placed at arbitrary locations and initial orientations on a ring-shaped arena are observed to eventually all march in the same direction. In this work we ask whether, and how fast, a similar phenomenon occurs in a stochastic swarm of simple locust-inspired agents. The agents are randomly initiated as marching either clockwise or counterclockwise on a discretized, wide ring-shaped region, which we subdivide into k concentric tracks of length n. Collisions cause agents to change their direction of motion. To avoid this, agents may decide to switch tracks to merge with platoons of agents marching in their direction. We prove that such agents must eventually converge to a local consensus about their direction of motion, meaning that all agents on each narrow track must eventually march in the same direction. We give asymptotic bounds for the expected time it takes for such convergence or “stabilization” to occur, which depends on the number of agents, the length of the tracks, and the number of tracks. We show that when agents also have a small probability of “erratic”, random track-jumping behavior, a global consensus on the direction of motion across all tracks will eventually be reached. Finally, we verify our theoretical findings in numerical simulations.     

Characterizing Topics in Social Media Using Dynamics of Conversation

Online social media provides massive open-ended platforms for users of a wide variety of backgrounds, interests, and beliefs to interact and debate, facilitating countless discussions across a myriad of subjects. With numerous unique voices being lent to the ever-growing information stream, it is essential to consider how the types of conversations that result from a social media post represent the post itself. We hypothesize that the biases and predispositions of users cause them to react to different topics in different ways not necessarily entirely intended by the sender. In this paper, we introduce a set of unique features that capture patterns of discourse, allowing us to empirically explore the relationship between a topic and the conversations it induces. Utilizing “microscopic” trends to describe “macroscopic” phenomena, we set a paradigm for analyzing information dissemination through the user reactions that arise from a topic, eliminating the need to analyze the involved text of the discussions. Using a Reddit dataset, we find that our features not only enable classifiers to accurately distinguish between content genre, but also can identify more subtle semantic differences in content under a single topic as well as isolating outliers whose subject matter is substantially different from the norm.     

Video Action Recognition Using Motion and Multi-View Excitation with Temporal Aggregation

Spatiotemporal and motion feature representations are the key to video action recognition. Typical previous approaches are to utilize 3D CNNs to cope with both spatial and temporal features, but they suffer from huge computations. Other approaches are to utilize (1+2)D CNNs to learn spatial and temporal features in an efficient way, but they neglect the importance of motion representations. To overcome problems with previous approaches, we propose a novel block which makes it possible to alleviate the aforementioned problems, since our block can capture spatial and temporal features more faithfully and efficiently learn motion features. This proposed block includes Motion Excitation (ME), Multi-view Excitation (MvE), and Densely Connected Temporal Aggregation (DCTA). The purpose of ME is to encode feature-level frame differences; MvE is designed to enrich spatiotemporal features with multiple view representations adaptively; and DCTA is to model long-range temporal dependencies. We inject the proposed building block, which we refer to as the META block (or simply “META”), into 2D ResNet-50. Through extensive experiments, we demonstrate that our proposed method architecture outperforms previous CNN-based methods in terms of “Val Top-1 %” measure with Something-Something v1 and Jester datasets, while the META yielded competitive results with the Moment-in-Time Mini dataset.