Dynamical Field Inference and Supersymmetry

Knowledge on evolving physical fields is of paramount importance in science, technology, and economics. Dynamical field inference (DFI) addresses the problem of reconstructing a stochastically-driven, dynamically-evolving field from finite data. It relies on information field theory (IFT), the information theory for fields. Here, the relations of DFI, IFT, and the recently developed supersymmetric theory of stochastics (STS) are established in a pedagogical discussion. In IFT, field expectation values can be calculated from the partition function of the full space-time inference problem. The partition function of the inference problem invokes a functional Dirac function to guarantee the dynamics, as well as a field-dependent functional determinant, to establish proper normalization, both impeding the necessary evaluation of the path integral over all field configurations. STS replaces these problematic expressions via the introduction of fermionic ghost and bosonic Lagrange fields, respectively. The action of these fields has a supersymmetry, which means there exists an exchange operation between bosons and fermions that leaves the system invariant. In contrast to this, measurements of the dynamical fields do not adhere to this supersymmetry. The supersymmetry can also be broken spontaneously, in which case the system evolves chaotically. This affects the predictability of the system and thereby makes DFI more challenging. We investigate the interplay of measurement constraints with the non-linear chaotic dynamics of a simplified, illustrative system with the help of Feynman diagrams and show that the Fermionic corrections are essential to obtain the correct posterior statistics over system trajectories.     

Information Theory for Fields

A physical field has an infinite number of degrees of freedom since it has a field value at each location of a continuous space. Therefore, it is impossible to know a field from finite measurements alone and prior information on the field is essential for field inference. An information theory for fields is needed to join the measurement and prior information into probabilistic statements on field configurations. Such an information field theory (IFT) is built upon the language of mathematical physics, in particular, on field theory and statistical mechanics. IFT permits the mathematical derivation of optimal imaging algorithms, data analysis methods, and even computer simulation schemes. The application of IFT algorithms to astronomical datasets provides high fidelity images of the Universe and facilitates the search for subtle statistical signals from the Big Bang. The concepts of IFT may even pave the road to novel computer simulations that are aware of their own uncertainties.     

Security concerns on machine learning solutions for 6G networks in mmWave beam prediction

6G – sixth generation – is the latest cellular technology currently under development for wireless communication systems. In recent years, machine learning (ML) algorithms have been applied widely in various fields, such as healthcare, transportation, energy, autonomous cars, and many more. Those algorithms have also been used in communication technologies to improve the system performance in terms of frequency spectrum usage, latency, and security. With the rapid developments of ML techniques, especially deep learning (DL), it is critical to consider the security concern when applying the algorithms. While ML algorithms offer significant advantages for 6G networks, security concerns on artificial intelligence (AI) models are typically ignored by the scientific community so far. However, security is also a vital part of AI algorithms because attackers can poison the AI model itself. This paper proposes a mitigation method for adversarial attacks against proposed 6G ML models for the millimeter-wave (mmWave) beam prediction using adversarial training. The main idea behind generating adversarial attacks against ML models is to produce faulty results by manipulating trained DL models for 6G applications for mmWave beam prediction. We also present a proposed adversarial learning mitigation method’s performance for 6G security in mmWave beam prediction application a fast gradient sign method attack. The results show that the defended model under attack’s mean square errors (i.e., the prediction accuracy) are very close to the undefended model without attack.     

TERFDA: Tensor Embedding RF Domain Adaptation for varying noise interference

Radio frequency machine learning (RFML) can be loosely termed as a field that machine learning (ML) and deep learning (DL) techniques to applications related to wireless communications. However, traditional RFML basically assume that the data of training set and test set are independent and identically distributed and only a large number of labeled data can train a classification model which can effectively classify test set data. In other words, without enough training samples, it is impossible to learn an automatic modulation classifier that performs well in varying noise interference environment. Feature-based transfer learning minimizes the distribution difference between historical modulated signal data and new data by learning similarity-maximizing feature spaces. Therefore, in this paper, Dynamic Distribution Adaptation (DDA) is adopted to address the above challenges. We propose a Tensor Embedding RF Domain Adaptation (TERFDA) approach, which learns the latent subspace of the tensors formed by the time–frequency maps of the signals, so that use the multi-dimensional domain information of the signals to jointly learn the shared feature subspace of the source domain and the target domain, then perform DDA in the shared subspace. The experimental results show that under the modulated signal data, compared with the state-of-the-art DA algorithm, TERFDA has less requirements on the number of samples and categories, and has superior performance for confrontation the varying noise interference between source domain and target domain.     

Deep learning-based channel estimation using Gaussian mixture distribution and expectation maximum algorithm

In massive multiple-input multiple-output (MIMO), it is much challenging to obtain accurate channel state information (CSI) after radio frequency (RF) chain reduction due to the high dimensions. With the fast development of machine learning(ML), it is widely acknowledged that ML is an effective method to deal with channel models which are typically unknown and hard to approximate. In this paper, we use the low complexity vector approximate messaging passing (VAMP) algorithm for channel estimation, combined with a deep learning framework for soft threshold shrinkage function training. Furthermore, in order to improve the estimation accuracy of the algorithm for massive MIMO channels, an optimized threshold function is proposed. This function is based on Gaussian mixture (GM) distribution modeling, and the expectation maximum Algorithm (EM Algorithm) is used to recover the channel information in beamspace. This contraction function and deep neural network are improved on the vector approximate messaging algorithm to form a high-precision channel estimation algorithm. Simulation results validate the effectiveness of the proposed network.     

Active Online Learning in the Binary Perceptron Problem

The binary perceptron is the simplest artificial neural network formed by N input units and one output unit, with the neural states and the synaptic weights all restricted to ±1 values. The task in the teacher-student scenario is to infer the hidden weight vector by training on a set of labeled patterns. Previous efforts on the passive learning mode have shown that learning from independent random patterns is quite inefficient. Here we consider the active online learning mode in which the student designs every new Ising training pattern. We demonstrate that it is mathematically possible to achieve perfect(error-free) inference using only N designed training patterns, but this is computationally unfeasible for large systems. We then investigate two Bayesian statistical designing protocols, which require 2.3N and 1.9N training patterns, respectively, to achieve error-free inference. If the training patterns are instead designed through deductive reasoning, perfect inference is achieved using N + log_2N samples. The performance gap between Bayesian and deductive designing strategies may be shortened in future work by taking into account the possibility of ergodicity breaking in the version space of the binary perceptron.     

Unsupervised LoS/NLoS identification in mmWave communication using two-stage machine learning framework

Identification of line-of-sight (LoS)/ non-LoS (NLoS) condition in millimeter wave (mmWave) communication is important for localization and unobstructed transmission between a base station (BS) and a user. A sudden obstruction in a link between a BS and a user can result in poorly received signal strength or termination of communication. Channel features obtained by the estimation of channel state information (CSI) of a user at the BS can be used for identifying LoS/NLoS condition. With the assumption of labeled CSI, existing machine learning (ML) methods have achieved satisfactory performance for LoS/NLoS identification. However, in a real communication environment, labeled CSI is not available. In this paper, we propose a two-stage unsupervised ML based LoS/NLoS identification framework to address the lack of labeled data. We conduct experiments for the outdoor scenario by generating data from the NYUSIM simulator. We compare the performance of our method with the supervised deep neural network (SDNN) in terms of accuracy and receiver characteristic curves. The proposed framework can achieve an accuracy of 87.4% and it outperforms SDNN. Further, we compare the performance of our method with other state-of-the-art LoS/NLoS identification schemes in terms of accuracy, recall, precision, and F1-score.     

Bert-Enhanced Text Graph Neural Network for Classification

Text classification is a fundamental research direction, aims to assign tags to text units. Recently, graph neural networks (GNN) have exhibited some excellent properties in textual information processing. Furthermore, the pre-trained language model also realized promising effects in many tasks. However, many text processing methods cannot model a single text unit’s structure or ignore the semantic features. To solve these problems and comprehensively utilize the text’s structure information and semantic information, we propose a Bert-Enhanced text Graph Neural Network model (BEGNN). For each text, we construct a text graph separately according to the co-occurrence relationship of words and use GNN to extract text features. Moreover, we employ Bert to extract semantic features. The former part can take into account the structural information, and the latter can focus on modeling the semantic information. Finally, we interact and aggregate these two features of different granularity to get a more effective representation. Experiments on standard datasets demonstrate the effectiveness of BEGNN.     

Machine Learning and Micromagnetic Studies of Magnetization Switching

Magnetization switching is one of the most fundamental topics in the field of magnetism.Machine learning(ML)models of random forest(RF),support vector machine(SVM),deep neural network(DNN)methods are built and trained to classify the magnetization reversal and non-reversal cases of single-domain particle,and the classification performances are evaluated by comparison with micromagnetic simulations.The results show that the ML models have achieved great accuracy and the DNN model reaches the best area under curve(AUC)of 0.997,even with a small training dataset,and RF and SVM models have lower AUCs of 0.964 and 0.836,respectively.This work validates the potential of ML applications in studies of magnetization switching and provides the benchmark for further ML studies in magnetization switching.     

Regularization,Bayesian Inference,and Machine Learning Methods for Inverse Problems

Classical methods for inverse problems are mainly based on regularization theory, in particular those, that are based on optimization of a criterion with two parts: a data-model matching and a regularization term. Different choices for these two terms and a great number of optimization algorithms have been proposed. When these two terms are distance or divergence measures, they can have a Bayesian Maximum A Posteriori (MAP) interpretation where these two terms correspond to the likelihood and prior-probability models, respectively. The Bayesian approach gives more flexibility in choosing these terms and, in particular, the prior term via hierarchical models and hidden variables. However, the Bayesian computations can become very heavy computationally. The machine learning (ML) methods such as classification, clustering, segmentation, and regression, based on neural networks (NN) and particularly convolutional NN, deep NN, physics-informed neural networks, etc. can become helpful to obtain approximate practical solutions to inverse problems. In this tutorial article, particular examples of image denoising, image restoration, and computed-tomography (CT) image reconstruction will illustrate this cooperation between ML and inversion.     

A real signal and its states

The paper describes the general physical theory of signals, carriers of information, which supplements Shannon’s abstract classical theory and is applicable in much broader fields, including nuclear physics. It is shown that in the absence of classical noise its place should be taken by the physical threshold of signal perception for objects of both macrocosm and microcosm. The signal perception threshold allows the presence of subthreshold (virtual) signal states. For these states, Boolean algebra of logic (A = 0/1) is transformed into the “algebraic logic” of probabilities (0 ≤ a ≤ 1). The similarity and difference of virtual states of macroand microsignals are elucidated. “Real” and “quantum” information for computers is considered briefly. The maximum information transmission rate is estimated based on physical constants.     

Generic Structure Extraction with Bi-Level Optimization for Graph Structure Learning

Currently, most Graph Structure Learning (GSL) methods, as a means of learning graph structure, improve the robustness of GNN merely from a local view by considering the local information related to each edge and indiscriminately applying the mechanism across edges, which may suffer from the local structure heterogeneity of the graph (i.e., the uneven distribution of inter-class connections over nodes). To overcome the drawbacks, we extract the graph structure as a learnable parameter and jointly learn the structure and common parameters of GNN from the global view. Excitingly, the common parameters contain the global information for nodes features mapping, which is also crucial for structure optimization (i.e., optimizing the structure relies on global mapping information). Mathematically, we apply a generic structure extractor to abstract the graph structure and transform GNNs in the form of learning structure and common parameters. Then, we model the learning process as a novel bi-level optimization, i.e., Generic Structure Extraction with Bi-level Optimization for Graph Structure Learning (GSEBO), which optimizes GNN parameters in the upper level to obtain the global mapping information and graph structure is optimized in the lower level with the global information learned from the upper level. We instantiate the proposed GSEBO on classical GNNs and compare it with the state-of-the-art GSL methods. Extensive experiments validate the effectiveness of the proposed GSEBO on four real-world datasets.     

A Block-Based Adaptive Decoupling Framework for Graph Neural Networks

Graph neural networks (GNNs) with feature propagation have demonstrated their power in handling unstructured data. However, feature propagation is also a smooth process that tends to make all node representations similar as the number of propagation increases. To address this problem, we propose a novel Block-Based Adaptive Decoupling (BBAD) Framework to produce effective deep GNNs by utilizing backbone networks. In this framework, each block contains a shallow GNN with feature propagation and transformation decoupled. We also introduce layer regularizations and flexible receptive fields to automatically adjust the propagation depth and to provide different aggregation hops for each node, respectively. We prove that the traditional coupled GNNs are more likely to suffer from over-smoothing when they become deep. We also demonstrate the diversity of outputs from different blocks of our framework. In the experiments, we conduct semi-supervised and fully supervised node classifications on benchmark datasets, and the results verify that our method can not only improve the performance of various backbone networks, but also is superior to existing deep graph neural networks with less parameters.     

Multi-task learning approach for modulation and wireless signal classification for 5G and beyond: Edge deployment via model compression

Future communication networks must address the scarce spectrum to accommodate extensive growth of heterogeneous wireless devices. Efforts are underway to address spectrum coexistence, enhance spectrum awareness, and bolster authentication schemes. Wireless signal recognition is becoming increasingly more significant for spectrum monitoring, spectrum management, secure communications, among others. Consequently, comprehensive spectrum awareness on the edge has the potential to serve as a key enabler for the emerging beyond 5G (fifth generation) networks. State-of-the-art studies in this domain have (i) only focused on a single task – modulation or signal (protocol) classification – which in many cases is insufficient information for a system to act on, (ii) consider either radar or communication waveforms (homogeneous waveform category), and (iii) does not address edge deployment during neural network design phase. In this work, for the first time in the wireless communication domain, we exploit the potential of deep neural networks based multi-task learning (MTL) framework to simultaneously learn modulation and signal classification tasks while considering heterogeneous wireless signals such as radar and communication waveforms in the electromagnetic spectrum. The proposed MTL architecture benefits from the mutual relation between the two tasks in improving the classification accuracy as well as the learning efficiency with a lightweight neural network model. We additionally include experimental evaluations of the model with over-the-air collected samples and demonstrate first-hand insight on model compression along with deep learning pipeline for deployment on resource-constrained edge devices. We demonstrate significant computational, memory, and accuracy improvement of the proposed model over two reference architectures. In addition to modeling a lightweight MTL model suitable for resource-constrained embedded radio platforms, we provide a comprehensive heterogeneous wireless signals dataset for public use.     

Statistical physics of inference: thresholds and algorithms

Many questions of fundamental interest in today's science can be formulated as inference problems: some partial, or noisy, observations are performed over a set of variables and the goal is to recover, or infer, the values of the variables based on the indirect information contained in the measurements. For such problems, the central scientific questions are: Under what conditions is the information contained in the measurements sufficient for a satisfactory inference to be possible? What are the most efficient algorithms for this task? A growing body of work has shown that often we can understand and locate these fundamental barriers by thinking of them as phase transitions in the sense of statistical physics. Moreover, it turned out that we can use the gained physical insight to develop new promising algorithms. The connection between inference and statistical physics is currently witnessing an impressive renaissance and we review here the current state-of-the-art, with a pedagogical focus on the Ising model which, formulated as an inference problem, we call the planted spin glass. In terms of applications we review two classes of problems: (i) inference of clusters on graphs and networks, with community detection as a special case and (ii) estimating a signal from its noisy linear measurements, with compressed sensing as a case of sparse estimation. Our goal is to provide a pedagogical review for researchers in physics and other fields interested in this fascinating topic.     

Tomography Methods for Vector Field Study Using Space-Distributed Fiber Optic Sensors with Integral Sensitivity

The problem of tomographic reconstruction of vector physical fields is studied. This problem can be solved by using fiber optic measuring lines (MLs) of special shape. In the case that the ML output signal is proportional to the vector's projection, the ML must be shaped like a narrow loop. This problem can be solved by means of the integral theorem. If an ML output signal is proportional to projection of a vector derivative with respect to ML direction, the ML with a step shape can be used. In this case the potential component of a vector field can be reconstructed. This approach can be applied to research on distributions of electromagnetic, deforming, and other vector fields and can be used for developing systems to monitor vector physical fields.     

Multiple-symbol noncoherent learning detection of coded QAM signals in IEEE 802.15.3 Wireless Multi-media Networks

We consider the noncoherent deep learning problem for coded signal detection under the phase noncoherent channels for remote home healthcare applications with high data rate. In particular, a multiple-symbol noncoherent learning detection (MNLD) scheme based on neural networks is proposed for low-density parity-check (LDPC) coded noncoherent quadrature amplitude modulation (QAM) signals in IEEE 802.15.3 Wireless Multi-media Networks. Our derivation shows that extensive operations for the first kind zero-order modified Bessel function is unavoidable for the implementation of the optimal bit log-likelihood ratio (LLR) for decoding in traditional multiple-symbol detection (MSD) scheme. The perfect estimation of the channel state information (CSI), i.e., a priori information about the variance of the additive white Gaussian noise (AWGN), is also required for the receiver. This is clearly not computationally practical for Wireless Multi-media Networks. Consequently, we developed an improved approach based on feed-forward neural networks to accurately calculate the bit LLR. Furthermore, to decrease the generation size of training set and thus increase the training speed of the proposed neural networks, we uniformly quantize the continuous carrier phase offset (CPO), which is random and unknown, into discrete status. Our simulation results verify the learning efficiency of this simplified training-set generation configuration. The decoding convergence is successfully accelerated and much performance gain is finally achieved when compared with traditional decoding using the perfect bit LLR. This is clearly critical for high reliable transmission of home healthcare information.     

Approximate Bayesian Computation for Discrete Spaces

Many real-life processes are black-box problems, i.e., the internal workings are inaccessible or a closed-form mathematical expression of the likelihood function cannot be defined. For continuous random variables, likelihood-free inference problems can be solved via Approximate Bayesian Computation (ABC). However, an optimal alternative for discrete random variables is yet to be formulated. Here, we aim to fill this research gap. We propose an adjusted population-based MCMC ABC method by re-defining the standard ABC parameters to discrete ones and by introducing a novel Markov kernel that is inspired by differential evolution. We first assess the proposed Markov kernel on a likelihood-based inference problem, namely discovering the underlying diseases based on a QMR-DTnetwork and, subsequently, the entire method on three likelihood-free inference problems: (i) the QMR-DT network with the unknown likelihood function, (ii) the learning binary neural network, and (iii) neural architecture search. The obtained results indicate the high potential of the proposed framework and the superiority of the new Markov kernel.     

Active Inference: Applicability to Different Types of Social Organization Explained through Reference to Industrial Engineering and Quality Management

Active inference is a physics of life process theory of perception, action and learning that is applicable to natural and artificial agents. In this paper, active inference theory is related to different types of practice in social organization. Here, the term social organization is used to clarify that this paper does not encompass organization in biological systems. Rather, the paper addresses active inference in social organization that utilizes industrial engineering, quality management, and artificial intelligence alongside human intelligence. Social organization referred to in this paper can be in private companies, public institutions, other for-profit or not-for-profit organizations, and any combination of them. The relevance of active inference theory is explained in terms of variational free energy, prediction errors, generative models, and Markov blankets. Active inference theory is most relevant to the social organization of work that is highly repetitive. By contrast, there are more challenges involved in applying active inference theory for social organization of less repetitive endeavors such as one-of-a-kind projects. These challenges need to be addressed in order for active inference to provide a unifying framework for different types of social organization employing human and artificial intelligence.     

Adaptive Neuro-Fuzzy Inference System and a Multilayer Perceptron Model Trained with Grey Wolf Optimizer for Predicting Solar Diffuse Fraction

The accurate prediction of the solar diffuse fraction (DF), sometimes called the diffuse ratio, is an important topic for solar energy research. In the present study, the current state of Diffuse irradiance research is discussed and then three robust, machine learning (ML) models are examined using a large dataset (almost eight years) of hourly readings from Almeria, Spain. The ML models used herein, are a hybrid adaptive network-based fuzzy inference system (ANFIS), a single multi-layer perceptron (MLP) and a hybrid multi-layer perceptron grey wolf optimizer (MLP-GWO). These models were evaluated for their predictive precision, using various solar and DF irradiance data, from Spain. The results were then evaluated using frequently used evaluation criteria, the mean absolute error (MAE), mean error (ME) and the root mean square error (RMSE). The results showed that the MLP-GWO model, followed by the ANFIS model, provided a higher performance in both the training and the testing procedures.