Ensemble Neuroevolution-Based Approach for Multivariate Time Series Anomaly Detection

Multivariate time series anomaly detection is a widespread problem in the field of failure prevention. Fast prevention means lower repair costs and losses. The amount of sensors in novel industry systems makes the anomaly detection process quite difficult for humans. Algorithms that automate the process of detecting anomalies are crucial in modern failure prevention systems. Therefore, many machine learning models have been designed to address this problem. Mostly, they are autoencoder-based architectures with some generative adversarial elements. This work shows a framework that incorporates neuroevolution methods to boost the anomaly detection scores of new and already known models. The presented approach adapts evolution strategies for evolving an ensemble model, in which every single model works on a subgroup of data sensors. The next goal of neuroevolution is to optimize the architecture and hyperparameters such as the window size, the number of layers, and the layer depths. The proposed framework shows that it is possible to boost most anomaly detection deep learning models in a reasonable time and a fully automated mode. We ran tests on the SWAT and WADI datasets. To the best of our knowledge, this is the first approach in which an ensemble deep learning anomaly detection model is built in a fully automatic way using a neuroevolution strategy.     

Unsupervised Anomaly Detection with Distillated Teacher-Student Network Ensemble

We address the problem of unsupervised anomaly detection for multivariate data. Traditional machine learning based anomaly detection algorithms rely on specific assumptions of normal patterns and fail to model complex feature interactions and relations. Recently, existing deep learning based methods are promising for extracting representations from complex features. These methods train an auxiliary task, e.g., reconstruction and prediction, on normal samples. They further assume that anomalies fail to perform well on the auxiliary task since they are never trained during the model optimization. However, the assumption does not always hold in practice. Deep models may also perform the auxiliary task well on anomalous samples, leading to the failure detection of anomalies. To effectively detect anomalies for multivariate data, this paper introduces a teacher-student distillation based framework Distillated Teacher-Student Network Ensemble (DTSNE). The paradigm of the teacher-student distillation is able to deal with high-dimensional complex features. In addition, an ensemble of student networks provides a better capability to avoid generalizing the auxiliary task performance on anomalous samples. To validate the effectiveness of our model, we conduct extensive experiments on real-world datasets. Experimental results show superior performance of DTSNE over competing methods. Analysis and discussion towards the behavior of our model are also provided in the experiment section.     

Perfect Density Models Cannot Guarantee Anomaly Detection

Thanks to the tractability of their likelihood, several deep generative models show promise for seemingly straightforward but important applications like anomaly detection, uncertainty estimation, and active learning. However, the likelihood values empirically attributed to anomalies conflict with the expectations these proposed applications suggest. In this paper, we take a closer look at the behavior of distribution densities through the lens of reparametrization and show that these quantities carry less meaningful information than previously thought, beyond estimation issues or the curse of dimensionality. We conclude that the use of these likelihoods for anomaly detection relies on strong and implicit hypotheses, and highlight the necessity of explicitly formulating these assumptions for reliable anomaly detection.     

一种基于无监督主动学习的苹果品质光谱无损检测模型构建方法

利用光谱技术实现农产品、食品品质无损检测的实质是建立样本光谱信息与样本品质参数之间的机器学习模型。为了获得具有良好泛化性能的机器学习模型,通常需要大量的标记样本,然而,获取样本的光谱信息相对容易,但标注样本品质参数的过程往往涉及到大量的时间和经济成本,并且具有破坏性。主动学习是一种减少训练集有标记样本数量的方法,通过选择最有价值的样本进行标记,而不是随机选择。因此,主动学习能够控制向训练集添加哪些样本,模型不再是被动地接受用于建模的样本。在分类任务中已经提出较多关于主动学习的算法,但回归任务中的研究却相对较少,且现有的用于回归任务的主动学习算法大多是有监督的,即需要少量有标记样本训练初始模型。本文提出了一种基于无监督主动学习方法的训练样本选择策略。该方法首先通过层次凝聚聚类对无标记（标准值）光谱数据集进行多样性划分,获得不同的聚类簇;然后通过局部线性重建算法在每个聚类簇中选择最具代表性的样本构成训练样本集,最后基于训练集构建模型。利用两个年份三个品种苹果的近红外光谱数据,构建了其可溶性固形物含量和硬度的偏最小二乘预测模型,用于验证所提出方法的有效性。实验结果表明：所提出的方法要优于已有的样本选择策略,可以有效地提高模型精度,减少在模型训练中的破坏性理化实验。同时,与随机采样（RS）、Kennard-Stone算法（KS）、光谱-理化值共生距离算法（SPXY）这三种光谱领域常用的样本选择算法相比,该研究所提出的方法表现出了最佳的性能, 基于所提出的无监督主动学习算法选取200个样本作为训练集所建立的可溶性固形物含量预测模型的预测均方根误差相对于其他三种算法降低了2.0%~13.2%,硬度预测模型的预测均方根误差相对降低了1.2%~15.7%。     

潜变量机器学习方法在咖啡NIR定量分析中的应用

采用近红外（NIR）光谱快检技术实现对咖啡蛋白质的定量检测,研究支持向量机（SVM）和极限学习机（ELM）等机器学习方法在建模分析中的实用性。结合潜变量分析技术,建立潜变量SVM（LV-SVM）模型和潜变量ELM（LV-ELM）模型,通过调试潜变量个数和机器学习关键参数的联合优选,实现数据降维和机器学习关键参数的同过程优化。运用定标-验证-测试机制,利用定标集样本建立咖啡蛋白质的NIR分析模型,随参数变动形成三维随动优选结构的建模预测结果,结合验证集样本对模型进行联合优选,然后将优化模型应用于测试集样本进行模型评价。LV-SVM建模优选的验证集预测均方根误差为6.797,对应的测试集预测均方根误差为8.384。LV-ELM建模优选的验证集预测均方根误差为6.118,对应的测试集预测均方根误差为7.837。与常规偏最小二乘（PLS）方法相比较,LV-SVM和LV-ELM方法均取得更好的预测结果,验证了潜变量机器学习方法在近红外定量分析中的应用优势,该方法有望应用于不同类型的咖啡各成分含量检测。     

A Novel Lightweight Real-Time Traffic Sign Detection Integration Framework Based on YOLOv4

As a popular research direction in the field of intelligent transportation, various scholars have widely concerned themselves with traffic sign detection However, there are still some key issues that need to be further solved in order to thoroughly apply related technologies to real scenarios, such as the feature extraction scheme of traffic sign images, the optimal selection of detection methods, and the objective limitations of detection tasks. For the purpose of overcoming these difficulties, this paper proposes a lightweight real-time traffic sign detection integration framework based on YOLO by combining deep learning methods. The framework optimizes the latency concern by reducing the computational overhead of the network, and facilitates information transfer and sharing at diverse levels. While improving the detection efficiency, it ensures a certain degree of generalization and robustness, and enhances the detection performance of traffic signs in objective environments, such as scale and illumination changes. The proposed model is tested and evaluated on real road scene datasets and compared with the current mainstream advanced detection models to verify its effectiveness. In addition, this paper successfully finds a reasonable balance between detection performance and deployment difficulty by effectively reducing the computational cost, which provides a possibility for realistic deployment on edge devices with limited hardware conditions, such as mobile devices and embedded devices. More importantly, the related theories have certain application potential in technology industries such as artificial intelligence or autonomous driving.     

A Soar-Based Space Exploration Algorithm for Mobile Robots

Space exploration is a hot topic in the application field of mobile robots. Proposed solutions have included the frontier exploration algorithm, heuristic algorithms, and deep reinforcement learning. However, these methods cannot solve space exploration in time in a dynamic environment. This paper models the space exploration problem of mobile robots based on the decision-making process of the cognitive architecture of Soar, and three space exploration heuristic algorithms (HAs) are further proposed based on the model to improve the exploration speed of the robot. Experiments are carried out based on the Easter environment, and the results show that HAs have improved the exploration speed of the Easter robot at least 2.04 times of the original algorithm in Easter, verifying the effectiveness of the proposed robot space exploration strategy and the corresponding HAs.     

基于基线校正和主元分析的紫外-可见光光谱在线水质异常检测方法

近年来,饮用水安全问题引起社会的广泛关注。采用紫外-可见光吸收光谱对水质进行异常检测,具有现场原位、无需试剂、分析快速等优点,适合快速在线监测。然而,紫外-可见光光谱数据量大,且易受仪器和水质正常波动的干扰,从而影响水质异常检测结果。提出一种基于基线校正和主元分析的紫外-可见光光谱法来检测污染物引起的水质异常,该方法利用非对称最小二乘校正基线,采用主元分析法从基线校正后的光谱矩阵中降维并提取特征,然后根据残差子空间的Q统计量评估测试样本的离群点,最后采用累计概率来更新异常报告结果。通过苯酚注入的实验,验证了该算法的有效性,实验结果表明,提出的方法与单波长法相比,有效地提高了污染物的检出下限;与未经基线校正采用主元分析进行的异常检测方法相比,提高了检出率,降低了误报率。     

Transfer-Learning-Based Approach for the Diagnosis of Lung Diseases from Chest X-ray Images

Using chest X-ray images is one of the least expensive and easiest ways to diagnose patients who suffer from lung diseases such as pneumonia and bronchitis. Inspired by existing work, a deep learning model is proposed to classify chest X-ray images into 14 lung-related pathological conditions. However, small datasets are not sufficient to train the deep learning model. Two methods were used to tackle this: (1) transfer learning based on two pretrained neural networks, DenseNet and ResNet, was employed; (2) data were preprocessed, including checking data leakage, handling class imbalance, and performing data augmentation, before feeding the neural network. The proposed model was evaluated according to the classification accuracy and receiver operating characteristic (ROC) curves, as well as visualized by class activation maps. DenseNet121 and ResNet50 were used in the simulations, and the results showed that the model trained by DenseNet121 had better accuracy than that trained by ResNet50.     

Experience-driven learning-based intelligent hybrid beamforming for massive MIMO mmWave communications

We have studied massive MIMO hybrid beamforming (HBF) for millimeter-wave (mmWave) communications, where the transceivers only have a few radio frequency chain (RFC) numbers compared to the number of antenna elements. We propose a hybrid beamforming design to improve the system’s spectral, hardware, and computational efficiencies, where finding the precoding and combining matrices are formulated as optimization problems with practical constraints. The series of analog phase shifters creates a unit modulus constraint, making this problem non-convex and subsequently incurring unaffordable computational complexity. Advanced deep reinforcement learning techniques effectively handle non-convex problems in many domains; therefore, we have transformed this non-convex hybrid beamforming optimization problem using a reinforcement learning framework. These frameworks are solved using advanced deep reinforcement learning techniques implemented with experience replay schemes to maximize the spectral and learning efficiencies in highly uncertain wireless environments. We developed a twin-delayed deep deterministic (TD3) policy gradient-based hybrid beamforming scheme to overcome Q-learning’s substantial overestimation. We assumed a complete channel state information (CSI) to design our beamformers and then challenged this assumption by proposing a deep reinforcement learning-based channel estimation method. We reduced hybrid beamforming complexity using soft target double deep Q-learning to exploit mmWave channel sparsity. This method allowed us to construct the analog precoder by selecting channel dominant paths. We have demonstrated that the proposed approaches improve the system’s spectral and learning efficiencies compared to prior studies. We have also demonstrated that deep reinforcement learning is a versatile technique that can unleash the power of massive MIMO hybrid beamforming in mmWave systems for next-generation wireless communication.     

Detecting Errors with Zero-Shot Learning

Error detection is a critical step in data cleaning. Most traditional error detection methods are based on rules and external information with high cost, especially when dealing with large-scaled data. Recently, with the advances of deep learning, some researchers focus their attention on learning the semantic distribution of data for error detection; however, the low error rate in real datasets makes it hard to collect negative samples for training supervised deep learning models. Most of the existing deep-learning-based error detection algorithms solve the class imbalance problem by data augmentation. Due to the inadequate sampling of negative samples, the features learned by those methods may be biased. In this paper, we propose an AEGAN (Auto-Encoder Generative Adversarial Network)-based deep learning model named SAT-GAN (Self-Attention Generative Adversarial Network) to detect errors in relational datasets. Combining the self-attention mechanism with the pre-trained language model, our model can capture semantic features of the dataset, specifically the functional dependency between attributes, so that no rules or constraints are needed for SAT-GAN to identify inconsistent data. For the lack of negative samples, we propose to train our model via zero-shot learning. As a clean-data tailored model, SAT-GAN tries to recognize error data as outliers by learning the latent features of clean data. In our evaluation, SAT-GAN achieves an average F1-score of 0.95 on five datasets, which yields at least 46.2% F1-score improvement over rule-based methods and outperforms state-of-the-art deep learning approaches in the absence of rules and negative samples.     

GTAD: Graph and Temporal Neural Network for Multivariate Time Series Anomaly Detection

The rapid development of smart factories, combined with the increasing complexity of production equipment, has resulted in a large number of multivariate time series that can be recorded using sensors during the manufacturing process. The anomalous patterns of industrial production may be hidden by these time series. Previous LSTM-based and machine-learning-based approaches have made fruitful progress in anomaly detection. However, these multivariate time series anomaly detection algorithms do not take into account the correlation and time dependence between the sequences. In this study, we proposed a new algorithm framework, namely, graph attention network and temporal convolutional network for multivariate time series anomaly detection (GTAD), to address this problem. Specifically, we first utilized temporal convolutional networks, including causal convolution and dilated convolution, to capture temporal dependencies, and then used graph neural networks to obtain correlations between sensors. Finally, we conducted sufficient experiments on three public benchmark datasets, and the results showed that the proposed method outperformed the baseline method, achieving detection results with F1 scores higher than 95% on all datasets.     

Consistent,Covariant and Multiplicative Anomalies

It is shown that the multiplicative anomaly in the vector-axial-vector model, which apparently has nothing to do with the breaking of classical current symmetries, nevertheless is strictly related to the well known consistent and covariant anomalies.     

Optimal Policy of Multiplayer Poker via Actor-Critic Reinforcement Learning

Poker has been considered a challenging problem in both artificial intelligence and game theory because poker is characterized by imperfect information and uncertainty, which are similar to many realistic problems like auctioning, pricing, cyber security, and operations. However, it is not clear that playing an equilibrium policy in multi-player games would be wise so far, and it is infeasible to theoretically validate whether a policy is optimal. Therefore, designing an effective optimal policy learning method has more realistic significance. This paper proposes an optimal policy learning method for multi-player poker games based on Actor-Critic reinforcement learning. Firstly, this paper builds the Actor network to make decisions with imperfect information and the Critic network to evaluate policies with perfect information. Secondly, this paper proposes a novel multi-player poker policy update method: asynchronous policy update algorithm (APU) and dual-network asynchronous policy update algorithm (Dual-APU) for multi-player multi-policy scenarios and multi-player sharing-policy scenarios, respectively. Finally, this paper takes the most popular six-player Texas hold ’em poker to validate the performance of the proposed optimal policy learning method. The experiments demonstrate the policies learned by the proposed methods perform well and gain steadily compared with the existing approaches. In sum, the policy learning methods of imperfect information games based on Actor-Critic reinforcement learning perform well on poker and can be transferred to other imperfect information games. Such training with perfect information and testing with imperfect information models show an effective and explainable approach to learning an approximately optimal policy.     

Algebraic classification of Weyl anomalies in arbitrary dimensions

Conformally invariant systems involving only dimensionless parameters are known to describe particle physics at very high energy. In the presence of an external gravitational field, the conformal symmetry may generalize to the Weyl invariance of classical massless field systems in interaction with gravity. In the quantum theory, the latter symmetry no longer survives: A Weyl anomaly appears. Anomalies are a cornerstone of quantum field theory, and, for the first time, a general, purely algebraic understanding of the universal structure of the Weyl anomalies is obtained, in arbitrary dimensions and independently of any regularization scheme.     

Self-Dual Strings in Six Dimensions: Anomalies,the <Emphasis Type="Italic">ADE</Emphasis>-Classification,and the World-Sheet <Emphasis Type="Italic">WZW</Emphasis>-Model

We consider the (2,0) supersymmetric theory of tensor multiplets and self-dual strings in six space-time dimensions. Space-time diffeomorphisms that leave the string world-sheet invariant appear as gauge transformations on the normal bundle of the world-sheet. The naive invariance of the model under such transformations is however explicitly broken by anomalies: The electromagnetic coupling of the string to the two-form gauge field of the tensor multiplet suffers from a classical anomaly, and there is also a one-loop quantum anomaly from the chiral fermions on the string world-sheet. Both of these contributions are proportional to the Euler class of the normal bundle of the string world-sheet, and consistency of the model requires that they cancel. This imposes strong constraints on possible models, which are found to obey an ADE-classification. We then consider the decoupled world-sheet theory that describes low-energy fluctuations (compared to the scale set by the string tension) around a configuration with a static, straight string. The anomaly structure determines this to be a supersymmetric version of the level one Wess-Zumino-Witten model based on the group      

Electromechanical Properties of Charge-Density-Wave Conductors:AC-DC Coupling Effects

We calculate perturbatively the Shapirc-Type anomalies in Young's modulus Y and internal friction δ of the CDW conductors when a combined ac and dc electric field presents within a classical model of two incommensurate, interacting, many-body eystem proposed by Sneddon. It is found that this model produces the experimental results very well at the weak pinning. Some experiments are suggested.     

宽度学习的虾新鲜度检测方法

为了提升虾新鲜度判别的准确性,提出了一种基于宽度学习(BLS)的虾新鲜度检测方法。首先采用多元散射校正（MSC）、标准正态变量校正（SNV）和直接正交信号校正（DOSC）对不同冷藏天数虾的原始高光谱进行预处理,再使用t分布随机邻域嵌入（t-SNE）将预处理之后的数据可视化,可视化结果表明DOSC聚类效果最佳。然后使用随机森林（RF）、主成分分析（PCA）和二维相关光谱分析（2D-COS）对经DOSC预处理之后的光谱数据进行特征选择。最后基于选择的特征波长对虾新鲜度进行建模分析。将宽度学习（BLS）首次用于虾新鲜度建模,同时与偏最小二乘判别（PLS-DA）和极限学习机（ELM）等经典判别模型做比较。研究结果表明RF方法最大限度地消除了光谱中的冗余信息,而BLS与线性建模方法PLS-DA以及非线性建模方法ELM相比,准确率更高并且判别时间更短,因此RF-BLS组合模型获得了最佳新鲜度判别效果,表明高光谱成像技术结合宽度学习识别虾的新鲜度是可行的,可以为在线检测虾新鲜度系统的开发提供理论依据。     

Local approach to dilatation invariance

Scale invariance is analyzed locally by coupling the energy-momentum tensor to a source which is the metric field of curved space-time. The resulting theory at the classical level has no mass parameters only if the general coordinate transformation group can be represented in Weyl's scheme. We further discuss the quantum extension of the theory; the Ward identities become anomalous under radiative corrections and the anomaly is shown to be connected with the instability of the classical metric field representation. The anomalies, recognized as the well-known trace anomalies for the energy-momentum tensor, are then reabsorbed by a perturbative alteration of the original metric field transformation law and we prove the modified Ward identities to be renormalizable in the flat limit. Finally we show that our approach is equivalent to the well-known parametric equations of the Callan-Symanzik type only if the dilatation invariance is not spontaneously broken. In the presence of spontaneous scale breaking we derive a functional equation which will be applied to cases of physical interest in a forthcoming paper.