Unsupervised Anomaly Detection with Distillated Teacher-Student Network Ensemble

We address the problem of unsupervised anomaly detection for multivariate data. Traditional machine learning based anomaly detection algorithms rely on specific assumptions of normal patterns and fail to model complex feature interactions and relations. Recently, existing deep learning based methods are promising for extracting representations from complex features. These methods train an auxiliary task, e.g., reconstruction and prediction, on normal samples. They further assume that anomalies fail to perform well on the auxiliary task since they are never trained during the model optimization. However, the assumption does not always hold in practice. Deep models may also perform the auxiliary task well on anomalous samples, leading to the failure detection of anomalies. To effectively detect anomalies for multivariate data, this paper introduces a teacher-student distillation based framework Distillated Teacher-Student Network Ensemble (DTSNE). The paradigm of the teacher-student distillation is able to deal with high-dimensional complex features. In addition, an ensemble of student networks provides a better capability to avoid generalizing the auxiliary task performance on anomalous samples. To validate the effectiveness of our model, we conduct extensive experiments on real-world datasets. Experimental results show superior performance of DTSNE over competing methods. Analysis and discussion towards the behavior of our model are also provided in the experiment section.     

A Review on Machine Learning Approaches for Network Malicious Behavior Detection in Emerging Technologies

Network anomaly detection systems (NADSs) play a significant role in every network defense system as they detect and prevent malicious activities. Therefore, this paper offers an exhaustive overview of different aspects of anomaly-based network intrusion detection systems (NIDSs). Additionally, contemporary malicious activities in network systems and the important properties of intrusion detection systems are discussed as well. The present survey explains important phases of NADSs, such as pre-processing, feature extraction and malicious behavior detection and recognition. In addition, with regard to the detection and recognition phase, recent machine learning approaches including supervised, unsupervised, new deep and ensemble learning techniques have been comprehensively discussed; moreover, some details about currently available benchmark datasets for training and evaluating machine learning techniques are provided by the researchers. In the end, potential challenges together with some future directions for machine learning-based NADSs are specified.     

Portrait Segmentation Using Ensemble of Heterogeneous Deep-Learning Models

Image segmentation plays a central role in a broad range of applications, such as medical image analysis, autonomous vehicles, video surveillance and augmented reality. Portrait segmentation, which is a subset of semantic image segmentation, is widely used as a preprocessing step in multiple applications such as security systems, entertainment applications, video conferences, etc. A substantial amount of deep learning-based portrait segmentation approaches have been developed, since the performance and accuracy of semantic image segmentation have improved significantly due to the recent introduction of deep learning technology. However, these approaches are limited to a single portrait segmentation model. In this paper, we propose a novel approach using an ensemble method by combining multiple heterogeneous deep-learning based portrait segmentation models to improve the segmentation performance. The Two-Models ensemble and Three-Models ensemble, using a simple soft voting method and weighted soft voting method, were experimented. Intersection over Union (IoU) metric, IoU standard deviation and false prediction rate were used to evaluate the performance. Cost efficiency was calculated to analyze the efficiency of segmentation. The experiment results show that the proposed ensemble approach can perform with higher accuracy and lower errors than single deep-learning-based portrait segmentation models. The results also show that the ensemble of deep-learning models typically increases the use of memory and computing power, although it also shows that the ensemble of deep-learning models can perform more efficiently than a single model with higher accuracy using less memory and less computing power.     

Using deep neural networks to diagnose engine pre-ignition

Engine downsizing and boosting have been recognized as effective strategies for improving engine efficiency. However, operating the engines at high load promotes abnormal combustion events, such as pre-ignition and potential superknock. Currently the most effective method for detecting pre-ignition is by using in-cylinder pressure sensors that have high precision and sensitivity, but also high cost. Due to rapid advances in automotive technology such as autonomous driving, computer-aided designs and future connectivity, we propose to use a complimentary data-driven strategy for diagnosing abnormal combustion events. To this end, a data-driven diagnostics approach for pre-ignition detection with deep neural networks is proposed. The success of convolutional neural networks (CNNs) in object detection and recurrent neural networks (RNNs) in sequence forecasting inspired us to develop these models for pre-ignition detection. For a cost-effective strategy, we use data from less expensive sensors, such as lambda and low-resolution exhaust back pressure (EBP), instead of high resolution in-cylinder pressure measurements. The first deep learning model is combined with a commonly used dimensionality reduction tool–Principal Component Analysis (PCA). The second model eliminates this step and directly processes time-series data. Results indicate that the first model with reduced input dimensions, and correspondingly smaller size of the network, shows better performance in detecting pre-ignition cycles with an F1 score of 79%. Overall, the proposed deep learning approach is a promising alternative for abnormal combustion diagnostics using data from low resolution sensors.     

Application of Improved Asynchronous Advantage Actor Critic Reinforcement Learning Model on Anomaly Detection

Anomaly detection research was conducted traditionally using mathematical and statistical methods. This topic has been widely applied in many fields. Recently reinforcement learning has achieved exceptional successes in many areas such as the AlphaGo chess playing and video gaming etc. However, there were scarce researches applying reinforcement learning to the field of anomaly detection. This paper therefore aimed at proposing an adaptable asynchronous advantage actor-critic model of reinforcement learning to this field. The performances were evaluated and compared among classical machine learning and the generative adversarial model with variants. Basic principles of the related models were introduced firstly. Then problem definitions, modelling processes and testing were detailed. The proposed model differentiated the sequence and image from other anomalies by proposing appropriate neural networks of attention mechanism and convolutional network for the two kinds of anomalies, respectively. Finally, performances with classical models using public benchmark datasets (NSL-KDD, AWID and CICIDS-2017, DoHBrw-2020) were evaluated and compared. Experiments confirmed the effectiveness of the proposed model with the results indicating higher rewards and lower loss rates on the datasets during training and testing. The metrics of precision, recall rate and F1 score were higher than or at least comparable to the state-of-the-art models. We concluded the proposed model could outperform or at least achieve comparable results with the existing anomaly detection models.     

GTAD: Graph and Temporal Neural Network for Multivariate Time Series Anomaly Detection

The rapid development of smart factories, combined with the increasing complexity of production equipment, has resulted in a large number of multivariate time series that can be recorded using sensors during the manufacturing process. The anomalous patterns of industrial production may be hidden by these time series. Previous LSTM-based and machine-learning-based approaches have made fruitful progress in anomaly detection. However, these multivariate time series anomaly detection algorithms do not take into account the correlation and time dependence between the sequences. In this study, we proposed a new algorithm framework, namely, graph attention network and temporal convolutional network for multivariate time series anomaly detection (GTAD), to address this problem. Specifically, we first utilized temporal convolutional networks, including causal convolution and dilated convolution, to capture temporal dependencies, and then used graph neural networks to obtain correlations between sensors. Finally, we conducted sufficient experiments on three public benchmark datasets, and the results showed that the proposed method outperformed the baseline method, achieving detection results with F1 scores higher than 95% on all datasets.     

Anomaly Detection of Complex Networks Based on Intuitionistic Fuzzy Set Ensemble

Ensemble learning for anomaly detection of data structured into a complex network has been barely studied due to the inconsistent performance of complex network characteristics and the lack of inherent objective function. We propose the intuitionistic fuzzy set(IFS)-based anomaly detection, a new two-phase ensemble method for anomaly detection based on IFS, and apply it to the abnormal behavior detection problem in temporal complex networks.Firstly, it constructs the IFS of a single network characteristic, which quantifies the degree of membership,non-membership and hesitation of each network characteristic to the defined linguistic variables so that makes the unuseful or noise characteristics become part of the detection. To build an objective intuitionistic fuzzy relationship, we propose a Gaussian distribution-based membership function which gives a variable hesitation degree. Then, for the fuzzification of multiple network characteristics, the intuitionistic fuzzy weighted geometric operator is adopted to fuse multiple IFSs and to avoid the inconsistence of multiple characteristics. Finally, the score function and precision function are used to sort the fused IFS. Finally, we carry out extensive experiments on several complex network datasets for anomaly detection, and the results demonstrate the superiority of our method to state-of-the-art approaches, validating the effectiveness of our method.     

Perfect Density Models Cannot Guarantee Anomaly Detection

Thanks to the tractability of their likelihood, several deep generative models show promise for seemingly straightforward but important applications like anomaly detection, uncertainty estimation, and active learning. However, the likelihood values empirically attributed to anomalies conflict with the expectations these proposed applications suggest. In this paper, we take a closer look at the behavior of distribution densities through the lens of reparametrization and show that these quantities carry less meaningful information than previously thought, beyond estimation issues or the curse of dimensionality. We conclude that the use of these likelihoods for anomaly detection relies on strong and implicit hypotheses, and highlight the necessity of explicitly formulating these assumptions for reliable anomaly detection.     

A Novel Framework for Anomaly Detection for Satellite Momentum Wheel Based on Optimized SVM and Huffman-Multi-Scale Entropy

The health status of the momentum wheel is vital for a satellite. Recently, research on anomaly detection for satellites has become more and more extensive. Previous research mostly required simulation models for key components. However, the physical models are difficult to construct, and the simulation data does not match the telemetry data in engineering applications. To overcome the above problem, this paper proposes a new anomaly detection framework based on real telemetry data. First, the time-domain and frequency-domain features of the preprocessed telemetry signal are calculated, and the effective features are selected through evaluation. Second, a new Huffman-multi-scale entropy (HMSE) system is proposed, which can effectively improve the discrimination between different data types. Third, this paper adopts a multi-class SVM model based on the directed acyclic graph (DAG) principle and proposes an improved adaptive particle swarm optimization (APSO) method to train the SVM model. The proposed method is applied to anomaly detection for satellite momentum wheel voltage telemetry data. The recognition accuracy and detection rate of the method proposed in this paper can reach 99.60% and 99.87%. Compared with other methods, the proposed method can effectively improve the recognition accuracy and detection rate, and it can also effectively reduce the false alarm rate and the missed alarm rate.     

Financial Fraud Detection and Prediction in Listed Companies Using SMOTE and Machine Learning Algorithms

This paper proposes a new method that can identify and predict financial fraud among listed companies based on machine learning. We collected 18,060 transactions and 363 indicators of finance, including 362 financial variables and a class variable. Then, we eliminated 9 indicators which were not related to financial fraud and processed the missing values. After that, we extracted 13 indicators from 353 indicators which have a big impact on financial fraud based on multiple feature selection models and the frequency of occurrence of features in all algorithms. Then, we established five single classification models and three ensemble models for the prediction of financial fraud records of listed companies, including LR, RF, XGBOOST, SVM, and DT and ensemble models with a voting classifier. Finally, we chose the optimal single model from five machine learning algorithms and the best ensemble model among all hybrid models. In choosing the model parameter, optimal parameters were selected by using the grid search method and comparing several evaluation metrics of models. The results determined the accuracy of the optimal single model to be in a range from 97% to 99%, and that of the ensemble models as higher than 99%. This shows that the optimal ensemble model performs well and can efficiently predict and detect fraudulent activity of companies. Thus, a hybrid model which combines a logistic regression model with an XGBOOST model is the best among all models. In the future, it will not only be able to predict fraudulent behavior in company management but also reduce the burden of doing so.     

Tracking noisy limit cycle oscillation with nonlinear filters

For engineering systems, the dynamic state estimates provide valuable information for the detection and prediction of failure due to noise and vibration. From this perspective, nonlinear filtering techniques are applied to the problem of state estimation of dynamical systems undergoing noisy limit cycle oscillation. Specifically, the extended Kalman filter, ensemble Kalman filter and particle filter are used to track the limit cycle oscillations of a Duffing oscillator using noisy observational data. The noisy limit cycle oscillations feature highly non-Gaussian trends. The efficiency and limitations of the extended Kalman filter, ensemble Kalman filter and particle filter in tracking limit cycle oscillations are examined with respect to the model and measurement noise and sparsity of measurement data. For the limit cycle oscillations considered here, it is demonstrated that the ensemble Kalman filter and particle filter outperform the extended Kalman filter in the presence of sparse observational data or strong measurement noise. For moderate measurement noise and frequent measurement data, the ensemble Kalman filter and particle filter perform equally well in comparison to the extended Kalman filter.     

Scale-space point spread function based framework to boost infrared target detection algorithms

Small target detection is one of the major concern in the development of infrared surveillance systems. Detection algorithms based on Gaussian target modeling have attracted most attention from researchers in this field. However, the lack of accurate target modeling limits the performance of this type of infrared small target detection algorithms. In this paper, signal to clutter ratio (SCR) improvement mechanism based on the matched filter is described in detail and effect of Point Spread Function (PSF) on the intensity and spatial distribution of the target pixels is clarified comprehensively. In the following, a new parametric model for small infrared targets is developed based on the PSF of imaging system which can be considered as a matched filter. Based on this model, a new framework to boost model-based infrared target detection algorithms is presented. In order to show the performance of this new framework, the proposed model is adopted in Laplacian scale-space algorithms which is a well-known algorithm in the small infrared target detection field. Simulation results show that the proposed framework has better detection performance in comparison with the Gaussian one and improves the overall performance of IRST system. By analyzing the performance of the proposed algorithm based on this new framework in a quantitative manner, this new framework shows at least 20% improvement in the output SCR values in comparison with Laplacian of Gaussian (LoG) algorithm.     

A data-efficient self-supervised deep learning model for design and characterization of nanophotonic structures

With its tremendous success in many machine learning and pattern recognition tasks, deep learning, as one type of data-driven models, has also led to many breakthroughs in other disciplines including physics, chemistry and material science. Nevertheless,the supremacy of deep learning over conventional optimization approaches heavily depends on the huge amount of data collected in advance to train the model, which is a common bottleneck of such a data-driven technique. In this work, we present a comprehensive deep learning model for the design and characterization of nanophotonic structures, where a self-supervised learning mechanism is introduced to alleviate the burden of data acquisition. Taking reflective metasurfaces as an example, we demonstrate that the self-supervised deep learning model can effectively utilize randomly generated unlabeled data during training, with the total test loss and prediction accuracy improved by about 15% compared with the fully supervised counterpart.The proposed self-supervised learning scheme provides an efficient solution for deep learning models in some physics-related tasks where labeled data are limited or expensive to collect.     

基于光谱图像的森林病虫害自动检测方法

我国天然林区分布范围广,地形复杂,依靠传统的护林员巡检方式进行林木病虫害防治,效率较低,难于及时发现早期的林木病虫害,可能因此错过防治的最佳时机。针对该问题,设计了一种基于多光谱图像检测林木病虫害的深度学习网络,研发了一套检测软件,通过无人机挂飞实验,利用搭建的深度学习网络,完成林区染病区检测,对检测结果进行了分析。     

基于深度学习的故障检测方法

针对传统故障诊断方法中特征提取技术难度大、故障样本获取困难等问题,在深度学习计算框架下提出了一种半监督训练的故障检测方法,利用深度信念网络中的受限波茨曼机堆栈结构实现了数据高层特征的自动提取,结合支持向量数据描述方法实现了异常数据检测,只需利用正常工况的数据样本进行网络训练和模型拟合,无需故障样本数据,也无需人工干预进行信号特征提取,即能实现对故障数据进行的实时检测和判别。经采用标准轴承实验数据的三组故障数据进行验证,故障识别率达到100%,具有很强的工程应用价值。     

Unmanned aerial vehicles assisted rice seedling detection using shark smell optimization with deep learning model

Rice seedling classification using an unmanned aerial vehicle (UAV) images remains a challenging problem that needs to be addressed. It is still a difficult task because it is prone to low temporal and spatial resolution images. Recently, machine learning (ML) and deep learning (DL) models can be employed for several image preprocessing tasks such as classification, object detection, and segmentation. Therefore, this study focuses on the design of shark smell optimization with deep learning based rice seedling detection (SSODL-RSD) on UAV imagery. The presented SSODL-RSD technique recognizes the UAV images into arable land and rice seedlings. To achieve this, the SSODL-RSD technique employs the adaptive Wiener filtering (AWF) technique for the noise removal procedure. In addition, the SSODL-RSD technique exploits the NestNet feature extractor model. Moreover, the SSO algorithm is used for the hyperparameter tuning of the NestNet model. Finally, the long short term memory-recurrent neural network (LSTM-RNN) model is employed for the classification of rice seedlings. The extensive comparative study highlighted the improved outcomes of the SSODL-RSD technique over other existing models.     

Source Ranging Using Ensemble Convolutional Networks in the Direct Zone of Deep Water

Using deep convolutional neural networks as primary learners and a deep neural network as meta-learner, source ranging is solved as a regression problem with the ensemble learning method. Simulated acoustic data from the acoustic propagation model are used as the training data. Real data from an experiment in the South China Sea are used as the test data to demonstrate the performance. The results indicate that in the direct zone of deep water, signals received by a very deep receiver can be used to estimate the range of underwater sound source.Within 30 km, the mean absolute error of the range predictions is 1.0 km and the mean absolute percentage error is 7.9%.     

行人跟踪算法及应用综述

行人跟踪是计算机视觉领域中研究的热点和难点,通过对视频资料中行人的跟踪,可以提取出行人的运动轨迹,进而分析个体或群体的行为规律.本文首先对行人跟踪与行人检测问题之间的差别进行了阐述,其次从传统跟踪算法和基于深度学习的跟踪算法两个方面分别综述了相关算法与技术,并对经典的行人动力学模型进行了介绍,最终对行人跟踪在智能监控、拥堵人群分析、异常行为检测等场景的应用进行了系统讲解.在深度学习浪潮席卷计算机视觉领域的背景下,行人跟踪领域的研究取得了飞跃式发展,随着深度学习算法在计算机视觉领域的应用日益成熟,利用这一工具提取和量化个体和群体的行为模式,进而对大规模人群行为开展精确、实时的分析成为了该领域的发展趋势.