基于混合神经网络和注意力机制的混沌时间序列预测

为提高混沌时间序列的预测精度,提出一种基于混合神经网络和注意力机制的预测模型(Att-CNNLSTM),首先对混沌时间序列进行相空间重构和数据归一化,然后利用卷积神经网络(CNN)对时间序列的重构相空间进行空间特征提取,再将CNN提取的特征和原时间序列组合,用长短期记忆网络(LSTM)根据空间特征提取时间特征,最后通过注意力机制捕获时间序列的关键时空特征,给出最终预测结果.将该模型对Logistic,Lorenz和太阳黑子混沌时间序列进行预测实验,并与未引入注意力机制的CNN-LSTM模型、单一的CNN和LSTM网络模型、以及传统的机器学习算法最小二乘支持向量机(LSSVM)的预测性能进行比较.实验结果显示本文提出的预测模型预测误差低于其他模型,预测精度更高.     

Permutation Entropy-Based Interpretability of Convolutional Neural Network Models for Interictal EEG Discrimination of Subjects with Epileptic Seizures vs. Psychogenic Non-Epileptic Seizures

The differential diagnosis of epileptic seizures (ES) and psychogenic non-epileptic seizures (PNES) may be difficult, due to the lack of distinctive clinical features. The interictal electroencephalographic (EEG) signal may also be normal in patients with ES. Innovative diagnostic tools that exploit non-linear EEG analysis and deep learning (DL) could provide important support to physicians for clinical diagnosis. In this work, 18 patients with new-onset ES (12 males, 6 females) and 18 patients with video-recorded PNES (2 males, 16 females) with normal interictal EEG at visual inspection were enrolled. None of them was taking psychotropic drugs. A convolutional neural network (CNN) scheme using DL classification was designed to classify the two categories of subjects (ES vs. PNES). The proposed architecture performs an EEG time-frequency transformation and a classification step with a CNN. The CNN was able to classify the EEG recordings of subjects with ES vs. subjects with PNES with 94.4% accuracy. CNN provided high performance in the assigned binary classification when compared to standard learning algorithms (multi-layer perceptron, support vector machine, linear discriminant analysis and quadratic discriminant analysis). In order to interpret how the CNN achieved this performance, information theoretical analysis was carried out. Specifically, the permutation entropy (PE) of the feature maps was evaluated and compared in the two classes. The achieved results, although preliminary, encourage the use of these innovative techniques to support neurologists in early diagnoses.     

A Hybrid Method Based on Extreme Learning Machine and Wavelet Transform Denoising for Stock Prediction

The trend prediction of the stock is a main challenge. Accidental factors often lead to short-term sharp fluctuations in stock markets, deviating from the original normal trend. The short-term fluctuation of stock price has high noise, which is not conducive to the prediction of stock trends. Therefore, we used discrete wavelet transform (DWT)-based denoising to denoise stock data. Denoising the stock data assisted us to eliminate the influences of short-term random events on the continuous trend of the stock. The denoised data showed more stable trend characteristics and smoothness. Extreme learning machine (ELM) is one of the effective training algorithms for fully connected single-hidden-layer feedforward neural networks (SLFNs), which possesses the advantages of fast convergence, unique results, and it does not converge to a local minimum. Therefore, this paper proposed a combination of ELM- and DWT-based denoising to predict the trend of stocks. The proposed method was used to predict the trend of 400 stocks in China. The prediction results of the proposed method are a good proof of the efficacy of DWT-based denoising for stock trends, and showed an excellent performance compared to 12 machine learning algorithms (e.g., recurrent neural network (RNN) and long short-term memory (LSTM)).     

结合高光谱成像和机器学习的棉种年份鉴别

棉花精量播种技术目前已经在新疆兵团全面推广,该技术能精确实现一穴一粒的农艺技术指标,但是也对高质量棉种的筛选提出了更高的要求。为了避免播种往年活力不足的棉种而导致发芽率降低的问题,结合机器学习和近红外（NIR）高光谱成像技术（HSI）进行棉种年份精确鉴别,实现棉种的快速无损筛选。采集2016年—2019年近四年外观无明显差异的棉种各360粒,共1 440粒棉种（按照3∶1∶1划分训练集、验证集和测试集）作为样本,按照每批60粒采集915~1 698 nm范围的棉种高光谱图像,去除首尾两端噪声大的光谱,保留1 002~1 602 nm范围的光谱为原始数据。利用Savitzky-Golay（SG）平滑算法对光谱进行预处理,采用主成分载荷方法（PCA-loading）选取13个特征波段,基于全部光谱数据和特征波段（±10 nm）数据建立逻辑回归（LR）、偏最小二乘判别分析（PLS-DA）、支持向量机（SVM）、循环神经网络（RNN）、长短记忆网络（LSTM）和卷积神经网络（CNN）六种分类模型。使用全光谱数据建模时,六种分类模型在测试集上的鉴别准确率分别为96.27%,98.98%,99.32%,96.95%,97.63%和100%,其中CNN和SVM模型取得了较好的结果;使用特征光谱数据建模时,六种分类模型在测试集上的鉴别精度分别为93.56%,97.29%,98.30%,95.25%,94.24%和99.66%,其中CNN和SVM模型仍有较好的分类结果。结果表明,使用全光谱数据建模时,六种分类模型都可以实现较高精度的棉种年份鉴别,使用特征光谱数据建模时CNN和SVM模型的鉴别精度仍可达到98%;其中深度学习方法优于传统机器学习方法,但是传统机器学习方法仍能保持较好的鉴别准确率。因此,结合近红外高光谱成像技术和机器学习方法能够实现棉种年份的高精度鉴别,为棉花精量播种过程中的优质棉种选种技术提供理论依据和方法。     

Statistics-Based Outlier Detection and Correction Method for Amazon Customer Reviews

People nowadays use the internet to project their assessments, impressions, ideas, and observations about various subjects or products on numerous social networking sites. These sites serve as a great source to gather data for data analytics, sentiment analysis, natural language processing, etc. Conventionally, the true sentiment of a customer review matches its corresponding star rating. There are exceptions when the star rating of a review is opposite to its true nature. These are labeled as the outliers in a dataset in this work. The state-of-the-art methods for anomaly detection involve manual searching, predefined rules, or traditional machine learning techniques to detect such instances. This paper conducts a sentiment analysis and outlier detection case study for Amazon customer reviews, and it proposes a statistics-based outlier detection and correction method (SODCM), which helps identify such reviews and rectify their star ratings to enhance the performance of a sentiment analysis algorithm without any data loss. This paper focuses on performing SODCM in datasets containing customer reviews of various products, which are (a) scraped from Amazon.com and (b) publicly available. The paper also studies the dataset and concludes the effect of SODCM on the performance of a sentiment analysis algorithm. The results exhibit that SODCM achieves higher accuracy and recall percentage than other state-of-the-art anomaly detection algorithms.     

Hybrid network with difference degree and attention mechanism combined with radiomics (H-DARnet) for MVI prediction in HCC

MVI is a risk assessment factor related to hepatocellular carcinoma (HCC) recurrence after hepatectomy or liver transplantation. The goal of this paper is to study the preoperative diagnosis of microvascular invasion (MVI) by using a deep learning algorithm in non-contrast T2 weighted magnetic resonance imaging (MRI) images instead of pathological images. Herein, an ensemble learning algorithm named H-DARnet—based on the difference degree and attention mechanism, combined with radiomics, for MVI prediction—is proposed. Our hybrid network combines the fine-grained, high-level semantic, and radiomics features and exhibits a rich multilevel-feature architecture composed of global-local-prior knowledge with suitable complementarity. The total loss function comprises two regularization items––the triplet and the cross-entropy loss function––which are selected for the triplet network and SE-DenseNet, respectively. The hard triplet sample selection strategy for a triplet network and data augmentation for small-scale liver image datasets in convolutional neural network (CNN) training is indispensable. For 200 patch level test samples (135 positive samples and 65 negative samples), our method can obtain the best prediction results, the AUC, sensitivity, and specificity were 0.826, 79.5% and 73.8%, respectively. The experiment results show that MVI can be predicted by using MRI images, and the proposed method is better than other deep learning algorithms and hand-crafted feature algorithms. The proposed ensemble learning algorithm is proved to be an effective method for MVI prediction.     

深度学习在超声检测缺陷识别中的应用与发展*

深度学习（Deep Learning）是目前最强大的机器学习算法之一,其中卷积神经网络（Convolutional Neural Network, CNN）模型具有自动学习特征的能力,在图像处理领域较其他深度学习模型有较大的性能优势。本文先简述了深度学习的发展史,然后综述了深度学习在超声检测缺陷识别中的应用与发展,从早期浅层神经网络到现在深度学习的应用现状,并借鉴医学影像识别和射线图像识别领域的方法,分析了卷积神经网络对超声图像缺陷识别的适用性。最后,探讨归纳了目前在超声检测图像识别中使用CNN存在的一些问题,及其主要应对策略的研究方向。     

基于深度学习的无波前探测自适应光学系统研究进展

近年来自适应光学（AO）系统向着小型化和低成本化趋势发展,无波前探测自适应光学（WFSless AO）系统由于结构简单、应用范围广,成为目前相关领域的研究热点。硬件环境确定后,系统控制算法决定了WFSless AO系统的校正效果和系统收敛速度。新兴的深度学习及人工神经网络为WFSless AO系统控制算法注入了新的活力,进一步推动了WFSless AO系统的理论发展与应用发展。在回顾前期WFSless AO系统控制算法的基础上,全面介绍了近年来卷积神经网络（CNN）、长短期记忆神经网络（LSTM）、深度强化学习在WFSless AO系统控制中的应用,并对WFSless AO系统中各种深度学习模型的特点进行了总结。概述了WFSless AO技术在天文观测、显微成像、眼底成像、激光通信等领域的应用。     

基于深度学习的太赫兹时域光谱识别研究

太赫兹时域光谱技术,由于其具有物质"指纹谱"特性,是一种可以快速无损地鉴别物质的重要手段,在毒品和爆炸物的无损检测等方面有广阔的应用前景.其中,光谱识别是太赫兹时域光谱技术应用研究的重要方向之一.现有的光谱识别方法多是依靠手工选取特征后进行机器学习分类,或是通过设置吸收峰阈值门限进行判断.由于一些物质在太赫兹波段内并没...     

基于STA-CRNN模型的语声情感识别*

语声情感识别对人机交互和情感计算研究领域具有重要作用,各类研究方法层出不穷。近期研究学者应用卷积神经网络和长短期记忆网络方法提取对数Mel谱图空间特征和时间特征,取得了一定的成果。然而不论是卷积神经网络还是长短期记忆网络提取特征时,都会产生特征冗余,导致语声情感识别效果下降。针对这一问题,该文提出了一种基于时空注意力机制的卷积-递归神经网络模型,采用对数Mel谱图和其一阶差分、二阶差分作为特征输入,在使用卷积神经网络提取空间特征和长短期记忆网络提取时间特征时,加入空间注意力和时间注意力机制,从而使上述网络能够更好地提取到对数Mel谱图中有效表征情感的空间特征和时间特征。该模型在Emo-DB和IEMOCAP语声数据集上的加权准确率分别达到86.8%、69.4%,未加权准确率分别达到84.7%、65.5%,优于当前大多数先进方法。     

基于GADF变换和多尺度CNN的哈密瓜表面农药残留可见-近红外光谱判别方法

针对哈密瓜表面农药残留化学检测方法成本高且具有破坏性等问题,探索了可见-近红外（Vis-NIR）光谱技术对农药残留定性判别的可行性。以哈密瓜为载体,百菌清和吡虫啉农药为研究对象,采集哈密瓜表面无残留、百菌清和吡虫啉残留的可见-近红外漫反射光谱,利用格拉姆角场（GAF）将一维光谱数据转换为二维彩色图像,构建GAF图像数据集。设计一种包含Inception结构的多尺度卷积神经网络模型用于哈密瓜表面农药残留种类判别,包括1层输入层、3层卷积层、1层融合层、1层平坦层、2层全连接层和1层输出层。模型测试混淆矩阵结果表明,格拉姆角差场（GADF）变换对哈密瓜表面农药残留的可见-近红外光谱表达能力较强。此外,构建AlexNet、VGG-16卷积神经网络（CNN）模型和支持向量机（SVM）、极限学习机（ELM）机器学习模型与提出的多尺度CNN模型进行性能对比。结果表明,3种CNN模型对哈密瓜表面有无农药残留的判别效果较好,综合判别准确率均高于SVM和ELM模型。对比3种CNN模型性能,多尺度CNN模型的性能最佳,训练耗时为14 s,综合判别准确率为98.33%。多尺度CNN模型结构利用多种小尺寸滤波器组合（1×1,3×3和5×5）和并行卷积模块,能够捕获不同层次和尺度的特征,通过级联融合模式进行深度特征融合,提高了模型的特征提取能力。与传统深度CNN模型相比,在保证计算复杂度不变的情况下,多尺度CNN模型的精度得到了有效提高。实验结果表明,GADF变换结合多尺度CNN模型可以有效进行光谱数据解析,利用可见-近红外光谱技术可以实现哈密瓜表面农药残留的定性判别。研究结果为大型瓜果表面农药残留的快速无损检测技术的研发提供了理论参考。     

An Overview of Infrared Spectroscopy Based on Continuous Wavelet Transform Combined with Machine Learning Algorithms: Application to Chinese Medicines,Plant Classification,and Cancer Diagnosis

Abstract

Infrared spectroscopy has been a workhorse technique for materials analysis and can result in positively identifying many different types of material. In recent years there have been reports using wavelet analysis and machine learning algorithms to extract features of Fourier transform infrared spectrometry (FTIR). The machine learning algorithms contain back-propagation neural network (BPNN), radial basis function neural network (RBFNN), and support vector machine (SVM). This article reviews the important advances in FTIR analysis employing a continuous wavelet transform (CWT) and machine learning algorithms, especially in the applications of the method for Chinese medicine identification, plant classification, and cancer diagnosis.     

One- and Two-Phase Software Requirement Classification Using Ensemble Deep Learning

Recently, deep learning (DL) has been utilized successfully in different fields, achieving remarkable results. Thus, there is a noticeable focus on DL approaches to automate software engineering (SE) tasks such as maintenance, requirement extraction, and classification. An advanced utilization of DL is the ensemble approach, which aims to reduce error rates and learning time and improve performance. In this research, three ensemble approaches were applied: accuracy as a weight ensemble, mean ensemble, and accuracy per class as a weight ensemble with a combination of four different DL models—long short-term memory (LSTM), bidirectional long short-term memory (BiLSTM), a gated recurrent unit (GRU), and a convolutional neural network (CNN)—in order to classify the software requirement (SR) specification, the binary classification of SRs into functional requirement (FRs) or non-functional requirements (NFRs), and the multi-label classification of both FRs and NFRs into further experimental classes. The models were trained and tested on the PROMISE dataset. A one-phase classification system was developed to classify SRs directly into one of the 17 multi-classes of FRs and NFRs. In addition, a two-phase classification system was developed to classify SRs first into FRs or NFRs and to pass the output to the second phase of multi-class classification to 17 classes. The experimental results demonstrated that the proposed classification systems can lead to a competitive classification performance compared to the state-of-the-art methods. The two-phase classification system proved its robustness against the one-phase classification system, as it obtained a 95.7% accuracy in the binary classification phase and a 93.4% accuracy in the second phase of NFR and FR multi-class classification.     

Acoustic detection and classification of Microchiroptera using machine learning: lessons learned from automatic speech recognition

Current automatic acoustic detection and classification of microchiroptera utilize global features of individual calls (i.e., duration, bandwidth, frequency extrema), an approach that stems from expert knowledge of call sonograms. This approach parallels the acoustic phonetic paradigm of human automatic speech recognition (ASR), which relied on expert knowledge to account for variations in canonical linguistic units. ASR research eventually shifted from acoustic phonetics to machine learning, primarily because of the superior ability of machine learning to account for signal variation. To compare machine learning with conventional methods of detection and classification, nearly 3000 search-phase calls were hand labeled from recordings of five species: Pipistrellus bodenheimeri, Molossus molossus, Lasiurus borealis, L. cinereus semotus, and Tadarida brasiliensis. The hand labels were used to train two machine learning models: a Gaussian mixture model (GMM) for detection and classification and a hidden Markov model (HMM) for classification. The GMM detector produced 4% error compared to 32% error for a baseline broadband energy detector, while the GMM and HMM classifiers produced errors of 0.6 +/- 0.2% compared to 16.9 +/- 1.1% error for a baseline discriminant function analysis classifier. The experiments showed that machine learning algorithms produced errors an order of magnitude smaller than those for conventional methods.     

A convolutional neural network approach to calibrating the rotation axis for X‐ray computed tomography

This paper presents an algorithm to calibrate the center‐of‐rotation for X‐ray tomography by using a machine learning approach, the Convolutional Neural Network (CNN). The algorithm shows excellent accuracy from the evaluation of synthetic data with various noise ratios. It is further validated with experimental data of four different shale samples measured at the Advanced Photon Source and at the Swiss Light Source. The results are as good as those determined by visual inspection and show better robustness than conventional methods. CNN has also great potential for reducing or removing other artifacts caused by instrument instability, detector non‐linearity, etc. An open‐source toolbox, which integrates the CNN methods described in this paper, is freely available through GitHub at tomography/xlearn and can be easily integrated into existing computational pipelines available at various synchrotron facilities. Source code, documentation and information on how to contribute are also provided.     

基于多算法优化SVM的粗糙面参数反演

支持向量机（SVM）是粗糙面参数反演中常用的一种反演算法,SVM反演中的惩罚参数C和核函数参数G对反演结果精度的影响较大,若参数取值不当,会使模型产生"过学习"或者"欠学习"的现象,从而降低预测精度.给出几种SVM参数C和参数G的优化算法,如K折交叉验证（K-CV）、遗传算法（GA）和粒子群算法（PSO）,并在此基础上提出一种基于K-CV和GA改进的PSO算法（GA-CV-PSO）.利用矩量法（MoM）获得的粗糙面后向散射系数构造训练集和测试集,通过不同参数反演的仿真结果对比不同优化算法的反演精度和计算时间,表明GA-CV-PSO算法克服了单一优化算法的缺陷,具有更精确的反演精度和更强的泛化能力.     

基于双向长短期记忆网络的太赫兹光谱识别

特征提取是太赫兹光谱识别的关键处理步骤,通常利用降维方法作为特征提取手段。然而,当一些化合物的太赫兹光谱曲线整体差异度较小时,降维方法往往会缺失样本差异的重要特征信息,从而导致分类错误。如果不采用降维方法提取特征,传统机器学习分类算法对维数较高的原始太赫兹光谱数据又不能很好的分类。针对此问题,提出了一种基于双向长短期记忆网络（BLSTM-RNN）自动提取太赫兹光谱特征的识别方法。BLSTM-RNN作为一种特殊的循环神经网络,利用其LSTM单元可以有效解决原始太赫兹光谱数据维数较高使得模型难以训练问题。再结合模型的双向频谱信息利用架构模式,可以增强模型对复杂光谱数据自动提取有效特征信息的能力。采用三类、15种化合物太赫兹透射光谱作为测试对象,首先利用S-G滤波和三次样条插值对Anthraquinone,Benomyl和Carbazole等十五种化合物在0.9～6 THz内的太赫兹透射光谱数据进行归一化处理,然后通过构建一个具有双向长短期记忆的循环神经网络对太赫兹光谱的全频谱信息进行自动特征提取并利用Softmax分类器进行分类。通过试验优化网络结构和各项参数,最终获得了针对复杂太赫兹透射光谱数据的预测模型,并与传统机器学习算法SVM,KNN及神经网络算法MLP,CNN进行对比实验。结果表明,dataset-1和dataset-2分别作为差异度较大和无明显峰值特征的五种化合物太赫兹透射光谱数据集,其平均识别率分别为100%和98.51%,与其他方法相比识别率有所提高;最重要的是,dataset-3作为5种化合物谱线极为相似的太赫兹透射光谱数据集,其平均识别率为96.56%,与其他方法相比识别率提高显著;dataset-4作为dataset-1,dataset-2和dataset-3的透射光谱数据集集合,其平均识别率为98.87%。从而验证了BLSTM-RNN模型能自动提取有效的太赫兹光谱特征,同时又能保证复杂太赫兹光谱的预测精度。在选择模型训练优化算法方面,使用Adam优化算法要好于RMSProp,SGD和AdaGrad,其模型的目标函数损失值收敛速度最快。同时随着模型训练迭代次数增加,相似太赫兹透射光谱数据集的预测准确率也不断提升。可为复杂太赫兹光谱数据库的光谱识别检索提供一种新的识别方法。     

基于卷积神经网络与投票机制的蒲黄炮制品近红外判别方法

蒲黄炭是由香蒲花粉炮制而成,具有止血、化瘀、通淋等多种功效,被广泛应用于临床抗血栓,创面和出血。然而蒲黄炭在炒炭过程中,常常会出现炭化过轻或者炭化过重的现象,从而出现不同炭化程度的蒲黄炭药品,主要为轻度炭化、标准炭化与重度炭化三种不同的蒲黄炭药品。由于炭化程度不同,蒲黄炭的凝血效果优劣不等,其中标准炭化的蒲黄炭药品药效最优。目前,鉴别蒲黄炭药品的方法多为人工凭借肉眼与经验进行判别。基于人工的蒲黄炭药品判别方法判别效率低,受主观因素影响大,判别结果不稳定,难以区分出标准炭化的蒲黄炭。为有效地对不同炭化程度的蒲黄炭进行识别,提出一种基于卷积神经网络与投票机制的蒲黄炮制品近红外判别方法。该方法创新性地结合深度学习与机器学习算法,有效利用卷积神经网络强大表征提取能力的同时通过投票决策提升算法模型的泛化能力与鲁棒性。首先通过近红外光谱技术获取蒲黄炭的近红外光谱,并通过卷积神经网络分别提取样本经过四种预处理方法所得到光谱图的高阶特征,并计算预测结果。按照样本准确率与损失值为四种预处理方法分配相应权重得到蒲黄炮制品预测模型。该模型将所得到的四种预测结果结合权重共同投票出样本的最终结果,从而鉴别出蒲黄炭的炭化程度。实验结果表明所提方法可以有效判别蒲黄炮制品的炭化程度。当训练集所占样本比例为80%时,预测准确率达到95.4%。所提方法与传统卷积神经网络方法、线性判别分析方法以及标准正太变量变换-线性判别分析方法相比预测准确率分别提高8.6%,4.3%和2.6%。同时,所提方法具有一定的稳定性,当训练集所占样本比例大于70%时,测试准确率高于90%;当训练集比例仅占10%时,预测准确性仍然能够达到约80%。     

URLLC occasional large time delay prediction based on unbalanced regression and LSTM

Occasional large time delay is one of the bottlenecks for 5G applied to the industrial field. Accurate measurement, analysis, and prediction of communication network latency are of great significance. Due to the strong randomness and small number of samples in ultra-reliable and low latency communication (URLLC), the prediction task of occasional large time delay is very challenging. This paper proposes an URLLC occasional large time delay prediction method based on unbalanced regression algorithms and long–short term memory (LSTM). We first decompose the original sequence by using the variational mode decomposition (VMD) to obtain relatively stable component sequences. The parameters of the VMD are automatically optimized by grasshopper optimization algorithm (GOA). In order to improve the prediction effect of URLLC occasional large time delay, we introduce unbalanced regression algorithms. Before the VMD decomposition of data samples, the SMOGN is introduced to balance the number of large-time delay samples and common-time delay samples. Next we calculate the weight of each sample based on the rarity of each sample point and our proposed LDSWeight. Finally, LSTM is used to complete cost-sensitive learning. The experimental results show that the URLLC occasional large time delay prediction method proposed in this paper has better prediction accuracy than other methods.     

Improving Document-Level Sentiment Classification Using Importance of Sentences

Previous researchers have considered sentiment analysis as a document classification task, in which input documents are classified into predefined sentiment classes. Although there are sentences in a document that support important evidences for sentiment analysis and sentences that do not, they have treated the document as a bag of sentences. In other words, they have not considered the importance of each sentence in the document. To effectively determine polarity of a document, each sentence in the document should be dealt with different degrees of importance. To address this problem, we propose a document-level sentence classification model based on deep neural networks, in which the importance degrees of sentences in documents are automatically determined through gate mechanisms. To verify our new sentiment analysis model, we conducted experiments using the sentiment datasets in the four different domains such as movie reviews, hotel reviews, restaurant reviews, and music reviews. In the experiments, the proposed model outperformed previous state-of-the-art models that do not consider importance differences of sentences in a document. The experimental results show that the importance of sentences should be considered in a document-level sentiment classification task.