基于Dense-UNet++的关节滑膜磁共振图像分割

为解决以往基于深度学习的滑膜磁共振图像分割模型存在的分割精度较低、鲁棒性较差、训练耗时等问题,本文提出了一种基于Dense-UNet++网络的新模型,将DenseNet模块插入UNet++网络中,并使用Swish激活函数进行训练．利用1 036张滑膜磁共振图像数据增广后的14 512张滑膜图像对模型进行训练,并利用68张图像进行测试．结果显示,模型的平均DSC系数为0.819 9,交叉联合度量（IOU）为0.927 9．相较于UNet、ResUNet和VGG-UNet++网络结构,DSC系数和IOU均有提升,DSC振荡系数降低．另外在应用于相同滑膜图像数据集和使用相同的网络结构时,Swish函数相比ReLu函数有助于提升分割精度．实验结果表明,本文提出的算法对于滑膜磁共振图像的病灶区域的分割有较好的效果,能够辅助医生对病情做出判断．     

Deep-learning-based prediction of living cells mitosis via quantitative phase microscopy

We present a deep learning approach for living cells mitosis classification based on label-free quantitative phase imaging with transport of intensity equation methods. In the approach, we applied a pretrained deep convolutional neural network using transfer learning for binary classification of mitosis and non-mitosis. As a validation, we demonstrated the performances of the network trained by phase images and intensity images, respectively. The convolutional neural network trained by phase images achieved an average accuracy of 98.9% on the validation data, which outperforms the average accuracy 89.6% obtained by the network trained by intensity images. We believe that the quantitative phase microscopy in combination with deep learning enables researchers to predict the mitotic status of living cells noninvasively and efficiently.     

An Efficient DenseNet-Based Deep Learning Model for Malware Detection

Recently, there has been a huge rise in malware growth, which creates a significant security threat to organizations and individuals. Despite the incessant efforts of cybersecurity research to defend against malware threats, malware developers discover new ways to evade these defense techniques. Traditional static and dynamic analysis methods are ineffective in identifying new malware and pose high overhead in terms of memory and time. Typical machine learning approaches that train a classifier based on handcrafted features are also not sufficiently potent against these evasive techniques and require more efforts due to feature-engineering. Recent malware detectors indicate performance degradation due to class imbalance in malware datasets. To resolve these challenges, this work adopts a visualization-based method, where malware binaries are depicted as two-dimensional images and classified by a deep learning model. We propose an efficient malware detection system based on deep learning. The system uses a reweighted class-balanced loss function in the final classification layer of the DenseNet model to achieve significant performance improvements in classifying malware by handling imbalanced data issues. Comprehensive experiments performed on four benchmark malware datasets show that the proposed approach can detect new malware samples with higher accuracy (98.23% for the Malimg dataset, 98.46% for the BIG 2015 dataset, 98.21% for the MaleVis dataset, and 89.48% for the unseen Malicia dataset) and reduced false-positive rates when compared with conventional malware mitigation techniques while maintaining low computational time. The proposed malware detection solution is also reliable and effective against obfuscation attacks.     

DFTSA-Net: Deep Feature Transfer-Based Stacked Autoencoder Network for DME Diagnosis

Diabetic macular edema (DME) is the most common cause of irreversible vision loss in diabetes patients. Early diagnosis of DME is necessary for effective treatment of the disease. Visual detection of DME in retinal screening images by ophthalmologists is a time-consuming process. Recently, many computer-aided diagnosis systems have been developed to assist doctors by detecting DME automatically. In this paper, a new deep feature transfer-based stacked autoencoder neural network system is proposed for the automatic diagnosis of DME in fundus images. The proposed system integrates the power of pretrained convolutional neural networks as automatic feature extractors with the power of stacked autoencoders in feature selection and classification. Moreover, the system enables extracting a large set of features from a small input dataset using four standard pretrained deep networks: ResNet-50, SqueezeNet, Inception-v3, and GoogLeNet. The most informative features are then selected by a stacked autoencoder neural network. The stacked network is trained in a semi-supervised manner and is used for the classification of DME. It is found that the introduced system achieves a maximum classification accuracy of 96.8%, sensitivity of 97.5%, and specificity of 95.5%. The proposed system shows a superior performance over the original pretrained network classifiers and state-of-the-art findings.     

基于高光谱图像技术与迁移学习的水晶梨早期损伤检测

梨在储藏、包装和运输等过程中均可能发生不同程度的机械损伤,若不及时剔除损伤梨,损伤可能会逐渐严重而演变成腐烂,造成严重的经济损失。为建立一种梨早期损伤检测及损伤时间评估的快速、无损检测方法,采用高光谱图像结合迁移学习模型对损伤早期水晶梨进行识别。以无损伤、挤压损伤24 h和挤压损伤48 h的水晶梨为研究对象,应用高光谱成像系统采集样品的高光谱图像,共获取无损伤、挤压损伤24 h和挤压损伤48 h的水晶梨高光谱图像各80帧。对高光谱图像进行主成分分析,选择主成分图像4,5,6（PC4,PC5,PC6）作为检测水晶梨损伤的特征图像,将3个主成分图像拼接后进行数据扩充共得到无损伤、挤压损伤24 h和挤压损伤48 h的特征图像各160帧。按照9∶1比例划分样本训练集和测试集后,分别建立了支持向量机（SVM）、k-近邻（k-NN）和基于ResNet50网络的迁移学习损伤识别模型。SVM、k-NN和基于ResNet50网络的迁移学习模型对测试集样本总体识别准确率分别为83.33%,85.42%和93.75%,基于ResNet50网络的迁移学习模型识别效果最佳,其对测试集中无损伤、挤压损伤24 h和挤压损伤48 h的样本正确识别率分别达到100%,83%和95%。该研究结果表明,高光谱图像技术结合基于ResNet50网络的迁移学习模型可实现水晶梨早期损伤检测,并对损伤时间有较好的预测效果,且损伤时间越长,识别准确率越高。     

Electromagnetic Modulation Signal Classification Using Dual-Modal Feature Fusion CNN

AMC (automatic modulation classification) plays a vital role in spectrum monitoring and electromagnetic abnormal signal detection. Up to now, few studies have focused on the complementarity between features of different modalities and the importance of the feature fusion mechanism in the AMC method. This paper proposes a dual-modal feature fusion convolutional neural network (DMFF-CNN) for AMC to use the complementarity between different modal features fully. DMFF-CNN uses the gram angular field (GAF) image coding and intelligence quotient (IQ) data combined with CNN. Firstly, the original signal is converted into images by GAF, and the GAF images are used as the input of ResNet50. Secondly, it is converted into IQ data and as the complex value network (CV-CNN) input to extract features. Furthermore, a dual-modal feature fusion mechanism (DMFF) is proposed to fuse the dual-modal features extracted by GAF-ResNet50 and CV-CNN. The fusion feature is used as the input of DMFF-CNN for model training to achieve AMC of multi-type signals. In the evaluation stage, the advantages of the DMFF mechanism proposed in this paper and the accuracy improvement compared with other feature fusion algorithms are discussed. The experiment shows that our method performs better than others, including some state-of-the-art methods, and has superior robustness at a low signal-to-noise ratio (SNR), and the average classification accuracy of the dataset signals reaches 92.1%. The DMFF-CNN proposed in this paper provides a new path for the AMC field.     

Measuring Dynamics in Evacuation Behaviour with Deep Learning

Bounded rationality is one crucial component in human behaviours. It plays a key role in the typical collective behaviour of evacuation, in which heterogeneous information can lead to deviations from optimal choices. In this study, we propose a framework of deep learning to extract a key dynamical parameter that drives crowd evacuation behaviour in a cellular automaton (CA) model. On simulation data sets of a replica dynamic CA model, trained deep convolution neural networks (CNNs) can accurately predict dynamics from multiple frames of images. The dynamical parameter could be regarded as a factor describing the optimality of path-choosing decisions in evacuation behaviour. In addition, it should be noted that the performance of this method is robust to incomplete images, in which the information loss caused by cutting images does not hinder the feasibility of the method. Moreover, this framework provides us with a platform to quantitatively measure the optimal strategy in evacuation, and this approach can be extended to other well-designed crowd behaviour experiments.     

Co-Training for Visual Object Recognition Based on Self-Supervised Models Using a Cross-Entropy Regularization

Automatic recognition of visual objects using a deep learning approach has been successfully applied to multiple areas. However, deep learning techniques require a large amount of labeled data, which is usually expensive to obtain. An alternative is to use semi-supervised models, such as co-training, where multiple complementary views are combined using a small amount of labeled data. A simple way to associate views to visual objects is through the application of a degree of rotation or a type of filter. In this work, we propose a co-training model for visual object recognition using deep neural networks by adding layers of self-supervised neural networks as intermediate inputs to the views, where the views are diversified through the cross-entropy regularization of their outputs. Since the model merges the concepts of co-training and self-supervised learning by considering the differentiation of outputs, we called it Differential Self-Supervised Co-Training (DSSCo-Training). This paper presents some experiments using the DSSCo-Training model to well-known image datasets such as MNIST, CIFAR-100, and SVHN. The results indicate that the proposed model is competitive with the state-of-art models and shows an average relative improvement of 5% in accuracy for several datasets, despite its greater simplicity with respect to more recent approaches.     

融合数据增强与改进ResNet34的奶牛热红外图像乳腺炎检测

乳腺炎是奶牛生产养殖中最为严重的疾病之一,奶牛乳腺炎的早期检测可以为后续治疗提供依据,从而提高疾病治疗效率,降低养殖风险。为了对自然行走的奶牛实现快速、高精度的“一步式”乳腺炎疾病检测,提出了一种基于热红外图像,融合数据增强与改进ResNet34的奶牛乳腺炎疾病检测方法。相对于现有的“多步式”奶牛红外图像乳腺炎检测方法,该方法无需奶牛关键部分如乳房和眼睛的定位以及温度提取等,可有效避免“多步式”造成的误差累计,从而实现更高效的乳腺炎检测。首先,将包含奶牛关键部位的局部图片水平拼接成信息完整的整体图片,结合RandAugment数据增强方法扩增训练样本;其次,采用ResNet34残差网络作为实验的基础网络,并根据热红外图像特性对模型进行如下改进：（1）精简网络内部冗余层使得模型更轻量化;（2）中间层添加辅助分类器弥补由于模型精简带来的特征损失;（3）将改进的多融合池化层代替原有单一池化层,使得特征提取内容更丰富。随机选取3 298张热红外图像（66头奶牛）作为实验对象,并设置多组对比实验,结果表明: 与传统ResNet34相比改进后ResNet34模型分类准确率提高3.4%,基于改进ResNet34并融合迁移学习和数据增强的模型验证准确率达到90.3%,测试准确率为88.4%,分类时间仅需3.39×10-3 s。为了保证实验数据集的样本独立性,进一步将奶牛个体数量按照3∶1∶1划分为训练集、验证集和测试集,测得模型测试准确率达到80.3%,证明所提出模型具有很好的鲁棒性。根据测试结果,计算出模型查准率为91.2%、查全率为91.6%、F1分数为91.4%,与前人所做实验相比准确率提高了5.1%,特异度提升5.3%。该研究方法可以为初期奶牛乳腺疾病筛选和医学诊断提供辅助和参考。     

基于规范化样本拆分的轴承缺陷检测

表面缺陷对轴承的性能和寿命存在严重影响.近年来,深度学习在缺陷检测中发挥了重要的作用,然而对于轴承检测而言,缺陷样本的采集耗时耗力.选择轴承内径作为研究对象,根据轴承的对称性特性提出一种规范化样本拆分方法,可有效扩充轴承样本数据集.分别采用不同的样本处理方法,而后利用ResNet网络训练轴承缺陷检测模型,进行多组对比实...     

Transfer learning in deep neural network based under-sampled MR image reconstruction

In Magnetic Resonance Imaging (MRI), the success of deep learning-based under-sampled MR image reconstruction depends on: (i) size of the training dataset, (ii) generalization capabilities of the trained neural network. Whenever there is a mismatch between the training and testing data, there is a need to retrain the neural network from scratch with thousands of MR images obtained using the same protocol. This may not be possible in MRI as it is costly and time consuming to acquire data. In this research, a transfer learning approach i.e. end-to-end fine tuning is proposed for U-Net to address the data scarcity and generalization problems of deep learning-based MR image reconstruction. First the generalization capabilities of a pre-trained U-Net (initially trained on the human brain images of 1.5 T scanner) are assessed for: (a) MR images acquired from MRI scanners of different magnetic field strengths, (b) MR images of different anatomies and (c) MR images under-sampled by different acceleration factors. Later, end-to-end fine tuning of the pre-trained U-Net is proposed for the reconstruction of the above-mentioned MR images (i.e. (a), (b) and (c)). The results show successful reconstructions obtained from the proposed method as reflected by the Structural SIMilarity index, Root Mean Square Error, Peak Signal-to-Noise Ratio and central line profile of the reconstructed images.     

A New Look at the Spin Glass Problem from a Deep Learning Perspective

Spin glass is the simplest disordered system that preserves the full range of complex collective behavior of interacting frustrating elements. In the paper, we propose a novel approach for calculating the values of thermodynamic averages of the frustrated spin glass model using custom deep neural networks. The spin glass system was considered as a specific weighted graph whose spatial distribution of the edges values determines the fundamental characteristics of the system. Special neural network architectures that mimic the structure of spin lattices have been proposed, which has increased the speed of learning and the accuracy of the predictions compared to the basic solution of fully connected neural networks. At the same time, the use of trained neural networks can reduce simulation time by orders of magnitude compared to other classical methods. The validity of the results is confirmed by comparison with numerical simulation with the replica-exchange Monte Carlo method.     

An Interpretation Architecture for Deep Learning Models with the Application of COVID-19 Diagnosis

The Coronavirus disease 2019 (COVID-19) has become one of the threats to the world. Computed tomography (CT) is an informative tool for the diagnosis of COVID-19 patients. Many deep learning approaches on CT images have been proposed and brought promising performance. However, due to the high complexity and non-transparency of deep models, the explanation of the diagnosis process is challenging, making it hard to evaluate whether such approaches are reliable. In this paper, we propose a visual interpretation architecture for the explanation of the deep learning models and apply the architecture in COVID-19 diagnosis. Our architecture designs a comprehensive interpretation about the deep model from different perspectives, including the training trends, diagnostic performance, learned features, feature extractors, the hidden layers, the support regions for diagnostic decision, and etc. With the interpretation architecture, researchers can make a comparison and explanation about the classification performance, gain insight into what the deep model learned from images, and obtain the supports for diagnostic decisions. Our deep model achieves the diagnostic result of 94.75%, 93.22%, 96.69%, 97.27%, and 91.88% in the criteria of accuracy, sensitivity, specificity, positive predictive value, and negative predictive value, which are 8.30%, 4.32%, 13.33%, 10.25%, and 6.19% higher than that of the compared traditional methods. The visualized features in 2-D and 3-D spaces provide the reasons for the superiority of our deep model. Our interpretation architecture would allow researchers to understand more about how and why deep models work, and can be used as interpretation solutions for any deep learning models based on convolutional neural network. It can also help deep learning methods to take a step forward in the clinical COVID-19 diagnosis field.     

A New Deep Dual Temporal Domain Adaptation Method for Online Detection of Bearings Early Fault

With the quick development of sensor technology in recent years, online detection of early fault without system halt has received much attention in the field of bearing prognostics and health management. While lacking representative samples of the online data, one can try to adapt the previously-learned detection rule to the online detection task instead of training a new rule merely using online data. As one may come across a change of the data distribution between offline and online working conditions, it is challenging to utilize the data from different working conditions to improve detection accuracy and robustness. To solve this problem, a new online detection method of bearing early fault is proposed in this paper based on deep transfer learning. The proposed method contains an offline stage and an online stage. In the offline stage, a new state assessment method is proposed to determine the period of the normal state and the degradation state for whole-life degradation sequences. Moreover, a new deep dual temporal domain adaptation (DTDA) model is proposed. By adopting a dual adaptation strategy on the time convolutional network and domain adversarial neural network, the DTDA model can effectively extract domain-invariant temporal feature representation. In the online stage, each sequentially-arrived data batch is directly fed into the trained DTDA model to recognize whether an early fault occurs. Furthermore, a health indicator of target bearing is also built based on the DTDA features to intuitively evaluate the detection results. Experiments are conducted on the IEEE Prognostics and Health Management (PHM) Challenge 2012 bearing dataset. The results show that, compared with nine state-of-the-art fault detection and diagnosis methods, the proposed method can get an earlier detection location and lower false alarm rate.     

Deep-learning approach with convolutional neural network for classification of maximum intensity projections of dynamic contrast-enhanced breast magnetic resonance imaging

PurposeWe aimed to evaluate deep learning approach with convolutional neural networks (CNNs) to discriminate between benign and malignant lesions on maximum intensity projections of dynamic contrast-enhanced breast magnetic resonance imaging (MRI).MethodsWe retrospectively gathered maximum intensity projections of dynamic contrast-enhanced breast MRI of 106 benign (including 22 normal) and 180 malignant cases for training and validation data. CNN models were constructed to calculate the probability of malignancy using CNN architectures (DenseNet121, DenseNet169, InceptionResNetV2, InceptionV3, NasNetMobile, and Xception) with 500 epochs and analyzed that of 25 benign (including 12 normal) and 47 malignant cases for test data. Two human readers also interpreted these test data and scored the probability of malignancy for each case using Breast Imaging Reporting and Data System. Sensitivity, specificity, accuracy, and area under the receiver operating characteristic curve (AUC) were calculated.ResultsThe CNN models showed a mean AUC of 0.830 (range, 0.750–0.895). The best model was InceptionResNetV2. This model, Reader 1, and Reader 2 had sensitivities of 74.5%, 72.3%, and 78.7%; specificities of 96.0%, 88.0%, and 80.0%; and AUCs of 0.895, 0.823, and 0.849, respectively. No significant difference arose between the CNN models and human readers (p > 0.125).ConclusionOur CNN models showed comparable diagnostic performance in differentiating between benign and malignant lesions to human readers on maximum intensity projection of dynamic contrast-enhanced breast MRI.     

Deep-learning-based cryptanalysis of two types of nonlinear optical cryptosystems

The two types of nonlinear optical cryptosystems (NOCs) that are respectively based on amplitude-phase retrieval algorithm (APRA) and phase retrieval algorithm (PRA) have attracted a lot of attention due to their unique mechanism of encryption process and remarkable ability to resist common attacks. In this paper, the securities of the two types of NOCs are evaluated by using a deep-learning (DL) method, where an end-to-end densely connected convolutional network (DenseNet) model for cryptanalysis is developed. The proposed DL-based method is able to retrieve unknown plaintexts from the given ciphertexts by using the trained DenseNet model without prior knowledge of any public or private key. The results of numerical experiments with the DenseNet model clearly demonstrate the validity and good performance of the proposed the DL-based attack on NOCs.     

一种基于GAN和自适应迁移学习的样本生成方法

研究了基于生成式对抗网络(GAN)和跨域自适应迁移学习的样本生成和自动标注方法。该方法利用自适应迁移学习网络,基于已有的少量可见光图像样本集,挖掘目标在红外和可见光图像中特征内在相关性,构建自适应的转换迁移学习网络模型,生成标注好的目标图像。提出的方法解决了红外图像样本数量少且标注费时的问题,为后续多频段协同目标检测和识别获得了足够的样本数据。实验结果表明:自动标注算法对实际采集的装甲目标图像和生成的装甲目标图像各1 000张进行自动标注测试,对实际装甲目标图像的标注准确率达到95%以上,对生成的装甲目标标注准确率达到83%以上;利用真实图像和生成图像的混合数据集训练的分类器的性能和使用纯真实图像时基本一致。     

Boosting COVID-19 Image Classification Using MobileNetV3 and Aquila Optimizer Algorithm

Currently, the world is still facing a COVID-19 (coronavirus disease 2019) classified as a highly infectious disease due to its rapid spreading. The shortage of X-ray machines may lead to critical situations and delay the diagnosis results, increasing the number of deaths. Therefore, the exploitation of deep learning (DL) and optimization algorithms can be advantageous in early diagnosis and COVID-19 detection. In this paper, we propose a framework for COVID-19 images classification using hybridization of DL and swarm-based algorithms. The MobileNetV3 is used as a backbone feature extraction to learn and extract relevant image representations as a DL model. As a swarm-based algorithm, the Aquila Optimizer (Aqu) is used as a feature selector to reduce the dimensionality of the image representations and improve the classification accuracy using only the most essential selected features. To validate the proposed framework, two datasets with X-ray and CT COVID-19 images are used. The obtained results from the experiments show a good performance of the proposed framework in terms of classification accuracy and dimensionality reduction during the feature extraction and selection phases. The Aqu feature selection algorithm achieves accuracy better than other methods in terms of performance metrics.     

基于Mask R-CNN结合边缘分割的颗粒物图像检测

对颗粒物的尺寸检测是生产中重要的环节,使用相机采集图像并处理是常用的非接触检测方法。围绕颗粒物的识别与尺寸检测需求,选用沙粒为检测对象,提出了一种改进颗粒物边界掩膜的Mask R-CNN模型。该模型结合经典的边缘检测技术,并利用深度学习模型预测掩膜,根据边缘分割的结果来得到更高精度的掩膜。使用DenseNet作为检测网络的主干网络,使得整体网络参数量更少,并利用通道注意力机制加强网络的特征提取能力。实验结果表明,改进的网络可以提高检测的精度,且结合图像处理的方式能够改善掩膜尺寸检测的准确度,为颗粒物的工业检测提供了一种有意义的方法。     

基于NPU的实时深度学习跟踪算法实现北大核心CSCD

深度学习在检测领域高速发展,但受限于训练数据和计算效率,在基于嵌入式平台的边缘计算领域,尤其是实时跟踪应用中深度学习的智能化算法应用并不广泛。针对这一现象,同时为满足现阶段国产化、智能化的技术需求,提出了一种改进的孪生网络深度学习跟踪算法。在特征网络加入微调网络,解决了网络模型无法在线更新的问题,提升了跟踪的准确性;在IoUNet损失函数中加入中心距离惩罚项,解决了IoUNet当IoU相同时位置跳跃,存在收敛盲区和收敛速度慢的问题;将训练后的网络通过通道剪枝,缩减网络模型尺寸,提升了模型加载和运行的速度。在华为Atlas200NPU平台上实现了实时运行,算法准确率高达0.90(IoU>0.7),帧率达到66 Hz。