Deep-learning-based prediction of living cells mitosis via quantitative phase microscopy

We present a deep learning approach for living cells mitosis classification based on label-free quantitative phase imaging with transport of intensity equation methods. In the approach, we applied a pretrained deep convolutional neural network using transfer learning for binary classification of mitosis and non-mitosis. As a validation, we demonstrated the performances of the network trained by phase images and intensity images, respectively. The convolutional neural network trained by phase images achieved an average accuracy of 98.9% on the validation data, which outperforms the average accuracy 89.6% obtained by the network trained by intensity images. We believe that the quantitative phase microscopy in combination with deep learning enables researchers to predict the mitotic status of living cells noninvasively and efficiently.     

Diabetic Retinal Grading Using Attention-Based Bilinear Convolutional Neural Network and Complement Cross Entropy

Diabetic retinopathy (DR) is a common complication of diabetes mellitus (DM), and it is necessary to diagnose DR in the early stages of treatment. With the rapid development of convolutional neural networks in the field of image processing, deep learning methods have achieved great success in the field of medical image processing. Various medical lesion detection systems have been proposed to detect fundus lesions. At present, in the image classification process of diabetic retinopathy, the fine-grained properties of the diseased image are ignored and most of the retinopathy image data sets have serious uneven distribution problems, which limits the ability of the network to predict the classification of lesions to a large extent. We propose a new non-homologous bilinear pooling convolutional neural network model and combine it with the attention mechanism to further improve the network’s ability to extract specific features of the image. The experimental results show that, compared with the most popular fundus image classification models, the network model we proposed can greatly improve the prediction accuracy of the network while maintaining computational efficiency.     

基于深度递归级联卷积神经网络的并行磁共振成像方法

快速磁共振成像是磁共振研究领域重要的课题之一.随着大数据和深度学习的兴起,神经网络成为快速磁共振技术的重要方法.然而网络性能表现和网络参数量之间较难取得平衡,且对于多通道数据重建的并行成像问题,相关研究较少.本文构建了一种深度递归级联卷积神经网络结构,用于处理并行成像问题.这种网络结构在减少网络参数量的同时,能够尽可能地提高网络的表达能力,提高网络重建的精确度.实验结果表明,相较于传统并行成像方法,通过训练好的神经网络对欠采样磁共振数据进行重建,可以得到更准确的重建结果,且重建时间大大缩短.     

深度卷积网络的多品种多厂商药品近红外光谱分类

近红外光谱（NIR）分析具有分析高效、样品无损、环境无污染以及可现场检测等优点,特别适合药品的快速建模分析。但NIR存在吸收强度弱以及谱带重叠等缺点,需要建立稳健可靠的化学计量学模型对其进行分析。深度卷积神经网络是深度学习方法中一个重要分支,它通过逐层抽取数据特征并进行组合、转换,形成更高层的语义特征,具有极强的建模能力,广泛应用于计算机视觉、语音识别等领域,而在药品NIR分析方面尚未见报道。基于深度卷积网络模型,对药品NIR多分类建模进行研究。针对药品NIR数据的特点,设计若干个面向多品种、多厂商药品NIR分类的一维深度卷积网络模型。模型中卷积层和池化层交叠排列用于逐层抽取NIR数据特征,输出层连接softmax分类器,对药品NIR数据进行分类概率预测。在输出层之前采用全局最大池化层,将特征图进行整体池化,形成一个特征点,用于解决全连接层存在的限制输入维度大小,参数过多的问题。同时,在网络模型中引入批处理操作和dropout机制,以防止梯度消失和减小网络过拟合的风险。在网络模型的设计过程中,通过设计不同的卷积网络层数以及不同的卷积核尺寸大小,分析其对建模效果的影响,同时分析五种经典数据预处理方法对NIR分析的影响。以我国7个厂商生产的头孢克肟片和11个厂商生产的苯妥英钠片样本NIR为实验对象, 建立药品的多品种、多厂商分类模型,该模型在二分类、多分类实验中取得了良好的分类效果。在十八分类实验中,当训练集与测试集比例为7∶3时,分类准确率为99.37±0.45,比SVM, BP, AE和ELM算法取得更优的分类性能。同时,深度卷积神经网络模型推理速度较快,优于SVM和ELM算法,但训练速度慢于二者。大量实验结果表明,深度卷积神经网络可对多品种、多厂商药品NIR数据准确、可靠地判别分类,且模型具有良好的鲁棒性和可扩展性。该方法也可推广到烟草、石化等其他领域的NIR数据分类应用中。     

基于一维卷积神经网络的化爆和地震次声分类*

次声事件的分类识别方法应用广泛,传统分类方法在很多方面进行了尝试,但由于次声信号具备非线性的特点,致使分类难度较大,分类精度不高,这对次声事件的分类工作提出了挑战。针对次声事件中的化学爆炸与天然地震信号分类问题,文章构建了一种改进的深度卷积神经网络分类模型用于实现两类次声信号的分类。论文采用"全面禁止核试验条约组织"官网上收集到的化学爆炸和天然地震次声信号进行数据集的构建,使用改进的深度卷积神经网络分别与BP网络和一维LeNet-5网络进行对比分析。实验结果表明,论文的测试识别率能够达到82.72%,较上述算法有优势。     

Transfer-Learning-Based Approach for the Diagnosis of Lung Diseases from Chest X-ray Images

Using chest X-ray images is one of the least expensive and easiest ways to diagnose patients who suffer from lung diseases such as pneumonia and bronchitis. Inspired by existing work, a deep learning model is proposed to classify chest X-ray images into 14 lung-related pathological conditions. However, small datasets are not sufficient to train the deep learning model. Two methods were used to tackle this: (1) transfer learning based on two pretrained neural networks, DenseNet and ResNet, was employed; (2) data were preprocessed, including checking data leakage, handling class imbalance, and performing data augmentation, before feeding the neural network. The proposed model was evaluated according to the classification accuracy and receiver operating characteristic (ROC) curves, as well as visualized by class activation maps. DenseNet121 and ResNet50 were used in the simulations, and the results showed that the model trained by DenseNet121 had better accuracy than that trained by ResNet50.     

基于深度架构网络的矮新星自动分类研究

矮新星是一类特殊而稀少的半相接双星。发现更多的矮新星对于深入研究物质转移理论、理解密近双星演化过程意义深远。利用深度学习技术提取天体光谱特征并进而分类是天文数据处理领域的研究热点。传统的自编码器是仅包含一个隐层的经典神经网络模型,编码能力有限,数据表征学习能力不足。模块化拓宽神经网络的深度能够驱使网络继承地学习到天体光谱的特征,通过对底层特征的逐渐抽象学习获得高层特征,进而提高光谱的分类准确率。以自编码器为基础构建了由输入层、若干隐藏层和输出层组成的基于多层感知器架构的深度前馈堆栈式自编码器网络,用于处理海量的光谱数据集,挖掘隐藏在光谱内部具有区分度的深度结构特征,实现对矮新星光谱的准确分类。鉴于深度架构网络的参数设置会严重影响所构建网络的性能,将网络参数的优化分为逐层训练和反向传播两个过程。预处理后的光谱数据先由输入层进入网络,再经自编码器算法和权值共享实现对网络参数的逐层训练。反向传播阶段将初始样本数据再次输入网络,以逐层训练所得的权值对网络初始化,再把网络各层的局部优化训练结果融合起来,根据所设置的输出误差代价函数调整网络参数。反复地逐层训练和反向传播,直到获得全局最优的网络参数。最后由末隐层作为重构层搭建支持向量机分类器,实现对矮新星的特征提取与分类。网络参数优化过程中利用均值网络思想使网络隐层单元输出按照dropout系数衰减,并由反向传播算法微调整个网络,从而防止发生深度过拟合现象,减少因隐层神经元间的相互节制而学习到重复的数据表征,提高网络的泛化能力。该网络分布式的多层次架构能够提供有效的数据抽象和表征学习能力,其特征检测层可从无标注数据中隐式地学习到深度结构特征,有效刻画光谱数据的非线性和随机波动性,避免了光谱特征的显式提取,体现出较强的数据拟合和泛化能力。不同层之间的权值共享能够减少冗余信息的干扰,有效化解传统多层次架构网络易陷入权值局部最小化的风险。实验表明,该深度架构网络在矮新星分类任务中能达到95.81%的准确率,超过了经典的LM-BP网络。     

深度学习在核物理中的应用

深度学习是目前最好的模式识别工具,预期会在核物理领域帮助科学家从大量复杂数据中寻找与某些物理最相关的特征。本文综述了深度学习技术的分类,不同数据结构对应的最优神经网络架构,黑盒模型的可解释性与预测结果的不确定性。介绍了深度学习在核物质状态方程、核结构、原子核质量、衰变与裂变方面的应用,并展示如何训练神经网络预测原子核质量。结果发现使用实验数据训练的神经网络模型对未参与训练的实验数据拥有良好的预测能力。基于已有的实验数据外推,神经网络对丰中子的轻原子核质量预测结果与宏观微观液滴模型有较大偏离。此区域可能存在未被宏观微观液滴模型包含的新物理,需要进一步的实验数据验证。     

An Improved Residual Network for Pork Freshness Detection Using Near-Infrared Spectroscopy

Effective and rapid assessment of pork freshness is significant for monitoring pork quality. However, a traditional sensory evaluation method is subjective and physicochemical analysis is time-consuming. In this study, the near-infrared spectroscopy (NIRS) technique, a fast and non-destructive analysis method, is employed to determine pork freshness. Considering that commonly used statistical modeling methods require preprocessing data for satisfactory performance, this paper presents a one-dimensional squeeze-and-excitation residual network (1D-SE-ResNet) to construct the complex relationship between pork freshness and NIRS. The developed model enhances the one-dimensional residual network (1D-ResNet) with squeeze-and-excitation (SE) blocks. As a deep learning model, the proposed method is capable of extracting features from the input spectra automatically and can be used as an end-to-end model to simplify the modeling process. A comparison between the proposed method and five popular classification models indicates that the 1D-SE-ResNet achieves the best performance, with a classification accuracy of 93.72%. The research demonstrates that the NIRS analysis technique based on deep learning provides a promising tool for pork freshness detection and therefore is helpful for ensuring food safety.     

基于深度学习的气溶胶荧光光谱识别应用研究

空气中的高危病原微生物对人类社会存在着极大威胁,而传统的监测方法无法对空气中的微生物实现准确的识别与分类。因此采用激光诱导荧光技术原理,以单光子探测器为核心器件,设计并搭建了一种高效的荧光光谱仪用于空气中高危病原微生物的识别与分类,并且该光谱仪可以实现对微生物浓度的预测,其对于环境安全具有重要意义。对于该光谱仪采集的数据,探索了以一维向量和二维矩阵2种输入形式来实现荧光光谱的识别与分类,并研究对比了主成分分析网络、卷积神经网络和全卷积网络等深度学习网络的识别与分类效果。实验结果表明以矩阵形式输入的卷积神经网络模型在测试集中识别分类准确率达到98.05%。采用矩阵形式输入的全卷积网络模型在测试集中微生物浓度预测准确率达到98.97%。     

Deep neural network inspired by iterative shrinkage-thresholding algorithm with data consistency (NISTAD) for fast Undersampled MRI reconstruction

With the aim of developing a fast algorithm for high-quality MRI reconstruction from undersampled k-space data, we propose a novel deep neural Network, which is inspired by Iterative Shrinkage Thresholding Algorithm with Data consistency (NISTAD). NISTAD consists of three consecutive blocks: an encoding block, which models the flow graph of ISTA, a classical iteration-based compressed sensing magnetic resonance imaging (CS-MRI) method; a decoding block, which recovers the image from sparse representation; a data consistency block, which adaptively enforces consistency with the acquired raw data according to learned noise level. The ISTA method is thereby mapped to an end-to-end deep neural network, which greatly reduces the reconstruction time and simplifies the tuning of hyper-parameters, compared to conventional model-based CS-MRI methods. On the other hand, compared to general deep learning-based MRI reconstruction methods, the proposed method uses a simpler network architecture with fewer parameters. NISTAD has been validated on retrospectively undersampled diencephalon standard challenge data using different acceleration factors, and compared with DAGAN and Cascade CNN, two state-of-the-art deep neural network-based methods which outperformed many other state-of-the-art model-based and deep learning-based methods. Experimental results demonstrated that NISTAD reconstruction achieved comparable image quality with DAGAN and Cascade CNN reconstruction in terms of both PSNR and SSIM metrics, and subjective assessment, though with a simpler network structure.     

Multi-task learning approach for modulation and wireless signal classification for 5G and beyond: Edge deployment via model compression

Future communication networks must address the scarce spectrum to accommodate extensive growth of heterogeneous wireless devices. Efforts are underway to address spectrum coexistence, enhance spectrum awareness, and bolster authentication schemes. Wireless signal recognition is becoming increasingly more significant for spectrum monitoring, spectrum management, secure communications, among others. Consequently, comprehensive spectrum awareness on the edge has the potential to serve as a key enabler for the emerging beyond 5G (fifth generation) networks. State-of-the-art studies in this domain have (i) only focused on a single task – modulation or signal (protocol) classification – which in many cases is insufficient information for a system to act on, (ii) consider either radar or communication waveforms (homogeneous waveform category), and (iii) does not address edge deployment during neural network design phase. In this work, for the first time in the wireless communication domain, we exploit the potential of deep neural networks based multi-task learning (MTL) framework to simultaneously learn modulation and signal classification tasks while considering heterogeneous wireless signals such as radar and communication waveforms in the electromagnetic spectrum. The proposed MTL architecture benefits from the mutual relation between the two tasks in improving the classification accuracy as well as the learning efficiency with a lightweight neural network model. We additionally include experimental evaluations of the model with over-the-air collected samples and demonstrate first-hand insight on model compression along with deep learning pipeline for deployment on resource-constrained edge devices. We demonstrate significant computational, memory, and accuracy improvement of the proposed model over two reference architectures. In addition to modeling a lightweight MTL model suitable for resource-constrained embedded radio platforms, we provide a comprehensive heterogeneous wireless signals dataset for public use.     

稀疏降噪自编码算法用于近红外光谱鉴别真假药的研究

近红外光谱分析技术作为一种快速、无损检测技术十分适用于真假药品现场鉴别。自编码网络作为当前机器学习领域研究的热点受到广泛关注,自编码网络是一种典型的深度学习网络模型,它比传统的潜层学习方法具有更强的模型表示能力。自编码网络使用贪婪逐层预训练算法,通过最小化各层网络的重构误差,依次训练网络的每一层,进而训练整个网络。通过对数据进行白化预处理并使用无监督算法对输入数据进行逐层重构,使网络更有效的学习到数据的内部结构特征。之后使用带标签数据通过监督学习算法对整个网络进行调优。首先对真假琥乙红霉素片的近红外光谱数据进行预处理以及白化预处理,通过白化处理降低数据特征之间的相关性,使数据各特征具有相同的方差。数据处理之后利用稀疏降噪自编码网络针对真假药品光谱数据建立分类模型,并将稀疏降噪自编码网络模型与BP神经网络以及SVM算法在分类准确率及算法稳定性方面进行对比。结果表明对光谱数据进行白化预处理能有效提升稀疏降噪自编码网络的分类准确率。并且自编码网络分类准确率在不同训练样本数量下均高于BP神经网络,SVM算法在少量训练样本的情况下更有优势,但在训练数据集样本数达到一定数量后,自编码网络的分类准确率将优于SVM算法。在算法稳定性方面,自编码网络较之BP神经网络和SVM算法也更稳定。使用稀疏降噪自编码网络对真假药品近红外光谱数据进行建模,能对真假药品进行有效的鉴别。     

Unmanned aerial vehicles assisted rice seedling detection using shark smell optimization with deep learning model

Rice seedling classification using an unmanned aerial vehicle (UAV) images remains a challenging problem that needs to be addressed. It is still a difficult task because it is prone to low temporal and spatial resolution images. Recently, machine learning (ML) and deep learning (DL) models can be employed for several image preprocessing tasks such as classification, object detection, and segmentation. Therefore, this study focuses on the design of shark smell optimization with deep learning based rice seedling detection (SSODL-RSD) on UAV imagery. The presented SSODL-RSD technique recognizes the UAV images into arable land and rice seedlings. To achieve this, the SSODL-RSD technique employs the adaptive Wiener filtering (AWF) technique for the noise removal procedure. In addition, the SSODL-RSD technique exploits the NestNet feature extractor model. Moreover, the SSO algorithm is used for the hyperparameter tuning of the NestNet model. Finally, the long short term memory-recurrent neural network (LSTM-RNN) model is employed for the classification of rice seedlings. The extensive comparative study highlighted the improved outcomes of the SSODL-RSD technique over other existing models.     

基于子频带能量特征提取的汽车鸣笛声识别*

针对城市中汽车违法鸣笛声之间识别分类较难的问题，为了快速准确的识别鸣笛声并将不同种鸣笛声之间进行分类，在鸣笛声识别分类中提出了应用子频带能量提取鸣笛声的特征，并利用BP神经网络对提取的子频带能量特征值矩阵进行学习训练，且在神经网络学习过程中利用可变学习速度的方法，减小了神经网络的迭代次数。实验表明利用此种子频带能量特征提取法使鸣笛声与非鸣笛声的平均识别率达到了94.889%；使不同鸣笛声之间的分类正确率最大达到了93.75%，实现了不同鸣笛声之间的分类。     

基于深度聚类的目标细粒度分类方法北大核心CSCD

为提高光电系统对弱小目标的识别和分类能力,降低算法对硬件平台和数据的依赖,提出一种无监督分类方法−基于目标深度特征聚类的细粒度分类方法。该方法通过轮廓、颜色、对比度等浅层特征提取提示目标,经超分辨处理后,利用卷积神经网络对目标的深层特征进行编码,进一步采用基于注意机制的主成分分析方法进行降维生成表征矩阵,最后利用聚类的方式实现目标细粒度分类。实验验证了基于不同神经网络的深度聚类方法在不同数据集上的分类性能,其中采用ResNet-34聚类方法在CIFAR-10测试集上细粒度分类性能达92.71%,结果表明,基于深度聚类的目标细粒度方法能够取得与强监督学习方法相当的目标分类效果。此外,还可以根据不同簇数和聚类等级的选择实现不同细粒度的分类效果。     

Evaluating the Learning Procedure of CNNs through a Sequence of Prognostic Tests Utilising Information Theoretical Measures

Deep learning has proven to be an important element of modern data processing technology, which has found its application in many areas such as multimodal sensor data processing and understanding, data generation and anomaly detection. While the use of deep learning is booming in many real-world tasks, the internal processes of how it draws results is still uncertain. Understanding the data processing pathways within a deep neural network is important for transparency and better resource utilisation. In this paper, a method utilising information theoretic measures is used to reveal the typical learning patterns of convolutional neural networks, which are commonly used for image processing tasks. For this purpose, training samples, true labels and estimated labels are considered to be random variables. The mutual information and conditional entropy between these variables are then studied using information theoretical measures. This paper shows that more convolutional layers in the network improve its learning and unnecessarily higher numbers of convolutional layers do not improve the learning any further. The number of convolutional layers that need to be added to a neural network to gain the desired learning level can be determined with the help of theoretic information quantities including entropy, inequality and mutual information among the inputs to the network. The kernel size of convolutional layers only affects the learning speed of the network. This study also shows that where the dropout layer is applied to has no significant effects on the learning of networks with a lower dropout rate, and it is better placed immediately after the last convolutional layer with higher dropout rates.     

稀疏降噪自编码结合高斯过程的近红外光谱药品鉴别方法

提出一种稀疏降噪自编码结合高斯过程的近红外光谱药品鉴别方法。首先对近红外光谱数据进行小波变换以消除基线漂移,然后用稀疏降噪自编码(SDAE)网络提取光谱特征并降维表示,最后采用高斯过程(GP)进行二分类,其中GP选用光谱混合(SM)核函数作为协方差函数,记此分类网络为wSDAG_SM。自编码网络具有很强的模型表示能力,高斯过程分类器在处理小样本数据时具有优势。wSDAG_SM网络通过稀疏降噪自编码学习得到维数更低但更有价值的特征来表示输入数据,同时将具有很好表达力的光谱混合核作为高斯过程的协方差函数,有利于更准确的光谱数据分类。以琥乙红霉素及其他药品的近红外光谱为实验数据,将该方法与经过墨西哥帽小波变换的BP神经网络(wBP)、支持向量机(wSVM), SDAE结合Logistic二分类(wSDAL)、SDAE结合采用平方指数(SE)协方差核的GP二分类(wSDAG_SE),以及未采用小波变换的SDAG_SM网络等方法进行对比。实验结果表明,对光谱数据进行墨西哥帽小波变换预处理能有效提升SDAG_SM网络的分类准确率和稳定性。wSDAG_SM方法无论从分类准确率还是分类结果稳定性方面,都优于其他分类器。     

A 3D densely connected convolution neural network with connection-wise attention mechanism for Alzheimer's disease classification

PurposeAlzheimer's disease (AD) is a progressive and irreversible neurodegenerative disease. In recent years, machine learning methods have been widely used on analysis of neuroimage for quantitative evaluation and computer-aided diagnosis of AD or prediction on the conversion from mild cognitive impairment (MCI) to AD. In this study, we aimed to develop a new deep learning method to detect or predict AD in an efficient way.Materials and methodsWe proposed a densely connected convolution neural network with connection-wise attention mechanism to learn the multi-level features of brain MR images for AD classification. We used the densely connected neural network to extract multi-scale features from pre-processed images, and connection-wise attention mechanism was applied to combine connections among features from different layers to hierarchically transform the MR images into more compact high-level features. Furthermore, we extended the convolution operation to 3D to capture the spatial information of MRI. The features extracted from each 3D convolution layer were integrated with features from all preceding layers with different attention, and were finally used for classification. Our method was evaluated on the baseline MRI of 968 subjects from ADNI database to discriminate (1) AD versus healthy subjects, (2) MCI converters versus healthy subjects, and (3) MCI converters versus non-converters.ResultsThe proposed method achieved 97.35% accuracy for distinguishing AD patients from healthy control, 87.82% for MCI converters against healthy control, and 78.79% for MCI converters against non-converters. Compared with some neural networks and methods reported in recent studies, the classification performance of our proposed algorithm was among the top ranks and improved in discriminating MCI subjects who were in high risks of conversion to AD.ConclusionsDeep learning techniques provide a powerful tool to explore minute but intricate characteristics in MR images which may facilitate early diagnosis and prediction of AD.