A 3D densely connected convolution neural network with connection-wise attention mechanism for Alzheimer's disease classification

PurposeAlzheimer's disease (AD) is a progressive and irreversible neurodegenerative disease. In recent years, machine learning methods have been widely used on analysis of neuroimage for quantitative evaluation and computer-aided diagnosis of AD or prediction on the conversion from mild cognitive impairment (MCI) to AD. In this study, we aimed to develop a new deep learning method to detect or predict AD in an efficient way.Materials and methodsWe proposed a densely connected convolution neural network with connection-wise attention mechanism to learn the multi-level features of brain MR images for AD classification. We used the densely connected neural network to extract multi-scale features from pre-processed images, and connection-wise attention mechanism was applied to combine connections among features from different layers to hierarchically transform the MR images into more compact high-level features. Furthermore, we extended the convolution operation to 3D to capture the spatial information of MRI. The features extracted from each 3D convolution layer were integrated with features from all preceding layers with different attention, and were finally used for classification. Our method was evaluated on the baseline MRI of 968 subjects from ADNI database to discriminate (1) AD versus healthy subjects, (2) MCI converters versus healthy subjects, and (3) MCI converters versus non-converters.ResultsThe proposed method achieved 97.35% accuracy for distinguishing AD patients from healthy control, 87.82% for MCI converters against healthy control, and 78.79% for MCI converters against non-converters. Compared with some neural networks and methods reported in recent studies, the classification performance of our proposed algorithm was among the top ranks and improved in discriminating MCI subjects who were in high risks of conversion to AD.ConclusionsDeep learning techniques provide a powerful tool to explore minute but intricate characteristics in MR images which may facilitate early diagnosis and prediction of AD.     

High-performance rapid MR parameter mapping using model-based deep adversarial learning

PurposeTo develop and evaluate a deep adversarial learning-based image reconstruction approach for rapid and efficient MR parameter mapping.MethodsThe proposed method provides an image reconstruction framework by combining the end-to-end convolutional neural network (CNN) mapping, adversarial learning, and MR physical models. The CNN performs direct image-to-parameter mapping by transforming a series of undersampled images directly into MR parameter maps. Adversarial learning is used to improve image sharpness and enable better texture restoration during the image-to-parameter conversion. An additional pathway concerning the MR signal model is added between the estimated parameter maps and undersampled k-space data to ensure the data consistency during network training. The proposed framework was evaluated on T₂ mapping of the brain and the knee at an acceleration rate R = 8 and was compared with other state-of-the-art reconstruction methods. Global and regional quantitative assessments were performed to demonstrate the reconstruction performance of the proposed method.ResultsThe proposed adversarial learning approach achieved accurate T₂ mapping up to R = 8 in brain and knee joint image datasets. Compared to conventional reconstruction approaches that exploit image sparsity and low-rankness, the proposed method yielded lower errors and higher similarity to the reference and better image sharpness in the T₂ estimation. The quantitative metrics were normalized root mean square error of 3.6% for brain and 7.3% for knee, structural similarity index of 85.1% for brain and 83.2% for knee, and tenengrad measures of 9.2% for brain and 10.1% for the knee. The adversarial approach also achieved better performance for maintaining greater image texture and sharpness in comparison to the CNN approach without adversarial learning.ConclusionThe proposed framework by incorporating the efficient end-to-end CNN mapping, adversarial learning, and physical model enforced data consistency is a promising approach for rapid and efficient reconstruction of quantitative MR parameters.     

Fast prediction of aerodynamic noise induced by the flow around a cylinder based on deep neural network

Accurate and fast prediction of aerodynamic noise has always been a research hotspot in fluid mechanics and aeroacoustics. The conventional prediction methods based on numerical simulation often demand huge computational resources, which are difficult to balance between accuracy and efficiency. Here, we present a data-driven deep neural network (DNN) method to realize fast aerodynamic noise prediction while maintaining accuracy. The proposed deep learning method can predict the spatial distributions of aerodynamic noise information under different working conditions. Based on the large eddy simulation turbulence model and the Ffowcs Williams-Hawkings acoustic analogy theory, a dataset composed of 1216 samples is established. With reference to the deep learning method, a DNN framework is proposed to map the relationship between spatial coordinates, inlet velocity and overall sound pressure level. The root-mean-square-errors of prediction are below 0.82 dB in the test dataset, and the directivity of aerodynamic noise predicted by the DNN framework are basically consistent with the numerical simulation. This work paves a novel way for fast prediction of aerodynamic noise with high accuracy and has application potential in acoustic field prediction.     

基于遗传注意力机制的DLSTM电力系统混沌预测

提出一种基于遗传算法优化注意力机制的深度长短期记忆网络(DLSTM)方法,用于电力系统的混沌预测。通过传递共享参数,将遗传算法优化的注意力机制加入DLSTM模型中,可以挖掘时间序列中潜在特征,同时避免陷入局部优化。该方法是一种受进化计算方法启发的寻优方法,可以很好地学习注意力层中的参数。电力系统混沌预测实验表明所提模型比其他参考模型具有更高的预测精度和长期预测能力。     

Rapid determination of ethyl pentanoate in liquor using Fourier transform near-infrared spectroscopy coupled with chemometrics

ABSTRACT

The feasibility of Fourier transform near-infrared spectroscopy for rapid determination of ethyl pentanoate in Chinese liquor was investigated. A total of 108 liquor samples from production line were analyzed with Fourier transform near-infrared transmission spectroscopy. The calibration model for ethyl pentanoate content prediction was established with partial least square, and validated using internal cross validation. In a calibration set (80 samples), the coefficient of determination was 0.958, with the corresponding root mean square error of cross validation 0.020 g L^?1. In a validation set (28 samples), the coefficient of determination was 0.964, with the root mean square errors of prediction 0.023 g L^?1, and the bias value (0.005 g L^?1) for the validation set was less than the bias confidence limits value (0.009 g L^?1), indicating that the resulting near-infrared spectroscopy prediction model has good performance for online rapid determination of ethyl pentanoate in Chinese liquor. The determination of ethyl pentanoate in liquor can be completed in less than 2 min per sample using the near-infrared spectroscopy prediction model. The near-infrared spectroscopy combining chemometrics as a rapid analytical method has a promising application prospect of application in the liquor production field.     

集成学习结合波长选取的有机物红外光谱定量回归方法研究

研究集成学习方法在有机物红外光谱定量分析中的应用及特征波长选取方法对红外光谱集成学习建模效率和预测精度的影响。以柴油红外光谱的十六烷和总芳香烃含量为研究对象,首先采用极端随机森林(ERT)、线性核支持向量机(LinearSVM)、径向基核支持向量机(RBFSVM)和多项式核支持向量机(polySVM)作为基学习器,LinearSVM作为元学习器建立两层Stacking集成学习框架,分析比较单个基学习器与集成学习对柴油红外光谱的定量回归预测精度,与偏最小二乘(PLS)定量回归模型相比,Stacking集成学习模型对柴油光谱的两种有机物含量的预测精度均有提升,其中十六烷含量的ERT模型预测结果最优(r=0.848, RMSEP=1.603, RDP=2.627),总芳香烃含量的Stacking模型预测结果最优(r=0.991, RMSEP=0.526, RDP=9.243);进一步利用组合偏最小二乘(SiPLS)和连续投影算法(SPA)对红外光谱进行特征波长选取,利用优选出的红外光谱特征波长建立集成学习定量回归模型,其中十六烷含量的SiPLS-ERT模型预测结果最优(r=0.893, RMSEP=1.013, RDP=3.051),芳香烃含量的SiPLS-Stacking模型预测结果最优(r=0.998, RMSEP=0.354, RDP=11.475),且模型平均训练时间较全光谱训练时间减少50%以上,建模速度明显提高。研究结果表明,特征波长结合集成学习定量回归建模能够用于有机物红外光谱的定量分析中,与传统定量回归方法相比,该方法的建模效率和预测精度均有较大提高,为进一步研究机器学习在光谱定量分析中的应用提供相关方法支持。     

集成学习算法的红外光谱定量回归模型

近年来,深度学习在数据挖掘领域研究较多,深度学习中的集成学习算法也越来越多地应用到分类和定量回归中,但是,集成学习算法在红外光谱分析领域的应用研究较少。提出一种基于Blending模型融合的集成学习定量回归算法,利用GBDT算法、线性核支持向量机(LinearSVM)和径向基核支持向量机(RBF SVM)作为基学习器,将基学习器预测结果通过LinearSVM模型完成数据融合。以公开数据库中的药片和柴油近红外光谱数据为研究对象,首先对光谱数据进行一阶导数预处理,分别采用单核支持向量回归模型、GBDT模型和Blending集成学习模型,将模型预测结果进行分析比较。药片活性物含量和硬度性质采用RBF SVM模型的预测结果最优,RMSEP最小,RPD最大;其次为Blending集成学习模型;GBDT模型预测结果最差。药片质量采用Blending集成学习模型预测的R2最高,达到0.837 4;RBF SVM的RMSEP最小,为2.140 6,RPD最大,达到7.487 8;LinearSVM的预测结果最差。对于柴油沸点、闪点和总芳香烃三种性质,Blending模型预测效果最好,优于三种单模型预测结果。对于十六烷值,GBDT模型和RBF SVM模型预测结果优于Blending集成学习模型。对于密度,仅GBDT模型优于Blending集成模型,并且,使用单模型和集成模型的预测结果均较为理想,除了LinearSVM模型R2为0.944 5,其他模型R2均高于0.99。对于冰点的预测,RBF SVM和LinearSVM的预测效果优于Blending集成学习模型。对于黏性性质的预测,仅RBF SVM的预测效果优于Blending集成算法模型。由结果可以看出,由GBDT,LinearSVM和RBF SVM集成的Blending模型由于融合了单模型的特征,与单模型相比,预测效果较优或者最优,证明集成学习Blending模型用于红外光谱定量回归具有较强的适用性,且具有较高的预测精度和泛化能力,对于进一步研究集成学习算法在红外光谱定量回归中的应用具有重要的意义。     

Deep learning for radial SMS myocardial perfusion reconstruction using the 3D residual booster U-net

PurposeTo develop an end-to-end deep learning solution for quickly reconstructing radial simultaneous multi-slice (SMS) myocardial perfusion datasets with comparable quality to the pixel tracking spatiotemporal constrained reconstruction (PT-STCR) method.MethodsDynamic contrast enhanced (DCE) radial SMS myocardial perfusion data were obtained from 20 subjects who were scanned at rest and/or stress with or without ECG gating using a saturation recovery radial CAIPI turboFLASH sequence. Input to the networks consisted of complex coil combined images reconstructed using the inverse Fourier transform of undersampled radial SMS k-space data. Ground truth images were reconstructed using the PT-STCR pipeline. The performance of the residual booster 3D U-Net was tested by comparing it to state-of-the-art network architectures including MoDL, CRNN-MRI, and other U-Net variants.ResultsResults demonstrate significant improvements in speed requiring approximately 8 seconds to reconstruct one radial SMS dataset which is approximately 200 times faster than the PT-STCR method. Images reconstructed with the residual booster 3D U-Net retain quality of ground truth PT-STCR images (0.963 SSIM/40.238 PSNR/0.147 NRMSE). The residual booster 3D U-Net has superior performance compared to existing network architectures in terms of image quality, temporal dynamics, and reconstruction time.ConclusionResidual and booster learning combined with the 3D U-Net architecture was shown to be an effective network for reconstructing high-quality images from undersampled radial SMS datasets while bypassing the reconstruction time of the PT-STCR method.     

A knowledge-driven feature learning and integration method for breast cancer diagnosis on multi-sequence MRI

BackgroundThe classification of benign versus malignant breast lesions on multi-sequence Magnetic Resonance Imaging (MRI) is a challenging task since breast lesions are heterogeneous and complex. Recently, deep learning methods have been used for breast lesion diagnosis with raw image input. However, without the guidance of domain knowledge, these data-driven methods cannot ensure that the features extracted from images are comprehensive for breast cancer diagnosis. Specifically, these features are difficult to relate to clinically relevant phenomena.PurposeInspired by the cognition process of radiologists, we propose a Knowledge-driven Feature Learning and Integration (KFLI) framework, to discriminate between benign and malignant breast lesions using Multi-sequences MRI.MethodsStarting from sequence division based on characteristics, we use domain knowledge to guide the feature learning process so that the feature vectors of sub-sequence are constrained to lie in characteristic-related semantic space. Then, different deep networks are designed to extract various sub-sequence features. Furthermore, a weighting module is employed for the integration of the features extracted from different sub-sequence images adaptively.ResultsThe KFLI is a domain knowledge and deep network ensemble, which can extract sufficient and effective features from each sub-sequence for a comprehensive diagnosis of breast cancer. Experiments on 100 MRI studies have demonstrated that the KFLI achieves sensitivity, specificity, and accuracy of 84.6%, 85.7% and 85.0%, respectively, which outperforms other state-of-the-art algorithms.     

Development of fast deep learning quantification for magnetic resonance fingerprinting in vivo

PurposeA deep neural network was developed for magnetic resonance fingerprinting (MRF) quantification. This study aimed at extending previous studies of deep learning MRF to in vivo applications, allowing sub-second computation time for large-scale data.MethodsWe applied the deep learning methodology based on our previously published multi-layer perceptron. The number of layers was four, which was optimized to balance the model capacity and noise robustness. The training sets were obtained from MRF dictionaries with 9000 to 28,000 atoms, depending on the desired T1 and T2 ranges. The simulated MRF undersampling artifact based on the k-space acquisition scheme and noise were both added to the training data to reduce the error in estimates.ResultsThe neural network achieved high fidelity (R2 _ 0.98) as compared to the T1 and T2 values of the ISMRM standardized phantom. In brain MRF experiment, the model trained with simulated artifacts and noise showed less error compared to that without. The in vivo application of our neural network for liver and prostate were also demonstrated. For an MRF slice with 256 _ 256 image resolution, the computation time of our neural network was 0.12 s, compared with the _ 28 s-pre-slice for the conventional dictionary matching method.ConclusionOur neural network achieved fast computation speed for MRF quantification. The model trained with simulated artifacts and noise showed less error and achieved optimal performance for phantom experiment and in vivo normal brain and liver, and prostate cancer patient.     

Prediction carbon dioxide solubility in ionic liquids based on deep learning

Ionic liquids have a great potential in capture and separation of carbon dioxide (CO₂), and the solubility of CO₂ in ionic liquids is one of key data for engineering applications. In this paper, the critical properties of ionic liquids are combined with deep learning models (CP-DNN, CP-CNN, CP-RNN) to establish theoretical prediction models of CO₂ solubility in ionic liquids. The predictive performance of these framworks is able to meet or exceed the predicted effects of the method based on thermodynamic models (PR,SRK) and machine learning method (XGBoost). For CP-RNN, the coefficient of determination (R²) between experimental and predicted values is 0.988, CP-CNN is 0.999, and CP-DNN is 0.984. This research can avoid complex computational characterisation, it is to provide a theoretical method to further enrich and improve the data information analysis of the solubility of CO₂ in ionic liquids.     

Energy–entropy competition and the effectiveness of stochastic gradient descent in machine learning

ABSTRACT

Finding parameters that minimise a loss function is at the core of many machine learning methods. The Stochastic Gradient Descent (SGD) algorithm is widely used and delivers state-of-the-art results for many problems. Nonetheless, SGD typically cannot find the global minimum, thus its empirical effectiveness is hitherto mysterious. We derive a correspondence between parameter inference and free energy minimisation in statistical physics. The degree of undersampling plays the role of temperature. Analogous to the energy–entropy competition in statistical physics, wide but shallow minima can be optimal if the system is undersampled, as is typical in many applications. Moreover, we show that the stochasticity in the algorithm has a non-trivial correlation structure which systematically biases it towards wide minima. We illustrate our argument with two prototypical models: image classification using deep learning and a linear neural network where we can analytically reveal the relationship between entropy and out-of-sample error.     

Empirical Frequentist Coverage of Deep Learning Uncertainty Quantification Procedures

Uncertainty quantification for complex deep learning models is increasingly important as these techniques see growing use in high-stakes, real-world settings. Currently, the quality of a model’s uncertainty is evaluated using point-prediction metrics, such as the negative log-likelihood (NLL), expected calibration error (ECE) or the Brier score on held-out data. Marginal coverage of prediction intervals or sets, a well-known concept in the statistical literature, is an intuitive alternative to these metrics but has yet to be systematically studied for many popular uncertainty quantification techniques for deep learning models. With marginal coverage and the complementary notion of the width of a prediction interval, downstream users of deployed machine learning models can better understand uncertainty quantification both on a global dataset level and on a per-sample basis. In this study, we provide the first large-scale evaluation of the empirical frequentist coverage properties of well-known uncertainty quantification techniques on a suite of regression and classification tasks. We find that, in general, some methods do achieve desirable coverage properties on in distribution samples, but that coverage is not maintained on out-of-distribution data. Our results demonstrate the failings of current uncertainty quantification techniques as dataset shift increases and reinforce coverage as an important metric in developing models for real-world applications.     

基于深度学习的材料超声回波衰减预测方法

材料超声回波衰减是评价材料均匀一致性的常用方法, 针对具有复杂结构的航空发动机盘件难以进行材料底面超声回波衰减评价的问题, 本文提出了利用超声背散射波信号直接预测底面回波衰减的方法。采用10MHz聚焦探头进行超声背散射波数据的采集, 利用深度学习技术构建和训练模型,建立了基于深度学习的材料底面回波衰减预测方法, 同时讨论了采用不同信号形式的超声波信号分类识别模型的准确率差异。研究发现：基于深度学习技术可实现通过超声背散射波预测材料的底面回波衰减, 预测结果和实际底面回波衰减试验结果具有良好的一致性。     

基于XRF-CNN土壤重金属Zn元素含量预测模型研究

结合X射线荧光光谱法,针对土壤中重金属元素Zn含量的预测问题,提出基于深度卷积神经网络回归预测模型.对原始土壤进行相关预处理,用粉末压片法制作土壤压片,采用X射线荧光光谱法(X-Ray-fluorescence,XRF)获取土壤光谱,相比于传统检测方式,XRF法具有检测速度快、精度高、操作简单、不破坏样品属性并且可实现...     

Star-shaped discotic compounds with tetrazole and oxadiazole fragments

ABSTRACT

Two series of star-shaped discotic compounds (A and B) were studied to establish the relationship between their molecular structure and mesogenity. Series A included 19 three-arm compounds with known mesomorphism. Series B consisted of 132 new compounds with unknown mesomorphism: pyromellitic and cyanuric acid derivatives, 5,5′-azo-bis-isophthalic and 4,4′-azodiphthalic acids and triphenylene derivatives. The columnar mesomorphism prediction data for both series were obtained using the original program СМР ChemCard. The prediction data for series A are in good agreement with the experimental results and the reliability of the prediction was estimated to be 89.5%. The same method was applied for series B. The prediction results were approved by the synthesis of individual representatives of series B. A good correlation of the prediction with the experimental data was revealed.     

A robust electrical conductivity imaging method with total variation and wavelet regularization

PurposeThis study aims to develop and evaluate a robust conductivity imaging method that combines total variation and wavelet regularization to enhance the accuracy of conductivity maps.Theory and methodsThe proposed approach is based on a gradient-based method. The central equation is derived from Maxwell's equation and describes the relationship between conductivity and the transceive phase. A linear system equation is obtained via a finite-difference method and solved using a least-squares method. Total variation and wavelet transform regularization terms are added to the minimization problem and solved using the Split Bregman method to improve reconstruction stability. The proposed approach is compared with conventional and gradient-based methods. Numerical simulations are performed to validate the accuracy of the developed method, and the effects of noise are determined. Phantom and in vivo experiments are conducted at 3 T to verify the clinical applicability of the proposed method.ResultsNumerical simulations show that the proposed method is more robust than other methods and can suppress the effects of noise. The quantitative conductivity value of the phantom experiment agrees with the measured value. The in vivo experiment results present a clear structure, and the conductivity value of the tumor region is significantly higher than that around healthy tissues.ConclusionThe proposed electrical conductivity imaging method can improve the quality of conductivity reconstruction, and thus, has future clinical applications.     

深度学习在核物理中的应用

深度学习是目前最好的模式识别工具,预期会在核物理领域帮助科学家从大量复杂数据中寻找与某些物理最相关的特征。本文综述了深度学习技术的分类,不同数据结构对应的最优神经网络架构,黑盒模型的可解释性与预测结果的不确定性。介绍了深度学习在核物质状态方程、核结构、原子核质量、衰变与裂变方面的应用,并展示如何训练神经网络预测原子核质量。结果发现使用实验数据训练的神经网络模型对未参与训练的实验数据拥有良好的预测能力。基于已有的实验数据外推,神经网络对丰中子的轻原子核质量预测结果与宏观微观液滴模型有较大偏离。此区域可能存在未被宏观微观液滴模型包含的新物理,需要进一步的实验数据验证。     

Application of hierarchical clustering to multi-parametric MR in prostate: Differentiation of tumor and normal tissue with high accuracy

PurposeHierarchical clustering (HC), an unsupervised machine learning (ML) technique, was applied to multi-parametric MR (mp-MR) for prostate cancer (PCa). The aim of this study is to demonstrate HC can diagnose PCa in a straightforward interpretable way, in contrast to deep learning (DL) techniques.MethodsHC was constructed using mp-MR including intravoxel incoherent motion, diffusion kurtosis imaging, and dynamic contrast-enhanced MRI from 40 tumor and normal tissues in peripheral zone (PZ) and 23 tumor and normal tissues in transition zone (TZ). HC model was optimized by assessing the combinations of several dissimilarity and linkage methods. Goodness of HC model was validated by internal methods.ResultsAccuracy for differentiating tumor and normal tissue by optimal HC model was 96.3% in PZ and 97.8% in TZ, comparable to current clinical standards. Relationship between input (DWI and permeability parameters) and output (tumor and normal tissue cluster) was shown by heat maps, consistent with literature.ConclusionHC can accurately differentiate PCa and normal tissue, comparable to state-of-the-art diffusion based parameters. Contrary to DL techniques, HC is an operator-independent ML technique producing results that can be interpreted such that the results can be knowledgeably judged.     

Prediction of sgRNA Off-Target Activity in CRISPR/Cas9 Gene Editing Using Graph Convolution Network

CRISPR/Cas9 is a powerful genome-editing technology that has been widely applied in targeted gene repair and gene expression regulation. One of the main challenges for the CRISPR/Cas9 system is the occurrence of unexpected cleavage at some sites (off-targets) and predicting them is necessary due to its relevance in gene editing research. Very few deep learning models have been developed so far to predict the off-target propensity of single guide RNA (sgRNA) at specific DNA fragments by using artificial feature extract operations and machine learning techniques; however, this is a convoluted process that is difficult to understand and implement for researchers. In this research work, we introduce a novel graph-based approach to predict off-target efficacy of sgRNA in the CRISPR/Cas9 system that is easy to understand and replicate for researchers. This is achieved by creating a graph with sequences as nodes and by using a link prediction method to predict the presence of links between sgRNA and off-target inducing target DNA sequences. Features for the sequences are extracted from within the sequences. We used HEK293 and K562 t datasets in our experiments. GCN predicted the off-target gene knockouts (using link prediction) by predicting the links between sgRNA and off-target sequences with an auROC value of 0.987.