High-performance rapid MR parameter mapping using model-based deep adversarial learning

PurposeTo develop and evaluate a deep adversarial learning-based image reconstruction approach for rapid and efficient MR parameter mapping.MethodsThe proposed method provides an image reconstruction framework by combining the end-to-end convolutional neural network (CNN) mapping, adversarial learning, and MR physical models. The CNN performs direct image-to-parameter mapping by transforming a series of undersampled images directly into MR parameter maps. Adversarial learning is used to improve image sharpness and enable better texture restoration during the image-to-parameter conversion. An additional pathway concerning the MR signal model is added between the estimated parameter maps and undersampled k-space data to ensure the data consistency during network training. The proposed framework was evaluated on T₂ mapping of the brain and the knee at an acceleration rate R = 8 and was compared with other state-of-the-art reconstruction methods. Global and regional quantitative assessments were performed to demonstrate the reconstruction performance of the proposed method.ResultsThe proposed adversarial learning approach achieved accurate T₂ mapping up to R = 8 in brain and knee joint image datasets. Compared to conventional reconstruction approaches that exploit image sparsity and low-rankness, the proposed method yielded lower errors and higher similarity to the reference and better image sharpness in the T₂ estimation. The quantitative metrics were normalized root mean square error of 3.6% for brain and 7.3% for knee, structural similarity index of 85.1% for brain and 83.2% for knee, and tenengrad measures of 9.2% for brain and 10.1% for the knee. The adversarial approach also achieved better performance for maintaining greater image texture and sharpness in comparison to the CNN approach without adversarial learning.ConclusionThe proposed framework by incorporating the efficient end-to-end CNN mapping, adversarial learning, and physical model enforced data consistency is a promising approach for rapid and efficient reconstruction of quantitative MR parameters.     

Exploiting the wavelet structure in compressed sensing MRI

Sparsity has been widely utilized in magnetic resonance imaging (MRI) to reduce k-space sampling. According to structured sparsity theories, fewer measurements are required for tree sparse data than the data only with standard sparsity. Intuitively, more accurate image reconstruction can be achieved with the same number of measurements by exploiting the wavelet tree structure in MRI. A novel algorithm is proposed in this article to reconstruct MR images from undersampled k-space data. In contrast to conventional compressed sensing MRI (CS-MRI) that only relies on the sparsity of MR images in wavelet or gradient domain, we exploit the wavelet tree structure to improve CS-MRI. This tree-based CS-MRI problem is decomposed into three simpler subproblems then each of the subproblems can be efficiently solved by an iterative scheme. Simulations and in vivo experiments demonstrate the significant improvement of the proposed method compared to conventional CS-MRI algorithms, and the feasibleness on MR data compared to existing tree-based imaging algorithms.     

基于深度递归级联卷积神经网络的并行磁共振成像方法

快速磁共振成像是磁共振研究领域重要的课题之一.随着大数据和深度学习的兴起,神经网络成为快速磁共振技术的重要方法.然而网络性能表现和网络参数量之间较难取得平衡,且对于多通道数据重建的并行成像问题,相关研究较少.本文构建了一种深度递归级联卷积神经网络结构,用于处理并行成像问题.这种网络结构在减少网络参数量的同时,能够尽可能地提高网络的表达能力,提高网络重建的精确度.实验结果表明,相较于传统并行成像方法,通过训练好的神经网络对欠采样磁共振数据进行重建,可以得到更准确的重建结果,且重建时间大大缩短.     

Transfer learning in deep neural network based under-sampled MR image reconstruction

In Magnetic Resonance Imaging (MRI), the success of deep learning-based under-sampled MR image reconstruction depends on: (i) size of the training dataset, (ii) generalization capabilities of the trained neural network. Whenever there is a mismatch between the training and testing data, there is a need to retrain the neural network from scratch with thousands of MR images obtained using the same protocol. This may not be possible in MRI as it is costly and time consuming to acquire data. In this research, a transfer learning approach i.e. end-to-end fine tuning is proposed for U-Net to address the data scarcity and generalization problems of deep learning-based MR image reconstruction. First the generalization capabilities of a pre-trained U-Net (initially trained on the human brain images of 1.5 T scanner) are assessed for: (a) MR images acquired from MRI scanners of different magnetic field strengths, (b) MR images of different anatomies and (c) MR images under-sampled by different acceleration factors. Later, end-to-end fine tuning of the pre-trained U-Net is proposed for the reconstruction of the above-mentioned MR images (i.e. (a), (b) and (c)). The results show successful reconstructions obtained from the proposed method as reflected by the Structural SIMilarity index, Root Mean Square Error, Peak Signal-to-Noise Ratio and central line profile of the reconstructed images.     

DeepcomplexMRI: Exploiting deep residual network for fast parallel MR imaging with complex convolution

This paper proposes a multi-channel image reconstruction method, named DeepcomplexMRI, to accelerate parallel MR imaging with residual complex convolutional neural network. Different from most existing works which rely on the utilization of the coil sensitivities or prior information of predefined transforms, DeepcomplexMRI takes advantage of the availability of a large number of existing multi-channel groudtruth images and uses them as target data to train the deep residual convolutional neural network offline. In particular, a complex convolutional network is proposed to take into account the correlation between the real and imaginary parts of MR images. In addition, the k-space data consistency is further enforced repeatedly in between layers of the network. The evaluations on in vivo datasets show that the proposed method has the capability to recover the desired multi-channel images. Its comparison with state-of-the-art methods also demonstrates that the proposed method can reconstruct the desired MR images more accurately.     

Incorporating prior knowledge via volumetric deep residual network to optimize the reconstruction of sparsely sampled MRI

For sparse sampling that accelerates magnetic resonance (MR) image acquisition, non-linear reconstruction algorithms have been developed, which incorporated patient specific a prior information. More generic a prior information could be acquired via deep learning and utilized for image reconstruction. In this study, we developed a volumetric hierarchical deep residual convolutional neural network, referred to as T-Net, to provide a data-driven end-to-end mapping from sparsely sampled MR images to fully sampled MR images, where cartilage MR images were acquired using an Ultra-short TE sequence and retrospectively undersampled using pseudo-random Cartesian and radial acquisition schemes. The network had a hierarchical architecture that promoted the sparsity of feature maps and increased the receptive field, which were valuable for signal synthesis and artifact suppression. Relatively dense local connections and global shortcuts were established to facilitate residual learning and compensate for details lost in hierarchical processing. Additionally, volumetric processing was adopted to fully exploit spatial continuity in three-dimensional space. Data consistency was further enforced. The network was trained with 336 three-dimensional images (each consisting of 32 slices) and tested by 24 images. The incorporation of a priori information acquired via deep learning facilitated high acceleration factors (as high as 8) while maintaining high image fidelity (quantitatively evaluated using the structural similarity index measurement). The proposed T-Net had an improved performance as compared to several state-of-the-art networks.     

IKWI-net: A cross-domain convolutional neural network for undersampled magnetic resonance image reconstruction

Magnetic resonance imaging (MRI) is widely used to get the information of anatomical structure and physiological function with the advantages of high resolution and non-invasive scanning. But the long acquisition time limits its application. To reduce the time consumption of MRI, compressed sensing (CS) theory has been proposed to reconstruct MRI images from undersampled k-space data. But conventional CS methods mostly use iterative methods that take lots of time. Recently, deep learning methods are proposed to achieve faster reconstruction, but most of them only pay attention to a single domain, such as the image domain or k-space. To take advantage of the feature representation in different domains, we propose a cross-domain method based on deep learning, which first uses convolutional neural networks (CNNs) in the image domain, k-space and wavelet domain simultaneously. The combined order of the three domains is also first studied in this work, which has a significant effect on reconstruction. The proposed IKWI-net achieves the best performance in various combinations, which utilizes CNNs in the image domain, k-space, wavelet domain and image domain sequentially. Compared with several deep learning methods, experiments show it also achieves mean improvements of 0.91 dB in peak signal-to-noise ratio (PSNR) and 0.005 in structural similarity (SSIM).     

Triple-D network for efficient undersampled magnetic resonance images reconstruction

Compressed sensing (CS) theory can help accelerate magnetic resonance imaging (MRI) by sampling partial k-space measurements. However, conventional optimization-based CS-MRI methods are often time-consuming and are based on fixed transform or shallow image dictionaries, which limits modeling capabilities. Recently, deep learning models have been used to solve the CS-MRI problem. However, recent researches have focused on modeling in image domain, and the potential of k-space modeling capability has not been utilized seriously. In this paper, we propose a deep model called Dual Domain Dense network (Triple-D network), which consisted of some k-space and image domain sub-network. These sub-networks are connected with dense connections, which can utilize feature maps at different levels to enhance performance. To further promote model capabilities, we use two strategies: multi-supervision strategies, which can avoid loss of supervision information; channel-wise attention layer (CA layer), which can adaptively adjust the weight of the feature map. Experimental results show that the proposed Triple-D network provides promising performance in CS-MRI, and it can effectively work on different sampling trajectories and noisy settings.     

A multi-scale residual network for accelerated radial MR parameter mapping

A deep learning MR parameter mapping framework which combines accelerated radial data acquisition with a multi-scale residual network (MS-ResNet) for image reconstruction is proposed. The proposed supervised learning strategy uses input image patches from multi-contrast images with radial undersampling artifacts and target image patches from artifact-free multi-contrast images. Subspace filtering is used during pre-processing to denoise input patches. For each anatomy and relaxation parameter, an individual network is trained. in vivo T₁ mapping results are obtained on brain and abdomen datasets and in vivo T₂ mapping results are obtained on brain and knee datasets. Quantitative results for the T₂ mapping of the knee show that MS-ResNet trained using either fully sampled or undersampled data outperforms conventional model-based compressed sensing methods. This is significant because obtaining fully sampled training data is not possible in many applications. in vivo brain and abdomen results for T₁ mapping and in vivo brain results for T₂ mapping demonstrate that MS-ResNet yields contrast-weighted images and parameter maps that are comparable to those achieved by model-based iterative methods while offering two orders of magnitude reduction in reconstruction times. The proposed approach enables recovery of high-quality contrast-weighted images and parameter maps from highly accelerated radial data acquisitions. The rapid image reconstructions enabled by the proposed approach makes it a good candidate for routine clinical use.     

Deep learning for radial SMS myocardial perfusion reconstruction using the 3D residual booster U-net

PurposeTo develop an end-to-end deep learning solution for quickly reconstructing radial simultaneous multi-slice (SMS) myocardial perfusion datasets with comparable quality to the pixel tracking spatiotemporal constrained reconstruction (PT-STCR) method.MethodsDynamic contrast enhanced (DCE) radial SMS myocardial perfusion data were obtained from 20 subjects who were scanned at rest and/or stress with or without ECG gating using a saturation recovery radial CAIPI turboFLASH sequence. Input to the networks consisted of complex coil combined images reconstructed using the inverse Fourier transform of undersampled radial SMS k-space data. Ground truth images were reconstructed using the PT-STCR pipeline. The performance of the residual booster 3D U-Net was tested by comparing it to state-of-the-art network architectures including MoDL, CRNN-MRI, and other U-Net variants.ResultsResults demonstrate significant improvements in speed requiring approximately 8 seconds to reconstruct one radial SMS dataset which is approximately 200 times faster than the PT-STCR method. Images reconstructed with the residual booster 3D U-Net retain quality of ground truth PT-STCR images (0.963 SSIM/40.238 PSNR/0.147 NRMSE). The residual booster 3D U-Net has superior performance compared to existing network architectures in terms of image quality, temporal dynamics, and reconstruction time.ConclusionResidual and booster learning combined with the 3D U-Net architecture was shown to be an effective network for reconstructing high-quality images from undersampled radial SMS datasets while bypassing the reconstruction time of the PT-STCR method.     

Smoothly Clipped Absolute Deviation (SCAD) regularization for compressed sensing MRI Using an augmented Lagrangian scheme

PurposeCompressed sensing (CS) provides a promising framework for MR image reconstruction from highly undersampled data, thus reducing data acquisition time. In this context, sparsity-promoting regularization techniques exploit the prior knowledge that MR images are sparse or compressible in a given transform domain. In this work, a new regularization technique was introduced by iterative linearization of the non-convex smoothly clipped absolute deviation (SCAD) norm with the aim of reducing the sampling rate even lower than it is required by the conventional l₁ norm while approaching an l₀ norm.Materials and MethodsThe CS-MR image reconstruction was formulated as an equality-constrained optimization problem using a variable splitting technique and solved using an augmented Lagrangian (AL) method developed to accelerate the optimization of constrained problems. The performance of the resulting SCAD-based algorithm was evaluated for discrete gradients and wavelet sparsifying transforms and compared with its l₁-based counterpart using phantom and clinical studies. The k-spaces of the datasets were retrospectively undersampled using different sampling trajectories. In the AL framework, the CS-MRI problem was decomposed into two simpler sub-problems, wherein the linearization of the SCAD norm resulted in an adaptively weighted soft thresholding rule with a sparsity enhancing effect.ResultsIt was demonstrated that the proposed regularization technique adaptively assigns lower weights on the thresholding of gradient fields and wavelet coefficients, and as such, is more efficient in reducing aliasing artifacts arising from k-space undersampling, when compared to its l₁-based counterpart.ConclusionThe SCAD regularization improves the performance of l₁-based regularization technique, especially at reduced sampling rates, and thus might be a good candidate for some applications in CS-MRI.     

Deep learning for image reconstruction in thermoacoustic tomography

Microwave-induced thermoacoustic tomography(TAT)is a rapidly-developing noninvasive imaging technique that integrates the advantages of microwave imaging and ultrasound imaging.While an image reconstruction algorithm is critical for the TAT,current reconstruction methods often creates significant artifacts and are computationally costly.In this work,we propose a deep learning-based end-to-end image reconstruction method to achieve the direct reconstruction from the sinogram data to the initial pressure density image.We design a new network architecture TAT-Net to transfer the sinogram domain to the image domain with high accuracy.For the scenarios where realistic training data are scarce or unavailable,we use the finite element method(FEM)to generate synthetic data where the domain gap between the synthetic and realistic data is resolved through the signal processing method.The TAT-Net trained with synthetic data is evaluated through both simulations and phantom experiments and achieves competitive performance in artifact removal and robustness.Compared with other state-of-the-art reconstruction methods,the TAT-Net method can reduce the root mean square error to 0.0143,and increase the structure similarity and peak signal-to-noise ratio to 0.988 and 38.64,respectively.The results obtained indicate that the TAT-Net has great potential applications in improving image reconstruction quality and fast quantitative reconstruction.     

基于支撑先验与深度图像先验的无预训练磁共振图像重建方法

基于深度学习的磁共振成像(magnetic resonance imaging, MRI)方法需要大规模、高质量的病患数据样本集进行预训练.然而,由于病患隐私及设备等因素限制,获取大规模、高质量的磁共振数据集在实际临床应用中面临挑战.本文提出一种新的基于深度学习的欠采样磁共振图像重建方法,该方法无需预训练、不依赖训练数据集,而是充分利用待重建的目标MR图像的结构先验和支撑先验,并将其引入深度图像先验(deep image prior, DIP)框架,从而削减对训练数据集的依赖,提升学习效率.基于参考图像与目标图像的相似性,采用高分辨率参考图像作为深度网络输入,将结构先验信息引入网络;将参考图像在小波域中幅值大的系数索引集作为目标图像的已知支撑集,构造正则化约束项,将网络训练转化为网络参数的最优化求解过程.实验结果表明,本文方法可由欠采样k空间数据重建得到更精确的磁共振图像,且在保留组织特征、细节纹理方面具有明显优势.     

Compressed Sensing MRI Using Sparsity Averaging and FISTA

Magnetic resonance imaging (MRI) is widely adopted for clinical diagnosis due to its non-invasively detection. However, acquisition of full k-space data limits its imaging speed. Compressed sensing (CS) provides a new technique to significantly reduce the measurements with high-quality MR image reconstruction. The sparsity of the MR images is one of the crucial bases of CS-MRI. In this paper, we present to use sparsity averaging prior for CS-MRI reconstruction in the basis of that MR images have average sparsity over multiple wavelet frames. The problem is solved using a Fast Iterative Shrinkage Thresholding Algorithm (FISTA), each iteration of which includes a shrinkage step. The performance of the proposed method is evaluated for several types of MR images. The experiment results illustrate that our approach exhibits a better performance than those methods that using redundant frame or a single orthonormal basis to promote sparsity.     

基于DEMON谱和LSTM网络的水下运动目标噪声基频检测*

传统的船舶辐射噪声基频检测方法不仅依赖大量的先验知识,而且对背景噪声非常敏感。为了提高目标识别的稳定性和精确性,本文提出了一种基于深度神经网络的基频检测算法。首先从多通道水听器信号中提取DEMON谱,然后直接将二维谱特征矩阵输入由CNN和LSTM构成的级联网络,最后通过稠密层输出实现对基频的估计。从仿真和外场试验数据得到如下结论：（1）深度网络能够实现无先验知识和不同信噪比条件下的基频检测,具有良好的泛化性能。（2）LSTM网络能够高效地从时序DEMON谱中提取统计特征,提高基频估计精度。（3）输入信号的时间长短会影响网络的检测精度,更长时间的信号能够获得更好的检测结果。     

基于深度神经网络的时空编码磁共振成像超分辨率重建方法

单扫描时空编码磁共振成像是一种新型超快速磁共振成像技术,它对磁场不均匀和化学位移伪影有较强的抵抗性,但是其固有的空间分辨率较低,因此通常需要进行超分辨率重建,以在不增加采样点数的情况下提高时空编码磁共振图像的空间分辨率.然而,现有的重建方法存在迭代求解时间长、重建结果有混叠伪影残留等问题.为此,本文提出了一种基于深度神经网络的单扫描时空编码磁共振成像超分辨率重建方法.该方法采用模拟样本训练深度神经网络,再利用训练好的网络模型对实际采样信号进行重建.数值模拟、水模和活体鼠脑的实验结果表明,该方法能快速重建出无残留混叠伪影、纹理信息清楚的超分辨率时空编码磁共振图像.适当增加训练样本数量以及在训练样本中加入适当的随机噪声水平,有助于改善重建效果.     

Sensor and Actuator Fault Diagnosis for Robot Joint Based on Deep CNN

This paper proposes a data-driven method-based fault diagnosis method using the deep convolutional neural network (DCNN). The DCNN is used to deal with sensor and actuator faults of robot joints, such as gain error, offset error, and malfunction for both sensors and actuators, and different fault types are diagnosed using the trained neural network. In order to achieve the above goal, the fused data of sensors and actuators are used, where both types of fault are described in one formulation. Then, the deep convolutional neural network is applied to learn characteristic features from the merged data to try to find discriminative information for each kind of fault. After that, the fully connected layer does prediction work based on learned features. In order to verify the effectiveness of the proposed deep convolutional neural network model, different fault diagnosis methods including support vector machine (SVM), artificial neural network (ANN), conventional neural network (CNN) using the LeNet-5 method, and long-term memory network (LTMN) are investigated and compared with DCNN method. The results show that the DCNN fault diagnosis method can realize high fault recognition accuracy while needing less model training time.     

A Deep Neural Network Method for Arterial Blood Flow Profile Reconstruction

Arterial stenosis will reduce the blood flow to various organs or tissues, causing cardiovascular diseases. Although there are mature diagnostic techniques in clinical practice, they are not suitable for early cardiovascular disease prediction and monitoring due to their high cost and complex operation. In this paper, we studied the electromagnetic effect of arterial blood flow and proposed a method based on the deep neural network for arterial blood flow profile reconstruction. The potential difference and weight matrix are used as inputs to the method, and its output is an estimate of the internal blood flow velocity distribution for arterial blood flow profile reconstruction. Firstly, the weight matrix is input into the convolutional auto-encode (CAE) network to extract its features. Then, the weight matrix features and potential difference are combined to obtain the features of the blood velocity distribution. Finally, the velocity features are reconstructed into blood flow velocity distribution by a convolution neural network (CNN). All data sets are obtained from a model of the carotid artery with different rates of stenosis in a uniform magnetic field by COMSOL. The results show that the average root mean square error of the reconstruction results obtained by the proposed method is 0.0333, and the average correlation coefficient is 0.9721, which is better than the corresponding indicators of the Tikhonov, back propagation (BP) and CNN methods. The simulation results show that the proposed method can achieve high accuracy in blood flow profile reconstruction and is of great significance for the early diagnosis of arterial stenosis and other vessel diseases.     

基于级联卷积神经网络的前列腺磁共振图像分类

针对深度学习训练成本高,以及基于磁共振图像的前列腺癌临床诊断需要大量医学常识且极为耗时的问题,本文提出了一种基于级联卷积神经网络（Convolutional Neural Network,CNN）和磁共振图像的前列腺癌（Prostate Cancer,PCa）自动分类诊断方法,该网络以Faster-RCNN作为前网络,对前列腺区域进行提取分割,用于排除前列腺附近组织器官的干扰;以基于ResNet改进的网络结构CNN40_bottleneck作为后网络,用于对前列腺区域病变进行分类.后网络由瓶颈结构串联组成,其中使用批量标准化（Batch Normalization,BN）、全局平均池化（Global Average Pooling,GAP）进行优化.实验结果证明,本文方法对前列腺癌诊断结果较好,而且缩减了训练时间和参数量,有效降低了训练成本.     

GIS Partial Discharge Pattern Recognition Based on a Novel Convolutional Neural Networks and Long Short-Term Memory

Distinguishing the types of partial discharge (PD) caused by different insulation defects in gas-insulated switchgear (GIS) is a great challenge in the power industry, and improving the recognition accuracy of the relevant models is one of the key problems. In this paper, a convolutional neural network and long short-term memory (CNN-LSTM) model is proposed, which can effectively extract and utilize the spatiotemporal characteristics of PD input signals. First, the spatial characteristics of higher-level PD signals can be obtained through the CNN network, but because CNN is a deep feedforward neural network, it does not have the ability to process time-series data. The PD voltage signal is related to the time dimension, so LSTM saves and analyzes the previous voltage signal information, realizes the modeling of the time dependence of the data, and improves the accuracy of the PD signal pattern recognition. Finally, the pattern recognition results based on CNN-LSTM are given and compared with those based on other traditional analysis methods. The results show that the pattern recognition rate of this method is the highest, with an average of 97.9%, and its overall accuracy is better than that of other traditional analysis methods. The CNN-LSTM model provides a reliable reference for GIS PD diagnosis.