Deep neural network inspired by iterative shrinkage-thresholding algorithm with data consistency (NISTAD) for fast Undersampled MRI reconstruction

With the aim of developing a fast algorithm for high-quality MRI reconstruction from undersampled k-space data, we propose a novel deep neural Network, which is inspired by Iterative Shrinkage Thresholding Algorithm with Data consistency (NISTAD). NISTAD consists of three consecutive blocks: an encoding block, which models the flow graph of ISTA, a classical iteration-based compressed sensing magnetic resonance imaging (CS-MRI) method; a decoding block, which recovers the image from sparse representation; a data consistency block, which adaptively enforces consistency with the acquired raw data according to learned noise level. The ISTA method is thereby mapped to an end-to-end deep neural network, which greatly reduces the reconstruction time and simplifies the tuning of hyper-parameters, compared to conventional model-based CS-MRI methods. On the other hand, compared to general deep learning-based MRI reconstruction methods, the proposed method uses a simpler network architecture with fewer parameters. NISTAD has been validated on retrospectively undersampled diencephalon standard challenge data using different acceleration factors, and compared with DAGAN and Cascade CNN, two state-of-the-art deep neural network-based methods which outperformed many other state-of-the-art model-based and deep learning-based methods. Experimental results demonstrated that NISTAD reconstruction achieved comparable image quality with DAGAN and Cascade CNN reconstruction in terms of both PSNR and SSIM metrics, and subjective assessment, though with a simpler network structure.     

Incorporating prior knowledge via volumetric deep residual network to optimize the reconstruction of sparsely sampled MRI

For sparse sampling that accelerates magnetic resonance (MR) image acquisition, non-linear reconstruction algorithms have been developed, which incorporated patient specific a prior information. More generic a prior information could be acquired via deep learning and utilized for image reconstruction. In this study, we developed a volumetric hierarchical deep residual convolutional neural network, referred to as T-Net, to provide a data-driven end-to-end mapping from sparsely sampled MR images to fully sampled MR images, where cartilage MR images were acquired using an Ultra-short TE sequence and retrospectively undersampled using pseudo-random Cartesian and radial acquisition schemes. The network had a hierarchical architecture that promoted the sparsity of feature maps and increased the receptive field, which were valuable for signal synthesis and artifact suppression. Relatively dense local connections and global shortcuts were established to facilitate residual learning and compensate for details lost in hierarchical processing. Additionally, volumetric processing was adopted to fully exploit spatial continuity in three-dimensional space. Data consistency was further enforced. The network was trained with 336 three-dimensional images (each consisting of 32 slices) and tested by 24 images. The incorporation of a priori information acquired via deep learning facilitated high acceleration factors (as high as 8) while maintaining high image fidelity (quantitatively evaluated using the structural similarity index measurement). The proposed T-Net had an improved performance as compared to several state-of-the-art networks.     

An unsupervised deep learning technique for susceptibility artifact correction in reversed phase-encoding EPI images

Echo planar imaging (EPI) is a fast and non-invasive magnetic resonance imaging technique that supports data acquisition at high spatial and temporal resolutions. However, susceptibility artifacts, which cause the misalignment to the underlying structural image, are unavoidable distortions in EPI. Traditional susceptibility artifact correction (SAC) methods estimate the displacement field by optimizing an objective function that involves one or more pairs of reversed phase-encoding (PE) images. The estimated displacement field is then used to unwarp the distorted images and produce the corrected images. Since this conventional approach is time-consuming, we propose an end-to-end deep learning technique, named S-Net, to correct the susceptibility artifacts the reversed-PE image pair. The proposed S-Net consists of two components: (i) a convolutional neural network to map a reversed-PE image pair to the displacement field; and (ii) a spatial transform unit to unwarp the input images and produce the corrected images. The S-Net is trained using a set of reversed-PE image pairs and an unsupervised loss function, without ground-truth data. For a new image pair of reversed-PE images, the displacement field and corrected images are obtained simultaneously by evaluating the trained S-Net directly. Evaluations on three different datasets demonstrate that S-Net can correct the susceptibility artifacts in the reversed-PE images. Compared with two state-of-the-art SAC methods (TOPUP and TISAC), the proposed S-Net runs significantly faster: 20 times faster than TISAC and 369 times faster than TOPUP, while achieving a similar correction accuracy. Consequently, S-Net accelerates the medical image processing pipelines and makes the real-time correction for MRI scanners feasible. Our proposed technique also opens up a new direction in learning-based SAC.     

基于PCAU-Net的快速多通道磁共振成像方法

多通道磁共振成像方法采用多个接收线圈同时欠采样k空间以加快成像速度,并基于后处理算法重建图像,但在较高加速因子时,其图像重建质量仍然较差．本文提出了一种基于PCAU-Net的快速多通道磁共振成像方法,将单通道实数U型卷积神经网络拓展到多通道复数卷积神经网络,设计了一种结构不对称的U型网络结构,通过在解码部分减小网络规模以降低模型的复杂度．PCAU-Net网络在跳跃连接前增加了1×1卷积,以实现跨通道信息交互．输入和输出之间利用残差连接为误差的反向传播提供捷径．实验结果表明,使用规则和随机采样模板,在不同加速因子时,相比常规的GRAPPA重建算法和SPIRiT重建方法,本文提出的PCAU-Net方法可高质量重建出磁共振复数图像,并且相比于PCU-Net方法,PCAU-Net减少了模型参数、缩短了训练时间．     

High-performance rapid MR parameter mapping using model-based deep adversarial learning

PurposeTo develop and evaluate a deep adversarial learning-based image reconstruction approach for rapid and efficient MR parameter mapping.MethodsThe proposed method provides an image reconstruction framework by combining the end-to-end convolutional neural network (CNN) mapping, adversarial learning, and MR physical models. The CNN performs direct image-to-parameter mapping by transforming a series of undersampled images directly into MR parameter maps. Adversarial learning is used to improve image sharpness and enable better texture restoration during the image-to-parameter conversion. An additional pathway concerning the MR signal model is added between the estimated parameter maps and undersampled k-space data to ensure the data consistency during network training. The proposed framework was evaluated on T₂ mapping of the brain and the knee at an acceleration rate R = 8 and was compared with other state-of-the-art reconstruction methods. Global and regional quantitative assessments were performed to demonstrate the reconstruction performance of the proposed method.ResultsThe proposed adversarial learning approach achieved accurate T₂ mapping up to R = 8 in brain and knee joint image datasets. Compared to conventional reconstruction approaches that exploit image sparsity and low-rankness, the proposed method yielded lower errors and higher similarity to the reference and better image sharpness in the T₂ estimation. The quantitative metrics were normalized root mean square error of 3.6% for brain and 7.3% for knee, structural similarity index of 85.1% for brain and 83.2% for knee, and tenengrad measures of 9.2% for brain and 10.1% for the knee. The adversarial approach also achieved better performance for maintaining greater image texture and sharpness in comparison to the CNN approach without adversarial learning.ConclusionThe proposed framework by incorporating the efficient end-to-end CNN mapping, adversarial learning, and physical model enforced data consistency is a promising approach for rapid and efficient reconstruction of quantitative MR parameters.     

Singular Value Decomposition Using Jacobi Algorithm in pMRI and CS

Parallel magnetic resonance imaging (pMRI) and compressed sensing (CS) have been recently used to accelerate data acquisition process in MRI. Matrix inversion (for rectangular matrices) is required to reconstruct images from the acquired under-sampled data in various pMRI algorithms (e.g., SENSE, GRAPPA) and CS. Singular value decomposition (SVD) provides a mechanism to accurately estimate pseudo-inverse of a rectangular matrix. This work proposes the use of Jacobi SVD algorithm to reconstruct MR images from the acquired under-sampled data both in pMRI and in CS. The use of Jacobi SVD algorithm is proposed in advance MRI reconstruction algorithms, including SENSE, GRAPPA, and low-rank matrix estimation in L + S model for matrix inversion and estimation of singular values. Experiments are performed on 1.5T human head MRI data and 3T cardiac perfusion MRI data for different acceleration factors. The reconstructed images are analyzed using artifact power and central line profiles. The results show that the Jacobi SVD algorithm successfully reconstructs the images in SENSE, GRAPPA, and L + S algorithms. The benefit of using Jacobi SVD algorithm for MRI image reconstruction is its suitability for parallel computation on GPUs, which may be a great help in reducing the image reconstruction time.     

基于深度递归级联卷积神经网络的并行磁共振成像方法

快速磁共振成像是磁共振研究领域重要的课题之一.随着大数据和深度学习的兴起,神经网络成为快速磁共振技术的重要方法.然而网络性能表现和网络参数量之间较难取得平衡,且对于多通道数据重建的并行成像问题,相关研究较少.本文构建了一种深度递归级联卷积神经网络结构,用于处理并行成像问题.这种网络结构在减少网络参数量的同时,能够尽可能地提高网络的表达能力,提高网络重建的精确度.实验结果表明,相较于传统并行成像方法,通过训练好的神经网络对欠采样磁共振数据进行重建,可以得到更准确的重建结果,且重建时间大大缩短.     

Motion correction based reconstruction method for compressively sampled cardiac MR imaging

Respiratory motion during Magnetic Resonance (MR) acquisition causes strong blurring artifacts in the reconstructed images. These artifacts become more pronounced when used with the fast imaging reconstruction techniques like compressed sensing (CS). Recently, an MR reconstruction technique has been done with the help of compressed sensing (CS), to provide good quality sparse images from the highly under-sampled k-space data. In order to maximize the benefits of CS, it is obvious to use CS with the motion corrected samples. In this paper, we propose a novel CS based motion corrected image reconstruction technique. First, k-space data have been assigned to different respiratory state with the help of frequency domain phase correlation method. Then, multiple sparsity constraints has been used to provide good quality reconstructed cardiac cine images with the highly under-sampled k-space data. The proposed method exploits the multiple sparsity constraints, in combination with demon based registration technique and a novel reconstruction technique to provide the final motion free images. The proposed method is very simple to implement in clinical settings as compared to existing motion corrected methods. The performance of the proposed method is examined using simulated data and clinical data. Results show that this method performs better than the reconstruction of CS based method of cardiac cine images. Different acceleration rates have been used to show the performance of the proposed method.     

Deriving new soft tissue contrasts from conventional MR images using deep learning

Versatile soft tissue contrast in magnetic resonance imaging is a unique advantage of the imaging modality. However, the versatility is not fully exploited. In this study, we propose a deep learning-based strategy to derive more soft tissue contrasts from conventional MR images obtained in standard clinical MRI. Two types of experiments are performed. First, MR images corresponding to different pulse sequences are predicted from one or more images already acquired. As an example, we predict T_1ρ weighted knee image from T₂ weighted image and/or T₁ weighted image. Furthermore, we estimate images corresponding to alternative imaging parameter values. In a representative case, variable flip angle images are predicted from a single T₁ weighted image, whose accuracy is further validated in quantitative T₁ map subsequently derived. To accomplish these tasks, images are retrospectively collected from 56 subjects, and self-attention convolutional neural network models are trained using 1104 knee images from 46 subjects and tested using 240 images from 10 other subjects. High accuracy has been achieved in resultant qualitative images as well as quantitative T₁ maps. The proposed deep learning method can be broadly applied to obtain more versatile soft tissue contrasts without additional scans or used to normalize MR data that were inconsistently acquired for quantitative analysis.     

Deep learning for radial SMS myocardial perfusion reconstruction using the 3D residual booster U-net

PurposeTo develop an end-to-end deep learning solution for quickly reconstructing radial simultaneous multi-slice (SMS) myocardial perfusion datasets with comparable quality to the pixel tracking spatiotemporal constrained reconstruction (PT-STCR) method.MethodsDynamic contrast enhanced (DCE) radial SMS myocardial perfusion data were obtained from 20 subjects who were scanned at rest and/or stress with or without ECG gating using a saturation recovery radial CAIPI turboFLASH sequence. Input to the networks consisted of complex coil combined images reconstructed using the inverse Fourier transform of undersampled radial SMS k-space data. Ground truth images were reconstructed using the PT-STCR pipeline. The performance of the residual booster 3D U-Net was tested by comparing it to state-of-the-art network architectures including MoDL, CRNN-MRI, and other U-Net variants.ResultsResults demonstrate significant improvements in speed requiring approximately 8 seconds to reconstruct one radial SMS dataset which is approximately 200 times faster than the PT-STCR method. Images reconstructed with the residual booster 3D U-Net retain quality of ground truth PT-STCR images (0.963 SSIM/40.238 PSNR/0.147 NRMSE). The residual booster 3D U-Net has superior performance compared to existing network architectures in terms of image quality, temporal dynamics, and reconstruction time.ConclusionResidual and booster learning combined with the 3D U-Net architecture was shown to be an effective network for reconstructing high-quality images from undersampled radial SMS datasets while bypassing the reconstruction time of the PT-STCR method.     

Deep artifact suppression for spiral real-time phase contrast cardiac magnetic resonance imaging in congenital heart disease

PurposeReal-time spiral phase contrast MR (PCMR) enables rapid free-breathing assessment of flow. Target spatial and temporal resolutions require high acceleration rates often leading to long reconstruction times. Here we propose a deep artifact suppression framework for fast and accurate flow quantification.MethodsU-Nets were trained for deep artifact suppression using 520 breath-hold gated spiral PCMR aortic datasets collected in congenital heart disease patients. Two spiral trajectories (uniform and perturbed) and two losses (Mean Absolute Error -MAE- and average structural similarity index measurement -SSIM-) were compared in synthetic data in terms of MAE, peak SNR (PSNR) and SSIM. Perturbed spiral PCMR was prospectively acquired in 20 patients. Stroke Volume (SV), peak mean velocity and edge sharpness measurements were compared to Compressed Sensing (CS) and Cartesian reference.ResultsIn synthetic data, perturbed spiral consistently outperformed uniform spiral for the different image metrics. U-Net MAE showed better MAE and PSNR while U-Net SSIM showed higher SSIM based metrics.In-vivo, there were no significant differences in SV between any of the real-time reconstructions and the reference standard Cartesian data. However, U-Net SSIM had better image sharpness and lower biases for peak velocity when compared to U-Net MAE. Reconstruction of 96 frames took ~59 s for CS and 3.9 s for U-Nets.ConclusionDeep artifact suppression of complex valued images using an SSIM based loss was successfully demonstrated in a cohort of congenital heart disease patients for fast and accurate flow quantification.     

Deep unregistered multi-contrast MRI reconstruction

Multiple magnetic resonance images of different contrasts are normally acquired for clinical diagnosis. Recently, research has shown that the previously acquired multi-contrast (MC) images of the same patient can be used as anatomical prior to accelerating magnetic resonance imaging (MRI). However, current MC-MRI networks are based on the assumption that the images are perfectly registered, which is rarely the case in real-world applications. In this paper, we propose an end-to-end deep neural network to reconstruct highly accelerated images by exploiting the shareable information from potentially misaligned reference images of an arbitrary contrast. Specifically, a spatial transformation (ST) module is designed and integrated into the reconstruction network to align the pre-acquired reference images with the images to be reconstructed. The misalignment is further alleviated by maximizing the normalized cross-correlation (NCC) between the MC images. The visualization of feature maps demonstrates that the proposed method effectively reduces the misalignment between the images for shareable information extraction when applied to the publicly available brain datasets. Additionally, the experimental results on these datasets show the proposed network allows the robust exploitation of shareable information across the misaligned MC images, leading to improved reconstruction results.     

Supervised Domain Adaptation for Automated Semantic Segmentation of the Atrial Cavity

Atrial fibrillation (AF) is the most common cardiac arrhythmia. At present, cardiac ablation is the main treatment procedure for AF. To guide and plan this procedure, it is essential for clinicians to obtain patient-specific 3D geometrical models of the atria. For this, there is an interest in automatic image segmentation algorithms, such as deep learning (DL) methods, as opposed to manual segmentation, an error-prone and time-consuming method. However, to optimize DL algorithms, many annotated examples are required, increasing acquisition costs. The aim of this work is to develop automatic and high-performance computational models for left and right atrium (LA and RA) segmentation from a few labelled MRI volumetric images with a 3D Dual U-Net algorithm. For this, a supervised domain adaptation (SDA) method is introduced to infer knowledge from late gadolinium enhanced (LGE) MRI volumetric training samples (80 LA annotated samples) to a network trained with balanced steady-state free precession (bSSFP) MR images of limited number of annotations (19 RA and LA annotated samples). The resulting knowledge-transferred model SDA outperformed the same network trained from scratch in both RA (Dice equals 0.9160) and LA (Dice equals 0.8813) segmentation tasks.     

基于深度神经网络的时空编码磁共振成像超分辨率重建方法

单扫描时空编码磁共振成像是一种新型超快速磁共振成像技术,它对磁场不均匀和化学位移伪影有较强的抵抗性,但是其固有的空间分辨率较低,因此通常需要进行超分辨率重建,以在不增加采样点数的情况下提高时空编码磁共振图像的空间分辨率.然而,现有的重建方法存在迭代求解时间长、重建结果有混叠伪影残留等问题.为此,本文提出了一种基于深度神经网络的单扫描时空编码磁共振成像超分辨率重建方法.该方法采用模拟样本训练深度神经网络,再利用训练好的网络模型对实际采样信号进行重建.数值模拟、水模和活体鼠脑的实验结果表明,该方法能快速重建出无残留混叠伪影、纹理信息清楚的超分辨率时空编码磁共振图像.适当增加训练样本数量以及在训练样本中加入适当的随机噪声水平,有助于改善重建效果.     

GPU based parallel framework for receiver coil sensitivity estimation in SENSE reconstruction

Magnetic Resonance Imaging (MRI) uses non-ionizing radiations and is safer as compared to CT and X-ray imaging. MRI is broadly used around the globe for medical diagnostics. One main limitation of MRI is its long data acquisition time. Parallel MRI (pMRI) was introduced in late 1990's to reduce the MRI data acquisition time. In pMRI, data is acquired by under-sampling the Phase Encoding (PE) steps which introduces aliasing artefacts in the MR images. SENSitivity Encoding (SENSE) is a pMRI based method that reconstructs fully sampled MR image from the acquired under-sampled data using the sensitivity information of receiver coils. In SENSE, precise estimation of the receiver coil sensitivity maps is vital to obtain good quality images. Eigen-value method (a recently proposed method in literature for the estimation of receiver coil sensitivity information) does not require a pre-scan image unlike other conventional methods of sensitivity estimation. However, Eigen-value method is computationally intensive and takes a significant amount of time to estimate the receiver coil sensitivity maps. This work proposes a parallel framework for Eigen-value method of receiver coil sensitivity estimation that exploits its inherent parallelism using Graphics Processing Units (GPUs). We evaluated the performance of the proposed algorithm on in-vivo and simulated MRI datasets (i.e. human head and simulated phantom datasets) with Peak Signal-to-Noise Ratio (PSNR) and Artefact Power (AP) as evaluation metrics. The results show that the proposed GPU implementation reduces the execution time of Eigen-value method of receiver coil sensitivity estimation (providing up to 30 times speed up in our experiments) without degrading the quality of the reconstructed image.     

基于模糊标签和深度学习的TOF-MRA影像脑动脉瘤自动检测

脑动脉瘤破裂造成的蛛网膜下腔出血致死致残率极高，借助深度学习网络辅助医生实现高效筛查具有重要意义．为提高基于时间飞跃法磁共振血管造影（Time of Flight-Magnetic Resonance Angiography，TOF-MRA）的脑动脉瘤自动检测的精度，本文基于模糊标签方式，提出一种基于变体3D U-Net和双分支通道注意力（Dual-branch Channel Attention，DCA）的深度神经网络DCAU-Net，DCA模块可以自适应地调整通道特征的响应，提高特征提取能力．首先对260例病例的TOF-MRA影像预处理，将数据集分为174例训练集、43例验证集和43例测试集，然后使用处理后的数据训练和验证DCAU-Net，测试集实验结果表明DCAU-Net可以达到90.69%的敏感度，0.83个/例的假阳性计数和0.52的阳性预测值，有望为动脉瘤筛查提供参考．     

DeepcomplexMRI: Exploiting deep residual network for fast parallel MR imaging with complex convolution

This paper proposes a multi-channel image reconstruction method, named DeepcomplexMRI, to accelerate parallel MR imaging with residual complex convolutional neural network. Different from most existing works which rely on the utilization of the coil sensitivities or prior information of predefined transforms, DeepcomplexMRI takes advantage of the availability of a large number of existing multi-channel groudtruth images and uses them as target data to train the deep residual convolutional neural network offline. In particular, a complex convolutional network is proposed to take into account the correlation between the real and imaginary parts of MR images. In addition, the k-space data consistency is further enforced repeatedly in between layers of the network. The evaluations on in vivo datasets show that the proposed method has the capability to recover the desired multi-channel images. Its comparison with state-of-the-art methods also demonstrates that the proposed method can reconstruct the desired MR images more accurately.     

Dynamic cardiac MRI reconstruction using motion aligned locally low rank tensor (MALLRT)

Various sparse transform models have been explored for compressed sensing-based dynamic cardiac MRI reconstruction from vastly under-sampled k-space data. Recently emerged low rank tensor model using Tucker decomposition could be viewed as a special form of sparse model, where the core tensor, which is obtained using high-order singular value decomposition, is sparse in the sense that only a few elements have dominantly large magnitude. However, local details tend to be over-smoothed when the entire image is conventionally modeled as a global tensor. Moreover, low rankness is sensitive to motion as spatiotemporal correlation is corrupted by spatial misalignment between temporal frames. To overcome these limitations, this paper presents a novel motion aligned locally low rank tensor (MALLRT) model for dynamic MRI reconstruction. In MALLRT, low rank constraint is enforced on image patch-based local tensors, which correspond to overlapping blocks extracted from the reconstructed high-dimensional image after group-wise inter-frame motion registration. For solving the proposed model, this paper presents an efficient optimization algorithm by using variable splitting and alternating direction method of multipliers (ADMM). MALLRT demonstrated promising performance as validated on one cardiac perfusion MRI dataset and two cardiac cine MRI datasets using retrospective under-sampling with various acceleration factors, as well as one prospectively under-sampled cardiac perfusion MRI dataset. Compared to four state-of-the-art methods, MALLRT achieved substantially better image reconstruction quality in terms of both signal to error ratio (SER) and structural similarity index (SSIM) metrics, and visual perception in preserving spatial details and capturing temporal variations.     

Deep learning for image reconstruction in thermoacoustic tomography

Microwave-induced thermoacoustic tomography(TAT)is a rapidly-developing noninvasive imaging technique that integrates the advantages of microwave imaging and ultrasound imaging.While an image reconstruction algorithm is critical for the TAT,current reconstruction methods often creates significant artifacts and are computationally costly.In this work,we propose a deep learning-based end-to-end image reconstruction method to achieve the direct reconstruction from the sinogram data to the initial pressure density image.We design a new network architecture TAT-Net to transfer the sinogram domain to the image domain with high accuracy.For the scenarios where realistic training data are scarce or unavailable,we use the finite element method(FEM)to generate synthetic data where the domain gap between the synthetic and realistic data is resolved through the signal processing method.The TAT-Net trained with synthetic data is evaluated through both simulations and phantom experiments and achieves competitive performance in artifact removal and robustness.Compared with other state-of-the-art reconstruction methods,the TAT-Net method can reduce the root mean square error to 0.0143,and increase the structure similarity and peak signal-to-noise ratio to 0.988 and 38.64,respectively.The results obtained indicate that the TAT-Net has great potential applications in improving image reconstruction quality and fast quantitative reconstruction.