Deep unregistered multi-contrast MRI reconstruction

Multiple magnetic resonance images of different contrasts are normally acquired for clinical diagnosis. Recently, research has shown that the previously acquired multi-contrast (MC) images of the same patient can be used as anatomical prior to accelerating magnetic resonance imaging (MRI). However, current MC-MRI networks are based on the assumption that the images are perfectly registered, which is rarely the case in real-world applications. In this paper, we propose an end-to-end deep neural network to reconstruct highly accelerated images by exploiting the shareable information from potentially misaligned reference images of an arbitrary contrast. Specifically, a spatial transformation (ST) module is designed and integrated into the reconstruction network to align the pre-acquired reference images with the images to be reconstructed. The misalignment is further alleviated by maximizing the normalized cross-correlation (NCC) between the MC images. The visualization of feature maps demonstrates that the proposed method effectively reduces the misalignment between the images for shareable information extraction when applied to the publicly available brain datasets. Additionally, the experimental results on these datasets show the proposed network allows the robust exploitation of shareable information across the misaligned MC images, leading to improved reconstruction results.     

Scale and rotation robust line-based matching for high resolution images

A line-based matching method is proposed to overcome the low significant level of point feature and the shortage in the matching between weak texture images. Edges are extracted to generate line segments at first. Then, saliency-lines are detected and a control network is constructed in each image. Every remaining general-line is grouped into a saliency-line through a process of clustering. Finally, an iterative algorithm is applied to search for correspondences based on their position relative to the saliency-line. Sufficient spatial information is available to reduce the ambiguity and avoid false matches. Experimental results demonstrate that the proposed method is robust to image scale change and rotation, and the performance is better than point-based method in low texture area.     

MRI denoising using progressively distribution-based neural network

Magnetic Resonance (MR) images often suffer from noise pollution during image acquisition and transmission, which limits the accuracy of quantitative measurements from the data. Noise in magnitude MR images is usually governed by Rician distribution, due to the existence of uncorrelated Gaussian noise with zero-mean and equal variance in both the real and imaginary parts of the complex K-space data. Different from the existing MRI denoising methods that utilizing the spatial neighbor information around the pixels or patches, this work turns to capture the pixel-level distribution information by means of supervised network learning. A progressive network learning strategy is proposed via fitting the distribution of pixel-level and feature-level intensities. The proposed network consists of two residual blocks, one is used for fitting pixel domain without batch normalization layer and another one is applied for matching feature domain with batch normalization layer. Experimental results under synthetic, complex-valued and clinical MR brain images demonstrate great potential of the proposed network with substantially improved quantitative measures and visual inspections.     

基于PCAU-Net的快速多通道磁共振成像方法

多通道磁共振成像方法采用多个接收线圈同时欠采样k空间以加快成像速度,并基于后处理算法重建图像,但在较高加速因子时,其图像重建质量仍然较差．本文提出了一种基于PCAU-Net的快速多通道磁共振成像方法,将单通道实数U型卷积神经网络拓展到多通道复数卷积神经网络,设计了一种结构不对称的U型网络结构,通过在解码部分减小网络规模以降低模型的复杂度．PCAU-Net网络在跳跃连接前增加了1×1卷积,以实现跨通道信息交互．输入和输出之间利用残差连接为误差的反向传播提供捷径．实验结果表明,使用规则和随机采样模板,在不同加速因子时,相比常规的GRAPPA重建算法和SPIRiT重建方法,本文提出的PCAU-Net方法可高质量重建出磁共振复数图像,并且相比于PCU-Net方法,PCAU-Net减少了模型参数、缩短了训练时间．     

Hybrid network with difference degree and attention mechanism combined with radiomics (H-DARnet) for MVI prediction in HCC

MVI is a risk assessment factor related to hepatocellular carcinoma (HCC) recurrence after hepatectomy or liver transplantation. The goal of this paper is to study the preoperative diagnosis of microvascular invasion (MVI) by using a deep learning algorithm in non-contrast T2 weighted magnetic resonance imaging (MRI) images instead of pathological images. Herein, an ensemble learning algorithm named H-DARnet—based on the difference degree and attention mechanism, combined with radiomics, for MVI prediction—is proposed. Our hybrid network combines the fine-grained, high-level semantic, and radiomics features and exhibits a rich multilevel-feature architecture composed of global-local-prior knowledge with suitable complementarity. The total loss function comprises two regularization items––the triplet and the cross-entropy loss function––which are selected for the triplet network and SE-DenseNet, respectively. The hard triplet sample selection strategy for a triplet network and data augmentation for small-scale liver image datasets in convolutional neural network (CNN) training is indispensable. For 200 patch level test samples (135 positive samples and 65 negative samples), our method can obtain the best prediction results, the AUC, sensitivity, and specificity were 0.826, 79.5% and 73.8%, respectively. The experiment results show that MVI can be predicted by using MRI images, and the proposed method is better than other deep learning algorithms and hand-crafted feature algorithms. The proposed ensemble learning algorithm is proved to be an effective method for MVI prediction.     

Whole volume brain extraction for multi-centre,multi-disease FLAIR MRI datasets

Automatic segmentation of the brain from magnetic resonance images (MRI) is a fundamental step in many neuroimaging processing frameworks. There are mature technologies for this task for T1- and T2-weighted MRI; however, a widely-accepted brain extraction method for Fluid-Attenuated Inversion Recovery (FLAIR) MRI has yet to be established. FLAIR MRI are becoming increasingly important for the analysis of neurodegenerative diseases and tools developed for this sequence would have clinical value. To maximize translation opportunities and for large scale research studies, algorithms for brain extraction in FLAIR MRI should generalize to multi-centre (MC) data. To this end, this work proposes a fully automated, whole volume brain extraction methodology for MC FLAIR MRI datasets. The framework is built using a novel standardization framework which reduces acquisition artifacts, standardizes the intensities of tissues and normalizes the spatial coordinates of brain tissue across MC datasets. Using the standardized datasets, an intuitive set of features based on intensity, spatial location and gradients are extracted and classified using a random forest (RF) classifier to segment the brain tissue class. A series of experiments were conducted to optimize classifier parameters, and to determine segmentation accuracy for standardized and unstandardized (original) data, as a function of scanner vendor, feature type and disease type. The models are trained, tested and validated on 156 image volumes (∼8000 image slices) from two multi-centre, multi-disease datasets, acquired with varying imaging parameters from 30 centres and three scanner vendors. The image datasets, denoted as CAIN and ADNI for vascular and dementia disease, respectively, represent a diverse collection of MC data to test the generalization capabilities of the proposed design. Results demonstrate the importance of standardization for segmentation of MC data, as models trained on standardized data yielded a drastic improvement in brain extraction accuracy compared to the original, unstandardized data (CAIN: DSC = 91% and ADNI: DSC = 86% vs. CAIN: 78% and ADNI: 65%). It was also found that models created from one scanner vendor based on unstandardized data yielded poor segmentation results in data acquired from other scanner vendors, which was improved through standardization. These results demonstrate that to create consistency in segmentations from multi-institutional datasets it is paramount that MC variability be mitigated to improve stability and to ensure generalization of machine learning algorithms for MRI.     

基于多任务卷积神经网络的红外与可见光多分辨率图像融合

红外与可见光图像融合一直是图像领域研究的热点,融合技术能弥补单一传感器的不足,为图像理解与分析提供良好的成像基础。因生产工艺以及成本的限制,红外探测器的分辨率远低于可见光探测器,并在一定程度上因源图像分辨率的差异阻碍了实际应用。针对红外与可见光图像分辨率不一致的问题,提出了用于红外图像超分辨率重建与融合的多任务卷积网络框架,应用于多分辨率图像融合。在网络结构方面,首先设计了双通道网络分别提取红外与可见光特征,使算法不受源图像分辨率的限制;其次提出了特征上采样模块,先用双线性插值方法增加像素个数,再通过多层感知器精细化拟合像素平滑空间与高频空间的映射关系,无需重新训练模型即可实现任意尺度的红外图像上采样;接着将线性注意力引入网络,学习特征空间位置间的非线性关系,抑制无关信息并增强网络对全局信息的表达。在损失函数方面,提出了梯度损失,保留红外与可见光图像中绝对值较大的滤波器响应值,并计算该值与重建的融合图像响应值的Frobenius范数,无需理想的融合图像作为真值监督网络学习就能生成融合图像;此外,在梯度损失、像素损失的共同作用下对多任务模型进行优化,可以同时重建融合图像和高分辨率红外图像...     

A 3D densely connected convolution neural network with connection-wise attention mechanism for Alzheimer's disease classification

PurposeAlzheimer's disease (AD) is a progressive and irreversible neurodegenerative disease. In recent years, machine learning methods have been widely used on analysis of neuroimage for quantitative evaluation and computer-aided diagnosis of AD or prediction on the conversion from mild cognitive impairment (MCI) to AD. In this study, we aimed to develop a new deep learning method to detect or predict AD in an efficient way.Materials and methodsWe proposed a densely connected convolution neural network with connection-wise attention mechanism to learn the multi-level features of brain MR images for AD classification. We used the densely connected neural network to extract multi-scale features from pre-processed images, and connection-wise attention mechanism was applied to combine connections among features from different layers to hierarchically transform the MR images into more compact high-level features. Furthermore, we extended the convolution operation to 3D to capture the spatial information of MRI. The features extracted from each 3D convolution layer were integrated with features from all preceding layers with different attention, and were finally used for classification. Our method was evaluated on the baseline MRI of 968 subjects from ADNI database to discriminate (1) AD versus healthy subjects, (2) MCI converters versus healthy subjects, and (3) MCI converters versus non-converters.ResultsThe proposed method achieved 97.35% accuracy for distinguishing AD patients from healthy control, 87.82% for MCI converters against healthy control, and 78.79% for MCI converters against non-converters. Compared with some neural networks and methods reported in recent studies, the classification performance of our proposed algorithm was among the top ranks and improved in discriminating MCI subjects who were in high risks of conversion to AD.ConclusionsDeep learning techniques provide a powerful tool to explore minute but intricate characteristics in MR images which may facilitate early diagnosis and prediction of AD.     

Deep pancreas segmentation with uncertain regions of shadowed sets

Pancreas segmentation is a challenging task in medical image analysis especially for the patients with pancreatic cancer. First, the images often have poor contrast and blurred boundaries. Second, there exist large variations in gray scale, texture, location, shape and size among pancreas images. It becomes even worse with cases of pancreatic cancer. Besides, as an inevitable phenomenon, some of the slices have disconnected topology in pancreas part. All these problems lead to high segmentation uncertainties and make the results inaccurate. Existing pancreas segmentation methods rarely achieve sufficiently accurate and robust results especially for cancer cases. To tackle these problems, we propose a 2D deep learning-based method which can involve uncertainties in the process of segmentation iteratively. The proposed method describes the uncertain regions of pancreatic MRI images based on shadowed sets theory. The results are further corrected through increasing the weights of uncertain regions in iterative training. We evaluate our approach on a challenging pancreatic cancer MRI images dataset collected from the Changhai Hospital, and also validate our approach on the NIH pancreas segmentation dataset. The experimental results demonstrate that our proposed method outperforms the state-of-the-art methods in terms of the Dice similarity coefficient of 73.88% on cancer MRI dataset and 84.37% on NIH dataset respectively.     

Joint Reconstruction of Multi-contrast Images and Multi-channel Coil Sensitivities

Magnetic resonance imaging (MRI) has an important feature that it provides multiple images with different contrasts for complementary diagnostic information. However, a large amount of data is needed for multi-contrast images depiction, and thus, the scan is time-consuming. Many methods based on parallel magnetic resonance imaging (pMRI) and compressed sensing (CS) are applied to accelerate multi-contrast MR imaging. Nevertheless, the image reconstructed by sophisticated pMRI methods contains residual aliasing artifact that degrades the quality of the image when the acceleration factor is high. Other methods based on CS always suffer the regularization parameter-selecting problem. To address these issues, a new method is presented for joint multi-contrast image reconstruction and coil sensitivity estimation. The coil sensitivities can be shared during the reconstruction due to the identity of coil sensitivity profiles of different contrast images for imaging stationary tissues. The proposed method uses the coil sensitivities as sharable information during the reconstruction to improve the reconstruction quality. As a result, the residual aliasing artifact can be effectively removed in the reconstructed multi-contrast images even if the acceleration factor is high. Besides, as there is no regularization term in the proposed method, the troublesome regularization parameter selection in the CS can also be avoided. Results from multi-contrast in vivo experiments demonstrated that multi-contrast images can be jointly reconstructed by the proposed method with effective removal of the residual aliasing artifact at a high acceleration factor.     

IKWI-net: A cross-domain convolutional neural network for undersampled magnetic resonance image reconstruction

Magnetic resonance imaging (MRI) is widely used to get the information of anatomical structure and physiological function with the advantages of high resolution and non-invasive scanning. But the long acquisition time limits its application. To reduce the time consumption of MRI, compressed sensing (CS) theory has been proposed to reconstruct MRI images from undersampled k-space data. But conventional CS methods mostly use iterative methods that take lots of time. Recently, deep learning methods are proposed to achieve faster reconstruction, but most of them only pay attention to a single domain, such as the image domain or k-space. To take advantage of the feature representation in different domains, we propose a cross-domain method based on deep learning, which first uses convolutional neural networks (CNNs) in the image domain, k-space and wavelet domain simultaneously. The combined order of the three domains is also first studied in this work, which has a significant effect on reconstruction. The proposed IKWI-net achieves the best performance in various combinations, which utilizes CNNs in the image domain, k-space, wavelet domain and image domain sequentially. Compared with several deep learning methods, experiments show it also achieves mean improvements of 0.91 dB in peak signal-to-noise ratio (PSNR) and 0.005 in structural similarity (SSIM).     

Simulation of polarization-sensitive optical coherence tomography images by a Monte Carlo method

We introduce a new Monte Carlo (MC) method for simulating optical coherence tomography (OCT) images of complex multilayered turbid scattering media. We demonstrate, for the first time of our knowledge, the use of a MC technique to imitate two-dimensional polarization-sensitive OCT images with nonplanar boundaries of layers in the medium like a human skin. The simulation of polarized low-coherent optical radiation is based on the vector approach generalized from the iterative procedure of the solution of Bethe-Saltpeter equation. The performances of the developed method are demonstrated both for conventional and polarization-sensitive OCT modalities.     

An adaptive super-resolution method based on regional pixel information and ringing artifacts suppression

Multi-frame image super-resolution (SR) aims to utilize information from a set of low-resolution (LR) images to compose a high-resolution (HR) one. In this paper, a novel multi-frame image super-resolution algorithm is proposed based on regional pixel information and ringing artifacts suppression. Firstly, a new regularization term which adopts Regional Adaptive Weight Coefficients (RAWC) is produced to keep edges and flat regions. After detailed analysis, an iterative process is given for image reconstruction. Then an adaptive term according to the local variance of iterative correction image is designed to evaluate the ringing artifacts. Finally, the original iteration is updated by adding the restraint term for better visual effects and lower noise of reconstructive HR image. Thorough experimental results show the proposed algorithm is effective for SR reconstruction and ringing artifacts suppression.     

基于红外和可见光图像逐级自适应融合的场景深度估计

从图像中恢复场景的深度是计算机视觉领域中的一个关键问题。考虑到单一类型图像在深度估计中受场景不同光照的限制，提出了基于红外和可见光图像逐级自适应融合的场景深度估计方法（PF-CNN）。该方法包括双流滤波器部分耦合网络、自适应多模态特征融合网络以及自适应逐级特征融合网络。在双流卷积中红外和可见光图像的滤波器部分耦合使两者特征得到增强；自适应多模态特征融合网络学习红外和可见光图像的残差特征并将两者自适应加权融合，充分利用两者的互补信息；逐级特征融合网络学习多层融合特征的结合，充分利用不同卷积层的不同特征。实验结果表明:PF-CNN在测试集上获得了较好的效果，将阈值指标提高了5%，明显优于其他方法。     

A Texture Combined Multispectral Magnetic Resonance Imaging Segmentation for Nasopharyngeal Carcinoma

Tumor segmentation from magnetic resonance imaging (MRI) is important for volume estimation and visualization of nasopharyngeal carcinoma (NPC). In some cases, segmentation using the general multispectral (GM) method often obtained poor results due to the high false positives caused by complex anatomic structures and serious overlap in feature space. In this study, a texture combined multispectral fuzzy clustering (TCMFC) segmentation algorithm was proposed. A texture measure of T1-weighted (T1) MR image was introduced by calculating the two-order central statistical information of every pixel within a window after the window convolution operation. The texture measure and the intensities in T1 and contrast-enhanced T1 images formed the new 3-D feature vector for fuzzy clustering implemented by semi-supervised fuzzy c-means (SFCM). Testing showed that by reducing the false positives significantly, the TCMFC method achieved improved segmentation results, compared with the GM method.     

Fusion of infrared-visible images using improved multi-scale top-hat transform and suitable fusion rules

Integration of infrared and visible images is an active and important topic in image understanding and interpretation. In this paper, a new fusion method is proposed based on the improved multi-scale center-surround top-hat transform, which can effectively extract the feature information and detail information of source images. Firstly, the multi-scale bright (dark) feature regions of infrared and visible images are respectively extracted at different scale levels by the improved multi-scale center-surround top-hat transform. Secondly, the feature regions at the same scale in both images are combined by multi-judgment contrast fusion rule, and the final feature images are obtained by simply adding all scales of feature images together. Then, a base image is calculated by performing Gaussian fuzzy logic combination rule on two smoothed source images. Finally, the fusion image is obtained by importing the extracted bright and dark feature images into the base image with a suitable strategy. Both objective assessment and subjective vision of the experimental results indicate that the proposed method is superior to current popular MST-based methods and morphology-based methods in the field of infrared-visible images fusion.     

基于偏最小二乘算法的人脸图像超分辨率技术

提出了基于偏最小二乘法回归的超分辨率复原算法.介绍了偏最小二乘法回归算法的原理,研究和分析了基于偏最小二乘法回归的超分辨率复原算法.将高低分辨率图像块的高频信息和中频信息作为其特征,并采用分块重叠的方法解决了复原时存在的方块效应.通过对亚洲人脸和欧美人脸的实验结果表明,提出的方法无论是对亚洲人脸还是欧美人脸都能取得较好的复原效果,并且在放大倍数较大的情况下,复原的效果仍然显著.     

改进的Faster-RCNN算法在聚乙烯管接头内部缺陷检测中的应用

超声相控阵技术是目前聚乙烯管道热熔接头内部缺陷检测的一种主流方法。提出了基于注意力机制的改进Faster-RCNN目标检测网络用于超声相控阵D扫图聚乙烯管接头内部缺陷检测。针对聚乙烯管道热熔接头内部超声相控阵D扫图小缺陷较多、特征信息容易丢失的问题,将残差网络(ResNet50)与特征金字塔网络(FPN)相结合作为骨干网络,并引入卷积注意力模块(CBAM)自适应细化特征。将SSD网络框架和Faster-RCNN网络框架用于模型训练和测试,使用VGG16、ResNet50、ResNet50+FPN、ACBM+ResNet50+FPN作为骨干网络依次对超声相控阵聚乙烯管道热熔对接接头内部缺陷样本进行训练对比。结果表明,改进的Faster-RCNN网络模型在聚乙烯管接头内部缺陷检测和分类方面有明显改进,对小缺陷的检测性能有了显著的提高。     

Wavelet-based edge correlation incorporated iterative reconstruction for undersampled MRI

Undersampling k-space is an effective way to decrease acquisition time for MRI. However, aliasing artifacts introduced by undersampling may blur the edges of magnetic resonance images, which often contain important information for clinical diagnosis. Moreover, k-space data is often contaminated by the noise signals of unknown intensity. To better preserve the edge features while suppressing the aliasing artifacts and noises, we present a new wavelet-based algorithm for undersampled MRI reconstruction. The algorithm solves the image reconstruction as a standard optimization problem including a ?₂ data fidelity term and ?₁ sparsity regularization term. Rather than manually setting the regularization parameter for the ?₁ term, which is directly related to the threshold, an automatic estimated threshold adaptive to noise intensity is introduced in our proposed algorithm. In addition, a prior matrix based on edge correlation in wavelet domain is incorporated into the regularization term. Compared with nonlinear conjugate gradient descent algorithm, iterative shrinkage/thresholding algorithm, fast iterative soft-thresholding algorithm and the iterative thresholding algorithm using exponentially decreasing threshold, the proposed algorithm yields reconstructions with better edge recovery and noise suppression.