Are multi-contrast magnetic resonance images necessary for segmenting multiple sclerosis brains? A large cohort study based on deep learning

BackgroundMagnetic resonance images with multiple contrasts or sequences are commonly used for segmenting brain tissues, including lesions, in multiple sclerosis (MS). However, acquisition of images with multiple contrasts increases the scan time and complexity of the analysis, possibly introducing factors that could compromise segmentation quality.ObjectiveTo investigate the effect of various combinations of multi-contrast images as input on the segmented volumes of gray (GM) and white matter (WM), cerebrospinal fluid (CSF), and lesions using a deep neural network.MethodsU-net, a fully convolutional neural network was used to automatically segment GM, WM, CSF, and lesions in 1000 MS patients. The input to the network consisted of 15 combinations of FLAIR, T1-, T2-, and proton density-weighted images. The Dice similarity coefficient (DSC) was evaluated to assess the segmentation performance. For lesions, true positive rate (TPR) and false positive rate (FPR) were also evaluated. In addition, the effect of lesion size on lesion segmentation was investigated.ResultsHighest DSC was observed for all the tissue volumes, including lesions, when the input was combination of all four image contrasts. All other input combinations that included FLAIR also provided high DSC for all tissue classes. However, the quality of lesion segmentation showed strong dependence on the input images. The DSC and TPR values for inputs with the four contrast combination and FLAIR alone were very similar, but FLAIR showed a moderately higher FPR for lesion size <100 μl. For lesions smaller than 20 μl all image combinations resulted in poor performance. The segmentation quality improved with lesion size.ConclusionsBest performance for segmented tissue volumes was obtained with all four image contrasts as the input, and comparable performance was attainable with FLAIR only as the input, albeit with a moderate increase in FPR for small lesions. This implies that acquisition of only FLAIR images provides satisfactory tissue segmentation. Lesion segmentation was poor for very small lesions and improved rapidly with lesion size.     

水下高分辨率声图中小目标的深度网络分类方法

针对声成像数据缺少条件下的水下沉底小目标分类问题,提出一种深度网络分类算法。首先,采用高斯混合模型对声影区统计特性进行建模并提取声图阴影,在此基础上构建仿真数据集和真实数据集。将仿真数据集输入卷积神经网络进行训练,保留其特征提取部分,用于对真实数据集进行特征提取.重建网络分类部分并采用真实数据集的特征向量进行训练。结果表明,所提出的方法分类正确率可达88.24%,与6种对照方法相比平均分类正确率分别提升8.67%,20.47%,19.78%,11.59%,9.01%,11.58%。验证了所提出方法在小样本条件下具有较好对水下沉底小目标的分类能力。其学习曲线收敛到96.25%,仅比验证曲线高5.14%,说明在一定程度上缓解了过拟合问题。将改进的卷积神经网络应用于融合分类器,通过与逻辑回归分类器、支持向量机对目标进行分类并融合决策,正确率为93.33%,可进一步提高算法的正确率和稳定性.      

基于U-Net的盐体识别方法

卷积神经网络在计算机视觉领域取得重大突破，利用其强大的图像处理能力，将地下沉积盐体的识别问题转化为图像语义分割问题，应用深度卷积神经网络实现盐体地震图像的像素级语义分割.本文在U-Net基础上，增加网络深度并同时引入批归一化和Dropout处理，使得神经网络模型具有更高的可信度和更强的泛化能力.通过实验发现，在卷积层之后引入批归一化处理，并在池化层和叠加层之后引入Dropout可以稳定提升模型对盐体图像的分割性能.     

Automated acoustic seabed classification of multibeam images of Stanton Banks

Dividing sidescan images into regions that have similar seabeds is often done by expert interpretation. Automated classification systems are becoming more widely used. This paper describes techniques, based on image amplitudes and texture, that lead to useful and practical automated segmentation of multibeam images. Seabed (or riverbed or lakebed) type affects amplitudes and texture, but so do system operating details and survey geometry. Effects of the last two must be compensated to isolate the effects of seabed type. Images from multibeam surveys are accompanied by bathymetric data from which grazing angles of all sonar footprints can be calculated. By compiling tables of amplitude against range and grazing angle, systematic changes in amplitude with these two variables can be removed consistently. Classification, based on a large number of features, is done in image space to avoid artifacts common in mosaics. Unsupervised segmentation requires clustering, in which records are divided into their natural classes. An objective clustering method using simulated annealing assigns points to classes based on their Bayesian distances from cluster centres. Stanton Banks is a rocky area 100 km north of County Donegal, Ireland, that rises about 100 m above the ocean floor at 180 m. Multibeam images and data from an 80-km² survey were classified into regions of acoustic similarity. Assigning labels of physical properties to these regions requires non-acoustic ground truth, which was obtained from a series of 105 photographs. Photographic geological assignments were found to correlate well with the acoustic classes.     

Neural network-based segmentation of dynamic MR mammographic images

The usefulness of neural networks for the classification of signal-time curves from dynamic MR mammography was recently demonstrated by our group. The multi-layer perceptron under study consists of 28 input, 4 hidden, and 3 output nodes, and was trained to classify signal-time curves into three tissue classes: "carcinoma," "benign lesion," and "parenchyma." Extending this approach, it was the aim of the present study to evaluate the performance of the developed network in the segmentation of dynamic MR mammographic images in comparison to a pixel-by-pixel two-compartment pharmacokinetic analysis. The population investigated in this pilot study comprised 15 women with suspicious lesions in the breast, which were confirmed histologically after the MR examination. The neural network classified the same areas as malignant as those which were marked as being highly suspicious by the pharmacokinetic mapping approach but with the advantage that no a priori knowledge on tissue microcirculation was needed, that computation proved to be much faster, and that it yielded a unique classification into just three tissue classes.     

A New Pooling Approach Based on Zeckendorf’s Theorem for Texture Transfer Information

The pooling layer is at the heart of every convolutional neural network (CNN) contributing to the invariance of data variation. This paper proposes a pooling method based on Zeckendorf’s number series. The maximum pooling layers are replaced with Z pooling layer, which capture texels from input images, convolution layers, etc. It is shown that Z pooling properties are better adapted to segmentation tasks than other pooling functions. The method was evaluated on a traditional image segmentation task and on a dense labeling task carried out with a series of deep learning architectures in which the usual maximum pooling layers were altered to use the proposed pooling mechanism. Not only does it arbitrarily increase the receptive field in a parameterless fashion but it can better tolerate rotations since the pooling layers are independent of the geometric arrangement or sizes of the image regions. Different combinations of pooling operations produce images capable of emphasizing low/high frequencies, extract ultrametric contours, etc.     

一种基于2D-CNN的激光超声表面缺陷检测方法

激光超声表面缺陷检测的过程中,缺陷的定量表征通常依赖于操作者的判断,易受到人为因素干扰,致使检测结果不稳定.针对这一问题,提出一种基于图像识别的二维卷积神经网络(2D-CNN)的缺陷自动分类检测方法.利用有限元方法模拟激光超声检测过程,并采集超声信号数据用于训练分类模型;使用连续小变换(CWT)处理超声信号得到小波时频...     

一种基于图像特征和神经网络的苹果图像分割算法

苹果识别是开发苹果采摘机器人的关键环节,利用图像处理技术和神经网络分类器探索苹果图像分割算法.从苹果树图片中选取苹果图像样本和背景网像样本.分别计算这两类图像样本的颜色特征和纹理特征.颜色特征的计算基于RGB色彩模型,纹理特征的计算基于灰度共生矩阵.选取适当的颜色特征(R/B值)和纹理特征(对比度值和相关性值)作为输入节点,利用反向传播神经网络分类器建模,输出值是一个O～1之间的计算值.通过阈值将输出结果分类为苹果或背景.试验结果表明,该算法正确率大于87.6%,对光照的影响不敏感,是一利较为实用的苹果分割算法.     

基于深度学习的无人车夜视图像语义分割

为了增强无人车对夜视图像的场景理解,在夜间模式下更快更精确地探测和识别周围环境,将深度学习应用于夜视图像的场景语义分割,提出了一种基于卷积-反卷积神经网络的无人车夜视图像语义分割方法。在传统的卷积神经网络中加入反卷积网络,构建卷积-反卷积神经网络,无需手工选取特征。通过像素到像素的学习和训练,得到图像语义分割模型,可直接用该模型预测夜视图像中每个像素所属的场景语义类别,实现无人车夜间行驶时的环境感知。实验结果表明,该方法具有较好的准确性和实时性,平均IU达到68.47。     

点云投影结合轻量化卷积神经网络实现三维成像声呐快速目标分类*

三维成像声呐的成像结果是三维点云,基于点云的三维成像声呐目标分类方法具有网络结构复杂,计算量大的特点,针对这一问题本文提出了一种将三维成像声呐成像结果从三维点云投影至二维图像的方法,并且使用轻量化卷积神经网络实现了三维成像声呐快速目标分类。该方法首先对三维成像声呐波束形成后的波束域数据进行最大值滤波和阈值滤波,降低点云数据维度;接着,依据三维成像声呐的波束方向,将点云投影为深度图和强度图,分别保存点云的位置信息和强度信息;然后,利用深度图和强度图分别作为第一个通道和第二个通道构建混合通道图,将混合通道图作为目标分类网络的输入,从而将三维点云的目标分类问题转换为二维图像的目标分类问题;最后使用MobileNetV2网络实现了三维成像声呐快速目标分类。实验结果表明,通过本文提出的投影方法可以用二维图像分类网络完成三维成像声呐点云的目标分类任务;而且混合通道图比单独的强度图和深度图收敛速度更快,结合目标识别网络可以25fps实时的进行目标分类,在真实数据集上分类精度达到了91.13%。     

残差网络分层融合的高光谱地物分类

高光谱图像具有较高的空间分辨率,蕴含着丰富的空间光谱信息,近年来被广泛用于城市地物分类中。在高光谱图像分类过程中,空间光谱特征的提取直接影响着分类精度;传统的高光谱图像特征提取方法只利用了4或8邻域的像素进行简单卷积处理,因而丢失了大量的复杂、有效信息;卷积神经网络（CNN）虽然可以自动提取空间光谱特征,在保留图像空间信息的同时,简化网络模型,但是,随着网络深度增加,网络分类产生退化现象,而且网络间缺乏相关信息的互补性,从而影响分类精度。该工作引入CNN自动提取空间光谱特征,并且针对CNN深度增加所导致的退化问题,设计了面向地物分类的高光谱特征融合残差网络。首先,为了降低高光谱图像的光谱冗余度,利用PCA提取主要光谱波段;然后,为了逐级提取光谱图像的空间光谱特征,定义了卷积核为16,32,64的低、中、高3层残差网络模块,并利用64个1×1的卷积核对3层特征输出进行卷积,完成维度匹配与特征图融合;接着,对融合后的特征图进行全局平均池化(GAP)生成用于分类的特征向量;最后,引入具有可调节机制的Large-Margin Softmax损失函数,监督模型完成训练过程,实现高光谱图像分类。实验采用Indian Pines,University of Pavia和Salinas地区的高光谱图像来验证方法有效性,设置批次训练的样本集为100,网络训练的初始学习率为0.1,当损失函数稳定后学习率降低为0.001,动量为0.9,权重延迟为0.000 1,最大训练迭代次数为2×104,当3个数据集的样本块像素分别设置为25×25,23×23,27×27,网络深度分别为28,32和28时,3个数据集的分类准确率最高,其平均总体准确率（OA）为98.75%、平均准确率（AA）的评价值为98.1%,平均Kappa系数为0.98。实验结果表明,基于残差网络的分类方法能够自动学习更丰富的空间光谱特征,残差网络层数的增加和不同网络层融合可以提高高光谱分类精度;Large-Margin Softmax实现了类内紧凑和类间分离,可以进一步提高高光谱图像分类精度。     

不均匀光照的路面裂缝检测和分类新方法

针对受光照不均影响的路面裂缝图像,提出一种基于Sobel算子和最大熵法的图像分割算法,并采用长线段与原图进行与操作和判断黑色像素所占比例的方法去除图像孤立噪声点.根据不同类型裂缝的几何形态,提取投影向量、分布密度和空洞数等特征值作为路面裂缝分类的依据,设计径向基函数神经网络的分类器实现对裂缝的准确分类.实验结果表明,较传统全局阈值算法,本文算法对光照不均图像的处理不仅能很好的提取裂缝边缘,且具有很强的抗噪能力,对路面裂缝的分类准确率高.     

结合小波融合和深度学习的脑胶质瘤自动分割

针对水肿区域边界模糊和瘤内结构复杂多变导致的脑胶质瘤分割不精确问题,本文提出了一种基于小波融合和3D-UNet网络的脑胶质瘤磁共振图像自动分割算法.首先,对脑胶质瘤磁共振图像的T1、T1ce、T2、Flair四种模态进行小波融合以及偏置场校正;然后,提取待分类的图像块;再利用提取的图像块训练3D-UNet网络以对图像块中的像素进行分类;最后加载损失率较小的网络模型进行分割,并采用基于连通区域的轮廓提取方法,以降低假阳性率.对57组Brats2018（Brain Tumor Segmentation 2018）磁共振图像测试集进行分割的结果显示,肿瘤的整体、核心和水肿部分的平均分割准确率（DSC）分别达到90.64%、80.74%和86.37%,这表明该算法分割脑胶质瘤准确率较高,与金标准相近.相比多模态图像融合前,该算法在减少输入网络数据量和图像冗余信息的同时,还一定程度上解决了胶质瘤边界模糊、分割不精确的问题,提高了分割的准确度和鲁棒性.     

Secret Communication Systems Using Chaotic Wave Equations with Neural Network Boundary Conditions

In a secret communication system using chaotic synchronization, the communication information is embedded in a signal that behaves as chaos and is sent to the receiver to retrieve the information. In a previous study, a chaotic synchronous system was developed by integrating the wave equation with the van der Pol boundary condition, of which the number of the parameters are only three, which is not enough for security. In this study, we replace the nonlinear boundary condition with an artificial neural network, thereby making the transmitted information difficult to leak. The neural network is divided into two parts; the first half is used as the left boundary condition of the wave equation and the second half is used as that on the right boundary, thus replacing the original nonlinear boundary condition. We also show the results for both monochrome and color images and evaluate the security performance. In particular, it is shown that the encrypted images are almost identical regardless of the input images. The learning performance of the neural network is also investigated. The calculated Lyapunov exponent shows that the learned neural network causes some chaotic vibration effect. The information in the original image is completely invisible when viewed through the image obtained after being concealed by the proposed system. Some security tests are also performed. The proposed method is designed in such a way that the transmitted images are encrypted into almost identical images of waves, thereby preventing the retrieval of information from the original image. The numerical results show that the encrypted images are certainly almost identical, which supports the security of the proposed method. Some security tests are also performed. The proposed method is designed in such a way that the transmitted images are encrypted into almost identical images of waves, thereby preventing the retrieval of information from the original image. The numerical results show that the encrypted images are certainly almost identical, which supports the security of the proposed method.     

Snake-based method for the segmentation of objects in multichannel images degraded by speckle

We recently proposed a new approach for the segmentation of speckled images based on active contours (snakes) [e.g., Opt. Commun. 137, 382 (1997)]. We propose an extension of this approach to multichannel data. Two solutions are compared based on hypotheses on the possible mean intensity variation between the channels. Each solution is optimal for a certain class of input images, but one solution shows better or equivalent performance for both input image classes. This result opens new perspectives for the segmentation of multichannel images with the snake-based approach.     

基于时空双流卷积神经网络的红外行为识别

针对红外视频人体行为识别问题,提出了一种基于时空双流卷积神经网络的红外人体行为识别方法。通过将整个红外视频进行平均分段,然后将每一段视频中随机抽取的红外图像和对应的光流图像输入空间卷积神经网络,空间卷积神经网络通过融合光流信息可以有效地学习到红外图像中真正发生运动的空间信息,再将每一小段的识别结果进行融合得到空间网络结果。同时将每一段视频中随机抽取的光流图像序列输入时间卷积神经网络,融合每一小段的结果后得到时间网络结果。最后再将空间网络结果和时间网络结果进行加权求和,从而得到最终的视频分类结果。实验中,采用此方法对包含23种红外行为动作类别的红外视频数据集上的动作进行识别,正确识别率为92.0%。结果表明,该算法可以有效地对红外视频行为进行准确识别。     

Electromagnetic Modulation Signal Classification Using Dual-Modal Feature Fusion CNN

AMC (automatic modulation classification) plays a vital role in spectrum monitoring and electromagnetic abnormal signal detection. Up to now, few studies have focused on the complementarity between features of different modalities and the importance of the feature fusion mechanism in the AMC method. This paper proposes a dual-modal feature fusion convolutional neural network (DMFF-CNN) for AMC to use the complementarity between different modal features fully. DMFF-CNN uses the gram angular field (GAF) image coding and intelligence quotient (IQ) data combined with CNN. Firstly, the original signal is converted into images by GAF, and the GAF images are used as the input of ResNet50. Secondly, it is converted into IQ data and as the complex value network (CV-CNN) input to extract features. Furthermore, a dual-modal feature fusion mechanism (DMFF) is proposed to fuse the dual-modal features extracted by GAF-ResNet50 and CV-CNN. The fusion feature is used as the input of DMFF-CNN for model training to achieve AMC of multi-type signals. In the evaluation stage, the advantages of the DMFF mechanism proposed in this paper and the accuracy improvement compared with other feature fusion algorithms are discussed. The experiment shows that our method performs better than others, including some state-of-the-art methods, and has superior robustness at a low signal-to-noise ratio (SNR), and the average classification accuracy of the dataset signals reaches 92.1%. The DMFF-CNN proposed in this paper provides a new path for the AMC field.     

时频谱图和数据增强的水声信号深度学习目标识别方法

水声目标识别一直是水声领域研究的重点问题之一,深度学习方法可以有效地解决目标识别问题,然而,水声样本的稀少限制了该方法的应用.该文提出一种基于数据增强的水声信号深度学习目标识别方法,该方法以Mel功率谱作为网络的输入特征,通过对原始信号在时域和时频域的拉伸和掩蔽等变换,实现数据扩展和增加泛化性能的目的,最后,利用改进的...