Pulse Coupled Neural Network-Based Multimodal Medical Image Fusion via Guided Filtering and WSEML in NSCT Domain

Multimodal medical image fusion aims to fuse images with complementary multisource information. In this paper, we propose a novel multimodal medical image fusion method using pulse coupled neural network (PCNN) and a weighted sum of eight-neighborhood-based modified Laplacian (WSEML) integrating guided image filtering (GIF) in non-subsampled contourlet transform (NSCT) domain. Firstly, the source images are decomposed by NSCT, several low- and high-frequency sub-bands are generated. Secondly, the PCNN-based fusion rule is used to process the low-frequency components, and the GIF-WSEML fusion model is used to process the high-frequency components. Finally, the fused image is obtained by integrating the fused low- and high-frequency sub-bands. The experimental results demonstrate that the proposed method can achieve better performance in terms of multimodal medical image fusion. The proposed algorithm also has obvious advantages in objective evaluation indexes VIFF, Q_W, API, SD, EN and time consumption.     

基于变换域PCNN的近红外与彩色可见光融合

针对近红外与彩色可见光图像融合后对比度低、细节丢失和颜色失真等问题,提出一种基于多尺度变换和自适应脉冲耦合神经网络(PCNN-pulse coupled neural network,PCNN)的红外与彩色可见光图像融合的新算法。首先将彩色可见光图像转换到HSI（hue saturation intensity)空间,HSI色彩空间包含亮度、色度和饱和度三个分量,并且这三个分量互不相关,因此利用这个特点可对三个分量分别进行处理。将其亮度分量与近红外图像分别进行多尺度变换,变换方法选择Tetrolet变换。变换后分别得到低频和高频分量,针对图像低频分量,提出一种期望最大的低频分量融合规则;针对图像高频分量,采用高斯差分算子调节PCNN模型的阈值,提出一种自适应的PCNN模型作为融合规则。处理后的高低频分量经过Tetrolet逆变换得到的融合图像作为新的亮度图像。然后将新的亮度图像和原始的色度和饱和度分量反向映射到RGB空间,得到融合后的彩色图像。为了解决融合带来的图像平滑化和原始图像光照不均的问题,引入颜色与锐度校正机制(colour and sharpness correction, CSC)来提高融合图像的质量。为了验证方法的有效性,选取了5组分辨率为1 024×680近红外与彩色可见光图像进行试验,并与当前高效的四种融合方法以及未进行颜色校正的本方法进行了对比。实验结果表明,同其他图像融合算法进行对比分析,该方法在有无CSC颜色的情况下均能保留最多的细节和纹理,可见度均大大提高,同时本方法的结果在光照条件较弱的情况下具有更多的细节和纹理,均具有更好的对比度和良好的色彩再现性。在信息保留度、颜色恢复、图像对比度和结构相似性等客观指标上均具有较大优势。     

Multi-focus image fusion based on robust principal component analysis and pulse-coupled neural network

Multi-focus image fusion combines multiple source images with different focus points into one image, so that the resulting image appears all in-focus. In order to improve the accuracy of focused region detection and fusion quality, a novel multi-focus image fusion scheme based on robust principal component analysis (RPCA) and pulse-coupled neural network (PCNN) is proposed. In this method, registered source images are decomposed into principal component matrices and sparse matrices with RPCA decomposition. The local sparse features computed from the sparse matrix construct a composite feature space to represent the important information from the source images, which become inputs to PCNN to motivate the PCNN neurons. The focused regions of the source images are detected by the firing maps of PCNN and are integrated to construct the final, fused image. Experimental results demonstrate that the superiority of the proposed scheme over existing methods and highlight the expediency and suitability of the proposed method.     

A medical image fusion method based on energy classification of BEMD components

A medical image fusion method based on bi-dimensional empirical mode decomposition (BEMD) and dual-channel PCNN is proposed in this paper. The multi-modality medical images are decomposed into intrinsic mode function (IMF) components and a residue component. IMF components are divided into high-frequency and low-frequency components based on the component energy. Fusion coefficients are achieved by the following fusion rule: high frequency components and the residue component are superimposed to get more textures; low frequency components contain more details of the source image which are input into dual-channel PCNN to select fusion coefficients, the fused medical image is achieved by inverse transformation of BEMD. BEMD is a self-adaptive tool for analyzing nonlinear and non-stationary data; it doesn’t need to predefine filter or basis function. Dual-channel PCNN reduces the computational complexity and has a good ability in selecting fusion coefficients. A combined application of BEMD and dual-channel PCNN can extract the details of the image information more effectively. The experimental result shows the proposed algorithm gets better fusion result and has more advantages comparing with traditional fusion algorithms.     

结合深度学习的非下采样剪切波遥感图像融合

遥感图像融合是指将不同传感器得到的具有不同观测特性的图像信息有选择、有策略地结合起来,以得到具有更优观测特性的新图像的方法。提出一种深度学习结合非下采样剪切波变换(NSST)的遥感图像融合算法,利用改进的超分辨率重建网络对多光谱图像(MS)进行空间分辨率增强,全色图像(PAN)参考重建后的多光谱图像的每个分量进行直方图匹配。将对应通道的图像进行NSST变换,分别得到低频子带和若干高频子带。低频子带通过使用基于梯度域的自适应加权平均规则来获得低频融合系数,高频子带采用局部空间频率最大值规则来获得高频融合系数,最后经逆NSST变换重构获得融合图像。对不同数据集中的City和Inland多光谱图像采用双三次插值方法进行上采样,作者提出算法的通用图像质量指数(UIQI)分别为0.988 6和0.932 1,光谱角映射(SAM)分别为1.872 1和2.143 2。实验结果表明,图像结构更加清晰,保存的光谱信息更加完整,融合图像质量优于对比算法,融合图像更利于人类视觉观察。     

Multi-focus image fusion algorithm based on compound PCNN in Surfacelet domain

In order to effectively retain details and suppress noise, a multi-focus image fusion method based on Surfacelet transform and compound PCNN is proposed. Surfacelet transform is a powerful multi-resolution analysis tool which is able to decompose the original image into a number of different frequency band sub-images, compound PCNN model is a combined model of PCNN and dual-channel PCNN which is to select the fusion coefficients from the decomposed coefficients, the Local sum-modified-Laplacian (LSML) is selected as external stimulus of compound PCNN, fusion coefficients are decided by compound PCNN. The experimental results show that the new method has a good performance, fusion image has more texture details and it is more similar to the original images, the objective evaluation indexes show that this method is superior to the traditional image fusion methods.     

Fusion of visible and infrared images using saliency analysis and detail preserving based image decomposition

Image fusion for visible and infrared images is a significant task in image analysis. The target regions in infrared image and abundant detail information in visible image should be both extracted into the fused result. Thus, one should preserve or even enhance the details from original images in fusion process. In this paper, an algorithm using pixel value based saliency detection and detail preserving based image decomposition is proposed. Firstly, the multi-scale decomposition is constructed using weighted least squares filter for original infrared and visible images. Secondly, the pixel value based saliency map is designed and utilized for image fusion in different decomposition level. Finally, the fusion result is reconstructed by synthesizing different scales with synthetic weights. Since the information of original signals can be well preserved and enhanced with saliency extraction and multi scale decomposition process, the fusion algorithm performs robustly and excellently. The proposed approach is compared with other state-of the-art methods on several image sets to verify the effectiveness and robustness.     

一种基于NSCT变换的红外与可见光图像融合算法

针对红外与可见光图像融合,提出了一种基于NSCT变换的图像融合方法。对经NSCT变换的低频子带系数采用基于区域能量自适应加权的融合规则,对高频子带系数采用混合的融合方法,即对于低层,采用基于区域方差选大的融合方法,对于高层采用像素点的绝对值选大的融合方法。实验结果表明,该融合算法可以获得更多的细节信息,能获得较理想的融合图像。     

A multi-focus image fusion algorithm based on an improved dual-channel PCNN in NSCT domain

This paper presents a multi-focus image fusion algorithm based on dual-channel PCNN in NSCT domain. The fusion algorithm based on multi-scale transform is likely to produce the pseudo-Gibbs effects and it is not effective to fuse the dim or partial bright images. To solve these problems, this algorithm will get a number of different frequency sub-image of the two images by using the NSCT transform, the selection principles of different subband coefficients obtained by the NSCT decomposition are discussed in detail, and the images are fused based on the improved dual-channel PCNN in order to determine the band-pass sub-band coefficient, at last fused image is obtained by using the inverse NSCT transform. Fusion rules based on dual-channel PCNN are used to solve the complexity of the PCNN parameter settings and long computing time problems. The experimental results show that the algorithm has overcome the defects of the traditional multi-focus image fusion algorithm and improved the fusion effect.     

A novel image fusion algorithm based on nonsubsampled shearlet transform

To overcome the shortcoming of traditional image fusion method based on multi-scale transform, a novel adaptive image fusion algorithm based on nonsubsampled shearlet transform (NSST) is proposed. Firstly, the NSST is utilized to decompose the source images on various scales and in different directions, and the low frequency sub-band and bandpass sub-band coefficients are obtained. Secondly, for the low frequency sub-band coefficients, the singular value decomposition method in the gradient domain is used to estimate the local structure information of image, and an adaptive ‘weighted averaging’ fusion rule based on the sigmoid function and the extracted features is presented. To improve the quality of fused image, a novel sum-modified-Laplacian (NSML), which can extract more useful information from source images, is employed as the measurement to select bandpass sub-band coefficients. Finally, the fused image is obtained by performing the inverse NSST on the combined coefficients. The proposed fusion method is verified on several sets of multi-source images, and the experimental results show that the proposed approach can significantly outperform the conventional image fusion methods in terms of both objective evaluation criteria and visual quality.     

Multi-Focus Image Fusion Method Based on Multi-Scale Decomposition of Information Complementary

Multi-focus image fusion is an important method used to combine the focused parts from source multi-focus images into a single full-focus image. Currently, to address the problem of multi-focus image fusion, the key is on how to accurately detect the focus regions, especially when the source images captured by cameras produce anisotropic blur and unregistration. This paper proposes a new multi-focus image fusion method based on the multi-scale decomposition of complementary information. Firstly, this method uses two groups of large-scale and small-scale decomposition schemes that are structurally complementary, to perform two-scale double-layer singular value decomposition of the image separately and obtain low-frequency and high-frequency components. Then, the low-frequency components are fused by a rule that integrates image local energy with edge energy. The high-frequency components are fused by the parameter-adaptive pulse-coupled neural network model (PA-PCNN), and according to the feature information contained in each decomposition layer of the high-frequency components, different detailed features are selected as the external stimulus input of the PA-PCNN. Finally, according to the two-scale decomposition of the source image that is structure complementary, and the fusion of high and low frequency components, two initial decision maps with complementary information are obtained. By refining the initial decision graph, the final fusion decision map is obtained to complete the image fusion. In addition, the proposed method is compared with 10 state-of-the-art approaches to verify its effectiveness. The experimental results show that the proposed method can more accurately distinguish the focused and non-focused areas in the case of image pre-registration and unregistration, and the subjective and objective evaluation indicators are slightly better than those of the existing methods.     

基于支持度变换和top-hat分解的双色中波红外图像融合

为了解决用多尺度top-hat分解法融合双色中波红外图像时经常存在对比度提升有限、边缘区域失真较重的问题,提出了基于支持度变换和top-hat分解相结合的融合方法。先用支持度变换法将双色中波图像分解为低频图像和支持度图像序列;再从最后一层低频图像中用多尺度top-hat分解法提取各自的亮信息和暗信息;用灰度值取大法分别融合亮信息和暗信息;通过灰度值归一化和高斯滤波分别增强亮、暗信息融合图像;然后融合两低频图像和亮、暗信息增强图像;将融合图像作为新的低频图像和用灰度值取大法融合得到的支持度融合图像序列进行支持度逆变换,得到最终融合图像。该方法的实验结果同采用单一的支持度变换法融合和多尺度top-hat分解法融合相比,融合图像的对比度提升了11.69%,失真度降低了63.42%,局部粗糙度提高了38.12%。说明提出的从低频图像提取亮暗信息,并经过分别融合、增强,再与低频图像进行融合,能有效破解红外融合图像对比度提升和边缘区域失真度降低之间的矛盾,为提高图像融合质量提供了新方法。     

基于区域分割和Counterlet变换的图像融合算法

提出了一种基于区域分割和Contourlet变换的图像融合算法。首先,对各源图像做区域分割,并利用区域能量比和区域清晰比的概念来度量和提取区域信息;然后,对各源图像进行多尺度非子采样Contourlet分解,分解后的高频部分采用绝对值取大算子进行融合,低频部分则采用基于区域的融合规则和算子进行融合;最后进行重构得到融合图像。对红外与可见光图像进行了融合实验,并与基于像素的àtrous小波变换和Contourlet变换的融合效果进行了比较。结果表明,采用本文算法的融合图像既保留了可见光图像的光谱信息,又继承了红外图像的目标信息,其熵值高于基于像素的融合方法约10%,交叉熵仅为基于像素的融合方法的1%左右。     

Fusion method of infrared and visible images based on neighborhood characteristic and regionalization in NSCT domain

On fusing infrared and visible image, the traditional fusion method cannot get the better image quality. Based on neighborhood characteristic and regionalization in NSCT (Nonsubsampled Contourlet Transform) domain, the fusion algorithm was proposed. Firstly, NSCT was adopted to decompose infrared and visible images at different scales and directions for the low and high frequency coefficients, the low frequency coefficients which were fused with improving regional weighted fusion method based on neighborhood energy, and the high-frequency coefficients were fused with multi-judgment rule based on neighborhood characteristic regional process. Finally, the coefficients were reconstructed to obtain the fused image. The experimental results show that, compared with the other three related methods, the proposed method can get the biggest value of IE (information entropy), MI(VI,F) (mutual information from visible image), MI(VI,F) (mutual information from infrared image), MI (sum of mutual information), and Q^AB/F (edge retention). The proposed method can leave enough information in the original images and its details, and the fused images have better visual effects.     

基于潜在低秩表示与复合滤波的红外与弱可见光增强图像融合方法

针对传统红外与弱可见光图像融合算法中存在的亮度与对比度低、细节轮廓信息缺失、可视性差等问题,提出一种基于潜在低秩表示与复合滤波的红外与弱可见光增强图像融合方法.该方法首先利用改进的高动态范围压缩增强方法增强可见光图像提高亮度;然后利用基于潜在低秩表示与复合滤波的分解方法分别对红外与增强后的弱可见光图像进行分解,得到相应的低频和高频层;再分别使用改进的对比度增强视觉显著图融合方法与改进的加权最小二乘优化融合方法对得到的低频和高频层进行融合;最后将得到的低频和高频融合层进行线性叠加得到最终的融合图像.与其他方法的对比实验结果表明,用该方法得到的融合图像细节信息丰富,清晰度高,具有良好的可视性.     

基于区域分维和非采样Contourlet变换的图像融合算法

针对源图像有用信息的提取,提出了基于区域分维和非下采样Contourlet变换相结合的红外与可见光图像融合算法.将图像的区域属性、区域大小、边缘强度以及纹理显著程度等特点用图像不同尺度上的区域分维进行描述,对于非下采样Contourlet变换低频系数,根据源图像不同尺度上的区域分维进行基于系数选择的融合.针对带通子带系数设计了系数局部匹配度算子,依据匹配度不同采用加权和系数选取相结合的融合规则.与其他常规融合方法进行比较,该算法可有效实现红外与可见光图像的融合.     

基于二代curvelet变换的图像融合研究

曲波(Curvelet)作为一种新的多尺度分析方法比小波更加适合分析二维图像中的曲线或直线状边缘特征,而且具有更高的逼近精度和更好的稀疏表达能力.将curvelet变换引入图像融合,能够更好地提取原始图像的特征,为融合图像提供更多的信息.第二代curvelet理论的提出也使得其理论更易理解和实现.因此,提出了一种基于第二代curvelet变换的图像融合方法,首先将图像进行curvelet变换,然后在相应尺度上利用融合规则将curvelet系数融合,最后进行重构得到融合结果.对多聚焦图像进行了实验,采用均方误差、偏差指数和相关系数对融合结果进行了客观评价,并与基于小波变换的融合进行了比较,实验结果表明该方法除分解2层时与小波性能相当,取其他分解层数时均获得更好的融合效果.     

Multifocus image fusion scheme based on features of multiscale products and PCNN in lifting stationary wavelet domain

Multifocus image fusion aims at overcoming imaging cameras's finite depth of field by combining information from multiple images with the same scene. For the fusion problem of the multifocus image of the same scene, a novel algorithm is proposed based on multiscale products of the lifting stationary wavelet transform (LSWT) and the improved pulse coupled neural network (PCNN), where the linking strength of each neuron can be chosen adaptively. In order to select the coefficients of the fused image properly with the source multifocus images in a noisy environment, the selection principles of the low frequency subband coefficients and bandpass subband coefficients are discussed, respectively. For choosing the low frequency subband coefficients, a new sum modified-Laplacian (NSML) of the low frequency subband, which can effectively represent the salient features and sharp boundaries of the image in the LSWT domain, is an input to motivate the PCNN neurons; when choosing the high frequency subband coefficients, a novel local neighborhood sum of Laplacian of multiscale products is developed and taken as one type of feature of high frequency to motivate the PCNN neurons. The coefficients in the LSWT domain with large firing times are selected as coefficients of the fused image. Experimental results demonstrate that the proposed fusion approach outperforms the traditional discrete wavelet transform (DWT)-based, LSWT-based and LSWT-PCNN-based image fusion methods even though the source image is in a noisy environment in terms of both visual quality and objective evaluation.     

基于嵌入式多尺度分解和可能性理论的多波段纹理图像融合

将多尺度变换和“高频取大、低频加权平均”融合规则相结合是融合双波段图像的有效方法。但用该类方法融合多波段图像时,序贯式加权常常会导致原图像间固有的差异信息在融合图像中被弱化,从而影响后续的目标识别和场景理解。该问题在融合具有纹理特征的多波段图像时更为突出。为此,提出了一个基于嵌入式多尺度分解和可能性理论的多波段纹理图像融合新方法。首先,利用一种多尺度变换方法把多波段原图像分别分解为高频和低频成分,并对多波段图像中标准差最大的一幅原图像的低频成分利用另一种多尺度方法进行分块,再以该分块图像的大小和位置为标准对其余波段的原图像进行分块。然后,基于可能性理论的相关融合规则逐一融合对应的多波段块图像,再把块融合图像进行拼接,以拼接结果作为低频融合图像。最后,将该低频融合图像和利用取大规则融合得到的高频成分一起通过多尺度逆变换获得最终的融合图像。这种方法不仅将像素级和特征级融合方法综合在一起, 而且将空间域和变换域技术综合在一起, 并通过对大小块采用不同融合规则解决了目标边缘的锯齿效应问题。实验表明该方法效果显著。     

基于Shearlet变换的自适应图像融合算法

针对多聚焦图像与多光谱和全色图像的成像特点,结合Shearlet变换具有较好的稀疏表示图像特征的性质,提出了一种新的图像融合规则.并基于此融合规则,提出了基于Shearlet变换的自适应图像融合算法.在多聚焦图像的融合算法中,分别对聚焦不同的图像进行Shearlet变换,并基于本文提出的融合规则,对分解后的高低频系数进行融合处理. 通过与多种算法的比较实验证明了本文提出的算法融合的图像具有更高的清晰度和更加丰富的细节信息.在多光谱和全色图像的融合处理中,提出了一种基于Shearlet变换与HSV变换相结合的图像融合方法.该算法首先对多光谱图像作HSV变换,将得到的V分量与全色图像进行Shearlet分解与融合,在融合过程中对分解系数选用特定的融合准则进行融合,最后将融合生成新的分量与H、S分量进行HSV逆变换产生新的RGB融合图像. 该算法在空间分辨率和光谱特性两方面达到了良好的平衡,融合后的图像在减少光谱失真的同时,有效增强了空间分辨率. 仿真实验证明,本文算法融合的图像与传统的多光谱和全色图像融合算法相比,具有更佳的融合性能和视觉效果.