Image fusion based on nonsubsampled contourlet transform for infrared and visible light image

In this paper, a new method based on nonsubsampled contourlet transform (NSCT) is proposed to fuse the infrared image and the visible light image, which will produce a new fused image by which the target can be identified more easily. Firstly, two original images were decomposed into low frequency subband coefficients and the bandpass direction subband coefficients by using NSCT. Secondly, the selection of the low frequency subband coefficient and the bandpass direction subband coefficient is discussed in detail. The low frequency subband coefficients are selected based on the regional visual characteristics. For the selection of bandpass direction subband coefficients, this paper proposes a minimum regional cross-gradient method, and the cross-gradient is gained by calculating the gradient between the pixel of bandpass subbands and the adjacent pixel in the fused image of the low-frequency components. Comparison experiments have been performed on different image sets, and experimental results demonstrate that the proposed method performs better in both subjective and objective qualities.     

基于非采样提升小波-Contourlet变换的图像融合算法

为克服非采样Contourlet变换中金字塔分解的不足,首先在提升小波变换的基础上,通过取消其奇偶分裂环节得到具有平移不变性的非采样提升小波变换,然后用此变换来取代非采样Contourlet变换中的金字塔分解,得到新的非采样提升小波-Contourlet变换。将此变换与一定的融合规则相结合,提出了一种基于非采样提升小波-Contourlet变换的图像融合算法。实验表明,该算法相对于非采样Contourlet变换能从源图像中提取更多有用信息注入到融合图像中,可得到更高性能的融合图像。     

Fusion method of infrared and visible images based on neighborhood characteristic and regionalization in NSCT domain

On fusing infrared and visible image, the traditional fusion method cannot get the better image quality. Based on neighborhood characteristic and regionalization in NSCT (Nonsubsampled Contourlet Transform) domain, the fusion algorithm was proposed. Firstly, NSCT was adopted to decompose infrared and visible images at different scales and directions for the low and high frequency coefficients, the low frequency coefficients which were fused with improving regional weighted fusion method based on neighborhood energy, and the high-frequency coefficients were fused with multi-judgment rule based on neighborhood characteristic regional process. Finally, the coefficients were reconstructed to obtain the fused image. The experimental results show that, compared with the other three related methods, the proposed method can get the biggest value of IE (information entropy), MI(VI,F) (mutual information from visible image), MI(VI,F) (mutual information from infrared image), MI (sum of mutual information), and Q^AB/F (edge retention). The proposed method can leave enough information in the original images and its details, and the fused images have better visual effects.     

A fusion algorithm for infrared and visible images based on RDU-PCNN and ICA-bases in NSST domain

The aim of infrared and visible image fusion is to enhance the feature in infrared image and preserve abundant detail information in visible image. Based on the fact that the human sense system accepts external stimulation only when the stimulus intensity is greater than a certain value and the reaction of neuronal cells have obvious regional characters, an image fusion algorithm based on region dual-channel unit-linking pulse coupled neural networks (RDU-PCNN) and independent component analysis (ICA) bases in non-subsampled shearlet transform (NSST) domain for infrared and visible images is proposed. RDU-PCNN we constructed has obvious regional characters and much lower computational costs. We trained ICA-bases using a number of images that the content and statistical properties are similar with the fusion images but applied it as low-frequency ICA-bases, which can reduce calculation complexity. Experimental results demonstrate that the proposed method can significantly improved the fusion quality and need less computational costs.     

非下采样变换的红外与可见光图像融合

基于非下采样Contourlet变换（NSCT）,提出了一种红外和可见光图像融合算法。针对低频子带系数和各带通方向子带系数分别提出了基于图像物理特征的系数加权选择方式与基于区域能量匹配的系数选择方式,即低频基于区域梯度信息、高频基于区域特征因子的加权与选择结合的图像融合算法。实验结果表明：非下采样Contourlet变换具有较快的运算速度,且经非下采样变换后能量更加集中,可提供更多的图像信息。相对于基于像素的图像融合算法,本文的图像融合算法具有更高的融合性能,是一种更适合图像融合的多尺度几何分析（MGA）工具。     

非下采样变换的红外与可见光图像融合

基于非下采样Contourlet变换(NSCT),提出了一种红外和可见光图像融合算法。针对低频子带系数和各带通方向子带系数分别提出了基于图像物理特征的系数加权选择方式与基于区域能量匹配的系数选择方式,即低频基于区域梯度信息、高频基于区域特征因子的加权与选择结合的图像融合算法。实验结果表明:非下采样Contourlet变换具有较快的运算速度,且经非下采样变换后能量更加集中,可提供更多的图像信息。相对于基于像素的图像融合算法,本文的图像融合算法具有更高的融合性能,是一种更适合图像融合的多尺度几何分析(MGA)工具。     

An adaptive fusion approach for infrared and visible images based on NSCT and compressed sensing

A novel nonsubsampled contourlet transform (NSCT) based image fusion approach, implementing an adaptive-Gaussian (AG) fuzzy membership method, compressed sensing (CS) technique, total variation (TV) based gradient descent reconstruction algorithm, is proposed for the fusion computation of infrared and visible images.Compared with wavelet, contourlet, or any other multi-resolution analysis method, NSCT has many evident advantages, such as multi-scale, multi-direction, and translation invariance. As is known, a fuzzy set is characterized by its membership function (MF), while the commonly known Gaussian fuzzy membership degree can be introduced to establish an adaptive control of the fusion processing. The compressed sensing technique can sparsely sample the image information in a certain sampling rate, and the sparse signal can be recovered by solving a convex problem employing gradient descent based iterative algorithm(s).In the proposed fusion process, the pre-enhanced infrared image and the visible image are decomposed into low-frequency subbands and high-frequency subbands, respectively, via the NSCT method as a first step. The low-frequency coefficients are fused using the adaptive regional average energy rule; the highest-frequency coefficients are fused using the maximum absolute selection rule; the other high-frequency coefficients are sparsely sampled, fused using the adaptive-Gaussian regional standard deviation rule, and then recovered by employing the total variation based gradient descent recovery algorithm.Experimental results and human visual perception illustrate the effectiveness and advantages of the proposed fusion approach. The efficiency and robustness are also analyzed and discussed through different evaluation methods, such as the standard deviation, Shannon entropy, root-mean-square error, mutual information and edge-based similarity index.     

A novel multi-scale segmentation algorithm for high resolution remote sensing images based on wavelet transform and improved JSEG algorithm

Considering the difficulties in image segmentation caused by the complexity of diverse ecological environments and various artificial targets in high resolution remote sensing images, especially in city scene, and in order to overcome the limitations existing in the traditional segmentation algorithm, JSEG (J-Segmentation), for high resolution remote sensing image segmentation and to further improve the segmentation accuracy, WJSEG (Wavelet-JSEG), a novel multi-scale segmentation algorithm based on wavelet transform, is proposed, which is an improved JSEG algorithm. WJSEG is an improved form of JSEG in relation to three aspects, including color quantization, multi-scale segmentation and region merging by introducing the multi-scale analysis tool based on wavelet transform. Experiments have been conducted on high resolution SPOT 5 pan-sharpened multispectral image and IKONOS panchromatic image. These experimental results were compared with those gained by the traditional JSEG algorithm and the famous commercial software named eCognition, which validated the effectiveness and reliability of the proposed WJSEG algorithm.     

红外和彩色可见光图像亮度-对比度传递融合算法

以红外和彩色可见光图像为研究对象,提出了一种基于亮度-对比度传递（LCT）技术的彩色图像融合算法。首先借助灰度融合方法将红外图像与彩色可见光图像亮度分量融合,然后用LCT技术改善灰度融合结果的亮度和对比度,最后利用快速YCBCR变换融合策略在RGB空间内直接生成彩色融合图像。文中利用像素平均融合法和多分辨率融合法作为不同的灰度融合措施以分别满足高实时性和高融合质量的需求。实验结果表明,提出算法的融合结果不仅具有与输入彩色可见光图像相近的自然色彩,而且具备令人满意的亮度和对比度,即使采用运算简单的像素平均法进行灰度融合,同样可以获得良好的融合效果。     

A fusion method for visible and infrared images based on contrast pyramid with teaching learning based optimization

This paper proposes a novel image fusion scheme based on contrast pyramid (CP) with teaching learning based optimization (TLBO) for visible and infrared images under different spectrum of complicated scene. Firstly, CP decomposition is employed into every level of each original image. Then, we introduce TLBO to optimizing fusion coefficients, which will be changed under teaching phase and learner phase of TLBO, so that the weighted coefficients can be automatically adjusted according to fitness function, namely the evaluation standards of image quality. At last, obtain fusion results by the inverse transformation of CP. Compared with existing methods, experimental results show that our method is effective and the fused images are more suitable for further human visual or machine perception.     

改进的非下采样Contourlet变形红外弱小目标检测方法

针对存在复杂背景干扰和噪声的红外图像弱小目标检测问题,提出了一种非下采样Contourlet变换（NSCT）的改进的红外弱小目标检测方法。首先对含弱小目标的红外图像进行预处理,然后利用NSCT进行变换,并利用改进的非线性映射函数和能量交叉融合相结合的方法实现了背景杂波的抑制,最后引入Otsu算法进行阈值分割分离出红外弱小目标。通过与同类弱小目标检测算法的对比实验,验证了该方法的有效性。     

Fusion of infrared-visible images using improved multi-scale top-hat transform and suitable fusion rules

Integration of infrared and visible images is an active and important topic in image understanding and interpretation. In this paper, a new fusion method is proposed based on the improved multi-scale center-surround top-hat transform, which can effectively extract the feature information and detail information of source images. Firstly, the multi-scale bright (dark) feature regions of infrared and visible images are respectively extracted at different scale levels by the improved multi-scale center-surround top-hat transform. Secondly, the feature regions at the same scale in both images are combined by multi-judgment contrast fusion rule, and the final feature images are obtained by simply adding all scales of feature images together. Then, a base image is calculated by performing Gaussian fuzzy logic combination rule on two smoothed source images. Finally, the fusion image is obtained by importing the extracted bright and dark feature images into the base image with a suitable strategy. Both objective assessment and subjective vision of the experimental results indicate that the proposed method is superior to current popular MST-based methods and morphology-based methods in the field of infrared-visible images fusion.     

A novel image fusion algorithm based on nonsubsampled shearlet transform

To overcome the shortcoming of traditional image fusion method based on multi-scale transform, a novel adaptive image fusion algorithm based on nonsubsampled shearlet transform (NSST) is proposed. Firstly, the NSST is utilized to decompose the source images on various scales and in different directions, and the low frequency sub-band and bandpass sub-band coefficients are obtained. Secondly, for the low frequency sub-band coefficients, the singular value decomposition method in the gradient domain is used to estimate the local structure information of image, and an adaptive ‘weighted averaging’ fusion rule based on the sigmoid function and the extracted features is presented. To improve the quality of fused image, a novel sum-modified-Laplacian (NSML), which can extract more useful information from source images, is employed as the measurement to select bandpass sub-band coefficients. Finally, the fused image is obtained by performing the inverse NSST on the combined coefficients. The proposed fusion method is verified on several sets of multi-source images, and the experimental results show that the proposed approach can significantly outperform the conventional image fusion methods in terms of both objective evaluation criteria and visual quality.     

Multi-window visual saliency extraction for fusion of visible and infrared images

Fusion for visible and infrared images aims to combine the source images of the same scene into a single image with more feature information and better visual performance. In this paper, the authors propose a fusion method based on multi-window visual saliency extraction for visible and infrared images. To extract feature information from infrared and visible images, we design local-window-based frequency-tuned method. With this idea, visual saliency maps are calculated for variable feature information under different local window. These maps show the weights of people’s attention upon images for each pixel and region. Enhanced fusion is done using simple weight combination way. Compared with the classical and state-of-the-art approaches, the experimental results demonstrate the proposed approach runs efficiently and performs better than other methods, especially in visual performance and details enhancement.     

Region-based fusion of infrared and visible images using nonsubsampled contourlet transform

With the nonsubsampled contourlet transform (NSCT), a novel region-segmentation-based fusion algorithm for infrared (IR) and visible images is presented.The IR image is segmented according to the physical features of the target.The source images are decomposed by the NSCT, and then, different fusion rules for the target regions and the background regions are employed to merge the NSCT coefficients respectively.Finally, the fused image is obtained by applying the inverse NSCT.Experimental results show that the proposed algorithm outperforms the pixel-based methods, including the traditional wavelet-based method and NSCT-based method.     

Two-scale image fusion of visible and infrared images using saliency detection

Military, navigation and concealed weapon detection need different imaging modalities such as visible and infrared to monitor a targeted scene. These modalities provide complementary information. For better situation awareness, complementary information of these images has to be integrated into a single image. Image fusion is the process of integrating complementary source information into a composite image. In this paper, we propose a new image fusion method based on saliency detection and two-scale image decomposition. This method is beneficial because the visual saliency extraction process introduced in this paper can highlight the saliency information of source images very well. A new weight map construction process based on visual saliency is proposed. This process is able to integrate the visually significant information of source images into the fused image. In contrast to most of the multi-scale image fusion techniques, proposed technique uses only two-scale image decomposition. So it is fast and efficient. Our method is tested on several image pairs and is evaluated qualitatively by visual inspection and quantitatively using objective fusion metrics. Outcomes of the proposed method are compared with the state-of-art multi-scale fusion techniques. Results reveal that the proposed method performance is comparable or superior to the existing methods.     

基于小波变换的红外热波无损检测融合算法

在红外热波无损检测中获取的热像序列存在着背景噪声大、缺陷边缘模糊、对比度低等特点。为了提高由红外热像序列重构的数字图像的缺陷显示能力,以小波变换为热像处理工具,采用基于像素级和特征级的图像融合算法对热像序列进行了处理,并采用基于统计学的图像评估标准对处理效果作了定量评价。通过对铝合金试件的检测实验说明该方法可用于材料内部缺陷的红外热波无损检测。研究结果表明,此种图像融合算法可对不同深度缺陷所对应的两幅最佳热像进行有效地融合,在一幅融合图像中直观地反映出全部缺陷,并能有效地减少加热不均和背景噪声对缺陷识别的不利影响。     

基于小波框架的红外/可见光图像融合

针对小波框架所具有的冗余性和平移不变性在图像处理中能较好保留图像细节特征的特点,提出了一种将l2上的离散小波框架和交叉子带像素融合方案(Cross band pixel selection)结合起来的融合算法,并用于红外和可见光图像的融合。仿真试验表明该算法较好地保留了源图像的细节信息,是一种行之有效的融合算法。     

基于小波变换和区域分割的红外可见光彩色图像融合算法

提出一种基于小波变换和区域分割的YCb Cr变换域红外-可见光彩色图像融合算法,以小波变换融合为基础,将融合结果作为YCb Cr域参量,以区域分割方法为基础,与参考图像进行色彩传递。实验结果表明,采用方法比传统的线性色彩传递方法具有更好的彩色图像融合效果,同时参考图像适应性较好,适合单一图像以及视频的融合。     

基于红外和可见光图像逐级自适应融合的场景深度估计

从图像中恢复场景的深度是计算机视觉领域中的一个关键问题。考虑到单一类型图像在深度估计中受场景不同光照的限制,提出了基于红外和可见光图像逐级自适应融合的场景深度估计方法（PF-CNN）。该方法包括双流滤波器部分耦合网络、自适应多模态特征融合网络以及自适应逐级特征融合网络。在双流卷积中红外和可见光图像的滤波器部分耦合使两者特征得到增强;自适应多模态特征融合网络学习红外和可见光图像的残差特征并将两者自适应加权融合,充分利用两者的互补信息;逐级特征融合网络学习多层融合特征的结合,充分利用不同卷积层的不同特征。实验结果表明:PF-CNN在测试集上获得了较好的效果,将阈值指标提高了5%,明显优于其他方法。