Two-scale image fusion of visible and infrared images using saliency detection

Military, navigation and concealed weapon detection need different imaging modalities such as visible and infrared to monitor a targeted scene. These modalities provide complementary information. For better situation awareness, complementary information of these images has to be integrated into a single image. Image fusion is the process of integrating complementary source information into a composite image. In this paper, we propose a new image fusion method based on saliency detection and two-scale image decomposition. This method is beneficial because the visual saliency extraction process introduced in this paper can highlight the saliency information of source images very well. A new weight map construction process based on visual saliency is proposed. This process is able to integrate the visually significant information of source images into the fused image. In contrast to most of the multi-scale image fusion techniques, proposed technique uses only two-scale image decomposition. So it is fast and efficient. Our method is tested on several image pairs and is evaluated qualitatively by visual inspection and quantitatively using objective fusion metrics. Outcomes of the proposed method are compared with the state-of-art multi-scale fusion techniques. Results reveal that the proposed method performance is comparable or superior to the existing methods.     

Fusion for visible and infrared images using visual weight analysis and bilateral filter-based multi scale decomposition

Fusion for visible and infrared images has been an important and challenging work in image analysis. Both the feature information in infrared image and abundant detail information in visible image should be preserved and enhanced in fused result. In this paper, a detail enhanced fusion algorithm through visual weight analysis based on smooth-inspired multi scale decomposition is proposed. With variable parameter, bilateral filter-based idea successfully decomposes the two source image into several scales. At each scale level, visual weight map is calculated and used for fusion. Finally, those levels are synthetized with proper weights. Using this idea, the detail information could be enhanced easily. The experimental results demonstrate the proposed approach performs better than other methods, especially in visual effect and keeping details.     

基于小波框架的红外/可见光图像融合

针对小波框架所具有的冗余性和平移不变性在图像处理中能较好保留图像细节特征的特点,提出了一种将l2上的离散小波框架和交叉子带像素融合方案(Cross band pixel selection)结合起来的融合算法,并用于红外和可见光图像的融合。仿真试验表明该算法较好地保留了源图像的细节信息,是一种行之有效的融合算法。     

Multi-window visual saliency extraction for fusion of visible and infrared images

Fusion for visible and infrared images aims to combine the source images of the same scene into a single image with more feature information and better visual performance. In this paper, the authors propose a fusion method based on multi-window visual saliency extraction for visible and infrared images. To extract feature information from infrared and visible images, we design local-window-based frequency-tuned method. With this idea, visual saliency maps are calculated for variable feature information under different local window. These maps show the weights of people’s attention upon images for each pixel and region. Enhanced fusion is done using simple weight combination way. Compared with the classical and state-of-the-art approaches, the experimental results demonstrate the proposed approach runs efficiently and performs better than other methods, especially in visual performance and details enhancement.     

有限离散剪切波域的红外可见光图像融合

针对目前图像融合过程中的不足之处,结合有限离散剪切波具有高的方向敏感性和抛物尺度化特性,提出了一种有限离散剪切波变换下的图像融合算法。首先对严格配准的多传感器图像进行有限离散剪切波变换,得到低频子带系数和不同尺度不同方向的高频子带系数;然后对低频子带系数采用全局特征值和像素点之间的差异性与区域空间频率匹配度相结合的融合算法,高频方向子带系数采用方向权重对比度与相对区域平均梯度和相对区域方差相结合的方案;最后通过有限离散剪切波逆变换得到融合图像。实验结果表明,与其他的融合算法相比较,本文算法不但有良好的主观视觉效果,而且3幅图像的客观评价指标分别平均提高了0.9%、3.8%、3.1%,2.6%、3.8%、2.9%和1.5%、125%、59%,充分说明了本文融合算法的优越性。     

Fusion method for infrared and visible images by using non-negative sparse representation

In this paper, an interesting fusion method, named as NNSP, is developed for infrared and visible image fusion, where non-negative sparse representation is used to extract the features of source images. The characteristics of non-negative sparse representation coefficients are described according to their activity levels and sparseness levels. Multiple methods are developed to detect the salient features of the source images, which include the target and contour features in the infrared images and the texture features in the visible images. The regional consistency rule is proposed to obtain the fusion guide vector for determining the fused image automatically, where the features of the source images are seamlessly integrated into the fused image. Compared with the classical and state-of-the-art methods, our experimental results have indicated that our NNSP method has better fusion performance in both noiseless and noisy situations.     

Fusion of visible and infrared images using global entropy and gradient constrained regularization

Infrared and visible image fusion has been an important and popular topic in imaging science. Dual-band image fusion aims to extract both target regions in infrared image and abundant detail information in visible image into fused result, preserving even enhancing the information that inherits from source images. In our study, we propose an optimization-based fusion method by combining global entropy and gradient constrained regularization. We design a cost function by taking the advantages of global maximum entropy as the first term, together with gradient constraint as the regularized term. In this cost function, global maximum entropy could make the fused result inherit as more information as possible from sources. And using gradient constraint, the fused result would have clear details and edges with noise suppression. The fusion is achieved based on the minimization of the cost function by adding weight value matrix. Experimental results indicate that the proposed method performs well and has obvious superiorities over other typical algorithms in both subjective visual performance and objective criteria.     

Infrared image enhancement through saliency feature analysis based on multi-scale decomposition

To improve contrast between dim target region and background in infrared (IR) long-range surveillance, this paper proposes a fast image enhancement approach using saliency feature extraction based on multi-scale decomposition. Firstly, a smooth based multi-scale decomposition is designed and applied to original infrared image, generating sub-images with various frequency components at different decomposition levels. The dim target regions of sub-images are extracted by a local frequency-tuned based saliency feature detection method, secondly. With saliency maps created by saliency extraction using multi-scale local windows with different sizes, the sub-images are enhanced at different decomposition scales. Finally, the enhanced result is reconstructed by synthesizing the all sub-images with adjustable synthetic weights. Since salient areas are analyzed based on fast multi-scale image decomposition, IR image can be s enhanced with good contrast successfully and rapidly. Compared with other algorithms, the experimental results prove that the proposed method is robust and efficient for IR image enhancement.     

Classification of visible and infrared hyperspectral images based on image segmentation and edge-preserving filtering

The classification of hyperspectral images with a few labeled samples is a major challenge which is difficult to meet unless some spatial characteristics can be exploited. In this study, we proposed a novel spectral-spatial hyperspectral image classification method that exploited spatial autocorrelation of hyperspectral images. First, image segmentation is performed on the hyperspectral image to assign each pixel to a homogeneous region. Second, the visible and infrared bands of hyperspectral image are partitioned into multiple subsets of adjacent bands, and each subset is merged into one band. Recursive edge-preserving filtering is performed on each merged band which utilizes the spectral information of neighborhood pixels. Third, the resulting spectral and spatial feature band set is classified using the SVM classifier. Finally, bilateral filtering is performed to remove “salt-and-pepper” noise in the classification result. To preserve the spatial structure of hyperspectral image, edge-preserving filtering is applied independently before and after the classification process. Experimental results on different hyperspectral images prove that the proposed spectral-spatial classification approach is robust and offers more classification accuracy than state-of-the-art methods when the number of labeled samples is small.     

Fusion method of infrared and visible images based on neighborhood characteristic and regionalization in NSCT domain

On fusing infrared and visible image, the traditional fusion method cannot get the better image quality. Based on neighborhood characteristic and regionalization in NSCT (Nonsubsampled Contourlet Transform) domain, the fusion algorithm was proposed. Firstly, NSCT was adopted to decompose infrared and visible images at different scales and directions for the low and high frequency coefficients, the low frequency coefficients which were fused with improving regional weighted fusion method based on neighborhood energy, and the high-frequency coefficients were fused with multi-judgment rule based on neighborhood characteristic regional process. Finally, the coefficients were reconstructed to obtain the fused image. The experimental results show that, compared with the other three related methods, the proposed method can get the biggest value of IE (information entropy), MI(VI,F) (mutual information from visible image), MI(VI,F) (mutual information from infrared image), MI (sum of mutual information), and Q^AB/F (edge retention). The proposed method can leave enough information in the original images and its details, and the fused images have better visual effects.     

一种基于区域生长的红外与可见光的图像融合方法

针对目前红外与可见光目标跟踪的视频序列融合难以满足实时性问题，利用红外图像目标与背景显著的灰度差异特征，结合目标跟踪中目标分割时常用到的区域生长法，通过区域生长方法从红外图像中提取目标区域，再将得到的红外目标区域与已经过图像配准的可见光图像的背景区域进行融合处理，最终得到既具有红外图像较好的目标指示特性又具有可见光图像清晰场景信息的融合图像。实验表明：该算法不仅简单易行，而且所得到的融合图像视觉效果优于其他融合算法得到的图像。     

Fusion of the low-light-level visible and infrared images for night-vision context enhancement

For better night-vision applications using the low-light-level visible and infrared imaging, a fusion framework for night-vision context enhancement(FNCE) method is proposed. An adaptive brightness stretching method is first proposed for enhancing the visible image. Then, a hybrid multi-scale decomposition with edge-preserving filtering is proposed to decompose the source images. Finally, the fused result is obtained via a combination of the decomposed images in three different rules. Experimental results demonstrate that the FNCE method has better performance on the details(edges), the contrast, the sharpness, and the human visual perception. Therefore,better results for the night-vision context enhancement can be achieved.     

Fast image enhancement using multi-scale saliency extraction in infrared imagery

We propose a multi-scale saliency extraction based fast infrared image enhancement approach. A local frequency-tuned based saliency extraction technique is designed for highlighting the salient regions, firstly. Then, multi-scale saliency extraction is demonstrated, introducing multi-scale local windows with different sizes to extract regions of interest at different scales. Finally, the original image is enhanced with combining multi-scale salient image regions into one image. The experimental results prove the proposed approach is robust and efficient for infrared image enhancement.     

基于小波分解的医学显微图像融合

提出了基于小波分解的医学显微图像融合。通过先将图像进行小波分解,并对小波分解后的不同频率域分别进行了讨论,即分别讨论了选择高频系数和低频系数的原则。选择低频系数时,设计了两种选择方案,即区域能量法和边缘选择法。选择高频系数时,设计了两种选择方案,即区域能量法和目标清晰与模糊判据法。最后选择了在低频系数时采用边缘选择法,在高频系数时采用目标清晰与模糊判据法。从仿真结果的比较来看,在将多个目标聚焦图像融合后,图像能很好地保留多幅原来图像清晰部分的信息,融合后,提高了整幅图像的信噪比,从而获得了整幅高清晰图像。     

Infrared and visible image fusion method based on saliency detection in sparse domain

Infrared and visible image fusion is a key problem in the field of multi-sensor image fusion. To better preserve the significant information of the infrared and visible images in the final fused image, the saliency maps of the source images is introduced into the fusion procedure. Firstly, under the framework of the joint sparse representation (JSR) model, the global and local saliency maps of the source images are obtained based on sparse coefficients. Then, a saliency detection model is proposed, which combines the global and local saliency maps to generate an integrated saliency map. Finally, a weighted fusion algorithm based on the integrated saliency map is developed to achieve the fusion progress. The experimental results show that our method is superior to the state-of-the-art methods in terms of several universal quality evaluation indexes, as well as in the visual quality.     

基于小波变换和区域分割的红外可见光彩色图像融合算法

提出一种基于小波变换和区域分割的YCb Cr变换域红外-可见光彩色图像融合算法,以小波变换融合为基础,将融合结果作为YCb Cr域参量,以区域分割方法为基础,与参考图像进行色彩传递。实验结果表明,采用方法比传统的线性色彩传递方法具有更好的彩色图像融合效果,同时参考图像适应性较好,适合单一图像以及视频的融合。     

Feature-based fusion of infrared and visible dynamic images using target detection

We employ the target detection to improve the performance of the feature-based fusion of infrared and visible dynamic images, which forms a novel fusion scheme. First, the target detection is used to segment the source image sequences into target and background regions. Then, the dual-tree complex wavelet transform (DT-CWT) is proposed to decompose all the source image sequences. Different fusion rules are applied respectively in target and background regions to preserve the target information as much as possible. Real world infrared and visible image sequences are used to validate the performance of the proposed novel scheme. Compared with the previous fusion approaches of image sequences, the improvements of shift invariance, temporal stability and consistency, and computation cost are all ensured.     

Region-based fusion of infrared and visible images using nonsubsampled contourlet transform

With the nonsubsampled contourlet transform (NSCT), a novel region-segmentation-based fusion algorithm for infrared (IR) and visible images is presented.The IR image is segmented according to the physical features of the target.The source images are decomposed by the NSCT, and then, different fusion rules for the target regions and the background regions are employed to merge the NSCT coefficients respectively.Finally, the fused image is obtained by applying the inverse NSCT.Experimental results show that the proposed algorithm outperforms the pixel-based methods, including the traditional wavelet-based method and NSCT-based method.     

红外和彩色可见光图像亮度-对比度传递融合算法

以红外和彩色可见光图像为研究对象,提出了一种基于亮度-对比度传递（LCT）技术的彩色图像融合算法。首先借助灰度融合方法将红外图像与彩色可见光图像亮度分量融合,然后用LCT技术改善灰度融合结果的亮度和对比度,最后利用快速YCBCR变换融合策略在RGB空间内直接生成彩色融合图像。文中利用像素平均融合法和多分辨率融合法作为不同的灰度融合措施以分别满足高实时性和高融合质量的需求。实验结果表明,提出算法的融合结果不仅具有与输入彩色可见光图像相近的自然色彩,而且具备令人满意的亮度和对比度,即使采用运算简单的像素平均法进行灰度融合,同样可以获得良好的融合效果。