A fusion method for visible and infrared images based on contrast pyramid with teaching learning based optimization

This paper proposes a novel image fusion scheme based on contrast pyramid (CP) with teaching learning based optimization (TLBO) for visible and infrared images under different spectrum of complicated scene. Firstly, CP decomposition is employed into every level of each original image. Then, we introduce TLBO to optimizing fusion coefficients, which will be changed under teaching phase and learner phase of TLBO, so that the weighted coefficients can be automatically adjusted according to fitness function, namely the evaluation standards of image quality. At last, obtain fusion results by the inverse transformation of CP. Compared with existing methods, experimental results show that our method is effective and the fused images are more suitable for further human visual or machine perception.     

An improved fusion algorithm for infrared and visible images based on multi-scale transform

In this paper, an improved fusion algorithm for infrared and visible images based on multi-scale transform is proposed. First of all, Morphology-Hat transform is used for an infrared image and a visible image separately. Then two images were decomposed into high-frequency and low-frequency images by contourlet transform (CT). The fusion strategy of high-frequency images is based on mean gradient and the fusion strategy of low-frequency images is based on Principal Component Analysis (PCA). Finally, the final fused image is obtained by using the inverse contourlet transform (ICT). The experiments and results demonstrate that the proposed method can significantly improve image fusion performance, accomplish notable target information and high contrast and preserve rich details information at the same time.     

Image fusion based on nonsubsampled contourlet transform for infrared and visible light image

In this paper, a new method based on nonsubsampled contourlet transform (NSCT) is proposed to fuse the infrared image and the visible light image, which will produce a new fused image by which the target can be identified more easily. Firstly, two original images were decomposed into low frequency subband coefficients and the bandpass direction subband coefficients by using NSCT. Secondly, the selection of the low frequency subband coefficient and the bandpass direction subband coefficient is discussed in detail. The low frequency subband coefficients are selected based on the regional visual characteristics. For the selection of bandpass direction subband coefficients, this paper proposes a minimum regional cross-gradient method, and the cross-gradient is gained by calculating the gradient between the pixel of bandpass subbands and the adjacent pixel in the fused image of the low-frequency components. Comparison experiments have been performed on different image sets, and experimental results demonstrate that the proposed method performs better in both subjective and objective qualities.     

基于红外和可见光图像逐级自适应融合的场景深度估计

从图像中恢复场景的深度是计算机视觉领域中的一个关键问题。考虑到单一类型图像在深度估计中受场景不同光照的限制,提出了基于红外和可见光图像逐级自适应融合的场景深度估计方法（PF-CNN）。该方法包括双流滤波器部分耦合网络、自适应多模态特征融合网络以及自适应逐级特征融合网络。在双流卷积中红外和可见光图像的滤波器部分耦合使两者特征得到增强;自适应多模态特征融合网络学习红外和可见光图像的残差特征并将两者自适应加权融合,充分利用两者的互补信息;逐级特征融合网络学习多层融合特征的结合,充分利用不同卷积层的不同特征。实验结果表明:PF-CNN在测试集上获得了较好的效果,将阈值指标提高了5%,明显优于其他方法。     

A multi-focus image fusion algorithm based on an improved dual-channel PCNN in NSCT domain

This paper presents a multi-focus image fusion algorithm based on dual-channel PCNN in NSCT domain. The fusion algorithm based on multi-scale transform is likely to produce the pseudo-Gibbs effects and it is not effective to fuse the dim or partial bright images. To solve these problems, this algorithm will get a number of different frequency sub-image of the two images by using the NSCT transform, the selection principles of different subband coefficients obtained by the NSCT decomposition are discussed in detail, and the images are fused based on the improved dual-channel PCNN in order to determine the band-pass sub-band coefficient, at last fused image is obtained by using the inverse NSCT transform. Fusion rules based on dual-channel PCNN are used to solve the complexity of the PCNN parameter settings and long computing time problems. The experimental results show that the algorithm has overcome the defects of the traditional multi-focus image fusion algorithm and improved the fusion effect.     

Fusion method of infrared and visible images based on neighborhood characteristic and regionalization in NSCT domain

On fusing infrared and visible image, the traditional fusion method cannot get the better image quality. Based on neighborhood characteristic and regionalization in NSCT (Nonsubsampled Contourlet Transform) domain, the fusion algorithm was proposed. Firstly, NSCT was adopted to decompose infrared and visible images at different scales and directions for the low and high frequency coefficients, the low frequency coefficients which were fused with improving regional weighted fusion method based on neighborhood energy, and the high-frequency coefficients were fused with multi-judgment rule based on neighborhood characteristic regional process. Finally, the coefficients were reconstructed to obtain the fused image. The experimental results show that, compared with the other three related methods, the proposed method can get the biggest value of IE (information entropy), MI(VI,F) (mutual information from visible image), MI(VI,F) (mutual information from infrared image), MI (sum of mutual information), and Q^AB/F (edge retention). The proposed method can leave enough information in the original images and its details, and the fused images have better visual effects.     

A fusion algorithm for infrared and visible images based on RDU-PCNN and ICA-bases in NSST domain

The aim of infrared and visible image fusion is to enhance the feature in infrared image and preserve abundant detail information in visible image. Based on the fact that the human sense system accepts external stimulation only when the stimulus intensity is greater than a certain value and the reaction of neuronal cells have obvious regional characters, an image fusion algorithm based on region dual-channel unit-linking pulse coupled neural networks (RDU-PCNN) and independent component analysis (ICA) bases in non-subsampled shearlet transform (NSST) domain for infrared and visible images is proposed. RDU-PCNN we constructed has obvious regional characters and much lower computational costs. We trained ICA-bases using a number of images that the content and statistical properties are similar with the fusion images but applied it as low-frequency ICA-bases, which can reduce calculation complexity. Experimental results demonstrate that the proposed method can significantly improved the fusion quality and need less computational costs.     

Learning-based compressed sensing for infrared image super resolution

This paper presents an infrared image super-resolution method based on compressed sensing (CS). First, the reconstruction model under the CS framework is established and a Toeplitz matrix is selected as the sensing matrix. Compared with traditional learning-based methods, the proposed method uses a set of sub-dictionaries instead of two coupled dictionaries to recover high resolution (HR) images. And Toeplitz sensing matrix allows the proposed method time-efficient. Second, all training samples are divided into several feature spaces by using the proposed adaptive k-means classification method, which is more accurate than the standard k-means method. On the basis of this approach, a complex nonlinear mapping from the HR space to low resolution (LR) space can be converted into several compact linear mappings. Finally, the relationships between HR and LR image patches can be obtained by multi-sub-dictionaries and HR infrared images are reconstructed by the input LR images and multi-sub-dictionaries. The experimental results show that the proposed method is quantitatively and qualitatively more effective than other state-of-the-art methods.     

Infrared and visible image fusion scheme based on NSCT and low-level visual features

Multi-scale transform (MST) is an efficient tool for image fusion. Recently, many fusion methods have been developed based on different MSTs, and they have shown potential application in many fields. In this paper, we propose an effective infrared and visible image fusion scheme in nonsubsampled contourlet transform (NSCT) domain, in which the NSCT is firstly employed to decompose each of the source images into a series of high frequency subbands and one low frequency subband. To improve the fusion performance we designed two new activity measures for fusion of the lowpass subbands and the highpass subbands. These measures are developed based on the fact that the human visual system (HVS) percept the image quality mainly according to its some low-level features. Then, the selection principles of different subbands are presented based on the corresponding activity measures. Finally, the merged subbands are constructed according to the selection principles, and the final fused image is produced by applying the inverse NSCT on these merged subbands. Experimental results demonstrate the effectiveness and superiority of the proposed method over the state-of-the-art fusion methods in terms of both visual effect and objective evaluation results.     

Multi-window visual saliency extraction for fusion of visible and infrared images

Fusion for visible and infrared images aims to combine the source images of the same scene into a single image with more feature information and better visual performance. In this paper, the authors propose a fusion method based on multi-window visual saliency extraction for visible and infrared images. To extract feature information from infrared and visible images, we design local-window-based frequency-tuned method. With this idea, visual saliency maps are calculated for variable feature information under different local window. These maps show the weights of people’s attention upon images for each pixel and region. Enhanced fusion is done using simple weight combination way. Compared with the classical and state-of-the-art approaches, the experimental results demonstrate the proposed approach runs efficiently and performs better than other methods, especially in visual performance and details enhancement.     

一种基于选择性测量的自适应压缩感知方法

针对低信噪比条件下现有压缩感知系统重构性能严重恶化的问题,提出了一种基于选择性测量的自适应压缩感知结构.首先推导并分析了经过压缩测量的噪声的统计特性及其对重构性能的影响;然后基于输出能量最小化准则,设计了一种压缩域投影滤波联合噪声检测的自适应感知器,感知获得噪声子空间的位置信息;进一步利用该信息构造选择性压缩测量矩阵,智能选择测量信号,同时"屏蔽"噪声分量,极大提高了压缩测量值的信噪比.仿真结果表明,相对于现有压缩感知结构,选择性测量的压缩感知结构明显改善了含噪稀疏信号的重构性能,可更好地应用于吸波材料的前端特性分析、认知无线电的频谱感知等领域.     

Two-scale image fusion of visible and infrared images using saliency detection

Military, navigation and concealed weapon detection need different imaging modalities such as visible and infrared to monitor a targeted scene. These modalities provide complementary information. For better situation awareness, complementary information of these images has to be integrated into a single image. Image fusion is the process of integrating complementary source information into a composite image. In this paper, we propose a new image fusion method based on saliency detection and two-scale image decomposition. This method is beneficial because the visual saliency extraction process introduced in this paper can highlight the saliency information of source images very well. A new weight map construction process based on visual saliency is proposed. This process is able to integrate the visually significant information of source images into the fused image. In contrast to most of the multi-scale image fusion techniques, proposed technique uses only two-scale image decomposition. So it is fast and efficient. Our method is tested on several image pairs and is evaluated qualitatively by visual inspection and quantitatively using objective fusion metrics. Outcomes of the proposed method are compared with the state-of-art multi-scale fusion techniques. Results reveal that the proposed method performance is comparable or superior to the existing methods.     

Synthetic Aperture Radar image nonlinear enhancement algorithm based on NSCT transform

Due to the imaging mechanism, Synthetic Aperture Radar (SAR) images are susceptible to speckle noise, which affects radar image interpretation. So image denoising and enhancement are important topics of improving SAR image performance. A nonlinear image enhancement algorithm based on nonsubsampled contourlet transform (NSCT) is proposed in this paper. The image is decomposed into coefficients of different scales and directions through nonsubsampled contourlet transform. It is denoised by the threshold method of the multi-scale product of NSCT coefficients. Then thresholds of the nonlinear enhancement function are determined according to the coefficients of each scale. The two parameters of the function, among which one is used to control the range of enhancement and the other can determine the strength of enhancement, are obtained by solving nonlinear equations. The coefficients processed by the enhancement function are used to reconstruct the image. The simulation results on the Matlab platform show that the algorithm has a good effect of enhancing details of images and suppressing noise signals meanwhile.     

红外和彩色可见光图像亮度-对比度传递融合算法

以红外和彩色可见光图像为研究对象,提出了一种基于亮度-对比度传递（LCT）技术的彩色图像融合算法。首先借助灰度融合方法将红外图像与彩色可见光图像亮度分量融合,然后用LCT技术改善灰度融合结果的亮度和对比度,最后利用快速YCBCR变换融合策略在RGB空间内直接生成彩色融合图像。文中利用像素平均融合法和多分辨率融合法作为不同的灰度融合措施以分别满足高实时性和高融合质量的需求。实验结果表明,提出算法的融合结果不仅具有与输入彩色可见光图像相近的自然色彩,而且具备令人满意的亮度和对比度,即使采用运算简单的像素平均法进行灰度融合,同样可以获得良好的融合效果。     

Image fusion algorithm based on redundant-lifting NSWMDA and adaptive PCNN

Pyramid decomposition in the NSCT transformation is a band-pass filtering process in the frequency domain where different scales of images are orthogonal. However, from the perspective of the image content, correlation is likely to exist between the fused images, and this kind of decomposition makes images of different scales contain redundant information, as a result of which the fused image may not capture the subtle information from the original images. In order to overcome the above-mentioned problem, an effective image fusion method based on redundant-lifting non-separable wavelet multi-directional analysis (NSWMDA) and adaptive pulse coupled neural network (PCNN) has been proposed. The original images are firstly decomposed by using the NSWMDA into several sub-bands in order to retain texture detail and contrast information of the images, and then adaptive PCNN algorithm is applied on the high-frequency directional sub-bands to extract the high-frequency information. The low-frequency sub-bands are evaluated by weighted average based on Gaussian kernel with a chosen maximum fusion rule. Results from experiments show that the proposed method can make the fused image maintains more texture details and contrast information.     

Study on compressed sensing imaging based on intensity modulation in Fourier domain

In 4F system, compressed sensing is usually implemented by using phase modulation in Fourier domain. In this paper, we present a type of 4F system based on intensity modulation in Fourier domain as the measurement system for compressed sensing. The feasibility of this system is demonstrated. At the point of coherence, the two modulation methods are compared and superiority of intensity modulation in Fourier domain was verified. Simulations are presented and the conclusion we presented is validated. Finally, we analyze the results.     

基于压缩感知耦合梯度下降的红外-可见光图像自适应融合算法

设计了一种压缩感知耦合梯度下降的IR-VI图像自适应融合方案。引入S-函数对IR图像进行预处理,增强其对比度。利用非下采样Contourlet变换对IR与VI图像分解,分别得到低频与高频系数。对低频系数,利用自适应区域平均能量准则对其进行融合,以减少边缘模糊。对于高频部分,引入压缩感知进行稀疏采样,再采用绝对最大值选择与自适应高斯区域标准差的融合规则,通过高斯模糊隶属度建立的自适应控制融合过程,并利用基于梯度下降迭代算法来求解稀疏信号,形成高频融合系数。通过逆NSCT生成最终融合图像。实验表明,与当前流行的红外-可见光融合算法比较,所提算法具有更高的融合质量,输出图像的信息更丰富,边缘与纹理更为清晰。所提算法具有较高的融合质量,在红外、安防以及模式识别等领域具有一定的应用价值。     

Infrared super-resolution imaging based on compressed sensing

The theoretical basis of traditional infrared super-resolution imaging method is Nyquist sampling theorem. The reconstruction premise is that the relative positions of the infrared objects in the low-resolution image sequences should keep fixed and the image restoration means is the inverse operation of ill-posed issues without fixed rules. The super-resolution reconstruction ability of the infrared image, algorithm’s application area and stability of reconstruction algorithm are limited. To this end, we proposed super-resolution reconstruction method based on compressed sensing in this paper. In the method, we selected Toeplitz matrix as the measurement matrix and realized it by phase mask method. We researched complementary matching pursuit algorithm and selected it as the recovery algorithm. In order to adapt to the moving target and decrease imaging time, we take use of area infrared focal plane array to acquire multiple measurements at one time. Theoretically, the method breaks though Nyquist sampling theorem and can greatly improve the spatial resolution of the infrared image. The last image contrast and experiment data indicate that our method is effective in improving resolution of infrared images and is superior than some traditional super-resolution imaging method. The compressed sensing super-resolution method is expected to have a wide application prospect.     

Fleet algorithm for X-ray pulsar profile construction and TOA solution based on compressed sensing

X-ray pulse profile and time of arrival (TOA) are the two important physical quantities for pulsar navigation. With the standard and integrated X-ray pulse profiles modeled, X-ray pulse profile construction is studied and TOA is solved using compressed sensing (CS) technology. The observation matrix and waveform complete dictionary are mainly examined. A column vector-based matching pursuit algorithm is presented. The feasibility of obtaining X-ray pulse profile construction by compressed sensing technology is verified by numerical simulation. Compared with the X-ray pulse profile construction method based on epoch folding, the proposed method exhibits improved real-time performance, and its detection time for integrated X-ray pulse profile could be reduced by one order of magnitude. This proposed method can also solve for TOA solution and construct the X-ray pulse profile simultaneously, which is essential to improve pulsar navigation efficiency.