Novel averaging window filter for SIFT in infrared face recognition

The extraction of stable local features directly affects the performance of infrared face recognition algorithms.Recent studies on the application of scale invariant feature transform(SIFT) to infrared face recognition show that star-styled window filter(SWF) can filter out errors incorrectly introduced by SIFT.The current letter proposes an improved filter pattern called Y-styled window filter(YWF) to further eliminate the wrong matches.Compared with SWF,YWF patterns are sparser and do not maintain rotation invariance;thus,they are more suitable to infrared face recognition.Our experimental results demonstrate that a YWF-based averaging window outperforms an SWF-based one in reducing wrong matches,therefore improving the reliability of infrared face recognition systems.     

基于兴趣区域掩码卷积神经网络的红外-可见光图像融合与目标识别算法研究

建立权重独立的双通道残差卷积神经网络,对可见光与红外频段下的目标图像进行特征提取,生成多尺度复合频段特征图组.基于像点间的欧式距离计算双频段特征图显著性,根据目标在不同成像频段下的特征贡献值进行自适应融合.通过热源能量池化核与视觉注意力机制,分别生成目标在双频段下的兴趣区域逻辑掩码并叠加在融合图像上,凸显目标特征并抑制...     

尺度不变特征与几何特征融合的人耳识别方法

要提高人耳的识别率,关键是特征的提取与表达.尺度不变特征变换(SIFT)技术是局部点特征提取算法,在尺度空间寻找极值点,提取对图像的尺度和旋转变化具有不变性,对光照变化和图像变形具有较强的适应性的特征向量.尝试用SIFT技术来提取外耳图像的结构特征点以形成稳定的特征描述子,为了克服一幅图像中有多个局部描述子相似的问题,在SIFT特征描述子中融入一个耳廓几何特征.最后采用特征向量的欧氏距离作为两幅图像相似性度量标准进行人耳识别.在耳图像库七进行实验.结果表明,该方法不仅可以有效地提取人耳特征,通过少量特征可获得较高的识别率,而且对耳图像刚体变化具有较强的稳健性.     

Infrared and visible image fusion based on visual saliency map and weighted least square optimization

The goal of infrared (IR) and visible image fusion is to produce a more informative image for human observation or some other computer vision tasks. In this paper, we propose a novel multi-scale fusion method based on visual saliency map (VSM) and weighted least square (WLS) optimization, aiming to overcome some common deficiencies of conventional methods. Firstly, we introduce a multi-scale decomposition (MSD) using the rolling guidance filter (RGF) and Gaussian filter to decompose input images into base and detail layers. Compared with conventional MSDs, this MSD can achieve the unique property of preserving the information of specific scales and reducing halos near edges. Secondly, we argue that the base layers obtained by most MSDs would contain a certain amount of residual low-frequency information, which is important for controlling the contrast and overall visual appearance of the fused image, and the conventional “averaging” fusion scheme is unable to achieve desired effects. To address this problem, an improved VSM-based technique is proposed to fuse the base layers. Lastly, a novel WLS optimization scheme is proposed to fuse the detail layers. This optimization aims to transfer more visual details and less irrelevant IR details or noise into the fused image. As a result, the fused image details would appear more naturally and be suitable for human visual perception. Experimental results demonstrate that our method can achieve a superior performance compared with other fusion methods in both subjective and objective assessments.     

基于改进SIFT特征点匹配的图像拼接算法

基于传统SIFT方法和图像像素加权平滑融合的思想,提出了一种改进SIFT特征点的图像拼接方法。该方法首先利用Canny边缘检测算法获得图像的边缘点坐标,通过和SIFT算法关键点坐标进行对比,去除不稳定响应点;其次通过K-L变换降低算法复杂度,对得到的匹配点对,使用RANSAC算法进行提纯,计算投影变换模型参数;最后使用渐入渐出的加权融合算法平滑图像,消除图像之间的拼接缝隙。该算法的可行性和有效性通过实验结果可以得到证明。     

基于特征级融合的多波段舰船目标识别方法

针对复杂情况下海上舰船目标单波段特征识别能力不足的问题,研究可见光、中波红外和长波红外三波段特征图像融合技术,重点解决图像融合方法中存在的算法耗时和融合策略选择的问题,提出了一种新的基于区域协方差矩阵的多波段特征级融合方法,针对可见光图像和红外图像分别设计11维和5维特征向量,协方差矩阵可以将多个特征进行融合,既保证了不同目标之间的区别性,同时又减小计算量。该方法首先利用显著性检测,快速定位图像中的目标区域,然后,针对不同波段图像设计的特征向量定义协方差阵的距离计算公式并进行匹配,通过对图像的一次遍历操作获得积分图像,在协方差计算时达到快速计算的目的,最后利用k-阶最近邻算法对多种舰船目标进行分类识别。利用实拍的3 400余张三波段舰船目标图像作为测试数据。实验主要分为两部分,首先对比单波段和三波段融合识别的识别率,验证所提出的融合方法具有更广的应用范围;然后,在计算效率上对比多种传统的像素级方法,验证采用的特征级融合在计算时间上的优势。实验结果表明,该方法可达到95.1%的识别率,单帧计算耗时约为0.5 s,在实时性和检测率方面都有明显提高。     

A categorization method of infrared polarization and intensity image fusion algorithm based on the transfer ability of difference features

The fusion of infrared polarization and intensity image can significantly improve the detection performance of target, and the fused image is more suitable for human visual perception and further image-processing tasks. In this paper, a new categorization method of infrared polarization and intensity image fusion algorithm based on the transfer ability of difference feature is proposed. Firstly, the difference feature between two kinds of image and the characteristics of different fusion algorithms are analyzed and summarized. Second, an evaluation vector of fusion algorithm for difference feature transform ability is constructed. Thirdly, the transfer ability of fusion algorithm for difference feature is estimated by the evaluation vector, and the degree of transfer ability of fusion algorithm for difference feature is analyzed. Finally the fusion algorithms are classified by the degree of transfer ability of fusion algorithm for difference feature. The results shows that the proposed fusion algorithm categorization method helps select fusion algorithms in actual scene.     

融合多尺度特征的短时音频场景识别方法EI北大核心CSCD

为解决短时音频场景识别任务中识别性能差的问题,提出一种融合多尺度特征的音频场景识别方法。首先将双声道音频中左右声道的和差作为输入,并使用长时帧长进行分帧处理,以保证提取出的帧级特征中包含足够多的音频信息。然后将特征逐帧输入到融合多尺度特征的一维卷积神经网络中,以充分利用网络中不同尺度的浅层、中层和深层嵌入特征。最后综合所有帧级软标签得到短时音频的场景分类结果。实验结果表明,该方法在国际声学场景和事件检测与分类挑战赛(DCASE) 2021短时音频场景数据集上的准确率为79.02%,实现了该数据集上目前为止的最优性能。     

Two-scale image fusion of visible and infrared images using saliency detection

Military, navigation and concealed weapon detection need different imaging modalities such as visible and infrared to monitor a targeted scene. These modalities provide complementary information. For better situation awareness, complementary information of these images has to be integrated into a single image. Image fusion is the process of integrating complementary source information into a composite image. In this paper, we propose a new image fusion method based on saliency detection and two-scale image decomposition. This method is beneficial because the visual saliency extraction process introduced in this paper can highlight the saliency information of source images very well. A new weight map construction process based on visual saliency is proposed. This process is able to integrate the visually significant information of source images into the fused image. In contrast to most of the multi-scale image fusion techniques, proposed technique uses only two-scale image decomposition. So it is fast and efficient. Our method is tested on several image pairs and is evaluated qualitatively by visual inspection and quantitatively using objective fusion metrics. Outcomes of the proposed method are compared with the state-of-art multi-scale fusion techniques. Results reveal that the proposed method performance is comparable or superior to the existing methods.     

基于特征点自动匹配的红外图像配准研究

在模糊和含噪声的红外图像配准中,利用角点检测实现特征点的选择,在提高角点提取效率的同时又保证了角点提取的精度。根据互相关的双向匹配实现对应特征点的自动匹配,然后由对应的特征点对估计出仿射变换的参数。实测的数据和计算结果表明,这种方法对于双波段红外图像的配准是有效的,而且有利于后续的图像融合。     

Multi-window visual saliency extraction for fusion of visible and infrared images

Fusion for visible and infrared images aims to combine the source images of the same scene into a single image with more feature information and better visual performance. In this paper, the authors propose a fusion method based on multi-window visual saliency extraction for visible and infrared images. To extract feature information from infrared and visible images, we design local-window-based frequency-tuned method. With this idea, visual saliency maps are calculated for variable feature information under different local window. These maps show the weights of people’s attention upon images for each pixel and region. Enhanced fusion is done using simple weight combination way. Compared with the classical and state-of-the-art approaches, the experimental results demonstrate the proposed approach runs efficiently and performs better than other methods, especially in visual performance and details enhancement.     

Fusion of infrared-visible images using improved multi-scale top-hat transform and suitable fusion rules

Integration of infrared and visible images is an active and important topic in image understanding and interpretation. In this paper, a new fusion method is proposed based on the improved multi-scale center-surround top-hat transform, which can effectively extract the feature information and detail information of source images. Firstly, the multi-scale bright (dark) feature regions of infrared and visible images are respectively extracted at different scale levels by the improved multi-scale center-surround top-hat transform. Secondly, the feature regions at the same scale in both images are combined by multi-judgment contrast fusion rule, and the final feature images are obtained by simply adding all scales of feature images together. Then, a base image is calculated by performing Gaussian fuzzy logic combination rule on two smoothed source images. Finally, the fusion image is obtained by importing the extracted bright and dark feature images into the base image with a suitable strategy. Both objective assessment and subjective vision of the experimental results indicate that the proposed method is superior to current popular MST-based methods and morphology-based methods in the field of infrared-visible images fusion.     

A fusion method for infrared–visible image and infrared-polarization image based on multi-scale center-surround top-hat transform

This paper presents a fusion method for infrared–visible image and infrared-polarization image based on multi-scale center-surround top-hat transform which can effectively extract the feature information and detail information of source images. Firstly, the multi-scale bright (dark) feature regions of source images at different scale levels are respectively extracted by multi-scale center-surround top-hat transform. Secondly, the bright (dark) feature regions at different scale levels are refined for eliminating the redundancies by spatial scale. Thirdly, the refined bright (dark) feature regions from different scales are combined into the fused bright (dark) feature regions through adding. Then, a base image is calculated by performing dilation and erosion on the source images with the largest scale outer structure element. Finally, the fusion image is obtained by importing the fused bright and dark features into the base image with a reasonable strategy. Experimental results indicate that the proposed fusion method can obtain state-of-the-art performance in both aspects of objective assessment and subjective visual quality.     

A feature-based robust digital image watermarking scheme using bandelet transform

Desynchronization attacks are among the most difficult attacks to resist, for it can desynchronize the location of the watermark and hence cause incorrect watermark detection. The design of an image watermarking scheme that is robust against desynchronization attacks is challenging. Based on a multi-scale SIFT (scale invariant feature transform) detector and Bandelet transform theory, we propose a new content based image watermarking algorithm with good visual quality and reasonable resistance toward desynchronization attacks. Firstly, the stable image feature points are extracted from the original host by using the multi-scale SIFT detector, and the local feature regions (LFRs) are constructed adaptively according to the feature scale theory. The Bandelet transform is then performed on the LFRs. Finally, the digital watermark is embedded into the LFRs by modifying the significant Bandelet coefficients. By binding the watermark with the geometrically invariant image features, the watermark detection can be done without synchronization error. Experimental results show that the proposed image watermarking is not only invisible and robust against common signal processing such as sharpening, noise adding, JPEG compression, etc., but also robust against the desynchronization attacks such as rotation, translation, scaling, row or column removal, cropping, etc.     

基于图像融合的动态轮廓线跟踪新方法

红外与可见光传感器是目标跟踪识别系统中常用的两种传感器,对这两种传感器图像进行融合能有效提高系统跟踪检测的准确性。将动态轮廓线模型与图像融合结合,在特征搜索过程中利用特征点准确地完成了图像配准,同时使用了一种新的特征级融合方法,将两种图像中目标轮廓的B样条曲线控制点进行实时微分耦合。这种耦合将Curwen提出的微分耦合机制作了改进,利用图像配准把刚性硬模板改变为实时的变换模板并推导了融合后动态轮廓线的新的动力学方程。这种融合利用了红外图像目标轮廓信息约束可见光图像中动态轮廓线的收敛形状,有效地提高了可见光图像目标跟踪的准确性。对运动人手序列图像的对比跟踪实验表明,这种融合使得可见光图像中动态轮廓线平均跟踪误差减小了60．25％。     

多波段红外图像的海面舰船目标检测

现有的基于单个红外宽波段的海面舰船目标探测系统在面对复杂海天背景、岛岸背景、恶劣天气、亮带干扰或诱饵弹干扰等情况时,系统的探测率、虚警率、探测距离等性能指标均会受到严重的影响;为此,开展了基于多波段红外图像的海面舰船目标检测方法的研究。通过中波红外多波段数据采集系统实际采集107组五个中波红外波段的图像;波段1-5分别为3.7～4.8,3.7～4.1,4.4～4.8,3.7～3.9和4.65～4.75 μm;对多波段图像进行手动标注构建样本数据集,其中,正样本舰船目标298个,负样本非舰船目标353个。对于多波段红外图像,首先进行PCA降维并采用选择性搜索算法生成初始目标候选区域;针对候选区域中存在大量明显的非舰船目标区域的问题,利用积分图像计算候选区域的局部对比度,依据红外舰船目标的几何和灰度特征从初始目标候选区域中筛选出舰船目标可能性大的区域作为舰船目标候选区域。然后对舰船目标候选区域进行拓展以融入局部上下文信息,对于候选区域对应的5波段红外图像,分别提取每个波段图像的稠密SIFT特征,并将128维SIFT特征向量降为64维,融入SIFT特征的空间和波段位置分布信息得到新的特征向量,基于高斯混合模型对候选区域的特征向量集合进行编码融合得到舰船目标候选区域的费舍尔向量表示,最后利用线性SVM分类器识别出舰船目标。对多波段图像进行舰船目标候选区域生成实验,所提出的基于红外舰船目标的几何和灰度特征的约束方法可以有效地克服选择性搜索算法的不足,从初始目标候选区域中快速定位出舰船目标候选区域,对25组多波段图像进行实验,舰船目标候选区域生成的整体耗时为0.353 s,定位舰船目标区域耗时0.005 s。对100个正负样本进行目标识别测试,所提出的目标识别算法融合了目标的多波段图像特征信息,通过引入费舍尔向量挖掘了多波段图像梯度统计特征的深层次信息,算法的识别率达到了0.97,显著高于单波段红外图像的目标识别率。对25组多波段图像进行舰船目标检测实验,所提出的舰船目标检测方法能够在海天背景、岛岸背景以及亮带干扰等不同场景下完成海面舰船目标的检测工作,舰船目标定位准确,舰船目标召回率达到了0.95,每组多波段图像的平均检测耗时为1.33 s。研究结果表明,充分考虑海面舰船目标在红外图像中与局部海洋背景的辐射差异以及有效地融合舰船目标在多个红外波段图像中的辐射特征,可以增强舰船目标的可分性,提高舰船目标的识别率以及检测率,为基于多波段红外图像的海面舰船目标检测提供了新的技术支持。     

海空背景下红外-紫外双色探测技术研究

舰船采用紫外-红外多传感器探测低空飞行的导弹、飞机等目标，对各个传感器采集的目标图像数据进行融合处理，可提高对目标的探测识别概率。本文分析了红外传感器器和紫外传感器的各自的优越性、海空背景下导弹羽烟的紫外辐射传输特性，得出采用红外-紫外图像融合技术能使红外、紫外传感器的优势得到互补，融合级别适合在特征级上进行，并对红外-紫外双色探测技术进行了展望。     

Infrared image enhancement through saliency feature analysis based on multi-scale decomposition

To improve contrast between dim target region and background in infrared (IR) long-range surveillance, this paper proposes a fast image enhancement approach using saliency feature extraction based on multi-scale decomposition. Firstly, a smooth based multi-scale decomposition is designed and applied to original infrared image, generating sub-images with various frequency components at different decomposition levels. The dim target regions of sub-images are extracted by a local frequency-tuned based saliency feature detection method, secondly. With saliency maps created by saliency extraction using multi-scale local windows with different sizes, the sub-images are enhanced at different decomposition scales. Finally, the enhanced result is reconstructed by synthesizing the all sub-images with adjustable synthetic weights. Since salient areas are analyzed based on fast multi-scale image decomposition, IR image can be s enhanced with good contrast successfully and rapidly. Compared with other algorithms, the experimental results prove that the proposed method is robust and efficient for IR image enhancement.     

Thermal 3D modeling system based on 3-view geometry

In this paper, we propose a novel thermal three-dimensional (3D) modeling system that includes 3D shape, visual, and thermal infrared information and solves a registration problem among these three types of information. The proposed system consists of a projector, a visual camera and, a thermal camera (PVT). To generate 3D shape information, we use a structured light technique, which consists of a visual camera and a projector. A thermal camera is added to the structured light system in order to provide thermal information. To solve the correspondence problem between the three sensors, we use three-view geometry. Finally, we obtain registered PVT data, which includes visual, thermal, and 3D shape information. Among various potential applications such as industrial measurements, biological experiments, military usage, and so on, we have adapted the proposed method to biometrics, particularly for face recognition. With the proposed method, we obtain multi-modal 3D face data that includes not only textural information but also data regarding head pose, 3D shape, and thermal information. Experimental results show that the performance of the proposed face recognition system is not limited by head pose variation which is a serious problem in face recognition.