基于时空双流卷积神经网络的红外行为识别

针对红外视频人体行为识别问题,提出了一种基于时空双流卷积神经网络的红外人体行为识别方法。通过将整个红外视频进行平均分段,然后将每一段视频中随机抽取的红外图像和对应的光流图像输入空间卷积神经网络,空间卷积神经网络通过融合光流信息可以有效地学习到红外图像中真正发生运动的空间信息,再将每一小段的识别结果进行融合得到空间网络结果。同时将每一段视频中随机抽取的光流图像序列输入时间卷积神经网络,融合每一小段的结果后得到时间网络结果。最后再将空间网络结果和时间网络结果进行加权求和,从而得到最终的视频分类结果。实验中,采用此方法对包含23种红外行为动作类别的红外视频数据集上的动作进行识别,正确识别率为92.0%。结果表明,该算法可以有效地对红外视频行为进行准确识别。     

Pixel-clarity-based multifocus image fusion

Due to the limited depth-of-field of optical lenses, it is difficult to get an image with all objects in focus. One way to overcome this problem is to take several images with different focus points and combine them into a single composite which contains all the regions full focused. This paper describes a pixel-clarity-based multifocus image fusion algorithm. The characteristic of this approach is that the pixels of the fused image are selected from the clearest pixels in the input images according to pixel clarity criteria. For each pixel in the source images, the pixel clarity is calculated. The fusion procedure is performed by a selection mode according to the magnitude of pixel clarity. Consistency verification is performed on the selected pixels. Experiments show that the proposed algorithm works well in multifocus image fusion.     

基于改进YOLOv3网络的无人车夜间环境感知

环境感知是无人车夜间行驶中的一项关键任务，提出一种改进的YOLOv3网络，以实现夜间对无人车获取的红外图像中行人、车辆的检测，将判断周边车辆的行驶方向问题转化为预测车辆位置的角度大小问题，并与深度估计信息进行融合对周边车辆行驶的距离和速度作出判断，从而实现夜间无人车对周边车辆行驶意图的感知。该网络具有端到端的优点，能实现整张图像作为网络的输入，直接在输出层回归检测目标的边界框位置、所属的类别和车辆的角度预测结果，并和深度估计信息融合得到周边车辆的距离和速度信息。实验结果表明，使用改进的YOLOv3网络对夜间无人车获取的红外图像进行目标检测的时间为0.04 s/帧，角度和速度预测效果较好，准确性和实时性达到了实际应用要求。     

基于改进DeepLabv3+的无人车夜间红外图像语义分割

为了增强无人车对夜间场景的理解能力，针对无人车在夜间获取的红外图像，提出了一种基于改进DeepLabv3+网络的无人车夜间红外图像语义分割算法。由于自动驾驶场景中的对象往往显示出非常大的尺度变化，该算法在DeepLabv3+网络的基础上，通过引入密集连接的空洞卷积空间金字塔模块，使网络生成的多尺度特征能覆盖更大的尺度范围。此外，该算法将编码器模块的多层结果拼接在译码器模块中，以恢复更多在降采样过程中丢失的空间信息和低级特征。通过端到端的学习和训练，可直接用于对夜间红外图像的语义分割。实验结果表明，该算法在红外数据集上的分割精度优于原DeepLabv3+算法，平均交并比达到80.42，具有良好的实时性和准确性。     

Fusion of visible and infrared imagery for face recognition

In recent years face recognition has received substantial attention, but still remained very challenging in real applications. Despite the variety of approaches and tools studied, face recognition is not accurate or robust enough to be used in uncontrolled environments. Infrared (IR) imagery of human faces offers a promising alternative to visible imagery, however, IR has its own limitations. In this paper, a scheme to fuse information from the two modalities is proposed. The scheme is based on eigenfaces and probabilistic neural network (PNN), using fuzzy integral to fuse the objective evidence supplied by each modality. Recognition rate is used to evaluate the fusion scheme. Experimental results show that the scheme improves recognition performance substantially.     

基于方向金字塔框架变换的遥感图像融合算法

为了综合利用多光谱遥感图像与全色遥感图像之间的互补信息,提出了一种方向金字塔框架变换(SPFT),并基于此变换提出了一种遥感图像融合算法。具体融合过程是将多光谱图像的每个波段分别与高分辨力全色图像进行融合,首先将高分辨力全色图像与多光谱图像的待融合波段进行直方图匹配,然后对该波段图像以及直方图匹配后的高分辨力全色图像分别进行方向金字塔框架变换分解,融合过程就是对两图像方向金字塔框架变换分解后的系数进行组合,最后对组合后的系数进行方向金字塔框架逆变换即可得到该波段图像与高分辨力全色图像的融合图像。实验结果表明该算法在性能上优于基于亮度-色调-饱和度(1HS)的彩色空间变换以及基于离散小波框架变换(DWFT)的遥感图像融合方法,尤其对源图像之间存在配准误差的情况。     

Coloring night vision imagery for depth perception

Depth perception for night vision （NV） imagery could largely improve scene comprehension. We present a novel scheme to give fused multi-band NV imagery smoothly natural color appearance as well as depth sense from color. Our approach is based on simulating color cues by varying saturation values of each object in the color NV image, in correspondence with the ratio between the infrared and low-light-level sensor outputs which in practice is the depth feature for same materials. We render the NV image segment- by-segment by taking advantage image fusion. Experiments have of image segmentation, dominant shown that the proposed scheme color transfer, saturation variation, and can achieve satisfying results.     

Multi-sensor image fusion using discrete wavelet frame transform

An algorithm is presented for multi-sensor image fusion using discrete wavelet frame transform (DWFT).The source images to be fused are firstly decomposed by DWFT. The fusion process is the combining of the source coefficients. Before the image fusion process, image segmentation is performed on each source image in order to obtain the region representation of each source image. For each source image, the salience of each region in its region representation is calculated. By overlapping all these region representations of all the source images, we produce a shared region representation to label all the input images. The fusion process is guided by these region representations. Region match measure of the source images is calculated for each region in the shared region representation. When fusing the similar regions, weighted averaging mode is performed; otherwise selection mode is performed. Experimental results using real data show that the proposed algorithm outperforms the traditional pyramid transform based or discrete wavelet transform (DWT) based algorithms in multi-sensor image fusion.     

Image fusion based on expectation maximization algorithm and steerable pyramid

In this paper, a novel image fusion method based on the expectation maximization (EM) algorithm and steerable pyramid is proposed. The registered images are first decomposed by using steerable pyramid. The EM algorithm is used to fuse the image components in the low frequency band. The selection method involving the informative importance measure is applied to those in the high frequency band. The final fused image is then computed by taking the inverse transform on the composite coefficient representations. Experimental results show that the proposed method outperforms conventional image fusion methods.     

基于轮廓跟踪的车载红外视频彩色化方法

提出一种车载红外视频快速彩色化方法,利用轮廓特征点跟踪获取每帧物体类别的轮廓区域,采用类别特征色彩对各区域传递色彩。构建各景物样本特征色彩集,以各类景物在自然彩色图像中表现出来的特征色彩作为红外图像中对应景物的色彩;利用改进的高效K Means方法对红外关键帧进行聚类,得到分割区域,提取轮廓特征点;通过KLT算法跟踪特征点,得到其在下一帧中的位置并同时修正,采用B样条插值进行轮廓复原,得到该帧的各类别轮廓区域;最后将特征色彩按类别赋予各区域,从而给各帧图像着上合适的颜色,实现红外视频序列的快速彩色化。实验结果表明, 该方法与基于运动估计的算法相比可提高近5倍的处理速度,并且能够得到与自然景物色彩较接近的彩色化视觉效果。