图像序列的显著性目标区域检测方法

针对传统视觉显著性模型在自顶向下的任务指导和动态信息处理方面的不足,设计并实现了融入运动特征的视觉显著性模型。利用该模型提取了图像的静态特征和动态特征,静态特征的提取在图像的亮度、颜色和方向通道进行,运动特征的提取采用基于多尺度差分的特征提取方法实现,然后各通道分别通过滤波、差分得到显著图,在生成全局显著图时,提出多通道参数估计方法,计算图像感兴趣区域与眼动感兴趣区域的相似度,从而可在图像上准确定位目标位置。针对20组视频图像序列(每组50帧)进行了实验,结果表明:本文算法提取注意焦点即目标区域的平均相似度为0.87,使用本文算法能够根据不同任务情境,选择各特征通道的权重参数,从而可有效提高目标搜索的效率。     

基于双阈值视觉注意模型的图像关注焦点检测

 图像关注焦点（FOA）检测是基于人眼视觉关注模型的图像感兴趣区提取的关键技术。为了更加精确、合理地搜索图像关注焦点,提出一种基于双阈值视觉关注模型的FOA检测算法。算法首先提取图像的亮度、颜色、方向和离散矩变换（DMT）特征,生成各个特征对应的特征图;然后将各特征图合并为一张包含多种特征的显著图;最后根据显著图的灰度直方图建立静态阈值与动态阈值,确定图像关注焦点的位置与数量。实验结果表明：新算法在图像关注焦点检测的准确性上较Itti模型有更为优秀的表现,更符合人眼视觉的关注习惯。     

Novel biological visual attention mechanism via Gaussian harmony search

Most of the visual attention models are based on the concept of a two-dimensional saliency map, which encodes the conspicuity of the object in the visual scene. The visual attention model proposed by Laurent Itti is used in this work. In Itti's model, the saliency map is calculated via combining the information across several modalities, including color, intensity, and orientation. In this work, we propose a pre-training process to select the weightings used in the combining of feature maps to make the target more conspicuity in the saliency map. Harmony search (HS) algorithm is used in the pre-training process to obtain the weightings. HS is a new heuristic algorithm, which mimics the improvisation of music players. Its performance has been verified by many benchmark problems. We modify the pitch adjustment process of the original HS to improve the optimization performance and accelerate the convergence rate. The modified algorithm is named Gaussian harmony search (GHS).     

基于视觉注意机制的感兴趣区检测

提出了一种基于视觉注意机制的感兴趣区检测方法：使用分水岭方法分割图像区域;根据生物的视觉注意机制特性,选用中央周边差的采样方式提取图像特征,将不同维的图像特征融合为显著图;显著点经过竞争得到的注意焦点作为分水岭分割的种子点,然后融合显著图和分水岭分割区得到感兴趣区;遵循返回抑制和邻近优先的准则选择并转移注意焦点,从而计算区域的重要性或兴趣度.实验结果表明该方法符合生物的视觉注意机制,在自动检测感兴趣区时可以有效减少过分割,也能较好的处理大对象.     

Infrared image enhancement through saliency feature analysis based on multi-scale decomposition

To improve contrast between dim target region and background in infrared (IR) long-range surveillance, this paper proposes a fast image enhancement approach using saliency feature extraction based on multi-scale decomposition. Firstly, a smooth based multi-scale decomposition is designed and applied to original infrared image, generating sub-images with various frequency components at different decomposition levels. The dim target regions of sub-images are extracted by a local frequency-tuned based saliency feature detection method, secondly. With saliency maps created by saliency extraction using multi-scale local windows with different sizes, the sub-images are enhanced at different decomposition scales. Finally, the enhanced result is reconstructed by synthesizing the all sub-images with adjustable synthetic weights. Since salient areas are analyzed based on fast multi-scale image decomposition, IR image can be s enhanced with good contrast successfully and rapidly. Compared with other algorithms, the experimental results prove that the proposed method is robust and efficient for IR image enhancement.     

基于生物视觉特征和视觉心理学的视频显著性检测算法

提出了一种空域和时域相结合的视频显著性检测算法.对单帧图像,受视觉皮层层次化感知特性和Gestalt视觉心理学的启发,提出了一种层次化的静态显著图检测方法.在底层,通过符合生物视觉特性的特征图像（双对立颜色特征及亮度特征图像）的非线性简化模型来合成特征图像,形成多个候选显著区域;在中层,根据矩阵的最小Frobenius-范数（F-范数）性质选取竞争力最强的候选显著区域作为局部显著区域;在高层,利用Gestalt视觉心理学的核心理论,对在中层得到的局部显著区域进行整合,得到具有整体感知的空域显著图.对序列帧图像,基于运动目标在位置、运动幅度和运动方向一致性的假设,对Lucas-Kanade算法检测出的光流点进行二分类,排除噪声点的干扰,并利用光流点的运动幅度来衡量运动目标运动显著性.最后,基于人类视觉对动态信息与静态信息敏感度的差异提出了一种空域和时域显著图融合的通用模型.实验结果表明,该方法能够抑制视频背景中的噪声并且解决了运动目标稀疏等问题,能够较好地从复杂场景中检测出视频中的显著区域.     

Multi-window visual saliency extraction for fusion of visible and infrared images

Fusion for visible and infrared images aims to combine the source images of the same scene into a single image with more feature information and better visual performance. In this paper, the authors propose a fusion method based on multi-window visual saliency extraction for visible and infrared images. To extract feature information from infrared and visible images, we design local-window-based frequency-tuned method. With this idea, visual saliency maps are calculated for variable feature information under different local window. These maps show the weights of people’s attention upon images for each pixel and region. Enhanced fusion is done using simple weight combination way. Compared with the classical and state-of-the-art approaches, the experimental results demonstrate the proposed approach runs efficiently and performs better than other methods, especially in visual performance and details enhancement.     

Dual multi-scale filter with SSS and GW for infrared small target detection

Multi-scale analysis is a powerful tool in the field of signal processing. In this paper, we propose an efficient small target detection algorithm that is mainly based on the dual multi-scale filters which work sequentially. The algorithm consists of two stages: at the first stage, Spectrum Scale-Space (SSS) is used as the pre-process procedure to obtain the multi-scale saliency maps, which can suppress the low frequency background noise and make the target region prominently at different scale levels. As a result, the more detail information and feature information can be exhibited in the different decomposition image level. After then, the least information entropy is used as the criterion to select the optimal salient map out; At the second stage, the Gabor wavelets (GW) algorithm is utilized to suppress the high frequency noise remained in the optimal salient map and match the feature of size and direction of small target at different scales and angles, and next, to ensure the robustness of the target detection, Non-negative Matrix Factorization (NMF) is applied to fuse all the GW multi-scale images into one optimal target image, which is the final output of the presented method. Experimental results show that, compared with the contrast method, the proposed algorithm has high SCRG and high correct target detection rate, and works well in different types of complex backgrounds.     

基于背景原型对比度的显著性物体检测

针对传统显著性模型在自然图像的显著性物体检测中存在的缺陷,提出了一种利用背景原型(background prototypes)进行对比的视觉关注模型,以实现显著性物体的检测与提取;传统显著性模型主要通过计算区域中心与四周区域差异性实现显著性检测,而自然场景中显著性区域和背景区域往往都存在较大差异,导致在复杂图像中难以获得理想检测效果;基于背景原型对比度的显著性物体检测方法在图像分割生成的超像素图基础上,选择距离图像中心较远的图像区域作为背景原型区域,通过计算图像中任意区域与这些背景原型区域的颜色对比度准确检测和提取图像中的显著性物体;实验结果表明,基于背景原型对比度的显著性模型可以更好地滤除杂乱背景,产生更稳定、准确的显著图,在准确率、召回率和F-measure等关键性能和直观视觉效果上均优于目前最先进的显著性模型,计算复杂度低,利于应用推广。     

视觉注意机制在图像增强中的应用研究

将视觉注意机制引入到直方图构造中,并在此基础上提出了一种新的基于灰度级信息量直方图的图像增强算法.该算法利用Itti视觉注意计算模型对图像的显著性进行分析,获得全局显著图;然后,将全局显著图划分为若干等大的子区域,求取各子区域的平均显著值,并做归一化处理,得到子区域的加权统计系数;再将各子区域的灰度级加权统计值相加,得到灰度级信息量直方图;最后,依据直方图均衡化的映射函数,调整灰度级的动态范围.实验结果表明,该算法明显优于经典的GHE算法和AHE算法,具有满意的视觉效果.     

A robust infrared dim target detection method based on template filtering and saliency extraction

Dim target detection in infrared image with complex background and low signal-clutter ratio (SCR) is a significant and difficult task in the infrared target tracking system. A robust infrared dim target detection method based on template filtering and saliency extraction is proposed in this paper. The weighted gray map is obtained from the infrared image to highlight the target which is brighter than its neighbors and has weak correlation with its background. The target saliency map is then calculated by phase spectrum of Fourier Transform, so that the dim target detection could be converted to salient region extraction. The potential targets are finally extracted by combining the two maps. Moreover, position discrimination between targets in the two maps is used to exclude the false alarms and extract the targets. Experimental results on measured images indicate that our method is feasible, adaptable and robust in different backgrounds. The ROC (Receiver Operating Characteristic) curves obtained from the simulated images demonstrate the proposed method outperforms some existing typical methods in both detection rate and false alarm rate, for target detection with low SCR.     

Semiotic Aggregation in Deep Learning

Convolutional neural networks utilize a hierarchy of neural network layers. The statistical aspects of information concentration in successive layers can bring an insight into the feature abstraction process. We analyze the saliency maps of these layers from the perspective of semiotics, also known as the study of signs and sign-using behavior. In computational semiotics, this aggregation operation (known as superization) is accompanied by a decrease of spatial entropy: signs are aggregated into supersign. Using spatial entropy, we compute the information content of the saliency maps and study the superization processes which take place between successive layers of the network. In our experiments, we visualize the superization process and show how the obtained knowledge can be used to explain the neural decision model. In addition, we attempt to optimize the architecture of the neural model employing a semiotic greedy technique. To the extent of our knowledge, this is the first application of computational semiotics in the analysis and interpretation of deep neural networks.     

Infrared and visible image fusion method based on saliency detection in sparse domain

Infrared and visible image fusion is a key problem in the field of multi-sensor image fusion. To better preserve the significant information of the infrared and visible images in the final fused image, the saliency maps of the source images is introduced into the fusion procedure. Firstly, under the framework of the joint sparse representation (JSR) model, the global and local saliency maps of the source images are obtained based on sparse coefficients. Then, a saliency detection model is proposed, which combines the global and local saliency maps to generate an integrated saliency map. Finally, a weighted fusion algorithm based on the integrated saliency map is developed to achieve the fusion progress. The experimental results show that our method is superior to the state-of-the-art methods in terms of several universal quality evaluation indexes, as well as in the visual quality.     

基于视觉注意的彩色图像检索方法

基于视觉注意模型提取的特征能够反映图像高层语义的新特征,将视觉注意机制引入到图像分析领域能有效地减小语义鸿沟,获得高效的图像检索性能.根据视觉感知的特点,对Itti视觉注意模型进行了改进.采用主分量图表示亮度图,将纹理粗糙度信息融入视觉注意模型,进而提出了一种基于视觉注意空间分布特征的图像检索算法.首先由改进视觉注意模型将图像分解得到38个视觉特征图,然后采用网格平分法提取视觉特征图的空间分布信息,组成特征矢量来多层次地对图像特征进行描述,用于图像检索.实验结果表明,该算法利用基于改进注意力模型方法来提取图像空间分布特征进行图像检索,能获得较高的检索率.     

通道注意网络和模糊划分熵图割的图像去雾

针对雾图成像时变化的场景光及去雾过程中不同雾相关信息在处理上的差异性,提出了通道注意网络和模糊划分熵图割的单幅图像去雾算法。以考虑变化场景光的大气散射物理成像模型为基础,首先使用通道注意的编码解码网络来估计透射率,并在编码器最后及解码器起始处添加通道注意模块,以便为编码器提取的不同雾相关特征图分配不同的权重,准确地计算透射率;然后利用所提出的模糊划分熵图割算法将透射率划分为不同场景光覆盖下的近景、中景、远景,此分割策略将考虑空间相关性的图割算法与模糊划分熵的阈值分割算法相结合,解决了单一阈值分割算法产生的区域误分问题;最后估计场景光和大气光,得到去雾图像。实验结果表明,算法在合成雾图及真实雾图上均有较好的去雾效果。与已有的去雾算法相比,本文算法在峰值信噪比及结构相似性上均有提升,单张图像的平均处理时间为3.9 s。     

Saliency detection via two-directional 2DPCA analysis of image patches

Salient object detection is an important and challenging problem in computer vision. In this paper, we present a model of salient region detection based on the fusion of contrast and distribution, computed by two-directional 2DPCA analysis of image patches under the combination of RGB space, LAB space and YCbCr space. First, non-overlap patches of three layers from the image are obtained in the three color spaces respectively and stacked for the combination of the three sapces in a single layer. For every layer, two-directional, two-dimensional PCA are utilized to realize automatic selection of effective features, then based on the high contrast and compact character of salient object, contrast values and distribution values of image patches are fused to get the saliency map. Finally, three saliency maps for three layers are combined to detect salient object. The experimental results on a publicly available database show that the proposed algorithm performs well and are in line with the human eye observation results.     

基于语义先验和几何约束的动态场景SLAM算法北大核心CSCD

实际场景中运动物体的特征点加入到相机位姿计算中,以及静态环境特征点过度稀疏都会导致移动机器人传统视觉同步定位与地图构建(simultaneous localization and mapping,SLAM)算法在位姿估计时精度低、鲁棒性差。设计了基于分支空洞卷积的双边语义分割算法,将环境区分为潜在运动区域和静态区域;结合几何约束进行静态特征点的二次判断及对没有先验动态标记而具有移动性的特征点的判断,并在事先均匀提取的全部特征点中进行移除,只应用静态特征点求解相机位姿和构建静态环境地图。在TUM公共数据集上进行实验,验证了提出算法在动态环境中SLAM的定位精度明显优于现有其他方法。在存在运动物体的真实环境下进行建图实验,与ORB-SLAM2算法进行对比,本文算法在动态场景中构建的地图更清晰。     

基于视觉注意力模型的背景斑点提取方法

针对传统人工选取背景斑点的不足,提出了一种基于视觉注意力模型的背景斑点提取方法。首先根据Itti显著图模型确定背景图中的显著区域;然后找出若干注意视点,并将这些视点作为种子点在原图上进行区域生长;最后在区域生长的基础上依据相关准则提取出斑点。实验结果表明,该方法可以准确地找到背景图中的明显斑点,使进一步的背景斑点分析成为可能。     

Moving target detection approach based on spatio-temporal salient perception

The differences in texture and motion between man-made object and natural scene are the key features for human biological visual system to detect moving object in scenery. The paper proposed a moving target detection approach based on spatio-temporal perception, which is a crucial function of the visual attention mechanism. The spatial feature including edge, orientation, texture and contrast of the image are extracted, and then the corresponding spatial salient map are constructed by fusing the features through difference of Gaussian (DOG) function, which can suppress the common and enhance the difference of local region. Then, the global motion, local motion and relative motion between continuous images are extracted by means of pyramid multi-resolution, and the moving salient map is constructed after the motion difference between moving target and background is confirmed. Finally, the spatio-temporal salient map is constructed by fusing the spatial salient map and the moving salient map through competition strategy, and the moving target could be detected by searching the maximum in the spatio-temporal salient map. Some experiments are included and the results show that the method can accurately detect the moving target in complex background.     

No-Reference Quality Assessment for 3D Synthesized Images Based on Visual-Entropy-Guided Multi-Layer Features Analysis

Multiview video plus depth is one of the mainstream representations of 3D scenes in emerging free viewpoint video, which generates virtual 3D synthesized images through a depth-image-based-rendering (DIBR) technique. However, the inaccuracy of depth maps and imperfect DIBR techniques result in different geometric distortions that seriously deteriorate the users’ visual perception. An effective 3D synthesized image quality assessment (IQA) metric can simulate human visual perception and determine the application feasibility of the synthesized content. In this paper, a no-reference IQA metric based on visual-entropy-guided multi-layer features analysis for 3D synthesized images is proposed. According to the energy entropy, the geometric distortions are divided into two visual attention layers, namely, bottom-up layer and top-down layer. The feature of salient distortion is measured by regional proportion plus transition threshold on a bottom-up layer. In parallel, the key distribution regions of insignificant geometric distortion are extracted by a relative total variation model, and the features of these distortions are measured by the interaction of decentralized attention and concentrated attention on top-down layers. By integrating the features of both bottom-up and top-down layers, a more visually perceptive quality evaluation model is built. Experimental results show that the proposed method is superior to the state-of-the-art in assessing the quality of 3D synthesized images.