An in-depth analysis of hyperspectral target detection with shadow compensation via LiDAR

Shadowy areas present a big hindrance in target detection from HSI, as the reflectance data received from target materials is significantly diminished when measured from shadowy areas. In this work, we perform an in-depth analysis of hyperspectral target detection on targets in full illumination and in partial or full shadows; and analyze how much target detection can be improved if the hyperspectral data is corrected at the regions of shadows. To do this, first, we detect the shadows using LiDAR, and propose a way to correct them in the hyperspectral image using the physical radiance model. Then, using three target detectors, namely the spectral angle mapper (SAM), adaptive coherence estimator (ACE) and matched filter (MF), we compare the results of target detection with and without shadow correction. We analyze our results based on the target material (red felt and blue felt targets), the background (grass or gravel), the amount of shadow (partial or full) and based on the time of the data collection (in the morning or at noon). Our results indicate several interesting observations: (i) the red-felt material is much harder to detect than the blue-felt material even though they are made up of the same material; but this gap in detection decreases significantly if shadow correction is performed using the radiance model, (ii) both the red-felt and the blue-felt targets are hard to detect earlier in the day when the rays from the sun are inclined; but there is not a significant difference in making the data collection in the morning or at noon if shadow correction is performed, and (iii) the shadow compensation dramatically increases the detection rates and boosts up the area under the receiver operating curve (AUC) from around 0,7-0,9 band to the 0,95-1,00 band.In addition, we provide our shadow detection code¹, sky-view factor results and all the MODTRAN outputs for the parts of the Share2012 dataset used in this work. In doing so, we hope to provide a benchmark for researchers who would like to test their target detection or shadow correction algorithms on HSI-LiDAR data.     

Shadow detection via multi-scale feature fusion and unsupervised domain adaptation

Shadow detection is significant for scene understanding. As a common scenario, soft shadows have more ambiguous boundaries than hard shadows. However, they are rarely present in the available benchmarks since annotating for them is time-consuming and needs expert help. This paper discusses how to transfer the shadow detection capability from available shadow data to soft shadow data and proposes a novel shadow detection framework (MUSD) based on multi-scale feature fusion and unsupervised domain adaptation. Firstly, we set the existing labeled shadow dataset (i.e., SBU) as the source domain and collect an unlabeled soft shadow dataset (SSD) as the target domain to formulate an unsupervised domain adaptation problem. Next, we design an efficient shadow detection network based on the double attention module and multi-scale feature fusion. Then, we use the global–local feature alignment strategy to align the task-related feature distributions between the source and target domains. This allows us to obtain a robust model and achieve domain adaptation effectively. Extensive experimental results show that our method can detect soft shadows more accurately than existing state-of-the-art methods.     

Multi-scale spatial–temporal attention graph convolutional networks for driver fatigue detection

With the development of deep learning, fatigue detection technology for drivers has achieved remarkable achievements. Although the image-based approach achieves good accuracy, it inevitably leads to greater model complexity, which is unsuitable for mobile terminal devices. Luckily, human skeletal data significantly reduces the impact of noise and input data volume while retaining valid information, and it can better deal with real-world driving scenarios with the benefit of robustness in complex driving situations. This paper proposes a lightweight multi-scale spatio-temporal attention graph convolutional network (MS-STAGCN) to efficiently utilize skeleton data to identify driver states by aggregating locally and globally valid face information, which achieves good performance even for lightweight design. The experimental results show that the method achieves 92.4% accuracy on the NTHU-DDD dataset, which can be applied to fatigue detection tasks of the driver in real-world driving scenarios in the future.     

SDRNet: An end-to-end shadow detection and removal network

Image shadow detection and removal can effectively recover image information lost in the image due to the existence of shadows, which helps improve the accuracy of object detection, segmentation and tracking. Thus, aiming at the problem of the scale of the shadow in the image, and the inconsistency of the shadowed area with the original non-shadowed area after the shadow is removed, the multi-scale and global feature (MSGF) is used in the proposed method, combined with the non-local network and dense dilated convolution pyramid pooling network. Besides, aiming at the problem of inaccurate detection of weak shadows and complicated shape shadows in existing methods, the direction feature (DF) module is adopted to enhance the features of the shadow areas, thereby improving shadow segmentation accuracy. Based on the above two methods, an end-to-end shadow detection and removal network SDRNet is proposed. SDRNet completes the task of sharing two feature heights in a unified network without adding additional calculations. Experimental results on the two public datasets ISDT and SBU demonstrate that the proposed method achieves more than 10% improvement in the BER index for shadow detection and the RMSE index for shadow removal, which proves that the proposed SDRNet based on the MSGF module and DF module can achieve the best results compared with other existing methods.     

基于相对生成对抗网络的低清小脸幻构

针对当前代表性低清小脸幻构方法存在的视觉真实感弱、网络结构复杂等问题,提出了一种基于相对生成对抗网络的低清小脸幻构方法(tfh-RGAN).该文方法的网络架构包括幻构生成器和判别器两个部分,通过像素损失函数和相对生成对抗损失函数的联合最小化,实现生成器和判别器的交替迭代训练.其中,幻构生成器结合了残差块、稠密块以及深度...     

基于多通道多尺度卷积神经网络的单幅图像去雨方法

雨天等恶劣天气会严重影响到图像成像质量,从而影响到视觉处理算法的性能。为了改善雨天图像的成像质量,该文提出一种基于多通道多尺度卷积神经网络的去雨算法,建立了多通道多尺度卷积神经网络结构来提取雨线特征。首先利用小波阈值引导的双边滤波将有雨图像进行分解,得到高频雨线图像和轮廓保持度高的低频背景图像。然后为了使图像高频部分的雨线信息更为明显,减少雨线特征学习时高频图像中的背景误判,将得到的高频雨线图像再一次通过滤波器得到减弱背景信息同时增强雨线信息的到更高频雨线图像。其次针对低频背景图像上也残留了大量雨痕,该文提出将低频背景图像和更高频雨线图像一起送入卷积神经网络进行特征学习,其中对图像提取的是多尺度特征信息,最后得到雨线去除更彻底的复原图像。同时在构造网络模型时利用空洞卷积代替标准卷积来提取图像的特征信息,得到更丰富的图像特征,提高了算法的去雨性能。从实验结果可以看出去雨之后的图像清晰,细节保持度较高。     

Moving cast shadow detection using online sub-scene shadow modeling and object inner-edges analysis

In this paper, we propose an adaptive and accurate moving cast shadow detection method employing online sub-scene shadow modeling and object inner-edges analysis for applications of static-camera video surveillance. To describe shadow appearance more accurately, the proposed method builds adaptive online shadow models for sub-scenes with different conditions of irradiance and reflectance. The online shadow models are learned by utilizing Gaussian functions to fit the significant peaks of accumulating histograms, which are calculated from Hue, Saturation and Intensity (HSI) difference of moving objects between background and foreground. Additionally, object inner-edges analysis is adopted to reject camouflages, which are misclassified foreground regions that are highly similar to shadows. Finally, the main shadow regions are expanded to recycle the misclassified shadow pixels based on local color constancy. The proposed algorithm can adaptively handle the shadow appearance changes and camouflages without prior information about illuminations and scenarios. Experimental results demonstrate that the proposed method outperforms state-of-the-art methods.     

Saliency detection via cross-scale deep inference

The small, moderate, and large scale saliency patterns in images are valuable to be extracted in saliency detection. By the observation that the probability of small and large saliency patterns appearing in datasets is lower than that of moderate scale saliency patterns. As results, a deep saliency model trained on such datasets would converge to moderate scale saliency patterns, and it is hard to well infer the small and large scale saliency patterns because they are not encoded efficiently in the model for their low probability. Thus a novel but simple saliency detection method using cross-scale deep inference is presented in this paper. Moreover, a new network architecture, in which the attention mechanism is exploited by multiple layers, is proposed to improve the receptive fields of various scale saliency patterns in different scale images. The presented cross-scale deep inference could improve the representation power of small and large scale saliency patterns encoded in multiple scale images efficiently. The quantitative and qualitative evaluation demonstrates our deep model achieves a promising results across a wide of metrics.     

Research on the parallelization of image quality analysis algorithm based on deep learning

Image quality assessment is an indispensable in computer vision applications, such as image classification, image parsing. With the development of Internet, image data acquisition becomes more conveniently. However, image distortion is inevitable due to imperfect image acquisition system, image transmission medium and image recording equipment. Traditional image quality assessment algorithms only focus on low-level visual features such as color or texture, which could not encode high-level features effectively. CNN-based methods have shown satisfactory results in image quality assessment. However, existing methods have problems such as incomplete feature extraction, partial image block distortion, and inability to determine scores. So in this paper, we propose a novel framework for image quality assessment based on deep learning. We incorporate both low-level visual features and high-level semantic features to better describe images. And image quality is analyzed in a parallel processing mode. Experiments are conducted on LIVE and TID2008 datasets demonstrate the proposed model can predict the quality of the distorted image well, and both SROCC and PLCC can reach 0.92 or higher.     

Water leakage image recognition of shield tunnel via learning deep feature representation

With the development of urban metro, the research on structural diseases of shield tunnels has been becoming a hot research topic, especially the leakage water diseases. Deep learning-based algorithms have shown impressive performance in image processing domain, such as image classification, image recognition or image retrieval. In this paper, we propose a novel image recognition algorithm for water leakage diseases of shield tunnels based on deep learning algorithm. Water leakage images are classified into six categories, each of which are extracted deep representation for image recognition. We compare our method with Otsu algorithm (OA), Region Growing Algorithm (RGA), and Watershed Algorithm (WA) to show the effectiveness of our proposed method.     

A comparative study between single and multi-frame anomaly detection and localization in recorded video streams

Video anomaly detection is usually studied by considering the spatial and temporal contexts. This paper focuses first on spatial context and shows that it can be a fast real-time solution. In the first part of this work there are two main contributions: employing a new deep network for reconstruction and introducing a new regularity scoring function. The new deep architecture is based on pyramid of input images and compared to UNet, the proposed architecture boosts AUC by 15% and the new regularity scoring function is based on SSIM. The second part employs a multiframe approach to distinguish temporal behavior anomalies. The second approach enhances the results by 7% compared to spatial anomaly detection. Comparing the two approaches, if computing power is limited and real time anomaly detection is looked for, single frame detection is preferred while multi frame analysis offers a much wider possibility of anomaly detection.     

Generative image inpainting via edge structure and color aware fusion

Very recently, with the widespread research of deep learning, its achievements are increasingly evident in image inpainting tasks. However, many existing methods fail to effectively reconstruct vivid contents and refine structures. In order to solve this issue, in this paper, a novel two-stage generative adversarial network based on the fusion of edge structures and color aware maps is proposed. In the first-stage network, edges with missing regions are employed to train an edge structure generator. Meanwhile, the input image with missing regions is transformed into a global color feature map after the content aware fill algorithm and a large kernel size Gaussian filtering. In the second-stage network, the image fused from the edge map and the color map is used as a label to guide the network to reconstruct the refined image. Qualitative and quantitative experiments conducted on multiple public datasets demonstrate that the method proposed in this paper has superior performance.     

Attention mechanism enhancement algorithm based on cycle consistent generative adversarial networks for single image dehazing

This paper proposes AMEA-GAN, an attention mechanism enhancement algorithm. It is cycle consistency-based generative adversarial networks for single image dehazing, which follows the mechanism of the human retina and to a great extent guarantees the color authenticity of enhanced images. To address the color distortion and fog artifacts in real-world images caused by most image dehazing methods, we refer to the human visual neurons and use the attention mechanism of similar Horizontal cell and Amazon cell in the retina to improve the structure of the generator adversarial networks. By introducing our proposed attention mechanism, the effect of haze removal becomes more natural without leaving any artifacts, especially in the dense fog area. We also use an improved symmetrical structure of FUNIE-GAN to improve the visual color perception or the color authenticity of the enhanced image and to produce a better visual effect. Experimental results show that our proposed model generates satisfactory results, that is, the output image of AMEA-GAN bears a strong sense of reality. Compared with state-of-the-art methods, AMEA-GAN not only dehazes images taken in daytime scenes but also can enhance images taken in nighttime scenes and even optical remote sensing imagery.     

基于深度卷积神经网络的红外图像行人检测

红外图像中的行人检测一直是计算机视觉领域的研究热点与难点。针对传统的红外行人检测方法需要人工设计目标表达特征的弊端,本文从深度学习的角度出发,提出一种可以自动构建目标表达特征的红外行人检测卷积神经网络。在对卷积神经网络的实现原理进行分析的基础上,设计了红外行人检测卷积神经网络的初始结构,然后通过实验对初始结构进行调整,得到最终的检测神经网络。对实拍红外人体数据库进行行人检测的实验结果表明,该方法在保持低虚警率的同时可以对红外图像中的行人进行稳健检测,优于传统方法。     

Underwater image processing method for fish localization and detection in submarine environment

Object detection is an important process in image processing, it aims to detect instances of semantic objects of a certain class in digital images and videos. Object detection has applications in many areas of computer vision such as underwater fish detection. In this paper we present a method for preprocessing and fish localization in underwater images. We are based on a Poisson–Gauss theory, because it can accurately describe the noise present in a large variety of imaging systems. In the preprocessing step we denoise and restore the raw images. These images are split into regions utilizing the mean shift algorithm. For each region, statistical estimation is done independently in order to combine regions into objects. The method is tested under different underwater conditions. Experimental results show that the proposed approach outperforms state of the art methods.     

High compression image coding via directional filtering

A new image coding technique is presented as derived from an image decomposition into a low frequency component and many high frequency directional components. The directional filters and their properties are introduced. Then the implementation of the directional decomposition and the selection of the information to be coded are described. The combination of transform domain coding of the low frequency component and spatial domain coding of the directional components led to acceptable results with compression ratios higher than 30 to 1.     

Salient object detection in video using deep non-local neural networks

Detection of salient objects in image and video is of great importance in many computer vision applications. In spite of the fact that the state of the art in saliency detection for still images has been changed substantially over the last few years, there have been few improvements in video saliency detection. This paper proposes a novel non-local fully convolutional network architecture for capturing global dependencies more efficiently and investigates the use of recently introduced non-local neural networks in video salient object detection. The effect of non-local operations is studied separately on static and dynamic saliency detection in order to exploit both appearance and motion features. A novel deep non-local fully convolutional network architecture is introduced for video salient object detection and tested on two well-known datasets DAVIS and FBMS. The experimental results show that the proposed algorithm outperforms state-of-the-art video saliency detection methods.     

Single image deraining using multi-scales context information and attention network

The existing deraining methods based on convolutional neural networks (CNNs) have made great success, but some remaining rain streaks can degrade images drastically. In this work, we proposed an end-to-end multi-scale context information and attention network, called MSCIANet. The proposed network consists of multi-scale feature extraction (MSFE) and multi-receptive fields feature extraction (MRFFE). Firstly, the MSFE can pick up features of rain streaks in different scales and propagate deep features of the two layers across stages by skip connections. Secondly, the MRFFE can refine details of the background by attention mechanism and the depthwise separable convolution of different receptive fields with different scales. Finally, the fusion of these outputs of two subnetworks can reconstruct the clean background image. Extensive experimental results have shown that the proposed network achieves a good effect on the deraining task on synthetic and real-world datasets. The demo can be available at https://github.com/CoderLi365/MSCIANet.     

Three Degree Binary Graph and Shortest Edge Clustering for re-ranking in multi-feature image retrieval

Graph methods have been widely employed in re-ranking for image retrieval. Although we can effectively find visually similar images through these methods, the ranking lists given by those approaches may contain some candidates which appear to be irrelevant to a query. Most of these candidates fall into two categories: (1) the irrelevant outliers located near to the query images in a graph; and (2) the images from another cluster which close to the query. Therefore, eliminating these two types of images from the ordered retrieval sets is expected to further boost the retrieval precision. In this paper, we build a Three Degree Binary Graph (TDBG) to eliminate the outliers and utilize a set-based greedy algorithm to reduce the influence of adjacent manifolds. Moreover, a multi-feature fusion method is proposed to enhance the retrieval performance further. Experimental results obtained on three public datasets demonstrate the superiority of the proposed approach.     

Deep-NC: A secure image transmission using deep learning and network coding

Visual communications have played an important part in our daily life as a non-verbal way of conveying information using symbols, gestures and images. With the advances of technology, people can visually communicate with each other in a number of forms via digital communications. Recently Image Super-Resolution (ISR) with Deep Learning (DL) has been developed to reproduce the original image from its low-resolution version, which allows us to reduce the image size for saving transmission bandwidth. Although many benefits can be realised, the image transmission over wireless media experiences inevitable loss due to environment noise and inherent hardware issues. Moreover, data privacy is of vital importance, especially when the eavesdropper can easily overhear the communications over the air. To this end, this paper proposes a secure ISR protocol, namely Deep-NC, for the image communications based on the DL and Network Coding (NC). Specifically, two schemes, namely Per-Image Coding (PIC) and Per-Pixel Coding (PPC), are designed so as to protect the sharing of private image from the eavesdropper. Although the PPC scheme achieves a better performance than the PIC scheme for the entire image, it requires a higher computational complexity on every pixel of the image. In the proposed Deep-NC, the intended user can easily recover the original image achieving a much higher performance in terms of Peak Signal-to-Noise Ratio (PSNR) and Structural Similarity Index Measure (SSIM) than those at the eavesdropper. Simulation results show that an improvement of up to 32 dB in the PSNR can be obtained when the eavesdropper does not have any knowledge of the parameters and the reference image used in the mixing schemes. Furthermore, the original image can be downscaled to a much lower resolution for saving significantly the transmission bandwidth with negligible performance loss.