Deeply fusing multi-model quality-aware features for sophisticated human activity understanding

Detecting and understanding human action under sophisticated lighting condition and backgrounds, also known as human action recognition in real-world context, is an indispensable component in modern intelligent systems and has becoming a hot research topic currently. Nowadays, human action recognition is still a tough challenge due to intra-class and inter-class, environment and temporal-level differences of the same action. Algorithms based on the single visual channel cannot achieve satisfactory performance. Thus, in this paper, we propose a novel action recognition framework towards sophisticated activity understanding, focusing on intelligently combining multimodel quality-related action features. Specifically, we first design a multi-channel feature fusion (MCFF) algorithm to capture visual appearance, motion and acoustic patterns from each video frame, where image-level labels are characterized by choosing high quality multimodel features. Subsequently, we design an adaptive key frame selection algorithm that can be applied to characterize human action from human action video stream. Thereafter, we engineer a multimodel feature based on an auxiliary human action retrieval system to achieve sophisticated activity understanding. Extensive experimental evaluations have demonstrated that the effectiveness and robustness of our proposed method.     

LSCN:一种用于动作识别的长短时序关注网络

相较于图像分析,如何分析时序信息是动作识别中的一个主要问题.大多数先前的方法,如3D卷积网络、双流卷积网络,仅使用包含全局时域信息的特征作为视频的表征,忽略了局部时序特征的重要性.考虑到这样的问题,本文提出一种基于时序交互感知模块的长短时序关注网络——Long and Short Sequence Concerned Networks（LSCN）,融合不同的时序信息,利用不同卷积层时序特征的交互加强对不同时序长度的动作实例的表示,兼顾长短动作实例对时序信息的需求.实验结果表明,基于3D ResNext101的LSCN在两个公共数据集（UCF101和HMDB51）上,相较于基础的网络分别有0.4%和2.9%的准确率提升.     

Human–robot interaction based on gesture and movement recognition

Human–robot interaction (HRI) has become a research hotspot in computer vision and robotics due to its wide application in human–computer interaction (HCI) domain. Based on the explored algorithms of gesture recognition and limb movement recognition in somatosensory interaction, an HRI model of a robotic arm is proposed for robot arm manipulation. More specifically, 3D SSD architecture is used for the location and identification of gesture and arm movement. Then, DTW template matching algorithm is adopted to trace the dynamic gestures. The interactive scenarios and interactive modes are designed for experiment and implementation. Virtual interactive experimental results have demonstrated the usefulness of our method.     

Multi-modality medical image fusion based on separable dictionary learning and Gabor filtering

Sparse representation (SR) has been widely used in image fusion in recent years. However, source image, segmented into vectors, reduces correlation and structural information of texture with conventional SR methods, and extracting texture with the sliding window technology is more likely to cause spatial inconsistency in flat regions of multi-modality medical fusion image. To solve these problems, a novel fusion method that combines separable dictionary optimization with Gabor filter in non-subsampled contourlet transform (NSCT) domain is proposed. Firstly, source images are decomposed into high frequency (HF) and low frequency (LF) components by NSCT. Then the HF components are reconstructed sparsely by separable dictionaries with iterative updating sparse coding and dictionary training. In the process, sparse coefficients and separable dictionaries are updated by orthogonal matching pursuit (OMP) and manifold-based conjugate gradient method, respectively. Meanwhile, the Gabor energy as weighting factor is utilized to guide the LF components fusion, and this further improves the fusion degree of low-significant feature in the flat regions. Finally, the fusion components are transformed to obtain fusion image by inverse NSCT. Experimental results demonstrate the more competitive results of the proposal, leading to the state-of-art performance on both visual quality and objective assessment.     

人体胃肠道智能仿生系统设计

消化系统是人体构成的重要系统之一,食物的消化与吸收给人体源源不断地提供能量,促使其他系统正常运转,这是一个相当繁琐的过程,因此所产生的胃肠道疾病也是错综复杂,传统的胃肠道疾病的诊断和治疗主要采用胃肠道内窥镜,这个过程是相当痛苦的。而传统的医学研究大多采用活体(小白鼠)进行实验,但这又不符合人道主义。所以本文建立了一个能够在离体条件下直观地模拟胃肠道工作的系统。     

基于稀疏表示的红外空中目标分类算法

针对红外空中目标,提出了一种基于稀疏表示的快速分类算法.该工作的技术难点表现在训练样本较少,算法需要具有旋转不变性、较高的抗噪性和实时性.针对这些难点,首先根据红外空中面目标的梯度信息和统计特性,计算出图像主方向,然后将主方向旋转至同一参考方向.接着基于稀疏表示原理,把分类问题转化为1范数最小化问题,最后用快速收敛方法得到分类结果.实验结果表明该方法能够达到98.3%的正确率,给测试图像50%的像素叠加噪声后,分类正确率仍大于80%.     

A deep genetic algorithm for human activity recognition leveraging fog computing frameworks

With modern e-healthcare developments, ambulatory healthcare has become a prominent requirement for physical or mental ailed, elderly, childhood people. One of the major challenges in such applications is timing and precision. A potential solution to this problem is the fog-assisted cloud computing architecture. The activity recognition task is performed with the hybrid advantages of deep learning and genetic algorithms. The video frames captured from vision cameras are subjected to the genetic change detection algorithm, which detects changes in activities of subsequent frames. Consequently, the deep learning algorithm recognizes the activity of the changed frame. This hybrid algorithm is run on top of fog-assisted cloud framework, fogbus and the performance measures including latency, execution time, arbitration time and jitter are observed. Empirical evaluations of the proposed model against three activity data sets shows that the proposed deep genetic algorithm exhibits higher accuracy in inferring human activities as compared to the state-of-the-art algorithms.     

Facial micro-expression recognition based on accordion spatio-temporal representation and random forests

Micro-expressions are very brief involuntary facial expressions which appear on the face of humans when they unconsciously conceal an emotion. Creating a solution allowing an automatic recognition of the facial micro-expressions from video sequences has garnered increasing attention from experts across such different disciplines as computer science, security, and psychology. This paper offered a solution to facial micro-expressions recognition, based on accordion spatio-temporal representation and Random Forests. The proposed feature space, called “Uniform Local Binary Patterns on an Accordion 2D representation of sub-regions presented by a Pyramid of levels (LBPAccP^u2)”, exploits the effectiveness of uniform LBP patterns applied on an accordion representation of sub-regions at different sizes. Random Forests were used to select the most discriminating features and reduce the classification ambiguity of similar micro-expressions through a new proximity measure. The main objective of our paper was to demonstrate that the use of few features could be more efficient to produce a strong micro-expression recognition classifier that outperforms the approaches that rely on high dimensional features space. The experimental results across six micro-expression datasets show the effectiveness of the proposed solution with an accuracy rate that can reach 81.38% on CasmeII dataset. Compared to some famous competitive state-of-the-art approaches, the proposed solution proved its performance thanks to its accuracy rate as well as the number of features it uses.