Motion-based region growing segmentation of image sequences

This paper proposes a motion-based region growing segmentation scheme for the object-based video coding, which segments an image into homogeneous regions characterized by a coherent motion. It adopts a block matching algorithm to estimate motion vectors and uses morphological tools such as open-close by reconstruction and the region-growing version of the watershed algorithm for spatial segmentation to improve the temporal segmentation. In order to determine the reliable motion vectors, this paper also proposes a change detection algorithm and a multi-candidate pro- screening motion estimation method. Preliminary simulation results demonstrate that the proposed scheme is feasible. The main advantage of the scheme is its low computational load.     

一种用于MPEG-4形状编码的快速运动估计算法

该文提出了一种用于MPEG-4形状编码的快速运动估计算法。该算法利用了形状编码及形状信息的固有特性,即基于上下文的运动估计特性、相邻二值alpha块的运动矢量相关性以及形状信息的二值特性。模拟结果表明,该算法具有运算量少、处理速度快的特点,适用于MPEG-4形状编码的实时软件实现。     

图像的区域描述方法及运动估计

文中提出一种任意形状区域的图像描述方法进行图像序列的运动估计。首先,对初始帧图像采用保边界平滑,然后,对平滑帧进行分水岭分割。根据标记图像,将图像以区域邻接图的方式进行存储。为解决过度分割的问题,给出了区域的灰度特性和边界强度及区域尺寸相结合的融合准则。最后,每个区域采用仿射运动模型及非线性最小均方法进行参数优化。得到的运动矢量场同基于像素点的估计方法相比具有更高的一致性和抗干扰性,运动补偿差图像的峰值信噪比得以提高。     

A simple and efficient block motion estimation algorithm based on full-search array architecture

This paper first presents an array structure using ±1 full-search (FS) architecture as the search engine of block motion estimation which takes advantage of the design regularity of FS. An efficient algorithm named modified gradient-descent search (MGDS) is then introduced based on the proposed architecture. MGDS utilizes an adaptive computation distribution mechanism to efficiently allocate computation of the employed ±1 FS array to blocks or frames of video sequences. Experimental results indicate that MGDS uniformly achieves a higher quality than FS by an amount that is dependent on motion activities of sequences.     

Fast motion estimation for H.264

Several specific features have been incorporated into Motion estimation (ME) in H.264 coding standard to improve its coding efficiency. However, they result in very high computational load. In this paper, a fast ME algorithm is proposed to reduce the computational complexity. First, a mode discriminant method is used to free the encoder from checking the small block size modes in homogeneous regions. Second, a condensed hierarchical block matching method and a spatial neighbor searching scheme are employed to find the best full-pixel motion vector. Finally, direction-based selection rule is utilized to reduce the searching range in sub-pixel ME process. Experimental results on commonly used QCIF and CIF format test sequences have shown that the proposed algorithm achieves a reduction of 88% ME process time on average, while incurring only 0.033 dB loss in PSNR and 0.50% increment on the total bit rate compared with that of exhaustive ME process, which is a default approach adopted in the JM reference software.     

Spatiotemporal texture synthesis and region-based motion compensation for video compression

In this paper, a content-based approach for video compression is proposed. The main novelty relies on the complete texture analysis/synthesis framework, which enables the use of multiple algorithms, depending on texture characteristics. The idea comes from the efficient MPEG prediction based on a best mode selection. Existing synthesis algorithms cannot be efficient in synthesizing every kind of texture but a certain range of them.This approach is designed to be jointly used with current and future standard compression schemes. At encoder side, texture analysis includes segmentation and characterization tools, in order to localize candidate regions for synthesis: motion compensation or texture synthesis. The corresponding areas are not encoded. The decoder fills them using texture synthesis. The remaining regions in images are classically encoded. They can potentially serve as input for texture synthesis. The chosen tools are developed and adapted in order to ensure the coherency of the whole scheme. Thus, a texture characterization step provides required parameters to the texture synthesizer. Two texture synthesizers, including a pixel-based and a patch-based approach, are used on different types of texture, complementing each other.The scheme is coupled with a motion estimator in order to segment coherent regions and to interpolate rigid motions using an affine model. Inter frame adapted synthesis is therefore used for non-rigid texture regions.The framework has been validated within an H.264/MPEG4-AVC video codec. Experimental results show significant bit-rate saving at similar visual quality levels, assessed using subjective tests. The method can be coupled with the future HEVC in which blocks can be skipped by the encoder to be synthesized at decoder side.     

Motion compensation for very low bit-rate video

A combination of spatial transformation and image segmentation is used to compensate for non-uniform intensity changes in moving scenes. The method efficiently tracks movements such that the motion vectors alone can be employed to represent a moving object with complex motion. Using fast transformation and interpolation algorithms, it is shown that while the compression efficiency of the presented method is far superior to that of the conventional full search block-matching motion estimation, its computational complexity is still affordable.     

Homography-based block motion estimation for video coding of PTZ cameras

Due to the constrained movement of pan-tilt-zoom (PTZ) cameras, two frames in the video sequences captured by such cameras can be geometrically related by a relationship (homography). This geometric relationship is helpful for reducing the spatial redundancy in video coding. In this paper, by exploiting the homography between two frames with optical flow tracking algorithm, we propose a novel homography-based search (HBS) algorithm for block motion estimation in coding the sequences captured by PTZ cameras. In addition, adaptive thresholds are adopted in our method to classify different kinds of blocks. Compared with other traditional fast algorithms, the proposed HBS algorithm is proved to be more efficient for the sequences captured by PTZ cameras. And compared to our previous work in ICME (Cui et al., 2011), which only deals with pan-tilt (PT) camera and calculates the homography with mechanical devices, in this extended work we compute the homography by using information on images instead.     

Multi-resolution motion estimation and compensation based on adjacent prediction of frame difference in wavelet domain

Aiming at the higher bit-rate occupation of motion vector encoding and more time load of full-searching strategies,a multi-resolution motion estimation and compensation algorithm based on adjacent prediction of frame difference was proposed.Differential motion detection was employed to image sequences and proper threshold was adopted to identify the connected region.Then the motion region was extracted to carry out motion estimation and motion compensation on it.The experiment results show that the encoding efficiency of motion vector is promoted,the complexity of motion es timation is reduced and the quality of the reconstruction image at the same bit-rate as Multi-Reso lution Motion Estimation (MRME) is improved.     

一种减少块匹配运动估值象素点的方法

在分析了块匹配运动估值模型的基础上，指同处于块中央的象素眯在块匹配时作用 小，最了一种减少块匹配运算象素的方法。不仅如此，提出的方法可与任一基于减少匹配次数的快速算法相结合，从而得到各种事型的块匹配快速算法。最后试验结果，提出了的方法是有效的。     

Fuzzy quantization based bit transform for low bit-resolution motion estimation

This study proposes a novel fuzzy quantization based bit transform for low bit-resolution motion estimation. We formalize the procedure of bit resolution reduction by two successive steps, namely interval partitioning and interval mapping. The former is a many-to-one mapping which determines motion estimation performance, while the latter is a one-to-one mapping. To gain a reasonable interval partitioning, we propose a non-uniform quantization method to compute coarse thresholds. They are then refined by using a membership function to solve the mismatch of pixel values near threshold caused by camera noise, coding distortion, etc. Afterwards, we discuss that the sum of absolute difference (SAD) is one of the fast matching metrics suitable for low bit-resolution motion estimation in the sense of mean squared errors. A fuzzy quantization based low bit-resolution motion estimation algorithm is consequently proposed. Our algorithm not only can be directly employed in video codecs, but also be applied to other fast or complexity scalable motion estimation algorithms. Extensive experimental results show that the proposed algorithm can always achieve good motion estimation performances for video sequences with various characteristics. Compared with one-bit transform, multi-thresholding two-bit transform, and adaptive quantization based two-bit transform, our bit transform separately gains 0.98 dB, 0.42 dB, and 0.24 dB improvement in terms of average peak signal-to-noise ratio, with less computational cost as well.     

A new distortion measure for motion estimation in motion-compensated hybrid video coding

In the video coding standards MPEG-x and H.26x, a motion-compensated prediction technique is used for enhancing the coding performance of bitrate reduction or peak signal to noise ratio (PSNR) improvement. This technique takes advantage of the correlation between consecutive frames in the time domain, which is relatively higher than that between adjacent blocks in the spatial domain. In order to utilize the correlation between consecutive frames, the conventional video coding standards have used the motion estimation (ME) and compensation technique, where the Sum of the Absolute Differences (SAD) is usually used as the distortion measure. The ME estimates the reference block that could minimize the residual signal between the current and reference blocks. However, the SAD is not appropriate to the specific sequences that have global or local illumination changes. In addition, the high-resolution video sequences have higher spatial correlation than the low-resolution video sequences in general. Therefore, a new distortion measure that can consider spatial and temporal correlation simultaneously may be helpful to enhance the coding performance. The proposed distortion measure searches for a reference block that minimizes the motion-compensated residual signal when the DC-component is predicted. In our proposed algorithm, the maximum BD-rate improvement is up to 13.6% for illumination-changed video sequences, and the average BD-rate improvement is 6.6% for various high-resolution video sequences in the baseline profile.     

Small-diamond-based search algorithm for fast block motion estimation

A good fast motion search algorithm should efficiently speed up the encoding time and keep the quality of encoded video stable at the same time. Researches have shown that many fast algorithms lose the quality requirement in some special video sequences. These video sequences often have heavy motions and need large search windows for motion vector search. E3SS, DS, and E-HEXBS, which are famous algorithms, are not good enough in these sequences. As to UMHexagonS, it is able to meet the high video quality requirement very well, but it costs too much computation. This paper introduces a multi-stage motion estimation algorithm. The algorithm ensures getting good video quality while decreases the motion search time efficiently. It divides the search regions into many un-overlapped small-diamond regions and forces the motion search to go outward for larger motion vectors. This method is also designed to avoid mistaking local optimal motion vectors. For this reason, the selected motion vector is refined by several stages. Experimental results show that the proposed algorithm uses almost the same number of checking points as E3SS but achieves a better quality. Furthermore, the proposed algorithm is also tested in H.264/AVC JM9.5 encoder; the experimental results show that this algorithm is also suitable for variable block-size motion estimation.     

基于可变块运动矢量的边信息生成算法*

在分布式视频编码系统中,针对图像中细节丰富的区域易造成严重的块效应,提出了一种基于可变块运动矢量的边信息生成算法。根据前后相邻关键帧对应块的相关性,将像素块分为保留块和运动块。对保留块直接作保留处理,对运动块中的像素块继续进行分割并计算子块的初始运动矢量,最后将所有对应块的运动矢量进行加权自适应运动补偿得到改进的边信息。实验结果表明,对于运动较剧烈复杂的视频序列,该算法能够提高边信息生成质量, 并且使得改进后的边信息PSNR值提高了0.98~1.33 dB。     

MOTION COMPENSATION BASED ON MOTION VECTOR INTERPOLATION

A modified block matching algorithm (BMA) with motion correlation constraint (MCCBMA) is proposed at first. Then a novel motion compensation algorithm (INTPMC) which computes motion vector for each pixel by interpolating motion vectors is presented. In order to increase interframe prediction performance and decrease the computational complexity, an optimizing algorithm for partial motion vectors is described at last. Experimental results show that the proposed algorithm can improve the prediction performance obviously with a moderately increased complexity compared with the conventional full search block matching algorithm (FSBMA).     

一种适用于AVS-M的自适应快速运动估计算法

为了减小移动视频编码标准(AVS-M)中运动估计模块的复杂度,提出了一种快速、有效的块匹配运动估计算法.该算法充分利用了视频图像中运动矢量场的中心偏置特性和时空相关性,根据运动类型自适应的选择搜索起点和搜索策略,结合改进的搜索模板和高效搜索中止准则,有效地降低了运动估计的运算量.实验结果表明,该算法在保证搜索精度的同时,大大减少了搜索点数.     

一种提高H.261编码性能的实现方法

研究表明采用最小均方误差或绝对误差准则的块匹配运动估值算法对H.261编码器来说不是最好的。本文提出了一种改进的块匹配运动估值算法。它所采用的准则不仅考虑了预测误差能量的大小,还考虑了运动矢量信息以及帧间预测误差的编码比特数目的多少.实验结果表明新的准则能够显著地改善H.261的编码性能。     

A coding scheme using global motion estimation for AVS P-frame

Block matching has been used for motion estimation and motion compensation in the Audio and Video Standard （AVS） for years. While having an acceptable performance in describing motion between frames, it requires quite a few bits to represent the motion vectors. In certain circumstances, the use of global motion estimation and compensation would perform equally well or even better than the block matching in terms of motion accuracy, while it results in the coding of global motion model parameters. In this letter, we modify an AVS coder by adding （1） six global motion model parameters to the frame header, and （2） mode selection among INTRA, SKIP, INTER-16×16, INTER-16×8, INTER-8×16, INTER-8×8, and Global Motion Compensation （GMC） modes by Lagrange optimal rate-distortion criteria. Simulation results demonstrate that over 0.1dB improvement in PSNR is obtained compared to the AVS coder for an average coded P-frame with the same bitrate.     

一种新的自适应二值运动估计算法

利用增强位平面的二值特性和运动矢量场自适应思想,提出了一种新的自适应二值运动估计算法.算法在图像的位平面中完成运动矢量的搜索;匹配时对静止块进行判定直接中止搜索,同时采用了起始点预测和运动类型判定,自适应使用LDSP和SDSP模板进行搜索.实验结果表明:该算法的搜索速度优于绝大多数现有的运动估计算法,且预测精度接近于FS算法.     

Global-motion estimation in image sequences of 3-D scenes for coding applications

A technique for global-motion estimation and compensation in image sequences of 3-D scenes is described in this paper. Each frame is segmented into regions whose motion can be described by a single set of parameters and a set of motion parameters is estimated for each segment. This is done using an iterative block-based image segmentation combined with the estimation of the parameters describing the global motion of each segment. The segmentation is done using a Gibbs-Markov model-based iterative technique for finding a local optimum solution to a maximum a posteriori probability (MAP) segmentation problem. The initial condition for this process is obtained by applying a Hough transform to the motion vectors of each block in the frame obtained by block matching. In each iteration, given a segmentation, the motion parameters are estimated using the least-squares (LS) technique. To obtain the final segmentation and the more appropriate higher-order motion model for each segment, a final stage of splitting/merging of segments is needed. This step is performed on the basis of maximum-likelihood decisions combined with the determination of the higher-order model parameters by LS. The incorporation of the proposed global-motion estimation technique in an image-sequence coder was found to bring about a substantial reduction in bit-rate without degrading the perceived quality or the PSNR.