A multi-resolution approach for massively-parallel hardware-friendly optical flow estimation

This paper presents a novel hardware-friendly motion estimation for real-time applications such as robotics or autonomous navigation. Our approach is based on the well-known Lucas & Kanade local algorithm, whose main problem is the unreliability of its estimations for large-range displacements. This disadvantage is solved in the literature by adding the sequential multiscale-with-warping extension, although it dramatically increases the computational cost. Our choice is the implementation of a multiresolution scheme that avoids the warping computation and allows the estimation of large-range motion. This alternative allows the parallel computation of the scale-by-scale motion estimation which makes the whole computation lighter and significantly reduces the processing time compared with the multiscale-with-warping approach. Furthermore, this last fact also means reducing the hardware resource cost for its potential implementation in digital hardware devices such as GPUs, ASICs, or FPGAs. In the discussion, we analyze the speedup of the multiresolution approach compared to the multiscale-with-warping scheme. For an FPGA implementation, we obtain a reduction of latency between 40% and 50% and a resource reduction of 30%. The final solution copes with large-range motion estimations with a simplified architecture very well-suited for customized digital hardware datapath implementations as well as current multicore architectures.     

Occlusion-aware optical flow estimation

Optical flow can be reliably estimated between areas visible in two images, but not in occlusion areas. If optical flow is needed in the whole image domain, one approach is to use additional views of the same scene. If such views are unavailable, an often-used alternative is to extrapolate optical flow in occlusion areas. Since the location of such areas is usually unknown prior to optical flow estimation, this is usually performed in three steps. First, occlusion-ignorant optical flow is estimated, then occlusion areas are identified using the estimated (unreliable) optical flow, and, finally, the optical flow is corrected using the computed occlusion areas. This approach, however, does not permit interaction between optical flow and occlusion estimates. In this paper, we permit such interaction by proposing a variational formulation that jointly computes optical flow, implicitly detects occlusions and extrapolates optical flow in occlusion areas. The extrapolation mechanism is based on anisotropic diffusion and uses the underlying image gradient to preserve structure, such as optical flow discontinuities. Our results show significant improvements in the computed optical flow fields over other approaches, both qualitatively and quantitatively.     

Evaluation of differential optical flow techniques on synthesizedecho images

The performance of three methods for evaluation of motion on synthesized 2-D echo image sequences with features similar to real ones are examined. The selected techniques based on the computation of optical flow are of the differential type and assume that the image brightness pattern is constant over time. They differ in the choice of the smoothing term and in the local or global treatment of the domain. The images were synthesized by simulating the process of echo formation, considering the interaction between ultrasonic fields and human tissues. Moreover, two different approaches were followed to generate the sequences: (1) a known motion field was applied to the intensity distribution of the synthesized images; (2) a known motion field was applied directly to the point scatterer distribution of the tissue. Favorable results were obtained by applying Lucas-Kanade and Horn-Schunck techniques to the sequences of the first type, while all the techniques produced large errors when applied to the other type of sequences. A discussion about the suitability of the above-mentioned techniques for evaluation of motion on real echocardiographic images is also presented together with some results     

Errors-in-variables modeling in optical flow estimation

Gradient-based optical flow estimation methods typically do not take into account errors in the spatial derivative estimates. The presence of these errors causes an errors-in-variables (EIV) problem. Moreover, the use of finite difference methods to calculate these derivatives ensures that the errors are strongly correlated between pixels. Total least squares (TLS) has often been used to address this EIV problem. However, its application in this context is flawed as TLS implicitly assumes that the errors between neighborhood pixels are independent. In this paper, a new optical flow estimation method (EIVM) is formulated to properly treat the EIV problem in optical flow. EIVM is based on Sprent's (1966) procedure which allows the incorporation of a general EIV model in the estimation process. In EIVM, the neighborhood size acts as a smoothing parameter. Due to the weights in the EIVM objective function, the effect of changing the neighborhood size is more complex than in other local model methods such as Lucas and Kanade (1981). These weights, which are functions of the flow estimate, can alter the effective size and orientation of the neighborhood. In this paper, we also present a data-driven method for choosing the neighborhood size based on Stein's unbiased risk estimators (SURE).     

Measuring confidence in optical flow estimation

The performance of a number of techniques proposed for estimating the quality of optical flow estimates formed using differential methods is considered. The existence of a bias in the optical flow estimates computed using numerical derivatives is demonstrated, as well as the correlation between observed errors and predictions made by using a variety of published confidence estimates     

Fast technique for phase-based disparity estimation with noexplicit calculation of phase

A simpler and faster technique for depth estimation, based on phase measurements of disparity, is presented. The technique provides direct evaluations of phase differences, avoids explicit calculations of single phases and the attendant problem of phase wrapping, and is suitable for efficient software and hardware implementations     

Optimal brightness functions for optical flow estimation ofdeformable motion

Estimation accuracy of Horn and Schunck's (1981) classical optical flow algorithm depends on many factors including the brightness pattern of the measured images. Since some applications can select brightness functions with which to "paint" the object, it is desirable to know what patterns will lead to the best motion estimates. The paper presents a method for determining this pattern a priori using mild assumptions about the velocity field and imaging process. The method is based on formulating Horn and Schunck's algorithm as a linear smoother and rigorously deriving an expression for the corresponding error covariance function. The authors then specify a scalar performance measure and develop an approach to select an optimal brightness function which minimizes this performance measure from within a parametrized class. Conditions for existence of an optimal brightness function are also given. The resulting optimal performance is demonstrated using simulations, and a discussion of these results and potential future research is given.     

Wavelet-based compression algorithm for still omnidirectional 3d integral images

Three dimensional (3D) integral imaging is a method that allows the display of full colour images with continuous parallax within a wide viewing zone. Due to the significant quantity of data required to represent a captured 3D integral image with high resolution, image compression becomes mandatory for the storage and transmission of integral images. This paper investigates the use of 2D discrete wavelet transform (2D-DWT) for the compression of omnidirectional 3D integral images (OII). The method requires the extraction of different viewpoint images from the integral image. A single viewpoint image is constructed by extracting one pixel from each microlens, then each viewpoint image is decomposed using a 2D-DWT. The resulting array of coefficients contains several frequency bands. The lower frequency bands of the viewpoint images are assembled and compressed using a 3D discrete cosine transform (3D-DCT) followed by Huffman coding. Whereas, the remaining higher bands are fed directly into a quantisation process followed by arithmetic coding. Simulations are performed on a set of several grey level 3D OII using a uniform scalar quantizer with deadzone. It was found that the algorithm achieves better rate-distortion performance and reconstructs the images with much better image quality at very low bit rates than previously reported 3D-DCT-based scheme.     

Scale space analysis and active contours for omnidirectional images.

A new generation of optical devices that generate images covering a larger part of the field of view than conventional cameras, namely catadioptric cameras, is slowly emerging. These omnidirectional images will most probably deeply impact computer vision in the forthcoming years, provided that the necessary algorithmic background stands strong. In this paper, we propose a general framework that helps define various computer vision primitives. We show that geometry, which plays a central role in the formation of omnidirectional images, must be carefully taken into account while performing such simple tasks as smoothing or edge detection. Partial differential equations (PDEs) offer a very versatile tool that is well suited to cope with geometrical constraints. We derive new energy functionals and PDEs for segmenting images obtained from catadioptric cameras and show that they can be implemented robustly using classical finite difference schemes. Various experimental results illustrate the potential of these new methods on both synthetic and natural images.     

B-spline image model for energy minimization-based optical flow estimation.

Robust estimation of the optical flow is addressed through a multiresolution energy minimization. It involves repeated evaluation of spatial and temporal gradients of image intensity which rely usually on bilinear interpolation and image filtering. We propose to base both computations on a single pyramidal cubic B-spline model of image intensity. We show empirically improvements in convergence speed and estimation error and validate the resulting algorithm on real test sequences.     

基于三维立体线阵的全方位DOA估计方法

本文基于3D立体线阵以及累量的灵活定义,实现了信号DOA的全方位估计,该算法不依赖于信号的具体结构,并构造了三个累量域DOA矩阵,通过对DOA矩阵的特征分解得到配对特征向量,从而获得信号DOA的全方位估计,该方法避免了2D或1D的谱峰搜索,仿真实验结果表明了这种方法的正确性。     

A harmonic retrieval framework for discontinuous motion estimation

Motion discontinuities arise when there are occlusions or multiple moving objects in the scene that is imaged. Conventional regularization techniques use smoothness constraints but are not applicable to motion discontinuities. In this paper, we show that discontinuous (or multiple) motion estimation can be viewed as a multicomponent harmonic retrieval problem. From this viewpoint, a number of established techniques for harmonic retrieval ran be applied to solve the challenging problem of discontinuous (or multiple) motion. Compared with existing techniques, the resulting algorithm is not iterative, which not only implies computational efficiency but also obviates concerns regarding convergence or local minima. It also adds flexibility to spatio-temporal techniques which have suffered from lack of explicit modeling of discontinuous motion. Experimental verification of our framework on both synthetic data as well as real image data is provided.     

A framework for optical burst switching network design

We analyze optical burst switching (OBS) systems. The analysis leads to a framework which provides guidelines for OBS design. We identify conditions for OBS feasibility and the relationship between burst size, or equivalently burst assembly delay, and throughput, taking into consideration control packet processing and the number of available wavelengths per fiber     

Robust optical flow estimation based on a sparse motion trajectory set

This paper presents an approach to the problem of estimating a dense optical flow field. The approach is based on a multiframe, irregularly spaced motion trajectory set, where each trajectory describes the motion of a given point as a function of time. From this motion trajectory set a dense flow field is estimated using a process of interpolation. A set of localized motion models are estimated, with each pixel labeled as belonging to one of the motion models. A Markov random field framework is adopted, allowing the incorporation of contextual constraints to encourage region-like structures. The approach is compared with a number of conventional optical flow estimation algorithms taken over a number of real and synthetic sequences. Results indicate that the method produces more accurate results for sequences with known ground truth flow. Also, applying the method to real sequences with unknown flow results in lower DFD, for all of the sequences tested.     

A comprehensive cardiac motion estimation framework using both untagged and 3-D tagged MR images based on nonrigid registration

In this paper, we present a novel technique based on nonrigid image registration for myocardial motion estimation using both untagged and 3-D tagged MR images. The novel aspect of our technique is its simultaneous usage of complementary information from both untagged and 3-D tagged MR images. To estimate the motion within the myocardium, we register a sequence of tagged and untagged MR images during the cardiac cycle to a set of reference tagged and untagged MR images at end-diastole. The similarity measure is spatially weighted to maximize the utility of information from both images. In addition, the proposed approach integrates a valve plane tracker and adaptive incompressibility into the framework. We have evaluated the proposed approach on 12 subjects. Our results show a clear improvement in terms of accuracy compared to approaches that use either 3-D tagged or untagged MR image information alone. The relative error compared to manually tracked landmarks is less than 15% throughout the cardiac cycle. Finally, we demonstrate the automatic analysis of cardiac function from the myocardial deformation fields.     

A unifying framework for partial volume segmentation of brain MR images

Accurate brain tissue segmentation by intensity-based voxel classification of magnetic resonance (MR) images is complicated by partial volume (PV) voxels that contain a mixture of two or more tissue types. In this paper, we present a statistical framework for PV segmentation that encompasses and extends existing techniques. We start from a commonly used parametric statistical image model in which each voxel belongs to one single tissue type, and introduce an additional downsampling step that causes partial voluming along the borders between tissues. An expectation-maximization approach is used to simultaneously estimate the parameters of the resulting model and perform a PV classification. We present results on well-chosen simulated images and on real MR images of the brain, and demonstrate that the use of appropriate spatial prior knowledge not only improves the classifications, but is often indispensable for robust parameter estimation as well. We conclude that general robust PV segmentation of MR brain images requires statistical models that describe the spatial distribution of brain tissues more accurately than currently available models.     

A novel steganalysis framework of heterogeneous images based on GMM clustering

The current steganalysis frameworks involve a large number of techniques for feature extraction and classification. However, one of their common defects is treating all images as equal, thus ignoring the variability of statistical properties of different images, which motivates us to propose a novel steganalysis framework based on Gaussian mixture model (GMM) clustering in the study, targeting at heterogeneous images with different texture complexity. There are two main improvements compared to the current steganalysis frameworks. First, in the training stage, the GMM clustering algorithm is exploited to classify the training samples into limited categories automatically, and then design corresponding steganalyzers for each category; second, in the testing stage, the posterior probability of testing samples belonging to each category is calculated, and the samples are submitted to the steganalyzers corresponding to the maximum posterior probability for test. Extensive experimental results aiming at least significant bit matching (LSBM) steganography and two adaptive steganography algorithms show that the proposed framework outperforms the steganalysis system that is directly trained on a mixed dataset, and also indicate that our framework exhibits better detection performance compared to the representative framework for using image contents in most circumstances and similar detection performance in few cases.     

A management and visualization framework for reconfigurable WDM optical networks

As the phenomenal advance in optical WDM networking technologies continues, optical WDM network equipment has been deployed not only in backbone networks, but also in regional, metropolitan, and access networks. It is widely believed that a major component of the next-generation Internet will be an IP-based optical network employing WDM. WDM wavelength routing and signaling have become an active research field, and dynamic and adaptive wavelength routing and assignment algorithms have been proposed. However, there is less work on reporting network control and management system implementation efforts over testbed WDM networks. This article presents a network management and visualization framework aimed at guiding the development of management applications for reconfigurable WDM optical networks. A layered framework architecture including element and network management and visualization is provided, and an object-based information model representing the WDM network is introduced. Functional components on reconfiguration, software agent, and network visualization services are presented, and important issues related to optical lightpath generation are discussed. A network visualization service also provides WDM control and management APIs to applications and access networks such as an IP network management system. To illustrate the usage of the framework, we share our experience in implementing the MONET network control and management system, and present network visualization views obtained from the MONET WDM network to highlight the framework features.     

Optical flow estimation in cardiac CT images using the steered Hermite transform

This paper describes a new method to estimate the heart's motion in computer tomography images with the inclusion of a bio-inspired image representation model. Our proposal is based on the polynomial decomposition of each of the images using the steered Hermite transform as a representation of the local characteristics of images from an perceptual approach within a multiresolution scheme.The Hermite transform is a model that incorporates some of the more important properties of the first stages of the human visual system, such as the overlapping Gaussian receptive fields, the Gaussian derivative model of early vision and the multiresolution analysis.We propose an approach for optical flow estimation that incorporates image structure information extracted from the steered Hermite coefficients, that is later used as local motion constraints in a differential estimation method that involves several of the constraints seen in the current differential methods, which allows obtaining accurate flows.Considering the importance of understanding the movement of certain structures such as left ventricular and myocardial wall for better medical diagnosis, our main goal is to find an estimation method useful to assist diagnosis tasks in computer tomography images.