一种新的体视对应方法

选择特征点作为匹配特征,采用视差估计方法,在满足一定匹配准则情况下,可以得到一约束最小化能量函数。模拟实验结果表明,使用模拟退火算法,可使能量函数达到全局最小,从而实现立体匹配。     

A new parallel hardware architecture for high-performance stereo matching calculation

In this paper, we propose a hardware (H/W) architecture to find disparities for stereo matching in real time. After analyzing the arithmetic characteristic of stereo matching, we propose a new calculating method that reuses the intermediate results to minimize the calculation load and memory access. From this, we propose a stereo matching calculation cell and a new H/W architecture. Finally, we propose a new stereo matching processor. The implemented H/W can operate at the clock frequency of 250 MHz at least in the FPGA (field programmable gate array) environment and produce about 120 disparity images per second for HD stereo images.     

User assisted disparity remapping for stereo images

This study concentrates on user assisted disparity remapping for stereo image footage, i.e. the disparity of an object of interest is altered while leaving the remaining scene unattended. This application is useful in the sense that it provides a method for emphasizing/de-emphasizing an object on the scene by adjusting its depth with respect to the camera. The proposed technique can also be used as a post-processing step for retargeting stereoscopic footage on different display sizes and resolutions. The proposed technique involves an MRF-based energy minimization step for interactive stereo image segmentation, for which user assistance on only one of the stereo pairs is required for determining the location of stereo object pair. A key contribution of the proposed study is elimination of dense disparity estimation step from the pipeline. This step is realized through a sparse feature matching technique between the stereo pairs. Moreover, by the help of the proposed technique, novel disparity adjusted views are synthesized using the produced stereo object segments and background information for the images. Qualitative and quantitative evaluation of the generated segments and the disparity adjusted images prove the functionality and superiority of the proposed technique.     

Efficient local stereo matching algorithm based on fast gradient domain guided image filtering

Guided image filtering (GIF) based cost aggregation or disparity refinement stereo matching algorithms are studied extensively owing to the edge-aware preserved smoothing property. However, GIF suffers from halo artifacts in sharp edges and shows high computational costs on high-resolution images. The performance of GIF in stereo matching would be limited by the above two defects. To solve these problems, a novel fast gradient domain guided image filtering (F-GDGIF) is proposed. To be specific, halo artifacts are effectively alleviated by incorporating an efficient multi-scale edge-aware weighting into GIF. With this multi-scale weighting, edges can be preserved much better. In addition, high computational costs are cut down by sub-sampling strategy, which decreases the computational complexity from O(N) to O(N/s²) (s: sub-sampling ratio) To verify the effectiveness of the algorithm, F-GDGIF is applied to cost aggregation and disparity refinement in stereo matching algorithms respectively. Experiments on the Middlebury evaluation benchmark demonstrate that F-GDGIF based stereo matching method can generate more accuracy disparity maps with low computational cost compared to other GIF based methods.     

光栅式双目立体视觉传感器的立体匹配方法

光栅式双目立体视觉传感器的难点之一在于立体匹配问题,为此,提出了一种基于极线约束和空间点最小距离搜索的立体匹配方法.该方法将光栅式双目立体视觉传感器看作两个光栅结构光传感器,分别标定后可测定光条中心点关于某个结构光模型的三维坐标,若两点匹配,则其三维坐标间的距离理论上为零.引入极线约束,在左摄像机成像光条上找一个特征点,在右摄像机所成像中便可计算出一条极线与之对应,在极线与各光条中心的交点中寻找匹配点.该方法在三维空间进行匹配,计算量小,能够实现点与点的唯一匹配.仿真实验表明了该方法的有效性.     

High dynamic range video reconstruction from a stereo camera setup

To overcome the dynamic range limitations in images taken with regular consumer cameras, several methods exist for creating high dynamic range (HDR) content. Current low-budget solutions apply a temporal exposure bracketing which is not applicable for dynamic scenes or HDR video. In this article, a framework is presented that utilizes two cameras to realize a spatial exposure bracketing, for which the different exposures are distributed among the cameras. Such a setup allows for HDR images of dynamic scenes and HDR video due to its frame by frame operating principle, but faces challenges in the stereo matching and HDR generation steps. Therefore, the modules in this framework are selected to alleviate these challenges and to properly handle under- and oversaturated regions. In comparison to existing work, the camera response calculation is shifted to an offline process and a masking with a saturation map before the actual HDR generation is proposed. The first aspect enables the use of more complex camera setups with different sensors and provides robust camera responses. The second one makes sure that only necessary pixel values are used from the additional camera view, and thus, reduces errors in the final HDR image. The resulting HDR images are compared with the quality metric HDR-VDP-2 and numerical results are given for the first time. For the Middlebury test images, an average gain of 52 points on a 0-100 mean opinion score is achieved in comparison to temporal exposure bracketing with camera motion. Finally, HDR video results are provided.     

Combining texture and stereo disparity cues for real-time face detection

Intuitively, integrating information from multiple visual cues, such as texture, stereo disparity, and image motion, should improve performance on perceptual tasks, such as object detection. On the other hand, the additional effort required to extract and represent information from additional cues may increase computational complexity. In this work, we show that using biologically inspired integrated representation of texture and stereo disparity information for a multi-view facial detection task leads to not only improved detection performance, but also reduced computational complexity. Disparity information enables us to filter out 90% of image locations as being less likely to contain faces. Performance is improved because the filtering rejects 32% of the false detections made by a similar monocular detector at the same recall rate. Despite the additional computation required to compute disparity information, our binocular detector takes only 42 ms to process a pair of 640×480 images, 35% of the time required by the monocular detector. We also show that this integrated detector is computationally more efficient than a detector with similar performance where texture and stereo information is processed separately.     

用于自由曲面视觉测量的立体精匹配方法

根据立体视觉原理,针对自由曲面视觉测量的实际情况,文中将多种方法融合为一体用于立体匹配,提出一种亚像素级的立体精匹配方法。首先利用投影光栅,在曲面上形成变形条纹,采用小波变换检测出像素级的离散边缘点;在此基础上,提出搜索式无监督聚类拟合匹配算法,将边缘点按实际边缘情况分为不同组别,并用三次B样条将离散的边缘点拟合成连续曲线;根据对极约束特性,实现亚像素级的立体精匹配,解决了交向摆放姿态的双目立体视觉系统的匹配问题。     

Disparity based stereo image retrieval through univariate and bivariate models

The widespread use of stereovision in various application fields has led to the constitution of very huge stereo image databases. Therefore, the design of effective content based image retrieval system devoted to stereo pairs becomes an issue of importance. To this end, we propose in this paper two retrieval methods which combine the visual contents of the stereo images with their corresponding disparity information. After modeling the distribution of their associated wavelet coefficients by the generalized Gaussian statistical model, the resulting distribution parameters are selected as salient features. While the two views are processed separately through a univariate modeling in the first method, the second one exploits the correlation between the views by resorting to a bivariate modeling. Experimental results show the benefits which can be drawn from the proposed retrieval approaches.     

基于视差补偿预测的立体视频图像压缩编码

本文介绍了立体视频编码方法,并对其关键技术-视差补偿预测技术进行深入研究.本文所提出的基于视差分割的视差补偿预测算法是建立在可变尺寸块匹配算法的基础上,充分利用视差信息实现对目标图像帧的有效分割,并采用相适应的视差向量编码方案.与传统算法相比,在相同预测精度下,明显降低了视差信息编码开销.     

Convolutional neural network based deep conditional random fields for stereo matching

Stereo matching has been studied for many years and is still a challenge problem. The Markov Random Fields (MRF) model and the Conditional Random Fields (CRF) model based methods have achieved good performance recently. Based on these pioneer works, a deep conditional random fields based stereo matching algorithm is proposed in this paper, which draws a connection between the Convolutional Neural Network (CNN) and CRF. The object knowledge is used as a soft constraint, which can effectively improve the depth estimation accuracy. Moreover, we proposed a CNN potential function that learns the potentials of CRF in a CNN framework. The inference of the CRF model is formulated as a Recurrent Neural Network (RNN). A variety of experiments have been conducted on KITTI and Middlebury benchmark. The results show that the proposed algorithm can produce state-of-the-art results and outperform other MRF-based or CRF-based methods.     

Novel shrinking residual convolutional neural network for efficient accurate stereo matching

For stereo matching based on patch comparing using convolutional neural networks (CNNs), the matching cost estimation is highly dependent on the network structure, and the patch comparing is time consuming for traditional CNNs. Accordingly, we propose a stereo matching method based on a novel shrinking residual CNN, which consists of convolutional layers and skip-connection layers, and the size of the fully connected layers decreases progressively. Firstly, a layer-by-layer shrinking size model is adopted for the full-connection layers to greatly increase the running speed. Secondly, the convolutional layer and the residual structure are fused to improve patch comparing. Finally, the Loss function is re-designed to give higher weights to hard-classified examples compared with the standard cross entropy loss. Experimental results on KITTI2012 and KITTI2015 demonstrate that the proposed method can improve the operation speed while maintaining high accuracy.     

Stacking learning with coalesced cost filtering for accurate stereo matching

Deep learning based stereo matching algorithms have produced impressive disparity estimation for recent years; and the success of them has once overshadowed the conventional ones. In this paper, we intend to reverse this inferiority, by leveraging Stacking Learning with Coalesced Cost Filtering to make the conventional algorithms achieve or even surpass the results of deep learning ones. Four classical and Discriminative Dictionary Learning (DDL) algorithms are adopted as base-models for Stacking. For the former ones, four classical stereo matching algorithms are employed and regarded as ‘Coalesced Cost Filtering Module’; for the latter supervised learning one, we utilize the Discriminative Dictionary Learning (DDL) stereo matching algorithm. Then three categories of features are extracted from the predictions of base-models to train the meta-model. For the meta-model (final classifier) of Stacking, the Random Forest (RF) classifier is selected. In addition, we also employ an advanced one-view disparity refinement strategy to compute the final refined results more efficiently. Performance evaluations on Middlebury v.2 and v.3 stereo data sets demonstrate that the proposed algorithm outperforms other four most challenging stereo matching algorithms. Besides, the submitted online results even show better results than deep learning ones.     

采用小波变换的立体匹配:一种基于相位的方法

本义提出了一种采用小波变换基于相位的立体匹配算法。该算法引入多分辨率分析思想,借助小波变换的Ｍａｌｌａｔ算法,采用金字塔式的多尺度匹配结构,从而显著地提高了匹配效率。同时,该算法利用平滑性约束条件实现了相位不稳定区域视差的自适应插值,保证了良好的匹配效果。立体象对的测试结果表明了该算法的有效性。     

Edge-preserving guided filtering based cost aggregation for stereo matching

Stereo matching has been widely used in various computer applications and it is still a challenging problem. In stereo matching, the filter-based stereo matching methods have achieved outstanding performance. A local stereo matching method based on adaptive edge-preserving guided filter is presented in this paper, which can achieve proper cost-volume filtering and keep edges well. We introduce a gradient vector of the enhanced image generated by the proposed filter into the cost computation and the Census transform is adopted in the cost measurement. This cost computation method is robust against radiometric variations and textureless areas. The edge-preserving guided filter approach is proposed to aggregate the cost volume, which further proves the effectiveness of edge-preserving filter for stereo matching. The experiments conducted on Middlebury benchmark and KITTI benchmark demonstrate that the proposed algorithm produces better results compared with other edge-aware filter-based methods.     

基于相位的尺度自适应立体匹配方法

本文实现了一种高效的基于相位的尺度自适应的立体匹配方法,基于相位的立体匹配算法,是目前最为先进的立体匹配算法,具有视差精度高,稳定性好,可以并行计算等优点,对于常见的相位“卷绕”问题,常采用“由粗及细”的逐步求粗策略,但是,相位信号在尺度,位置空间中的极点的邻域内不稳定,此时,逐步求精策略可能产生不可恢复的错误,存在鲁棒性问题,解决的方案是采有某种尺度自适应的方法,我们针对多尺度滤波器的构造问题提     

Reliable fusion of ToF and stereo data based on joint depth filter

To obtain reliable depth images with high resolution, a novel method is proposed in this study that fuses data acquired from time-of-flight (ToF) and stereo cameras, through which the advantages of both active and passive sensing are utilised. Based on the classic error model of the ToF, gradient information is introduced to establish the likelihood distribution for all disparity candidates. The stereo likelihood is estimated in parallel based on a 3D adaptive support-weight approach. The two independent likelihoods are unified using a maximum likelihood estimation, a process also referred to as a joint depth filter herein. Conventional post-processing methods such as a mutual consistency check are also used after applying a joint depth filter. We also propose a novel hole-filling method based on the seed-growing algorithm to retrieve missing disparities. Experiment results show that the proposed fusion method can produce reliable high-resolution depth maps and outperforms other compared methods.     

Dense-CNN: Dense convolutional neural network for stereo matching using multiscale feature connection

In spite of the fact that convolutional neural network-based stereo matching models have shown good performance in both accuracy and robustness, the issue of image feature loss in regions of texture-less, complex scenes and occlusions remains. In this paper, we present a dense convolutional neural network-based stereo matching method with multiscale feature connection, named Dense-CNN. First, we construct a novel densely connected network with multiscale convolutional layers to extract rich image features, in which the merged multiscale features with context information are utilized to estimate the cost volume for stereo matching. Second, we plan a novel loss-function strategy to learn the network parameters more reasonably, which can develop the performance of the proposed Dense-CNN model on disparity computation. Finally, we run our Dense-CNN model on the Middlebury and KITTI databases to conduct a comprehensive comparison with several state-of-the-art approaches. The experimental results demonstrate that the proposed method achieved superior performance on computational accuracy and robustness of disparity estimation, especially achieving the significant benefit of feature preservation in ill-posed regions.     

SAR stereo imaging algorithm based on Capon spectrum estimation

This letter presents a novel approach for the Synthetic Aperture Radar （SAR） stereo imaging based on the Capon spectrum estimation technique. In order to deal with nonuniform sampling space and lead to super resolution in the elevation direction, Capon approach is used to focus the SAR data on a certain height. Results obtained on simulated data demonstrate the feasibility of the Capon based algorithm. Compared with the classical Fast Fourier Transform （FFT）, the Capon based algorithm shows better resolution quality.