Depth map misalignment correction and dilation for DIBR view synthesis

The quality of the synthesized views by Depth Image Based Rendering (DIBR) highly depends on the accuracy of the depth map, especially the alignment of object boundaries of texture image. In practice, the misalignment of sharp depth map edges is the major cause of the annoying artifacts at the disoccluded regions of the synthesized views. Conventional smooth filter approach blurs the depth map to reduce the disoccluded regions. The drawbacks are the degradation of 3D perception of the reconstructed 3D videos and the destruction of the texture in background regions. Conventional edge preserving filter utilizes the color image information in order to align the depth edges with color edges. Unfortunately, the characteristics of color edges and depth edges are very different which causes annoying boundaries artifacts in the synthesized virtual views. Recent solution of reliability-based approach uses reliable warping information from other views to fill the holes. However, it is not suitable for the view synthesis in video-plus-depth based DIBR applications. In this paper, a new depth map preprocessing approach is proposed. It utilizes Watershed color segmentation method to correct the depth map misalignment and then the depth map object boundaries are extended to cover the transitional edge regions of color image. This approach can handle the sharp depth map edges lying inside or outside the object boundaries in 2D sense. The quality of the disoccluded regions of the synthesized views can be significantly improved and unknown depth values can also be estimated. Experimental results show that the proposed method achieves superior performance for view synthesis by DIBR especially for generating large baseline virtual views.     

Virtual view synthesis using layered depth image generation and depth-based inpainting for filling disocclusions and translucent disocclusions

View synthesis is an efficient solution to produce content for 3DTV and FTV. However, proper handling of the disocclusions is a major challenge in the view synthesis. Inpainting methods offer solutions for handling disocclusions, though limitations in foreground-background classification causes the holes to be filled with inconsistent textures. Moreover, the state-of-the art methods fail to identify and fill disocclusions in intermediate distances between foreground and background through which background may be visible in the virtual view (translucent disocclusions). Aiming at improved rendering quality, we introduce a layered depth image (LDI) in the original camera view, in which we identify and fill occluded background so that when the LDI data is rendered to a virtual view, no disocclusions appear but views with consistent data are produced also handling translucent disocclusions. Moreover, the proposed foreground-background classification and inpainting fills the disocclusions with neighboring background texture consistently. Based on the objective and subjective evaluations, the proposed method outperforms the state-of-the art methods at the disocclusions.     

An efficient depth image-based rendering with depth reliability maps for view synthesis

In this paper, an efficient depth image-based rending (DIBR) with depth reliability maps (DRM) is proposed to improve the quality of synthesized images. First, a DRM-based occlusion-aware approach is developed to obtain a segmentation mask, which can explicitly indicate where the information in an intermediate image should be blended preferably. Next, an improved weight model for view creation is introduced to enhance the quality of synthesized images. Finally, a distance and depth-based sub-pixel weighted (DDSPW) algorithm is presented to solve the visibility and resampling problems. Experimental results demonstrate that the treated DIBR schemes have better performance for view synthesis than the other three methods through the subjective visual perception and objective assessments in terms of peak signal to noise ratio and structural similarity index.     

Adaptive depth truncation filter for MVC based compressed depth image

In multiview video plus depth (MVD) format, virtual views are generated from decoded texture videos with corresponding decoded depth images through depth image based rendering (DIBR). 3DV-ATM is a reference model for the H.264/AVC based multiview video coding (MVC) and aims at achieving high coding efficiency for 3D video in MVD format. Depth images are first downsampled then coded by 3DV-ATM. However, sharp object boundary characteristic of depth images does not well match with the transform coding based nature of H.264/AVC in 3DV-ATM. Depth boundaries are often blurred with ringing artifacts in the decoded depth images that result in noticeable artifacts in synthesized virtual views. This paper presents a low complexity adaptive depth truncation filter to recover the sharp object boundaries of the depth images using adaptive block repositioning and expansion for increasing the depth values refinement accuracy. This new approach is very efficient and can avoid false depth boundary refinement when block boundaries lie around the depth edge regions and ensure sufficient information within the processing block for depth layers classification. Experimental results demonstrate that the sharp depth edges can be recovered using the proposed filter and boundary artifacts in the synthesized views can be removed. The proposed method can provide improvement up to 3.25 dB in the depth map enhancement and bitrate reduction of 3.06% in the synthesized views.     

基于边缘差异的虚拟视图像质量评价方法

为了降低多视视频中的传输数据量,可适当减少包含纹理图及其对应深度图的多视数量,而在自由立体视频系统的终端采用基于深度图的绘制技术,生成新的虚拟视点。由于不准确的深度图估计、遮挡及合成算法等原因,合成虚拟视的边缘会出现高频噪声。对于这种特有的失真,用一般的评价方法并不能准确反映人眼的视觉感知。该文在分析失真虚拟视与原始视像素差异基础上,对各差异像素进行分类,并分配权重,提出了一种基于视觉加权的边缘差异质量评价方法,对边缘像素施加较高权重。经多视视频序列实验证明,该文方法相比于其它评价方法,能更好地预测人类视觉对虚拟视图像的主观感知。     

Segment-based view synthesis optimization scheme in 3D-HEVC

The 3D extension of high efficiency video coding (3D-HEVC) adopts a view synthesis optimization (VSO) technique to improve the quality of synthesized views for depth map coding. The exact synthesized view distortion change (SVDC) is calculated in VSO which in turn brings huge coding complexity to the 3D-HEVC encoder due to the real view synthesis process. This work presents a scheme aimed at reducing coding complexity of the SVDC calculating process in the 3D-HEVC encoder. It skips line segments of pixels with variable lengths based on information from both of the textures and depth maps in the SVDC calculation. Experimental results demonstrate that the proposed scheme can reduce the coding complexity without any significant loss in rate distortion performance for the synthesized views.