基于卷积神经网络的虚拟视点空洞填补

为了提高虚拟视点绘制中空洞填补的质量,本文提 出一种基于卷积神经网络的虚拟视点空洞填补算 法,包括图像预处理、特征提取和空洞填补三部分。首先,对双视点虚拟视点图像进行预处 理,即利用深 度图像绘制(Depth Image Based Rendering,DIBR)技术得到空洞填补前的图像和空洞掩膜 ；然后,通过卷积 神经网络对虚拟视点图像提取多维特征,针对空洞区域,使用结构相似性损失函数优化网络 ；最后,利用 提取的特征填补虚拟视点中的空洞。实验结果表明,本文算法能较好地保持虚拟视点图像前 景和背景的锐 利边缘,主观视觉感知效果佳,同时,对不同场景测试序列的客观评价指标PS NR和SSIM都能取得满意结果,该方法能有效地填补虚拟视点中的空洞。

Virtual view hole filling based on convolutional neural network

In order to improve the quality of hole filling in the virtual view rendering,t his paper proposes an algorithm of virtual view hole filling based on convolutional neural network.Th e proposed algorithm includes image preprocessing,feature extraction and hole filling.Firstly,implement the image preprocessing of double virtual view, that is,using the depth image based rendering (DIBR) technique to get the image before hole filling and the hole mask.Then,use the convolutional neural network to extract the feature,as for the hole,use the structure similarity loss function to optimize the network;finally,fill the hole in the virtual vie w with the extracted features. Experimental results indicate that the proposed algorithm effectively fills the hole of the double virtual view and achieves better performance in terms of the quality of subjective visual view ef fect.