Occlusion removal technique for improved recognition of partially occluded 3D objects in computational integral imaging

In this paper, we propose an occlusion removal technique for improved recognition of 3D objects that are partially occluded in computational integral imaging (CII). In the reconstruction process of a 3D object which is partially occluded by other objects, occlusion degrades the resolution of reconstructed 3D images and thus this affects negatively the recognition of a 3D object in CII. To overcome this problem, we introduce a method to eliminate occluding objects in elemental image array (EIA) and the proposed method is applied to 3D object recognition by use of CII. To our best knowledge, this is the first time to remove occlusion in CII. In our method, we apply the elemental image to sub-image (ES) transform to EIA obtained by a pickup process and those sub-images are employed for occlusion removal. After the transformation, we correlate those sub-images with a reference sub-image to locate occluding objects and then we eliminate the objects. The inverse ES transform provides a modified EIA. Actually, the modified EIA is considered to be an EIA without the object that occludes the object to be reconstructed. This can provide a substantial gain in terms of the image quality of 3D objects and in terms of recognition performance. To verify the usefulness of the proposed technique, some experimental results are carried out and the results are presented.     

Optical 3D watermark based digital image watermarking for telemedicine

Region of interest (ROI) of a medical image is an area including important diagnostic information and must be stored without any distortion. This algorithm for application of watermarking technique for non-ROI of the medical image preserving ROI. The paper presents a 3D watermark based medical image watermarking scheme. In this paper, a 3D watermark object is first decomposed into 2D elemental image array (EIA) by a lenslet array, and then the 2D elemental image array data is embedded into the host image. The watermark extraction process is an inverse process of embedding. The extracted EIA through the computational integral imaging reconstruction (CIIR) technique, the 3D watermark can be reconstructed. Because the EIA is composed of a number of elemental images possesses their own perspectives of a 3D watermark object. Even though the embedded watermark data badly damaged, the 3D virtual watermark can be successfully reconstructed. Furthermore, using CAT with various rule number parameters, it is possible to get many channels for embedding. So our method can recover the weak point having only one transform plane in traditional watermarking methods. The effectiveness of the proposed watermarking scheme is demonstrated with the aid of experimental results.     

Resolution-enhanced 3D image correlator using computationally reconstructed integral images

In this paper, a novel resolution-enhanced three-dimensional (3D) image correlator using the computationally reconstructed integral images is proposed in order to extract target object’s 3D location data in a scene. Elemental images of the reference and target objects are picked up by lenslet arrays and using these elemental images, reference and target plane images are reconstructed on the output plane by means of a modified computational integral imaging reconstruction technique. Then, through cross-correlations between the reconstructed reference and the target plane images, 3D location data of the target object can be extracted from the correlation outputs. With the purpose of showing the feasibility of the proposed method, some computational and optical experiments on the target objects in space are carried out and the results are presented.     

Computational integral imaging reconstruction scheme of far 3D objects by additional use of an imaging lens

In this paper, we propose a novel performance-enhanced computational integral imaging reconstruction (CIIR) scheme by additional use of an imaging lens. In the proposed scheme, elemental images can be obtained by using a simultaneous pickup scheme of far three-dimensional (3D) objects from the lenslet array in both real and virtual image fields. And additional imaging lens produces an image shift effect of 3D objects located far away from the lenslet array and improve the visual quality of reconstructed images in CIIR by overcoming limitation of pickup range in integral imaging. To show the usefulness of the proposed system, some experiments are carried out for real 3D objects and its results are presented.     

Modified computational integral imaging-based double image encryption using fractional Fourier transform

In this paper, we propose an image encryption technique to simultaneously encrypt double or multiple images into one encrypted image using computational integral imaging (CII) and fractional Fourier transform (FrFT). In the encryption, each of the input plane images are located at different positions along a pickup plane, and simultaneously recorded in the form of an elemental image array (EIA) through a lenslet array. The recorded EIA to be encrypted is multiplied by FrFT with two different fractional orders. In order to mitigate the drawbacks of occlusion noise in computational integral imaging reconstruction (CIIR), the plane images can be reconstructed using a modified CIIR technique. To further improve the solution of the reconstructed plane images, a block matching algorithm is also introduced. Numerical simulation results verify the feasibility and effectiveness of the proposed method.     

Three-dimensional recognition of occluded objects by using computational integral imaging

We have proposed a method to recognize partially occluded three-dimensional (3D) objects by using 3D volumetric reconstruction integral imaging (II). An II system captures multiple perspectives of occluded objects by using a microlens array. The reconstruction of the occluded 3D scene and target recognition are done digitally to reduce the effects of the occlusion. To verify system performance, we have implemented an optimum filter for object recognition. Both two-dimensional (2D) images and 3D II volumetric reconstructed images are considered. The correlation results of occluded 3D images for volumetric reconstruction show substantial improvements compared with those for conventional 2D imaging of occluded images.     

Enhanced computational integral imaging system for partially occluded 3D objects using occlusion removal technique and recursive PCA reconstruction

In this paper, we propose an enhanced computational integral imaging system by both eliminating the occlusion in the elemental images recorded from the partially occluded 3D object and recovering the entire elemental images of the 3D object. In the proposed system, we first obtain the elemental images for partially occluded object using computational integral imaging system and it is transformed to sub-images. Then we eliminate the occlusion within the sub-images by use of an occlusion removal technique. To compensate the removed part from occlusion-removed sub-images, we use a recursive application of PCA reconstruction and error compensation. Finally, we generate the entire elemental images without a loss from the newly reconstructed sub-images and perform the process of object recognition. To show the usefulness of the proposed system, we carry out the computational experiments for face recognition and its results are presented. Our experimental results show that the proposed system might improve the recognition performance dramatically.     

A novel three-dimensional digital watermarking scheme basing on integral imaging

A novel integral imaging-based three-dimensional (3D) digital watermarking scheme is presented. In the proposed method, an elemental image array (EIA) obtained by recording the rays coming from a 3D object through a pinhole array in the integral imaging system is employed as a new 3D watermark. The EIA is composed of a number of small elemental images having their own perspectives of a 3D object, and from this recorded EIA various depth-dependent 3D object images can be reconstructed by using the computational integral imaging reconstruction (CIIR) technique. This 3D property of the EIA watermark can make a robust reconstruction of the watermark image available even though there are some data losses in the embedded watermark by attacks. To show the robustness of the proposed scheme against attacks, some experiments are carried out and the results are discussed.     

Three-dimensional imaging and visualization of partially occluded objects using axially distributed stereo image sensing

In this Letter, we propose a multiperspective three-dimensional (3D) imaging system using axially distributed stereo image sensing. In this proposed method, the stereo camera is translated along its optical axis and multiple axial elemental image pairs for a 3D scene are collected. The captured elemental images are reconstructed in 3D using a computational reconstruction algorithm based on ray back-projection. The proposed method is applied to partially occluded object visualization. Optical experiments are performed to verify the approach.     

A 3D image encryption technique using computer-generated integral imaging and cellular automata transform

A novel three-dimensional (3D) image encryption approach by using the computer-generated integral imaging and cellular automata transform (CAT) is proposed, in which, the two-dimensional (2D) elemental image array (EIA) digitally recorded by light rays coming from the 3D image is mapped inversely through the virtual pinhole array according to the ray-tracing theory. Next, the encrypted image is generated by using the 2D CAT scrambling transform for the 2D EIA. The reconstructed process is carried out by using the modified computational integral-imaging reconstruction (CIIR) technique; the depth-dependent plane images are reconstructed on the output plane. The reconstructed 3D image quality of the proposed scheme can be greatly improved, because the proposed encryption scheme carries out in a computer which can avoid the light diffraction caused by optical device CIIR, and solves blur problem caused by CIIR by using the pixel-averaging algorithm. Furthermore, the CAT-based encryption algorithm is an error-free encryption method; CAT as an orthogonal transformation offers considerable simplicity in the calculation of the transform coefficient, that is, it can improve the quality of the reconstructed image by reducing energy loss compared with the traditional complicated transform process. To show the effectiveness of the proposed scheme, we perform computational experiments. Experimental results show that the proposed scheme outperforms conventional encryption methods.     

Real-time sensing and three-dimensional display of far outdoor scenes based on asymmetric integral imaging

A new asymmetric integral imaging (AII) system for real-time pickup and three-dimensional (3-D) display of far outdoor scenes based on dynamic-pixel-mapping (DPM) is proposed. DPM is a digital process to transform the elemental images captured with a lens array into the perspective-variant object images (POIs) whose structures are matched with those of display lenses, where the orders of pixels in each POI are reversely mapped, and then capture a set of virtual elemental images (EIs) at the specific depth planes from the back-propagated POIs. This DPM enables an asymmetrical use of pickup and display lens arrays, allowing the long-ranged pickup of far outdoor scenes and their resolution-enhanced 3-D reconstruction. Experiments with a pair of pickup and display lens arrays whose pitches and focal lengths are given by 7.5 mm, 30 mm and 1.2 mm, 8 mm, respectively, show that the effective pickup-range and resolution of the proposed system have been increased up to 6 m and 1600×1600 pixels, respectively, from 0.064 m and 480×480 pixels of the conventional systems employing the same pickup and display lens arrays. In addition, experiments with an implemented test bed confirms that the proposed system can provide real-time 3-D images in 25 frames per second.     

Space-variant filtering with holographic multifacet elements

We present a spatial filtering method for space-variant operations. The method is based on the combination of a holographic lenslet array and an appropriate filter array. The lenslets produce local Fourier transforms of the object which are filtered in parallel and independently by the spatial filters. The most obvious space-variant setup based on lenslet arrays would consist of many narrow 4f-setups, arranged side by side. The total object would be partitioned into many small object patches, as many as there are lens elements in the lenslet array. However this setup shows the undesired effect of local inversion. Therefore we choose a setup with only one lenslet array and an imaging lens which avoids the local inversion.     

利用匹配区域的纹理特征改善重构三维图像的视觉质量

研究了计算机重构三维图像时分辨率低的问题,提出一种改善3D计算机全景重构图像的视觉质量的方法,该方法利用3D空间的物体部分在每个元素图像中形成的匹配区域的纹理特征,从两个相邻的元素图像中的匹配区域提取出多个像素,经过加权计算重构出相应的图像区域.该方法与传统的计算机重构方法相比,提高了图像分辨率,同时也消除了从每个元素图像中提取多个像素直接重构图像时存在的"像素块"效应,改善了重构图像的视觉质量.     

Analysis on axial squeeze distortion in resolution-priority integral imaging display systems

When the three-dimensional (3D) image is out of focus, a significant axial squeeze distortion is generally produced by position mismatching between the geometric convergences of the lenslet array and the convergence of each lenslet. In this paper, a rigorous explanation of a phenomenon that produces axial distortions is first presented. Then, the phenomenon is theoretically analyzed and the axial shifted position of the 3D image point is derived based on geometrical optics and the modulation transform function (MTF) theory. Simulated results show the dependence of 3D image axial shift error on the focal length of the lenslet array. Finally, a solution for eliminating the axial shift and the axial distortion is brought out and the suitable range of the focal length is discussed.     

3D image reconstruction with controllable spatial filtering based on correlation of multiple periodic functions in computational integral imaging

We propose a novel method of slice image reconstruction with controllable spatial filtering by using the correlation of periodic delta-function arrays(PDFAs) with elemental images in computational integral imaging. The multiple PDFAs, whose spatial periods correspond to object's depths with the elemental image array(EIA), can generate a set of spatially filtered EIAs for multiple object depths compared with the conventional method for the depth of a single object. We analyze a controllable spatial filtering effect by the proposed method.To show the feasibility of the proposed method, we carry out preliminary experiments for multiple objects and present the results.     

Geometric analysis of spatial distortion in projection-type integral imaging

In projection-type integral imaging, positional errors in elemental images and elemental lenses affect three-dimensional (3D) image quality. We analyzed the relationships between the geometric distortion in elemental images caused by a projection lens and the spatial distortion in the reconstructed 3D image. As a result, we clarified that 3D images that were reconstructed far from the lens array were largely affected, and that the reconstructed images were significantly distorted in the depth direction at the corners of the displayed images.     

Three-dimensional imaging with detector arrays on arbitrarily shaped surfaces

Conventional multiview imaging systems commonly utilize a planar lenslet array and a rigid, flat image sensor in the pickup stage to capture different views of the scene. In this Letter, we remove this constraint by proposing the concept of three-dimensional (3D) imaging with detector arrays that may conform to arbitrarily shaped surfaces or platforms. A nonplanar detector array configuration can be used in combination with a flexible lenslet array to capture different views. The orientation and optical axes of individual image sensing elements could vary. A point-by-point 3D reconstruction algorithm is described and the feasibility of the proposed approach is demonstrated through simulated imagery.     

4-D image reconstruction for Streak Tube Imaging Lidar

The Streak Tube Imaging Lidar (STIL) which is the flash laser radar is developed in recently years. It can output the 4-D image (3-D range image + 1-D intensity image). The core part of STIL is the streak tube, which is a line array detector. STIL can directly collect the image of returned light pulse, and consequently, it can’t output the 4-D target image. In the paper, the peak detection as the reconstruction algorithm is used to reconstruct the multiple streak tube images, and the clear 4-D target images are obtained. The outdoor imaging experiments are completed for the far distance buildings. The reconstruction for the multiple streak tube images is finished, and the results state that the peak detection algorithm can output the high-resolution 4-D target images. Original Text ? Astro, Ltd., 2009. The article is published in the original.     

Robust image encryption by combined use of integral imaging and pixel scrambling techniques

A robust image encryption method by using the integral imaging and pixel scrambling (PS) techniques is proposed. In this method, pixels of the cover image are scrambled with the PS technique and elemental images for this scrambled image are picked up through a lenslet array. Subsequently, an encrypted image is obtained by scrambling these picked-up elemental images. Since this encrypted image has the hologram-like property of data redundancy resulted from the integral imaging scheme, while it can as well be decoded by multiple keys such as the orders of pixel scrambling and the pickup conditions of the elemental images, its security against the various attacks could be dramatically improved. Good experimental results also confirm that the proposed method could provide more enhanced robustness against data loss and Gaussian noises compared to the conventional methods.     

基于微透镜阵列的实时三维物体识别

提出一种基于微透镜阵列多视角成像特点,将三维物体的深度信息转化为二维透射像阵列的角度信息,利用光学二维图像识别技术,实现对三维物体识别的方法.对识别过程进行了理论分析和计算,用匹配滤波的方法实现了对三维物体骰子的实时识别.实验结果表明,本方法的相关识别能力较高,并且具有很强的灵活性,对于有微小旋转、微小平移的三维物体也可进行识别.