Improved depth extraction method of 3D objects using computational integral imaging reconstruction based on multiple windowing techniques

This paper presents an improved depth extraction method of 3D objects using computational integral imaging reconstruction (CIIR) based on the multiple windowing models. The proposed method records 3D objects using the lenslet array; and it reconstructs multiple sets of slice images from multiple CIIR methods based on the different windowing models. A depth map is then extracted by a block matching algorithm among multiple set of slice images. A preliminary experiment is carried out to show the feasibility of the proposed method. Experimental results indicate the proposed method outperforms the previous method with two windowing models.     

Computational integral imaging reconstruction method of 3D images using pixel-to-pixel mapping and image interpolation

In this paper, we propose a novel computational integral imaging reconstruction (CIIR) method to improve the visual quality of the reconstructed images using a pixel-to-pixel mapping and an interpolation technique. Since an elemental image is magnified inversely through the corresponding pinhole and mapped on the reconstruction output plane based on pinhole-array model in the conventional CIIR method, the visual quality of reconstructed output image (ROI) degrades due to the interference problem between adjacent pixels during the superposition of the magnified elemental images. To avoid this problem, the proposed CIIR method generates dot-pattern ROIs using a pixel-to-pixel mapping and substitutes interpolated values for the empty pixels within the dot-pattern ROIs using an interpolation technique. The interpolated ROIs provides a much improved visual quality compared with the conventional method because of the exact regeneration of pixel positions sampled in the pickup process without interference between pixels. Moreover, it can enable us to reduce a computational cost by eliminating the magnification process used in the conventional CIIR. To confirm the feasibility of the proposed system, some experiments are carried out and the results are presented.     

Three-dimensional image correlator using fast computational integral imaging reconstruction method based on pixel-to-pixel mapping

In this paper, a three-dimensional (3D) image correlator using a fast computational integral imaging reconstruction (CIIR) method based on a pixel-to-pixel mapping is proposed. In order to implement the fast CIIR method, we replace the magnification process in the conventional CIIR by a pixel-to-pixel mapping. The proposed fast CIIR method reconstructs two sorts of plane images; a plane image whose quality is sufficient, and a dot pattern plane image insufficient to view. This property is very useful to enhance the performance of a CIIR-based image correlator. Thus, we apply the fast CIIR method to a CIIR-based image correlator. To show the feasibility of the proposed method, some preliminary experiments on both pattern correlation and computational cost are carried out, and the results are presented. Our experimental results indicate that the proposed image correlator is superior to the previous method in terms of both correlation performance and complexity.     

Orthoscopic integral imaging display by use of the computational method based on lenslet model

In this paper, we propose a computational depth conversion method based on the lenslet model to display the orthoscopic 3D images in 3D integral imaging display. The proposed method permits the synthesis of elemental images for the orthoscopic 3D images at any arbitrary position without any restrictions and requires no additional procedure during the depth conversion process. Due to the lenslet model involved in the depth conversion procedure, the proposed method can broaden the flexibility of 3D image reconstruction in the integral imaging display system. We carry out the preliminary experiments to prove the feasibility of the proposed method. The experimental results reveal that the proposed method is an effective depth conversion method that allows the reconstruction of the orthoscopic 3D images at any arbitrary position.     

Enhanced computational integral imaging system for partially occluded 3D objects using occlusion removal technique and recursive PCA reconstruction

In this paper, we propose an enhanced computational integral imaging system by both eliminating the occlusion in the elemental images recorded from the partially occluded 3D object and recovering the entire elemental images of the 3D object. In the proposed system, we first obtain the elemental images for partially occluded object using computational integral imaging system and it is transformed to sub-images. Then we eliminate the occlusion within the sub-images by use of an occlusion removal technique. To compensate the removed part from occlusion-removed sub-images, we use a recursive application of PCA reconstruction and error compensation. Finally, we generate the entire elemental images without a loss from the newly reconstructed sub-images and perform the process of object recognition. To show the usefulness of the proposed system, we carry out the computational experiments for face recognition and its results are presented. Our experimental results show that the proposed system might improve the recognition performance dramatically.     

Computational integral imaging reconstruction scheme of far 3D objects by additional use of an imaging lens

In this paper, we propose a novel performance-enhanced computational integral imaging reconstruction (CIIR) scheme by additional use of an imaging lens. In the proposed scheme, elemental images can be obtained by using a simultaneous pickup scheme of far three-dimensional (3D) objects from the lenslet array in both real and virtual image fields. And additional imaging lens produces an image shift effect of 3D objects located far away from the lenslet array and improve the visual quality of reconstructed images in CIIR by overcoming limitation of pickup range in integral imaging. To show the usefulness of the proposed system, some experiments are carried out for real 3D objects and its results are presented.     

Modified smart pixel mapping method for displaying orthoscopic 3D images in integral imaging

In this paper, we propose a modified smart pixel mapping (MSPM) method for displaying orthoscopic three-dimensional (3D) images with a function of depth control in integral imaging system. In the proposed MSPM, the depth-converted elemental image array (EIA) is obtained through the pixel mapping process and the image interpolation technique. The proposed method gives us the depth conversion at distances different from the position of 3D object and provides various types of EIAs using only an original EIA for orthoscopic images. To show the usefulness of the proposed method, we carry out the preliminary experiments and present the experimental results.     

Optical display of true 3D objects in depth-priority integral imaging using an active sensor

In this paper, we propose a system combining the pickup process using an active sensor and the display process using depth-priority integral imaging (DPII) system to display true three-dimensional (3D) objects within large depth through real and virtual image fields. The active sensor provides depth map and color images of 3D objects. Using captured depth map and original color images, elemental images are computationally synthesized and displayed optically in DPII system. Proposed system provides scaling of 3D scenes for true 3D object. To show the usefulness of proposed system, we carry out the experiment for true 3D objects of three character patterns and present the experimental results.     

Compression scheme of sub-images using Karhunen-Loeve transform in three-dimensional integral imaging

In this paper, a compression scheme of sub-image-transformed elemental images using Karhunen-Loeve transform (KLT) in three-dimensional integral imaging is proposed. The proposed scheme provides improved compression efficiency by improving the similarity between elemental images using sub-image transformation. To test the proposed scheme, various elemental images of 3D objects are picked up and the compression process is carried out using KLT. From the experimental results, it is showed that the proposed compression scheme gives us an improved efficiency of 26% as compared with the conventional compression method.     

用于集成成像3D显示的曲面微透镜阵列设计与仿真

针对集成成像3D显示观看视场范围小的问题和微透镜之间间隙的杂散光干扰导致的显示质量下降问题,提出了适合柔性显示及曲面显示器的曲面针孔/微透镜阵列结构.采用TracePro光学仿真软件对基于曲面针孔/微透镜阵列的集成成像3D显示的记录和重构过程进行仿真,结果显示:在记录和重构阶段,曲面针孔/微透镜阵列可以有效地减少透镜阵列之间杂散光引起的图像质量变差的问题;当记录和重构阶段均用曲面/针孔微透镜阵列时,记录三维物体的视角大,获得重构图像的视场角也相对较高.采用旋转接收屏法获取不同观看视角下的图像质量,当曲面针孔/微透镜阵列的曲面度数为30°时,重构图像质量最好.     

Occlusion removal technique for improved recognition of partially occluded 3D objects in computational integral imaging

In this paper, we propose an occlusion removal technique for improved recognition of 3D objects that are partially occluded in computational integral imaging (CII). In the reconstruction process of a 3D object which is partially occluded by other objects, occlusion degrades the resolution of reconstructed 3D images and thus this affects negatively the recognition of a 3D object in CII. To overcome this problem, we introduce a method to eliminate occluding objects in elemental image array (EIA) and the proposed method is applied to 3D object recognition by use of CII. To our best knowledge, this is the first time to remove occlusion in CII. In our method, we apply the elemental image to sub-image (ES) transform to EIA obtained by a pickup process and those sub-images are employed for occlusion removal. After the transformation, we correlate those sub-images with a reference sub-image to locate occluding objects and then we eliminate the objects. The inverse ES transform provides a modified EIA. Actually, the modified EIA is considered to be an EIA without the object that occludes the object to be reconstructed. This can provide a substantial gain in terms of the image quality of 3D objects and in terms of recognition performance. To verify the usefulness of the proposed technique, some experimental results are carried out and the results are presented.     

Investigation of resolution of the stereoscopic 3D images

Evaluation of 3D images is an important part of 3D display developments. In this paper, we propose an evaluation method which can reflect the characteristics of the stereoscopic 3D images in the depth direction. Perception of the depth planes is verified using a subjective evaluation method. Resolution of the stereoscopic 3D images which represents the distribution of the 3D images in the depth direction quantitatively is calculated and analyzed.     

A 3D image encryption technique using computer-generated integral imaging and cellular automata transform

A novel three-dimensional (3D) image encryption approach by using the computer-generated integral imaging and cellular automata transform (CAT) is proposed, in which, the two-dimensional (2D) elemental image array (EIA) digitally recorded by light rays coming from the 3D image is mapped inversely through the virtual pinhole array according to the ray-tracing theory. Next, the encrypted image is generated by using the 2D CAT scrambling transform for the 2D EIA. The reconstructed process is carried out by using the modified computational integral-imaging reconstruction (CIIR) technique; the depth-dependent plane images are reconstructed on the output plane. The reconstructed 3D image quality of the proposed scheme can be greatly improved, because the proposed encryption scheme carries out in a computer which can avoid the light diffraction caused by optical device CIIR, and solves blur problem caused by CIIR by using the pixel-averaging algorithm. Furthermore, the CAT-based encryption algorithm is an error-free encryption method; CAT as an orthogonal transformation offers considerable simplicity in the calculation of the transform coefficient, that is, it can improve the quality of the reconstructed image by reducing energy loss compared with the traditional complicated transform process. To show the effectiveness of the proposed scheme, we perform computational experiments. Experimental results show that the proposed scheme outperforms conventional encryption methods.     

Efficient compression of motion-compensated sub-images with Karhunen-Loeve transform in three-dimensional integral imaging

An approach to highly enhance the compression efficiency of the integral images by applying the Karhunen-Loeve transform (KLT) algorithm to the motion-compensated sub-images is proposed. The sub-images transformed from the elemental images picked-up from the three-dimensional (3D) object might represent the different perspectives of the object. Thus, the similarity among the sub-images gets better than that among the elemental images, so that an improvement of compression efficiency of the sub-images could be obtained. However, motion vectors occurred among the sub-images might result in an additional increase of image data to be compressed. Accordingly, in this paper, motion vectors have been estimated and compensated in all sub-image in advance. Then the KLT algorithm was applied to these motion-compensated sub-images for compression. It is shown from some experimental results that compression efficiency of the proposed method has been improved up to 24.44%, 40.62%, respectively, on the average compared to that of the conventional KLT compression method and that of the JPEG.     

Efficient compression of rearranged time-multiplexed elemental image arrays in MALT-based three-dimensional integral imaging

In this paper, an approach to efficiently compress the time-multiplexed EIAs picked up from the MALT-based integral imaging system is proposed. In this method, the time-multiplexed EIAs are rearranged by collecting the elemental images occupied at the same position in each EIA to enhance the similarity among the elemental images. Then, MPEG-4 is applied to these rearranged elemental images for compression. From the experimental results, it is shown that the average correlation quality (ACQ) value representing a degree of similarity between the elemental images, and the resultant compression efficiency have been enhanced by 11.50% and 9.97%, respectively on the average for three kinds of test scenarios in the proposed method, compared to those of the conventional method. Good experimental results finally confirmed the feasibility of the proposed scheme.     

平面三维显示技术的研究现状

平面三维显示技术是近年来最新出现的虚拟现实显示技术,其最大的特点是观察者无需使用任何辅助附加设备,直接用肉眼就可看到屏幕上显示的三维图像。为推进三维显示技术的发展,进一步研究了视差立体成像原理,并据此介绍几种平面三维显示方法及其工作原理,包括障栅立体显示、微柱透镜阵列立体显示、偏振片立体显示和基于微柱透镜立体显示原理的多视点系统,阐述并分析了系统的优缺点。以日本三洋公司的四视角立体显示装置、南京大学的多视点三维显示系统和NEC液晶科技的HDDP三维显示系统为例,描述了国内外该项技术近期的研究现状,分析了存在的技术难点,展望了该应用领域的发展前景。     

Optimising image quality in 2D and 3D confocal Raman mapping

This article compares the quality of Raman images obtained using metallurgical and oil immersion objectives to map complex structures in two and three dimensions. While the performance of these objectives for depth profiling planar structures has been discussed at length in the literature, the same comparison has not yet been made for 3D mapping of complex objects, where additional complications are introduced by non‐planar geometries. Studying samples with increasing complexity shows that the oil immersion objective is strongly preferred because it yields brighter images with better contrast, and eliminates some new and confusing artefacts that do not arise with simple planar objects. Copyright © 2013 John Wiley & Sons, Ltd.     

3D imager using dual color-balanced lights

A novel method for grabbing 3D shape of an object is proposed. It uses a pair of color-coded light sources to create a 3D-coordinated illumination space. The intensities of two modulation colors are complementally balanced, which makes the sum of the intensities of the colors a constant. This method demonstrates the abilities of uniquely representing any point in the 3D-coordinated illumination space, reducing the measuring problems in blind area, and compensating the effect caused by changes of surface color and reflection. In addition, this method has the ability of acquiring the 3D shape information in parallel and the algorithm is fairly simple, so the 3D imaging speed is basically restricted by the frame rate of the color CCD camera.     

High-accuracy and real-time 3D positioning,tracking system for medical imaging applications based on 3D digital image correlation

This paper presents a system for positioning markers and tracking the pose of a rigid object with 6 degrees of freedom in real-time using 3D digital image correlation, with two examples for medical imaging applications. Traditional DIC method was improved to meet the requirements of the real-time by simplifying the computations of integral pixel search. Experiments were carried out and the results indicated that the new method improved the computational efficiency by about 4–10 times in comparison with the traditional DIC method. The system was aimed for orthognathic surgery navigation in order to track the maxilla segment after LeFort I osteotomy. Experiments showed noise for the static point was at the level of 10⁻³ mm and the measurement accuracy was 0.009 mm. The system was demonstrated on skin surface shape evaluation of a hand for finger stretching exercises, which indicated a great potential on tracking muscle and skin movements.