Three-dimensional color photon counting microscopy using Bayesian estimation with adaptive priori information

In this Letter, we propose a novel three-dimensional(3D) color microscopy for microorganisms under photonstarved conditions using photon counting integral imaging and Bayesian estimation with adaptive priori information. In photon counting integral imaging, 3D images can be visualized using maximum likelihood estimation(MLE). However, since MLE does not consider a priori information of objects, the visual quality of 3D images may not be accurate. In addition, the only grayscale image can be reconstructed. Therefore, to enhance the visual quality of 3D images, we propose photon counting microscopy using maximum a posteriori with adaptive priori information. In addition, we consider a wavelength of each basic color channel to reconstruct 3D color images. To verify our proposed method, we carry out optical experiments.     

Three-dimensional free view reconstruction in axially distributed image sensing

In this Letter, we propose a three-dimensional(3D) free view reconstruction technique in axially distributed image sensing(ADS). In typical integral imaging, free view reconstructed images can be obtained by tilting all elemental images or tilting the reconstruction plane due to large lateral perspectives for 3D objects. In conventional ADS, the reconstructed images at only a front view can be generated since the sensor is moved along with its optical axis so that it has small lateral perspectives for 3D objects. However, the reconstructed 3D images at any viewing point may be obtained because the virtual viewing camera may capture these slightly different perspectives for 3D objects. Therefore, in this Letter, we employ the virtual viewing camera to visualize the 3D images at the arbitrary viewing point. To support our proposed method, we show the experimental results.     

Three-dimensional imaging and visualization of partially occluded objects using axially distributed stereo image sensing

In this Letter, we propose a multiperspective three-dimensional (3D) imaging system using axially distributed stereo image sensing. In this proposed method, the stereo camera is translated along its optical axis and multiple axial elemental image pairs for a 3D scene are collected. The captured elemental images are reconstructed in 3D using a computational reconstruction algorithm based on ray back-projection. The proposed method is applied to partially occluded object visualization. Optical experiments are performed to verify the approach.     

Occlusion removal technique for improved recognition of partially occluded 3D objects in computational integral imaging

In this paper, we propose an occlusion removal technique for improved recognition of 3D objects that are partially occluded in computational integral imaging (CII). In the reconstruction process of a 3D object which is partially occluded by other objects, occlusion degrades the resolution of reconstructed 3D images and thus this affects negatively the recognition of a 3D object in CII. To overcome this problem, we introduce a method to eliminate occluding objects in elemental image array (EIA) and the proposed method is applied to 3D object recognition by use of CII. To our best knowledge, this is the first time to remove occlusion in CII. In our method, we apply the elemental image to sub-image (ES) transform to EIA obtained by a pickup process and those sub-images are employed for occlusion removal. After the transformation, we correlate those sub-images with a reference sub-image to locate occluding objects and then we eliminate the objects. The inverse ES transform provides a modified EIA. Actually, the modified EIA is considered to be an EIA without the object that occludes the object to be reconstructed. This can provide a substantial gain in terms of the image quality of 3D objects and in terms of recognition performance. To verify the usefulness of the proposed technique, some experimental results are carried out and the results are presented.     

3D image reconstruction with a controllable overlapping number of elemental images in computational integral imaging

In this Letter, we propose a three-dimensional(3D) image reconstruction method with a controllable overlapping number of elemental images in computational integral imaging. The proposed method can control the overlapping number of pixels coming from the elemental images by using the subpixel distance based on ray optics between a 3D object and an image sensor. The use of a controllable overlapping number enables us to provide an improved 3D image visualization by controlling the inter-pixel interference within the reconstructed pixels.To find the optimal overlapping number, we simulate the pickup and reconstruction processes and utilize the numerical reconstruction results using a peak signal-to-noise ratio(PSNR) metric. To demonstrate the feasibility of our work in optical experiments, we carry out the preliminary experiments and present the results.     

Three-dimensional integral imaging with improved visualization using subpixel optical ray sensing

In this Letter, we propose an improved three-dimensional (3D) image reconstruction method for integral imaging. We use subpixel sensing of the optical rays of the 3D scene projected onto the image sensor. When reconstructing the 3D image, we use a calculated minimum subpixel distance for each sensor pixel instead of the average pixel value of integrated pixels from elemental images. The minimum subpixel distance is defined by measuring the distance between the center of the sensor pixel and the physical position of the imaging lens point spread function onto the sensor, which is projected from each reconstruction point for all elemental images. To show the usefulness of the proposed method, preliminary 3D imaging experiments are presented. Experimental results reveal that the proposed method may improve 3D imaging visualization because of the superior sensing and reconstruction of optical ray direction and intensity information for 3D objects.     

利用匹配区域的纹理特征改善重构三维图像的视觉质量

研究了计算机重构三维图像时分辨率低的问题,提出一种改善3D计算机全景重构图像的视觉质量的方法,该方法利用3D空间的物体部分在每个元素图像中形成的匹配区域的纹理特征,从两个相邻的元素图像中的匹配区域提取出多个像素,经过加权计算重构出相应的图像区域.该方法与传统的计算机重构方法相比,提高了图像分辨率,同时也消除了从每个元素图像中提取多个像素直接重构图像时存在的"像素块"效应,改善了重构图像的视觉质量.     

Three-dimensional visualization of objects in scattering medium using integral imaging and spectral analysis

This paper presents a three-dimensional visualization method of 3D objects in a scattering medium. The proposed method employs integral imaging and spectral analysis to improve the visual quality of 3D images. The images observed from 3D objects in the scattering medium such as turbid water suffer from image degradation due to scattering. The main reason is that the observed image signal is very weak compared with the scattering signal. Common image enhancement techniques including histogram equalization and contrast enhancement works improperly to overcome the problem. Thus, integral imaging that enables to integrate the weak signals from multiple images was discussed to improve image quality. In this paper, we apply spectral analysis to an integral imaging system such as the computational integral imaging reconstruction. Also, we introduce a signal model with a visibility parameter to analyze the scattering signal. The proposed method based on spectral analysis efficiently estimates the original signal and it is applied to elemental images. The visibility-enhanced elemental images are then used to reconstruct 3D images using a computational integral imaging reconstruction algorithm. To evaluate the proposed method, we perform the optical experiments for 3D objects in turbid water. The experimental results indicate that the proposed method outperforms the existing methods.     

Computational integral imaging reconstruction scheme of far 3D objects by additional use of an imaging lens

In this paper, we propose a novel performance-enhanced computational integral imaging reconstruction (CIIR) scheme by additional use of an imaging lens. In the proposed scheme, elemental images can be obtained by using a simultaneous pickup scheme of far three-dimensional (3D) objects from the lenslet array in both real and virtual image fields. And additional imaging lens produces an image shift effect of 3D objects located far away from the lenslet array and improve the visual quality of reconstructed images in CIIR by overcoming limitation of pickup range in integral imaging. To show the usefulness of the proposed system, some experiments are carried out for real 3D objects and its results are presented.     

No-Reference Quality Assessment for 3D Synthesized Images Based on Visual-Entropy-Guided Multi-Layer Features Analysis

Multiview video plus depth is one of the mainstream representations of 3D scenes in emerging free viewpoint video, which generates virtual 3D synthesized images through a depth-image-based-rendering (DIBR) technique. However, the inaccuracy of depth maps and imperfect DIBR techniques result in different geometric distortions that seriously deteriorate the users’ visual perception. An effective 3D synthesized image quality assessment (IQA) metric can simulate human visual perception and determine the application feasibility of the synthesized content. In this paper, a no-reference IQA metric based on visual-entropy-guided multi-layer features analysis for 3D synthesized images is proposed. According to the energy entropy, the geometric distortions are divided into two visual attention layers, namely, bottom-up layer and top-down layer. The feature of salient distortion is measured by regional proportion plus transition threshold on a bottom-up layer. In parallel, the key distribution regions of insignificant geometric distortion are extracted by a relative total variation model, and the features of these distortions are measured by the interaction of decentralized attention and concentrated attention on top-down layers. By integrating the features of both bottom-up and top-down layers, a more visually perceptive quality evaluation model is built. Experimental results show that the proposed method is superior to the state-of-the-art in assessing the quality of 3D synthesized images.     

Axially distributed sensing for three-dimensional imaging with unknown sensor positions

Recently, an axially distributed sensing system was proposed for three-dimensional (3D) imaging where the sensors are distributed along the optical axis. In this previously reported system, a priori knowledge of exact sensor positions was required for 3D image reconstruction. In this Letter, we present an axially distributed sensing with unknown sensor positions along the optical axis. In this system, only the relative positions of two sensors are needed, whereas all other sensor positions are assumed unknown. Experiments are presented to illustrate the feasibility of the proposed system and illustrate the visual quality of reconstructed 3D images by using the proposed calibrated sensor positions. To the best of our knowledge, this is the first report on axially distributed sensing with unknown sensor positions.     

Geometric analysis of spatial distortion in projection-type integral imaging

In projection-type integral imaging, positional errors in elemental images and elemental lenses affect three-dimensional (3D) image quality. We analyzed the relationships between the geometric distortion in elemental images caused by a projection lens and the spatial distortion in the reconstructed 3D image. As a result, we clarified that 3D images that were reconstructed far from the lens array were largely affected, and that the reconstructed images were significantly distorted in the depth direction at the corners of the displayed images.     

Depth-controlled reconstruction of 3D integral image using synthesized intermediate sub-images

In this paper, we propose a method that controls the depth of the three-dimensional (3D) object existing over the depth-of-focus in integral imaging. The depth control method is performed only in a computer by synthesizing the intermediate sub-images between original sub-images obtained by transforming the captured elemental images. In the reconstruction process, we can obtain reconstructed 3D images with the better image quality within depth-of-focus than that reconstructed over the depth-of-focus. To demonstrate the feasibility of our method, optical and computational experiments are carried out and its results are presented.     

Computational integral imaging reconstruction method of 3D images using pixel-to-pixel mapping and image interpolation

In this paper, we propose a novel computational integral imaging reconstruction (CIIR) method to improve the visual quality of the reconstructed images using a pixel-to-pixel mapping and an interpolation technique. Since an elemental image is magnified inversely through the corresponding pinhole and mapped on the reconstruction output plane based on pinhole-array model in the conventional CIIR method, the visual quality of reconstructed output image (ROI) degrades due to the interference problem between adjacent pixels during the superposition of the magnified elemental images. To avoid this problem, the proposed CIIR method generates dot-pattern ROIs using a pixel-to-pixel mapping and substitutes interpolated values for the empty pixels within the dot-pattern ROIs using an interpolation technique. The interpolated ROIs provides a much improved visual quality compared with the conventional method because of the exact regeneration of pixel positions sampled in the pickup process without interference between pixels. Moreover, it can enable us to reduce a computational cost by eliminating the magnification process used in the conventional CIIR. To confirm the feasibility of the proposed system, some experiments are carried out and the results are presented.     

Designing a genetic watermarking approach for 3D scenes

A new watermarking algorithm based on genetic algorithm (GA) in the transform domain is proposed. Unlike the existing computer-generated integral imaging based watermarking methods, the proposed method utilizes GA searching to the optimized transform domain to serve as a trade-off for watermark embedding. In this paper, 3D scene to be captured by using a virtual pinhole array and be computationally recorded as an elemental image array (EIA), watermarking with GA optimization and computer-generated holography is implemented. In the proposed GA optimization process, we utilize the fitness function to improve the visual quality of watermarked images and the robustness. Simulation results show that the proposed algorithm yields a holographic watermark that is imperceptibility to human eyes and robust to standard watermarking attacks. A comparison of the proposed watermarking method to the existing similar watermarking methods demonstrated that the proposed method generally outperforms completing methods in terms of imperceptibility and robustness.     

Enhanced computational integral imaging system for partially occluded 3D objects using occlusion removal technique and recursive PCA reconstruction

In this paper, we propose an enhanced computational integral imaging system by both eliminating the occlusion in the elemental images recorded from the partially occluded 3D object and recovering the entire elemental images of the 3D object. In the proposed system, we first obtain the elemental images for partially occluded object using computational integral imaging system and it is transformed to sub-images. Then we eliminate the occlusion within the sub-images by use of an occlusion removal technique. To compensate the removed part from occlusion-removed sub-images, we use a recursive application of PCA reconstruction and error compensation. Finally, we generate the entire elemental images without a loss from the newly reconstructed sub-images and perform the process of object recognition. To show the usefulness of the proposed system, we carry out the computational experiments for face recognition and its results are presented. Our experimental results show that the proposed system might improve the recognition performance dramatically.     

Modified smart pixel mapping method for displaying orthoscopic 3D images in integral imaging

In this paper, we propose a modified smart pixel mapping (MSPM) method for displaying orthoscopic three-dimensional (3D) images with a function of depth control in integral imaging system. In the proposed MSPM, the depth-converted elemental image array (EIA) is obtained through the pixel mapping process and the image interpolation technique. The proposed method gives us the depth conversion at distances different from the position of 3D object and provides various types of EIAs using only an original EIA for orthoscopic images. To show the usefulness of the proposed method, we carry out the preliminary experiments and present the experimental results.     

Robust visual correspondence computation using monogenic curvature phase based mutual information

Visual correspondence has been a major research topic in the fields of image registration, 3D reconstruction, and object tracking for some decades. However, due to the radiometric variations of images, conventional approaches fail to produce robust matching results. The traditional method of intensity-based mutual information performs very good for global variations between images, however, its performance degrades in the case of local radiometric variations. Monogenic curvature phase information, as an important local feature of the image, has the advantage of being robust against brightness variation. Hence, in this Letter, we propose an approach to compute the visual correspondence by coupling the advantages of mutual information and monogenic curvature phase. Experimental results demonstrate that the proposed approach can work robustly under radiometric variations.     

Three-dimensional photon counting integral imaging using moving array lens technique

In this Letter, we present three-dimensional (3D) photon counting integral imaging using the moving array-lens technique (MALT) to improve the visualization of a reconstructed 3D scene. In 3D scene reconstruction of photon counting integral imaging, various techniques such as maximum likelihood estimation may be used. However, the visual quality depends on the number of scene photons or detector pixels activated by photons. We show that MALT may improve the viewing resolution of integral imaging for reconstructed 3D scene under photon-starved conditions.     

A novel three-dimensional digital watermarking scheme basing on integral imaging

A novel integral imaging-based three-dimensional (3D) digital watermarking scheme is presented. In the proposed method, an elemental image array (EIA) obtained by recording the rays coming from a 3D object through a pinhole array in the integral imaging system is employed as a new 3D watermark. The EIA is composed of a number of small elemental images having their own perspectives of a 3D object, and from this recorded EIA various depth-dependent 3D object images can be reconstructed by using the computational integral imaging reconstruction (CIIR) technique. This 3D property of the EIA watermark can make a robust reconstruction of the watermark image available even though there are some data losses in the embedded watermark by attacks. To show the robustness of the proposed scheme against attacks, some experiments are carried out and the results are discussed.