Occlusion removal technique for improved recognition of partially occluded 3D objects in computational integral imaging

In this paper, we propose an occlusion removal technique for improved recognition of 3D objects that are partially occluded in computational integral imaging (CII). In the reconstruction process of a 3D object which is partially occluded by other objects, occlusion degrades the resolution of reconstructed 3D images and thus this affects negatively the recognition of a 3D object in CII. To overcome this problem, we introduce a method to eliminate occluding objects in elemental image array (EIA) and the proposed method is applied to 3D object recognition by use of CII. To our best knowledge, this is the first time to remove occlusion in CII. In our method, we apply the elemental image to sub-image (ES) transform to EIA obtained by a pickup process and those sub-images are employed for occlusion removal. After the transformation, we correlate those sub-images with a reference sub-image to locate occluding objects and then we eliminate the objects. The inverse ES transform provides a modified EIA. Actually, the modified EIA is considered to be an EIA without the object that occludes the object to be reconstructed. This can provide a substantial gain in terms of the image quality of 3D objects and in terms of recognition performance. To verify the usefulness of the proposed technique, some experimental results are carried out and the results are presented.     

Enhanced computational integral imaging system for partially occluded 3D objects using occlusion removal technique and recursive PCA reconstruction

In this paper, we propose an enhanced computational integral imaging system by both eliminating the occlusion in the elemental images recorded from the partially occluded 3D object and recovering the entire elemental images of the 3D object. In the proposed system, we first obtain the elemental images for partially occluded object using computational integral imaging system and it is transformed to sub-images. Then we eliminate the occlusion within the sub-images by use of an occlusion removal technique. To compensate the removed part from occlusion-removed sub-images, we use a recursive application of PCA reconstruction and error compensation. Finally, we generate the entire elemental images without a loss from the newly reconstructed sub-images and perform the process of object recognition. To show the usefulness of the proposed system, we carry out the computational experiments for face recognition and its results are presented. Our experimental results show that the proposed system might improve the recognition performance dramatically.     

Three-dimensional imaging and visualization of partially occluded objects using axially distributed stereo image sensing

In this Letter, we propose a multiperspective three-dimensional (3D) imaging system using axially distributed stereo image sensing. In this proposed method, the stereo camera is translated along its optical axis and multiple axial elemental image pairs for a 3D scene are collected. The captured elemental images are reconstructed in 3D using a computational reconstruction algorithm based on ray back-projection. The proposed method is applied to partially occluded object visualization. Optical experiments are performed to verify the approach.     

Computational integral imaging reconstruction scheme of far 3D objects by additional use of an imaging lens

In this paper, we propose a novel performance-enhanced computational integral imaging reconstruction (CIIR) scheme by additional use of an imaging lens. In the proposed scheme, elemental images can be obtained by using a simultaneous pickup scheme of far three-dimensional (3D) objects from the lenslet array in both real and virtual image fields. And additional imaging lens produces an image shift effect of 3D objects located far away from the lenslet array and improve the visual quality of reconstructed images in CIIR by overcoming limitation of pickup range in integral imaging. To show the usefulness of the proposed system, some experiments are carried out for real 3D objects and its results are presented.     

Three-dimensional visualization of objects in scattering medium using integral imaging and spectral analysis

This paper presents a three-dimensional visualization method of 3D objects in a scattering medium. The proposed method employs integral imaging and spectral analysis to improve the visual quality of 3D images. The images observed from 3D objects in the scattering medium such as turbid water suffer from image degradation due to scattering. The main reason is that the observed image signal is very weak compared with the scattering signal. Common image enhancement techniques including histogram equalization and contrast enhancement works improperly to overcome the problem. Thus, integral imaging that enables to integrate the weak signals from multiple images was discussed to improve image quality. In this paper, we apply spectral analysis to an integral imaging system such as the computational integral imaging reconstruction. Also, we introduce a signal model with a visibility parameter to analyze the scattering signal. The proposed method based on spectral analysis efficiently estimates the original signal and it is applied to elemental images. The visibility-enhanced elemental images are then used to reconstruct 3D images using a computational integral imaging reconstruction algorithm. To evaluate the proposed method, we perform the optical experiments for 3D objects in turbid water. The experimental results indicate that the proposed method outperforms the existing methods.     

3D reconstruction of a carotid bifurcation from 2D transversal ultrasound images

Visualizing and analyzing the morphological structure of carotid bifurcations are important for understanding the etiology of carotid atherosclerosis, which is a major cause of stroke and transient ischemic attack. For delineation of vasculatures in the carotid artery, ultrasound examinations have been widely employed because of a noninvasive procedure without ionizing radiation. However, conventional 2D ultrasound imaging has technical limitations in observing the complicated 3D shapes and asymmetric vasodilation of bifurcations. This study aims to propose image-processing techniques for better 3D reconstruction of a carotid bifurcation in a rat by using 2D cross-sectional ultrasound images. A high-resolution ultrasound imaging system with a probe centered at 40 MHz was employed to obtain 2D transversal images. The lumen boundaries in each transverse ultrasound image were detected by using three different techniques; an ellipse-fitting, a correlation mapping to visualize the decorrelation of blood flow, and the ellipse-fitting on the correlation map. When the results are compared, the third technique provides relatively good boundary extraction. The incomplete boundaries of arterial lumen caused by acoustic artifacts are somewhat resolved by adopting the correlation mapping and the distortion in the boundary detection near the bifurcation apex was largely reduced by using the ellipse-fitting technique. The 3D lumen geometry of a carotid artery was obtained by volumetric rendering of several 2D slices. For the 3D vasodilatation of the carotid bifurcation, lumen geometries at the contraction and expansion states were simultaneously depicted at various view angles. The present 3D reconstruction methods would be useful for efficient extraction and construction of the 3D lumen geometries of carotid bifurcations from 2D ultrasound images.     

Regeneration of elemental images in integral imaging for occluded objects using a plenoptic camera

In this Letter, we propose an elemental image regeneration method of three-dimensional(3D) integral imaging for occluded objects using a plenoptic camera. In conventional occlusion removal techniques, the information of the occlusion layers may be lost. Thus, elemental images have cracked parts, so the visual quality of the reconstructed 3D image is degraded. However, these cracked parts can be interpolated from adjacent elemental images. Therefore, in this Letter, we try to improve the visual quality of reconstructed 3D images by interpolating and regenerating virtual elemental images with adjacent elemental images after removing the occlusion layers. To prove our proposed method, we carry out optical experiments and calculate performance metrics such as the mean square error(MSE) and the peak signal-to-noise ratio(PSNR).     

Optical display of true 3D objects in depth-priority integral imaging using an active sensor

In this paper, we propose a system combining the pickup process using an active sensor and the display process using depth-priority integral imaging (DPII) system to display true three-dimensional (3D) objects within large depth through real and virtual image fields. The active sensor provides depth map and color images of 3D objects. Using captured depth map and original color images, elemental images are computationally synthesized and displayed optically in DPII system. Proposed system provides scaling of 3D scenes for true 3D object. To show the usefulness of proposed system, we carry out the experiment for true 3D objects of three character patterns and present the experimental results.     

Performance-enhanced recognition of a far and partially occluded 3-D object by use of direct pixel-mapping in computational curving-effective integral imaging

In this paper, we propose a novel approach to enhance the recognition performance of a far and partially occluded three-dimensional (3-D) target in computational curving-effective integral imaging by using the direct pixel-mapping (DPM) method. With this scheme, the elemental image array (EIA) originally picked up from a far and partially occluded 3-D target can be converted into a new EIA just like the one virtually picked up from a target located close to the lenslet array. Due to this characteristic of DPM, resolution and quality of the reconstructed target image can be highly enhanced, which results in a significant improvement of recognition performance of a far 3-D object. In addition, the computational time required for reconstruction of a far 3-D target could be also reduced because the distance between the lenslet array and image plane is virtually shortened in the new EIA transformed by DPM. Experimental results reveal that image quality of the reconstructed target image and object recognition performance of the proposed system have been improved by 1.75 dB and 4.56% on the average in PSNR (peak-to-peak signal-to-noise ratio) and NCC (normalized correlation coefficient), respectively, compared to the conventional system.     

Three-dimensional integral imaging with improved visualization using subpixel optical ray sensing

In this Letter, we propose an improved three-dimensional (3D) image reconstruction method for integral imaging. We use subpixel sensing of the optical rays of the 3D scene projected onto the image sensor. When reconstructing the 3D image, we use a calculated minimum subpixel distance for each sensor pixel instead of the average pixel value of integrated pixels from elemental images. The minimum subpixel distance is defined by measuring the distance between the center of the sensor pixel and the physical position of the imaging lens point spread function onto the sensor, which is projected from each reconstruction point for all elemental images. To show the usefulness of the proposed method, preliminary 3D imaging experiments are presented. Experimental results reveal that the proposed method may improve 3D imaging visualization because of the superior sensing and reconstruction of optical ray direction and intensity information for 3D objects.     

Resolution-enhanced 3D image correlator using computationally reconstructed integral images

In this paper, a novel resolution-enhanced three-dimensional (3D) image correlator using the computationally reconstructed integral images is proposed in order to extract target object’s 3D location data in a scene. Elemental images of the reference and target objects are picked up by lenslet arrays and using these elemental images, reference and target plane images are reconstructed on the output plane by means of a modified computational integral imaging reconstruction technique. Then, through cross-correlations between the reconstructed reference and the target plane images, 3D location data of the target object can be extracted from the correlation outputs. With the purpose of showing the feasibility of the proposed method, some computational and optical experiments on the target objects in space are carried out and the results are presented.     

Volume-resolved flame chemiluminescence and laser-induced fluorescence imaging

There is significant need for optical diagnostic techniques to measure instantaneous volumetric vector and scalar distributions in fluid flows and combustion processes. This is especially true for investigations where only limited optical access is available, such as in internal combustion engines, furnaces, flow reactors, etc. While techniques such as tomographic PIV for velocity measurement have emerged and reached a good level of maturity, instantaneous 3D measurements of scalar quantities are not available at the same level. Recently, developments in light field technology have progressed to a degree where implementation into scientific 3D imaging becomes feasible. Others have already demonstrated the utility of light field technology toward imaging high-contrast particles for PIV and for imaging flames when treated as single-surface objects. Here, the applicability and shortcomings of current commercially available light field technology toward volumetric imaging of translucent scalar distributions and flames are investigated. Results are presented from imaging canonical chemiluminescent and laser-induced fluorescent systems. While the current light field technology is able to qualitatively determine the position of surfaces by locating high-contrast features, the correlation-based reconstruction algorithm is unable to fully reconstruct the imaged objects for quantitative diagnostics. Current analysis algorithms are based on high-contrast correlation schemes, and new tools, possibly based on tomographic concepts, will have to be implemented to reconstruct the full 3D structure of translucent objects for quantitative analysis.     

Enhancing the discrimination accuracy between metastases, gliomas and meningiomas on brain MRI by volumetric textural features and ensemble pattern recognition methods

Three-dimensional (3D) texture analysis of volumetric brain magnetic resonance (MR) images has been identified as an important indicator for discriminating among different brain pathologies. The purpose of this study was to evaluate the efficiency of 3D textural features using a pattern recognition system in the task of discriminating benign, malignant and metastatic brain tissues on T1 postcontrast MR imaging (MRI) series. The dataset consisted of 67 brain MRI series obtained from patients with verified and untreated intracranial tumors. The pattern recognition system was designed as an ensemble classification scheme employing a support vector machine classifier, specially modified in order to integrate the least squares features transformation logic in its kernel function. The latter, in conjunction with using 3D textural features, enabled boosting up the performance of the system in discriminating metastatic, malignant and benign brain tumors with 77.14%, 89.19% and 93.33% accuracy, respectively. The method was evaluated using an external cross-validation process; thus, results might be considered indicative of the generalization performance of the system to "unseen" cases. The proposed system might be used as an assisting tool for brain tumor characterization on volumetric MRI series.     

Three-dimensional free view reconstruction in axially distributed image sensing

In this Letter, we propose a three-dimensional(3D) free view reconstruction technique in axially distributed image sensing(ADS). In typical integral imaging, free view reconstructed images can be obtained by tilting all elemental images or tilting the reconstruction plane due to large lateral perspectives for 3D objects. In conventional ADS, the reconstructed images at only a front view can be generated since the sensor is moved along with its optical axis so that it has small lateral perspectives for 3D objects. However, the reconstructed 3D images at any viewing point may be obtained because the virtual viewing camera may capture these slightly different perspectives for 3D objects. Therefore, in this Letter, we employ the virtual viewing camera to visualize the 3D images at the arbitrary viewing point. To support our proposed method, we show the experimental results.     

三维磁共振电阻抗成像的改进分层灵敏度算法

针对人体组织电导率的三维成像问题,提出一种改进的分层灵敏度磁共振电阻抗重建算法.利用单方向磁感应强度信息,对三维电导率图像实行分层重建,每层重建仅利用该层磁通密度分量测量数据,然后对单层重建结果进行修正以获得三维电导率重建图像.三介质长方体模型上的仿真实验证明,改进的分层重建算法改善了层间串扰现象,可以获得比一般分层算法甚至整体算法更高的图像分辨率,而且重建时间较整体算法显著减少;基于人体腿模型的仿真实验表明该算法对复杂模型三维重构的可行性;最后通过仿体实验验证算法的重建效果.改进的分层灵敏度重建算法降低了灵敏度矩阵法的计算机硬件需求,减少了重建时间,对MREIT的三维重建具有较高的成像精度和求解效率.     

Enhanced resolution for EPR imaging by two-step deblurring

The broad spectrum of spin probes used for electron paramagnetic resonance imaging (EPRI) result in poor spatial resolution of the reconstructed images. Conventional deconvolution procedures can enhance the resolution to some extent but obtaining high resolution EPR images is still a challenge. In this work, we have implemented and analyzed the performance of a postacquisition deblurring technique to enhance the spatial resolution of the EPR images. The technique consists of two steps; noniterative deconvolution followed by iterative deconvolution of the acquired projections which are then projected back using filtered backprojection (FBP) to reconstruct a high resolution image. Further, we have proposed an analogous technique for iterative reconstruction algorithms such as multiplicative simultaneous iterative reconstruction technique (MSIRT) which can be a method of choice for many applications. The performance of the suggested deblurring approach is evaluated using computer simulations and EPRI experiments. Results suggest that the proposed procedure is superior to the standard FBP and standard iterative reconstruction algorithms in terms of mean-square-error (MSE), spatial resolution, and visual judgment. Although the procedure is described for 2D imaging, it can be readily extended to 3D imaging.     

Three-dimensional diffuse optical imaging of hand joints: System description and phantom studies

In this paper we describe a three-dimensional (3D) continuous wave (CW) diffuse optical tomography (DOT) system and present 3D volumetric reconstruction studies using this DOT system with simple phantom models that simulate hand joints. The CCD-based DOT system consists of 64×64 source/detector fiber optic channels, which are arranged in four layers, forming a cylindrical fiber optic/tissue interface. Phantom experiments are used to evaluate system performance with respective to axial spatial resolution, optical contrast and target position for detection of osteoarthritis where cartilage is the primary target region of interest. These phantom studies suggest that we are able to quantitatively resolve a 2 mm thick “cartilage” and qualitatively resolve a 1 mm thick “cartilage” using our 3D reconstruction approach. Our results also show that optical contrast of 3:1–7:1 between the “disease cartilage” and normal cartilage can be quantitatively recovered. Finally, the target position along axial direction on image reconstruction is studied. All the images are obtained using our 3D finite-element-based reconstruction algorithm.     

Optoranger: A 3D pattern matching method for bin picking applications

This paper presents a new method, based on 3D vision, for the recognition of free-form objects in the presence of clutters and occlusions, ideal for robotic bin picking tasks. The method can be considered as a compromise between complexity and effectiveness. A 3D point cloud representing the scene is generated by a triangulation-based scanning system, where a fast camera acquires a blade projected by a laser source. Image segmentation is based on 2D images, and on the estimation of the distances between point pairs, to search for empty areas. Object recognition is performed using commercial software libraries integrated with custom-developed segmentation algorithms, and a database of model clouds created by means of the same scanning system.Experiments carried out to verify the performance of the method have been designed by randomly placing objects of different types in the Robot work area. The preliminary results demonstrate the excellent ability of the system to perform the bin picking procedure, and the reliability of the method proposed for automatic recognition of identity, position and orientation of the objects.     

Improved depth extraction method of 3D objects using computational integral imaging reconstruction based on multiple windowing techniques

This paper presents an improved depth extraction method of 3D objects using computational integral imaging reconstruction (CIIR) based on the multiple windowing models. The proposed method records 3D objects using the lenslet array; and it reconstructs multiple sets of slice images from multiple CIIR methods based on the different windowing models. A depth map is then extracted by a block matching algorithm among multiple set of slice images. A preliminary experiment is carried out to show the feasibility of the proposed method. Experimental results indicate the proposed method outperforms the previous method with two windowing models.     

Three-dimensional optical microscopy using axially distributed image sensing

We propose three-dimensional (3D) optical microscopy using axially distributed image sensing. In the proposed method, the micro-objects are optically magnified and their axially distributed images are recorded by moving the image sensor along a common optical axis. The 3D volumetric images are generated from the recorded axial image set using a computational reconstruction algorithm based on ray backprojection. Preliminary experimental results are presented. To the best of our knowledge, this is the first report on 3D optical microscopy using axially distributed sensing.