3DSEM++: Adaptive and intelligent 3D SEM surface reconstruction

Structural analysis of microscopic objects is a longstanding topic in several scientific disciplines, such as biological, mechanical, and materials sciences. The scanning electron microscope (SEM), as a promising imaging equipment has been around for decades to determine the surface properties (e.g., compositions or geometries) of specimens by achieving increased magnification, contrast, and resolution greater than one nanometer. Whereas SEM micrographs still remain two-dimensional (2D), many research and educational questions truly require knowledge and facts about their three-dimensional (3D) structures. 3D surface reconstruction from SEM images leads to remarkable understanding of microscopic surfaces, allowing informative and qualitative visualization of the samples being investigated. In this contribution, we integrate several computational technologies including machine learning, contrario methodology, and epipolar geometry to design and develop a novel and efficient method called 3DSEM++ for multi-view 3D SEM surface reconstruction in an adaptive and intelligent fashion. The experiments which have been performed on real and synthetic data assert the approach is able to reach a significant precision to both SEM extrinsic calibration and its 3D surface modeling.     

3D reconstruction of road surfaces using an integrated multi-sensory approach

In this paper, we present our experience in building a mobile imaging system that incorporates multi-modality sensors for road surface mapping and inspection applications. Our proposed system leverages 3D laser-range sensors, video cameras, global positioning systems (GPS) and inertial measurement units (IMU) towards the generation of photo-realistic, geometrically accurate, geo-referenced 3D models of road surfaces. Based on our summary of the state-of-the-art systems for a road distress survey, we identify several challenges in the real-time deployment, integration and visualization of the multi-sensor data. Then, we present our data acquisition and processing algorithms as a novel two-stage automation procedure that can meet the accuracy requirements with real-time performance. We provide algorithms for 3D surface reconstruction to process the raw data and deliver detail preserving 3D models that possess accurate depth information for characterization and visualization of cracks as a significant improvement over contemporary commercial video-based vision systems.     

Depth-controlled reconstruction of 3D integral image using synthesized intermediate sub-images

In this paper, we propose a method that controls the depth of the three-dimensional (3D) object existing over the depth-of-focus in integral imaging. The depth control method is performed only in a computer by synthesizing the intermediate sub-images between original sub-images obtained by transforming the captured elemental images. In the reconstruction process, we can obtain reconstructed 3D images with the better image quality within depth-of-focus than that reconstructed over the depth-of-focus. To demonstrate the feasibility of our method, optical and computational experiments are carried out and its results are presented.     

Sectional image reconstruction and three-dimensional microscopy of small particles by angular spectrum method

In this letter, we demonstrate sectional image reconstruction and three-dimensional microscopy of small particles. We demonstrate sectional image reconstruction and holographic methods to obtain 2D and 3D images of small particles. A single hologram is sufficient to obtain a section containing only the focused parts of the reconstructed image. One can obtain images of different plane sections of a specimen in addition to its 3D display. The reconstruction of a digital hologram is based on the plane-wave expansion of the diffracted wave fields using Fourier optics (this method is also known as the angular spectrum method). With this method, the object-to-hologram distance can be quite small because the minimum-distance requirement does not apply. Furthermore, numerical reconstruction of transparent objects by this method may be interesting for micro-structure measurement.     

Reconstruction of 3D ultrasound images based on Cyclic Regularized Savitzky-Golay filters

This paper presents a new three-dimensional (3D) ultrasound reconstruction algorithm for generation of 3D images from a series of two-dimensional (2D) B-scans acquired in the mechanical linear scanning framework. Unlike most existing 3D ultrasound reconstruction algorithms, which have been developed and evaluated in the freehand scanning framework, the new algorithm has been designed to capitalize the regularity pattern of the mechanical linear scanning, where all the B-scan slices are precisely parallel and evenly spaced. The new reconstruction algorithm, referred to as the Cyclic Regularized Savitzky-Golay (CRSG) filter, is a new variant of the Savitzky-Golay (SG) smoothing filter. The CRSG filter has been improved upon the original SG filter in two respects: First, the cyclic indicator function has been incorporated into the least square cost function to enable the CRSG filter to approximate nonuniformly spaced data of the unobserved image intensities contained in unfilled voxels and reduce speckle noise of the observed image intensities contained in filled voxels. Second, the regularization function has been augmented to the least squares cost function as a mechanism to balance between the degree of speckle reduction and the degree of detail preservation. The CRSG filter has been evaluated and compared with the Voxel Nearest-Neighbor (VNN) interpolation post-processed by the Adaptive Speckle Reduction (ASR) filter, the VNN interpolation post-processed by the Adaptive Weighted Median (AWM) filter, the Distance-Weighted (DW) interpolation, and the Adaptive Distance-Weighted (ADW) interpolation, on reconstructing a synthetic 3D spherical image and a clinical 3D carotid artery bifurcation in the mechanical linear scanning framework. This preliminary evaluation indicates that the CRSG filter is more effective in both speckle reduction and geometric reconstruction of 3D ultrasound images than the other methods.     

3D image reconstruction with a controllable overlapping number of elemental images in computational integral imaging

In this Letter, we propose a three-dimensional(3D) image reconstruction method with a controllable overlapping number of elemental images in computational integral imaging. The proposed method can control the overlapping number of pixels coming from the elemental images by using the subpixel distance based on ray optics between a 3D object and an image sensor. The use of a controllable overlapping number enables us to provide an improved 3D image visualization by controlling the inter-pixel interference within the reconstructed pixels.To find the optimal overlapping number, we simulate the pickup and reconstruction processes and utilize the numerical reconstruction results using a peak signal-to-noise ratio(PSNR) metric. To demonstrate the feasibility of our work in optical experiments, we carry out the preliminary experiments and present the results.     

A high dynamic range structured light means for the 3D measurement of specular surface

This paper presents a novel structured light approach for the 3D reconstruction of specular surface. The binary shifting strip is adopted as structured light pattern instead of conventional sinusoidal pattern. Based on the framework of conventional High Dynamic Range (HDR) imaging technique, an efficient means is first introduced to estimate the camera response function. And then, dynamic range of the generated radiance map is compressed in the gradient domain by introducing an attenuation function. Subject to the change of lighting conditions caused by projecting different structured light patterns, the structure light image with middle exposure level is selected as the reference image and used for the slight adjustment of the primary fused image. Finally, the regenerated structured light images with well exposing condition are used for 3D reconstruction of the specular surface. To evaluate performance of the method, some stainless stamping parts with strong reflectivity are used for the experiments. And the results showed that, different specular targets with various shapes can be precisely reconstructed by the proposed method.     

Tilted objects EFI extracted at once by 3D output of the angular spectrum method

Since many optical imaging systems suffer from a limited depth of focus, three-dimensional (3D) objects, or even flat objects tilted with respect to the optical axis, are imaged with only a portion of them in focus. Typically, to overcome this problem, in-focus areas are extracted from different images taken at different depths. The in-focus regions are thus merged together to build the extended focus image (EFI). In this work, we propose a quasi-automatic method for the EFI construction of tilted objects, extracted at once by 3D output of the Angular Spectrum Method (ASM) from a single digitally recorded hologram. Results show that our method can be effectively used for the correction of the anamorphism due to the reconstruction on a tilted plane with respect to the hologram one.     

3D ultrasound reconstruction algorithms from analog and digital data

Freehand 3D ultrasound is increasingly being introduced in the clinic for diagnostics and image-assisted interventions. Various algorithms exist for combining 2D images of regular ultrasound probes to 3D volumes, being either voxel-, pixel- or function-based. Previously, the most commonly used input to 3D ultrasound reconstruction has been digitized analog video. However, recent scanners that offer access to digital image frames exist, either as processed or unprocessed data. To our knowledge, no comparison has been performed to determine which data source gives the best reconstruction quality. In the present study we compared both reconstruction algorithms and data sources using novel comparison methods for detecting potential differences in image quality of the reconstructed volumes. The ultrasound scanner used in this study was the Sonix RP from Ultrasonix Medical Corp (Richmond, Canada), a scanner that allow third party access to unprocessed and processed digital data. The ultrasound probe used was the L14-5/38 linear probe. The assessment is based on a number of image criteria: detectability of wire targets, spatial resolution, detectability of small barely visible structures, subjective tissue image quality, and volume geometry. In addition we have also performed the more “traditional” comparison of reconstructed volumes by removing a percentage of the input data. By using these evaluation methods and data from the specific scanner, the results showed that the processed video performed better than the digital scan-line data, digital video being better than analog video. Furthermore, the results showed that the choice of video source was more important than the choice of tested reconstruction algorithms.     

小鼠卵母细胞染色体三维双光子荧光图像的轴向衰减

双光子荧光显微镜作为一种高分辨光学仪器,已经被广泛应用于生物样品的非侵入式三维光学成像中。相比共聚焦显微镜,双光子荧光显微镜拥有更深的探测深度。然而,即便如此,在对较厚的生物样品进行非侵入式光学三维成像时,样品的成像质量也往往会随着探测深度的增加而下降。在临床和生物学领域对研究母性遗传起重要作用的小鼠卵母细胞拥有较大的直径(80～100 μm),吸收和散射效应较为明显。本文研究小鼠卵母细胞染色体的三维双光子荧光图像随探测深度增加图像质量的衰减程度。通过对所得图像进行轴向衰减矫正,利用体积作为参数,将矫正前后小鼠卵母细胞内染色体三维双光子荧光图像进行对比。结果表明,由于吸收和散射效应,卵母细胞存在较严重的光学轴向衰减问题,因此,对用双光子荧光三维成像手段获得的小鼠卵母细胞图像进行衰减矫正是有必要的。这为进一步精确定量的研究卵母细胞内染色体的三维构像打下良好的基础。     

Occlusion removal technique for improved recognition of partially occluded 3D objects in computational integral imaging

In this paper, we propose an occlusion removal technique for improved recognition of 3D objects that are partially occluded in computational integral imaging (CII). In the reconstruction process of a 3D object which is partially occluded by other objects, occlusion degrades the resolution of reconstructed 3D images and thus this affects negatively the recognition of a 3D object in CII. To overcome this problem, we introduce a method to eliminate occluding objects in elemental image array (EIA) and the proposed method is applied to 3D object recognition by use of CII. To our best knowledge, this is the first time to remove occlusion in CII. In our method, we apply the elemental image to sub-image (ES) transform to EIA obtained by a pickup process and those sub-images are employed for occlusion removal. After the transformation, we correlate those sub-images with a reference sub-image to locate occluding objects and then we eliminate the objects. The inverse ES transform provides a modified EIA. Actually, the modified EIA is considered to be an EIA without the object that occludes the object to be reconstructed. This can provide a substantial gain in terms of the image quality of 3D objects and in terms of recognition performance. To verify the usefulness of the proposed technique, some experimental results are carried out and the results are presented.     

基于IR-SFS算法空间目标红外影像3D重建

在利用单幅影像的明暗恢复形状（Shape From Shading,SFS）三维重构算法的基础上,提出同时考虑外界辐射源和目标自身红外辐射的IR-SFS（infrared-SFS）算法。首先通过分析SFS算法原理和空间目标的红外成像特性,建立IR-SFS辐射方程并进行了仿真研究,然后利用温度场估计获得红外差值图,在人工合成的理想半球体、半圆柱体卫星红外影像上进行算法测试,并以美国STS107真实红外影像作为实验目标进行三维重建。实验结果表明,所提出IR-SFS算法经过参数优化后,与原SFS算法相比,重建模型的峰值信噪比更高,对STS107顶部舱门、尾翼、机舱、机舱内方形部件具有更佳显示度,整体效果得到明显改善。     

Point-based integration for 3D object reconstruction from Ladar range images

A method of point-based integration from multiple registered laser radar (Ladar) range images for 3D object reconstruction is presented in this literature. Our method operates surface point data directly and provides a direct way for overlapping data removal. The overlapping areas were detected using a single distance threshold. In order to set the distance threshold, a local point density approach is introduced to solve it. Compared with the mesh-based, volumetric and other points-based integration approaches, our method is simple and fast, and need less storage memory. The approach is performed on various objects with different geometric shapes. The experimental results demonstrate the efficiency and feasibility of our method.     

3D卷积自动编码网络的高光谱异常检测

高光谱图像包含丰富的地物光谱信息,在遥感图像领域有着巨大的发展前景.高光谱图像异常检测无需任何先验光谱信息,便可检测出图像中的异常目标.因此,在国防军事和民用领域都有广泛的应用,是现阶段高光谱图像处理领域的研究热点.然而,高光谱图像存在数据复杂、冗余性强、未标记以及样本数量少等特点,这给高光谱图像异常检测带来了很大的挑...     

Volume reconstruction of freehand three-dimensional ultrasound using median filters

Objectives

This paper aims to apply median filters for reducing interpolation error and improving the quality of 3D images in a freehand 3D ultrasound (US) system.

Background and motivation

Freehand 3D US imaging has been playing an important role in obtaining the entire 3D impression of tissues and organs. Reconstructing a sequence of irregularly located 2D US images (B-scans) into a 3D data set is one of the key procedures for visualization and data analysis.

Methods

In this study, we investigated the feasibility of using median filters for the reconstruction of 3D images in a freehand 3D US system. The B-scans were collected using a 7.5 MHz ultrasound probe. Four algorithms including the standard median (SM), Gaussian weighted median (GWM) and two types of distance-weighted median (DWM) filters were proposed to filter noises and compute voxel intensities. Qualitative and quantitative comparisons were made among the results of different methods based on the image set captured in freehand from the forearm of a healthy subject. A leave-one-out approach was used to demonstrate the performance of the median filters for predicting the removed B-scan pixels.

Results

Compared with the voxel nearest-neighbourhood (VNN) and distance-weighted (DW) interpolation methods, the four median filters reduced the interpolation error by 8.0-24.0% and 1.2-21.8%, respectively, when 1/4 to 5 B-scans was removed from the raw B-scan sequence.

Conclusions

In summary, the median filters can improve the quality of volume reconstruction by reducing the interpolation errors and facilitate the following image analyses in clinical applications.     

3D reconstruction of the brain from magnetic resonance images using a connectivity algorithm

We present high resolution three dimensional (3D) connectivity, surface construction and display algorithms that detect, extract, and display the surface of a brain from contiguous magnetic resonance (MR) images. The algorithms identify the external brain surface and create a 3D image, showing the fissures and surface convolutions of the cerebral hemispheres, cerebellum, and brain stem. Images produced by these algorithms also show the morphology of other soft tissue boundaries such as the cerebral ventricular system and the skin of the patient. For the purposes of 3D reconstruction, our experiments show that T1 weighted images give better contrast between the surface of the brain and the cerebral spinal fluid than T2 weighted images. 3D reconstruction of MR data provides a non-invasive procedure for examination of the brain surface and other anatomical features.     

基于空间光调制器的多层图像的构建与可视化研究

 为解决实时直观地观察多层再现图像的问题,提出利用发光材料实现其可视化.针对由沿光轴方向的二维图层组成的空间图像,利用高效、快速的三维Gerchberg-Saxton算法,得到位相型计算全息图,并通过计算机进行了数字模拟再现.阐述了三维Gerchberg-Saxton算法的流程,并搭建了基于液晶空间光调制器的位相全息图光学再现与可视化光路.利用液晶空间光调制器的灰度-位相曲线把计算全息图转换为灰度图,加载在液晶空间光调制器上,再现出高质量的三维光场,同时利用量子点材料的荧光特性实现了图像的可视化.实验结果表明,光学再现与计算机模拟结果较吻合.该技术在医学、军事、三维显示、微加工以及显微技术等领域有重要应用价值.     

基于Kinect传感器多深度图像融合的物体三维重建

物体的三维重建技术一直是计算机视觉领域研究的热点问题,提出一种利用Kinect传感器获取的深度图像实现多幅深度图像融合完成物体三维重建的方法。在图像空间中对深度图像进行三角化,然后在尺度空间中融合所有三角化的深度图像构建分层有向距离场（hierarchical signed distance field）,对距离场中所有的体素应用整体Delaunay三角剖分算法产生一个涵盖所有体素的凸包,并利用Marching Tetrahedra算法构造等值面,完成物体表面重建。实验结果表明,该方法利用Kinect传感器采集的不同方向37幅分辨率为640480的深度图像完成目标物体的三维重建,仅需要48 s,并且得到非常精细的重建效果。     

Orthoscopic integral imaging display by use of the computational method based on lenslet model

In this paper, we propose a computational depth conversion method based on the lenslet model to display the orthoscopic 3D images in 3D integral imaging display. The proposed method permits the synthesis of elemental images for the orthoscopic 3D images at any arbitrary position without any restrictions and requires no additional procedure during the depth conversion process. Due to the lenslet model involved in the depth conversion procedure, the proposed method can broaden the flexibility of 3D image reconstruction in the integral imaging display system. We carry out the preliminary experiments to prove the feasibility of the proposed method. The experimental results reveal that the proposed method is an effective depth conversion method that allows the reconstruction of the orthoscopic 3D images at any arbitrary position.