Recent advances in 3D SEM surface reconstruction

The scanning electron microscope (SEM), as one of the most commonly used instruments in biology and material sciences, employs electrons instead of light to determine the surface properties of specimens. However, the SEM micrographs still remain 2D images. To effectively measure and visualize the surface attributes, we need to restore the 3D shape model from the SEM images. 3D surface reconstruction is a longstanding topic in microscopy vision as it offers quantitative and visual information for a variety of applications consisting medicine, pharmacology, chemistry, and mechanics. In this paper, we attempt to explain the expanding body of the work in this area, including a discussion of recent techniques and algorithms. With the present work, we also enhance the reliability, accuracy, and speed of 3D SEM surface reconstruction by designing and developing an optimized multi-view framework. We then consider several real-world experiments as well as synthetic data to examine the qualitative and quantitative attributes of our proposed framework. Furthermore, we present a taxonomy of 3D SEM surface reconstruction approaches and address several challenging issues as part of our future work.     

Deep sub micrometer imaging of defects in copper pillars by X-ray tomography in a SEM

The potential of X-ray nanotomography hosted in a SEM in presented in this paper. In order to improve the detail detectability of this system, which is directly related to the X-ray source size, thin metal layers have been studied and installed in the equipment. A 3D resolution pattern has been created in order to determine the smallest detectable features by this setup. This sample is a 25 μm diameter copper pillar in which size-controlled holes have been milled using a plasma-focused ion beam. This pattern has then been scanned and the resulting 3D reconstruction demonstrates that the instrument is able to detect 500 nm diameter voids in a copper interconnection, as used in 3D integration.     

Data-driven curvilinear reconstructions of 3D MR images: application to cryptogenic extratemporal epilepsy

This paper presents a data-driven method for the reconstruction and visualisation of curvilinear slices from three-dimensional (3D) magnetic resonance (MR) scans of the head. Visualisation of curvilinear slices, rather than standard planar slices, produces symmetrical views of the cortex and allows small abnormalities to be detected by comparing the two hemispheres of the brain. In our method, the surface defined by the upper half of the brain is used as a reference shape for curvilinear reconstructions. The brain is first segmented from the 3D scan using a 3D region growing method associated to an unsupervised threshold selection technique. The upper half of the segmented brain is then extracted and fitted by a deformable surface model. This surface is finally interactively moved by the operator in the 3D scan, to visualise the desired curvilinear slice, which is projected on the screen as a two-dimensional image. We show an application of this visualisation technique to the localisation of cerebral epileptogenic lesions. The procedure has proven efficient and handy in clinical use.     

3D reconstruction of road surfaces using an integrated multi-sensory approach

In this paper, we present our experience in building a mobile imaging system that incorporates multi-modality sensors for road surface mapping and inspection applications. Our proposed system leverages 3D laser-range sensors, video cameras, global positioning systems (GPS) and inertial measurement units (IMU) towards the generation of photo-realistic, geometrically accurate, geo-referenced 3D models of road surfaces. Based on our summary of the state-of-the-art systems for a road distress survey, we identify several challenges in the real-time deployment, integration and visualization of the multi-sensor data. Then, we present our data acquisition and processing algorithms as a novel two-stage automation procedure that can meet the accuracy requirements with real-time performance. We provide algorithms for 3D surface reconstruction to process the raw data and deliver detail preserving 3D models that possess accurate depth information for characterization and visualization of cracks as a significant improvement over contemporary commercial video-based vision systems.     

宽场光学相干断层成像系统的三维显微成像

宽场相干断层成像技术（WFOCT）具有提高OCT系统的扫描速率和实现高分辨率的三维显微技术等优点,成为当前研究的热点。本文阐述了WFOCT的基本原理,利用八步移相法重建出玻璃物体微细结构的断层图像,研究了宽场OCT系统对玻璃材料的纵向分辨率和探测深度,其中探测深度可达3.3 mm。在获得多幅断层图像的基础上,利用VC6.0和OpenGL混合编程,采用移动立方体（MC）算法重建出玻璃物体微细结构的三维图像。实验结果表明,WFOCT系统不但能够在生物组织检测等医学方面得到应用,而且对反射率较高的物体能够完成三维形貌显微成像探测和深度探测。     

Reconstruction of 3D ultrasound images based on Cyclic Regularized Savitzky-Golay filters

This paper presents a new three-dimensional (3D) ultrasound reconstruction algorithm for generation of 3D images from a series of two-dimensional (2D) B-scans acquired in the mechanical linear scanning framework. Unlike most existing 3D ultrasound reconstruction algorithms, which have been developed and evaluated in the freehand scanning framework, the new algorithm has been designed to capitalize the regularity pattern of the mechanical linear scanning, where all the B-scan slices are precisely parallel and evenly spaced. The new reconstruction algorithm, referred to as the Cyclic Regularized Savitzky-Golay (CRSG) filter, is a new variant of the Savitzky-Golay (SG) smoothing filter. The CRSG filter has been improved upon the original SG filter in two respects: First, the cyclic indicator function has been incorporated into the least square cost function to enable the CRSG filter to approximate nonuniformly spaced data of the unobserved image intensities contained in unfilled voxels and reduce speckle noise of the observed image intensities contained in filled voxels. Second, the regularization function has been augmented to the least squares cost function as a mechanism to balance between the degree of speckle reduction and the degree of detail preservation. The CRSG filter has been evaluated and compared with the Voxel Nearest-Neighbor (VNN) interpolation post-processed by the Adaptive Speckle Reduction (ASR) filter, the VNN interpolation post-processed by the Adaptive Weighted Median (AWM) filter, the Distance-Weighted (DW) interpolation, and the Adaptive Distance-Weighted (ADW) interpolation, on reconstructing a synthetic 3D spherical image and a clinical 3D carotid artery bifurcation in the mechanical linear scanning framework. This preliminary evaluation indicates that the CRSG filter is more effective in both speckle reduction and geometric reconstruction of 3D ultrasound images than the other methods.     

3D scanning electron microscopy applied to surface characterization of fluorosed dental enamel

The enamel surfaces of fluorotic teeth were studied by scanning electron stereomicroscopy. Different whitening treatments were applied to 25 pieces to remove stains caused by fluorosis and their surfaces were characterized by stereomicroscopy in order to obtain functional and amplitude parameters. The topographic features resulting for each treatment were determined through these parameters. The results obtained show that the 3D reconstruction achieved from the SEM stereo pairs is a valuable potential alternative for the surface characterization of this kind of samples.     

双探头三维表面轮廓测量系统

提出了一种将原子力显微(AFM)探头与位置敏感探测器测距探头相结合的双探头三维表面轮廓测量新方法, 可在获取样品表面轮廓的同时, 测定样品局部形貌。搭建了双探头三维表面轮廓测量系统, 阐述了系统的工作原理, 并对其结构组成包括双探头、步进扫描台和计算机控制平台进行了说明。用2000 line/mm的光栅进行了扫描实验, 对系统的测量范围进行了标定。以外径8 mm、内径4 mm的金属垫圈为样品, 进行了整体三维表面轮廓与局部表面形貌测量实验, 给出了垫圈表面图和局部三维形貌图。结果表明, 该系统能满足不同尺寸和材质的样品的测量要求, 即可实现对样品轮廓的大范围扫描测量, 又可对样品局部进行高精度形貌测量。     

Volumetric q-space imaging by 3D diffusion-weighted MRI

High b-value diffusion magnetic resonance imaging (MRI) enables us to detect far smaller architectures, by using q-space analysis, than the resolution in conventional MRI. Average displacement, one of the q-space parameters, quantitatively reflects architecture size and is very useful in observing small changes in microstructures in vivo (e.g., neurodegeneration, tumor heterogeneity, and others). Diffusion-weighted imaging (DWI) is performed by a two-dimensional (2D) multislice method; however, due to finite slice thickness and slice gap, there is a partial-volume effect that makes it difficult to detect the net q-space signal. On the other hand, three-dimensional (3D) MRI, having the advantages of very thin slice thickness and no slice gap (contiguous slices), allows volumetric evaluation acquired in a small isotropic voxel, as compared to 2D multislice imaging. Little is known about the isotropic high-resolution 3D DWI application to q-space analysis. In this study, we have developed and implemented a high b-value 3D DWI sequence, applied q-space analysis to study the reliability of high b-value 3D DWI and obtained a microscopic analytical map with isotropic high resolution and less contamination.     

A high dynamic range structured light means for the 3D measurement of specular surface

This paper presents a novel structured light approach for the 3D reconstruction of specular surface. The binary shifting strip is adopted as structured light pattern instead of conventional sinusoidal pattern. Based on the framework of conventional High Dynamic Range (HDR) imaging technique, an efficient means is first introduced to estimate the camera response function. And then, dynamic range of the generated radiance map is compressed in the gradient domain by introducing an attenuation function. Subject to the change of lighting conditions caused by projecting different structured light patterns, the structure light image with middle exposure level is selected as the reference image and used for the slight adjustment of the primary fused image. Finally, the regenerated structured light images with well exposing condition are used for 3D reconstruction of the specular surface. To evaluate performance of the method, some stainless stamping parts with strong reflectivity are used for the experiments. And the results showed that, different specular targets with various shapes can be precisely reconstructed by the proposed method.     

光学三维成像实验系统

建立光学投影层析三维成像系统,该系统包括光学成像、图像采集、断层重建及三维显示,重建算法为滤波反投影算法.实验结果表明:利用该系统得到的重建图像与样品的形状吻合,重建结果边缘清晰,伪迹较小.     

3D ultrasound reconstruction algorithms from analog and digital data

Freehand 3D ultrasound is increasingly being introduced in the clinic for diagnostics and image-assisted interventions. Various algorithms exist for combining 2D images of regular ultrasound probes to 3D volumes, being either voxel-, pixel- or function-based. Previously, the most commonly used input to 3D ultrasound reconstruction has been digitized analog video. However, recent scanners that offer access to digital image frames exist, either as processed or unprocessed data. To our knowledge, no comparison has been performed to determine which data source gives the best reconstruction quality. In the present study we compared both reconstruction algorithms and data sources using novel comparison methods for detecting potential differences in image quality of the reconstructed volumes. The ultrasound scanner used in this study was the Sonix RP from Ultrasonix Medical Corp (Richmond, Canada), a scanner that allow third party access to unprocessed and processed digital data. The ultrasound probe used was the L14-5/38 linear probe. The assessment is based on a number of image criteria: detectability of wire targets, spatial resolution, detectability of small barely visible structures, subjective tissue image quality, and volume geometry. In addition we have also performed the more “traditional” comparison of reconstructed volumes by removing a percentage of the input data. By using these evaluation methods and data from the specific scanner, the results showed that the processed video performed better than the digital scan-line data, digital video being better than analog video. Furthermore, the results showed that the choice of video source was more important than the choice of tested reconstruction algorithms.     

Whole 3D shape reconstruction of vascular segments under pressure via fringe projection techniques

Understanding and modelling vascular wall mechanics is a primary issue in the study of circulatory diseases. Although theoretical and numerical studies on arteries compliance are continuously increasing, relatively little work has been documented on the use of non-invasive imaging techniques for monitoring 3D vascular wall deformations. Usually, 2D video dimension analyzer (VDA) systems recover diameter and length variations during inflation/extension tests by tracking position changes of few markers put on the blood vessel surface. Then, strain determination relies on the assumption of axisymmetric deformations. However, more rigorous evaluations of whole wall deformation map are required for properly modelling the highly anisotropic and inhomogeneous vascular tissue mechanical response.

This paper describes the development and application of a fringe projection (FP)-based procedure for the 360° 3D shape reconstruction of tubular samples subjected to internal pressure. A specially designed fixture for mounting and inflating the tubular segment allows specimen rotation about its axis. Movement is controlled by a high-precision rotational stage. This yields accurate positioning of the surface to be investigated with respect to the viewing direction. Data point clouds obtained from multiple recorded images are then processed and merged in a CAD environment, thus providing the whole shape of the sample with very high spatial resolution.

The entire procedure has successfully been applied to latex specimens and porcine vascular segments. Further improvements will make the present procedure suitable for in vitro tests under more closely reproduced physiological conditions.     

High-resolution cardiac imaging using an interleaved 3D double slab technique

A three-dimensional (3D) gradient-echo sequence with interleaved double-slab excitation was developed and optimized for the requirements in pediatric cardiac imaging. For this purpose high contrast between blood and myocardium signal should be obtained without the use of contrast agents. An acceptable measuring time for a large region examined with high spatial resolution should be achieved as well, especially with regard to the small structures of the heart and vessels of infants. The presented approach works with gradient moment nulling and a short echo time of 5.5 ms resulting in generally high signal intensity and only minor signal losses due to turbulent flow. The sequence allows simultaneous ECG-gated recording of two separately excited slabs with small thickness (10 mm) and with a distance of several centimeters between them. Thus, common effects of presaturation in 3D imaging can be avoided, although a relatively short measuring time is achievable. In order to get a 3D data set with good signal homogeneity of blood and of the other structures across a large volume of interest several double-slab measurements with suitable positions must be performed. The latter aspect is especially important for postprocessing techniques as multiple planar reconstruction and maximum intensity projection. Examples of applications of the new technique and appropriately postprocessed images are presented allowing demonstration even of subtle cardiac malformations.     

基于物体特征有效提取和离散点三维重建的3D扫描系统原型研究

针对3D扫描技术的近期发展情况,以及市场关于被摄物体特征有效提取的迫切需求,通过系统的对照相机和投影仪测量的方法与计算,利用离散点三维重建算法、标准化算法详细分析了,作为3D扫描原型系统的关键技术,并深入的对三维坐标重复演算的过程,进行了实践研究与计算,论证了基于投影光编码技术的3D扫描仪原型系统,在物体特征有效提取和离散点三维重建中完全可以高效应用;同时,使用焦距标准化运算、逆向倾角变形运算、逆向扭曲变形运算三种方法,综合论述了物体特征有效提取和离散点三维重建过程的相关内容,测试了原型系统对真实文物的扫描情况,展开了该原型系统可应用于不同材料,不同种类物体的扫描,并能有效的进行数字化物体特征识别的研究结论。     

Fast 3D spatial EPR imaging using spiral magnetic field gradient

Electron paramagnetic resonance imaging (EPRI) provides direct detection and mapping of free radicals. The continuous wave (CW) EPRI technique, in particular, has been widely used in a variety of applications in the fields of biology and medicine due to its high sensitivity and applicability to a wide range of free radicals and paramagnetic species. However, the technique requires long image acquisition periods, and this limits its use for many in vivo applications where relatively rapid changes occur in the magnitude and distribution of spins. Therefore, there has been a great need to develop fast EPRI techniques. We report the development of a fast 3D CW EPRI technique using spiral magnetic field gradient. By spiraling the magnetic field gradient and stepping the main magnetic field, this approach acquires a 3D image in one sweep of the main magnetic field, enabling significant reduction of the imaging time. A direct one-stage 3D image reconstruction algorithm, modified for reconstruction of the EPR images from the projections acquired with the spiral magnetic field gradient, was used. We demonstrated using a home-built L-band EPR system that the spiral magnetic field gradient technique enabled a 4-7-fold accelerated acquisition of projections. This technique has great potential for in vivo studies of free radicals and their metabolism.     

3D shape reconstruction of teeth by shadow speckle correlation method

This paper presents a new method for 3D shape reconstruction in computer-aided dental prosthetics. A specklegram is projected onto the tooth to be measured. This shadow speckle image is recorded and then processed by a digital image correlation method, which enables the computation of 2D shapes based on the similar principle of shadow moiré method. By repeating the procedure for all the sides, i.e., one crown and several side surfaces, local 2D shapes can be measured precisely. Afterwards, these local 2D profiles are merged to form a 3D model, during which certain constraints such as the widths along perpendicular directions are introduced to guide the process. As the height information within an entire image field is recorded instantly, it has the potential to be employed in an intra-oral environment, which would make the patient feel more comfortable during the restoration process. In vitro experiments were carried out on gypsum teeth models and the results proved the effectiveness of the proposed method.     

复杂轮廓表面激光检测及三维重构技术的研究

基于零件实物样件的几何模型反求技术已成为CAD/CAM领域中的研究热点之一。以航空航天用MJ螺纹为对象 ,对逆向工程中的两大关键技术———表面数字化及三维重构技术进行了分析和研究。提出了用一种新型激光光纤传感器实现复杂轮廓表面的非接触数字化检测方法 ,描述了测量原理。通过二维截面轮廓图重构三维面型 ,初步的实验结果验证了此方法的有效性和实用性     

Macro and microsurface morphology reconstructions during laser-induced etching of silicon

Surface morphologies of the laser-etched silicon were studied as a function of the laser power densities. Scanning electron microscope (SEM) results show that different kind of microstructures develop. Pores like structures are formed at low laser power density and pillar like structures are obtained at higher laser power density. It is the etching rate, which is responsible for the surface morphology reconstructions. Etching rate was found to be a function of the laser power density. Atomic force microscope (AFM) results reveal that macro and microsurface morphology reconstructions take place simultaneously as a result of increasing etching rate. Macrosurface morphology reconstruction takes place on the silicon wafer surface and the microsurface morphology reconstruction takes place inside the pore wall.     

基于IR-SFS算法空间目标红外影像3D重建

在利用单幅影像的明暗恢复形状（Shape From Shading,SFS）三维重构算法的基础上,提出同时考虑外界辐射源和目标自身红外辐射的IR-SFS（infrared-SFS）算法。首先通过分析SFS算法原理和空间目标的红外成像特性,建立IR-SFS辐射方程并进行了仿真研究,然后利用温度场估计获得红外差值图,在人工合成的理想半球体、半圆柱体卫星红外影像上进行算法测试,并以美国STS107真实红外影像作为实验目标进行三维重建。实验结果表明,所提出IR-SFS算法经过参数优化后,与原SFS算法相比,重建模型的峰值信噪比更高,对STS107顶部舱门、尾翼、机舱、机舱内方形部件具有更佳显示度,整体效果得到明显改善。