A fast and accurate calibration method for the structured light system based on trapezoidal phase-shifting pattern

Calibration plays an important part in the structured light system. It used to be regarded as time-consuming, expensive and hard to implement. In this paper, we introduce a novel and fast method to calibrate the structured light system by using the camera to control the projector to “capture” images. Firstly, we projected just six trapezoidal grayscale pattern to establish the corresponding pixel between the camera and the projector, then we converted the camera image to its corresponding projector image. Thus the structured light system can be easy controlled by the camera calibration. Experiments showed that the present calibration method is fast, easy and accurate.     

A self-recalibration method based on scale-invariant registration for structured light measurement systems

The accuracy of structured light measurement depends on delicate offline calibration. However, in some practical applications, the system is supposed to be reconfigured so frequently to track the target that an online calibration is required. To this end, this paper proposes a rapid and autonomous self-recalibration method. For the proposed method, first, the rotation matrix and the normalized translation vector are attained from the fundamental matrix; second, the scale factor is acquired based on scale-invariant registration such that the actual translation vector is obtained. Experiments have been conducted to verify the effectiveness of our proposed method and the results indicate a high degree of accuracy.     

Simple and fast rail wear measurement method based on structured light

In this paper, a fast and accurate rail wear measurement method based on simple equipments is presented. The inner rail profile is measured by a line structured light vision sensor. Using the centers of the big and small circle from the rail waist profile as control points, the measured rail profile is registered to the reference profile. The rail wear, including the vertical and horizontal rail wear, is computed by comparing the registered measured profile with the reference profile. The method has three key contributions: (1) the rail waist light stripe center points in the images are located fast and accurately by first tracking the region containing the rail waist light stripe using the Kalman filter and then computing the sub-pixel precision image coordinates by Hessian matrix at pixels. (2) The rail waist profile is segmented automatically into arcs of big and small circles by thresholding the normal angle curve of the measured rail waist profile. The centers of the two circles are used as control points for registering the measured rail profile to the reference profile. (3) The fast location of rail wear points in the images is realized by projecting the rail wear constraint points to the image, which simplifies the problem of computing rail wear from 2d image processing to 1d searching along the line segment connecting two rail wear constraint points. Experiments show that the proposed method can achieve 500 fps measurement frequency. At a train speed of 350 km/h, the interval between two consecutive measurements is about 190 mm. The system is tested on a real running train, and the measurement results are compared with those rail wear measured manually by special gage. The RMS errors of vertical and horizontal rail wears are 0.34 and 0.30 mm, respectively.     

A high dynamic range structured light means for the 3D measurement of specular surface

This paper presents a novel structured light approach for the 3D reconstruction of specular surface. The binary shifting strip is adopted as structured light pattern instead of conventional sinusoidal pattern. Based on the framework of conventional High Dynamic Range (HDR) imaging technique, an efficient means is first introduced to estimate the camera response function. And then, dynamic range of the generated radiance map is compressed in the gradient domain by introducing an attenuation function. Subject to the change of lighting conditions caused by projecting different structured light patterns, the structure light image with middle exposure level is selected as the reference image and used for the slight adjustment of the primary fused image. Finally, the regenerated structured light images with well exposing condition are used for 3D reconstruction of the specular surface. To evaluate performance of the method, some stainless stamping parts with strong reflectivity are used for the experiments. And the results showed that, different specular targets with various shapes can be precisely reconstructed by the proposed method.     

Infrared object detection using global and local cues based on LARK

Object detection has become a challenging problem in computer vision. Locally Adaptive Regression Kernel (LARK) based detection methods are able to produce visually pleasing results without any training. We in this paper present an effective object detection method by exploring global and local cues based on LARK features. First, we encode the local context similarity by exploiting region Structural LARK (SLARK) features, which measure the likeness of a pixel to its surroundings in the query image and the test image. Second, a global constraint based on SLARK features via Heat equation is learned to detect similar features in the test image. Results from matrix cosine similarity are computed to estimate similar regions between these computed features. A compactness score is provided to refine these regions. Next, we detect the location of objects in the test image using non-maxima suppression. We show in experiments that the proposed method significantly outperforms other methods on the infrared image datasets, localizing the objects in the test images effectively.     

Robust pattern decoding in shape-coded structured light

Decoding is a challenging and complex problem in a coded structured light system. In this paper, a robust pattern decoding method is proposed for the shape-coded structured light in which the pattern is designed as grid shape with embedded geometrical shapes. In our decoding method, advancements are made at three steps. First, a multi-template feature detection algorithm is introduced to detect the feature point which is the intersection of each two orthogonal grid-lines. Second, pattern element identification is modelled as a supervised classification problem and the deep neural network technique is applied for the accurate classification of pattern elements. Before that, a training dataset is established, which contains a mass of pattern elements with various blurring and distortions. Third, an error correction mechanism based on epipolar constraint, coplanarity constraint and topological constraint is presented to reduce the false matches. In the experiments, several complex objects including human hand are chosen to test the accuracy and robustness of the proposed method. The experimental results show that our decoding method not only has high decoding accuracy, but also owns strong robustness to surface color and complex textures.     

Thermal 3D modeling system based on 3-view geometry

In this paper, we propose a novel thermal three-dimensional (3D) modeling system that includes 3D shape, visual, and thermal infrared information and solves a registration problem among these three types of information. The proposed system consists of a projector, a visual camera and, a thermal camera (PVT). To generate 3D shape information, we use a structured light technique, which consists of a visual camera and a projector. A thermal camera is added to the structured light system in order to provide thermal information. To solve the correspondence problem between the three sensors, we use three-view geometry. Finally, we obtain registered PVT data, which includes visual, thermal, and 3D shape information. Among various potential applications such as industrial measurements, biological experiments, military usage, and so on, we have adapted the proposed method to biometrics, particularly for face recognition. With the proposed method, we obtain multi-modal 3D face data that includes not only textural information but also data regarding head pose, 3D shape, and thermal information. Experimental results show that the performance of the proposed face recognition system is not limited by head pose variation which is a serious problem in face recognition.     

Robust geometric,phase and colour structured light projection system calibration

This paper introduces a new comprehensive procedure for both geometric and colour calibration of structured light system. In order to perform both geometric and colour calibration procedure, a new calibration artifact is proposed. The intrinsic and extrinsic parameters of projector and camera are estimated by using an extended pinhole camera model with a tangential and radial distortion. Camera image plane coordinates are obtained by extracting features from images of a calibration artifact. Projector image plane coordinates are calculated on the basis of continuous phase maps obtained from a fringe pattern phase reconstruction procedure. In order to stereo calibrate camera-projector system, pairs of corresponding image plane points are calculated with subpixel accuracy. In addition, one of three pattern views is used in colour calibration. RGB values of a colour field pattern detected by camera and their reference values are compared. This comparison leads to derivation of a colour transformation matrix. The performance of the proposed method is tested by measuring plane, sphere and distance reference. Also 360 degrees complex object 3D model from a set of measurements is obtained. Residual mean errors for all tests performed are calculated.     

Manipulation and behavior modeling of one-dimensional nanomaterials on a structured surface

Different diameters of multiwall carbon nanotubes (CNTs) are manipulated by a cantilever tip of an atomic force microscope (AFM) to investigate the motion properties of one-dimensional nanomaterials on a structured surface. To describe the mechanical behaviors of this kind of samples, two mechanical models based on continuum mechanics are proposed. Through foreseeable manipulation procedures, we are able to position the tubes onto pre-etched micro trenches, and then measure their Young's moduli by the three-point bending method. Both string-like and beam-like deformation forms are observed on the tested samples. Additionally, we present a reparable ‘collapse’ phenomenon of the nanotube bridges.     

Flexible and accurate implementation of a binocular structured light system

This paper presents a practical implementation of a binocular system for three-dimensional (3D) surface measurement. Manufacturing procedures are easy to implement because this system does not require that the optical axes of two cameras’ lenses be coplanar and the structured light sheets be perpendicular to that plane. The method of structured light calibration is proposed to obtain the parameters of the structured light sheets. This calibration method can tolerate the surface condition of the calibration board. To eliminate the effect caused by the distortion of projector lens, each structured light sheet is fitted to a conicoid. Meanwhile, data acquired from two cameras are transformed into a unique coordinate system to avoid 3D data matching. Experiments are conducted to validate the flexibility, accuracy, and desirability of our system.     

A novel calibration approach to structured light 3D vision inspection

Structured light 3D vision inspection is a commonly used method for various 3D surface profiling techniques. In this paper, a novel approach is proposed to generate the sufficient calibration points with high accuracy for structured light 3D vision. This approach is based on a flexible calibration target, composed of a photo-electrical aiming device and a 3D translation platform. An improved algorithm of back propagation (BP) neural network is also presented, and is successfully applied to the calibration of structured light 3D vision inspection. Finally, using the calibration points and the improved algorithm of BP neural network, the best network structure is established. The training accuracy for the best BP network structure is 0.083 mm, and its testing accuracy is 0.128 mm.     

结构光三维测量系统中投影仪标定技术研究

投影仪标定是单摄像机单投影仪结构光三维测量中的基础。采用双面标定块实现投影仪的标定。标定块正面为黑底白色圆点图案,通过Tsai两步法及非线性优化完成摄像机标定。背面为反射率均匀的白色平面,由投影仪投影双方向高亮度的直线来生成特征点。针对特征点三维坐标难以确定的问题,采用了两种方法:一是依靠摄像机标定结果;二是结合正面圆点图案,利用交比不变性获取更多的有效输入数据。标定设备简单,操作方便。实验结果证明具有较理想的标定精度。     

Application of the structured illumination method to study the topography of the sole of the foot during a walk

The structured illumination technique consists of projecting a fringe system onto a 3D object from a well defined space point, which results in a pattern that depends on the characteristic of the projected fringes, the viewpoint and the illuminated object morphology. Therefore, the structured illumination method enables to determine the topography of 3D objects. To implement this technique we set up an optoelectronic array that allows studying the sole of the foot during a walk. The method consists of projecting a Ronchi grid and capturing the images generated on the foot by a CCD camera. These obtained images are graphically processed and fringes converted into vectors. A depending algorithm on the experimental setup allows obtaining, from those vectors data, cotes for a discrete profile plotting of the studied object. The method enables the quantitative determination of the sole topography during the walk. Qualitatively, it can be used for diagnosis and control of deformation and injuries caused by accidents or illnesses. It can be introduced in the primary attention health system to study a great number of patients, due to its simplicity and low cost installation, and for being a non-invasive technique.     

Inspecting verticality of cylindrical workpieces via multi-vision sensors based on structured light

In mechanical manufacturing industry, cylindrical workpiece is one of the most commonly used type of man-made workpieces, and the verticality inspection is a very important task for guaranteeing the quality of the workpieces. In this paper, we proposed a system to inspect the verticality of cylindrical workpieces via multi-vision sensors based on structured light, which has many advantages compared with the traditional methods: fast, on-line, non-contact, flexible and remarkably more accurate. The principles and methods about how to inspect the verticality were given in details, and a real system was set up to carry out the experiments. In the system, a “sensor-unit” which consists of two stripe structured light sensors is used to address the problem of short light stripe. The experiment results indicate a high capability of the proposed system for inspecting large workpieces.     

Absolute phase recovery in structured light illumination systems: Sinusoidal vs. intensity discrete patterns

Structured light illumination is a well-established technology for noncontact 3D surface measurements. A common challenge in those systems is to obtain the absolute surface information using few measurement frames. This work discusses techniques based on the projection of multiple sinusoidal fringe patterns with different fringe period, as well as the projection of intensity discrete Gray Code and grey-level coded patterns. The use of sinusoidal multi-frequency techniques has been since years an on-going area of research, where various algorithms have been developed based on beats, look-up tables, or number-theoretical approaches. This work shows that a related technique, the so-called algebraic reconstruction technique that is borrowed from the area of multi-wavelength interferometry can be used for this purpose. This approach provides a robust analytical solution to the phase-unwrapping problem. However, this work argues that despite these advances, the acquisition of additional phase maps obtained with different fringe periods requires too many measurement frames, and hence is inefficient. Motivated by that, this work proposes a new grey level coding scheme that uses only few measurement frames, overcomes typical defocus errors, and has an error detecting feature. The latter feature makes the need of separate error detecting algorithms obsolete. This so-called closed-loop space filling curve can be implemented with an arbitrary number of N grey-levels enabling to code up to (2N) code-words. The performance of this so-called closed-loop space filling curve is demonstrated using experimental data.     

编码光系统的颜色耦合与颜色失衡校正

为了减小编码光系统中的颜色耦合与颜色失衡的干扰,提高形状与颜色重建的准确度,建立了彩色编码光系统,并对其所采用的编解码方法、颜色重建的两个相关环节及颜色校正方法进行了研究。首先,给出了系统的彩色梯形相移强度比编解码法,分析了编解码过程中存在的RGB基色相互耦合现象,设计了应用Caspi模型硬件标定的颜色耦合校正方案;然后,给出了系统的逐点颜色重建法,分析了颜色重建过程中由表面曲率导致的颜色失衡现象,设计了利用表面几何信息校正颜色失衡的方案。实验结果表明：校正后单一绿色图像中的红色分量值约为校正前的1/10;单色表面色差约为0.1,趋近于人眼颜色分辨率,远低于校正前的0.4;重建的单色复杂被测表面颜色均匀、视觉效果良好,符合实际情况。提出的方法符合编码光系统抗干扰能力强的要求,有助于提高形状与颜色重建的准确度。     

Error correcting coding-theory for structured light illumination systems

Intensity discrete structured light illumination systems project a series of projection patterns for the estimation of the absolute fringe order using only the temporal grey-level sequence at each pixel. This work proposes the use of error-correcting codes for pixel-wise correction of measurement errors. The use of an error correcting code is advantageous in many ways: it allows reducing the effect of random intensity noise, it corrects outliners near the border of the fringe commonly present when using intensity discrete patterns, and it provides a robustness in case of severe measurement errors (even for burst errors where whole frames are lost). The latter aspect is particular interesting in environments with varying ambient light as well as in critical safety applications as e.g. monitoring of deformations of components in nuclear power plants, where a high reliability is ensured even in case of short measurement disruptions. A special form of burst errors is the so-called salt and pepper noise, which can largely be removed with error correcting codes using only the information of a given pixel. The performance of this technique is evaluated using both simulations and experiments.     

基于平面互补靶标的线结构光标定系统

为提升线结构光传感器的标定效率与精度,设计了一种集成自背光可调节位姿的平面棋盘格-同心圆互补线结构光标定系统。该系统基于同心圆圆心的真实投影位置与投影椭圆圆心位置的几何关系,建立非线性优化偏心误差补偿模型,精确得到透射投影下圆心偏心误差补偿位置。该方法与传统标定方法对比降低重投影误差84.7%,有效解决了圆形标志物偏心误差补偿的高精度标定难题。通过将相机坐标系下过光心、光条中心线的平面与靶标平面结合,多次获取空间交线的坐标信息增加特征点,并使用最小二乘法拟合光平面方程,解决了因特征点少从而平面拟合标定精度较低的问题。在复杂环境下重复实验测得大尺寸砂轮外径误差均值为0.005 1 mm,结果表明该标定系统具有一定的准确性和简便实用性。     

基于相移的彩色结构光编码三维扫描技术

在物体三维轮廓扫描中,被动式双目立体视觉方法对表面特征不明显的物体难以获取足够的三维信息,针对这一问题,基于双目立体视觉原理,结合主动视觉中结构光投射方法,提出了基于相移的彩色结构光编码方法。该方法将四种颜色按照全排列的相关性质编码产生彩色光栅投射在被测物体表面,依据光栅相位移动增大光栅密度,结合极线约束实现立体匹配,最终获取物体三维数据。结果表明使用该三维扫描技术得到的三维曲面符合原物特征,特别适用于物体三维轮廓的致密测量。     

Three-dimensional image correlator using fast computational integral imaging reconstruction method based on pixel-to-pixel mapping

In this paper, a three-dimensional (3D) image correlator using a fast computational integral imaging reconstruction (CIIR) method based on a pixel-to-pixel mapping is proposed. In order to implement the fast CIIR method, we replace the magnification process in the conventional CIIR by a pixel-to-pixel mapping. The proposed fast CIIR method reconstructs two sorts of plane images; a plane image whose quality is sufficient, and a dot pattern plane image insufficient to view. This property is very useful to enhance the performance of a CIIR-based image correlator. Thus, we apply the fast CIIR method to a CIIR-based image correlator. To show the feasibility of the proposed method, some preliminary experiments on both pattern correlation and computational cost are carried out, and the results are presented. Our experimental results indicate that the proposed image correlator is superior to the previous method in terms of both correlation performance and complexity.