Adaptive multiscale regression for reliable Raman quantitative analysis

This paper presents a novel methodology, adaptive multiscale regression (AMR), to adaptively process Raman spectra for quantitative analysis. The proposed methodology aims to construct an optimal calibration model for a Raman spectrum at hand, regardless of its structural characteristics, thus facilitating the application of Raman spectroscopy as a general tool for analytical chemistry. AMR firstly splits the spectra in a calibration set into frequency components at different scales using adaptive wavelet transform (AWT). Parallel member models constructed at different scales are then fused into a final prediction. The contributions of member models to a fusion model are straightforwardly estimated by a partial least square (PLS) model that emerges from a cross-validation results matrix (X) and reference values (Y). This procedure avoids information leakage by fully utilizing the multiscale nature of the input Raman spectra instead of arbitrarily removing some part of the spectral information by calibrating to selected features. Theoretically, we establish that AMR represents an automatic data-driven strategy that captures the Raman spectral structures adaptively and accurately. Our work tests and refines the AMR method by drawing upon the systematic analysis of spectra formulated to yield challenges representative of those encountered in common Raman analyses. AMR compares favorably with other popular preprocessing methods. Satisfactory calibration results suggest that AMR has the capacity to improve robustness and reliability of Raman spectral analysis, and may well extend to other spectroscopic techniques.     

Data fusion in the wavelet domain by means of fuzzy aggregation connectives

Dual-domain classification analysis is proposed to identify pigments used in works of art studied by Raman spectroscopy and X-ray fluorescence spectrometry. By means of this methodology, Raman and X-ray fluorescence data are jointly processed by a high-level fusion approach. The system proposed aims to avoid the pre-processing stage and directly process raw data obtained from the instrument. The system is tested with spectra contaminated with background components of different shapes and intensities and with those with the background removed by line segment correction. The benefits of the approach were well demonstrated in a study of an ochre pigment classification.The approach is based on the main advantage of wavelet transform, which is multiresolution. Each spectrum is split into blocks, according to a specific frequency, to form a wavelet prism. Partial least squares-discriminant analysis (PLS-DA) is then applied to those blocks which contain the deterministic part of the signal and are not influenced by noise and background signal components. At the end, to obtain the final classification assignment, high-level data fusion of the classifications results (decision levels) obtained from PLS-DA analysis is done by means of fuzzy aggregation connective operators. Our study showed that fuzzy aggregation may be suitable for performing high-level data fusion on dual-domain data. This method can be automated so that classification can be rapid. It can handle classifications with different levels of difficulty and requires no prior knowledge of sample composition.     

基于多光谱特征融合技术的面粉掺杂定量分析方法

提出了一种基于拉曼光谱技术(Raman)和激光诱导击穿光谱技术(LIBS)的多光谱特征融合技术(MFFT),利用拉曼光谱中分子组分信息和激光诱导击穿光谱中原子组分信息之间的互补特性,采用自适应小波变换(AWT)-竞争性自适应加权(CARS)-偏最小二乘回归(PLS)建模技术,获取了面粉体系更为全面的特征信息。在多光谱特征融合技术中,首先采用AWT-CARS方法分别提取拉曼光谱和激光诱导击穿光谱中的特征变量,然后将两者的特征变量融合为一个向量,采用PLS方法构建MFFT模型,实现了面粉掺杂物的定量分析。通过对二氧化钛、硫酸铝钾等面粉掺杂体系建模分析,考察MFFT模型的有效性。结果表明,与单一拉曼光谱技术或激光诱导击穿光谱技术建立的预测模型相比,MFFT模型显著提升了模型的预测性能,二氧化钛和硫酸铝钾预测模型的线性相关系数分别从相对较差的Raman模型的0.884、0.877提升到0.981、0.980,其预测均方根误差分别从相对较差的Raman模型的0.151、0.154降低到0.069、0.068。表明多光谱特征融合技术可以准确提取Raman光谱中的分子信息和LIBS光谱中的元素信息,使其互为补充、互为校正,进而有效克服面粉基质对掺杂组分定量分析的干扰,显著提高模型的预测精度。     

A consensus least squares support vector regression (LS-SVR) for analysis of near-infrared spectra of plant samples

Consensus modeling of combining the results of multiple independent models to produce a single prediction avoids the instability of single model. Based on the principle of consensus modeling, a consensus least squares support vector regression (LS-SVR) method for calibrating the near-infrared (NIR) spectra was proposed. In the proposed approach, NIR spectra of plant samples were firstly preprocessed using discrete wavelet transform (DWT) for filtering the spectral background and noise, then, consensus LS-SVR technique was used for building the calibration model. With an optimization of the parameters involved in the modeling, a satisfied model was achieved for predicting the content of reducing sugar in plant samples. The predicted results show that consensus LS-SVR model is more robust and reliable than the conventional partial least squares (PLS) and LS-SVR methods.     

The Application of Complex Wavelet Transform to Spectral Signals Background Deduction

The accuracy of spectrograms may be affected by baseline excursion or drift when infrared spectrometers are used in the analyses of gases. Background deduction or baseline correction is one of the effective pretreatment methods that can improve measurement accuracy. This paper presents a novel methodology based on complex wavelet transform algorithm to perform background deduction. The complex wavelet transform methodology establishes a complex wavelet filter to decompose the spectral signals first, and set the decomposition coefficients in the high-frequency section to zero, and then reconstruct the background signals; finally, the background deduction can be realized by deducting the background signals. In this study, the complex wavelet established by Daubechies was selected to demonstrate background deduction aiming at simulative spectral signals with different backgrounds and the real spectral signal of SF6 decomposition gases. Compared with the results done by the real wavelet transform in the same conditions, the results indicate that complex wavelet transform methodology can perform background deduction more efficiently than real wavelet transform methodology, thus improving the effectiveness and precision of spectrogram measurements greatly, which is useful for SF6 gas decomposition compositions analysis     

Classification of normal and malignant human gastric mucosa tissue with confocal Raman microspectroscopy and wavelet analysis

Thirty-two samples from the human gastric mucosa tissue, including 13 normal and 19 malignant tissue samples were measured by confocal Raman microspectroscopy. The low signal-to-background ratio spectra from human gastric mucosa tissues were obtained by this technique without any sample preparation. Raman spectral interferences include a broad featureless sloping background due to fluorescence and noise. They mask most Raman spectral feature and lead to problems with precision and quantitation of the original spectral information. A preprocessed algorithm based on wavelet analysis was used to reduce noise and eliminate background/baseline of Raman spectra. Comparing preprocessed spectra of malignant gastric mucosa tissues with those of counterpart normal ones, there were obvious spectral changes, including intensity increase at approximately 1156cm(-1) and intensity decrease at approximately 1587cm(-1). The quantitative criterion based upon the intensity ratio of the approximately 1156 and approximately 1587cm(-1) was extracted for classification of the normal and malignant gastric mucosa tissue samples. This could result in a new diagnostic method, which would assist the early diagnosis of gastric cancer.     

An adaptive strategy for selecting representative calibration samples in the continuous wavelet domain for near-infrared spectral analysis

Sample selection is often used to improve the cost-effectiveness of near-infrared (NIR) spectral analysis. When raw NIR spectra are used, however, it is not easy to select appropriate samples, because of background interference and noise. In this paper, a novel adaptive strategy based on selection of representative NIR spectra in the continuous wavelet transform (CWT) domain is described. After pretreatment with the CWT, an extension of the Kennard–Stone (EKS) algorithm was used to adaptively select the most representative NIR spectra, which were then submitted to expensive chemical measurement and multivariate calibration. With the samples selected, a PLS model was finally built for prediction. It is of great interest to find that selection of representative samples in the CWT domain, rather than raw spectra, not only effectively eliminates background interference and noise but also further reduces the number of samples required for a good calibration, resulting in a high-quality regression model that is similar to the model obtained by use of all the samples. The results indicate that the proposed method can effectively enhance the cost-effectiveness of NIR spectral analysis. The strategy proposed here can also be applied to different analytical data for multivariate calibration.     

基于小波系数的近红外光谱局部建模方法与应用研究

局部建模方法使用与预测样本相似的样本建立模型,可解决光谱响应与浓度之间的非线性问题,扩大模型的适用范围,提高预测准确度。采用小波变换进行数据压缩并利用小波系数之间的欧氏距离作为光谱相似性的判据,实现了近红外光谱定量分析的局部建模方法,避免了样本之间的依赖性。将所建立的方法用于烟草样品中氯含量的测定,100次重复计算得到的预测集均方根误差(RMSEP)平均值为0.0665,标准偏差(σ)为0.0045,优于全局建模和基于主成分的局部建模方法。     

A new hybrid strategy for constructing a robust calibration model for near-infrared spectral analysis

A new hybrid algorithm is proposed for construction of a high-quality calibration model for near-infrared (NIR) spectra that is robust against both spectral interference (including background and noise) and multiple outliers. The algorithm is a combination of continuous wavelet transform (CWT) and a modified iterative reweighted PLS (mIRPLS) procedure. In the proposed algorithm the spectral interference is filtered by CWT at the first stage then mIRPLS is proposed to detect the multiple outliers in the CWT domain. Compared with the original IRPLS method, mIRPLS does not need to adjust variable parameters to achieve optimum calibration results, which makes it very convenient to perform in practice. The final PLS model is constructed robustly because both the spectral interference and multiple outliers are eliminated. In order to validate the effectiveness and universality of the algorithm, it was applied to two different sets of NIR spectra. The results indicate that the proposed strategy can greatly enhance the robustness and predictive ability of NIR spectral analysis.     

Elimination of interference information by a new hybrid algorithm for quantitative calibration of near infrared spectra

A new hybrid algorithm is proposed to eliminate the interference information for multivariate calibration of near-infrared (NIR) spectra that includes noise, background and systemic spectral variation irrelevant to concentration. The method consists of two parts: approximate derivative based on continuous wavelet transform (CWT) and orthogonal signal correction (OSC). After the approximate derivative calculated by CWT, OSC was performed. It was successfully applied to real complex NIR spectral data to eliminate the interference information. Correction for the interference of NIR spectra resulted in a substantial improvement in the predicted precision, and a more concise calibration model was obtained. The proposed procedure also compared favourably with several pretreatment methods, and the new method appears to provide a high-performance pretreatment tool for multivariate calibration of NIR spectra. In addition, the strategy proposed here can be applied to various other spectral data for quantitative purposes as well.     

The Application of Complex Wavelet Transform to Spectral Signals Background Deduction

The accuracy of spectrograms may be affected by baseline excursion or drift when infrared spectrometers are used in the analyses of gases. Background deduction or baseline correction is one of the effective pretreatment methods that can improve measurement accuracy. This paper presents a novel methodology based on complex wavelet transform algorithm to perform background deduction. The complex wavelet transform methodology establishes a complex wavelet filter to decompose the spectral signals first, and set the decomposition coefficients in the high-frequency section to zero, and then reconstruct the background signals; finally, the background deduction can be realized by deducting the background signals. In this study, the complex wavelet established by Daubechies was selected to demonstrate background deduction aiming at simulative spectral signals with different backgrounds and the real spectral signal of SF6 decomposition gases. Compared with the results done by the real wavelet transform in the same conditions, the results indicate that complex wavelet transform methodology can perform background deduction more efficiently than real wavelet transform methodology, thus improving the effectiveness and precision of spectrogram measurements greatly, which is useful for SF6 gas decomposition compositions analysis

     

基于迭代小波变换的光谱信号本底扣除方法研究

本底会对光谱分析结果产生很大的干扰作用,为获取特征峰的有效信息,必须首先去除本底。该文提出了一种基于小波变换的本底扣除算法,通过对光谱及后续光谱迭代进行小波变换,利用逼近系数估计本底,直到本底收敛。提出了判断多次估计的本底最大误差是否足够小的收敛准则。利用该算法去除本底后,即可进行特征峰信息的提取。分别利用仿真光谱和实验能量色散X射线荧光光谱对算法进行了验证,并与传统小波变换和多项式拟合法进行了对比。结果表明,该算法能够更准确扣除光谱本底,对其他光谱的本底扣除也具有借鉴意义。     

A Background and noise elimination method for quantitative calibration of near infrared spectra

A new hybrid algorithm is proposed to eliminate the varying background and noise simultaneously for multivariate calibration of near infrared (NIR) spectral signals. The method is based on the use of multi-resolution, which is one of the main advantages provided by wavelet transform. The signals are firstly split into different frequency components, which keep the same data points of the original signals. In conjunction with a modified uninformative variable elimination (mUVE) criterion, the new method can be used to remove the low-frequency varying background and the high-frequency noise simultaneously. The method is successfully applied to simulated spectral data set and experimental NIR spectral data, resulting in more parsimonious multivariate models with higher precision. In addition, the proposed strategy can be applied to other spectral signals as well.     

基于非接触式拉曼光谱分析人血与犬血的PCA-LDA鉴别方法

将拉曼光谱分析法与数理统计方法有机结合,构建人血与犬血种属判别模型,实现了不同种属血液样本的高效无损鉴别.采用拉曼光谱的无损测试模式对血液样本进行测试,考察了抗凝管管材、聚焦位置及曝光时间等对血液样本拉曼光谱的影响,在激发波长为632.8 nm,光谱扫描范围为200~1800 cm-1,功率衰减率50%,曝光时间5 s及累加次数为2次的优化条件下,获得了无损检测条件下的血液样本拉曼光谱图.针对血液样本组分复杂、拉曼光谱信号基底背景高等问题,提出了基于小波变换去噪,进行分段多项式基线校正的预处理方法,有效解决了血液样本拉曼光谱谱图的高噪音和基线漂移问题.实验选择30例正常人血和33例比格犬血为样本训练集,5例正常人血和5例比格犬血为测试集,基于主成分分析法(PCA)联合线性判别法(LDA)模型,训练集分类正确率达到95.23%,盲测集分类正确率达90.00%.这种基于非接触式血液样本拉曼光谱和PCA-LDA判断模型的测试方法在进出口检验检疫等涉及血液无损鉴别的领域具有广泛的应用价值和前景.     

A partial least squares and wavelet-transform hybrid model to analyze carbon content in coal using laser-induced breakdown spectroscopy

A partial least squares (PLS) and wavelet transform hybrid model are proposed to analyze the carbon content of coal by using laser-induced breakdown spectroscopy (LIBS). The hybrid model is composed of two steps of wavelet analysis procedures, which include environmental denoising and background noise reduction, to pretreat the LIBS spectrum. The processed wavelet coefficients, which contain the discrete line information of the spectra, were taken as inputs for the PLS model for calibration and prediction of carbon element. A higher signal-to-noise ratio of carbon line was obtained after environmental denoising, and the best decomposition level was determined after background noise reduction. The hybrid model resulted in a significant improvement over the conventional PLS method under different ambient environments, which include air, argon, and helium. The average relative error of carbon decreased from 2.74 to 1.67% under an ambient helium environment, which indicated a significantly improved accuracy in the measurement of carbon in coal. The best results obtained under an ambient helium environment could be partly attributed to the smallest interference by noise after wavelet denoising. A similar improvement was observed in ambient air and argon environments, thereby proving the applicability of the hybrid model under different experimental conditions.     

Daubechies小波主成分回归法机理及算法研究

将小波变换与主成分回归相结合,提出一种新型多元校正算法---小波基主成分回归法。理论分析和仿真实验表明,该法可更有效地去除噪声,提取有用信息。将其用于氯霉素及甲硝唑实际药物体系分析,与主成分回归法(PCR)相比,得到的回收率总平均相对误差由1.70%下降到0.90%。此外,通过将统计判据和小波多尺度分析相结合,发展了一种新的因子数判定方法。理论和实验研究表明,该法比传统因子数判定法具有更高的可靠性。     

A comparison of signal-to-noise ratios for single channel methods (sequential and multiplex) vs multichannel methods in optical spectroscopy

Four major types of spectroscopic systems for quantitative analysis of one or more spectral components we compared, with regard to signal-to-noise ratio for constant analysis time. These four methods are based on sequential-linear scan, sequential-slew scan, multichannel, and multiplex approaches. The multiplex methods can generally be classified into two types, namely Fourier transform spectroscopy and Hadamard transform spectroscopy. It is shown that for the same spectral source, for the same resolving power luminosity product of the optical system, and the same detector, the multichannel approach is the best and the sequential slew scan system is nearly as good for relatively simple spectra in the u.v.- Visible region. Multiplex methods have little to offer in the u.v.- Visible region, where the detector noise limitation seldom applies and where background shot and/or fluctuation noise are dominant but could find considerable use in the u.v.- Visible region for atomic fluorescence or emission spectroscopy especially if the density of spectral lines in the measurement region is not too great and the background intensity is low.     

A novel method to calculate the approximate derivative photoacoustic spectrum using continuous wavelet transform

A novel method based on continuous wavelet transform (CWT) using Haar wavelet function for approximate derivative calculation of analytical signals is proposed and successfully used in processing the photoacoustic signal. An approximate nth derivative of an analytical signal can be obtained by applying n times of the wavelet transform to the signal. The results obtained from four other different methods--the conventional numerical differentiation, the Fourier transform method, the Savitzky-Golay method, and the discrete wavelet transform (DWT) method--were compared with the proposed CWT method; it was demonstrated that all the results are almost the same for signals without noise, but the proposed CWT method is superior to the former four methods for noisy signals. The approximate first and second derivative of the photoacoustic spectrum of Pr(Gly)3Cl3.3H2O and PrCl3.6H2O were obtained using the proposed CWT method; the results are satisfactory.     

Determination of Total Sugar in Tobacco by Near-Infrared Spectroscopy and Wavelet Transformation-Based Calibration

Based on wavelet transformation (WT) and mutual information (MI), a simple and effective procedure is proposed for multivariate calibration of near-infrared spectroscopy. In such a procedure, the original spectra of the training set are first transformed into a set of wavelet representations by wavelet prism transform. Then, the MI value between each wavelet coefficient variable and the dependent variable is calculated, resulting in a MI spectrum; by retaining a subset set of coefficients with higher MI, an update training set consisting of wavelet coefficients is obtained and reconstructed/converted back to the original domain. Based on this, a partial least square (PLS) model can be constructed and optimized. The optimal wavelet and decomposition level are determined by experiment. A NIR quantitative problem involving the determination of total sugar in tobacco is used to demonstrate the overall performance of the proposed procedure, named RPLS, meaning PLS in reconstructed original domain coupled with MI-induced variable selection in wavelet domain (RPLS). Three kinds of procedures, that is, conventional full-spectrum PLS in original domain (FPLS), PLS in original domain coupled with MI-induced variable selection (OPLS), and direct PLS in MI-based wavelet coefficients (WPLS), are used as reference. The result confirms that it can build more accurate and robust calibration models without increasing the complexity.     

Ensemble wavelet modelling for determination of wheat and gasoline properties by near and middle infrared spectroscopy

The wavelet transform has been shown to be a useful tool for multivariate calibration. However, the choice of wavelet transform settings (wavelet family, length and number of decomposition levels) for a given application is still an open problem. The present paper proposes an alternative approach, which consists of generating an ensemble model by combining individual models obtained with different wavelet transform settings. The advantages of the proposed method are demonstrated in two analytical problems, namely the determination of moisture and protein in wheat by near infrared spectroscopy and the determination of specific mass and three distillation temperatures (T10, T50, T90) in gasoline by middle infrared spectroscopy. In these problems, the results varied considerably among individual models, which underlines the risk associated to an inadequate choice of wavelet transform settings. In contrast, the ensemble model always provided adequate results in terms of prediction error and noise sensitivity. The proposed method can be seen as an advantageous alternative for multivariate calibration in the wavelet domain, as it frees the analyst from the need to choose a particular configuration for the wavelet transform.