Determination of α-linolenic acid and linoleic acid in edible oils using near-infrared spectroscopy improved by wavelet transform and uninformative variable elimination

This paper proposes an analytical method for simultaneous near-infrared (NIR) spectrometric determination of α-linolenic and linoleic acid in eight types of edible vegetable oils and their blending. For this purpose, a combination of spectral wavelength selection by wavelet transform (WT) and elimination of uninformative variables (UVE) was proposed to obtain simple partial least square (PLS) models based on a small subset of wavelengths. WT was firstly utilized to compress full NIR spectra which contain 1413 redundant variables, and 42 wavelet approximate coefficients were obtained. UVE was then carried out to further select the informative variables. Finally, 27 and 19 wavelet approximate coefficients were selected by UVE for α-linolenic and linoleic acid, respectively. The selected variables were used as inputs of PLS model. Due to original spectra were compressed, and irrelevant variables were eliminated, more parsimonious and efficient model based on WT-UVE was obtained compared with the conventional PLS model with full spectra data. The coefficient of determination (r²) and root mean square error prediction set (RMSEP) for prediction set were 0.9345 and 0.0123 for α-linolenic acid prediction by WT-UVE-PLS model. The r² and RMSEP were 0.9054, 0.0437 for linoleic acid prediction. The good performance showed a potential application using WT-UVE to select NIR effective variables. WT-UVE can both speed up the calculation and improve the predicted results. The results indicated that it was feasible to fast determine α-linolenic acid and linoleic acid content in edible oils using NIR spectroscopy.     

A method for near-infrared spectral calibration of complex plant samples with wavelet transform and elimination of uninformative variables

An algorithm is proposed for extracting relevant information from near-infrared (NIR) spectra for multivariate calibration of routine components in complex plant samples. The algorithm is a combination of wavelet transform (WT) data compression and a procedure for uninformative variable elimination (UVE). After compression of the NIR spectra by WT, the UVE approach is used to eliminate the irrelevant wavelet coefficients. Finally, a calibration model is built from the retained wavelet coefficients to enable prediction. Because irrelevant information can be removed from the spectra used for multivariate calibration, the model based on the extracted relevant features is better than those obtained with full-spectrum data. Both prediction precision and calculation speed are improved.     

离散小波变换-遗传算法-交互检验法用于近红外光谱数据的高倍压缩与变量筛选

用遗传算法(GA)与交互检验(CV)相结合建立了一种用于对近红外光谱(NIR)数据及其离散小波变换(DWT)系数进行变量筛选的方法，并应用于烟草样品中总挥发碱和总氮的同时测定。结果表明：NIR数据经DWT压缩为原始大小的3．3％时基本没有光谱信息的丢失；有效的变量筛选可以极大地减少模型中的变量个数，降低模型的复杂程度，改善预测的准确度。     

Determination of Total Sugar in Tobacco by Near-Infrared Spectroscopy and Wavelet Transformation-Based Calibration

Based on wavelet transformation (WT) and mutual information (MI), a simple and effective procedure is proposed for multivariate calibration of near-infrared spectroscopy. In such a procedure, the original spectra of the training set are first transformed into a set of wavelet representations by wavelet prism transform. Then, the MI value between each wavelet coefficient variable and the dependent variable is calculated, resulting in a MI spectrum; by retaining a subset set of coefficients with higher MI, an update training set consisting of wavelet coefficients is obtained and reconstructed/converted back to the original domain. Based on this, a partial least square (PLS) model can be constructed and optimized. The optimal wavelet and decomposition level are determined by experiment. A NIR quantitative problem involving the determination of total sugar in tobacco is used to demonstrate the overall performance of the proposed procedure, named RPLS, meaning PLS in reconstructed original domain coupled with MI-induced variable selection in wavelet domain (RPLS). Three kinds of procedures, that is, conventional full-spectrum PLS in original domain (FPLS), PLS in original domain coupled with MI-induced variable selection (OPLS), and direct PLS in MI-based wavelet coefficients (WPLS), are used as reference. The result confirms that it can build more accurate and robust calibration models without increasing the complexity.     

Improvement of prediction ability of PLS models employing the wavelet packet transform: A case study concerning FT-IR determination of gasoline parameters

The wavelet packet transform (WPT) is a variant of the standard wavelet transform that offers greater flexibility in the decomposition of instrumental signals. Although encouraging results have been published concerning the use of WPT for signal compression and denoising, its application in multivariate calibration problems has received comparatively little attention, with very few contributions reported in the literature. This paper presents an investigation concerning the use of WPT as a feature extraction tool to improve the prediction ability of PLS models. The optimization of the wavelet packet tree is accomplished by using the classic dynamic programming algorithm and an entropy cost function modified to take into account the variance explained by the WPT coefficients. The selection of WPT coefficients for inclusion in the PLS model is carried out on the basis of correlation with the dependent variable, in order to exploit the joint statistics of the instrumental response and the parameter of interest. This WPT-PLS strategy is applied in a case study involving FT-IR spectrometric determination of four gasoline parameters, namely specific mass (SM) and the distillation temperatures at which 10%, 50%, 90% of the sample has evaporated. The dataset comprises 103 gasoline samples collected from gas stations and 6144 wavelengths in the range 2500-15000 nm. By applying WPT to the FT-IR spectra, considerable compression with respect to the original wavelength domain is achieved. The effect of varying the wavelet and the threshold level on the prediction ability of the resulting models is investigated. The results show that WPT-PLS outperforms standard PLS in most wavelet-threshold combinations for all determined parameters.     

Radial basis function network calibration model for near-infrared spectra in wavelet domain using a genetic algorithm

Near-infrared (NIR) spectrometry is now widely used in various fields and great attention is paid to the application of it to addressing complex problems, which brings about the need for the calibration of systems that fail to exhibit satisfactional linear relationship between input-output data. In this work we present a novel method to build a multivariate calibration model for NIR spectra, i.e. genetic algorithm-radial basis function network in wavelet domain (WT-GA-RBFN), which combines the advantages of wavelet transform and genetic algorithm. The variable selection is accomplished in two stages in wavelet domain: at the first stage, the variables are pre-selected (compressed) by variance and at the second stage the variables are further reduced by a special designed GA. The proposed method is illustrated through presenting its application to three NIR data sets in different fields and the comparison to PLS model.     

A partial least squares and wavelet-transform hybrid model to analyze carbon content in coal using laser-induced breakdown spectroscopy

A partial least squares (PLS) and wavelet transform hybrid model are proposed to analyze the carbon content of coal by using laser-induced breakdown spectroscopy (LIBS). The hybrid model is composed of two steps of wavelet analysis procedures, which include environmental denoising and background noise reduction, to pretreat the LIBS spectrum. The processed wavelet coefficients, which contain the discrete line information of the spectra, were taken as inputs for the PLS model for calibration and prediction of carbon element. A higher signal-to-noise ratio of carbon line was obtained after environmental denoising, and the best decomposition level was determined after background noise reduction. The hybrid model resulted in a significant improvement over the conventional PLS method under different ambient environments, which include air, argon, and helium. The average relative error of carbon decreased from 2.74 to 1.67% under an ambient helium environment, which indicated a significantly improved accuracy in the measurement of carbon in coal. The best results obtained under an ambient helium environment could be partly attributed to the smallest interference by noise after wavelet denoising. A similar improvement was observed in ambient air and argon environments, thereby proving the applicability of the hybrid model under different experimental conditions.     

Determination of chlorogenic acid in plant samples by using near-infrared spectrum with wavelet transform preprocessing.

By theoretical analysis, it is found that wavelet transform (WT) with a wavelet function can be regarded as a smoothing and a differentiation process, and that the order of differentiation is determined by the vanishing moment, which is an important property of a wavelet function. Therefore, a method based on the continuous wavelet transform (CWT) for removing the background in the near-infrared (NIR) spectrum is proposed, and it is used in the determination of the chlorogenic acid in plant samples as a preprocessing tool for partial least square (PLS) modeling. It is shown that the benefit of the proposed method lies not only in its performance to improve the quality of PLS model and the prediction precision, but also in its simplicity and practicability. It may become a convenient and efficient tool for preprocessing NIR spectral data sets in multivariate calibration.     

Quantitative Analysis of Berberine in Processed Coptis by Near-Infrared Diffuse Reflectance Spectroscopy

The near-infrared（NIR） diffuse reflectance spectroscopy was used to study the content of Berberine in the processed Coptis. The allocated proportions of Coptis to ginger, yellow liquor or Evodia rutaecarpa changed according to the results of orthogonal design as well as the temperature. For as withdrawing the full and effective information from the spectral data as possible, the spectral data was preprocessed through first derivative and multiplicative scatter correetion（MSC） according to the optimization results of different preprocessing methods. Firstly, the model was established by partial least squares（PLS）; the coefficient of determination（R2） of the prediction was 0.839, the root mean squared error of prediction（RMSEP） was 0.1422, and the mean relative error（RME） was 0.0276. Secondly, for reducing the dimension and removing noise, the spectral variables were highly effectively compressed via the wavelet transformation（WT） technology and the Haar wavelet was selected to decompose the spectral signals. After the wavelet coefficients from WT were input into the artificial neural network（ANN） instead of the spectra signal, the quantitative analysis model of Berberine in processed Coptis was established. The R^2 of the model was 0.9153, the RMSEP was 0.0444, and the RME was 0.0091. The values of appraisal index, namely R^2, RMSECV, and RME, indicate that the generalization ability and prediction precision of ANN are superior to those of PLS. The overall results show that NIR spectroscopy combined with ANN can be efficiently utilized for the rapid and accurate analysis of routine chemical compositions in Coptis. Accordingly, the result can provide technical support for the further analysis of Berberine and other components in processed Coptis. Simultaneously, the research can also offer the foundation of quantitative analysis of other NIR application.     

Determination of soil properties using Fourier transform mid-infrared photoacoustic spectroscopy

Photoacoustic spectroscopy (PAS) is based on the absorption of electromagnetic radiation by analyte molecules, and this technique has emerged as a valuable tool for the study of materials like biological, chemical and geological samples. In this paper, Fourier transform mid-infrared photoacoustic spectroscopy (FTIR-PAS) was used in the prediction of soil properties. Air-dried soil samples (n = 56) from Fengqiu Ecology Experimental Station Chinese Academy of Sciences were involved in this experiment, and FTIR-PAS spectra of these soil samples were recorded. These FTIR-PAS spectra indicated abundant soil information, but overlapping of absorption made it difficult to make direct measurement of soil properties. Partial least squares (PLS) models based on soil FTIR-PAS spectra was developed to predict available nitrogen (N), phosphorus (P), potassium (K) and organic matter content of soil. 42 soil samples were firstly used in leave-one-out cross-validation, and calibration error, calibration coefficient, validation error and ratio of standard deviation to prediction error (RPD) were obtained to optimize the PLS factor number; then based on the optimized PLS models the soil properties of the other 14 soil samples were predicted. The calibration statistics showed that the PLS model was suitable to use in the prediction of available N, P, K and organic matter content of soil. This prediction technique was non-destructive, and no sample pre-treatment was needed, which made FTIR-PAS a very promising method for fast evaluation of soil properties as well as soil quality.     

Resolution of overlapping signals in spectrometry using a wavelet packet transform and an Elman recurrent neural network

A novel method named a wavelet packet transform based Elman recurrent neural network (WPTERNN) was proposed for the simultaneous UV–visible spectrometric determination of Cu(II), Cd(II) and Zn(II). This method combined wavelet packet denoising with an Elman recurrent neural network. A wavelet packet transform was applied to perform data compression, to extract relevant information, and to eliminate noise and collinearity. An Elman recurrent network was applied for nonlinear multivariate calibration. In this case, using trials, the kind of wavelet function, the decomposition level, and the number of hidden nodes for the WPTERNN method were selected as Daubechies 14, 3, and 8, respectively. A program (PWPTERNN) was designed that could perform the simultaneous determination of Cu(II), Cd(II) and Zn(II). The relative standard errors of prediction (RSEP) obtained for all components using WPTERNN, a Elman recurrent neural network (ERNN), partial least squares (PLS), principal component regression (PCR), Fourier transform based PCR (FTPCR), and multivariate linear regression (MLR) were compared. Experimental results demonstrated that the WPTERRN method was successful even where there was severe overlap of spectra. The results obtained from an additional test case also demonstrated that the WPTERNN method performed very well. Figure The part of WP coefficients obtained by wavelet packet transforms     

Combining orthogonal signal correction and wavelet packet transform with partial least squares to analyze overlapping spectra of three kinds of metal ions

A novel method named OSC-WPT-PLS approach based on partial least squares (PLS) regression with orthogonal signal correction (OSC) and wavelet packet transform (WPT) as pre-processed tools was proposed for the simultaneous spectrophotometric determination of Al(III), Mn(II) and Co(II). This method combines the ideas of OSC and WPT with PLS regression for enhancing the ability of extracting characteristic information and the quality of regression. OSC is used to remove information in the response matrix D by subtracting the structured noise that is orthogonal to the concentration matrix C. Wavelet packet transform was applied to perform data compression, to extract relevant information, and to eliminate noise and collinearity. PLS was applied for multivariate calibration and noise reduction by eliminating the less important latent variables. In this case, using trials, the kind of wavelet function, the decomposition level, the number of OSC components and the number of PLS factors for the OSC-WPT-PLS method were selected as Daubechies 4, 3, 2 and 3, respectively. A program (POSCWPTPLS) was designed to perform the simultaneous spectrophotometric determination of Al(III), Mn(II) and Co(II). The relative standard errors of prediction (RSEP) obtained for total elements using OSC-WPT-PLS, WPT-PLS and PLS were compared. Experimental results demonstrated that the OSC-WPT-PLS method had the best performance among the three methods and was successful even when there was severe overlap of spectra.     

Baseline correction combined partial least squares algorithm and its application in on-line Fourier transform infrared quantitative analysis

In order to eliminate the lower order polynomial interferences, a new quantitative calibration algorithm “Baseline Correction Combined Partial Least Squares (BCC-PLS)”, which combines baseline correction and conventional PLS, is proposed. By embedding baseline correction constraints into PLS weights selection, the proposed calibration algorithm overcomes the uncertainty in baseline correction and can meet the requirement of on-line attenuated total reflectance Fourier transform infrared (ATR-FTIR) quantitative analysis. The effectiveness of the algorithm is evaluated by the analysis of glucose and marzipan ATR-FTIR spectra. BCC-PLS algorithm shows improved prediction performance over PLS. The root mean square error of cross-validation (RMSECV) on marzipan spectra for the prediction of the moisture is found to be 0.53%, w/w (range 7–19%). The sugar content is predicted with a RMSECV of 2.04%, w/w (range 33–68%).     

基于近红外光谱的食用植物油中反式脂肪酸含量快速定量检测及模型优化研究

利用近红外光谱技术对食用植物油中反式脂肪酸(Trans fatty acids,TFA)含量进行快速定量检测,并通过波段选择、预处理方法、变量筛选及建模方法对TFA含量预测模型进行优化.采用AntarisⅡ傅里叶变换近红外光谱仪在4000~10000 cm-1光谱范围采集98个食用植物油样本的近红外透射光谱,然后采用气相色谱法测定TFA的真实含量.首先,对样本原始光谱进行波段、预处理方法优选;在此基础上,采用竞争自适应重加权法(Competitive adaptive reweighted sampling,CARS)筛选TFA相关的重要变量,最后应用主成分回归、偏最小二乘和最小二乘支持向量机方法分别建立食用植物油中TFA含量的预测模型.研究结果表明,近红外光谱技术检测食用植物油中的TFA含量是可行的,优化后的最佳预测模型的校正集和预测集R2分别为0.992和0.989,RMSEC和RMSEP分别为0.071%和0.075%.最佳预测模型所用的变量仅26个,占全波段变量的0.854%.此外,与全波段偏最小二乘预测模型相比,其预测集R2由0.904上升为0.989,RMSEP由0.230%下降为0.075%.由此表明,模型优化非常必要,CARS能有效筛选TFA相关的重要变量,极大减少建模变量数,从而简化预测模型,并较大提高预测模型的精度和稳定性.     

基于OSC–WT–PLS近红外光谱法测定烟草中芸香苷

采用正交信号校正(OSC)结合小波变换(WT)对烟草光谱进行光谱预处理,将预处理后的烟草光谱结合偏最小二乘法(PLS)建立了烟草光谱对芸香苷的预测模型。利用OSC滤除光谱中与芸香苷含量无关的光谱信息,确定OSC提取的最佳主成分数为7,再选择WT中的最佳小波基函数bior1.1对OSC预处理后的光谱进行压缩及进一步滤噪,然后进行PLS建模,OSC–WT–PLS所建模型决定系数r~2=0.874,校正标准偏差RMSEC=0.85,预测均方根误差RMSEP=0.743,交互验证系数Q_(ext)~2=0.887。结果表明,用OSC–WT–PLS可滤除光谱信息中与待测样品含量无关的信息、减少光谱数据量,降低建立模型的复杂度、提高建模速度及模型的预测能力、准确度。     

A new hybrid strategy for constructing a robust calibration model for near-infrared spectral analysis

A new hybrid algorithm is proposed for construction of a high-quality calibration model for near-infrared (NIR) spectra that is robust against both spectral interference (including background and noise) and multiple outliers. The algorithm is a combination of continuous wavelet transform (CWT) and a modified iterative reweighted PLS (mIRPLS) procedure. In the proposed algorithm the spectral interference is filtered by CWT at the first stage then mIRPLS is proposed to detect the multiple outliers in the CWT domain. Compared with the original IRPLS method, mIRPLS does not need to adjust variable parameters to achieve optimum calibration results, which makes it very convenient to perform in practice. The final PLS model is constructed robustly because both the spectral interference and multiple outliers are eliminated. In order to validate the effectiveness and universality of the algorithm, it was applied to two different sets of NIR spectra. The results indicate that the proposed strategy can greatly enhance the robustness and predictive ability of NIR spectral analysis.     

基于小波系数的近红外光谱局部建模方法与应用研究

局部建模方法使用与预测样本相似的样本建立模型,可解决光谱响应与浓度之间的非线性问题,扩大模型的适用范围,提高预测准确度。采用小波变换进行数据压缩并利用小波系数之间的欧氏距离作为光谱相似性的判据,实现了近红外光谱定量分析的局部建模方法,避免了样本之间的依赖性。将所建立的方法用于烟草样品中氯含量的测定,100次重复计算得到的预测集均方根误差(RMSEP)平均值为0.0665,标准偏差(σ)为0.0045,优于全局建模和基于主成分的局部建模方法。     

Treating NIR data with orthogonal discrete wavelet transform: Predicting concentrations of a multi-component system through a small-scale calibration set

Through randomly arranging samples of a calibration set, treating their NIR spectra with orthogonal discrete wavelet transform, and selecting suitable variables in terms of correlation coefficient test (r-test), it is possible to extract features of each component in a multi-component system respectively and partial least squares (PLS) models based on these features are capable of predicting the concentration of every component. What is perhaps more important, with the proposed strategy, the predictive ability of the model is at least not impaired while the size of the calibration set can be obviously reduced. Therefore, it provides a more economical, rapid, as well as convenient approach of NIR quantitative analysis for multi-component system. In addition, all important factors and parameters related to the proposed strategy are discussed in detail.     

Butene concentration prediction in ethylene/propylene/1-butene terpolymers by FT-IR spectroscopy through multivariate statistical analysis and artificial neural networks

This paper reports the development of calibration models for quality control in the production of ethylene/propylene/1-butene terpolymers by the use of multivariate tools and FT-IR spectroscopy.1-Butene concentration prediction is achieved in terpolymers by coupling FT-IR spectroscopy to multivariate regression tools. A dataset of 26 terpolymers (14 coming from a constrained experimental design for mixtures, plus 12 terpolymers used for external validation) was analysed by FT-IR spectroscopy. An internal method of “Polimeri Europa” plant, based on ¹³C NMR spectroscopy is used to determine the percentage of 1-butene in the samples. Then, different multivariate tools are used for 1-butene concentration prediction based on the FT-IR spectra recorded. Different multivariate calibration methods were explored: principal component regression (PCR), partial least squares (PLS), stepwise OLS regression (SWR) and artificial neural networks (ANNs). The model obtained by back-propagation neural networks turned out to be the best one. The performances of the BP-ANN model were further improved by variable selection procedures based on the calculation of the first derivative of the network.The proposed approach allows the monitoring in real time of the polymer synthesis and the estimation of the characteristics of the product attainable from the concentration of 1-butene.     

激光诱导击穿光谱结合CARS变量选择方法定量检测倍硫磷含量

利用双脉冲激光诱导击穿光谱(LIBS)技术对溶液中的倍硫磷含量进行定量检测。采用二通道高精度光谱仪采集不同浓度倍硫磷样品在206.28~481.77 nm波段的LIBS光谱,并对光谱进行多元散射校正(MSC)、标准正态变量变换(SNV)及3点平滑预处理,根据偏最小二乘(PLS)建模确定最优的预处理方法。在此基础上,利用竞争性自适应重加权算法(CARS)筛选与倍硫磷相关的重要变量,然后应用PLS回归建立溶液中倍硫磷含量的定量分析模型,并与单变量定量分析模型及未变量选择的PLS定量分析模型进行比较。结果表明,相比单变量定量分析模型及原始光谱PLS定量分析模型,CARS-PLS定量分析模型的性能更优,其模型的校正集和预测集的决定系数及平均相对误差分别为0.969 4、15.537%和0.995 9、5.016%。此外,与原始光谱PLS模型相比,CARS-PLS模型仅使用其中1.9%的波长变量,但预测集平均误差却由9.829%下降为5.016%。由此可见,LIBS技术检测溶液中的倍硫磷含量具有一定的可行性,且CARS方法能简化定量分析模型,提高模型的预测精度。