Determination of α-linolenic acid and linoleic acid in edible oils using near-infrared spectroscopy improved by wavelet transform and uninformative variable elimination

This paper proposes an analytical method for simultaneous near-infrared (NIR) spectrometric determination of α-linolenic and linoleic acid in eight types of edible vegetable oils and their blending. For this purpose, a combination of spectral wavelength selection by wavelet transform (WT) and elimination of uninformative variables (UVE) was proposed to obtain simple partial least square (PLS) models based on a small subset of wavelengths. WT was firstly utilized to compress full NIR spectra which contain 1413 redundant variables, and 42 wavelet approximate coefficients were obtained. UVE was then carried out to further select the informative variables. Finally, 27 and 19 wavelet approximate coefficients were selected by UVE for α-linolenic and linoleic acid, respectively. The selected variables were used as inputs of PLS model. Due to original spectra were compressed, and irrelevant variables were eliminated, more parsimonious and efficient model based on WT-UVE was obtained compared with the conventional PLS model with full spectra data. The coefficient of determination (r²) and root mean square error prediction set (RMSEP) for prediction set were 0.9345 and 0.0123 for α-linolenic acid prediction by WT-UVE-PLS model. The r² and RMSEP were 0.9054, 0.0437 for linoleic acid prediction. The good performance showed a potential application using WT-UVE to select NIR effective variables. WT-UVE can both speed up the calculation and improve the predicted results. The results indicated that it was feasible to fast determine α-linolenic acid and linoleic acid content in edible oils using NIR spectroscopy.     

Multivariate calibration of near infrared spectra by orthogonal WAVElet correction using a genetic algorithm

Orthogonal WAVElet correction (OWAVEC) is a pre-processing method aimed at simultaneously accomplishing two essential needs in multivariate calibration, signal correction and data compression, by combining the application of an orthogonal signal correction algorithm to remove information unrelated to a certain response with the great potential that wavelet analysis has shown for signal processing. In the previous version of the OWAVEC method, once the wavelet coefficients matrix had been computed from NIR spectra and deflated from irrelevant information in the orthogonalization step, effective data compression was achieved by selecting those largest correlation/variance wavelet coefficients serving as the basis for the development of a reliable regression model. This paper presents an evolution of the OWAVEC method, maintaining the first two stages in its application procedure (wavelet signal decomposition and direct orthogonalization) intact but incorporating genetic algorithms as a wavelet coefficients selection method to perform data compression and to improve the quality of the regression models developed later. Several specific applications dealing with diverse NIR regression problems are analyzed to evaluate the actual performance of the new OWAVEC method. Results provided by OWAVEC are also compared with those obtained with original data and with other orthogonal signal correction methods.     

Elimination of interference information by a new hybrid algorithm for quantitative calibration of near infrared spectra

A new hybrid algorithm is proposed to eliminate the interference information for multivariate calibration of near-infrared (NIR) spectra that includes noise, background and systemic spectral variation irrelevant to concentration. The method consists of two parts: approximate derivative based on continuous wavelet transform (CWT) and orthogonal signal correction (OSC). After the approximate derivative calculated by CWT, OSC was performed. It was successfully applied to real complex NIR spectral data to eliminate the interference information. Correction for the interference of NIR spectra resulted in a substantial improvement in the predicted precision, and a more concise calibration model was obtained. The proposed procedure also compared favourably with several pretreatment methods, and the new method appears to provide a high-performance pretreatment tool for multivariate calibration of NIR spectra. In addition, the strategy proposed here can be applied to various other spectral data for quantitative purposes as well.     

Detecting the quality of glycerol monolaurate: A method for using Fourier transform infrared spectroscopy with wavelet transform and modified uninformative variable elimination

Glycerol monolaurate (GML) products contain many impurities, such as lauric acid and glucerol. The GML content is an important quality indicator for GML production. A hybrid variable selection algorithm, which is a combination of wavelet transform (WT) technology and modified uninformative variable eliminate (MUVE) method, was proposed to extract useful information from Fourier transform infrared (FT-IR) transmission spectroscopy for the determination of GML content. FT-IR spectra data were compressed by WT first; the irrelevant variables in the compressed wavelet coefficients were eliminated by MUVE. In the MUVE process, simulated annealing (SA) algorithm was employed to search the optimal cutoff threshold. After the WT-MUVE process, variables for the calibration model were reduced from 7366 to 163. Finally, the retained variables were employed as inputs of partial least squares (PLS) model to build the calibration model. For the prediction set, the correlation coefficient (r) of 0.9910 and root mean square error of prediction (RMSEP) of 4.8617 were obtained. The prediction result was better than the PLS model with full-spectra data. It was indicated that proposed WT-MUVE method could not only make the prediction more accurate, but also make the calibration model more parsimonious. Furthermore, the reconstructed spectra represented the projection of the selected wavelet coefficients into the original domain, affording the chemical interpretation of the predicted results. It is concluded that the FT-IR transmission spectroscopy technique with the proposed method is promising for the fast detection of GML content.     

An adaptive strategy for selecting representative calibration samples in the continuous wavelet domain for near-infrared spectral analysis

Sample selection is often used to improve the cost-effectiveness of near-infrared (NIR) spectral analysis. When raw NIR spectra are used, however, it is not easy to select appropriate samples, because of background interference and noise. In this paper, a novel adaptive strategy based on selection of representative NIR spectra in the continuous wavelet transform (CWT) domain is described. After pretreatment with the CWT, an extension of the Kennard–Stone (EKS) algorithm was used to adaptively select the most representative NIR spectra, which were then submitted to expensive chemical measurement and multivariate calibration. With the samples selected, a PLS model was finally built for prediction. It is of great interest to find that selection of representative samples in the CWT domain, rather than raw spectra, not only effectively eliminates background interference and noise but also further reduces the number of samples required for a good calibration, resulting in a high-quality regression model that is similar to the model obtained by use of all the samples. The results indicate that the proposed method can effectively enhance the cost-effectiveness of NIR spectral analysis. The strategy proposed here can also be applied to different analytical data for multivariate calibration.     

Removal of major interference sources in aqueous near-infrared spectroscopy techniques

This work describes a hybrid procedure for eliminating major interference sources in aqueous near-infrared (NIR) spectra, that include aqueous influence, noise, and systemic variations irrelevant to concentration. The scheme consists of two parts: extension of wavelet prism (WPe) and orthogonal signal correction (OSC). First, WPe is employed to remove variations due to aqueous absorbance and noise; then OSC is applied to remove systemic spectral variations irrelevant to concentration. Although water possesses strong absorption bands that overshadow and overlap the absorption bands of analytes, along with noise and systematic interference, successful calibration models can be generated by employing the method proposed here. We show that the elimination of major interference sources from the aqueous NIR spectra results in a substantial improvement in the precision of prediction, and reduces the required number of PLS components in the model. In addition, the strategy proposed here can be applied to various analytical data for quantitative purposes as well.     

自适应小波算法用于近红外光谱的多元校正

实现了一种构建自适应小波滤波器的方法,并将其用于近红外光谱数据的多元校正。该方法根据一定的目标函数,针对信号的特性自适应地构造小波滤波器。用该法构建的滤波器对烟草样品的近红外光谱进行压缩,并将压缩后的数据采用偏最小二乘法建模,实现了烟草样品常规组分的定量分析。     

离散小波变换-遗传算法-交互检验法用于近红外光谱数据的高倍压缩与变量筛选

用遗传算法(GA)与交互检验(CV)相结合建立了一种用于对近红外光谱(NIR)数据及其离散小波变换(DWT)系数进行变量筛选的方法，并应用于烟草样品中总挥发碱和总氮的同时测定。结果表明：NIR数据经DWT压缩为原始大小的3．3％时基本没有光谱信息的丢失；有效的变量筛选可以极大地减少模型中的变量个数，降低模型的复杂程度，改善预测的准确度。     

Determination of Total Sugar in Tobacco by Near-Infrared Spectroscopy and Wavelet Transformation-Based Calibration

Based on wavelet transformation (WT) and mutual information (MI), a simple and effective procedure is proposed for multivariate calibration of near-infrared spectroscopy. In such a procedure, the original spectra of the training set are first transformed into a set of wavelet representations by wavelet prism transform. Then, the MI value between each wavelet coefficient variable and the dependent variable is calculated, resulting in a MI spectrum; by retaining a subset set of coefficients with higher MI, an update training set consisting of wavelet coefficients is obtained and reconstructed/converted back to the original domain. Based on this, a partial least square (PLS) model can be constructed and optimized. The optimal wavelet and decomposition level are determined by experiment. A NIR quantitative problem involving the determination of total sugar in tobacco is used to demonstrate the overall performance of the proposed procedure, named RPLS, meaning PLS in reconstructed original domain coupled with MI-induced variable selection in wavelet domain (RPLS). Three kinds of procedures, that is, conventional full-spectrum PLS in original domain (FPLS), PLS in original domain coupled with MI-induced variable selection (OPLS), and direct PLS in MI-based wavelet coefficients (WPLS), are used as reference. The result confirms that it can build more accurate and robust calibration models without increasing the complexity.     

A variable differential consensus method for improving the quantitative near-infrared spectroscopic analysis

Consensus methods have presented promising tools for improving the reliability of quantitative models in near-infrared(NIR) spectroscopic analysis.A strategy for improving the performance of consensus methods in multivariate calibration of NIR spectra is proposed.In the approach,a subset of non-collinear variables is generated using successive projections algorithm(SPA) for each variable in the reduced spectra by uninformative variables elimination(UVE).Then sub-models are built using the variable subsets and the calibration subsets determined by Monte Carlo(MC) re-sampling,and the sub-model that produces minimal error in cross validation is selected as a member model.With repetition of the MC re-sampling,a series of member models are built and a consensus model is achieved by averaging all the member models.Since member models are built with the best variable subset and the randomly selected calibration subset,both the quality and the diversity of the member models are insured for the consensus model.Two NIR spectral datasets of tobacco lamina are used to investigate the proposed method.The superiority of the method in both accuracy and reliability is demonstrated.     

Radial basis function network calibration model for near-infrared spectra in wavelet domain using a genetic algorithm

Near-infrared (NIR) spectrometry is now widely used in various fields and great attention is paid to the application of it to addressing complex problems, which brings about the need for the calibration of systems that fail to exhibit satisfactional linear relationship between input-output data. In this work we present a novel method to build a multivariate calibration model for NIR spectra, i.e. genetic algorithm-radial basis function network in wavelet domain (WT-GA-RBFN), which combines the advantages of wavelet transform and genetic algorithm. The variable selection is accomplished in two stages in wavelet domain: at the first stage, the variables are pre-selected (compressed) by variance and at the second stage the variables are further reduced by a special designed GA. The proposed method is illustrated through presenting its application to three NIR data sets in different fields and the comparison to PLS model.     

Determination of chlorogenic acid in plant samples by using near-infrared spectrum with wavelet transform preprocessing.

By theoretical analysis, it is found that wavelet transform (WT) with a wavelet function can be regarded as a smoothing and a differentiation process, and that the order of differentiation is determined by the vanishing moment, which is an important property of a wavelet function. Therefore, a method based on the continuous wavelet transform (CWT) for removing the background in the near-infrared (NIR) spectrum is proposed, and it is used in the determination of the chlorogenic acid in plant samples as a preprocessing tool for partial least square (PLS) modeling. It is shown that the benefit of the proposed method lies not only in its performance to improve the quality of PLS model and the prediction precision, but also in its simplicity and practicability. It may become a convenient and efficient tool for preprocessing NIR spectral data sets in multivariate calibration.     

Quantitative Analysis of Berberine in Processed Coptis by Near-Infrared Diffuse Reflectance Spectroscopy

The near-infrared（NIR） diffuse reflectance spectroscopy was used to study the content of Berberine in the processed Coptis. The allocated proportions of Coptis to ginger, yellow liquor or Evodia rutaecarpa changed according to the results of orthogonal design as well as the temperature. For as withdrawing the full and effective information from the spectral data as possible, the spectral data was preprocessed through first derivative and multiplicative scatter correetion（MSC） according to the optimization results of different preprocessing methods. Firstly, the model was established by partial least squares（PLS）; the coefficient of determination（R2） of the prediction was 0.839, the root mean squared error of prediction（RMSEP） was 0.1422, and the mean relative error（RME） was 0.0276. Secondly, for reducing the dimension and removing noise, the spectral variables were highly effectively compressed via the wavelet transformation（WT） technology and the Haar wavelet was selected to decompose the spectral signals. After the wavelet coefficients from WT were input into the artificial neural network（ANN） instead of the spectra signal, the quantitative analysis model of Berberine in processed Coptis was established. The R^2 of the model was 0.9153, the RMSEP was 0.0444, and the RME was 0.0091. The values of appraisal index, namely R^2, RMSECV, and RME, indicate that the generalization ability and prediction precision of ANN are superior to those of PLS. The overall results show that NIR spectroscopy combined with ANN can be efficiently utilized for the rapid and accurate analysis of routine chemical compositions in Coptis. Accordingly, the result can provide technical support for the further analysis of Berberine and other components in processed Coptis. Simultaneously, the research can also offer the foundation of quantitative analysis of other NIR application.     

Using near-infrared process analysis to study gas-solid adsorption process as well as its data treatment based on artificial neural network and partial least squares

This paper demonstrates the application of near-infrared (NIR) process analysis to study gas-solid adsorption process non-invasively: its experimental setup, data treatment, and potentials as a convenient tool to investigate the gas-solid adsorption process. The experimental setup includes a differential adsorption bed (DAB) monitored by a NIR spectrometer via an optical fiber probe, which makes it convenient and reliable to construct adsorption mass-transfer models. A chemometrics strategy based on back propagation-artificial neural network (BP-ANN) and partial least squares (PLS) has been developed to treat NIR spectra collected during the adsorption process because of the obvious nonlinearity in concentration prediction. This nonlinear problem results from the great concentration variation of the adsorbate adsorbed by the adsorbent during the whole adsorption process, the extraordinarily low concentration of the adsorbed adsorbate at the beginning of the process, and probably NIR distinction between the adsorbate on the first adsorption layer at the beginning of the process and that on the other layers afterward. With the strategy, NIR spectra are pretreated with PLS for data compression and noise reduction, and then a BP-ANN is built as the nonlinear calibration model. As compared with linear calibration algorithm, our strategy has the higher predication ability for the whole adsorption process, even with less calibration samples. Finally, as an example the kinetics of aniline-silica gel adsorption process has been studied through the experimental setup and chemometrics strategy.     

Chemometrical strategies for feature selection and data compression applied to NIR and MIR spectra of extra virgin olive oils for cultivar identification

The possibility provided by Chemometrics to extract and combine (fusion) information contained in NIR and MIR spectra in order to discriminate monovarietal extra virgin olive oils according to olive cultivar (Casaliva, Leccino, Frantoio) has been investigated.Linear discriminant analysis (LDA) was applied as a classification technique on these multivariate and non-specific spectral data both separately and jointly (NIR and MIR data together).In order to ensure a more appropriate ratio between the number of objects (samples) and number of variables (absorbance at different wavenumbers), LDA was preceded either by feature selection or variable compression. For feature selection, the SELECT algorithm was used while a wavelet transform was applied for data compression.Correct classification rates obtained by cross-validation varied between 60% and 90% depending on the followed procedure. Most accurate results were obtained using the fused NIR and MIR data, with either feature selection or data compression.Chemometrical strategies applied to fused NIR and MIR spectra represent an effective method for classification of extra virgin olive oils on the basis of the olive cultivar.     

基于小波系数的近红外光谱局部建模方法与应用研究

局部建模方法使用与预测样本相似的样本建立模型,可解决光谱响应与浓度之间的非线性问题,扩大模型的适用范围,提高预测准确度。采用小波变换进行数据压缩并利用小波系数之间的欧氏距离作为光谱相似性的判据,实现了近红外光谱定量分析的局部建模方法,避免了样本之间的依赖性。将所建立的方法用于烟草样品中氯含量的测定,100次重复计算得到的预测集均方根误差(RMSEP)平均值为0.0665,标准偏差(σ)为0.0045,优于全局建模和基于主成分的局部建模方法。     

A consensus least squares support vector regression (LS-SVR) for analysis of near-infrared spectra of plant samples

Consensus modeling of combining the results of multiple independent models to produce a single prediction avoids the instability of single model. Based on the principle of consensus modeling, a consensus least squares support vector regression (LS-SVR) method for calibrating the near-infrared (NIR) spectra was proposed. In the proposed approach, NIR spectra of plant samples were firstly preprocessed using discrete wavelet transform (DWT) for filtering the spectral background and noise, then, consensus LS-SVR technique was used for building the calibration model. With an optimization of the parameters involved in the modeling, a satisfied model was achieved for predicting the content of reducing sugar in plant samples. The predicted results show that consensus LS-SVR model is more robust and reliable than the conventional partial least squares (PLS) and LS-SVR methods.     

近红外分析中光谱预处理及波长选择方法进展与应用

光谱预处理和波长选取方法在近红外光谱分析技术中相当重要。本文综述了常用的NIR预处理和波长选取方法及这一领域的最新进展,详细介绍正交信号校正(OSC)、净分析信号(NAS)和小波变换(WT)等新光谱预处理方法以及无信息变量消除(UVE)和遗传算法(GA)等波长选取方法,并给出了这些方法的具体算法和一些应用实例。     

Improvement of prediction ability of PLS models employing the wavelet packet transform: A case study concerning FT-IR determination of gasoline parameters

The wavelet packet transform (WPT) is a variant of the standard wavelet transform that offers greater flexibility in the decomposition of instrumental signals. Although encouraging results have been published concerning the use of WPT for signal compression and denoising, its application in multivariate calibration problems has received comparatively little attention, with very few contributions reported in the literature. This paper presents an investigation concerning the use of WPT as a feature extraction tool to improve the prediction ability of PLS models. The optimization of the wavelet packet tree is accomplished by using the classic dynamic programming algorithm and an entropy cost function modified to take into account the variance explained by the WPT coefficients. The selection of WPT coefficients for inclusion in the PLS model is carried out on the basis of correlation with the dependent variable, in order to exploit the joint statistics of the instrumental response and the parameter of interest. This WPT-PLS strategy is applied in a case study involving FT-IR spectrometric determination of four gasoline parameters, namely specific mass (SM) and the distillation temperatures at which 10%, 50%, 90% of the sample has evaporated. The dataset comprises 103 gasoline samples collected from gas stations and 6144 wavelengths in the range 2500-15000 nm. By applying WPT to the FT-IR spectra, considerable compression with respect to the original wavelength domain is achieved. The effect of varying the wavelet and the threshold level on the prediction ability of the resulting models is investigated. The results show that WPT-PLS outperforms standard PLS in most wavelet-threshold combinations for all determined parameters.     

Development of a univariate calibration model for pharmaceutical analysis based on NIR spectra

Near-infrared spectroscopy (NIRS) has been widely used in the pharmaceutical field because of its ability to provide quality information about drugs in near-real time. In practice, however, the NIRS technique requires construction of multivariate models in order to correct collinearity and the typically poor selectivity of NIR spectra. In this work, a new methodology for constructing simple NIR calibration models has been developed, based on the spectrum for the target analyte (usually the active principle ingredient, API), which is compared with that of the sample in order to calculate a correlation coefficient. To this end, calibration samples are prepared spanning an adequate concentration range for the API and their spectra are recorded. The model thus obtained by relating the correlation coefficient to the sample concentration is subjected to least-squares regression. The API concentration in validation samples is predicted by interpolating their correlation coefficients in the straight calibration line previously obtained. The proposed method affords quantitation of API in pharmaceuticals undergoing physical changes during their production process (e.g. granulates, and coated and non-coated tablets). The results obtained with the proposed methodology, based on correlation coefficients, were compared with the predictions of PLS1 calibration models, with which a different model is required for each type of sample. Error values lower than 1-2% were obtained in the analysis of three types of sample using the same model; these errors are similar to those obtained by applying three PLS models for granules, and non-coated and coated samples. Based on the outcome, our methodology is a straightforward choice for constructing calibration models affording expeditious prediction of new samples with varying physical properties. This makes it an effective alternative to multivariate calibration, which requires use of a different model for each type of sample, depending on its physical presentation.