Radial basis function network calibration model for near-infrared spectra in wavelet domain using a genetic algorithm

Near-infrared (NIR) spectrometry is now widely used in various fields and great attention is paid to the application of it to addressing complex problems, which brings about the need for the calibration of systems that fail to exhibit satisfactional linear relationship between input-output data. In this work we present a novel method to build a multivariate calibration model for NIR spectra, i.e. genetic algorithm-radial basis function network in wavelet domain (WT-GA-RBFN), which combines the advantages of wavelet transform and genetic algorithm. The variable selection is accomplished in two stages in wavelet domain: at the first stage, the variables are pre-selected (compressed) by variance and at the second stage the variables are further reduced by a special designed GA. The proposed method is illustrated through presenting its application to three NIR data sets in different fields and the comparison to PLS model.     

Continuous wavelet transform applied to removing the fluctuating background in near-infrared spectra

A novel method based on continuous wavelet transform (CWT) was proposed as a preprocessing tool for the near-infrared (NIR) spectra. Due to the property of the vanishing moments of the wavelet, the fluctuating background of the NIR spectra can be successfully removed through convolution of the spectra with an appropriate wavelet function. The vanishing moments of a wavelet and the scale parameter are two key factors that govern the result of the background elimination. The result of its application to both the simulated spectra and the NIR spectra of tobacco samples demonstrates that CWT is a competitive tool for removing fluctuating background in spectra.     

基于小波系数的近红外光谱局部建模方法与应用研究

局部建模方法使用与预测样本相似的样本建立模型,可解决光谱响应与浓度之间的非线性问题,扩大模型的适用范围,提高预测准确度。采用小波变换进行数据压缩并利用小波系数之间的欧氏距离作为光谱相似性的判据,实现了近红外光谱定量分析的局部建模方法,避免了样本之间的依赖性。将所建立的方法用于烟草样品中氯含量的测定,100次重复计算得到的预测集均方根误差(RMSEP)平均值为0.0665,标准偏差(σ)为0.0045,优于全局建模和基于主成分的局部建模方法。     

A new hybrid strategy for constructing a robust calibration model for near-infrared spectral analysis

A new hybrid algorithm is proposed for construction of a high-quality calibration model for near-infrared (NIR) spectra that is robust against both spectral interference (including background and noise) and multiple outliers. The algorithm is a combination of continuous wavelet transform (CWT) and a modified iterative reweighted PLS (mIRPLS) procedure. In the proposed algorithm the spectral interference is filtered by CWT at the first stage then mIRPLS is proposed to detect the multiple outliers in the CWT domain. Compared with the original IRPLS method, mIRPLS does not need to adjust variable parameters to achieve optimum calibration results, which makes it very convenient to perform in practice. The final PLS model is constructed robustly because both the spectral interference and multiple outliers are eliminated. In order to validate the effectiveness and universality of the algorithm, it was applied to two different sets of NIR spectra. The results indicate that the proposed strategy can greatly enhance the robustness and predictive ability of NIR spectral analysis.     

Wavelet unfolded partial least squares for near-infrared spectral quantitative analysis of blood and tobacco powder samples

Continuous wavelet transform (CWT) has been shown to be a high-performance signal processing technique in multivariate calibration. However, the signal processed by CWT with a specific wavelet may account for only a part of the information. To effectively utilize more abundant information contained in analytical signals, a method, named as wavelet unfolded partial least squares (WUPLS), was proposed. In the approach, the measured dataset is firstly extended by CWT with different wavelets, and then partial least squares (PLS) is employed to develop the quantitative model between the extended dataset and the target values. In order to select the representative wavelets, principal component analysis (PCA) is used to investigate the distribution of the signals obtained by CWT with different wavelets. The performance of the method was tested with blood and tobacco powder samples. Compared with the results obtained by PLS methods, the WUPLS method combined with signal processing techniques is proven to be a promising tool for improving the near-infrared (NIR) spectral analysis of complex samples.     

Simultaneous wavelength selection and outlier detection in multivariate regression of near-infrared spectra.

Near-infrared (NIR) spectrometry will present a more promising tool for quantitative measurement if the robustness and predictive ability of the partial least square (PLS) model are improved. In order to achieve the purpose, we present a new algorithm for simultaneous wavelength selection and outlier detection; at the same time, the problems of background and noise in multivariate calibration are also solved. The strategy is a combination of continuous wavelet transform (CWT) and modified iterative predictors and objects weighting PLS (mIPOW-PLS). CWT is performed as a pretreatment tool for eliminating background and noise synchronously; then, mIPOW-PLS is proposed to remove both the useless wavelengths and the multiple outliers in CWT domain. After pretreatment with CWT-mIPOW-PLS, a PLS model is built finally for prediction. The results indicate that the combination of CWT and mIPOW-PLS produces robust and parsimonious regression models with very few wavelengths.     

离散小波变换-遗传算法-交互检验法用于近红外光谱数据的高倍压缩与变量筛选

用遗传算法(GA)与交互检验(CV)相结合建立了一种用于对近红外光谱(NIR)数据及其离散小波变换(DWT)系数进行变量筛选的方法，并应用于烟草样品中总挥发碱和总氮的同时测定。结果表明：NIR数据经DWT压缩为原始大小的3．3％时基本没有光谱信息的丢失；有效的变量筛选可以极大地减少模型中的变量个数，降低模型的复杂程度，改善预测的准确度。     

Determination of Total Sugar in Tobacco by Near-Infrared Spectroscopy and Wavelet Transformation-Based Calibration

Based on wavelet transformation (WT) and mutual information (MI), a simple and effective procedure is proposed for multivariate calibration of near-infrared spectroscopy. In such a procedure, the original spectra of the training set are first transformed into a set of wavelet representations by wavelet prism transform. Then, the MI value between each wavelet coefficient variable and the dependent variable is calculated, resulting in a MI spectrum; by retaining a subset set of coefficients with higher MI, an update training set consisting of wavelet coefficients is obtained and reconstructed/converted back to the original domain. Based on this, a partial least square (PLS) model can be constructed and optimized. The optimal wavelet and decomposition level are determined by experiment. A NIR quantitative problem involving the determination of total sugar in tobacco is used to demonstrate the overall performance of the proposed procedure, named RPLS, meaning PLS in reconstructed original domain coupled with MI-induced variable selection in wavelet domain (RPLS). Three kinds of procedures, that is, conventional full-spectrum PLS in original domain (FPLS), PLS in original domain coupled with MI-induced variable selection (OPLS), and direct PLS in MI-based wavelet coefficients (WPLS), are used as reference. The result confirms that it can build more accurate and robust calibration models without increasing the complexity.     

Determination of α-linolenic acid and linoleic acid in edible oils using near-infrared spectroscopy improved by wavelet transform and uninformative variable elimination

This paper proposes an analytical method for simultaneous near-infrared (NIR) spectrometric determination of α-linolenic and linoleic acid in eight types of edible vegetable oils and their blending. For this purpose, a combination of spectral wavelength selection by wavelet transform (WT) and elimination of uninformative variables (UVE) was proposed to obtain simple partial least square (PLS) models based on a small subset of wavelengths. WT was firstly utilized to compress full NIR spectra which contain 1413 redundant variables, and 42 wavelet approximate coefficients were obtained. UVE was then carried out to further select the informative variables. Finally, 27 and 19 wavelet approximate coefficients were selected by UVE for α-linolenic and linoleic acid, respectively. The selected variables were used as inputs of PLS model. Due to original spectra were compressed, and irrelevant variables were eliminated, more parsimonious and efficient model based on WT-UVE was obtained compared with the conventional PLS model with full spectra data. The coefficient of determination (r²) and root mean square error prediction set (RMSEP) for prediction set were 0.9345 and 0.0123 for α-linolenic acid prediction by WT-UVE-PLS model. The r² and RMSEP were 0.9054, 0.0437 for linoleic acid prediction. The good performance showed a potential application using WT-UVE to select NIR effective variables. WT-UVE can both speed up the calculation and improve the predicted results. The results indicated that it was feasible to fast determine α-linolenic acid and linoleic acid content in edible oils using NIR spectroscopy.     

An adaptive strategy for selecting representative calibration samples in the continuous wavelet domain for near-infrared spectral analysis

Sample selection is often used to improve the cost-effectiveness of near-infrared (NIR) spectral analysis. When raw NIR spectra are used, however, it is not easy to select appropriate samples, because of background interference and noise. In this paper, a novel adaptive strategy based on selection of representative NIR spectra in the continuous wavelet transform (CWT) domain is described. After pretreatment with the CWT, an extension of the Kennard–Stone (EKS) algorithm was used to adaptively select the most representative NIR spectra, which were then submitted to expensive chemical measurement and multivariate calibration. With the samples selected, a PLS model was finally built for prediction. It is of great interest to find that selection of representative samples in the CWT domain, rather than raw spectra, not only effectively eliminates background interference and noise but also further reduces the number of samples required for a good calibration, resulting in a high-quality regression model that is similar to the model obtained by use of all the samples. The results indicate that the proposed method can effectively enhance the cost-effectiveness of NIR spectral analysis. The strategy proposed here can also be applied to different analytical data for multivariate calibration.     

Elimination of interference information by a new hybrid algorithm for quantitative calibration of near infrared spectra

A new hybrid algorithm is proposed to eliminate the interference information for multivariate calibration of near-infrared (NIR) spectra that includes noise, background and systemic spectral variation irrelevant to concentration. The method consists of two parts: approximate derivative based on continuous wavelet transform (CWT) and orthogonal signal correction (OSC). After the approximate derivative calculated by CWT, OSC was performed. It was successfully applied to real complex NIR spectral data to eliminate the interference information. Correction for the interference of NIR spectra resulted in a substantial improvement in the predicted precision, and a more concise calibration model was obtained. The proposed procedure also compared favourably with several pretreatment methods, and the new method appears to provide a high-performance pretreatment tool for multivariate calibration of NIR spectra. In addition, the strategy proposed here can be applied to various other spectral data for quantitative purposes as well.     

A strategy for enhancing the reliability of near-infrared spectral analysis

Near-infrared (NIR) spectroscopy will present a more promising tool for quantitative measurement if the reliability of the calibration model is further improved. To achieve this purpose, a new partial least squares (PLSs) technique based on Monte Carlo (MC) resampling is proposed, which is named as MCPLS. In this method, the outliers are firstly removed based on probability statistics. Then, the models without outliers are averaged and combined into a single prediction model as done in a consensus modeling, which can greatly enhance the reliability of PLS calibration. To validate the effectiveness and universality of the proposed method, it was applied to two different sets of NIR spectra. It was found that MCPLS could effectively avoid the swamping and masking effects caused by multiple outliers. The results show that the method is of value to enhance the reliability of PLS model involving complex NIR matrices with a small number of outliers.     

偏最小二乘回归用于近红外光谱分析的稳健策略

偏最小二乘法（PLS）在近红外光谱（NIR）定量分析中占有重要地位,但预测结果往往容易受到样本分组和奇异样本等因素的影响,稳健性不强。多模型PLS (EPLS）方法在模型稳健性上得到提高,然而它无法识别样本中存在的奇异样本。为了同时提高模型的预测准确性和稳健性,本文提出了一种根据取样概率重新取样的多模型PLS方法,称为稳健共识PLS（RE-PLS）方法。该方法通过迭代赋权偏最小二乘法(IRPLS)计算样本回归残差得到每个校正集样本的取样概率,然后根据样本的取样概率来选择训练子集建立多个PLS模型,最后将所有PLS模型的预测结果平均作为最终预测结果。该方法用于两种不同植物样品的近红外光谱建模,并与传统的PLS及EPLS方法进行比较。结果表明该方法可以有效的避免校正集中奇异样本对模型的影响,同时可以提高预测精确度和稳健性。对于含有较多奇异样本的,复杂近红外光谱烟草实际样本,利用简单PLS或者EPLS方法建模预测效果不是很理想,而RE-PLS凭借其独特优势则有望在这种复杂光谱定量分析中得到广泛的应用。     

近红外漫反射光谱的小波变换滤波

利用小波变换对５２个烟草样品的近红外漫反射光谱进行滤波处理，并用ＰＬＳ法来计算烟草样品的总氮含量，结果表明小波变换滤波后，预测集的相对标准偏差由原来的９．２％降为７．４％，此结果也优于傅里叶变换和五点三次平滑。     

A wavelength selection method based on random decision particle swarm optimization with attractor for near‐infrared spectral quantitative analysis

In this paper, we proposed a wavelength selection method based on random decision particle swarm optimization with attractor for near‐infrared (NIR) spectra quantitative analysis. The proposed method was incorporated with partial least square (PLS) to construct a prediction model. The proposed method chooses the current own optimal or the current global optimal to calculate the attractor. Then the particle updates its flight velocity by the attractor, and the particle state is updated by the random decision with the new velocity. Moreover, the root‐mean‐square error of cross‐validation is adopted as the fitness function for the proposed method. In order to demonstrate the usefulness of the proposed method, PLS with all wavelengths, uninformative variable elimination by PLS, elastic net, genetic algorithm combined with PLS, the discrete particle swarm optimization combined with PLS, the modified particle swarm optimization combined with PLS, the neighboring particle swarm optimization combined with PLS, and the proposed method are used for building the components quantitative analysis models of NIR spectral datasets, and the effectiveness of these models is compared. Two application studies are presented, which involve NIR data obtained from an experiment of meat content determination using NIR and a combustion procedure. Results verify that the proposed method has higher predictive ability for NIR spectral data and the number of selected wavelengths is less. The proposed method has faster convergence speed and could overcome the premature convergence problem. Furthermore, although improving the prediction precision may sacrifice the model complexity under a certain extent, the proposed method is overfitted slightly. Copyright © 2015 John Wiley & Sons, Ltd.     

一种基于小波变换的近红外化学指纹图谱分析方法

提出了一种快速无损的化学指纹图谱分析新方法. 根据小波变换多分辨分析的特点, 对近红外(NIR)光谱进行小波分解, 从中提取被测样品的化学特征信息, 并作数字信息可视化处理, 构建形成可直观识别样品模式特征的NIR指纹图谱. 将该方法用于中药材丹参的质量检测, 检测结果与色谱指纹图谱检测结果相符, 能快速有效地识别丹参质量模式间的差异, 有望发展成为一种快速分析检测天然产物质量的方法.     

Treating NIR data with orthogonal discrete wavelet transform: Predicting concentrations of a multi-component system through a small-scale calibration set

Through randomly arranging samples of a calibration set, treating their NIR spectra with orthogonal discrete wavelet transform, and selecting suitable variables in terms of correlation coefficient test (r-test), it is possible to extract features of each component in a multi-component system respectively and partial least squares (PLS) models based on these features are capable of predicting the concentration of every component. What is perhaps more important, with the proposed strategy, the predictive ability of the model is at least not impaired while the size of the calibration set can be obviously reduced. Therefore, it provides a more economical, rapid, as well as convenient approach of NIR quantitative analysis for multi-component system. In addition, all important factors and parameters related to the proposed strategy are discussed in detail.     

A consensus least squares support vector regression (LS-SVR) for analysis of near-infrared spectra of plant samples

Consensus modeling of combining the results of multiple independent models to produce a single prediction avoids the instability of single model. Based on the principle of consensus modeling, a consensus least squares support vector regression (LS-SVR) method for calibrating the near-infrared (NIR) spectra was proposed. In the proposed approach, NIR spectra of plant samples were firstly preprocessed using discrete wavelet transform (DWT) for filtering the spectral background and noise, then, consensus LS-SVR technique was used for building the calibration model. With an optimization of the parameters involved in the modeling, a satisfied model was achieved for predicting the content of reducing sugar in plant samples. The predicted results show that consensus LS-SVR model is more robust and reliable than the conventional partial least squares (PLS) and LS-SVR methods.     

Determination of benzo[a]pyrene in cigarette mainstream smoke by using mid-infrared spectroscopy associated with a novel chemometric algorithm

Determination of benzo[a]pyrene (BaP) in cigarette smoke can be very important for the tobacco quality control and the assessment of its harm to human health. In this study, mid-infrared spectroscopy (MIR) coupled to chemometric algorithm (DPSO-WPT-PLS), which was based on the wavelet packet transform (WPT), discrete particle swarm optimization algorithm (DPSO) and partial least squares regression (PLS), was used to quantify harmful ingredient benzo[a]pyrene in the cigarette mainstream smoke with promising result. Furthermore, the proposed method provided better performance compared to several other chemometric models, i.e., PLS, radial basis function-based PLS (RBF-PLS), PLS with stepwise regression variable selection (Stepwise-PLS) as well as WPT-PLS with informative wavelet coefficients selected by correlation coefficient test (rtest-WPT-PLS). It can be expected that the proposed strategy could become a new effective, rapid quantitative analysis technique in analyzing the harmful ingredient BaP in cigarette mainstream smoke.     

A novel method to calculate the approximate derivative photoacoustic spectrum using continuous wavelet transform

A novel method based on continuous wavelet transform (CWT) using Haar wavelet function for approximate derivative calculation of analytical signals is proposed and successfully used in processing the photoacoustic signal. An approximate nth derivative of an analytical signal can be obtained by applying n times of the wavelet transform to the signal. The results obtained from four other different methods--the conventional numerical differentiation, the Fourier transform method, the Savitzky-Golay method, and the discrete wavelet transform (DWT) method--were compared with the proposed CWT method; it was demonstrated that all the results are almost the same for signals without noise, but the proposed CWT method is superior to the former four methods for noisy signals. The approximate first and second derivative of the photoacoustic spectrum of Pr(Gly)3Cl3.3H2O and PrCl3.6H2O were obtained using the proposed CWT method; the results are satisfactory.