Multivariate calibration of near infrared spectra by orthogonal WAVElet correction using a genetic algorithm

Orthogonal WAVElet correction (OWAVEC) is a pre-processing method aimed at simultaneously accomplishing two essential needs in multivariate calibration, signal correction and data compression, by combining the application of an orthogonal signal correction algorithm to remove information unrelated to a certain response with the great potential that wavelet analysis has shown for signal processing. In the previous version of the OWAVEC method, once the wavelet coefficients matrix had been computed from NIR spectra and deflated from irrelevant information in the orthogonalization step, effective data compression was achieved by selecting those largest correlation/variance wavelet coefficients serving as the basis for the development of a reliable regression model. This paper presents an evolution of the OWAVEC method, maintaining the first two stages in its application procedure (wavelet signal decomposition and direct orthogonalization) intact but incorporating genetic algorithms as a wavelet coefficients selection method to perform data compression and to improve the quality of the regression models developed later. Several specific applications dealing with diverse NIR regression problems are analyzed to evaluate the actual performance of the new OWAVEC method. Results provided by OWAVEC are also compared with those obtained with original data and with other orthogonal signal correction methods.     

Orthogonal signal correction, wavelet analysis, and multivariate calibration of complicated process fluorescence data

In this paper, multivariate calibration of complicated process fluorescence data is presented. Two data sets related to the production of white sugar are investigated. The first data set comprises 106 observations and 571 spectral variables, and the second data set 268 observations and 3997 spectral variables. In both applications, a single response, ash content, is modelled and predicted as a function of the spectral variables. Both data sets contain certain features making multivariate calibration efforts non-trivial. The objective is to show how principal component analysis (PCA) and partial least squares (PLS) regression can be used to overview the data sets and to establish predictively sound regression models. It is shown how a recently developed technique for signal filtering, orthogonal signal correction (OSC), can be applied in multivariate calibration to enhance predictive power. In addition, signal compression is tested on the larger data set using wavelet analysis. It is demonstrated that a compression down to 4% of the original matrix size — in the variable direction — is possible without loss of predictive power. It is concluded that the combination of OSC for pre-processing and wavelet analysis for compression of spectral data is promising for future use.     

Elimination of interference information by a new hybrid algorithm for quantitative calibration of near infrared spectra

A new hybrid algorithm is proposed to eliminate the interference information for multivariate calibration of near-infrared (NIR) spectra that includes noise, background and systemic spectral variation irrelevant to concentration. The method consists of two parts: approximate derivative based on continuous wavelet transform (CWT) and orthogonal signal correction (OSC). After the approximate derivative calculated by CWT, OSC was performed. It was successfully applied to real complex NIR spectral data to eliminate the interference information. Correction for the interference of NIR spectra resulted in a substantial improvement in the predicted precision, and a more concise calibration model was obtained. The proposed procedure also compared favourably with several pretreatment methods, and the new method appears to provide a high-performance pretreatment tool for multivariate calibration of NIR spectra. In addition, the strategy proposed here can be applied to various other spectral data for quantitative purposes as well.     

Combining orthogonal signal correction and wavelet packet transform with partial least squares to analyze overlapping spectra of three kinds of metal ions

A novel method named OSC-WPT-PLS approach based on partial least squares (PLS) regression with orthogonal signal correction (OSC) and wavelet packet transform (WPT) as pre-processed tools was proposed for the simultaneous spectrophotometric determination of Al(III), Mn(II) and Co(II). This method combines the ideas of OSC and WPT with PLS regression for enhancing the ability of extracting characteristic information and the quality of regression. OSC is used to remove information in the response matrix D by subtracting the structured noise that is orthogonal to the concentration matrix C. Wavelet packet transform was applied to perform data compression, to extract relevant information, and to eliminate noise and collinearity. PLS was applied for multivariate calibration and noise reduction by eliminating the less important latent variables. In this case, using trials, the kind of wavelet function, the decomposition level, the number of OSC components and the number of PLS factors for the OSC-WPT-PLS method were selected as Daubechies 4, 3, 2 and 3, respectively. A program (POSCWPTPLS) was designed to perform the simultaneous spectrophotometric determination of Al(III), Mn(II) and Co(II). The relative standard errors of prediction (RSEP) obtained for total elements using OSC-WPT-PLS, WPT-PLS and PLS were compared. Experimental results demonstrated that the OSC-WPT-PLS method had the best performance among the three methods and was successful even when there was severe overlap of spectra.     

Data fusion in multivariate calibration transfer

We report the use of stacked partial least-squares regression and stacked dual-domain regression analysis with four commonly used techniques for calibration transfer to improve predictive performance from transferred multivariate calibration models. The predictive performance from three conventional calibration transfer methods, piecewise direct standardization (PDS), orthogonal signal correction (OSC) and model updating (MUP), requiring standards measured on both instruments, was significantly improved from data fusion either by stacking of wavelet scales or by stacking of spectral intervals, as demonstrated by transfer of calibrations developed on near-infrared spectra of synthetic gasoline. Stacking did not produce as significant an improvement for calibration transfer using a finite impulse response (FIR) filter, but application of SPLS regression to FIR-transferred spectra improves predictive performance of the transferred model.     

Orthogonal projection to latent structures combined with artificial neural network for quantitative analysis of phenoxymethylpenicillin potassium powder

A method for quantitative analysis of phenoxymethylpenicillin potassium powder on the basis of near-infrared (NIR) spectroscopy is investigated by using orthogonal projection to latent structures (O-PLS) combined with artificial neural network (ANN). Being a preprocessing method, O-PLS can remove systematic orthogonal variation from a given data set X without disturbing the correlation between X and the response set y. In this paper, O-PLS method was applied to preprocess the original spectral data of phenoxymethylpenicillin potassium powder, and the filtered data was used to establish the ANN model. In this model, the concentration of phenoxymethylpenicillin potassium as the active component was determined. The degree of approximation was employed as the selective criterion of the optimum network parameters. In order to compare with O-PLS-ANN model, the calibration models that use the original spectra and different preprocessing methods (first-derivative, second-derivative, standard normal variate (SNV) and multiplicative scatter correction (MSC)) of the spectra were also designed. Experimental results show that O-PLS-ANN model is the best.     

Improved analysis of multivariate data by variable stability scaling: application to NMR-based metabolic profiling

Variable scaling alters the covariance structure of data, affecting the outcome of multivariate analysis and calibration. Here we present a new method, variable stability (VAST) scaling, which weights each variable according to a metric of its stability. The beneficial effect of VAST scaling is demonstrated for a data set of ¹H NMR spectra of urine acquired as part of a metabonomic study into the effects of unilateral nephrectomy in an animal model. The application of VAST scaling improved the class distinction and predictive power of partial least squares discriminant analysis (PLS-DA) models. The effects of other data scaling and pre-processing methods, such as orthogonal signal correction (OSC), were also tested. VAST scaling produced the most robust models in terms of class prediction, outperforming OSC in this aspect. As a result the subtle, but consistent, metabolic perturbation caused by unilateral nephrectomy could be accurately characterised despite the presence of much greater biological differences caused by normal physiological variation. VAST scaling presents itself as an interpretable, robust and easily implemented data treatment for the enhancement of multivariate data analysis.     

A method for near-infrared spectral calibration of complex plant samples with wavelet transform and elimination of uninformative variables

An algorithm is proposed for extracting relevant information from near-infrared (NIR) spectra for multivariate calibration of routine components in complex plant samples. The algorithm is a combination of wavelet transform (WT) data compression and a procedure for uninformative variable elimination (UVE). After compression of the NIR spectra by WT, the UVE approach is used to eliminate the irrelevant wavelet coefficients. Finally, a calibration model is built from the retained wavelet coefficients to enable prediction. Because irrelevant information can be removed from the spectra used for multivariate calibration, the model based on the extracted relevant features is better than those obtained with full-spectrum data. Both prediction precision and calculation speed are improved.     

Influence of data pre-processing on the quantitative determination of the ash content and lipids in roasted coffee by near infrared spectroscopy

In near-infrared (NIR) measurements, some physical features of the sample can be responsible for effects like light scattering, which lead to systematic variations unrelated to the studied responses. These errors can disturb the robustness and reliability of multivariate calibration models. Several mathematical treatments are usually applied to remove systematic noise in data, being the most common derivation, standard normal variate (SNV) and multiplicative scatter correction (MSC). New mathematical treatments, such as orthogonal signal correction (OSC) and direct orthogonal signal correction (DOSC), have been developed to minimize the variability unrelated to the response in spectral data. In this work, these two new pre-processing methods were applied to a set of roasted coffee NIR spectra. A separate calibration model was developed to quantify the ash content and lipids in roasted coffee samples by PLS regression. The results provided by these correction methods were compared to those obtained with the original data and the data corrected by derivation, SNV and MSC. For both responses, OSC and DOSC treatments gave PLS calibration models with improved prediction abilities (4.9 and 3.3% RMSEP with corrected data versus 7.1 and 8.3% RMSEP with original data, respectively).     

小波变换结合多维偏最小二乘方法用于近红外光谱定量分析

将小波变换和多维偏最小二乘法相结合用于近红外光谱定量校正模型的建立。首先将原始光谱进行小波变换分解，得到系列小波细节系数，通过选取一组受外界因素少、信息强的小波系数组成三维光谱阵，然后再采用多维偏最小二乘法建立校正模型。实验结果表明，该方法所建近红外校正模捌的预测能力更强，并更具稳健性。     

小波变换方法的比较──红外光谱数据压缩

介绍了小波变换和多分辨分析的基本理论以及常用小波变换压缩数据的3种方法:(1)只保留模糊信号;(2)全部保留模糊信号及锐化信号中的较大值;(3)保留模糊信号及锐化信号中的较大值.将紧支集小波和正交三次B-样条小波压缩4-苯乙炔基-邻苯二甲酸酐的红外光谱数据进行了对比,计算表明正交三次B-样条小波变换方法效果较好,而在全部保留模糊信号及只保留锐化信号中数值较大的系数时,压缩比大而重建光谱数据与原始光谱数据间的均方差较小.     

Method comparison on estimation for an unknown specimen in a heteroscedastic controlled calibration model

In analytical chemistry applications, statistical calibration models are commonly used to estimate the true value of an unknown specimen. In this article, we consider a heteroscedastic controlled calibration model in which both dependent and independent variables are subject to heteroscedastic measurement errors. The main task of using this model is to estimate the true value of an unknown regressor (independent variable) under the condition that a set of observations on its corresponding response (dependent variable) is available. We introduce four estimation methods to the problem of interest, including generalized least squares (GLS), modified least squares, corrected score, and expectation maximization‐based (EM‐based) methods. Furthermore, an interval estimation based on an asymptotic method is also derived. We compare their performance through detailed simulation studies. In consequence, GLS and EM‐based methods are recommended in practical use. A real data example is given to illustrate the application of the calibration model. Copyright © 2013 John Wiley & Sons, Ltd.     

Application of artificial neural network to determination of active principle ingredient in pharmaceutical quality control based on near infrared spectroscopy

In the construction of a neural network, most attentions have been paid to the selection of the architecture, the selection of the learning parameters and the network validation while the selection of input variables shared little. This study focused on the selection of input variables by various data pre-treatment for constructing ANN models. The results showed that the validation results differed from each other when different data-pretreatment methods combined with near-infrared spectroscopy (NIRS) to build a model using artificial neural network (ANN) for quality control of paracetamol in coldrex. And wavelet coefficients after orthogonal signal correction (OSC) in the ANN models reduced RMSEP by up to 77% compared to ANN models using derivatives combined with PCA pretreatment. The selection of input variables has potent to improve the calibration ability of ANN, and the model can be used for pressure reduction of quality control in the pharmaceutical industry.     

Interrelationships between generalized Tikhonov regularization,generalized net analyte signal,and generalized least squares for desensitizing a multivariate calibration to interferences

Orthogonal pre‐processing (orthogonal projection) of spectral data is a common approach to generate analyte‐specific information for use in multivariate calibration. The goal of this pre‐processing is to remove from each spectrum the respective sample interferent contributions (spectral interferences from overlap, scatter, noise, etc.). Two approaches to accomplish orthogonal pre‐processing are net analyte signal (NAS) and generalized least squares (GLS). Developed in this paper is the mathematical relationship between NAS and GLS. It is also realized that orthogonal NAS pre‐processing can remove too much analyte signal and that the degree of interferent correction can be regulated. Similar to GLS, the degree of correction is accomplished by using a regularization (tuning) parameter to form generalized NAS (GNAS). Also developed in this paper is an alternative to GNAS and GLS based on generalized Tikhonov regularization (GTR). The mathematical relationships between GTR, GNAS, and GLS are derived. A result is the ability to express the model vector as the sum of two contributions: the orthogonal NAS contribution and a non‐NAS contribution from the interferent components. Thus, rather than the usual situation of sequentially pre‐processing data by either GNAS or GLS followed by model building with the pre‐processed data, the methods of GTR, GNAS, and GLS are expressed as direct computations of model vectors allowing concurrent pre‐processing and model building to occur. Simultaneous pre‐processing and model forming are shown to be natural to the GTR process. Two near‐infrared spectroscopic data sets are studied to compare the theoretical relationships between GTR, GNAS, and GLS. One data set covers basic calibration, and the other data set is for calibration maintenance. Filter factor representation is key to developing the interprocess relationships. Copyright © 2013 John Wiley & Sons, Ltd.     

Detecting the quality of glycerol monolaurate: A method for using Fourier transform infrared spectroscopy with wavelet transform and modified uninformative variable elimination

Glycerol monolaurate (GML) products contain many impurities, such as lauric acid and glucerol. The GML content is an important quality indicator for GML production. A hybrid variable selection algorithm, which is a combination of wavelet transform (WT) technology and modified uninformative variable eliminate (MUVE) method, was proposed to extract useful information from Fourier transform infrared (FT-IR) transmission spectroscopy for the determination of GML content. FT-IR spectra data were compressed by WT first; the irrelevant variables in the compressed wavelet coefficients were eliminated by MUVE. In the MUVE process, simulated annealing (SA) algorithm was employed to search the optimal cutoff threshold. After the WT-MUVE process, variables for the calibration model were reduced from 7366 to 163. Finally, the retained variables were employed as inputs of partial least squares (PLS) model to build the calibration model. For the prediction set, the correlation coefficient (r) of 0.9910 and root mean square error of prediction (RMSEP) of 4.8617 were obtained. The prediction result was better than the PLS model with full-spectra data. It was indicated that proposed WT-MUVE method could not only make the prediction more accurate, but also make the calibration model more parsimonious. Furthermore, the reconstructed spectra represented the projection of the selected wavelet coefficients into the original domain, affording the chemical interpretation of the predicted results. It is concluded that the FT-IR transmission spectroscopy technique with the proposed method is promising for the fast detection of GML content.     

Orthogonal projection to latent structures solution properties for chemometrics and systems biology data

Partial least squares (PLS) is a widely used algorithm in the field of chemometrics. In calibration studies, a PLS variant called orthogonal projection to latent structures (O‐PLS) has been shown to successfully reduce the number of model components while maintaining good prediction accuracy, although no theoretical analysis exists demonstrating its applicability in this context. Using a discrete formulation of the linear mixture model known as Beer's law, we explicitly analyze O‐PLS solution properties for calibration data. We find that, in the absence of noise and for large n, O‐PLS solutions are simpler but just as accurate as PLS solutions for systems in which analyte and background concentrations are uncorrelated. However, the same is not true for the most general chemometric data in which correlations between the analyte and background concentrations are nonzero and pure profiles overlap. On the contrary, forcing the removal of orthogonal components may actually degrade interpretability of the model. This situation can also arise when the data are noisy and n is small, because O‐PLS may identify and model the noise as orthogonal when it is statistically uncorrelated with the analytes. For the types of data arising from systems biology studies, in which the number of response variables may be much greater than the number of observations, we show that O‐PLS is unlikely to discover orthogonal variation whether or not it exists. In this case, O‐PLS and PLS solutions are the same. Copyright © 2011 John Wiley & Sons, Ltd.     

基于正交信号校正的秸秆青贮饲料粗蛋白近红外分析模型传递方法

探讨了基于不同数据预处理方法的正交信号校正在秸杆饲料近红外光谱模型传递中的应用.以141个秸杆青贮饲料样品为研究对象,以其粗蛋白含量为目标参数,研究了基于无处理、局部中心化、全局中心化和Z-score标准化预处理方法的正交信号校正,在源仪器(SPECTRUM ONE NTS)和目标仪器1(ANTA-RIS)与目标仪器2(FOSS 6500)之间的模型传递效果.实验表明:对于两台傅里叶变换型近红外光谱仪,采用局部中心化、全局中心化和Z-score标准化预处理方法的正交信号校正均可成功实现模型传递,其中局部中心化和全局中心化法的作用效果基本一致,且优于Z-score标准化法.对于傅立叶变换和光栅型近红外光谱仪,全局中心化的作用效果明显优于其它3组处理效果,且只有全局中心化预处理的正交信号校正传递后的模型可用于实际预测.     

Rapid and Nondestructive Analysis of Bacillus Calmette–Guerin Polysaccharide Nucleic Acid Injection by near-Infrared Spectroscopy with Chemometrics

A novel quantitative analytical method using near-infrared (NIR) spectroscopy combined with chemometrics has been developed to determine the polysaccharides and nucleic acids for routine quality analysis of Bacillus Calmette–Guerin polysaccharide and nucleic acid injections. A Monte-Carlo method was used to detect and discard outliers and to improve the predictive ability of the model. Various other spectral preprocessing methods such as smoothing, derivative, multiplicative scattering correction, standard normal variables, and orthogonal signal correction methods were used to remove noise and other irrelevant information from the spectra. Sample-set partitioning based on joint x–y distance method was utilized to divide the sample measurements into calibration and validation datasets. The optimal wavelength variables were determined by competitive adaptive weighted sampling. The model was established and cross-validated using partial least square regression. The root mean square errors of cross-validation for polysaccharides and nucleic acids were determined to be 0.0382 and 5.218, and the root mean square errors of prediction were 0.0229 and 6.282. The overall results show that NIR spectroscopy combined with chemometry is effective for the quantitative analysis of Bacillus Calmette–Guerin polysaccharide and nucleic acid injections.     

Application of orthogonal space regression to calibration transfer without standards

To transfer a calibration model in cases where the standardization samples are rare or unstable, a method based on orthogonal space regression (OSR) is proposed. It uses virtual standardization spectra to account for response changes between instruments or batches. A comparative study of the proposed OSR, piecewise direct standardization, finite impulse response, orthogonal signal correction, and model updating (MU) was conducted on both pharmaceutical tablet data and chlorogenic acid data. The results of these studies suggest that both the OSR and the MU are superior to the other transfer techniques in terms of root‐mean‐squared error of prediction and ratio of performance to interquartile distance. Moreover, OSR requires no identical standard samples, and it avoids re‐optimizing the transfer models. In conclusion, both the differences among spectra measured on different spectrometers and the differences between different batches can be corrected successfully using the OSR method. Copyright © 2013 John Wiley & Sons, Ltd.     

Simultaneous determination of cobalt, nickel and palladium in micellar media using partial least square regression and direct orthogonal signal correction

A simple, novel and sensitive spectrophotometric method was described for the simultaneous determination of cobalt, nickel and palladium. The method is based on the complex formation of Co, Ni and Pd with 1-(2-pyridylazo)-2-naphtol (PAN) in Tween-80 micellar media. All factors affecting on the sensitivity were optimized and the linear dynamic range for determination of Co, Ni and Pd was found. The experimental calibration matrix was designed by measuring the absorbance over the range of 520-700 nm for 21 samples of 0.10-1.0, 0.050-0.50 and 0.050-4.0 microg ml(-1) of Co, Ni and Pd, respectively. The partial least square (PLS) modeling based on singular value decomposition (SVD) was used for the multivariate calibration of the spectrophotometric data. The direct orthogonal signal correction was used for pre-processing of data matrices and the prediction results of model, with and without using direct orthogonal signal correction, were statistically compared. The effects of various anions and cations on selectivity of the method were investigated. The proposed method was successfully applied to the determination of Co, Ni and Pd in water and alloy samples.