Data fusion in multivariate calibration transfer

We report the use of stacked partial least-squares regression and stacked dual-domain regression analysis with four commonly used techniques for calibration transfer to improve predictive performance from transferred multivariate calibration models. The predictive performance from three conventional calibration transfer methods, piecewise direct standardization (PDS), orthogonal signal correction (OSC) and model updating (MUP), requiring standards measured on both instruments, was significantly improved from data fusion either by stacking of wavelet scales or by stacking of spectral intervals, as demonstrated by transfer of calibrations developed on near-infrared spectra of synthetic gasoline. Stacking did not produce as significant an improvement for calibration transfer using a finite impulse response (FIR) filter, but application of SPLS regression to FIR-transferred spectra improves predictive performance of the transferred model.     

Elimination of interference information by a new hybrid algorithm for quantitative calibration of near infrared spectra

A new hybrid algorithm is proposed to eliminate the interference information for multivariate calibration of near-infrared (NIR) spectra that includes noise, background and systemic spectral variation irrelevant to concentration. The method consists of two parts: approximate derivative based on continuous wavelet transform (CWT) and orthogonal signal correction (OSC). After the approximate derivative calculated by CWT, OSC was performed. It was successfully applied to real complex NIR spectral data to eliminate the interference information. Correction for the interference of NIR spectra resulted in a substantial improvement in the predicted precision, and a more concise calibration model was obtained. The proposed procedure also compared favourably with several pretreatment methods, and the new method appears to provide a high-performance pretreatment tool for multivariate calibration of NIR spectra. In addition, the strategy proposed here can be applied to various other spectral data for quantitative purposes as well.     

Spectrophotometric simultaneous determination of nitroaniline isomers by orthogonal signal correction-partial least squares

The simultaneous determination of nitroaniline isomer mixtures by using spectrophotometric methods is a difficult problem in analytical chemistry, due to spectral interferences. By multivariate calibration methods, such as partial least squares (PLS), it is possible to obtain a model adjusted to the concentration values of the mixtures used in the calibration range. Orthogonal signal correction (OSC) is a preprocessing technique used for removes the information unrelated to the target variables based on constrained principal component analysis. OSC is a suitable preprocessing method for partial least squares calibration of mixtures without loss of prediction capacity using spectrophotometric method. In this study, the calibration model is based on absorption spectra in the 200–500 nm range for 21 different mixtures of nitroaniline isomers. Calibration matrices were containing 1–21, 1–15 and 1–18 μg ml⁻¹ of m-nitroaniline, o-nitroaniline and p-nitroaniline, respectively. The RMSEP for m-nitroaniline, o-nitroaniline and p-nitroaniline with OSC and without OSC were 0.6567, 0.2692, and 0.3134, and 1.3818, 1.2181, and 0.3953, respectively. This procedure allows the simultaneous determination of nitroaniline isomers in real matrix samples and good reliability of the determination was proved.     

Combining orthogonal signal correction and wavelet packet transform with partial least squares to analyze overlapping spectra of three kinds of metal ions

A novel method named OSC-WPT-PLS approach based on partial least squares (PLS) regression with orthogonal signal correction (OSC) and wavelet packet transform (WPT) as pre-processed tools was proposed for the simultaneous spectrophotometric determination of Al(III), Mn(II) and Co(II). This method combines the ideas of OSC and WPT with PLS regression for enhancing the ability of extracting characteristic information and the quality of regression. OSC is used to remove information in the response matrix D by subtracting the structured noise that is orthogonal to the concentration matrix C. Wavelet packet transform was applied to perform data compression, to extract relevant information, and to eliminate noise and collinearity. PLS was applied for multivariate calibration and noise reduction by eliminating the less important latent variables. In this case, using trials, the kind of wavelet function, the decomposition level, the number of OSC components and the number of PLS factors for the OSC-WPT-PLS method were selected as Daubechies 4, 3, 2 and 3, respectively. A program (POSCWPTPLS) was designed to perform the simultaneous spectrophotometric determination of Al(III), Mn(II) and Co(II). The relative standard errors of prediction (RSEP) obtained for total elements using OSC-WPT-PLS, WPT-PLS and PLS were compared. Experimental results demonstrated that the OSC-WPT-PLS method had the best performance among the three methods and was successful even when there was severe overlap of spectra.     

小波变换结合多维偏最小二乘方法用于近红外光谱定量分析

将小波变换和多维偏最小二乘法相结合用于近红外光谱定量校正模型的建立。首先将原始光谱进行小波变换分解，得到系列小波细节系数，通过选取一组受外界因素少、信息强的小波系数组成三维光谱阵，然后再采用多维偏最小二乘法建立校正模型。实验结果表明，该方法所建近红外校正模捌的预测能力更强，并更具稳健性。     

Resolution of ternary mixtures of salicylic, salicyluric and gentisic acids by partial least squares and principal component regression: Optimization of the scanning path in the excitation-emission matrices

The resolution of ternary mixtures of salicylic, salicyluric and gentisic acids has been accomplished by partial least squares (PLS) and principal component regression (PCR) multivariate calibration. The total luminescence information of the compounds has been used to optimize the spectral data set to perform the calibration. A comparison between the predictive ability of the three multivariate calibration methods, PLS-1, PLS-2 and PCR, on three spectral data sets, excitation, emission and synchronous spectra, has been performed. The excitation spectrum has been the best scanning path for salicylic and salicyluric acid determinations, while the emission spectrum has been the best for the gentisic acid determination. The convenience of analysing the total luminescence spectrum information when using multivariate calibration methods on fluorescence data is demonstrated.     

Simultaneous spectrophotometric determination of cobalt,copper and nickel using 1-（2-thiazolylazo）-2-naphthol by chemometrics methods

The simultaneous determination of cobalt, copper and nickel using 1-（2-thiazolylazo）-2-naphthol （first figure of this article） by spectrophotometric method is a difficult problem in analytical chemistry, due to spectral interferences. By multivariate calibration methods, such as partial least squares （PLS） regression, it is possible to obtain a model adjusted to the concentration values of the mixtures used in the calibration range. Orthogonal signal correction （OSC） is a preprocessing technique used for removing the information unrelated to the target variables based on constrained principal component analysis. OSC is a suitable preprocessing method for PLS calibration of mixtures without loss of prediction capacity using spectrophotometric method. In this study, the calibration model is based on absorption spectra in the 550-750-nm range for 21 different mixtures of cobalt, copper and nickel. Calibration matrices were formed from samples containing 0.05-1.05, 0.05-1.30 and 0.05-0.80 μg·mL^-1 for cobalt, copper and nickel, respectively. The root mean square error of prediction （RMSEP） for cobalt, copper and nickel with OSC and without OSC were 0.007, 0.008, 0.011 and 0.031,0.037, 0.032 μg· mL^-1, respectively. This procedure allows the simultaneous determination of cobalt, copper and nickel in synthetic and real samples and good reliability of the determination was proved.     

小波变换方法的比较──红外光谱数据压缩

介绍了小波变换和多分辨分析的基本理论以及常用小波变换压缩数据的3种方法:(1)只保留模糊信号;(2)全部保留模糊信号及锐化信号中的较大值;(3)保留模糊信号及锐化信号中的较大值.将紧支集小波和正交三次B-样条小波压缩4-苯乙炔基-邻苯二甲酸酐的红外光谱数据进行了对比,计算表明正交三次B-样条小波变换方法效果较好,而在全部保留模糊信号及只保留锐化信号中数值较大的系数时,压缩比大而重建光谱数据与原始光谱数据间的均方差较小.     

Chemometrics: the issues of measurement and modelling

In this paper, two spectral data sets have been used to illustrate the importance of maintaining chemical information whilst generating predictive multivariate calibration models. The first data set is based on 26 duplicate UV/VIS spectra for four meal ions (Fe, Ni, Co, Cu) present at varying concentrations in aqueous solution. Spectra were collected across the range 180–800 nm at a resolution of 3.5 nm generating 211 data points for each sample. Calibration was carried out using multiple linear regression (MLR) and a K-matrix approach to demonstrate the advantages the latter method has in describing real spectral features. In addition, the limitation of MLR in accommodating noise and spectral overlap in the data is also illustrated. The second data set based on NIR spectroscopy, was generated using a four-level 2 factor Factorial design strategy and consisted of two additives present at a range of concentrations in an aqueous caustic system, with the spectra being collected over the range 10,000–3000 cm⁻¹. Whilst a conventional partial least squares (PLS) model was applied to the data, it was through the use of variable selection (VS) prior to PLS and the application of weighted ridge regression (WRR) techniques that the need to develop chemometric methodology which intuitively reflected chemical information has been demonstrated. The results will also illustrate how a poorly designed experimental design protocol and missing data can limit the performance of the calibration models generated. The aims of this paper are not to prescribe ideal calibration methodology but rather to demonstrate the relevance of selecting multivariate calibration methodology that relates more to the chem rather than just the metrics in chemometrics.     

Multivariate calibration of near infrared spectra by orthogonal WAVElet correction using a genetic algorithm

Orthogonal WAVElet correction (OWAVEC) is a pre-processing method aimed at simultaneously accomplishing two essential needs in multivariate calibration, signal correction and data compression, by combining the application of an orthogonal signal correction algorithm to remove information unrelated to a certain response with the great potential that wavelet analysis has shown for signal processing. In the previous version of the OWAVEC method, once the wavelet coefficients matrix had been computed from NIR spectra and deflated from irrelevant information in the orthogonalization step, effective data compression was achieved by selecting those largest correlation/variance wavelet coefficients serving as the basis for the development of a reliable regression model. This paper presents an evolution of the OWAVEC method, maintaining the first two stages in its application procedure (wavelet signal decomposition and direct orthogonalization) intact but incorporating genetic algorithms as a wavelet coefficients selection method to perform data compression and to improve the quality of the regression models developed later. Several specific applications dealing with diverse NIR regression problems are analyzed to evaluate the actual performance of the new OWAVEC method. Results provided by OWAVEC are also compared with those obtained with original data and with other orthogonal signal correction methods.     

Wavelet analysis used for spectral background removal in the determination of glucose from near-infrared single-beam spectra

Wavelet analysis is developed as a preprocessing tool for use in removing background information from near-infrared (near-IR) single-beam spectra before the construction of multivariate calibration models. Three data sets collected with three different near-IR spectrometers are investigated that involve the determination of physiological levels of glucose (1-30 mM) in a simulated biological matrix containing alanine, ascorbate, lactate, triacetin, and urea in phosphate buffer. A factorial design is employed to optimize the specific wavelet function used and the level of decomposition applied, in addition to the spectral range and number of latent variables associated with a partial least-squares calibration model. The prediction performance of the computed models is studied with separate data acquired after the collection of the calibration spectra. This evaluation includes one data set collected over a period of more than 6 months. Preprocessing with wavelet analysis is also compared to the calculation of second-derivative spectra. Over the three data sets evaluated, wavelet analysis is observed to produce better-performing calibration models, with improvements in concentration predictions on the order of 30% being realized relative to models based on either second-derivative spectra or spectra preprocessed with simple additive and multiplicative scaling correction. This methodology allows the construction of stable calibrations directly with single-beam spectra, thereby eliminating the need for the collection of a separate background or reference spectrum.     

Influence of data pre-processing on the quantitative determination of the ash content and lipids in roasted coffee by near infrared spectroscopy

In near-infrared (NIR) measurements, some physical features of the sample can be responsible for effects like light scattering, which lead to systematic variations unrelated to the studied responses. These errors can disturb the robustness and reliability of multivariate calibration models. Several mathematical treatments are usually applied to remove systematic noise in data, being the most common derivation, standard normal variate (SNV) and multiplicative scatter correction (MSC). New mathematical treatments, such as orthogonal signal correction (OSC) and direct orthogonal signal correction (DOSC), have been developed to minimize the variability unrelated to the response in spectral data. In this work, these two new pre-processing methods were applied to a set of roasted coffee NIR spectra. A separate calibration model was developed to quantify the ash content and lipids in roasted coffee samples by PLS regression. The results provided by these correction methods were compared to those obtained with the original data and the data corrected by derivation, SNV and MSC. For both responses, OSC and DOSC treatments gave PLS calibration models with improved prediction abilities (4.9 and 3.3% RMSEP with corrected data versus 7.1 and 8.3% RMSEP with original data, respectively).     

Improved analysis of multivariate data by variable stability scaling: application to NMR-based metabolic profiling

Variable scaling alters the covariance structure of data, affecting the outcome of multivariate analysis and calibration. Here we present a new method, variable stability (VAST) scaling, which weights each variable according to a metric of its stability. The beneficial effect of VAST scaling is demonstrated for a data set of ¹H NMR spectra of urine acquired as part of a metabonomic study into the effects of unilateral nephrectomy in an animal model. The application of VAST scaling improved the class distinction and predictive power of partial least squares discriminant analysis (PLS-DA) models. The effects of other data scaling and pre-processing methods, such as orthogonal signal correction (OSC), were also tested. VAST scaling produced the most robust models in terms of class prediction, outperforming OSC in this aspect. As a result the subtle, but consistent, metabolic perturbation caused by unilateral nephrectomy could be accurately characterised despite the presence of much greater biological differences caused by normal physiological variation. VAST scaling presents itself as an interpretable, robust and easily implemented data treatment for the enhancement of multivariate data analysis.     

Bayesian linear regression and variable selection for spectroscopic calibration

This paper presents a Bayesian approach to the development of spectroscopic calibration models. By formulating the linear regression in a probabilistic framework, a Bayesian linear regression model is derived, and a specific optimization method, i.e. Bayesian evidence approximation, is utilized to estimate the model “hyper-parameters”. The relation of the proposed approach to the calibration models in the literature is discussed, including ridge regression and Gaussian process model. The Bayesian model may be modified for the calibration of multivariate response variables. Furthermore, a variable selection strategy is implemented within the Bayesian framework, the motivation being that the predictive performance may be improved by selecting a subset of the most informative spectral variables. The Bayesian calibration models are applied to two spectroscopic data sets, and they demonstrate improved prediction results in comparison with the benchmark method of partial least squares.     

OWAVEC: a combination of wavelet analysis and an orthogonalization algorithm as a pre-processing step in multivariate calibration

In an spectroscopic context, when a calibration model based on partial least squares is developed to predict a response, it is often the case that a high percentage of variation in the data explained by the first latent variable is not accompanied by an equally high percentage of variation in the studied response. The addition of more components can slowly improve the calibration model, but with negative effects on the robustness and interpretability of the final model. To solve this problem, several pre-processing methods have been proposed to remove only a portion unrelated to the studied response from the spectral matrix.Moreover, the need for efficient compression methods is increasingly important due to the large size of the data currently collected. In this sense, discrete wavelet transform has proven that it can achieve good compression without losing relevant information when used on individual signals.This paper introduces a new pre-processing method, orthogonal wavelet correction (OWAVEC) that tries to lump together two important needs in multivariate calibration: signal correction and compression. The new method has been tested on a set of diesel fuels using viscosity as variable response, and its results have been compared not only with those obtained from original data but also with those provided by other correction methods. The first practical results are encouraging, as the method generates considerably better calibration models compared to the model developed from raw data and provides results as least so good as other orthogonal correction methods.     

Variable-weighted least-squares support vector machine for multivariate spectral analysis

Multivariate spectral analysis has been widely applied in chemistry and other fields. Spectral data consisting of measurements at hundreds and even thousands of analytical channels can now be obtained in a few seconds. It is widely accepted that before a multivariate regression model is built, a well-performed variable selection can be helpful to improve the predictive ability of the model. In this paper, the concept of traditional wavelength variable selection has been extended and the idea of variable weighting is incorporated into least-squares support vector machine (LS-SVM). A recently proposed global optimization method, particle swarm optimization (PSO) algorithm is used to search for the weights of variables and the hyper-parameters involved in LS-SVM optimizing the training of a calibration set and the prediction of an independent validation set. All the computation process of this method is automatic. Two real data sets are investigated and the results are compared those of PLS, uninformative variable elimination-PLS (UVE-PLS) and LS-SVM models to demonstrate the advantages of the proposed method.     

Orthogonal signal correction-partial least squares method for simultaneous spectrophotometric determination of cypermethrin and tetramethrin

The simultaneous determination of cypermethrin and tetramethrin mixtures by using spectrophotometric method is a difficult problem in analytical chemistry, due to spectral interferences. By multivariate calibration methods, such as partial least squares (PLS) regression, it is possible to obtain a model adjusted to the concentration values of the mixtures used in the calibration range. Orthogonal signal correction (OSC) is a preprocessing technique used for removing the information unrelated to the target variables based on constrained principal component analysis. OSC is a suitable preprocessing method for partial least squares calibration of mixtures without loss of prediction capacity using spectrophotometric method. In this study, the calibration model is based on absorption spectra in the 200-350 nm range for 25 different mixtures of cypermethrin and tetramethrin. Calibration matrices were containing 0.1-12.9 and 0.1-13.8 microg mL(-1) for cypermethrin and tetramethrin, respectively. The RMSEP for cypermethrin and tetramethrin with OSC and without OSC were 0.0884, 0.0614 and 0.2915, 0.2309, respectively. This procedure allows the simultaneous determination of cypermethrin and tetramethrin in synthetic and real samples good reliability of the determination was proved.     

Wavelet multiscale regression from the perspective of data fusion: new conceptual approaches

Wavelet regression is a very promising technique for modern multivariate calibration and calibration transfer. Multiscale analysis of wavelet scales provides a connection between wavelet regression and data fusion. In this paper, current wavelet regression methods are reviewed from the novel perspective of data fusion. Illustrated by analysis of a public domain near-infrared dataset, the advantages and drawbacks of these methods are examined. For wavelet regression, the non-uniformity of the wavelet components, the multiscale nature of the signal, and the prevention of information leakage are crucial issues that will be addressed.     

小波分析在化学中的应用进展

小波分析是信息处理的良好工具, 本文就其在化学中改善信号质量、图谱数据压缩、结构形态、模式识别、预测预报、过程控制、奇异诊断、动力学、量子化学和分形等方面的应用作了简要评述。     

Construction of stable multivariate calibration models using unsupervised segmented principal component regression

In multivariate spectral calibration by principal component regression (PCR), the principal components (PCs) are calculated from the response data measured at all employed instrument channels; however some channels are redundant and their responses do not possess useful information. Thus, the extracted PCs possess mixed information from both useful and redundant channels. In this work, we propose a segmentation approach based on unsupervised pattern recognition to identify the most informative spectral region and then to construct a stable multivariate calibration model by PCR. In this method, the instrument channels are clustered into different segments via Kohonen self‐organization map. The spectral data of each segment are then subjected to PCA and the derived PCs are used as input variables for an inverse least square (ILS) regression model employing stepwise selection of the informative PCs. The proposed method was evaluated by the analysis of four simulated and six experimental data sets. It was found that our proposed method can model the above data sets with prediction errors lower than conventional partial least squares (PLS) and PCR methods. In addition, the prediction ability of our method was better than the previously reported models for these data sets. Copyright © 2011 John Wiley & Sons, Ltd.