Maintaining the predictive abilities of multivariate calibration models by spectral space transformation

In quantitative on-line/in-line monitoring of chemical and bio-chemical processes using spectroscopic instruments, multivariate calibration models are indispensable for the extraction of chemical information from complex spectroscopic measurements. The development of reliable multivariate calibration models is generally time-consuming and costly. Therefore, once a reliable multivariate calibration model is established, it is expected to be used for an extended period. However, any change in the instrumental response or variations in the measurement conditions can render a multivariate calibration model invalid. In this contribution, a new method, spectral space transformation (SST), has been developed to maintain the predictive abilities of multivariate calibration models when the spectrometer or measurement conditions are altered. SST tries to eliminate the spectral differences induced by the changes in instruments or measurement conditions through the transformation between two spectral spaces spanned by the corresponding spectra of a subset of standardization samples measured on two instruments or under two sets of experimental conditions. The performance of the method has been tested on two data sets comprising NIR and MIR spectra. The experimental results show that SST can achieve satisfactory analyte predictions from spectroscopic measurements subject to spectrometer/probe alteration, when only a few standardization samples are used. Compared with the existing popular methods designed for the same purpose, i.e. global PLS, univariate slope and bias correction (SBC) and piecewise direct standardization (PDS), SST has the advantages of implementation simplicity, wider applicability and better performance in terms of predictive accuracy.     

Standardization of a multivariate calibration model applied to the determination of chromium in tanning sewage

A multivariate standardization procedure was used to extend the lifetime of a multivariate partial least squares (PLS) calibration model for determining chromium in tanning sewage. The Kennard/Stone algorithm was used to select the transfer samples and the F-test was used to decide whether slope/bias correction (SBC) or piecewise direct standardization (PDS) had to be applied. Special attention was paid to the transfer samples since the process can be invalidated if samples are selected which behave anomalously. The results of the F-test were extremely sensitive to heterogeneity in the transfer set. In these cases, it should be taken as an interpretation tool.     

Standardization of SPE signals in multicomponent analysis of three benzimidazolic pesticides by spectrofluorimetry

Piecewise direct standardization (PDS) is applied to multivariate standardization of fluorescence signals using partial least squares (PLS) and principal component regression (PCR) as the calibration models. The multivariate standardization was used to transfer spectra obtained after a step of solid phase extraction (SPE) to spectra registered in pure solvent in the determination of carbendazim, fuberidazole and thiabendazole in water samples. The influential parameters, such as tolerance, window size and the number of samples of the standardization subset were optimized by means of the root mean squared error of prediction (RMSEP). Similar RMSEP values were obtained by PLS and PCR using the optimized influential parameters in the standardization. However, better predictions of the compounds were obtained in test set by the PLS model.     

Near-infrared calibration transfer based on spectral regression

A calibration transfer method for near-infrared (NIR) spectra based on spectral regression is proposed. Spectral regression method can reveal low dimensional manifold structure in high dimensional spectroscopic data and is suitable to transfer the NIR spectra of different instruments. A comparative study of the proposed method and piecewise direct standardization (PDS) for standardization on two benchmark NIR data sets is presented. Experimental results show that spectral regression method outperforms PDS and is quite competitive with PDS with background correction. When the standardization subset has sufficient samples, spectral regression method exhibits excellent performance.     

Transfer of multivariate regression models between high‐resolution NMR instruments: application to authenticity control of sunflower lecithin

In recent years the number of spectroscopic studies utilizing multivariate techniques and involving different laboratories has been dramatically increased. In this paper the protocol for calibration transfer of partial least square regression model between high‐resolution nuclear magnetic resonance (NMR) spectrometers of different frequencies and equipped with different probes was established. As the test system previously published quantitative model to predict the concentration of blended soy species in sunflower lecithin was used. For multivariate modelling piecewise direct standardization (PDS), direct standardization, and hybrid calibration were employed. PDS showed the best performance for estimating lecithin falsification regarding its vegetable origin resulting in a significant decrease in root mean square error of prediction from 5.0 to 7.3% without standardization to 2.9–3.2% for PDS. Acceptable calibration transfer model was obtained by direct standardization, but this standardization approach introduces unfavourable noise to the spectral data. Hybrid calibration is least recommended for high‐resolution NMR data. The sensitivity of instrument transfer methods with respect to the type of spectrometer, the number of samples and the subset selection was also discussed. The study showed the necessity of applying a proper standardization procedure in cases when multivariate model has to be applied to the spectra recorded on a secondary NMR spectrometer even with the same magnetic field strength. Copyright © 2016 John Wiley & Sons, Ltd.     

A comparative study of calibration transfer methods for determination of gasoline quality parameters in three different near infrared spectrometers

This work presents a comparative study of calibration transfer among three near infrared spectrometers for determination of naphthenes and RON (Research Octane Number) in gasoline. Seven transfer methods are compared: direct standardization (DS), piecewise direct standardization (PDS), orthogonal signal correction (OSC), reverse standardization (RS), piecewise reverse standardization (PRS), slope and bias correction (SBC) and model updating (MU). Two pre-treatment procedures, namely standard normal variate (SNV) and multiplicative scatter correction (MSC), are also investigated. The choice of an appropriate number of transfer samples for each technique, as well as the effect of window size in PDS/PRS and OSC components, are discussed. A broad set of gasoline samples representative of the Northeastern states of Brazil is employed in the investigation. The results show that the use of calibration transfer yields prediction errors comparable to those obtained with complete recalibration of the secondary instrument. Overall, the results point to RS as the best method for the analytical problem under consideration. When storage and/or physical transportation of transfer samples are impractical, MU is more appropriate. The comprehensive investigation carried out in the present work will be of value for practitioners involved in networks of fuel monitoring.     

Efficient use of pure component and interferent spectra in multivariate calibration

Partial Least Squares (PLS) is by far the most popular regression method for building multivariate calibration models for spectroscopic data. However, the success of the conventional PLS approach depends on the availability of a ‘representative data set’ as the model needs to be trained for all expected variation at the prediction stage. When the concentration of the known interferents and their correlation with the analyte of interest change in a fashion which is not covered in the calibration set, the predictive performance of inverse calibration approaches such as conventional PLS can deteriorate. This underscores the need for calibration methods that are capable of building multivariate calibration models which can be robustified against the unexpected variation in the concentrations and the correlations of the known interferents in the test set. Several methods incorporating ‘a priori’ information such as pure component spectra of the analyte of interest and/or the known interferents have been proposed to build more robust calibration models. In the present study, four such calibration techniques have been benchmarked on two data sets with respect to their predictive ability and robustness: Net Analyte Preprocessing (NAP), Improved Direct Calibration (IDC), Science Based Calibration (SBC) and Augmented Classical Least Squares (ACLS) Calibration. For both data sets, the alternative calibration techniques were found to give good prediction performance even when the interferent structure in the test set was different from the one in the calibration set. The best results were obtained by the ACLS model incorporating both the pure component spectra of the analyte of interest and the interferents, resulting in a reduction of the RMSEP by a factor 3 compared to conventional PLS for the situation when the test set had a different interferent structure than the one in the calibration set.     

Modelling the background current with partial least squares regression and transference of the calibration models in the simultaneous determination of Tl and Pb by stripping voltammetry

With the aim of carrying out a calibration transfer for routine analysis, partial least squares (PLS) regression was successfully applied to simultaneously determine thallium and lead by stripping voltammetry when an interfering background current is present. The presence of a significant blank signal that overlaps the thallium peak, together with the overlapping thallium and lead signals were both suitably modelled by this multivariate regression technique. Moreover, once the PLS models are built, the piecewise direct standardization (PDS) method can be used to transfer these models over time in such a way that the number of calibration samples that will be needed in future determinations is reduced from 25 to 9, without a loss of quality in the analyses. The mean of the relative errors (in absolute values) obtained for thallium and lead is below 4.94% and 3.19%, respectively.     

Improving the performance of hollow waveguide-based infrared gas sensors via tailored chemometrics

The use of chemometrics in order to improve the molecular selectivity of infrared (IR) spectra has been evaluated using classic least squares (CLS), partial least squares (PLS), science-based calibration (SBC), and multivariate curve resolution-alternate least squares (MCR-ALS) techniques for improving the discriminatory and quantitative performance of infrared hollow waveguide gas sensors. Spectra of mixtures of isobutylene, methane, carbon dioxide, butane, and cyclopropane were recorded, analyzed, and validated for optimizing the prediction of associated concentrations. PLS, CLS, and SBC provided equivalent results in the absence of interferences. After addition of the spectral characteristics of water by humidifying the sample mixtures, CLS and SBC results were similar to those obtained by PLS only if the water spectrum was included in the calibration model. In the presence of an unknown interferant, CLS revealed errors up to six times higher than those obtained by PLS. However, SBC provided similar results compared to PLS by adding a measured noise matrix to the model. Using MCR-ALS provided an excellent estimation of the spectra of the unknown interference. Furthermore, this method also provided a qualitative and quantitative estimation of the components of an unknown set of samples. In summary, using the most suitable chemometrics approach could improve the selectivity and quality of the calibration model derived for a sensor system, and may avoid the need to analyze expensive calibration data sets. The results obtained in the present study demonstrated that (1) if all sample components of the system are known, CLS provides a sufficiently accurate solution; (2) the selection between PLS and SBC methods depends on whether it is easier to measure a calibration data set or a noise matrix; and (3) MCR-ALS appears to be the most suitable method for detecting interferences within a sample. However, the latter approach requires the most extensive calculations and may thus result in limited temporal resolution, if the concentration of a component should be continuously monitored.     

Data fusion in multivariate calibration transfer

We report the use of stacked partial least-squares regression and stacked dual-domain regression analysis with four commonly used techniques for calibration transfer to improve predictive performance from transferred multivariate calibration models. The predictive performance from three conventional calibration transfer methods, piecewise direct standardization (PDS), orthogonal signal correction (OSC) and model updating (MUP), requiring standards measured on both instruments, was significantly improved from data fusion either by stacking of wavelet scales or by stacking of spectral intervals, as demonstrated by transfer of calibrations developed on near-infrared spectra of synthetic gasoline. Stacking did not produce as significant an improvement for calibration transfer using a finite impulse response (FIR) filter, but application of SPLS regression to FIR-transferred spectra improves predictive performance of the transferred model.     

Influence of data pre-processing on the quantitative determination of the ash content and lipids in roasted coffee by near infrared spectroscopy

In near-infrared (NIR) measurements, some physical features of the sample can be responsible for effects like light scattering, which lead to systematic variations unrelated to the studied responses. These errors can disturb the robustness and reliability of multivariate calibration models. Several mathematical treatments are usually applied to remove systematic noise in data, being the most common derivation, standard normal variate (SNV) and multiplicative scatter correction (MSC). New mathematical treatments, such as orthogonal signal correction (OSC) and direct orthogonal signal correction (DOSC), have been developed to minimize the variability unrelated to the response in spectral data. In this work, these two new pre-processing methods were applied to a set of roasted coffee NIR spectra. A separate calibration model was developed to quantify the ash content and lipids in roasted coffee samples by PLS regression. The results provided by these correction methods were compared to those obtained with the original data and the data corrected by derivation, SNV and MSC. For both responses, OSC and DOSC treatments gave PLS calibration models with improved prediction abilities (4.9 and 3.3% RMSEP with corrected data versus 7.1 and 8.3% RMSEP with original data, respectively).     

Development of variance‐filtered instrumental transfer methods for high‐resolution NMR spectroscopy

The introduction of a variance‐filter to both direct standardization (DS) and piece‐wise direct standardization (PDS) instrumental transfer methods for the analysis of NMR spectral data is described. The variance‐filter modification allows for the identification of regions in the NMR spectra that are not adequately represented by the limited number of transfer calibration samples used during the calculation of the instrument‐to‐instrument transfer matrix. For these spectral frequencies, the corresponding portion of the transfer matrix is replaced by identity (or scaled identity) prior to the secondary instrumental data sets being transferred to the target instrument response. The spectral matching performance of the variance‐filtered instrumental transfer method as applied to high‐resolution ¹H NMR spectra is presented along with possible uses and limitations. Copyright © 2010 John Wiley & Sons, Ltd.     

Improving the linearity of spectroscopic data subjected to fluctuations in external variables by the extended loading space standardization

In process analytical applications, spectral measurements can be subject to changes in process temperature, pressure, flow turbulence, and compactness as well as other external variations. Generally, the variations of external variables influence spectral data in a non-linear manner which leads to the poor predictive ability of bilinear calibration models on raw spectral data. In this contribution, the influence of external variables on spectral data is generally classified into two different modes, multiplicative influential mode and composition-related influential mode. A new chemometric method, termed Extended Loading Space Standardization (ELSS), has been developed to explicitly model these two kinds of influential modes. ELSS was applied to two sets of spectral data with fluctuations in external variables and its performance evaluated and compared with global partial least squares (PLS) models and Loading Space Standardization (LSS). Results show that ELSS can efficiently model the external non-linear effects in both data sets and greatly improve the accuracy of predictions with the mean square error of prediction for test samples being 2-3 times smaller than those of LSS and global PLS.     

Calibration model transfer for near-infrared spectra based on canonical correlation analysis

In order to solve the calibration transformation problem in near-infrared (NIR) spectroscopy, a method based on canonical correlation analysis (CCA) for calibration model transfer is developed in this work. Two real NIR data sets were tested. A comparative study between the proposed method and piecewise direct standardization (PDS) was conducted. It is shown that the transfer results obtained with the proposed method based on CCA were better than those obtained by PDS when the subset had sufficient samples.     

Construction of global and robust near-infrared calibration models based on hybrid calibration sets using Partial Least Squares (PLS) regression

Near-infrared spectroscopy (NIR) models built on a particular instrument are often invalid on other instruments due to spectral inconsistencies between the instruments. In the present work, global and robust NIR calibration models were constructed by partial least square (PLS) regression based on hybrid calibration sets, which are composed of both primary and secondary spectra. Three datasets were used as case studies. The first consisted of 72 radix scutellaria samples measured on two NIR spectrometers with known baicalin content. The second was composed of 80 corn samples measured on two instruments with known moisture, oil, and protein concentrations. The third dataset included 279 primary samples of tobacco with known nicotine content and 78 secondary samples of tobacco with known nicotine concentrations. The effect of the number of secondary spectra in the hybrid calibration sets and the methods for selecting secondary spectra on the PLS model performance were investigated by comparing the results obtained from different calibration sets. This study shows that the global and robust calibration models accurately predicted both primary and secondary samples as long as the ratios of the number of primary spectra to the number of secondary spectra were less than 22. The models performance was not influenced by the selection method of the secondary spectra. The hybrid calibration sets included the primary spectral information and also the secondary spectra; information, rendering the constructed global and robust models applicable to both primary and secondary instruments.     

Orthogonal signal correction, wavelet analysis, and multivariate calibration of complicated process fluorescence data

In this paper, multivariate calibration of complicated process fluorescence data is presented. Two data sets related to the production of white sugar are investigated. The first data set comprises 106 observations and 571 spectral variables, and the second data set 268 observations and 3997 spectral variables. In both applications, a single response, ash content, is modelled and predicted as a function of the spectral variables. Both data sets contain certain features making multivariate calibration efforts non-trivial. The objective is to show how principal component analysis (PCA) and partial least squares (PLS) regression can be used to overview the data sets and to establish predictively sound regression models. It is shown how a recently developed technique for signal filtering, orthogonal signal correction (OSC), can be applied in multivariate calibration to enhance predictive power. In addition, signal compression is tested on the larger data set using wavelet analysis. It is demonstrated that a compression down to 4% of the original matrix size — in the variable direction — is possible without loss of predictive power. It is concluded that the combination of OSC for pre-processing and wavelet analysis for compression of spectral data is promising for future use.     

Robust absorbance computations in the analysis of glucose by near-infrared spectroscopy

Procedures for data acquisition and data processing are evaluated for the optimal computation of absorbance values based on Fourier transform near-infrared transmission spectra. Samples consisting of physiological levels (1–20 mM) of glucose in an aqueous matrix of variable levels of bovine serum albumin and triacetin are studied in the combination spectral region (5000–4000 cm⁻¹). The weak glucose signals in this region define a challenging analysis that is extremely sensitive to the effects of instrumental drift. The impact of different procedures for obtaining absorbance estimates is evaluated in the context of multivariate calibration models based on partial least-squares (PLS) regression. Replicate calibration and prediction data acquired over 6 months are used to study the robustness of PLS models with respect to time. The recommended protocol for the absorbance calculations is based on the collection of a large group of individual background spectra during the instrumental warm-up period. Seven procedures are tested for obtaining optimal backgrounds for use with either the calibration or prediction data sets. When the developed methodology is employed, standard errors of prediction are maintained in the range of 1.0 mM for spectra acquired up to 6 months after the collection of the calibration data. This level of performance compares favorably to daily internal cross-validation errors of 0.5–0.9 mM.     

Comparison of various chemometric approaches for large near infrared spectroscopic data of feed and feed products

In the present study, different multivariate regression techniques have been applied to two large near-infrared data sets of feed and feed ingredients in order to fulfil the regulations and laws that exist about the chemical composition of these products. The aim of this paper was to compare the performances of different linear and nonlinear multivariate calibration techniques: PLS, ANN and LS-SVM. The results obtained show that ANN and LS-SVM are very powerful methods for non-linearity but LS-SVM can also perform quite well in the case of linear models. Using LS-SVM an improvement of the RMS for independent test sets of 10% is obtained in average compared to ANN and of 24% compared to PLS.     

Using near-infrared spectroscopy and differential adsorption bed method to study adsorption kinetics of orthoxylene on silica gel

This paper has demonstrated the study on the adsorption kinetics of orthoxylene on silica gel with a novel experimental methodology. In the method, there was a differential adsorption bed (DAB) where the solid adsorbent always contacted with the same bulk concentration of the adsorbate vapor, and the DAB was monitored with near-infrared diffuse reflectance spectroscopy (NIRDRS) continuously as well as non-invasively. Local partial least squares (PLS) algorithm was suggested to replace normal global PLS method in multivariate calibration models for processing NIRDRS data, because the concentration of the adsorbate on the adsorbent varied greatly as the adsorption process was going on. In this way, we, conveniently as well as promptly, obtained instantaneous adsorption rates of several orthoxylene/silica gel adsorption processes under different conditions like partial pressure of orthoxylene vapor and velocity of gas, and discovered that the adsorption process was physical adsorption, and mainly controlled by external diffusion.