Partial least-squares regression: a tutorial

A tutorial on the partial least-squares (PLS) regression method is provided. Weak points in some other regression methods are outlined and PLS is developed as a remedy for those weaknesses. An algorithm for a predictive PLS and some practical hints for its use are given.     

Application of FT-Raman Spectroscopy and Chemometric Analysis for Determination of Adulteration in Milk Powder

In this work, FT-Raman spectroscopy is explored as a rapid technique for the assessment of the milk powder quality. Based on information provided by Raman spectra of samples adulterated with starch and whey, a quantitative method is developed to identify the fraud, using Partial Least Squares regression (PLS). In regression models using PLS the results are satisfactory, and such models can be used to identify and quantify samples presenting whey and starch in milk powder at concentrations of 2.32% and 1.64% (w/w), respectively. In the whey determination, the obtained values in the PLS model of the new samples are compared with those obtained by the spectrophotometric method of acid ninhydrin. This result shows that there is no significant difference with the 95% level of confidence between the values provided by the PLS regression method and the acid ninhydrin. The present work shows Raman spectroscopy as an analytical tool which can be used in quality control of milk powder, even in fraud processes, and the calculated figures of merit such as sensitivity, accuracy, limit of detection and limit of quantification clearly demonstrate this potential use. Although the multivariate models developed are not strictly quantitative, especially for low concentrations, they can be used as screening methods for routine analysis, as showed by this work.     

Sparse matrix transform based weight updating in partial least squares regression

Regression from high dimensional observation vectors is particularly difficult when training data is limited. Partial least squares (PLS) partly solves the high dimensional regression problem by projecting the data to latent variables space. The key issue in PLS is the computation of weight vector which describes the covariance between the responses and observations. For small-sample-size and high-dimensional regression problem, the covariance estimation is usually inaccurate and the correlated components in the predictors will distort the PLS weight. In this paper, we propose a sparse matrix transform (SMT) based PLS (SMT-PLS) method for high-dimensional spectroscopy regression. In SMT-PLS, the observation data is first decorrelated by SMT. Then, in the decorrelated data space, the PLS loading weight is computed by least squares regression. SMT technique provides an accurate data covariance estimation, which can overcome the effect of small-sample-size and benefit both the PLS weight computation and subsequent regression prediction. The proposed SMT-PLS method is compared, in terms of root mean square errors of prediction, to PLS, Power PLS and PLS with orthogonal scatter correction on four real spectroscopic data sets. Experimental results demonstrate the efficacy and effectiveness of our proposed method.     

Rapid determination of total trihalomethanes index in drinking water

A method for the rapid determination of total trihalomethanes (THMs) index in drinking water has been developed by using a headspace-mass spectrometry (HS-MS) system and partial least squares (PLS) multivariate regression approach. Due to the presence of residual amounts of chlorine and organic matter in the drinking water, the use of a quenching reagent in order to avoid THM generation during the sample manipulation is necessary. The optimization experiments revealed that ascorbic acid was the best quenching reagent compared with sodium thiosulfate and ammonium sulfate. The use of a classification chemometric technique as soft independent modeling of class analogy before the PLS regression improved the results obtained in the prediction of the total THMs index, lowering the relative standard error of prediction (RSEP) from 11.4% to lower than 6.0%. The results obtained by the proposed HS-MS method were compared with those provided by a conventional chromatographic method after analyzing 20 real drinking water samples. A good agreement in the results was observed and no systematic differences were found, which corroborates the good performance of the proposed method.     

Comparison of Different PLS Algorithms for Simultaneous Determination of Cd(II), Cu(II), Pb(II), and Zn(II) by Anodic Stripping Voltammetry at Bismuth Film Electrode

In this work we evaluated the use of different variable selection techniques combined with partial least‐squares regression (PLS) – genetic algorithm PLS (GA‐PLS), interval PLS (iPLS), and synergy interval PLS (siPLS) – in the simultaneous determination of Cd(II), Cu(II), Pb(II) and Zn(II) by anodic stripping voltammetry at a bismuth film. Generally, variable selection provided an improvement in prediction results when compared to full‐voltammogram PLS. The use of interval selection based algorithms have shown to be most adequate than the selection of discrete variables by GA. Excellent analytical performances were obtained despite the inherent complexity of the simultaneous determination.     

Determination of benzo[a]pyrene in cigarette mainstream smoke by using mid-infrared spectroscopy associated with a novel chemometric algorithm

Determination of benzo[a]pyrene (BaP) in cigarette smoke can be very important for the tobacco quality control and the assessment of its harm to human health. In this study, mid-infrared spectroscopy (MIR) coupled to chemometric algorithm (DPSO-WPT-PLS), which was based on the wavelet packet transform (WPT), discrete particle swarm optimization algorithm (DPSO) and partial least squares regression (PLS), was used to quantify harmful ingredient benzo[a]pyrene in the cigarette mainstream smoke with promising result. Furthermore, the proposed method provided better performance compared to several other chemometric models, i.e., PLS, radial basis function-based PLS (RBF-PLS), PLS with stepwise regression variable selection (Stepwise-PLS) as well as WPT-PLS with informative wavelet coefficients selected by correlation coefficient test (rtest-WPT-PLS). It can be expected that the proposed strategy could become a new effective, rapid quantitative analysis technique in analyzing the harmful ingredient BaP in cigarette mainstream smoke.     

The determination of the fatty acid content of sea buckthorn seed oil using near infrared spectroscopy and variable selection methods for multivariate calibration

This study proposes an analytical method for the simultaneous near infrared (NIR) spectrometric determination of palmitic, oleic, linoleic and linolenic acids in sea buckthorn seed oil. For this purpose, four different combinations of multivariate calibration methods and variable selections were evaluated: partial least squares (PLS) with full spectrum; PLS with uninformative variables elimination (UVE); PLS with competitive adaptive reweighted sampling (CARS); and multiple linear regression (MLR) with uninformative variable elimination combined with successive projections algorithm (UVE-SPA). An independent set of samples was employed to evaluate the performance of the resulting models. The UVE-SPA-MLR model developed with a few spectral variables provided the best results for each parameter. The values of relative errors of prediction (REP) from the UVE-SPA-MLR model for palmitic, oleic, linoleic and linolenic acids are 1.77%, 1.20%, 1.02% and 1.40%, respectively. These results indicate that this method is a feasible and fast method for the determination of the fatty acid content of sea buckthorn seed oil.     

Determination of total sulfur in diesel fuel employing NIR spectroscopy and multivariate calibration

A method for sulfur determination in diesel fuel employing near infrared spectroscopy, variable selection and multivariate calibration is described. The performances of principal component regression (PCR) and partial least square (PLS) chemometric methods were compared with those shown by multiple linear regression (MLR), performed after variable selection based on the genetic algorithm (GA) or the successive projection algorithm (SPA). Ninety seven diesel samples were divided into three sets (41 for calibration, 30 for internal validation and 26 for external validation), each of them covering the full range of sulfur concentrations (from 0.07 to 0.33% w/w). Transflectance measurements were performed from 850 to 1800 nm. Although principal component analysis identified the presence of three groups, PLS, PCR and MLR provided models whose predicting capabilities were independent of the diesel type. Calibration with PLS and PCR employing all the 454 wavelengths provided root mean square errors of prediction (RMSEP) of 0.036% and 0.043% for the validation set, respectively. The use of GA and SPA for variable selection provided calibration models based on 19 and 9 wavelengths, with a RMSEP of 0.031% (PLS-GA), 0.022% (MLR-SPA) and 0.034% (MLR-GA). As the ASTM 4294 method allows a reproducibility of 0.05%, it can be concluded that a method based on NIR spectroscopy and multivariate calibration can be employed for the determination of sulfur in diesel fuels. Furthermore, the selection of variables can provide more robust calibration models and SPA provided more parsimonious models than GA.     

A Partial Least Squares‐Based Consensus Regression Method for the Analysis of Near‐Infrared Complex Spectral Data of Plant Samples

Abstract

A consensus regression approach based on partial least square (PLS) regression, named as cPLS, for calibrating the NIR data was investigated. In this approach, multiple independent PLS models were developed and integrated into a single consensus model. The utility and merits of the cPLS method were demonstrated by comparing its results with those from a regular PLS method in predicting moisture, oil, protein, and starch contents of corn samples using the NIR spectral data. It was found that cPLS was superior to regular PLS with respect to prediction accuracy and robustness.     

Determining the quality of insulating oils using near infrared spectroscopy and wavelength selection

This study presents an analytical method for determining interfacial tension and relative density in insulating oils using near infrared spectrometry (NIR). Five different strategies of regression were evaluated: partial least squares (PLS) with significant regression coefficients selected by jack-knife algorithm; interval PLS (iPLS); multiple linear regression (MLR) with variable selection by genetic algorithm (MLR/GA), successive projections algorithm (MLR/SPA) and stepwise strategy (SR/MLR). The overall results point to MLR/SPA as the best modeling strategy. The strategy is simpler and uses fewer spectral variables.     

Regression with multiple regressor arrays

Extension of standard regression to the case of multiple regressor arrays is given via the Kronecker product. The method is illustrated using ordinary least squares regression (OLS) as well as the latent variable (LV) methods principal component regression (PCR) and partial least squares regression (PLS). Denoting the method applied to PLS as mrPLS, the latter was shown to explain as much or more variance for the first LV relative to the comparable L‐partial least squares regression (L‐PLS) model. The same relationship holds when mrPLS is compared to PLS or n‐way partial least squares (N‐PLS) and the response array is 2‐way or 3‐way, respectively, where the regressor array corresponding to the first mode of the response array is 2‐way and the second mode regressor array is an identity matrix. In a comparison with N‐PLS using fragrance data, mrPLS proved superior in a validation sense when model selection was used. Though the focus is on 2‐way regressor arrays, the method can be applied to n‐way regressors via N‐PLS. Copyright © 2007 John Wiley & Sons, Ltd.     

Comparison of different calibration methods for the determination by FT-NIR spectroscopy of the hydroxyl number in polyester resins

Different calibration methods have been applied for the determination of the Hydroxyl Number in polyester resins, namely Partial Least Squares (PLS), Principal Component Regression (PCR), Ordinary Least Squares with selection of the variables by genetic algorithm (OLS-GEN) and back-propagation Artificial Neural Networks (BP-ANN). The predictive ability of the regression models was estimated by splitting the dataset in training and test sets by application of the Kohonen self-organising maps. The linear methods (OLS-GEN, PLS and PCR) showed comparable results while artificial neural networks provided the best results both in fitting and prediction.     

Determination of the content and identity of lidocaine solutions with UV-visible spectroscopy and multivariate calibration

A method is proposed for the determination of the content and identity of the active compound in pharmaceutical solutions by means of ultraviolet-visible (UV-Vis) spectroscopy, orthogonal signal correction (OSC) and multivariate calibration with soft independent modelling of class analogy (SIMCA) classification and partial least squares (PLS) regression. The content was determined with PLS regression and the identity with PLS regression and SIMCA classification. The method was tested on the local anaesthetic compound lidocaine. For the validation, external test sets of both manufactured sample solutions and samples from a stability study were used. For comparison with this new method, liquid chromatography was used as a reference method. The results show that in respect of accuracy, precision and repeatability, the new method is comparable to the reference method. The main advantage over liquid chromatography is the much shorter time of analysis and the simpler analytical procedure. An estimate of the analysis time saved with the proposed method compared with using liquid chromatography, together with practical considerations, is given.     

偏最小二乘回归用于近红外光谱分析的稳健策略

偏最小二乘法（PLS）在近红外光谱（NIR）定量分析中占有重要地位,但预测结果往往容易受到样本分组和奇异样本等因素的影响,稳健性不强。多模型PLS (EPLS）方法在模型稳健性上得到提高,然而它无法识别样本中存在的奇异样本。为了同时提高模型的预测准确性和稳健性,本文提出了一种根据取样概率重新取样的多模型PLS方法,称为稳健共识PLS（RE-PLS）方法。该方法通过迭代赋权偏最小二乘法(IRPLS)计算样本回归残差得到每个校正集样本的取样概率,然后根据样本的取样概率来选择训练子集建立多个PLS模型,最后将所有PLS模型的预测结果平均作为最终预测结果。该方法用于两种不同植物样品的近红外光谱建模,并与传统的PLS及EPLS方法进行比较。结果表明该方法可以有效的避免校正集中奇异样本对模型的影响,同时可以提高预测精确度和稳健性。对于含有较多奇异样本的,复杂近红外光谱烟草实际样本,利用简单PLS或者EPLS方法建模预测效果不是很理想,而RE-PLS凭借其独特优势则有望在这种复杂光谱定量分析中得到广泛的应用。     

Determination of gadolinium in geological samples by inductively coupled plasma mass spectrometry and multivariate calibration

Gadolinium can be difficult to determine by ICP-MS. In a normal geological sample there are risks of spectroscopic interferences on all of its isotopes. In this study this problem has been solved by using partial least squares (PLS) regression. Two PLS models are investigated: the first is based on aqueous standards, and the second on reference materials. Both models are capable of determining Gd with good results in reference materials containing interfering elements. It was not necessary to correct for nonspectroscopic matrix interferences. PLS is compared to principal components regression (PCR), another multivariate calibration method. For the aqueous standards PLS leads to a simpler model, while similar results are obtained for the two methods in the model based on reference materials.     

Partial least-squares modeling of continuous nodes in Bayesian networks

In Bayesian networks it is necessary to compute relationships between continuous nodes. The standard Bayesian network methodology represents this dependency with a linear regression model whose parameters are estimated by a maximum likelihood (ML) calculation. Partial least-squares (PLS) is proposed as an alternative method for computing the model parameters. This new hybrid method is termed PLS-Bayes, as it uses PLS to calculate regression vectors for a Bayesian network. This alternative approach requires storing the raw data matrix rather than sequentially updating sufficient statistics, but results in a regression matrix that predicts with higher accuracy, requires less training data, and performs well in large networks.     

Rapid determination of sucrose in chocolate mass using near infrared spectroscopy

This paper reports the results of a rapid method to determine sucrose in chocolate mass using near infrared spectroscopy (NIRS). We applied a broad-based calibration approach, which consists in putting together in one single calibration samples of various types of chocolate mass. This approach increases the concentration range for one or more compositional parameters, improves the model performance and requires just one calibration model for several recipes. The data were modelled using partial least squares (PLS) and multiple linear regression (MLR). The MLR models were developed using a variable selection based on the coefficient regression of PLS and genetic algorithm (GA). High correlation coefficients (0.998, 0.997, 0.998 for PLS, MLR and GA-MLR, respectively) and low prediction errors confirms the good predictability of the models. The results show that NIR can be used as rapid method to determine sucrose in chocolate mass in chocolate factories.     

QSAR study of angiotensin II antagonists using robust boosting partial least squares regression

In the current study, robust boosting partial least squares (RBPLS) regression has been proposed to model the activities of a series of 4H-1,2,4-triazoles as angiotensin II antagonists. RBPLS works by sequentially employing PLS method to the robustly reweighted versions of the training compounds, and then combing these resulting predictors through weighted median. In PLS modeling, an F-statistic has been introduced to automatically determine the number of PLS components. The results obtained by RBPLS have been compared to those by boosting partial least squares (BPLS) repression and partial least squares (PLS) regression, showing the good performance of RBPLS in improving the QSAR modeling. In addition, the interaction of angiotensin II antagonists is a complex one, including topological, spatial, thermodynamic and electronic effects.