Incorporating interactions in multi‐block sequential and orthogonalised partial least squares regression

This paper is about how to incorporate interaction effects in multi‐block methodologies. The method proposed is inspired by polynomial regression modelling in the case with only a few independent variables but extends/generalises the idea to situations where the blocks are potentially very large with respect to the number of variables. The method follows a so‐called type I sums of squares strategy where the linear effects (main effects) are incorporated sequentially and before the interactions. The sequential and orthogonalised partial least squares (SO‐PLS) technique is used as a basis for the proposal. The SO‐PLS method is based on sequential estimation of each new block by the PLS regression method after orthogonalisation with respect to blocks already fitted. The new method preserves the invariance already established for SO‐PLS and can be used for blocks with different dimensionality. The method is tested on one real data set with two independent blocks with different complexity and on a simulated data set with a large number of variables in each block. Copyright © 2011 John Wiley & Sons, Ltd.     

Regression models with process variables and parallel blocks of raw material measurements

The topic of this paper is regression models based on designed experiments, where additional spectroscopic measurements are also available. This particular case describes a situation with two spectral blocks with no natural order: The blocks are parallel. Three methods are described, which combine least squares regression of the design variables with PCA or PLS on the spectra. The methods properties are explored in two simulation studies based on real experiments. The results show that the methods are equal when it comes to prediction, but interpretability varies. One of the methods, LS‐ParPLS, is especially interesting when it comes to interpretability because it splits the spectral information into two parts; information that is common in both blocks and information that is unique for each block. Copyright © 2008 John Wiley & Sons, Ltd.     

Quadratic PLS1 regression revisited

Within the framework of nonlinear partial least squares (PLS), the quadratic PLS regression approach, involving both linear and quadratic terms in the criterion, is discussed. A new algorithm for the determination of the components is proposed, and its advantages over the original algorithm are outlined. The approach of analysis is illustrated on the basis of simulated and real data. Copyright © 2012 John Wiley & Sons, Ltd.     

Bilinear modelling of batch processes. Part II: a comparison of PLS soft‐sensors

The on‐line monitoring of batch processes based on principal component analysis (PCA) has been widely studied. Nonetheless, researchers have not paid so much attention to the on‐line application of partial least squares (PLS). In this paper, the influence of several issues in the predictive power of a PLS model for the on‐line estimation of key variables in a batch process is studied. Some of the conclusions can help to better understand the capabilities of the proposals presented for on‐line PCA‐based monitoring. Issues like the convenience of batch‐wise or variable‐wise unfolding, the method for the imputation of future measurements and the use of several sub‐models are addressed. This is the first time that the adaptive hierarchical (or multi‐block) approach is extended to the PLS modelling. Also, the formulation of the so‐called trimmed scores regression (TSR), a powerful imputation method defined for PCA, is extended for its application with PLS modelling. Data from two processes, one simulated and one real, are used to illustrate the results. Copyright © 2008 John Wiley & Sons, Ltd.     

Construction of global and robust near-infrared calibration models based on hybrid calibration sets using Partial Least Squares (PLS) regression

Near-infrared spectroscopy (NIR) models built on a particular instrument are often invalid on other instruments due to spectral inconsistencies between the instruments. In the present work, global and robust NIR calibration models were constructed by partial least square (PLS) regression based on hybrid calibration sets, which are composed of both primary and secondary spectra. Three datasets were used as case studies. The first consisted of 72 radix scutellaria samples measured on two NIR spectrometers with known baicalin content. The second was composed of 80 corn samples measured on two instruments with known moisture, oil, and protein concentrations. The third dataset included 279 primary samples of tobacco with known nicotine content and 78 secondary samples of tobacco with known nicotine concentrations. The effect of the number of secondary spectra in the hybrid calibration sets and the methods for selecting secondary spectra on the PLS model performance were investigated by comparing the results obtained from different calibration sets. This study shows that the global and robust calibration models accurately predicted both primary and secondary samples as long as the ratios of the number of primary spectra to the number of secondary spectra were less than 22. The models performance was not influenced by the selection method of the secondary spectra. The hybrid calibration sets included the primary spectral information and also the secondary spectra; information, rendering the constructed global and robust models applicable to both primary and secondary instruments.     

Algorithms for multi‐group PLS

Several approaches of investigation of the relationships between two datasets where the individuals are structured into groups are discussed. These strategies fit within the framework of partial least squares (PLS) regression. Each strategy of analysis is introduced on the basis of a maximization criterion, which involves the covariances between components associated with the groups of individuals in each dataset. Thereafter, algorithms are proposed to solve these maximization problems. The strategies of analysis can be considered as extensions of multi‐group principal components analysis to the context of PLS regression. Copyright © 2014 John Wiley & Sons, Ltd.     

Post‐transformation of PLS2 (ptPLS2) by orthogonal matrix: a new approach for generating predictive and orthogonal latent variables

Partial Least Squares (PLS) is a wide class of regression methods aiming at modelling relationships between sets of observed variables by means of latent variables. Specifically, PLS2 was developed to correlate two blocks of data, the X‐block representing the independent or explanatory variables and the Y‐block representing the dependent or response variables. Lately, OPLS was introduced to further reduce model complexity by removing Y‐orthogonal sources of variation from X in the latent space, thus improving data interpretation through the generated predictive latent variables. Nevertheless, relationships between PLS2 and OPLS in case of multiple Y‐response have not yet been fully explored. With this perspective and taking inspiration from some basic mathematical properties of PLS2, we here present a novel and general approach consisting in a post‐transformation of PLS2 (ptPLS2), which results in a decomposition of the latent space into orthogonal and predictive components, while preserving the same goodness of fit and predictive ability of PLS2. Additionally, we discuss the application of ptPLS2 approach to two metabolomic data sets extracted from earlier published studies and its advantages in model interpretation as compared with the ‘standard’ PLS approach. Copyright © 2016 John Wiley & Sons, Ltd.     

OPLS filtered data can be obtained directly from non‐orthogonalized PLS1

The well‐known Martens factorization for PLS1 produces a single y‐related score, with all subsequent scores being y‐unrelated. The X‐explanatory value of these y‐orthogonal scores can be summarized by a simple expression, which is analogous to the ‘P’ loading weights in the orthogonalized NIPALS algorithm. This can be used to rearrange the factorization into entirely y‐related and y‐unrelated parts. Systematic y‐unrelated variation can thus be removed from the X data through a single post hoc calculation following conventional PLS, without any recourse to the orthogonal projections to latent structures (OPLS) algorithm. The work presented is consistent with the development by Ergon (PLS post‐processing by similarity transformation (PLS + ST): a simple alternative to OPLS. J. Chemom. 2005; 19 : 1–4), which shows that conventional PLS and OPLS are equivalent within a similarity transform. Copyright © 2009 John Wiley & Sons, Ltd.     

Assessment of bitter taste of pharmaceuticals with multisensor system employing 3 way PLS regression

The application of the potentiometric multisensor system (electronic tongue, ET) for quantification of the bitter taste of structurally diverse active pharmaceutical ingredients (API) is reported. The measurements were performed using a set of bitter substances that had been assessed by a professional human sensory panel and the in vivo rat brief access taste aversion (BATA) model to produce bitterness intensity scores for each substance at different concentrations. The set consisted of eight substances, both inorganic and organic – azelastine, caffeine, chlorhexidine, potassium nitrate, naratriptan, paracetamol, quinine, and sumatriptan. With the aim of enhancing the response of the sensors to the studied APIs, measurements were carried out at different pH levels ranging from 2 to 10, thus promoting ionization of the compounds. This experiment yielded a 3 way data array (samples × sensors × pH levels) from which 3wayPLS regression models were constructed with both human panel and rat model reference data. These models revealed that artificial assessment of bitter taste with ET in the chosen set of API's is possible with average relative errors of 16% in terms of human panel bitterness score and 25% in terms of inhibition values from in vivo rat model data. Furthermore, these 3wayPLS models were applied for prediction of the bitterness in blind test samples of a further set of API's. The results of the prediction were compared with the inhibition values obtained from the in vivo rat model.     

Quantitation of κ-, ι- and λ-carrageenans by mid-infrared spectroscopy and PLS regression

Carrageenans are natural products obtained from seaweeds which are used as additives in many industrial fields, mainly in the food industry (labelled as E407). The three most employed ones being the so-called κ-, ι- and λ-carrageenans. So far, their industrial characterisation is based on physical measurements, which exhibit a rather large uncertainty. The aim of this work is to develop a new analytical methodology for the quantitative determination of carrageenans in industrial mixtures employing FTIR and multivariate regression (partial least squares, PLS), avoiding complex sample pre-treatment steps and reducing the turnaround time. The methodology allows to handle liquid (dissolved) carrageenans at room temperature (to prevent degradation), and therefore, their straightforward IR measurement using thin films. The standard prediction errors for the different models selected range from 3.3 to 4.2%, which can be considered as excellent.     

Simultaneous spectrophotometric determination of carbamazepine and phenytoin in serum by PLS regression and comparison with HPLC

Carbamazepine (CBZ) and phenytoin (PHT) are two antiepileptic drugs which are used simultaneously. In this paper a partial least-squares (PLS) calibration method is described for the simultaneous spectrophotometric determination of CBZ and PHT in plasma. Standard binary mixtures of CBZ and PHT have been resolved by application of PLS-1 to their UV spectra. Then, the binary standard solutions, spiked to plasma, were prepared and after the extraction of the drugs, their corresponding UV spectrum were analyzed by PLS regression to calculate the concentration of drugs in unknown plasma. A leave one out cross-validation procedure was employed to find the optimum numbers of latent variables using PRESS. A HPLC method was also applied for simultaneous determination of two drugs in the plasma and in methanol. The mean recoveries obtained by PLS were 98.4 and 98.2 for CBZ and PHT and those obtained by HPLC were 100.1 and 101.7, respectively. Although, the HPLC method showed better performance than PLS, it was found that the results obtained by PLS were comparable with those obtained by HPLC method.     

A comparison of nine PLS1 algorithms

Nine PLS1 algorithms were evaluated, primarily in terms of their numerical stability, and secondarily their speed. There were six existing algorithms: (a) NIPALS by Wold; (b) the non‐orthogonalized scores algorithm by Martens; (c) Bidiag2 by Golub and Kahan; (d) SIMPLS by de Jong; (e) improved kernel PLS by Dayal; and (f) PLSF by Manne. Three new algorithms were created: (g) direct‐scores PLS1 based on a new recurrent formula for the calculation of basis vectors yielding scores directly from X and y; (h) Krylov PLS1 with its regression vector defined explicitly, using only the original X and y; (i) PLSPLS1 with its regression vector recursively defined from X and the regression vectors of its previous recursions. Data from IR and NIR spectrometers applied to food, agricultural, and pharmaceutical products were used to demonstrate the numerical stability. It was found that three methods (c, f, h) create regression vectors that do not well resemble the corresponding precise PLS1 regression vectors. Because of this, their loading and score vectors were also concluded to be deviating, and their models of X and the corresponding residuals could be shown to be numerically suboptimal in a least squares sense. Methods (a, b, e, g) were the most stable. Two of them (e, g) were not only numerically stable but also much faster than methods (a, b). The fast method (d) and the moderately fast method (i) showed a tendency to become unstable at high numbers of PLS factors. Copyright © 2009 John Wiley & Sons, Ltd.     

THEME‐SEER: a multidimensional exploratory technique to analyze a structural model using an extended covariance criterion

In this work, we present a new approach to path modeling based on an extended multiple covariance criterion: system extended multiple covariance (SEMC). SEMC is suitable to measure the quality of any structural equations system. We show why SEMC may be preferred to criteria based on usual covariance of components and also to criteria based on residual sums of squares. We give a pursuit algorithm ensuring that SEMC increases and converges. When one wishes to extract more than one component per variable group, a problem arises of component hierarchy. To solve it, we define a local nesting principle of component models that makes the role of each component statistically clear. We then embed the pursuit algorithm in a more general algorithm that extracts sequences of locally nested models. We finally provide a component backward selection strategy. The technique is applied to cigarette data to model the generation of chemical compounds in smoke through tobacco combustion. Copyright © 2012 John Wiley & Sons, Ltd.     

A partial least squares regression study with antioxidant flavonoid compounds

The quantitative structure-activity relationship of a set of 19 flavonoid compounds presenting antioxidant activity was studied by means of PLS (Partial Least Squares) regression. The optimization of the structures and calculation of electronic properties were done by using the semiempirical method AM1. A reliable model (r ²=0.806 and q ²=0.730) was obtained and from this model it was possible to consider some aspects of the structure of the flavonoid compounds studied that are related with their free radical scavenging ability. The quality of the PLS model obtained in this work indicates that it can be used in order to design new flavonoid compounds that present ability to scavenge free radicals.     

Evaluation of smoothing techniques in the run to run optimization of fed‐batch processes with u‐PLS

Run to run (R2R) optimization based on unfolded Partial Least Squares (u‐PLS) is a promising approach for improving the performance of batch and fed‐batch processes as it is able to continuously adapt to changing processing conditions. Using this technique, the regression coefficients of PLS are used to modify the input profile of the process in order to optimize the yield. When this approach was initially proposed, it was observed that the optimization performed better when PLS was combined with a smoothing technique, in particular a sliding window filtering, which constrained the regression coefficients to be smooth. In the present paper, this result is further investigated and some modifications to the original approach are proposed. Also, the suitability of different smoothing techniques in combination with PLS is studied for both end‐of‐batch quality prediction and R2R optimization. The smoothing techniques considered in this paper include the original filtering approach, the introduction of smoothing constraints in the PLS calibration (Penalized PLS), and the use of functional analysis (Functional PLS). Two fed‐batch process simulators are used to illustrate the results. Copyright © 2015 John Wiley & Sons, Ltd.     

偏最小二乘法在红外光谱识别茶叶中的应用

采用漫反射傅立叶变换红外光谱(FTIR)法结合主成分分析(PCA)、偏最小二乘法(PLS)、簇类的独立软模式(SIMCA)识别法对十三种茶叶进行了分类判别研究。研究结果表明,通过多元散射校正(MSC)对原始光谱进行预处理,可以提高模式识别技术的分类判别效果。在此基础上,选取1 900~900 cm-1波长范围内的茶叶红外光谱建立识别模型,三种方法都得到了满意的分类判别效果。在对检验集中全部130个样本的判别中,PCA仅有两类样本无法判别,SIMCA的识别率和拒绝率都在90%以上,而PLS的识别效果最佳,全部样本都得到了正确的归类。这一研究结果表明傅立叶变换红外光谱法与化学计量学方法相结合可以实现茶叶品种的快速鉴别,这为茶叶的客观评审提供了一种新思路。     

Simultaneous kinetic spectrophotometric determination of cyanide and thiocyanate using the partial least squares (PLS) regression

The partial least squares (PLS-1) calibration model based on spectrophotometric measurement, for the simultaneous determination of CN⁻ and SCN⁻ ions is described. The method is based on the difference in the rate of the reaction between CN⁻ and SCN⁻ ions with chloramine-T in a pH 4.0 buffer solution and at 30 °C. The produced cyanogen chloride (CNCl) reacts with pyridine and the product condenses with barbituric acid and forms a final colored product. The absorption kinetic profiles of the solutions were monitored by measuring absorbance at 578 nm in the time range 20-180 s after initiation of the reaction with 2 s intervals. The experimental calibration matrix for partial least squares (PLS-1) calibration was designed with 31 samples. The cross-validation method was used for selecting the number of factors. The results showed that simultaneous determination could be performed in the range 10.0-900.0 and 50.0-1200.0 ng mL⁻¹ for CN⁻ and SCN⁻ ions, respectively. The proposed method was successfully applied to the simultaneous determination of cyanide and thiocyanate in water samples.     

Model choice and squared prediction errors in PLS regression

Squared prediction errors (SPE) in are discussed in relation to the conventional PLSR versus bidiagonalization model and algorithm issue concerning residual and prediction consistency, with focus on process monitoring and fault detection. Our analysis leads to the conclusion that conventional PLSR based on the NIPALS algorithm is ambiguous in SPE values caused by process faults. The basic reason for this is that the sample residuals are not found as projections onto the orthogonal complement of the space where the scores and regression solution are located, and where also the statistical limit is defined. The alternative non‐orthogonalized PLSR and bidiagonalization (Bidiag2) algorithms, as well as a simple re‐formulation of the NIPALS algorithm (RE‐PLSR), give unambiguous SPE values, and the last two of these also retain orthogonal score vectors. While prediction results from all of these methods in theory are identical, our conclusion is that methods where the and SPE values for process faults are uncorrelated should be preferred. Tests with added errors on real data do not indicate that this conclusion should be altered because of such errors. Copyright © 2011 John Wiley & Sons, Ltd.     

Cross fitted partial least squares (CF‐PLS): an alternative algorithm for a more reliable PLS

This paper presents a modified version of the NIPALS algorithm for PLS regression with one single response variable. This version, denoted a CF‐PLS, provides significant advantages over the standard PLS. First of all, it strongly reduces the over‐fit of the regression. Secondly, R² for the null hypothesis follows a Beta distribution only function of the number of observations, which allows the use of a probabilistic framework to test the validity of a component. Thirdly, the models generated with CF‐PLS have comparable if not better prediction ability than the models fitted with NIPALS. Finally, the scores and loadings of the CF‐PLS are directly related to the R², which makes the model and its interpretation more reliable. Copyright © 2011 John Wiley & Sons, Ltd.