Incorporating interactions in multi‐block sequential and orthogonalised partial least squares regression

This paper is about how to incorporate interaction effects in multi‐block methodologies. The method proposed is inspired by polynomial regression modelling in the case with only a few independent variables but extends/generalises the idea to situations where the blocks are potentially very large with respect to the number of variables. The method follows a so‐called type I sums of squares strategy where the linear effects (main effects) are incorporated sequentially and before the interactions. The sequential and orthogonalised partial least squares (SO‐PLS) technique is used as a basis for the proposal. The SO‐PLS method is based on sequential estimation of each new block by the PLS regression method after orthogonalisation with respect to blocks already fitted. The new method preserves the invariance already established for SO‐PLS and can be used for blocks with different dimensionality. The method is tested on one real data set with two independent blocks with different complexity and on a simulated data set with a large number of variables in each block. Copyright © 2011 John Wiley & Sons, Ltd.     

Orthogonal signal correction‐based prediction of total antioxidant activity using partial least squares regression from chromatograms

The multivariate calibration methods—partial least squares (PLS), orthogonal signal correction and partial least squares (OSC‐PLS)—were employed for the prediction of total antioxidant activities of four Prunella L. species. High‐performance liquid chromatography (HPLC) and spectrophotometric approaches were used to determine the total antioxidant activity of the Prunella L. samples. Several preprocessing techniques such as smoothing and normalization were employed to extract the chemically relevant information from the data after alignment with correlation optimized warping. The importance of the preprocessing was investigated by calculating the root mean square error for the calibration set for the total antioxidant activity of Prunella L. samples. The models developed on the basis of the preprocessed data were able to predict the total antioxidant activity with a precision comparable to that of the reference 2,2‐azino‐di‐(3‐ethylbenzothialozine‐sulfonic acid) and 2,2‐diphenyl‐1‐picrylhydrazyl methods. The OSC‐PLS model seems preferable because of its predictive and describing abilities and good interpretability of the contribution of compounds to the total antioxidant activity. The contribution of individual phenolic compounds to the total antioxidant activity was identified by HPLC. Copyright © 2012 John Wiley & Sons, Ltd.     

Optimized sample-weighted partial least squares

In ordinary multivariate calibration methods, when the calibration set is determined to build the model describing the relationship between the dependent variables and the predictor variables, each sample in the calibration set makes the same contribution to the model, where the difference of representativeness between the samples is ignored. In this paper, by introducing the concept of weighted sampling into partial least squares (PLS), a new multivariate regression method, optimized sample-weighted PLS (OSWPLS) is proposed. OSWPLS differs from PLS in that it builds a new calibration set, where each sample in the original calibration set is weighted differently to account for its representativeness to improve the prediction ability of the algorithm. A recently suggested global optimization algorithm, particle swarm optimization (PSO) algorithm is used to search for the best sample weights to optimize the calibration of the original training set and the prediction of an independent validation set. The proposed method is applied to two real data sets and compared with the results of PLS, the most significant improvement is obtained for the meat data, where the root mean squared error of prediction (RMSEP) is reduced from 3.03 to 2.35. For the fuel data, OSWPLS can also perform slightly better or no worse than PLS for the prediction of the four analytes. The stability and efficiency of OSWPLS is also studied, the results demonstrate that the proposed method can obtain desirable results within moderate PSO cycles.     

Constrained kernelized partial least squares

Nonlinear kernel methods have been widely used to deal with nonlinear problems in latent variable methods. However, in the presence of structured noise, these methods have reduced efficacy. We have previously introduced constrained latent variable methods that make use of any available additional knowledge about the structured noise. These methods improve performance by introducing additional constraints into the algorithm. In this paper, we build upon our previous work and introduce hard‐constrained and soft‐constrained nonlinear partial least squares methods using nonlinear kernels. The addition of nonlinear kernels reduces the effects of structured noise in nonlinear spaces and improves the regression performance between the input and response variables. Copyright © 2014 John Wiley & Sons, Ltd.     

A robust partial least squares regression method with applications

Partial least squares (PLS) regression is a linear regression technique developed to relate many regressors to one or several response variables. Robust methods are introduced to reduce or remove the effect of outlying data points. In this paper, we show that if the sample covariance matrix is properly robustified further robustification of the linear regression steps of the PLS algorithm becomes unnecessary. The robust estimate of the covariance matrix is computed by searching for outliers in univariate projections of the data on a combination of random directions (Stahel—Donoho) and specific directions obtained by maximizing and minimizing the kurtosis coefficient of the projected data, as proposed by Peña and Prieto [1]. It is shown that this procedure is fast to apply and provides better results than other methods proposed in the literature. Its performance is illustrated by Monte Carlo and by an example, where the algorithm is able to show features of the data which were undetected by previous methods. Copyright © 2008 John Wiley & Sons, Ltd.     

QSAR study of angiotensin II antagonists using robust boosting partial least squares regression

In the current study, robust boosting partial least squares (RBPLS) regression has been proposed to model the activities of a series of 4H-1,2,4-triazoles as angiotensin II antagonists. RBPLS works by sequentially employing PLS method to the robustly reweighted versions of the training compounds, and then combing these resulting predictors through weighted median. In PLS modeling, an F-statistic has been introduced to automatically determine the number of PLS components. The results obtained by RBPLS have been compared to those by boosting partial least squares (BPLS) repression and partial least squares (PLS) regression, showing the good performance of RBPLS in improving the QSAR modeling. In addition, the interaction of angiotensin II antagonists is a complex one, including topological, spatial, thermodynamic and electronic effects.     

Robustness properties of a robust partial least squares regression method

The presence of multicollinearity in regression data is no exception in real life examples. Instead of applying ordinary regression methods, biased regression techniques such as principal component regression and ridge regression have been developed to cope with such datasets. In this paper, we consider partial least squares (PLS) regression by means of the SIMPLS algorithm. Because the SIMPLS algorithm is based on the empirical variance-covariance matrix of the data and on least squares regression, outliers have a damaging effect on the estimates. To reduce this pernicious effect of outliers, we propose to replace the empirical variance-covariance matrix in SIMPLS by a robust covariance estimator. We derive the influence function of the resulting PLS weight vectors and the regression estimates, and conclude that they will be bounded if the robust covariance estimator has a bounded influence function. Also the breakdown value is inherited from the robust estimator. We illustrate the results using the MCD estimator and the reweighted MCD estimator (RMCD) for low-dimensional datasets. Also some empirical properties are provided for a high-dimensional dataset.     

Study of adsorption processes of model drugs at supercritical conditions using partial least squares regression

The characterisation of adsorption or impregnation processes using conventional or supercritical fluid technologies becomes an increasing part of the research on drug formulations. The complexity of the relationships between these adsorption processes and the experimental variables potentially influencing them, however, makes these studies more problematic. In this paper, a chemometric approach based on nonlinear partial least squares (NL-PLS) modelling is applied to characterise the effect of the experimental variables on the supercritical impregnation process. Various adsorbent materials such as silica gel, zeolite and amberlite were investigated using the following model compounds as adsorbates: benzoic, salicylic and acetylsalicylic acids.     

A novel solid phase for selective separation of flavonoid compounds

A novel straightforward approach to selective separation for flavonoid compounds was reported. The solid phase material was prepared by copolymerization using allyl-bromide-modified chitosan as macromonomer, and ethylene glycol dimethacrylate as cross-linker. The material was evaluated by chromatographic analysis; it exhibited high selectivity separation for quercetin and its structural analogues using different mobile phases. The material could directly trap a specific class of compounds including quercetin and kaempferol from the hydrolyzate of Ginkgo biloba extract. These results demonstrated the possibility of direct extraction of certain constituents from herb using this material.     

Simultaneous kinetic spectrophotometric determination of cyanide and thiocyanate using the partial least squares (PLS) regression

The partial least squares (PLS-1) calibration model based on spectrophotometric measurement, for the simultaneous determination of CN⁻ and SCN⁻ ions is described. The method is based on the difference in the rate of the reaction between CN⁻ and SCN⁻ ions with chloramine-T in a pH 4.0 buffer solution and at 30 °C. The produced cyanogen chloride (CNCl) reacts with pyridine and the product condenses with barbituric acid and forms a final colored product. The absorption kinetic profiles of the solutions were monitored by measuring absorbance at 578 nm in the time range 20-180 s after initiation of the reaction with 2 s intervals. The experimental calibration matrix for partial least squares (PLS-1) calibration was designed with 31 samples. The cross-validation method was used for selecting the number of factors. The results showed that simultaneous determination could be performed in the range 10.0-900.0 and 50.0-1200.0 ng mL⁻¹ for CN⁻ and SCN⁻ ions, respectively. The proposed method was successfully applied to the simultaneous determination of cyanide and thiocyanate in water samples.     

偏最小二乘线性拟合滴定甘蔗糖蜜中多组分有机酸

将滴定体系调节至pH 2.0,用碱标准溶液滴定至特定pH所消耗滴定荆为测量指标,构建了多组分有机酸滴定数据阵,分别以主成分回归法、偏最小二乘法以及人工神经元网络法进行多组分拟合.结果表明,偏最小二乘法的拟合结果最佳,对混合体系中乙酸、乳酸、草酸、琥珀酸、柠檬酸和乌头酸总量的相对预测均方根误差分别为5.80%、8.88%...     

Averaged and weighted average partial least squares

Two alternative partial least squares (PLS) methods, averaged PLS and weighted average PLS, are proposed and compared with the classical PLS in terms of root mean square error of prediction (RMSEP) for three real data sets. These methods compute the (weighted) average of PLS models with different complexity. The prediction abilities of the alternative methods are comparable to that of the classical PLS but they do not require to determine how many components should be included in the model. They are also more robust in the sense that the quality of prediction depends less on a good choice of the number of components to be included. In addition, weighted average PLS is also compared with the weighted average part of LOCAL, a published method that also applies weighted average PLS, with however an entirely different weighting scheme.     

Boosted regression trees, multivariate adaptive regression splines and their two-step combinations with multiple linear regression or partial least squares to predict blood-brain barrier passage: a case study

The use of some unconventional non-linear modeling techniques, i.e. classification and regression trees and multivariate adaptive regression splines-based methods, was explored to model the blood-brain barrier (BBB) passage of drugs and drug-like molecules. The data set contains BBB passage values for 299 structural and pharmacological diverse drugs, originating from a structured knowledge-based database. Models were built using boosted regression trees (BRT) and multivariate adaptive regression splines (MARS), as well as their respective combinations with stepwise multiple linear regression (MLR) and partial least squares (PLS) regression in two-step approaches. The best models were obtained using combinations of MARS with either stepwise MLR or PLS. It could be concluded that the use of combinations of a linear with a non-linear modeling technique results in some improved properties compared to the individual linear and non-linear models and that, when the use of such a combination is appropriate, combinations using MARS as non-linear technique should be preferred over those with BRT, due to some serious drawbacks of the BRT approaches.     

紫外分光光度法-局部PLSR同时测定三种人造甜味剂

本文报道了一种简便、快速、准确的同时测定三种人造甜味剂安赛蜜、阿斯巴甜和糖精钠的方法。方法基于在pH为3.21的盐酸溶液中对安赛蜜、阿斯巴甜和糖精钠三组分混合溶液进行紫外光度测定,所得重叠光谱数据分别用偏最小二乘回归法（partial least squares regression,PLSR）、特征峰结合PLSR法和特征峰结合局部偏最小二乘回归法（local partial least squares regression,LPLSR）进行处理。结果表明,选取特征波段的峰值作为自变量,采用4个局部样本做拟合的预报误差最小,总相对偏差仅为3.05%。对果汁样品进行测定,获得了很好的定量分析结果。安赛蜜、阿斯巴甜和糖精钠的定量线性范围分别为1.0 - 30.0 mg/L、1.0 - 10.0 mg/L和1.0 – 10.0 mg/L。     

Comparison between ordinary least squares regression and weighted least squares regression in the calibration of metals present in human milk determined by ICP-OES

In this study we compared the use of ordinary least squares and weighted least squares in the calibration of the method for analyzing essential and toxic metals present in human milk by ICP-OES, in order to avoid systematic errors in the measurements used. Human milk samples were provided by maternity clinic Odete Valadares and digested by means of a high-performance microwave (MW) oven. Evaluation of plasma short and long-term stability was made using a solution of digested milk (1:50) with 2.0 mg L⁻¹ Mg in HNO₃ 2% (v/v). The detection power resulted to be at or below the μg L⁻¹ level, whilst the precision expressed as relative standard deviation R.S.D. was almost always equal to or better than 3.3%. Certified reference material Infant Formula (NIST SRM 1846) was used to assess the accuracy of the proposed method, which proved to be accurate and precise. Recovery rates were in the range of 83-117%. Aqueous calibration was carried out for each element under study.     

Spectral regions selection to improve prediction ability of PLS models by changeable size moving window partial least squares and searching combination moving window partial least squares

Changeable size moving window partial least squares (CSMWPLS) and searching combination moving window partial least squares (SCMWPLS) are proposed to search for an optimized spectral interval and an optimized combination of spectral regions from informative regions obtained by a previously proposed spectral interval selection method, moving window partial least squares (MWPLSR) [Anal. Chem. 74 (2002) 3555]. The utilization of informative regions aims to construct better PLS models than those based on the whole spectral points. The purpose of CSMWPLS and SCMWPLS is to optimize the informative regions and their combination to further improve the prediction ability of the PLS models. The results of their application to an open-path (OP)/FT-IR spectra data set show that the proposed methods, especially SCMWPLS can find out an optimized combination, with which one can improve, often significantly, the performance of the corresponding PLS model, in terms of low prediction error, root mean square error of prediction (RMSEP) with the reasonable latent variable (LVs) number, comparing with the results obtained using whole spectra or direct combination of informative regions for a compound. Regions consisting of the combinations obtained can easily be explained by the existence of IR absorption bands in those spectral regions.     

Partitioned partial least squares regression with application to a batch fermentation process

The performance of Partial Least Squares regression (PLS) in predicting the output with multivariate cross‐ and autocorrelated data is studied. With many correlated predictors of varying importance PLS does not always predict well and we propose a modified algorithm, Partitioned Partial Least Squares (PPLS). In PPLS the predictors are partitioned into smaller subgroups and the important subgroups with high prediction power are identified. Finally, regular PLS analysis using only those subgroups is performed. The proposed Partitioned PLS (PPLS) algorithm is used in the analysis of data from a real pharmaceutical batch fermentation process for which the process variables follow certain profiles during a specific fermentation period. We observed that PPLS leads to a more accurate prediction of the yield of the fermentation process and an easier interpretation, since fewer predictors are used in the final PLS prediction. In the application important issues such as alignment of the profiles from one batch to another and standardization of the predictors are also addressed. For instance, in PPLS noise magnification due to standardization does not seem to create problems as it might in regular PLS. Finally, PPLS is compared to several recently proposed functional PLS and PCR methods and a genetic algorithm for variable selection. More specifically for a couple of publicly available data sets with near infrared spectra it is shown that overall PPLS has lower cross‐validated error than PLS, PCR and the functional modifications hereof, and is similar in performance to a more complex genetic algorithm. Copyright © 2011 John Wiley & Sons, Ltd.     

Determination of sugars in depilatory formulations: a green analytical method employing infrared detection and partial least squares regression

A green analytical method was developed for the analysis of sugar-based depilatories. Three independent partial least squares (PLS) regression models were built for the direct determination of glucose, fructose and maltose without any sample pretreatment based on their attenuated total reflectance - Fourier transform infrared (ATR-FTIR) spectra. The models showed adequate prediction capabilities with root-mean-square-errors of prediction ranging from 7.04 to 12.55 mg sugar g⁻¹ sample. As a reference procedure, gradient liquid chromatography with on-line infrared detection, employing background correction based on cubic smoothing splines, was used. The analysis revealed changes in the sugar concentration due to the formulation process as compared to information on the ingredients provided by the manufacturers. Although fructose, glucose and sucrose were declared to be used for the production of depilatories, in the final products only fructose, glucose and maltose were determined. This fact was attributed to pH and temperature conditions employed during the production process as well as to the use of glucose syrup instead of crystalline glucose. The present ATR-FTIR-PLS method enables an accurate, cheap and fast determination without solvent consumption or toxic waste generation and offers therefore a green screening alternative to methods employing chromatographic techniques.     

A nonlinear partial least squares algorithm using quadratic fuzzy inference system

We introduce a new nonlinear partial least squares algorithm ‘Quadratic Fuzzy PLS (QFPLS)’ that combines the outer linear Partial Least Squares (PLS) framework and the Takagi–Sugeno–Kang (TSK) fuzzy inference system. The inner relation between the input and the output PLS score vectors is modeled by a quadratic TSK fuzzy inference system. The performance of the proposed QFPLS method is tested and compared against four other well‐known partial least squares methods (Linear PLS (LPLS), Quadratic PLS (QPLS), Linear Fuzzy PLS (LFPLS), and Neural Network PLS (NNPLS)) on various different types of randomly generated test data. QFPLS outperformed competitors based on two comparison measures: the output variables cumulative per cent variance captured by the PLS latent variables and the root mean‐square error of prediction (RMSEP). Copyright © 2009 John Wiley & Sons, Ltd.