Algorithms for multi‐group PLS

Several approaches of investigation of the relationships between two datasets where the individuals are structured into groups are discussed. These strategies fit within the framework of partial least squares (PLS) regression. Each strategy of analysis is introduced on the basis of a maximization criterion, which involves the covariances between components associated with the groups of individuals in each dataset. Thereafter, algorithms are proposed to solve these maximization problems. The strategies of analysis can be considered as extensions of multi‐group principal components analysis to the context of PLS regression. Copyright © 2014 John Wiley & Sons, Ltd.     

Cross fitted partial least squares (CF‐PLS): an alternative algorithm for a more reliable PLS

This paper presents a modified version of the NIPALS algorithm for PLS regression with one single response variable. This version, denoted a CF‐PLS, provides significant advantages over the standard PLS. First of all, it strongly reduces the over‐fit of the regression. Secondly, R² for the null hypothesis follows a Beta distribution only function of the number of observations, which allows the use of a probabilistic framework to test the validity of a component. Thirdly, the models generated with CF‐PLS have comparable if not better prediction ability than the models fitted with NIPALS. Finally, the scores and loadings of the CF‐PLS are directly related to the R², which makes the model and its interpretation more reliable. Copyright © 2011 John Wiley & Sons, Ltd.     

Co‐evolution of non‐linear PLS model components

The issue of outer model weight updating is important in extending partial least squares (PLS) regression to modelling data that shows significant non‐linearity. This paper presents a novel co‐evolutionary component approach to the weight updating problem. Specification of the non‐linear PLS model is achieved using an evolutionary computational (EC) method that can co‐evolve all non‐linear inner models and all input projection weights simultaneously. In this method, modular symbolic non‐linear equations are used to represent the inner models and binary sequences are used to represent the projection weights. The approach is flexible, and other representations could be employed within the same co‐evolutionary framework. The potential of these methods is illustrated using a simulated pH neutralisation process data set exhibiting significant non‐linearity. It is demonstrated that the co‐evolutionary component architecture can produce results which are competitive with non‐linear neural network‐based PLS algorithms that use iterative projection weight updating. In addition, a data sampling method for mitigating overfitting to the training data is described. Copyright © 2007 John Wiley & Sons, Ltd.     

Canonical partial least squares—a unified PLS approach to classification and regression problems

We propose a new data compression method for estimating optimal latent variables in multi‐variate classification and regression problems where more than one response variable is available. The latent variables are found according to a common innovative principle combining PLS methodology and canonical correlation analysis (CCA). The suggested method is able to extract predictive information for the latent variables more effectively than ordinary PLS approaches. Only simple modifications of existing PLS and PPLS algorithms are required to adopt the proposed method. Copyright © 2009 John Wiley & Sons, Ltd.     

OPLS filtered data can be obtained directly from non‐orthogonalized PLS1

The well‐known Martens factorization for PLS1 produces a single y‐related score, with all subsequent scores being y‐unrelated. The X‐explanatory value of these y‐orthogonal scores can be summarized by a simple expression, which is analogous to the ‘P’ loading weights in the orthogonalized NIPALS algorithm. This can be used to rearrange the factorization into entirely y‐related and y‐unrelated parts. Systematic y‐unrelated variation can thus be removed from the X data through a single post hoc calculation following conventional PLS, without any recourse to the orthogonal projections to latent structures (OPLS) algorithm. The work presented is consistent with the development by Ergon (PLS post‐processing by similarity transformation (PLS + ST): a simple alternative to OPLS. J. Chemom. 2005; 19 : 1–4), which shows that conventional PLS and OPLS are equivalent within a similarity transform. Copyright © 2009 John Wiley & Sons, Ltd.     

Computational performance and cross‐validation error precision of five PLS algorithms using designed and real data sets

An evaluation of computational performance and precision regarding the cross‐validation error of five partial least squares (PLS) algorithms (NIPALS, modified NIPALS, Kernel, SIMPLS and bidiagonal PLS), available and widely used in the literature, is presented. When dealing with large data sets, computational time is an important issue, mainly in cross‐validation and variable selection. In the present paper, the PLS algorithms are compared in terms of the run time and the relative error in the precision obtained when performing leave‐one‐out cross‐validation using simulated and real data sets. The simulated data sets were investigated through factorial and Latin square experimental designs. The evaluations were based on the number of rows, the number of columns and the number of latent variables. With respect to their performance, the results for both simulated and real data sets have shown that the differences in run time are statistically different. PLS bidiagonal is the fastest algorithm, followed by Kernel and SIMPLS. Regarding cross‐validation error, all algorithms showed similar results. However, in some situations as, for example, when many latent variables were in question, discrepancies were observed, especially with respect to SIMPLS. Copyright © 2010 John Wiley & Sons, Ltd.     

X‐tended target projection (XTP)—comparison with orthogonal partial least squares (OPLS) and PLS post‐processing by similarity transformation (PLS + ST)

Target projection (TP) also called target rotation (TR) was introduced to facilitate interpretation of latent‐variable regression models. Orthogonal partial least squares (OPLS) regression and PLS post‐processing by similarity transform (PLS + ST) represent two alternative algorithms for the same purpose. In addition, OPLS and PLS + ST provide components to explain systematic variation in X orthogonal to the response. We show, that for the same number of components, OPLS and PLS + ST provide score and loading vectors for the predictive latent variable that are the same as for TP except for a scaling factor. Furthermore, we show how the TP approach can be extended to become a hybrid of latent‐variable (LV) regression and exploratory LV analysis and thus embrace systematic variation in X unrelated to the response. Principal component analysis (PCA) of the residual variation after removal of the target component is here used to extract the orthogonal components, but X‐tended TP (XTP) permits other criteria for decomposition of the residual variation. If PCA is used for decomposing the orthogonal variation in XTP, the variance of the major orthogonal components obtained for OPLS and XTP is observed to be almost the same, showing the close relationship between the methods. The XTP approach is tested and compared with OPLS for a three‐component mixture analyzed by infrared spectroscopy and a multicomponent mixture measured by near infrared spectroscopy in a reactor. Copyright © 2008 John Wiley & Sons, Ltd.     

Bilinear modelling of batch processes. Part II: a comparison of PLS soft‐sensors

The on‐line monitoring of batch processes based on principal component analysis (PCA) has been widely studied. Nonetheless, researchers have not paid so much attention to the on‐line application of partial least squares (PLS). In this paper, the influence of several issues in the predictive power of a PLS model for the on‐line estimation of key variables in a batch process is studied. Some of the conclusions can help to better understand the capabilities of the proposals presented for on‐line PCA‐based monitoring. Issues like the convenience of batch‐wise or variable‐wise unfolding, the method for the imputation of future measurements and the use of several sub‐models are addressed. This is the first time that the adaptive hierarchical (or multi‐block) approach is extended to the PLS modelling. Also, the formulation of the so‐called trimmed scores regression (TSR), a powerful imputation method defined for PCA, is extended for its application with PLS modelling. Data from two processes, one simulated and one real, are used to illustrate the results. Copyright © 2008 John Wiley & Sons, Ltd.     

Diagnostics of calibration methods: model adequacy of UV‐based determinations

Models such as ordinary least squares, independent component analysis, principle component analysis, partial least squares, and artificial neural networks can be found in the calibration literature. Linear or nonlinear methods can be used to explain the structure of the same phenomenon. Each type of model has its own advantages with respect to the other. These methods are usually grouped taxonomically, but different models can sometimes be applied to the same data set. Taxonomically, ordinary least square and artificial neural network use completely different analytical procedures but are occasionally applied to the same data set. The aim of the study of methodological superiority is to compare the residuals of models because the model with the minimum error is preferred in real analyses. Calibration models, in general, are based on deterministic and stochastic parts; in other words, the data are equal to the model + the error. Explaining a model solely using statistics such as the coefficient of determination or its related significance values is sometimes inadequate. The errors of a model, also called its residuals, must have minimum variance compared to its alternatives. Additionally, the residuals must be unpredictable, uncorrelated, and symmetric. Under these conditions, the model can be considered adequate. In this study, calibration methods were applied to the raw materials, hydrochlorothiazide and amiloride hydrochloride, of a drug, as well as a sample of the drug tablet. The applied chemical procedure was fast, simple, and reproducible. The various linear and nonlinear calibration methods mentioned above were applied, and the adequacy of the calibration methods was compared according to their residuals.     

Genetic algorithm‐based wavelength selection method for spectral calibration

In this paper, we propose a genetic algorithm‐based wavelength selection (GAWLS) method for visible and near‐infrared (Vis/NIR) spectral calibration. The objective of GAWLS is to construct robust and predictive regression models by selecting informative wavelength regions. To demonstrate the ability of the proposed method, regression models for soil properties and sugar content of apples are constructed by using GAWLS and other variable selection methods. Copyright © 2010 John Wiley & Sons, Ltd.     

Evaluation of smoothing techniques in the run to run optimization of fed‐batch processes with u‐PLS

Run to run (R2R) optimization based on unfolded Partial Least Squares (u‐PLS) is a promising approach for improving the performance of batch and fed‐batch processes as it is able to continuously adapt to changing processing conditions. Using this technique, the regression coefficients of PLS are used to modify the input profile of the process in order to optimize the yield. When this approach was initially proposed, it was observed that the optimization performed better when PLS was combined with a smoothing technique, in particular a sliding window filtering, which constrained the regression coefficients to be smooth. In the present paper, this result is further investigated and some modifications to the original approach are proposed. Also, the suitability of different smoothing techniques in combination with PLS is studied for both end‐of‐batch quality prediction and R2R optimization. The smoothing techniques considered in this paper include the original filtering approach, the introduction of smoothing constraints in the PLS calibration (Penalized PLS), and the use of functional analysis (Functional PLS). Two fed‐batch process simulators are used to illustrate the results. Copyright © 2015 John Wiley & Sons, Ltd.     

PLS‐DA for compositional data with application to metabolomics

When quantifying information in metabolomics, the results are often expressed as data carrying only relative information. Vectors of these data have positive components, and the only relevant information is contained in the ratios between their parts; such observations are called compositional data. The aim of the paper is to demonstrate how partial least squares discriminant analysis (PLS‐DA)—the most widely used method in chemometrics for multivariate classification—can be applied to compositional data. Theoretical arguments are provided, and data sets from metabolomics are investigated. The data are related to the diagnosis of inherited metabolic disorders (IMDs). The first example analyzes the significance of the corresponding regression parameters (metabolites) using a small data set resulting from targeted metabolomics, where just a subset of potential markers is selected. The second example—the approach of untargeted metabolomics—was used for the analysis detecting almost 500 metabolites. The significance of the metabolites is investigated by applying PLS‐DA, accommodated according to a compositional approach. The significance of important metabolites (markers of diseases) is more clearly visible with the compositional method in both examples. Also, cross‐validation methods lead to better results in case of using the compositional approach. Copyright © 2014 John Wiley & Sons, Ltd.     

A co‐training algorithm for multi‐view data with applications in data fusion

In several scientific applications, data are generated from two or more diverse sources (views) with the goal of predicting an outcome of interest. Often it is the case that the outcome is not associated with any single view. However, the synergy of all measurements from each view may yield a more predictive classifier. For example, consider a drug discovery application in which individual molecules are described partially by several assay screens based on diverse profiles and partially by their chemical structural fingerprints. A common classification problem is to determine whether the molecule is associated with a particular disease. In this paper, a co‐training algorithm is developed to utilize data from diverse sources to predict the common class variable. Novel enhancements for variable importance, robustness to a mislabeled class variable, and a technique to handle unbalanced classes are applied to the motivating data set, highlighting that the approach attains strong performance and provides useful diagnostics for data analytic purposes. In addition, comparisons to a framework with data fusion using partial least squares (PLS) are also assessed on real data. An R package for performing the proposed approach is provided as Supporting information. Copyright © 2003 John Wiley & Sons, Ltd.     

Fusion strategies for selecting multiple tuning parameters for multivariate calibration and other penalty based processes: A model updating application for pharmaceutical analysis

New multivariate calibration methods and other processes are being developed that require selection of multiple tuning parameter (penalty) values to form the final model. With one or more tuning parameters, using only one measure of model quality to select final tuning parameter values is not sufficient. Optimization of several model quality measures is challenging. Thus, three fusion ranking methods are investigated for simultaneous assessment of multiple measures of model quality for selecting tuning parameter values. One is a supervised learning fusion rule named sum of ranking differences (SRD). The other two are non-supervised learning processes based on the sum and median operations. The effect of the number of models evaluated on the three fusion rules are also evaluated using three procedures. One procedure uses all models from all possible combinations of the tuning parameters. To reduce the number of models evaluated, an iterative process (only applicable to SRD) is applied and thresholding a model quality measure before applying the fusion rules is also used. A near infrared pharmaceutical data set requiring model updating is used to evaluate the three fusion rules. In this case, calibration of the primary conditions is for the active pharmaceutical ingredient (API) of tablets produced in a laboratory. The secondary conditions for calibration updating is for tablets produced in the full batch setting. Two model updating processes requiring selection of two unique tuning parameter values are studied. One is based on Tikhonov regularization (TR) and the other is a variation of partial least squares (PLS). The three fusion methods are shown to provide equivalent and acceptable results allowing automatic selection of the tuning parameter values. Best tuning parameter values are selected when model quality measures used with the fusion rules are for the small secondary sample set used to form the updated models. In this model updating situation, evaluation of all possible models, thresholding, and iterative SRD performed equivalently for the three fusion rules with TR and PLS performed worse. While the application is model updating, the fusion processes are applicable to other situations requiring selection of multiple tuning parameter values.     

Construction of global and robust near-infrared calibration models based on hybrid calibration sets using Partial Least Squares (PLS) regression

Near-infrared spectroscopy (NIR) models built on a particular instrument are often invalid on other instruments due to spectral inconsistencies between the instruments. In the present work, global and robust NIR calibration models were constructed by partial least square (PLS) regression based on hybrid calibration sets, which are composed of both primary and secondary spectra. Three datasets were used as case studies. The first consisted of 72 radix scutellaria samples measured on two NIR spectrometers with known baicalin content. The second was composed of 80 corn samples measured on two instruments with known moisture, oil, and protein concentrations. The third dataset included 279 primary samples of tobacco with known nicotine content and 78 secondary samples of tobacco with known nicotine concentrations. The effect of the number of secondary spectra in the hybrid calibration sets and the methods for selecting secondary spectra on the PLS model performance were investigated by comparing the results obtained from different calibration sets. This study shows that the global and robust calibration models accurately predicted both primary and secondary samples as long as the ratios of the number of primary spectra to the number of secondary spectra were less than 22. The models performance was not influenced by the selection method of the secondary spectra. The hybrid calibration sets included the primary spectral information and also the secondary spectra; information, rendering the constructed global and robust models applicable to both primary and secondary instruments.     

Integration of Commercial Screen‐printed Electrodes into a Voltammetric Electronic Tongue for the Analysis of Aminothiols

A voltammetric sensor array (or electronic tongue) is developed for the simultaneous quantification of cysteine, glutathione and homocysteine without need of previous separation. It is based on the integration of three commercial screen‐printed electrodes (gold curated at high and low temperature and carbon modified with carbon nanotubes). Linear sweep voltammograms measured simultaneously by all three sensors are processed by Partial Least Squares (PLS) regression and different variables selection algorithms such as Genetic Algorithm and interval‐Partial Least Squares. The method was applied to synthetic mixtures and successfully validated, with correlation coefficients of prediction (R_p²) of 0.9542, 0.9429 and 0.9589 for cysteine, glutathione, and homocysteine respectively.     

Improvement of residual bilinearization by particle swarm optimization for achieving the second‐order advantage with unfolded partial least‐squares

The combination of unfolded partial least‐squares (U‐PLS) with residual bilinearization (RBL) provides a second‐order multivariate calibration method capable of achieving the second‐order advantage. RBL is performed by varying the test sample scores in order to minimize the residues of a combined U‐PLS model for the calibrated components and a principal component model for the potential interferents. The sample scores are then employed to predict the analyte concentration, with regression coefficients taken from the calibration step. When the contribution of multiple potential interferents is severe, particle swarm optimization (PSO) helps in preventing RBL to be trapped by false minima, restoring its predictive ability and making it comparable to the standard parallel factor (PARAFAC) analysis. Both simulated and experimental systems are analyzed in order to show the potentiality of the new technique. Copyright © 2007 John Wiley & Sons, Ltd.     

Sorting variables by using informative vectors as a strategy for feature selection in multivariate regression

A new procedure with high ability to enhance prediction of multivariate calibration models with a small number of interpretable variables is presented. The core of this methodology is to sort the variables from an informative vector, followed by a systematic investigation of PLS regression models with the aim of finding the most relevant set of variables by comparing the cross‐validation parameters of the models obtained. In this work, seven main informative vectors i.e. regression vector, correlation vector, residual vector, variable influence on projection (VIP), net analyte signal (NAS), covariance procedures vector (CovProc), signal‐to‐noise ratios vector (StN) and their combinations were automated and tested with the main purpose of feature selection. Six data sets from different sources were employed to validate this methodology. They originated from: near‐Infrared (NIR) spectroscopy, Raman spectroscopy, gas chromatography (GC), fluorescence spectroscopy, quantitative structure‐activity relationships (QSAR) and computer simulation. The results indicate that all vectors and their combinations were able to enhance prediction capability with respect to the full data sets. However, regression and NAS informative vectors from partial least squares (PLS) regression, both built using more latent variables than when building the model presented in most of tested data sets, were the best informative vectors for variable selection. In all the applications, the selected variables were quite effective and useful for interpretation. Copyright © 2008 John Wiley & Sons, Ltd.     

Genetic‐Algorithm‐Based Wavelength Selection in Multicomponent Spectrophotometric Determination by PLS: Application on Ascorbic Acid and Uric Acid Mixture

Genetic algorithm (GA) is a suitable method for selecting wavelengths for partial least squares (PLS) calibration of mixtures with almost identical spectra without loss of prediction capacity using the spectrophotometric method. In this study, the concentration model is based on absorption spectra in the range of 200‐320 nm for 25 different mixtures of ascorbic acid (AA) and uric acid (UA). The calibration curve was linear over the concentration range of 1‐15 and 2‐16 μg mL^?1 for ascorbic acid and uric acid, respectively. The root mean square deviation (RMSD) for ascorbic acid and uric acid with GA and without GA were 0.3071 and 0.3006, 0.3971 and 0.7063, respectively. The proposed method was successfully applied to the simultaneous determination of both analytes in human serum and urine samples.     

A wavelength selection method based on random decision particle swarm optimization with attractor for near‐infrared spectral quantitative analysis

In this paper, we proposed a wavelength selection method based on random decision particle swarm optimization with attractor for near‐infrared (NIR) spectra quantitative analysis. The proposed method was incorporated with partial least square (PLS) to construct a prediction model. The proposed method chooses the current own optimal or the current global optimal to calculate the attractor. Then the particle updates its flight velocity by the attractor, and the particle state is updated by the random decision with the new velocity. Moreover, the root‐mean‐square error of cross‐validation is adopted as the fitness function for the proposed method. In order to demonstrate the usefulness of the proposed method, PLS with all wavelengths, uninformative variable elimination by PLS, elastic net, genetic algorithm combined with PLS, the discrete particle swarm optimization combined with PLS, the modified particle swarm optimization combined with PLS, the neighboring particle swarm optimization combined with PLS, and the proposed method are used for building the components quantitative analysis models of NIR spectral datasets, and the effectiveness of these models is compared. Two application studies are presented, which involve NIR data obtained from an experiment of meat content determination using NIR and a combustion procedure. Results verify that the proposed method has higher predictive ability for NIR spectral data and the number of selected wavelengths is less. The proposed method has faster convergence speed and could overcome the premature convergence problem. Furthermore, although improving the prediction precision may sacrifice the model complexity under a certain extent, the proposed method is overfitted slightly. Copyright © 2015 John Wiley & Sons, Ltd.