Erratum: On the numerical stability of two widely used PLS algorithms

The above article (DOI: 10.1002/cem.1112) was published online on 14 February 2008. An error was subsequently identified: the captions for Figures 1 and 2 were omitted; they should read as follows: Figure 1. Orthogonality criterion (θ_A) for the octane data as a function of number of components (A) calculated using the standard PLS algorithm and SIMPLS. Figure 2. Orthogonality criterion (θ_A) for the wines data as a function of number of components (A) calculated using the standard PLS algorithm and SIMPLS.     

A comparison of nine PLS1 algorithms

Nine PLS1 algorithms were evaluated, primarily in terms of their numerical stability, and secondarily their speed. There were six existing algorithms: (a) NIPALS by Wold; (b) the non‐orthogonalized scores algorithm by Martens; (c) Bidiag2 by Golub and Kahan; (d) SIMPLS by de Jong; (e) improved kernel PLS by Dayal; and (f) PLSF by Manne. Three new algorithms were created: (g) direct‐scores PLS1 based on a new recurrent formula for the calculation of basis vectors yielding scores directly from X and y; (h) Krylov PLS1 with its regression vector defined explicitly, using only the original X and y; (i) PLSPLS1 with its regression vector recursively defined from X and the regression vectors of its previous recursions. Data from IR and NIR spectrometers applied to food, agricultural, and pharmaceutical products were used to demonstrate the numerical stability. It was found that three methods (c, f, h) create regression vectors that do not well resemble the corresponding precise PLS1 regression vectors. Because of this, their loading and score vectors were also concluded to be deviating, and their models of X and the corresponding residuals could be shown to be numerically suboptimal in a least squares sense. Methods (a, b, e, g) were the most stable. Two of them (e, g) were not only numerically stable but also much faster than methods (a, b). The fast method (d) and the moderately fast method (i) showed a tendency to become unstable at high numbers of PLS factors. Copyright © 2009 John Wiley & Sons, Ltd.     

The geometry of PLS1 explained properly: 10 key notes on mathematical properties of and some alternative algorithmic approaches to PLS1 modelling

The insight from, and conclusions of this paper motivate efficient and numerically robust ‘new’ variants of algorithms for solving the single response partial least squares regression (PLS1) problem. Prototype MATLAB code for these variants are included in the Appendix. The analysis of and conclusions regarding PLS1 modelling are based on a rich and nontrivial application of numerous key concepts from elementary linear algebra. The investigation starts with a simple analysis of the nonlinear iterative partial least squares (NIPALS) PLS1 algorithm variant computing orthonormal scores and weights. A rigorous interpretation of the squared P ‐loadings as the variable‐wise explained sum of squares is presented. We show that the orthonormal row‐subspace basis of W ‐weights can be found from a recurrence equation. Consequently, the NIPALS deflation steps of the centered predictor matrix can be replaced by a corresponding sequence of Gram–Schmidt steps that compute the orthonormal column‐subspace basis of T ‐scores from the associated non‐orthogonal scores. The transitions between the non‐orthogonal and orthonormal scores and weights (illustrated by an easy‐to‐grasp commutative diagram), respectively, are both given by QR factorizations of the non‐orthogonal matrices. The properties of singular value decomposition combined with the mappings between the alternative representations of the PLS1 ‘truncated’ X data (including P ^t W ) are taken to justify an invariance principle to distinguish between the PLS1 truncation alternatives. The fundamental orthogonal truncation of PLS1 is illustrated by a Lanczos bidiagonalization type of algorithm where the predictor matrix deflation is required to be different from the standard NIPALS deflation. A mathematical argument concluding the PLS1 inconsistency debate (published in 2009 in this journal) is also presented. Copyright © 2014 John Wiley & Sons, Ltd.     

Comparison of the numerical stability of methods for anharmonic calculations of vibrational molecular energies

On model examples, we compare the performance of the vibrational self-consistent field, variational, and four perturbational schemes used for computations of vibrational energies of semi-rigid molecules, with emphasis on the numerical stability. Although the accuracy of the energies is primarily dependent on the quality of the potential energy surface, approximate approaches to the anharmonic vibrational problem often do not converge to the same results due to the approximations involved. For furan, the sensitivity to variations of the anharmonic potential was systematically investigated by adding random noise to the cubic and quartic constants. The self-consistent field methods proved to be the most resistant to the potential variations. The second order perturbational techniques are sensitive to random degeneracies and provided the least stable results. However, their stability could be significantly improved by a simple generalization of the perturbational formula. The variational configuration interaction is practically limited by the size of the matrix that can be diagonalized for larger molecules; however, relatively fewer states need to be involved than for smaller ones, in favor of the computing.     

Evaluation of smoothing techniques in the run to run optimization of fed‐batch processes with u‐PLS

Run to run (R2R) optimization based on unfolded Partial Least Squares (u‐PLS) is a promising approach for improving the performance of batch and fed‐batch processes as it is able to continuously adapt to changing processing conditions. Using this technique, the regression coefficients of PLS are used to modify the input profile of the process in order to optimize the yield. When this approach was initially proposed, it was observed that the optimization performed better when PLS was combined with a smoothing technique, in particular a sliding window filtering, which constrained the regression coefficients to be smooth. In the present paper, this result is further investigated and some modifications to the original approach are proposed. Also, the suitability of different smoothing techniques in combination with PLS is studied for both end‐of‐batch quality prediction and R2R optimization. The smoothing techniques considered in this paper include the original filtering approach, the introduction of smoothing constraints in the PLS calibration (Penalized PLS), and the use of functional analysis (Functional PLS). Two fed‐batch process simulators are used to illustrate the results. Copyright © 2015 John Wiley & Sons, Ltd.     

Selection of individual variables versus intervals of variables in PLSR

The selection abilities of the two well‐known techniques of variable selection, synergy interval‐partial least‐squares (SiPLS) and genetic algorithm‐partial least‐squares (GA‐PLS), have been examined and compared. By using different simulated and real (corn and metabolite) datasets, keeping in view the spectral overlapping of the components, the influence of the selection of either intervals of variables or individual variables on the prediction performances was examined. In the simulated datasets, with decrease in the overlapping of the spectra of components and cases with components of narrow bands, GA‐PLS results were better. In contrast, the performance of SiPLS was higher for data of intermediate overlapping. For mixtures of high overlapping analytes, GA‐PLS showed slightly better performance. However, significant differences between the results of the two selection methods were not observed in most of the cases. Although SiPLS resulted in slightly better performance of prediction in the case of corn dataset except for the prediction of the moisture content, the improvement obtained by SiPLS compared with that by GA‐PLS was not significant. For real data of less overlapped components (metabolite dataset), GA‐PLS that tends to select far fewer variables did not give significantly better root mean square error of cross‐validation (RMSECV), cross‐validated R² (Q²), and root mean square error of prediction (RMSEP) compared with SiPLS. Irrespective of the type of dataset, GA‐PLS resulted in models with fewer latent variables (LVs). When comparing the computational time of the methods, GA‐PLS is considered superior to SiPLS. Copyright © 2010 John Wiley & Sons, Ltd.     

Genetic Algorithm Based Potential Selection in Simultaneous Voltammetric Determination of Isoniazid and Hydrazine by Using Partial Least Squares (PLS) and Artificial Neural Networks (ANNs)

The voltammetric behavior of isoniazid and hydrazine at an overoxidized polypyrrole modified glassy carbon electrode has been investigated. The obtained cyclic voltammograms showed that their oxidation peaks were overlapped and it is difficult to determine them individually from a mixture without separation. To overcome this limitation, a procedure was proposed for resolution of overlapped voltammetric signals from mixtures of isoniazid and hydrazine. In this procedure, genetic algorithm was used for the selection of potentials for partial least squares. A feed forward artificial neural network with back propagation error algorithm was used to process the nonlinear relationship between currents and concentrations of hydrazine and isoniazid. The proposed method was suitable for determination of isoniazid in pharmaceutical tablets and detection of hydrazine impurities in the same samples.     

Post‐transformation of PLS2 (ptPLS2) by orthogonal matrix: a new approach for generating predictive and orthogonal latent variables

Partial Least Squares (PLS) is a wide class of regression methods aiming at modelling relationships between sets of observed variables by means of latent variables. Specifically, PLS2 was developed to correlate two blocks of data, the X‐block representing the independent or explanatory variables and the Y‐block representing the dependent or response variables. Lately, OPLS was introduced to further reduce model complexity by removing Y‐orthogonal sources of variation from X in the latent space, thus improving data interpretation through the generated predictive latent variables. Nevertheless, relationships between PLS2 and OPLS in case of multiple Y‐response have not yet been fully explored. With this perspective and taking inspiration from some basic mathematical properties of PLS2, we here present a novel and general approach consisting in a post‐transformation of PLS2 (ptPLS2), which results in a decomposition of the latent space into orthogonal and predictive components, while preserving the same goodness of fit and predictive ability of PLS2. Additionally, we discuss the application of ptPLS2 approach to two metabolomic data sets extracted from earlier published studies and its advantages in model interpretation as compared with the ‘standard’ PLS approach. Copyright © 2016 John Wiley & Sons, Ltd.     

On the lattice stability of metals

A tight-binding calculation for body-centred cubic (bcc) and face-centred cubic (fcc) lithium is carried out using muffin-tin potentials which differ only in the arrangement of the muffin-tin spheres. The essential results are not restricted to lithium but also hold for other metals with similar s-p-bands. The bcc structure can be stable for the lowest valence states. The stability of fcc increases with increasing valence electron concentration. This is due to the kinetic energy which behaves as in an empty lattice case. In accordance with the behaviour of the kinetic energy, the lowest energy states have the highest distribution probability between neighbouring atoms. They are mostly delocalized. The trend in the lattice stability is explained in terms of the differences in the packing of the lattices. In real cases where the virial theorem holds an appropriate part of the kinetic energy is changed into potential energy. Hybridization plays a completely different role from that in covalent compounds. It stabilizes a compound by delocalizing the charge density.     

An evaluation of the radial part of numerical integration commonly used in DFT

Recently, Gill and Chien introduced a new radial quadrature for multiexponential integrands (MultiExp grid) to deal with the radial part of the numerical integration. In this article, the MultiExp grid is studied and used to integrate the charge density. The MultiExp grid, along with an optimal pruning scheme, performed very well both in terms of accuracy and efficiency compared to other radial mappings commonly used in Density Functional Theory.     

Comparison study of several numerical integration schemes used in calculations of density functional theory

Several numerical integration schemes for the evaluation of matrix elements in density functional theory calculations have been studied and compared by computational practice. The best scheme was found to be the combination of the atomic partition function proposed by Becke with the scaled generalized Gauss-Laguerre quadrature formula for radial integration suggested by Yang, which achieve the highest convergence rate to the numerical integration. With the same number of integration points, the accuracy of the calculated results by this scheme is higher by 1 to 2 orders of magnitudes than that by other schemes. The reason for achieving higher accuracy by this scheme has been proposed preliminarily.     

Simultaneous kinetic spectrophotometric determination of cyanide and thiocyanate using the partial least squares (PLS) regression

The partial least squares (PLS-1) calibration model based on spectrophotometric measurement, for the simultaneous determination of CN⁻ and SCN⁻ ions is described. The method is based on the difference in the rate of the reaction between CN⁻ and SCN⁻ ions with chloramine-T in a pH 4.0 buffer solution and at 30 °C. The produced cyanogen chloride (CNCl) reacts with pyridine and the product condenses with barbituric acid and forms a final colored product. The absorption kinetic profiles of the solutions were monitored by measuring absorbance at 578 nm in the time range 20-180 s after initiation of the reaction with 2 s intervals. The experimental calibration matrix for partial least squares (PLS-1) calibration was designed with 31 samples. The cross-validation method was used for selecting the number of factors. The results showed that simultaneous determination could be performed in the range 10.0-900.0 and 50.0-1200.0 ng mL⁻¹ for CN⁻ and SCN⁻ ions, respectively. The proposed method was successfully applied to the simultaneous determination of cyanide and thiocyanate in water samples.     

On the nature of DNA-duplex stability

The unwinding free energy of 128 DNA octamers was correlated with the sum of interaction energies among DNA bases and their solvation energies. The former energies were determined by using the recently developed density functional theory procedure augmented by London dispersion energy (RI-DFT-D) that provides accurate hydrogen-bonding and stacking energies highly comparable with CCSD(T)/complete basis set limit benchmark data. Efficient tight-binding DFT covering dispersion energy was also used and yielded satisfactory results. The latter method can be used for extended systems. The solvation energy was determined by using a C-PCM continuum solvent at HF level calculations. Various models were adopted to correlate theoretical energies with experimental unwinding free energies. Unless all energy components (hydrogen-bonding, intra- and interstrand-stacking, and solvation energies) were included and weighted individually, no satisfactory correlation resulted. The most advanced model yielded very close correlation (RMSE=0.32 kcal mol(-1)) fully comparable with the entirely empirical correlation introduced in the original paper. Analysis of the theoretical results shows the importance of inter- and intramolecular stacking energies, and especially the latter term plays a key role in determining DNA-duplex stabilization.     

Automated parallel anionic polymerizations: Enhancing the possibilities of a widely used technique in polymer synthesis

Anionic polymerization techniques have been implemented successfully in a commercial automated synthesizer. The main problems for a successful adaptation of the experimental technique in the automated synthesizer are addressed, as well as some simple potential applications, such as the anionic polymerization of styrene, isoprene, and methyl methacrylate. The obtained results were reproducible and in concordance with literature knowledge. The apparent rate constant of the anionic polymerization of styrene in cyclohexane initiated by sec‐butyllithium could be determined at two different concentrations of the monomer and initiator in a temperature range of 10–60 °C. All the synthesis and characterization experiments of the polymers were performed within a short time period. Moreover, the syntheses of poly(styrene‐b‐isoprene) and poly(styrene‐b‐methyl methacrylate) block copolymers were also successfully carried out within the automated synthesizer. © 2005 Wiley Periodicals, Inc. J Polym Sci Part A: Polym Chem 43: 4151–4160, 2005     

On generalized Borgen plots II: The line‐moving algorithm and its numerical implementation

Borgen plots are geometric constructions that represent the set of all nonnegative factorizations of spectral data matrices for three‐component systems. The classical construction by Borgen and Kowalski (Anal. Chim. Acta 174, 1‐26 (1985)) is limited to nonnegative data and results in nonnegative factorizations. The new approach of generalized Borgen plots allows factors with small negative entries. This makes it possible to construct Borgen plots for perturbed or noisy spectral data and stabilizes the computation. In the first part of this paper, the mathematical theory of generalized Borgen plots has been introduced. This second part presents the line‐moving algorithm for the construction of generalized Borgen plots. The algorithm is justified, and the implementation in the FACPACK software is validated.     

Boosted leave-many-out cross-validation: the effect of training and test set diversity on PLS statistics

It is becoming increasingly common in quantitative structure/activity relationship (QSAR) analyses to use external test sets to evaluate the likely stability and predictivity of the models obtained. In some cases, such as those involving variable selection, an internal test set – i.e., a cross-validation set – is also used. Care is sometimes taken to ensure that the subsets used exhibit response and/or property distributions similar to those of the data set as a whole, but more often the individual observations are simply assigned `at random.' In the special case of MLR without variable selection, it can be analytically demonstrated that this strategy is inferior to others. Most particularly, D-optimal design performs better if the form of the regression equation is known and the variables involved are well behaved. This report introduces an alternative, non-parametric approach termed `boosted leave-many-out' (boosted LMO) cross-validation. In this method, relatively small training sets are chosen by applying optimizable k-dissimilarity selection (OptiSim) using a small subsample size (k = 4, in this case), with the unselected observations being reserved as a test set for the corresponding reduced model. Predictive errors for the full model are then estimated by aggregating results over several such analyses. The countervailing effects of training and test set size, diversity, and representativeness on PLS model statistics are described for CoMFA analysis of a large data set of COX2 inhibitors.     

Parallel factor analysis combined with PLS regression applied to the on-line monitoring of Pichia pastoris cultures

Various key variables (biomass, substrate and product) of bioprocesses should be monitored in order to retrieve useful information on the system, with the biomass (the cell density) the principal target. Although several analytical methods have been adapted and used to monitor the evolution of cell density evolution in cultures, a general method for performing this determination has not yet been established, as each technique has its own advantages and drawbacks. In the present work, noninduced glycerol batch cultures (for which biomass and substrate are the key variables) were monitored using multiwavelength fluorescence spectroscopy. The data gathered were modelled via PARAFAC-PLS chemometric methodologies, resulting in important qualitative and quantitative information about the behaviours of different biogenic fluorophors in batch cultures of the yeast Pichia pastoris. This information was used to predict the target process variables in such cultures; this permitted the applicability of this combined technique to bioprocess monitoring to be assessed.