Bilinear modelling of batch processes. Part I: theoretical discussion

When studying the principal component analysis (PCA) or partial least squares (PLS) modelling of batch process data, one realizes that there is a wide range of approaches. In many cases, new modelling approaches are presented just because they work properly for a particular application, for example, on‐line monitoring and a given number of processes. A clear understanding of why these approaches perform successfully and which are the advantages and disadvantages in front of the others is seldom supplied. Why does modelling after batch‐wise unfolding capture changing dynamics? What are the consequences of variable‐wise unfolding? Is there any best unfolding method? When should several models for a single process be used? In this paper, it is shown how these and other related questions can be answered by properly analyzing the dynamic covariance structures of the various approaches. Copyright © 2008 John Wiley & Sons, Ltd.     

Algorithms for multi‐group PLS

Several approaches of investigation of the relationships between two datasets where the individuals are structured into groups are discussed. These strategies fit within the framework of partial least squares (PLS) regression. Each strategy of analysis is introduced on the basis of a maximization criterion, which involves the covariances between components associated with the groups of individuals in each dataset. Thereafter, algorithms are proposed to solve these maximization problems. The strategies of analysis can be considered as extensions of multi‐group principal components analysis to the context of PLS regression. Copyright © 2014 John Wiley & Sons, Ltd.     

Evaluation of smoothing techniques in the run to run optimization of fed‐batch processes with u‐PLS

Run to run (R2R) optimization based on unfolded Partial Least Squares (u‐PLS) is a promising approach for improving the performance of batch and fed‐batch processes as it is able to continuously adapt to changing processing conditions. Using this technique, the regression coefficients of PLS are used to modify the input profile of the process in order to optimize the yield. When this approach was initially proposed, it was observed that the optimization performed better when PLS was combined with a smoothing technique, in particular a sliding window filtering, which constrained the regression coefficients to be smooth. In the present paper, this result is further investigated and some modifications to the original approach are proposed. Also, the suitability of different smoothing techniques in combination with PLS is studied for both end‐of‐batch quality prediction and R2R optimization. The smoothing techniques considered in this paper include the original filtering approach, the introduction of smoothing constraints in the PLS calibration (Penalized PLS), and the use of functional analysis (Functional PLS). Two fed‐batch process simulators are used to illustrate the results. Copyright © 2015 John Wiley & Sons, Ltd.     

Cross fitted partial least squares (CF‐PLS): an alternative algorithm for a more reliable PLS

This paper presents a modified version of the NIPALS algorithm for PLS regression with one single response variable. This version, denoted a CF‐PLS, provides significant advantages over the standard PLS. First of all, it strongly reduces the over‐fit of the regression. Secondly, R² for the null hypothesis follows a Beta distribution only function of the number of observations, which allows the use of a probabilistic framework to test the validity of a component. Thirdly, the models generated with CF‐PLS have comparable if not better prediction ability than the models fitted with NIPALS. Finally, the scores and loadings of the CF‐PLS are directly related to the R², which makes the model and its interpretation more reliable. Copyright © 2011 John Wiley & Sons, Ltd.     

Co‐evolution of non‐linear PLS model components

The issue of outer model weight updating is important in extending partial least squares (PLS) regression to modelling data that shows significant non‐linearity. This paper presents a novel co‐evolutionary component approach to the weight updating problem. Specification of the non‐linear PLS model is achieved using an evolutionary computational (EC) method that can co‐evolve all non‐linear inner models and all input projection weights simultaneously. In this method, modular symbolic non‐linear equations are used to represent the inner models and binary sequences are used to represent the projection weights. The approach is flexible, and other representations could be employed within the same co‐evolutionary framework. The potential of these methods is illustrated using a simulated pH neutralisation process data set exhibiting significant non‐linearity. It is demonstrated that the co‐evolutionary component architecture can produce results which are competitive with non‐linear neural network‐based PLS algorithms that use iterative projection weight updating. In addition, a data sampling method for mitigating overfitting to the training data is described. Copyright © 2007 John Wiley & Sons, Ltd.     

Fault detection and classification for a two‐stage batch process

Multiway principal component analysis (MPCA) has been extensively applied to batch process monitoring. In the case of monitoring a two‐stage batch process, the inter‐stage variation is neglected if MPCA models for each individual stage are used. On the other hand, if two stages of reference data are combined into a large dataset that MPCA is applied to, the dimensions of the unfolded matrix will increase dramatically. In addition, when an abnormal event is detected, it is difficult to identify which stage's operation induced this alarm. In this paper, partial least squares (PLS) is applied to monitor the inter‐stage relation of a two‐stage batch process. In post‐analysis of abnormalities, PLS can clarify whether root causes are from previous stage operations or due to the changes of the inter‐stage correlations. This approach was successfully applied to a semiconductor manufacturing process. Copyright © 2008 John Wiley & Sons, Ltd.     

OPLS filtered data can be obtained directly from non‐orthogonalized PLS1

The well‐known Martens factorization for PLS1 produces a single y‐related score, with all subsequent scores being y‐unrelated. The X‐explanatory value of these y‐orthogonal scores can be summarized by a simple expression, which is analogous to the ‘P’ loading weights in the orthogonalized NIPALS algorithm. This can be used to rearrange the factorization into entirely y‐related and y‐unrelated parts. Systematic y‐unrelated variation can thus be removed from the X data through a single post hoc calculation following conventional PLS, without any recourse to the orthogonal projections to latent structures (OPLS) algorithm. The work presented is consistent with the development by Ergon (PLS post‐processing by similarity transformation (PLS + ST): a simple alternative to OPLS. J. Chemom. 2005; 19 : 1–4), which shows that conventional PLS and OPLS are equivalent within a similarity transform. Copyright © 2009 John Wiley & Sons, Ltd.     

Bilinear modeling of batch processes. Part III: parameter stability

A paramount aspect in the development of a model for a monitoring system is the so‐called parameter stability. This is inversely related to the uncertainty, i.e., the variance in the parameters estimates. Noise affects the performance of the monitoring system, reducing its fault detection capability. Low parameters uncertainty is desired to ensure a reduced amount of noise in the model. Nonetheless, there is no sound study on the parameter stability in batch multivariate statistical process control (BMSPC). The aim of this paper is to investigate the parameter stability associated to the most used synchronization and principal component analysis‐based BMSPC methods. The synchronization methods included in this study are the following: indicator variable, dynamic time warping, relaxed greedy time warping, and time linear expanding/compressing‐based. In addition, different arrangements of the three‐way batch data into two‐way matrices are considered, namely single‐model, K‐models, and hierarchical‐model approaches. Results are discussed in connection with previous conclusions in the first two papers of the series. Copyright © 2013 John Wiley & Sons, Ltd.     

A co‐training algorithm for multi‐view data with applications in data fusion

In several scientific applications, data are generated from two or more diverse sources (views) with the goal of predicting an outcome of interest. Often it is the case that the outcome is not associated with any single view. However, the synergy of all measurements from each view may yield a more predictive classifier. For example, consider a drug discovery application in which individual molecules are described partially by several assay screens based on diverse profiles and partially by their chemical structural fingerprints. A common classification problem is to determine whether the molecule is associated with a particular disease. In this paper, a co‐training algorithm is developed to utilize data from diverse sources to predict the common class variable. Novel enhancements for variable importance, robustness to a mislabeled class variable, and a technique to handle unbalanced classes are applied to the motivating data set, highlighting that the approach attains strong performance and provides useful diagnostics for data analytic purposes. In addition, comparisons to a framework with data fusion using partial least squares (PLS) are also assessed on real data. An R package for performing the proposed approach is provided as Supporting information. Copyright © 2003 John Wiley & Sons, Ltd.     

Computational performance and cross‐validation error precision of five PLS algorithms using designed and real data sets

An evaluation of computational performance and precision regarding the cross‐validation error of five partial least squares (PLS) algorithms (NIPALS, modified NIPALS, Kernel, SIMPLS and bidiagonal PLS), available and widely used in the literature, is presented. When dealing with large data sets, computational time is an important issue, mainly in cross‐validation and variable selection. In the present paper, the PLS algorithms are compared in terms of the run time and the relative error in the precision obtained when performing leave‐one‐out cross‐validation using simulated and real data sets. The simulated data sets were investigated through factorial and Latin square experimental designs. The evaluations were based on the number of rows, the number of columns and the number of latent variables. With respect to their performance, the results for both simulated and real data sets have shown that the differences in run time are statistically different. PLS bidiagonal is the fastest algorithm, followed by Kernel and SIMPLS. Regarding cross‐validation error, all algorithms showed similar results. However, in some situations as, for example, when many latent variables were in question, discrepancies were observed, especially with respect to SIMPLS. Copyright © 2010 John Wiley & Sons, Ltd.     

X‐tended target projection (XTP)—comparison with orthogonal partial least squares (OPLS) and PLS post‐processing by similarity transformation (PLS + ST)

Target projection (TP) also called target rotation (TR) was introduced to facilitate interpretation of latent‐variable regression models. Orthogonal partial least squares (OPLS) regression and PLS post‐processing by similarity transform (PLS + ST) represent two alternative algorithms for the same purpose. In addition, OPLS and PLS + ST provide components to explain systematic variation in X orthogonal to the response. We show, that for the same number of components, OPLS and PLS + ST provide score and loading vectors for the predictive latent variable that are the same as for TP except for a scaling factor. Furthermore, we show how the TP approach can be extended to become a hybrid of latent‐variable (LV) regression and exploratory LV analysis and thus embrace systematic variation in X unrelated to the response. Principal component analysis (PCA) of the residual variation after removal of the target component is here used to extract the orthogonal components, but X‐tended TP (XTP) permits other criteria for decomposition of the residual variation. If PCA is used for decomposing the orthogonal variation in XTP, the variance of the major orthogonal components obtained for OPLS and XTP is observed to be almost the same, showing the close relationship between the methods. The XTP approach is tested and compared with OPLS for a three‐component mixture analyzed by infrared spectroscopy and a multicomponent mixture measured by near infrared spectroscopy in a reactor. Copyright © 2008 John Wiley & Sons, Ltd.     

Partial least‐squares discriminant analysis optimized by particle swarm optimization: application to 1H nuclear magnetic resonance analysis of lung cancer metabonomics

The complexity of metabolic profiles makes chemometric tools indispensable for extracting the most significant information. Partial least‐squares discriminant analysis (PLS‐DA) acts as one of the most effective strategies for data analysis in metabonomics. However, its actual efficacy in metabonomics is often weakened by the high similarity of metabolic profiles, which contain excessive variables. To rectify this situation, particle swarm optimization (PSO) was introduced to improve PLS‐DA by simultaneously selecting the optimal sample and variable subsets, the appropriate variable weights, and the best number of latent variables (SVWL) in PLS‐DA, forming a new algorithm named PSO‐SVWL‐PLSDA. Combined with ¹H nuclear magnetic resonance‐based metabonomics, PSO‐SVWL‐PLSDA was applied to recognize the patients with lung cancer from the healthy controls. PLS‐DA was also investigated as a comparison. Relatively to the recognition rates of 86% and 65%, which were yielded by PLS‐DA, respectively, for the training and test sets, those of 98.3% and 90% were offered by PSO‐SVWL‐PLSDA. Moreover, several most discriminative metabolites were identified by PSO‐SVWL‐PLSDA to aid the diagnosis of lung cancer, including lactate, glucose (α‐glucose and β‐glucose), threonine, valine, taurine, trimethylamine, glutamine, glycoprotein, proline, and lipid. Copyright © 2015 John Wiley & Sons, Ltd.     

Near‐infrared spectroscopy for the quantitative determination of mass transfer and water absorption kinetics by a polymer solution

This article reports a new method to quantify the water absorption kinetics and the mass transfer in a polymer solution by using near‐infrared (NIR) spectroscopy and partial least‐squares (PLS) models, while it is exposed to a humid atmosphere. Polymer solutions used in this study were made with highly polar solvents exhibiting both a high affinity for water and a low volatility such as dimethylformamide, dimethylacetamide, and N‐methylpyrrolidone. Poly(ethersulfone) and poly(etherimide) were chosen as polymer models as the method could provide useful information for coating process and membrane fabrication monitoring. Whereas gravimetric kinetics yield data on the overall mass transfer, including both water absorption and solvent evaporation, in situ analyses using NIR can quantify separately the solvent and nonsolvent concentration change in the polymer solution. Quantitative models were developed using PLS regression to predict the local water, polymer, and solvent weight fractions in the polymer solution. The method was proved to be suitable for the different studied systems and allowed to infer mass transfers until the onset of the phase separation process. © 2010 Wiley Periodicals, Inc. J Polym Sci Part B: Polym Phys 48: 1960–1969, 2010     

Diagnostics of calibration methods: model adequacy of UV‐based determinations

Models such as ordinary least squares, independent component analysis, principle component analysis, partial least squares, and artificial neural networks can be found in the calibration literature. Linear or nonlinear methods can be used to explain the structure of the same phenomenon. Each type of model has its own advantages with respect to the other. These methods are usually grouped taxonomically, but different models can sometimes be applied to the same data set. Taxonomically, ordinary least square and artificial neural network use completely different analytical procedures but are occasionally applied to the same data set. The aim of the study of methodological superiority is to compare the residuals of models because the model with the minimum error is preferred in real analyses. Calibration models, in general, are based on deterministic and stochastic parts; in other words, the data are equal to the model + the error. Explaining a model solely using statistics such as the coefficient of determination or its related significance values is sometimes inadequate. The errors of a model, also called its residuals, must have minimum variance compared to its alternatives. Additionally, the residuals must be unpredictable, uncorrelated, and symmetric. Under these conditions, the model can be considered adequate. In this study, calibration methods were applied to the raw materials, hydrochlorothiazide and amiloride hydrochloride, of a drug, as well as a sample of the drug tablet. The applied chemical procedure was fast, simple, and reproducible. The various linear and nonlinear calibration methods mentioned above were applied, and the adequacy of the calibration methods was compared according to their residuals.     

Improvement of residual bilinearization by particle swarm optimization for achieving the second‐order advantage with unfolded partial least‐squares

The combination of unfolded partial least‐squares (U‐PLS) with residual bilinearization (RBL) provides a second‐order multivariate calibration method capable of achieving the second‐order advantage. RBL is performed by varying the test sample scores in order to minimize the residues of a combined U‐PLS model for the calibrated components and a principal component model for the potential interferents. The sample scores are then employed to predict the analyte concentration, with regression coefficients taken from the calibration step. When the contribution of multiple potential interferents is severe, particle swarm optimization (PSO) helps in preventing RBL to be trapped by false minima, restoring its predictive ability and making it comparable to the standard parallel factor (PARAFAC) analysis. Both simulated and experimental systems are analyzed in order to show the potentiality of the new technique. Copyright © 2007 John Wiley & Sons, Ltd.     

Multivariate quality control solved by one‐class partial least squares regression: identification of adulterated peanut oils by mid‐infrared spectroscopy

The Partial least squares class model (PLSCM) was recently proposed for multivariate quality control based on a partial least squares (PLS) regression procedure. This paper presents a case study of quality control of peanut oils based on mid‐infrared (MIR) spectroscopy and class models, focusing mainly on the following aspects: (i) to explain the meanings of PLSCM components and make comparisons between PLSCM and soft independent modeling of class analogy (SIMCA); (ii) to correct the estimation of the original PLSCM confidence interval by considering a nonzero intercept term for center estimation; (iii) to investigate the potential of MIR spectroscopy combined with class models for identifying peanut oils with low doping concentrations of other edible oils. It is demonstrated that PLSCM is actually different from the ordinary PLS procedure, but it estimates the class center and class dispersion in the framework of a latent variable projection model. While SIMCA projects the original variables onto a few dimensions explaining most of the data variances, PLSCM components consider simultaneously the explained variances and the compactness of samples belonging to the same class. The analysis results indicate PLSCM is an intuitive and easy‐to‐use tool to tackle one‐class problems and has comparable performance with SIMCA. The advantages of PLSCM might be attributed to the great success and well‐established foundations of PLS. For PLSCM, the optimization of model complexity and estimation of decision region can be performed as in multivariate calibration routines. Copyright © 2011 John Wiley & Sons, Ltd.     

Quantitative determination and evaluation of Paris polyphylla var. yunnanensis with different harvesting times using UPLC‐UV‐MS and FT‐IR spectroscopy in combination with partial least squares discriminant analysis

A rapid method was developed and validated by ultra‐performance liquid chromatography–triple quadrupole mass spectroscopy with ultraviolet detection (UPLC‐UV‐MS) for simultaneous determination of paris saponin I, paris saponin II, paris saponin VI and paris saponin VII. Partial least squares discriminant analysis (PLS‐DA) based on UPLC and Fourier transform infrared (FT‐IR) spectroscopy was employed to evaluate Paris polyphylla var. yunnanensis (PPY) at different harvesting times. Quantitative determination implied that the various contents of bioactive compounds with different harvesting times may lead to different pharmacological effects; the average content of total saponins for PPY harvested at 8 years was higher than that from other samples. The PLS‐DA of FT‐IR spectra had a better performance than that of UPLC for discrimination of PPY from different harvesting times.     

Developing a miniaturized setup for in‐tube simultaneous determination of three alkaloids using electromembrane extraction in combination with ultraviolet spectrophotometry

Herein, electromembrane extraction was combined with ultraviolet spectrophotometry using a customized manifold for preconcentration and simultaneous determination of morphine, codeine, and papaverine in water and human urine samples. Absorption spectra of the extracts were recorded inside the lumen of the hollow fiber using two fiber optics connected to a miniature spectrophotometer. Partial least squares regression was applied to resolve the overlapped spectra of the analytes. Performance of the model was validated by an independent test set. Central composite design was applied to optimize the extraction parameters. The optimized extraction conditions are as follows; supporting liquid membrane: 2‐nitrophenyl octyl ether containing 15% v/v bis(2‐ethylhexyl) phosphate, applied voltage: 80 V, donor pH: 3.0, acceptor pH: 1.0, extraction time: 20 min. Finally, the optimized extraction method was validated for determination of the mentioned alkaloids in human urine samples. The method showed good linearity (R² > 0.995) for all of the mentioned alkaloids. The limits of detection for morphine, codeine, and papaverine in diluted human urine were found to be 0.6, 1.1, and 0.6 ng/mL, respectively with acceptable relative standard deviations. Enrichment factors of 104, 108, and 102 were achieved for morphine, codeine, and papaverine, respectively.