Comparison of robust regression methods based on least-median and adaptive kalman filtering approaches applied to linear calibration data

Five algorithms for data analysis are evaluated for their abilities to discriminate against outliers in small data sets (4–10 points). These methods included least-squares regression, the least absolute -deviation method, the least median of squares method, and two techniques based on an adaptive Kalman filter. For data sets consisting of 4–9 points with one outlier, the average errors in the estimation of the slope were found to be 18.9 % by least-squares, 17.7% by the least absolute deviation method, 0.5% by the least median of squares algorithm, 9.1% by an adaptive Kalman filter algorithm, and 0.9% by a zero-lag adaptive Kalman filter algorithm. Based on these results, the conclusion is that the zero-lag adaptive Kalman filter and the least median of squares approaches are best suited for the detection of outliers in small calibration data sets.     

The Kalman Filter as an On-Line Drift Compensator in Multicomponent Analysis Determinations

Abstract

A method is proposed for the on-line compensation of linear drift in analytical measurements. The robustness of the compensator, which is based on the Kalman filter, has been investigated by means of a parametric study. Special emphasis is given to the choice of the initial error covariance matrix, which is an important design quantity for the Kalman filter. The presented method can be applied when there is uncertainty about the presence of drift in the measurements. Some results are compared with results obtained with the non-recursive least squares method. The excellent performance of the on-line compensator suggests a possible solution for the linear drift problem. Of some theoretical importance is the fact that the proposed solution has a strong intuitive appeal: the linear drift can be considered as “two extra components” in a spectrum or equivalently as the time varying mean of the measurement noise.     

Characterization of a Stable Adaptive Calibration Model Using Near-Infrared Spectroscopy and Partial Least Squares with a Kalman Filter

The calibration model of near-infrared (NIR) spectra established using the Kalman filter-partial least square (partial least squares combined with a Kalman filter) method can be adapted to outdated equipment, environmental changes, external samples, and other applications. However, the variance of the measurement noise estimation for NIR spectrum measurements cannot be easily obtained using Kalman filter-partial least squares; therefore, the variance in the measurement noise is often assumed to be zero for the Kalman filter-partial least square calibration model, which affects the stability of the model. In this study, the measured input and output data were used effectively, and the gamma test method for estimating the measurement noise variance was used to improve the stability of the Kalman filter-partial least square calibration model. First, an accurate estimation of the measurement noise variance was obtained, and accurate modeling was then performed using Kalman filter-partial least squares. Finally, 600 abandoned drilling fluid samples were used to confirm the validity of the proposed method. The Kalman filter-partial least square and gamma test-Kalman filter-partial least square methods are compared. Testing of external samples 401–600 demonstrated that the stability of the Kalman filter-partial least square model decreased. The root mean square error of the prediction of the Kalman filter-partial least square model was 27.135, which was worse than that of the gamma test-Kalman filter-partial least square model (20.307). The validation results show that the proposed method has better stability in tracking the evolution of the NIR spectrometer’s measurement state.     

State estimation in discrete titrations with kalman filtering and fixed-interval smoothing

The suitability of a state space model for processing titration curves and their derivatives is described. The Kalman filter gives a systematic deviation in the evaluated inflection points, which is removed completely by subsequent fixed-interval smoothing. The estimation procedures can be used for on-line control in order to obtain equidistant measurements with variable volume increments. The precision is shown to depend on the minimal delivery of the buret and on the noise variance of the detector. The algorithms are illustrated with a simulated example and applied successfully in practice.     

新极谱法实验数据的卡尔曼滤波

本文提出的"1:n+2"新卡尔曼滤波器,具有算法简单、占用计算机内存少、运行速度快等优点;同时又能有效地消除信号中的随机噪音和电容电流,讨论了"1:3"滤波器的结构性能,采用Cd⁺离子的阶梯扫描伏安法的实测数据来分析滤波结果的最优性,用于处理TI⁺、pb²⁺、In³⁺等离子的实测数据,均获得理想的滤波效果。     

Comparison of the extended kalman filter and marquardt algorithms for processing kinetic data

The extended Kalman filter and Marquardt's gradient expansion algorithm for nonlinear least squares are compared with respect to accuracy and precision of parameter extimates, computational burden, sensitivity to initial parameter estimates and ability to indicate model errors. Fits of synthetic first-order data and combined first- and zero-order data produce estimates of equivalent precision and accuracy in most cases. Similar results were obtained for both simulated and experimental data for combined zero-order/first-order processes. However, for the simulated zero-order/first-order data with small zero-order components processed over two half-lives of the first-order process, the Kalman filter overestimated the zero-order rate constant by a substantially larger amount than the Marquardt algorithm. Significant differences in computational burden and sensitivity to initial parameter estimates are demonstrated; however, neither algorithm has a significant advantage over the other for the detection of model error.     

One-dimensional Kalman filter in a robot system for total analysis of solid dosage formulations

A one-dimensional Kalman filter for peak resolution is applied to a totally automated robot system for liquid chromatographic analysis of solid dosage formulations. Sample solutions were prepared from four tablets of clemastine fumarate (1 mg/tablet) and injected automatically at regular, short intervals onto the column by the robot system. The overlapped peaks in the resulting complicated chromatogram were resolved and evaluated quantitatively by the one-dimensional filter. The peak-resolving powers are shown to be reliable by comparison with a multidimensional Kalman filter. The applicability of the whole analytical system with the linear Kalman filter is discussed.     

Kalman滤波用于荧光,吸收,导数与同步光谱的研究

将Ｋａｌｍａｎ滤波用于常规荧光、同步荧光、导数荧光、吸收光谱及导数光谱，实现了不经分离对苯酚与间苯二酚混合物及复方药物制剂的同时测定。结果良好，操作简便。     

通用模拟退火用于稳健多元分析校正

模拟退火是一种全局优化算法，具有跨越局部最优点的机制，最小一乘是一种较常用的最小二乘更为稳健的优化准则，更适用于可能偏离正态分布的实际数据集，本文探讨了用最小一乘为准则并利用模拟退火方法同时测定多组分体系的可能性。应用于2－3组分药物体系分析，获得了满意的结果，本文还探讨了改变步长提高模拟退火算法优化精度的方法。     

The Kalman filter in analytical chemistry

During the past ten years, the means by which more information can be extracted from experimental data have become an important area of research in analytical chemistry. Digital filters have been demonstrated to have a number of applications to analytical problems. These techniques typically involve a least-squares fit of experimental data to some model of the process being filtered. One method for filtering experimental data is based on the Kalman filter, a recursive, linear digital filter first developed for use in navigation, but now used in many fields. This paper discusses the implementation of Kalman filters in analytical chemistry. The principles of state-space digital filtering are reviewed, and the development of state/space models is discussed. Discussion is focused on the discrete Kalman algorithms. Two examples are provided to demonstrate the operation of the discrete Kalman filtering algorithm. Similarities between Kalman filtering and weighted least-squares methods are considered, and the specific advantages and disadvantages of linear and nonlinear Kalman filtering approaches are evaluated. To illustrate the range of problems which benefit from use of the filter, a comprehensive literature survey of the application of Kalman filtering to chemical problems is provided.     

Genetic‐Algorithm‐Based Wavelength Selection in Multicomponent Spectrophotometric Determination by PLS: Application on Ascorbic Acid and Uric Acid Mixture

Genetic algorithm (GA) is a suitable method for selecting wavelengths for partial least squares (PLS) calibration of mixtures with almost identical spectra without loss of prediction capacity using the spectrophotometric method. In this study, the concentration model is based on absorption spectra in the range of 200‐320 nm for 25 different mixtures of ascorbic acid (AA) and uric acid (UA). The calibration curve was linear over the concentration range of 1‐15 and 2‐16 μg mL^?1 for ascorbic acid and uric acid, respectively. The root mean square deviation (RMSD) for ascorbic acid and uric acid with GA and without GA were 0.3071 and 0.3006, 0.3971 and 0.7063, respectively. The proposed method was successfully applied to the simultaneous determination of both analytes in human serum and urine samples.     

化学计量学在我国光度分析中的进展

本文介绍了化学计量学在我国光度分析中的研究应用及某些进展。这些方法可用于消除干扰,改善选择性,实现无机、有机及药物多组分混合物以及复方制剂的同时测定。     

Two‐way and three‐way approaches to ultra high performance liquid chromatography–photodiode array dataset for the quantitative resolution of a two‐component mixture containing ciprofloxacin and ornidazole

Two‐way and three‐way calibration models were applied to ultra high performance liquid chromatography with photodiode array data with coeluted peaks in the same wavelength and time regions for the simultaneous quantitation of ciprofloxacin and ornidazole in tablets. The chromatographic data cube (tensor) was obtained by recording chromatographic spectra of the standard and sample solutions containing ciprofloxacin and ornidazole with sulfadiazine as an internal standard as a function of time and wavelength. Parallel factor analysis and trilinear partial least squares were used as three‐way calibrations for the decomposition of the tensor, whereas three‐way unfolded partial least squares was applied as a two‐way calibration to the unfolded dataset obtained from the data array of ultra high performance liquid chromatography with photodiode array detection. The validity and ability of two‐way and three‐way analysis methods were tested by analyzing validation samples: synthetic mixture, interday and intraday samples, and standard addition samples. Results obtained from two‐way and three‐way calibrations were compared to those provided by traditional ultra high performance liquid chromatography. The proposed methods, parallel factor analysis, trilinear partial least squares, unfolded partial least squares, and traditional ultra high performance liquid chromatography were successfully applied to the quantitative estimation of the solid dosage form containing ciprofloxacin and ornidazole.     

A Kalman filter for calibration,evaluation of unknown samples and quality control in drifting systems : Part 1. Theory and Simulations

The suitability of a Kalman filter for processing slowly varying parameters of a linear calibration graph is described. After calibration, the recursive algorithm predicts the changing parameters in time, which are used for the evaluation of unknown samples. With a preselected precision of the final results as a proper analytical goal, one may decide successively either to do another unknown sample or to calibrate again. The application of the proposed algorithm is demonstrated with a simulated example based on realistic data.     

Series expansion methods for extracting ternary diffusion coefficients from Rayleigh interferometric data

Three series expansions are examined for use in extracting ternary diffusion coefficients from Rayleigh interferometric data for free diffusion experiments. Convergence comparisons show that, providing enough terms are used, two of the three are suitable, including one used earlier by Albright and Sherrill. A new simpler analysis, using only linear least squares, has been applied to simulated and real data with good results. Advantages and disadvantages of series methods are compared to a direct non-linear least squares analysis.     

High-speed chromatographic analysis of high-fructose corn syrup for process monitoring

High-speed chromatography is coupled with numerical methods for analyzing unresolved chromatograms and applied to a process analysis of high-fructose corn syrup. A column selection process is demonstrated where a minimum amount of resolution is sacrificed in order to decrease analysis time from over 5 min to 25 sec. Two data analysis methods, linear least squares regression and the sequential chromatogram ratio technique coupled with sequential suppression, are compared for their ability to quantitate the poorly resolved chromatograms. Both methods fit pure component analyte chromatograms, collected on a computer, to a sample chromatogram with unknown concentrations of each analyte. For a high-fructose corn syrup sample with a nominal fructose concentration of 55%, linear least squares analysis gave a fructose concentration percentage of 57.2 +/- 0.9%. The sequential chromatogram ratio algorithm gave a fructose concentration percentage of 57.9 +/- 0.7%.     

Simultaneous Voltammetric Determination of Three Herbicides in Food and Water Samples with the Aid of Chemometrics

Differential pulse stripping voltammetry method(DPSV) was applied to the determination of three herbicides,ametryn,cyanatryn,and dimethametryn.It was found that their voltammograms overlapped strongly,and it is difficult to determine these compounds individually from their mixtures.With the aid of chemometrics,classical least squares(CLS),principal component regression(PCR) and partial least squares(PLS),voltammogram resolution and quantitative analysis of the synthetic mixtures of the three compounds were successfully performed.The proposed method was also applied to the analysis of some real samples with satisfactory results.     

Chemometrics-assisted simultaneous determination of cobalt(II) and chromium(III) with flow-injection chemiluminescence method

In this paper, a flow-injection chemiluminescence (CL) system is proposed for simultaneous determination of Co(II) and Cr(III) with partial least squares calibration. This method is based on the fact that both Co(II) and Cr(III) catalyze the luminol-H(2)O(2) CL reaction, and that their catalytic activities are significantly different on the same reaction condition. The CL intensity of Co(II) and Cr(III) was measured and recorded at different pH of reaction medium, and the obtained data were processed by the chemometric approach of partial least squares. The experimental calibration set was composed with nine sample solutions using orthogonal calibration design for two component mixtures. The calibration curve was linear over the concentration range of 2 x 10(-7) to 8 x 10(-10) and 2 x 10(-6) to 4 x 10(-9) g/ml for Co(II) and Cr(III), respectively. The proposed method offers the potential advantages of high sensitivity, simplicity and rapidity for Co(II) and Cr(III) determination, and was successfully applied to the simultaneous determination of both analytes in real water sample.