Learning and approximation by Gaussians on Riemannian manifolds

Learning function relations or understanding structures of data lying in manifolds embedded in huge dimensional Euclidean spaces is an important topic in learning theory. In this paper we study the approximation and learning by Gaussians of functions defined on a d-dimensional connected compact C ^∞ Riemannian submanifold of which is isometrically embedded. We show that the convolution with the Gaussian kernel with variance σ provides the uniform approximation order of O(σ ^s ) when the approximated function is Lipschitz s ∈(0, 1]. The uniform normal neighborhoods of a compact Riemannian manifold play a central role in deriving the approximation order. This approximation result is used to investigate the regression learning algorithm generated by the multi-kernel least square regularization scheme associated with Gaussian kernels with flexible variances. When the regression function is Lipschitz s, our learning rate is (log² m)/m) ^{s/(8 s + 4 d)} where m is the sample size. When the manifold dimension d is smaller than the dimension n of the underlying Euclidean space, this rate is much faster compared with those in the literature. By comparing approximation orders, we also show the essential difference between approximation schemes with flexible variances and those with a single variance. Supported partially by the Research Grants Council of Hong Kong [Project No. CityU 103405], City University of Hong Kong [Project No. 7001983], National Science Fund for Distinguished Young Scholars of China [Project No. 10529101], and National Basic Research Program of China [Project No. 973-2006CB303102].     

Quasi-Monte Carlo for Highly Structured Generalised Response Models

Highly structured generalised response models, such as generalised linear mixed models and generalised linear models for time series regression, have become an indispensable vehicle for data analysis and inference in many areas of application. However, their use in practice is hindered by high-dimensional intractable integrals. Quasi-Monte Carlo (QMC) is a dynamic research area in the general problem of high-dimensional numerical integration, although its potential for statistical applications is yet to be fully explored. We survey recent research in QMC, particularly lattice rules, and report on its application to highly structured generalised response models. New challenges for QMC are identified and new methodologies are developed. QMC methods are seen to provide significant improvements compared with ordinary Monte Carlo methods.      

Weighted nonlinear regression

Minimization of the weighted nonlinear sum of squares of differences may be converted to the minimization of sum of squares. The Gauss-Newton method is recalled and the length of the step of the steepest descent method is determined by substituting the steepest descent direction in the Gauss-Newton formula. The existence of minimum is shown.     

Determination of effective wavelengths for discrimination of fruit vinegars using near infrared spectroscopy and multivariate analysis

Near infrared (NIR) spectroscopy based on effective wavelengths (EWs) and chemometrics was proposed to discriminate the varieties of fruit vinegars including aloe, apple, lemon and peach vinegars. One hundred eighty samples (45 for each variety) were selected randomly for the calibration set, and 60 samples (15 for each variety) for the validation set, whereas 24 samples (6 for each variety) for the independent set. Partial least squares discriminant analysis (PLS-DA) and least squares-support vector machine (LS-SVM) were implemented for calibration models. Different input data matrices of LS-SVM were determined by latent variables (LVs) selected by explained variance, and EWs selected by x-loading weights, regression coefficients, modeling power and independent component analysis (ICA). Then the LS-SVM models were developed with a grid search technique and RBF kernel function. All LS-SVM models outperformed PLS-DA model, and the optimal LS-SVM model was achieved with EWs (4021, 4058, 4264, 4400, 4853, 5070 and 5273 cm⁻¹) selected by regression coefficients. The determination coefficient (R²), RMSEP and total recognition ratio with cutoff value ±0.1 in validation set were 1.000, 0.025 and 100%, respectively. The overall results indicted that the regression coefficients was an effective way for the selection of effective wavelengths. NIR spectroscopy combined with LS-SVM models had the capability to discriminate the varieties of fruit vinegars with high accuracy.     

Multiblock variance partitioning: a new approach for comparing variation in multiple data blocks

More than one multi-informative analytical technique is often applied when describing the condition of a set of samples. Often a part of the information found in these data blocks is redundant and can be extracted from more blocks. This study puts forward a method (multiblock variance partitioning—MVP) to compare the information/variation in different data blocks using simple quantitative measures. These measures are the unique part of the variation only found in one data block and the common part that can be found in more data blocks. These different parts are found using PLS models between predictor blocks and a common response. MVP provides a different view on the information in different blocks than normal multiblock analysis. It will be shown that this has many applications in very diverse fields such as process control, assessor performance in sensory analysis, efficiency of preprocessing methods and as complementary information to an interval PLS analysis. Here the ideas of the MVP approach are presented in detail using a study of red wines from different regions measured with GC-MS and FT-IR instruments providing different kinds of data representations.     

Multi‐Channel Piezoelectric Quartz Crystal Sensor with Principal Component Analysis and Back‐Propagation Neural Network for Organic Pollutants from Petrochemical Plants

A multi‐channel piezoelectric quartz crystal gas sensor comprising arrays coated with various organic materials and a home‐made computer interface for data processing were prepared and employed to detect six kinds of common organic pollutants from petrochemical plants including benzene, styrene, chloroform, octane, hexene and hexyne. The principal component analysis (PCA) method was employed to select six kinds of appropriate coating materials for these organic pollutants from 22 adsorbents onto piezoelectric crystals. After performing a PCA assay, six representative coating materials, namely Polyisobutylene, Poly(dimethylsiloxane) (SE30), 4‐tert‐Butylcalix[6]arene, Cholesteryl chloroformate, C60‐Polyphenyl acetylene (C60‐PPA) and Ag(I)/cryptand‐2,2/Ethylene diamine/NH₃/Polyvinyl chloride were selected. Moreover, effects of coating load of adsorbents and concentration of pollutants were also investigated. Three kinds of recognition techniques including 2D PCA score map, radar plot and back‐propagation neural network (BPN) were employed for qualitative analysis of these organic pollutants, and a quantitative analysis method could be established by creating calibration curves for each organic pollutant. This homemade multi‐channel piezoelectric quartz crystal gas sensor showed a good detection limit of 0.068‐1.127 mg/L for these organic pollutants. The multi‐channel piezoelectric gas sensor exhibited good reproducibility with a relative standard deviation (RSD) of 1.1‐9.6%. Furthermore, this multi‐channel piezoelectric crystal detection system with BPN recognition technique was also utilized to successfully distinguish and identify each component of the mixture of organic gas samples. Multivariate linear regression (MLR) analysis was employed to quantitatively compute the concentration of species in the organic mixtures.     

Simultaneous Voltammetric Determination of Lead and Tin by Adsorptive Differential Pulse Stripping Method and Orthogonal Signal Correction‐Partial Least Squares in Water Samples

An adsorptive differential pulse stripping method for the simultaneous determination of lead and tin is proposed. The procedure involves an adsorptive accumulation of lead and tin on a hanging mercury drop electrode (HMDE), followed by oxidation of adsorbed lead and tin by voltammetric scan using differential pulse modulation. The optimum experimental conditions are: 0.2 mol L^?1 HNO₃, accumulation potential of ?900 mV versus Ag/AgCl, accumulation time of 200 s, scan rate of 20 mV s^?1 and pulse height of 80 mV. Lead and tin peak currents were observed in the same potential region at about ?400 mV. The simultaneous determination of lead and tin by using voltammetry is a difficult problem in analytical chemistry, due to voltammogram interferences. The resolution of a mixture of lead and tin by the application of orthogonal signal correction‐partial least squares (OSC‐PLS) was performed. The linear dynamic ranges were 0.003‐0.35 and 0.008‐0.50 μg mL^?1 and detection limits were land 3 ng mL^?1 for lead and tin, respectively. The RMSEP for lead and tin with OSC and without OSC were 2.8737, 6.0557 and 8.0941, 9.5151, respectively. The capability of the method for the analysis of real samples was evaluated by the determination of lead and tin in water samples with satisfactory results.     

Genetic‐Algorithm‐Based Wavelength Selection in Multicomponent Spectrophotometric Determination by PLS: Application on Ascorbic Acid and Uric Acid Mixture

Genetic algorithm (GA) is a suitable method for selecting wavelengths for partial least squares (PLS) calibration of mixtures with almost identical spectra without loss of prediction capacity using the spectrophotometric method. In this study, the concentration model is based on absorption spectra in the range of 200‐320 nm for 25 different mixtures of ascorbic acid (AA) and uric acid (UA). The calibration curve was linear over the concentration range of 1‐15 and 2‐16 μg mL^?1 for ascorbic acid and uric acid, respectively. The root mean square deviation (RMSD) for ascorbic acid and uric acid with GA and without GA were 0.3071 and 0.3006, 0.3971 and 0.7063, respectively. The proposed method was successfully applied to the simultaneous determination of both analytes in human serum and urine samples.     

Simple outlier labeling based on quantile regression,with application to the steelmaking process

This paper introduces some methods for outlier identification in the regression setting, motivated by the analysis of steelmaking process data. The proposed methodology extends to the regression setting the boxplot rule, commonly used for outlier screening with univariate data. The focus here is on bivariate settings with a single covariate, but extensions are possible. The proposal is based on quantile regression, including an additional transformation parameter for selecting the best scale for linearity of the conditional quantiles. The resulting method is used to perform effective labeling of potential outliers, with a quite low computational complexity, allowing for simple implementation within statistical software as well as commonly used spreadsheets. Some simulation experiments have been carried out to study the swamping and masking properties of the proposal. The methodology is also illustrated by some real life examples, taking as the response variable the energy consumed in the melting process. Copyright © 2015 John Wiley & Sons, Ltd.