ASTM clustering for improving coal analysis by near-infrared spectroscopy

Multivariate analysis techniques have been applied to near-infrared (NIR) spectra coals to investigate the relationship between nine coal properties (moisture (%), ash (%), volatile matter (%), fixed carbon (%), heating value (kcal/kg), carbon (%), hydrogen (%), nitrogen (%) and sulphur (%)) and the corresponding predictor variables. In this work, a whole set of coal samples was grouped into six more homogeneous clusters following the ASTM reference method for classification prior to the application of calibration methods to each coal set. The results obtained showed a considerable improvement of the error determination compared with the calibration for the whole sample set. For some groups, the established calibrations approached the quality required by the ASTM/ISO norms for laboratory analysis. To predict property values for a new coal sample it is necessary the assignation of that sample to its respective group. Thus, the discrimination and classification ability of coal samples by Diffuse Reflectance Infrared Fourier Transform Spectroscopy (DRIFTS) in the NIR range was also studied by applying Soft Independent Modelling of Class Analogy (SIMCA) and Linear Discriminant Analysis (LDA) techniques. Modelling of the groups by SIMCA led to overlapping models that cannot discriminate for unique classification. On the other hand, the application of Linear Discriminant Analysis improved the classification of the samples but not enough to be satisfactory for every group considered.     

Chemometric analysis of groundwater quality data of alluvial aquifer of Gangetic plain, North India

Water quality data set from the alluvial region in the Gangetic plain in northern India, which is known for high fluoride levels in soil and groundwater, has been analysed by chemometric techniques, such as principal component analysis (PCA), discriminant analysis (DA) and partial least squares (PLS) in order to investigate the compositional differences between surface and groundwater samples, spatial variations in groundwater composition and influence of natural and anthropogenic factors. Trilinear plots of major ions showed that the groundwater in this region is mainly of Na/K-bicarbonate type. PCA performed on complete data matrix yielded six significant PCs explaining 65% of the data variance. Although, PCA rendered considerable data reduction, it could not clearly group and distinguish the sample types (dug well, hand-pump and surface water). However, a visible differentiation between the water samples pertaining to two watersheds (Khar and Loni) was obtained. DA identified six discriminating variables between surface and groundwater and also between different types of samples (dug well, hand pump and surface water). Distinct grouping of the surface and groundwater samples was achieved using the PLS technique. It further showed that the groundwater samples are dominated by variables having origin both in natural and anthropogenic sources in the region, whereas, variables of industrial origin dominate the surface water samples. It also suggested that the groundwater sources are contaminated with various industrial contaminants in the region.     

Reflection and transmission mid-infrared spectroscopy for rapid determination of coal properties by multivariate analysis

In the present paper, the influence of different acquisition techniques (transmission, diffuse reflectance infrared Fourier transform and attenuated total reflectance) in the determination of nine coal properties related to combustion power plants has been studied. Raw coal samples of different origins were pooled for developing a correlation between the resultant spectra and the corresponding coal properties by multivariate analysis techniques. Thus, the existent collinearity in mid-infrared coal spectra led to the application of partial least squares regression (PLS), studying simultaneously the influence of different spectroscopic units as well as several spectral data mathematical pre-treatments. On the other hand, a principal component analysis (PCA) revealed a relationship between principal components and coal composition in both transmission and reflection techniques. Although the best accuracy and precision results were obtained for coal properties related to organic matter, the system was also able to differentiate coal samples attending to the presence of a specific mineral matter, kaolinite.     

Identification of discriminatory variables in proteomics data analysis by clustering of variables

This article presents a data analysis method for biomarker discovery in proteomics data analysis. In factor analysis-based discriminate models, the latent variables (LV's) are calculated from the response data measured at all employed instrument channels. Since some channels are irrelevant and their responses do not possess useful information, the extracted LV's possess mixed information from both useful and irrelevant channels. In this work, clustering of variables (CLoVA) based on unsupervised pattern recognition is suggested as an efficient method to identify the most informative spectral region and then it is used to construct a more predictive multivariate classification model. In the suggested method, the instrument channels (m/z value) are clustered into different clusters via self-organization map. Subsequently, the spectral data of each cluster are separately used as the input variables of classification methods such as partial least square-discriminate analysis (PLS-DA) and extended canonical variate analysis (ECVA). The proposed method is evaluated by the analysis of two experimental data sets (ovarian and prostate cancer data set). It is found that our proposed method is able to detect cancerous from healthy samples with much higher sensitivity and selectivity than conventional PLS-DA and ECVA methods.     

A new regression method based on independent component analysis

Based on independent component analysis (ICA), a new regression method, independent component regression (ICR), was developed to build the model of NIR spectra and the routine components of plant samples. It is found that ICR and principal component regression (PCR) are completely equivalent when they are applied in quantitative prediction. However, independent components (ICs) can give more chemical explanation than principal components (PCs) because independence is a high-order statistic that is a much stronger condition than orthogonality. Three ICs are obtained by ICA from the NIR spectra of plant samples; it is found that they are strongly correlated to the NIR spectra of water, hydrocarbons and organonitrogen compounds, respectively. Therefore, ICA may be a promising tool to retrieve both quantitative and qualitative information from complex chemical data sets.     

Using combinations of principal component scores from different spectral ranges in near-infrared region to improve discrimination for samples of complex composition

Principal component analysis (PCA) is widely used as an exploratory data analysis tool in the field of vibrational spectroscopy, particularly near-infrared (NIR) spectroscopy. PCA represents original spectral data containing large variables into a few feature-containing variables, or scores. Although multiple spectral ranges can be simultaneously used for PCA, only one series of scores generated by merging the selected spectral ranges is generally used for qualitative analysis. Alternatively, the combined use of an independent series of scores generated from separate spectral ranges has not been exploited.The aim of this study is to evaluate the use of PCA to discriminate between two geographical origins of sesame samples, when scores independently generated from separate spectral ranges are optimally combined. An accurate and rapid analytical method to determine the origin is essentially required for the correct value estimation and proper production distribution. Sesame is chosen in this study because it is difficult to visually discriminate the geographical origins and its composition is highly complex. For this purpose, we collected diffuse reflectance near-infrared (NIR) spectroscopic data from geographically diverse sesame samples over a period of eight years. The discrimination error obtained by applying linear discriminant analysis (LDA) was improved when separate scores from two spectral ranges were optimally combined, compared to the discrimination errors obtained when scores from singly merged two spectral ranges were used.     

Orthogonal signal correction, wavelet analysis, and multivariate calibration of complicated process fluorescence data

In this paper, multivariate calibration of complicated process fluorescence data is presented. Two data sets related to the production of white sugar are investigated. The first data set comprises 106 observations and 571 spectral variables, and the second data set 268 observations and 3997 spectral variables. In both applications, a single response, ash content, is modelled and predicted as a function of the spectral variables. Both data sets contain certain features making multivariate calibration efforts non-trivial. The objective is to show how principal component analysis (PCA) and partial least squares (PLS) regression can be used to overview the data sets and to establish predictively sound regression models. It is shown how a recently developed technique for signal filtering, orthogonal signal correction (OSC), can be applied in multivariate calibration to enhance predictive power. In addition, signal compression is tested on the larger data set using wavelet analysis. It is demonstrated that a compression down to 4% of the original matrix size — in the variable direction — is possible without loss of predictive power. It is concluded that the combination of OSC for pre-processing and wavelet analysis for compression of spectral data is promising for future use.     

Parallel vector analysis based soft resolution of spectral data from pH‐metric titration using symmetry constraint

In this study, a soft method is proposed to calculate concentration and spectral profiles for the two‐way spectral data from dissociation equilibria of polyprotic acids (H_nA). This method has four main distinct steps: (i) a fixed size moving window evolving factor analysis (FSMWEFA) was used to identify the local rank map, (ii) WFA was applied to calculate the concentration profiles of H_nA and Aⁿ⁻ (selection of the window for application of WFA was performed using EFA), (iii) PVA was used to calculate H_{n − 1}A to HA spectral profiles, and (iv) a symmetry constraint, in addition to the non‐negativity constraint, was utilized to obtain the unique concentration and spectral profiles from different acceptable sets of profiles. In the absence of any selective region in the spectral data, the proposed soft method resulted in unique solution without rotational ambiguity. This study is the first application of symmetry constraint on concentration profiles. The rotational ambiguity drastically decreased on considering the constraint of symmetry of the H_{n − 1}A and HA concentration profiles, in addition to non‐negativity of profiles. Simulated examples were used to confirm these approaches. Effect of closeness of dissociation constants on the estimated values of constants was investigated. The results showed that when the difference between pK_a values is more than 1.2, the obtained errors in the estimation of pK_a values are less than about 6.5%. The considered real data were from pH‐metric titration of fluorescein. The obtained spectral and concentration profiles and the estimated pK_a values for fluorescein were in good agreement with the previously reported data. Copyright © 2009 John Wiley & Sons, Ltd.     

Methods of multivariate data analysis applied to the investigation of fen soils

Fen soils from two sites of the Rhin-Havel-Luch, a peatland in the north-east of Germany, have been investigated. The samples have been collected in two horizons, representing different degrees of degradation and mineralisation of peat. Gravimetric measurements, energy dispersive X-ray fluorescence (EDXRF), elemental analysis, and ¹H low resolution nuclear magnetic resonance (LR-NMR) of the fen soil samples have been performed. By multivariate analysis of all the experimental data, especially by the principal component analysis (PCA) and by the cluster analysis, respectively, it was possible to classify the fen soils, to identify their characteristic properties, to detect temporal and local variations, and to prove representative field sampling. Furthermore, the correlation between variables of the applied analytical methods could be interpreted in context to the composition of fen soils and mutual influences of their properties.     

The effect of sequential dual-gas testing on laser-induced breakdown spectroscopy-based discrimination: Application to brass samples and bacterial strains

Four Cu–Zn brass alloys with different stoichiometries and compositions have been analyzed by laser-induced breakdown spectroscopy (LIBS) using nanosecond laser pulses. The intensities of 15 emission lines of copper, zinc, lead, carbon, and aluminum (as well as the environmental contaminants sodium and calcium) were normalized and analyzed with a discriminant function analysis (DFA) to rapidly categorize the samples by alloy. The alloys were tested sequentially in two different noble gases (argon and helium) to enhance discrimination between them. When emission intensities from samples tested sequentially in both gases were combined to form a single 30-spectral line “fingerprint” of the alloy, an overall 100% correct identification was achieved. This was a modest improvement over using emission intensities acquired in argon gas alone. A similar study was performed to demonstrate an enhanced discrimination between two strains of Escherichia coli (a Gram-negative bacterium) and a Gram-positive bacterium. When emission intensities from bacteria sequentially ablated in two different gas environments were combined, the DFA achieved a 100% categorization accuracy. This result showed the benefit of sequentially testing highly similar samples in two different ambient gases to enhance discrimination between the samples.     

New discrimination method combining hit quality index based spectral matching and voting

A new discrimination method, called hit quality index (HQI)-voting, that uses the HQI for discriminant analysis has been developed. HQI indicates the degree of spectral matching between two spectra as known. In this method, a library sample yielding the highest HQI value for an unknown sample was initially searched and a group containing this sample was chosen as the group for the unknown sample. When overall spectral features of two groups are quite close to each other, many library samples with similar HQI values could be available for an unknown sample. In this situation, the simultaneous consideration of multiple votes (several library samples with close HQI values) for final decision would be more robust. In order to evaluate the discrimination performance of HQI-voting, three different near-infrared (NIR) spectroscopic datasets composed of two sample groups were used: (1) domestic and imported sesame samples, (2) domestic and imported Angelica gigas samples, and (3) diesel and light gas oil (LGO) samples. For the purpose of comparison, principal component analysis–linear discriminant analysis (PCA–LDA), partial least squares–discriminant analysis (PLS–DA) as well as k-nearest neighbor (k-NN) were also performed using the same datasets and the resulting accuracies were compared. The discrimination performances improved with the use of HQI-voting in comparison with those resulted from PCA–LDA and PLS–DA. The overall results support that HQI-voting is a comparable discrimination method to that of existing factor-based multivariate methods.     

交互移动窗口因子分析法(AMWFA)用于二维数据重叠色谱峰的比较分析

采用交互移动窗口因子分析法(AMWFA), 通过挖掘两个体系中的选择性信息, 获得了不同样本间的共有组分数, 还同时得到了各物质对应的光谱或质谱信息. 详细阐述了本法的原理和计算方法, 并用一个模拟的GC-MS 数据对方法进行了验证.     

Principal component analysis versus fuzzy principal component analysis A case study: the quality of danube water (1985-1996)

Principal component analysis (PCA) is a favorite tool in environmetrics for data compression and information extraction. PCA finds linear combinations of the original measurement variables that describe the significant variations in the data. However, it is well-known that PCA, as with any other multivariate statistical method, is sensitive to outliers, missing data, and poor linear correlation between variables due to poorly distributed variables. As a result data transformations have a large impact upon PCA. In this regard one of the most powerful approach to improve PCA appears to be the fuzzification of the matrix data, thus diminishing the influence of the outliers. In this paper we discuss and apply a robust fuzzy PCA algorithm (FPCA). The efficiency of the new algorithm is illustrated on a data set concerning the water quality of the Danube River for a period of 11 consecutive years. Considering, for example, a two component model, FPCA accounts for 91.7% of the total variance and PCA accounts only for 39.8%. Much more, PCA showed only a partial separation of the variables and no separation of scores (samples) onto the plane described by the first two principal components, whereas a much sharper differentiation of the variables and scores is observed when FPCA is applied.     

Supervised principal components: a new method for multivariate spectral analysis

The supervised principal components (SPC) method was proposed by Bair and Tibshirani for statistics regression problems where the number of variables greatly exceeds the number of samples. This case is extremely common in multivariate spectral analysis. The objective of this research is to apply SPC to near‐infrared and Raman spectral calibration. SPC is similar to traditional principal components analysis except that it selects the most significant part of wavelength from the high‐dimensional spectral data, which can reduce the risk of overfitting and the effect of collinearity in modeling according to a semi‐supervised strategy. In this study, four conventional regression methods, including principal component regression, partial least squares regression, ridge regression, and support vector regression, were compared with SPC. Three evaluation criteria, coefficient of determination (R²), external correlation coefficient (Q²), and root mean square error of prediction, were calculated to evaluate the performance of each algorithm on both near‐infrared and Raman datasets. The comparison results illustrated that the SPC model had a desirable ability of regression and prediction. We believe that this method might be an alternative method for multivariate spectral analysis. Copyright © 2013 John Wiley & Sons, Ltd.     

A pilot study on colonic mucosal tissues by fluorescence spectroscopy technique: Discrimination by principal component analysis (PCA) and artificial neural network (ANN) analysis

Pulsed laser‐induced autofluorescence spectra of pathologically certified normal and malignant colonic mucosal tissues were recorded at 325 nm excitation. The spectra were analysed using three different methods for discrimination purposes. First, all the spectra were subjected to the principal component analysis (PCA) and the discrimination between normal and malignant cases were achieved using parameters like, spectral residuals, Mahalanobis distance and scores of factors. Second, to understand the changes in tissue composition between the two classes (normal, and malignant), difference spectrum was constructed by subtracting mean spectrum of calibration set samples from simulated mean of all spectra of any one class (normal/malignant) and in third, artificial neural network (ANN) analysis was carried out on the same set of spectral data by training the network with spectral features like, mean, median, spectral residual, energy, standard deviation, number of peaks for different thresholds (100, 250 and 500) after carrying out 1st‐order differentiation of the training set samples and discrimination between normal and malignant conditions were achieved. The specificity and sensitivity were determined in PCA and ANN analyses and they were found to be 100 and 91.3% in PCA, and 100 and 93.47% in ANN, respectively. Copyright © 2008 John Wiley & Sons, Ltd.     

基于超高效液相色谱-紫外检测定量指纹图谱结合化学模式识别的复方金钱草颗粒质量评价

复方金钱草颗粒具有利尿、抑制泌尿系结石形成、抗炎、抗氧化作用,且具有较大的市场需求。因此,采用超高效液相色谱-紫外检测(UPLC-UV)法建立定量指纹图谱,并结合化学模式识别技术对不同年份的复方金钱草颗粒进行质量评价,可为其质量控制提供依据。采用聚类分析(HCA)和主成分分析(PCA)等化学模式识别技术对35批复方金钱草颗粒样品的指纹图谱数据进行分析,筛选出质量差异标志物芒果苷和异芒果苷,并对二者进行含量测定。在复方金钱草颗粒指纹图谱中共指认出12个共有峰,且35批样品的相似度均在0.952以上。在HCA中,将35批样品分为了两类,其中2018年和2019年的样品为一类,2020年和2021年的样品为一类。此外,PCA结果显示了与聚类分析相同的聚类趋势。在此基础上,进一步通过正交偏最小二乘法分析 (OPLS-DA)筛选出了导致2018年、2019年与2020年、2021年的样品产生差异的差异标志物芒果苷和异芒果苷。以两个差异标志物芒果苷和异芒果苷为指标进行含量测定,结果显示色谱峰的分离度良好,线性关系良好,平均加标回收率分别为101.7%~105.6%和103.4%~105.5%,且相对标准偏差(RSD)均低于1.43%。在35批样品中,2020年、2021年的样品与2018年、2019年的样品相比,芒果苷与异芒果苷含量更高且波动范围更小。该研究建立了准确、可靠的复方金钱草颗粒质控方法,实现了对不同年份的复方金钱草颗粒样品合理、有效的质量评价,可为建立更系统、更全面的质量控制标准提供借鉴与参考。     

Optimization of Fish Quality by Evaluation of Total Volatile Basic Nitrogen (TVB-N) and Texture Profile Analysis (TPA) by Near-Infrared (NIR) Hyperspectral Imaging

Hyperspectral images contain both spectral and spatial image information and were investigated to characterize the freshness of fish. However, most studies of this application have focused on spectral signals rather than image features. The goal of this work was to investigate the ability of spectral and image textural variables for predicting the chemical and physical qualities of fish, respectively, and to optimize the variables for the specific quality determination. The chemical (total volatile basic nitrogen, TVB-N) and physical (texture profile analysis, TPA) properties were investigated. Partial least square (PLS) was applied to develop fish quality prediction models with the spectral and textural variables from the hyperspectral images. The results showed that the TVB-N content of fish fillets was accurately predicted using the spectra. Meanwhile, the TPA parameters were determined through the image textural features with high accuracy, which indicated image textural features were highly related with the TPA parameters. Moreover, spectral and textural features were also extracted from fish eyes and gills and were further used to predict the intact fish quality, taking advantage of the freshness sensitivity of the eyes and gills. The results illustrate that spectra from fish eyes and gills are a potential tool to predict the TVB-N content and TPA parameters for intact fish.     

A practical approach for near infrared spectral quantitative analysis of complex samples using partial least squares modeling

The number of latent variables (LVs) or the factor number is a key parameter in PLS modeling to obtain a correct prediction. Although lots of work have been done on this issue, it is still a difficult task to determine a suitable LV number in practical uses. A method named independent factor diagnostics (IFD) is proposed for investigation of the contribution of each LV to the predicted results on the basis of discussion about the determination of LV number in PLS modeling for near infrared (NIR) spectra of complex samples. The NIR spectra of three data sets of complex samples, including a public data set and two tobacco lamina ones, are investigated. It is shown that several high order LVs constitute main contributions to the predicted results, albeit the contribution of the low order LVs should not be neglected in the PLS models. Therefore, in practical uses of PLS for analysis of complex samples, it may be better to use a slightly large LV number for NIR spectral analysis of complex samples. Supported by the National Natural Science Foundation of China (Grant Nos. 20775036 & 20835002)     

Development of near infrared sensors: Detection of influential factors by the AComDim method

The development of near infrared (NIR) sensors has to go through different steps of testing. Once a prototype is ready to be used, it is necessary to evaluate and optimize the experimental conditions and the data collection, in terms of accuracy, repeatability, reproducibility and speed. This paper studies the effects of controllable experimental factors on the quality of the spectral response, to determine the influence of each instrumental parameter and to improve the predictions obtained from the collected data. The AComDim method, based on the multi-block analysis of ANOVA matrices, was used here to evaluate the impact of experimental factors on the responses from the different sensors tested.