光谱多元分析校正集和验证集样本分布优选方法研究

分析了校正集和验证集样品数随性质分布不均匀性对光谱多元分析校正的不良影响，揭示了实际光谱多元校正中“均值化”现象，即性质值小的样本预测值结果偏大，性质值大的则偏小，提出了一种优选样品新方法—Rank-KS。其综合考虑光谱空间和性质空间对样本进行挑选，将性质空间平均分为若干小区间，在每个小区间内分别利用Kennard-Stone法和随机法进行校正集和验证集样本的挑选，这样得到的校正集和验证集可明显改善样本数随性质分布的均匀性。以红外光谱测定汽油中碳酸二甲酯(DMC)含量和近红外光谱测定二甲亚砜溶液二甲亚砜浓度为研究对象，分别采用Rank-KS、随机法、Kennard-Stone、浓度梯度法和SPXY等方法选择校正集和验证集样品，使用多元线性回归和偏最小二乘法建立模型，比较这些方法对光谱多元校正分析的影响，结果表明Rank-KS方法可改善校正集和验证集样品数随性质分布的均匀性;对于样本数分布中间局部样本多和两端局部少、或者局部没有样本的样本集，使用Rank-KS算法挑选校正集，无论使用MLR还是PLS1建立多元分析模型，均能明显改善其模型预测能力，使得到的模型的预测均方根最小。

An Optimal Selection Method of Samples of Calibration Set and Validation Set for Spectral Multivariate Analysis

The side effects in spectral multivariate modeling caused by the uneven distribution of sample numbers in the region of the calibration set and validation set were analyzed, and the “average” phenomenon that samples with small property values are predicted with larger values, and those with large property values are predicted with less values in spectral multivariate calibration is showed in this paper. Considering the distribution feature of spectral space and property space simultaneously, a new method of optimal sample selection named Rank-KS is proposed. Rank-KS aims at improving the uniformity of calibration set and validation set. Y-space was divided into some regions uniformly, samples of calibration set and validation set were extracted by Kennard-Stone(KS) and Random-Select(RS) algorithm respectively in every region, so the calibration set was distributed evenly and had a strong presentation. The proposed method were applied to the prediction of dimethylcarbonate (DMC) content in gasoline with infrared spectra and dimethylsulfoxide in its aqueous solution with near infrared spectra. The “average” phenomenon showed in the prediction of multiple linear regression (MLR) model of dimethylsulfoxide was weakened effectively by Rank-KS. For comparison, the MLR models and PLS1 models of MDC and dimethylsulfoxide were constructed by using RS, KS, Rank-Select, sample set partitioning based on joint X- and Y-blocks (SPXY) and proposed Rank-KS algorithms to select the calibration set, respectively. Application results verified that the best prediction was achieved by using Rank-KS. Especially, for the distribution of sample set with more in the middle and less on the boundaries, or none in the local, prediction of the model constructed by calibration set selected using Rank-KS can be improved obviously.