稳健算法及其在红外光谱分析中的应用

总结了稳健主成分分析、稳健主成分回归、稳健偏最小二乘回归和稳健连续回归等各种稳健算法的新近成果. 研究表明,稳健算法可以检测并规避异常值的影响. 稳健算法应用红外光谱分析中可望优化定性、定量预测模型.     

An enhanced Monte Carlo outlier detection method

Outlier detection is crucial in building a highly predictive model. In this study, we proposed an enhanced Monte Carlo outlier detection method by establishing cross‐prediction models based on determinate normal samples and analyzing the distribution of prediction errors individually for dubious samples. One simulated and three real datasets were used to illustrate and validate the performance of our method, and the results indicated that this method outperformed Monte Carlo outlier detection in outlier diagnosis. After these outliers were removed, the value of validation by Kovats retention indices and the root mean square error of prediction decreased from 3.195 to 1.655, and the average cross‐validation prediction error decreased from 2.0341 to 1.2780. This method helps establish a good model by eliminating outliers. © 2015 Wiley Periodicals, Inc.     

ATM交易状态特征分析与异常检测

考虑ATM交易过程当中产生的一系列参数,如交易量、交易成功率和响应时间等,对交易状态特征进行分析并建立了异常检测模型。针对成功率与响应时间2个参数,利用聚类算法将数据点划分为正常点、疑似异常点、异常点3大类。对于疑似的异常点,再根据其时间序列周围点的分布情况确定是否确实为异常点;对于交易量参数,首先通过LOF局部离群因子对离群点进行识别,再结合交易量随时间的移动均线及标准差加以辅助筛选,得到初步的疑似异常点,进一步通过与不同天同一时刻数据进行比较,最终确定是否为异常点。根据上述模型,本文将异常情况划分为3个预警等级,并对重大故障情况进行预测。     

抗野值自适应卡尔曼滤波方法的研究

针对陀螺漂移信号中野值的不良影响。从量测噪声自适应Kalman滤波算法出发，首次提出了抗野值自适应Kalman滤波方法，并在其基础上进行了分析，提出一种改进的算法．通过对某型号高精度寻北仪中的试验证明，该算法能够有效去除寻北仪信号中的野值，提高寻北仪的寻北精度，并且对系统的量测噪声不敏感．     

基于灰色预测和KID离群点检测的AUV故障诊断

提高故障诊断能力对于确保水下机器人AUV系统的稳定运行具有重要意义。针对水下机器人推进器系统,提出一种基于离群点检测的AUV故障检测方法。首先,将传感器采集的数据进行灰色预测处理;然后,提出了一种结合K-mean和DBSCAN的改进迭代聚类(Iterative K-mean DBSCAN,IKD)算法进行离群点检测;最后,与K-mean和DBSCAN算法相比,仿真实验结果表明基于灰色预测和KID离群点检测算法的故障检测准确率高,能够有效地实现水下机器人AUV的无监督故障诊断。     

Detection of material property errors in handbooks and databases using artificial neural networks with hidden correlations

The authors have discovered a systematic, intelligent and potentially automatic method to detect errors in handbooks and stop their transmission using unrecognised relationships between materials properties. The scientific community relies on the veracity of scientific data in handbooks and databases, some of which have a long pedigree covering several decades. Although various outlier-detection procedures are employed to detect and, where appropriate, remove contaminated data, errors, which had not been discovered by established methods, were easily detected by our artificial neural network in tables of properties of the elements. We started using neural networks to discover unrecognised relationships between materials properties and quickly found that they were very good at finding inconsistencies in groups of data. They reveal variations from 10 to 900% in tables of property data for the elements and point out those that are most probably correct. Compared with the statistical method adopted by Ashby and co-workers [Proc. R. Soc. Lond. Ser. A 454 (1998) p. 1301, 1323], this method locates more inconsistencies and could be embedded in database software for automatic self-checking. We anticipate that our suggestion will be a starting point to deal with this basic problem that affects researchers in every field. The authors believe it may eventually moderate the current expectation that data field error rates will persist at between 1 and 5%.     

模糊卡尔曼滤波器在初始对准抗野值修正中应用

Kalman滤波器广泛应用于捷联导航系统，在实际系统中量测序列由于各种原因可能出现错误的量测值或野值，会影响滤波器的估计效果。采用基于模糊控制器的卡尔曼滤波器，可以有效减小野值对实际系统的影响，提高滤波质量，改善系统精度。     

异常交易行为的甄别研究

本文在无指导学习的研究框架下,运用分位数回归模型结合变点检验,对中国证券市场的异常交易行为进行甄别研究。通过分析持股比例变动与股价收益率间协同演化关系的异常,为甄别异常交易行为设立判别标准并客观的界定阈值提供了一种新的方法。基于这一方法监管者可以构建分期、分级、分类的实时监管体系,提高监管效率。     

On Selection Criteria for the Tuning Parameter in Robust Divergence

Although robust divergence, such as density power divergence and γ-divergence, is helpful for robust statistical inference in the presence of outliers, the tuning parameter that controls the degree of robustness is chosen in a rule-of-thumb, which may lead to an inefficient inference. We here propose a selection criterion based on an asymptotic approximation of the Hyvarinen score applied to an unnormalized model defined by robust divergence. The proposed selection criterion only requires first and second-order partial derivatives of an assumed density function with respect to observations, which can be easily computed regardless of the number of parameters. We demonstrate the usefulness of the proposed method via numerical studies using normal distributions and regularized linear regression.     

海量星系光谱中的超新星候选范围自动约减

超新星是恒星世界中已知道的最剧烈的天文现象之一,但目前发现的超新星数量相比于已探测到的数百亿天体而言又是有限的,所以有必要寻找快速高效的超新星搜寻方法或辅助手段。拟在Ⅰa型超新星统计特征描述的基础上提出了一种海量星系光谱下Ⅰa型超新星候选体选择范围自动约减的方法。该方法首先对Ⅰa型超新星模板PCA分析获得特征谱,并获得每条待检星系光谱的低维超新星特征描述,然后通过引入样本的局部孤立性因子进行离群搜索,最后获得总样本数的1%作为继续搜寻证认超新星候选体选择的初始范围。实验表明该方法有效可行,这一方法在海量光谱中自动去除大量不含超新星的星系光谱,为超新星的进一步搜寻证认和后续观测提供了较可靠的候选范围,从而成为直接利用光谱巡天的海量数据获得超新星的高效途径。