光谱流量标准化的高效计算

流量标准化是光谱数据挖掘中的一个基本环节,他对挖掘结果的精度和系统的效率均有重要影响,常用方法存在效率较低的问题,为此研究了光谱数据挖掘中流量标准化的算法设计和效率比较问题。首先,探讨了光谱流量标准化技术不同实现方案的渐进效率,给出了实现高效计算的算法,并分析了它们的时间复杂度和空间复杂度。然后,通过SDSS(sloan digital sky survey)的实测光谱数据,横向比较了不同流量标准化算法的效率差异。在光谱流量标准化算法的纵向理论研究中,主要考虑的是计算效率随数据规模增长的变化规律,是在极限意义下进行探讨。在横向实验比较中,考虑重点是不同算法中基本操作时间复杂度的差异及其对算法效率的影响。理论研究和实验结果表明,虽然四种标准化方法Smax,Smedian,Smean和Sunit的渐进效率的类型相同,但对常见的观测规模光谱数据来说,Smax和Smean的效率远远高于Sunit和Smedian,且常用的Sunit标准化方法效率最低。该研究对于在光谱数据挖掘和开发中,如何根据数据的规模,具体需求,从整体上考虑精度和效率的折衷,以确定合适的流量标准化方法有重要的参考价值。

Efficient computation of spectral flux normalization

Flux normalization is a key procedure in spectral data mining, and is important for the efficiency and accuracy of automatic processing of massive astronomical spectral data, information extraction and sharing. Since the usual implementation of flux normalizing methods is inefficient, the present work focuses on the algorithm designing of spectral flux normalization. Firstly, the authors investigated the limit efficiency characteristics of the available flux normalization methods, introduced four efficient flux normalizing algorithms, and studied their time complexity and space complexity. Secondly, the authors evaluated the efficiency of the proposed algorithms experimentally and horizontally based on the SDSS (Sloan Digital Sky Survey) released spectral data. In the theoretical research, the main consideration is the computational complexity characteristics of the flux normalization methods when the data size increases unlimitedly. The experimental research focuses on the difference in the computational burden between the basic operations in different flux normalization methods. It is shown that, although the four flux normalization methods S(max), S(median), S(mean) and S(unit) belong to the same limit efficiency type, on the spectra with usual observing scale, S(max) and S(median) are much more efficient than S(mean) and S(unit), and S(unit) is the most inefficient one. This work is helpful for choosing the appropriate flux normalization method based on the size of spectra database and the scientific needs in automatic spectra analysis.