Adversarially Training MCMC with Non-Volume-Preserving Flows

Recently, flow models parameterized by neural networks have been used to design efficient Markov chain Monte Carlo (MCMC) transition kernels. However, inefficient utilization of gradient information of the target distribution or the use of volume-preserving flows limits their performance in sampling from multi-modal target distributions. In this paper, we treat the training procedure of the parameterized transition kernels in a different manner and exploit a novel scheme to train MCMC transition kernels. We divide the training process of transition kernels into the exploration stage and training stage, which can make full use of the gradient information of the target distribution and the expressive power of deep neural networks. The transition kernels are constructed with non-volume-preserving flows and trained in an adversarial form. The proposed method achieves significant improvement in effective sample size and mixes quickly to the target distribution. Empirical results validate that the proposed method is able to achieve low autocorrelation of samples and fast convergence rates, and outperforms other state-of-the-art parameterized transition kernels in varieties of challenging analytically described distributions and real world datasets.     

Using Perturbed Underdamped Langevin Dynamics to Efficiently Sample from Probability Distributions

In this paper we introduce and analyse Langevin samplers that consist of perturbations of the standard underdamped Langevin dynamics. The perturbed dynamics is such that its invariant measure is the same as that of the unperturbed dynamics. We show that appropriate choices of the perturbations can lead to samplers that have improved properties, at least in terms of reducing the asymptotic variance. We present a detailed analysis of the new Langevin sampler for Gaussian target distributions. Our theoretical results are supported by numerical experiments with non-Gaussian target measures.     

基于XRF与RBF神经网络的锌浸出渣有价组分精准定量分析研究

锌冶炼浸出渣是湿法炼锌工艺产出的冶炼固废渣,占锌冶炼固废产出总量的75％ 以上,因含有Zn,Cu,Pb,Ag,Cd和As等多种有价金属元素,其资源化利用潜力巨大.然而由于其成分含量不稳定,检测精度不足等原因,导致关键元素的资源转化效率难以保证,因此对浸出渣关键资源组分的精准定量分析在锌冶炼行业绿色发展方面具有重大意义....     

基于神经网络的粗糙表面双向反射分布函数建模

分析粗糙表面双向反射分布函数的测量方法,提出一种使用人工神经网络技术建立目标表面材料双向反射分布函数模型的方法。给出测量样品多个入射角度下的BRDF随散射角变化的曲线,从中选取部分曲线输入到神经网络,使用贝叶斯正则化方法训练网络,最终获取双向反射分布函数和入射角、散射角的映射关系模型。使用网络模型计算参与训练和未参与训练的输入角度的散射分布曲线,与实验测量曲线进行比较,结果表明：建立的模型正确,具有应用价值。     

利用Bayesian-MCMC方法从雷达回波反演海洋波导

应用贝叶斯-蒙特卡罗（Bayesian-MCMC）方法将海洋波导参数的先验信息描述为先验概率密度,结合雷达回波资料（电磁波传播损耗）,得到待反演海洋波导参数的后验概率密度,用马尔可夫链蒙特卡罗（MCMC）-Gibbs采样器采样后验概率密度分布,并用样本最大似然估计值作为对海洋波导参数分布的估计.数值实验结果表明,该方法对先验信息进行了有效利用,反演精度高于遗传算法的反演精度.该方法较为充分利用先验信息,得到解的概率分布,即解的不确定性分析,这在实际应用中有一定的参考价值. 关键词： 波导 电磁波传播损耗 贝叶斯-蒙特卡罗 概率分布     

A Field Study of Three Collocated Ambient PM10 Samplers

The performances of three ambient PM₁₀ samplers were studied at three monitoring stations in Taiwan. It was found that differences in the daily measured PM₁₀ concentrations of the SA 1200 and Wedding high-volume samplers are now within ± 10% since the former now has a closer cut-point to the latter than in the earlier SA 321 A model. The Wedding beta gauge automatic sampler was found to be applieable in rainy and humid weather conditions in Taiwan. Its daily PM₁₀ concentrations are typically within ± 10% of those of the Wedding highvolume sampler. The particle loading effect of Wedding highvolume and beta gauge samplers was found to be important. To avoid sampling errors due to the loading effect with ambient PM₁₀ samplers, they must be cleaned regularly at an interval depending on the ambient particulate level.     

蒙特卡罗粒子输运软件JMCT抽样工具库设计

为通用型蒙特卡罗粒子输运程序JMCT设计了抽样工具库,通过两种技术途径提供各分布的抽样。一是针对各种常见分布提供特定抽样子程序;二是提供一个通用型的抽样子程序,可以实现任意离散分布和任意一维有限区间上连续分布的自动抽样。在设计任意一维有限区间上连续分布的自动抽样工具时利用了部分开源代码,利用其功能提供给用户最大的方便性。对抽样工具库的检验表明,其可以正确、方便地实现各种输运模拟中常见分布的抽样。     

Optimal Randomness for Stochastic Configuration Network (SCN) with Heavy-Tailed Distributions

Stochastic Configuration Network (SCN) has a powerful capability for regression and classification analysis. Traditionally, it is quite challenging to correctly determine an appropriate architecture for a neural network so that the trained model can achieve excellent performance for both learning and generalization. Compared with the known randomized learning algorithms for single hidden layer feed-forward neural networks, such as Randomized Radial Basis Function (RBF) Networks and Random Vector Functional-link (RVFL), the SCN randomly assigns the input weights and biases of the hidden nodes in a supervisory mechanism. Since the parameters in the hidden layers are randomly generated in uniform distribution, hypothetically, there is optimal randomness. Heavy-tailed distribution has shown optimal randomness in an unknown environment for finding some targets. Therefore, in this research, the authors used heavy-tailed distributions to randomly initialize weights and biases to see if the new SCN models can achieve better performance than the original SCN. Heavy-tailed distributions, such as Lévy distribution, Cauchy distribution, and Weibull distribution, have been used. Since some mixed distributions show heavy-tailed properties, the mixed Gaussian and Laplace distributions were also studied in this research work. Experimental results showed improved performance for SCN with heavy-tailed distributions. For the regression model, SCN-Lévy, SCN-Mixture, SCN-Cauchy, and SCN-Weibull used less hidden nodes to achieve similar performance with SCN. For the classification model, SCN-Mixture, SCN-Lévy, and SCN-Cauchy have higher test accuracy of 91.5%, 91.7% and 92.4%, respectively. Both are higher than the test accuracy of the original SCN.     

A Method for Estimating the Entropy of Time Series Using Artificial Neural Networks

Measuring the predictability and complexity of time series using entropy is essential tool designing and controlling a nonlinear system. However, the existing methods have some drawbacks related to the strong dependence of entropy on the parameters of the methods. To overcome these difficulties, this study proposes a new method for estimating the entropy of a time series using the LogNNet neural network model. The LogNNet reservoir matrix is filled with time series elements according to our algorithm. The accuracy of the classification of images from the MNIST-10 database is considered as the entropy measure and denoted by NNetEn. The novelty of entropy calculation is that the time series is involved in mixing the input information in the reservoir. Greater complexity in the time series leads to a higher classification accuracy and higher NNetEn values. We introduce a new time series characteristic called time series learning inertia that determines the learning rate of the neural network. The robustness and efficiency of the method is verified on chaotic, periodic, random, binary, and constant time series. The comparison of NNetEn with other methods of entropy estimation demonstrates that our method is more robust and accurate and can be widely used in practice.     

A Bayesian Approach to the Estimation of Parameters and Their Interdependencies in Environmental Modeling

We present a case study for Bayesian analysis and proper representation of distributions and dependence among parameters when calibrating process-oriented environmental models. A simple water quality model for the Elbe River (Germany) is referred to as an example, but the approach is applicable to a wide range of environmental models with time-series output. Model parameters are estimated by Bayesian inference via Markov Chain Monte Carlo (MCMC) sampling. While the best-fit solution matches usual least-squares model calibration (with a penalty term for excessive parameter values), the Bayesian approach has the advantage of yielding a joint probability distribution for parameters. This posterior distribution encompasses all possible parameter combinations that produce a simulation output that fits observed data within measurement and modeling uncertainty. Bayesian inference further permits the introduction of prior knowledge, e.g., positivity of certain parameters. The estimated distribution shows to which extent model parameters are controlled by observations through the process of inference, highlighting issues that cannot be settled unless more information becomes available. An interactive interface enables tracking for how ranges of parameter values that are consistent with observations change during the process of a step-by-step assignment of fixed parameter values. Based on an initial analysis of the posterior via an undirected Gaussian graphical model, a directed Bayesian network (BN) is constructed. The BN transparently conveys information on the interdependence of parameters after calibration. Finally, a strategy to reduce the number of expensive model runs in MCMC sampling for the presented purpose is introduced based on a newly developed variant of delayed acceptance sampling with a Gaussian process surrogate and linear dimensionality reduction to support function-valued outputs.     

Geometric Partition Entropy: Coarse-Graining a Continuous State Space

Entropy is re-examined as a quantification of ignorance in the predictability of a one dimensional continuous phenomenon. Although traditional estimators for entropy have been widely utilized in this context, we show that both the thermodynamic and Shannon’s theory of entropy are fundamentally discrete, and that the limiting process used to define differential entropy suffers from similar problems to those encountered in thermodynamics. In contrast, we consider a sampled data set to be observations of microstates (unmeasurable in thermodynamics and nonexistent in Shannon’s discrete theory), meaning, in this context, it is the macrostates of the underlying phenomenon that are unknown. To obtain a particular coarse-grained model we define macrostates using quantiles of the sample and define an ignorance density distribution based on the distances between quantiles. The geometric partition entropy is then just the Shannon entropy of this finite distribution. Our measure is more consistent and informative than histogram-binning, especially when applied to complex distributions and those with extreme outliers or under limited sampling. Its computational efficiency and avoidance of negative values can also make it preferable to geometric estimators such as k-nearest neighbors. We suggest applications that are unique to this estimator and illustrate its general utility through an application to time series in the approximation of an ergodic symbolic dynamics from limited observations.     

How to Promote the Performance of Parametric Volatility Forecasts in the Stock Market? A Neural Networks Approach

This study uses the fourteen stock indices as the sample and then utilizes eight parametric volatility forecasting models and eight composed volatility forecasting models to explore whether the neural network approach and the settings of leverage effect and non-normal return distribution can promote the performance of volatility forecasting, and which one of the sixteen models possesses the best volatility forecasting performance. The eight parametric volatility forecasts models are composed of the generalized autoregressive conditional heteroskedasticity (GARCH) or GJR-GARCH volatility specification combining with the normal, Student’s t, skewed Student’s t, and generalized skewed Student’s t distributions. Empirical results show that, the performance for the composed volatility forecasting approach is significantly superior to that for the parametric volatility forecasting approach. Furthermore, the GJR-GARCH volatility specification has better performance than the GARCH one. In addition, the non-normal distribution does not have better forecasting performance than the normal distribution. In addition, the GJR-GARCH model combined with both the normal distribution and a neural network approach has the best performance of volatility forecasting among sixteen models. Thus, a neural network approach significantly promotes the performance of volatility forecasting. On the other hand, the setting of leverage effect can encourage the performance of volatility forecasting whereas the setting of non-normal distribution cannot.     

飞行试验中的动态压力测试方法与实现

测量动态压力信号的传感器频响较高,信号范围较宽;针对被测对象的特点,依托现有采集器具有的多通道同时采集、FRI(有限长冲击响应)数字滤波、单通道高采样率、16位的高分辨率等技术特点[3],设计一款高性能调节器;它与采集器的特性形成优势互补,组成的采集系统既能准确采集动态压力的全压信号、同时又能分离出其中的交流分量进行采集;通过验证,该测试系统能够为飞行试验提供准确可靠的动态压力数据,成功的解决了以往的动态压力测试技术难题。     

白矮主序双星光谱的卷积特征提取

通过卷积运算提取白矮主序双星的光谱特征是提高识别精度的有效手段。通过设计一维卷积神经网络,以判别学习的方式从大量混合光谱中拟合出具有稳定分布的12个卷积核,有效提取白矮主序双星的卷积特征。通过引入相对松弛的光谱类别先验分布,提出反贝叶斯学习策略以解决由于光谱抽样有偏带来的问题,显著提高识别精度。通过比较光谱在不同信噪比下的交叉熵测试误差,分析卷积特征的提取过程对光谱信噪比的鲁棒性。实验发现,基于反贝叶斯学习策略的一维卷积神经网络对白矮主序双星的识别准确率达到99.0(±0.3),超过了经典的PCA+SVM模型。卷积特征谱的池化过程以降低光谱分辨率的形式缓解了光谱噪声对识别精度的影响。当信噪比小于3时,必须通过增加模型在光谱上的迭代次数以形成稳定的卷积核;当信噪比介于3与6之间时,光谱卷积特征较为稳定;当信噪比大于6时,光谱卷积特征的稳定性显著上升,信噪比对于模型识别精度带来的影响可以忽略。     

SELECTION OF TRAINING SAMPLES FOR MODEL UPDATING USING NEURAL NETWORKS

One unique feature of neural networks is that they have to be trained to function. In developing an iterative neural network technique for model updating of structures, it has been shown that the number of training samples required increases exponentially as the number of parameters to be updated increases. Training the neural network using these samples becomes a time-consuming task. In this study, we investigate the use of orthogonal arrays for the sample selection. A comparison between this orthogonal arrays method and four other methods is illustrated by two numerical examples. One is the update of the felxural rigidities of a simply supported beam and the other is the update of the material properties and the boundary conditions of a circular plate. The results indicate that the orthogonal arrays method can significantly reduce the number of training samples without affecting too much the accuracy of the neural network prediction.     

Estimation for Entropy and Parameters of Generalized Bilal Distribution under Adaptive Type II Progressive Hybrid Censoring Scheme

Entropy measures the uncertainty associated with a random variable. It has important applications in cybernetics, probability theory, astrophysics, life sciences and other fields. Recently, many authors focused on the estimation of entropy with different life distributions. However, the estimation of entropy for the generalized Bilal (GB) distribution has not yet been involved. In this paper, we consider the estimation of the entropy and the parameters with GB distribution based on adaptive Type-II progressive hybrid censored data. Maximum likelihood estimation of the entropy and the parameters are obtained using the Newton–Raphson iteration method. Bayesian estimations under different loss functions are provided with the help of Lindley’s approximation. The approximate confidence interval and the Bayesian credible interval of the parameters and entropy are obtained by using the delta and Markov chain Monte Carlo (MCMC) methods, respectively. Monte Carlo simulation studies are carried out to observe the performances of the different point and interval estimations. Finally, a real data set has been analyzed for illustrative purposes.     

基于遗传神经网络的多光谱辐射测温法

针对BP神经网络易陷入局部极小等缺陷,将遗传算法(GA)与神经网络相结合,提出了一种将GA-BP算法应用于多光谱辐射测温的数据处理方法,并对基于亮度温度模型的多光谱辐射测温数据进行了仿真实验。结果表明：已训练样本的真实温度识别精度,GA-BP算法为±5 K,BP神经网络为±10 K;未训练样本的真实温度识别精度,GA-BP算法为±10 K,BP神经网络为±20 K;无论是GA-BP算法还是BP神经网络,已训练样本的真实温度识别精度比未训练样本的真实温度识别精度都更精确些,靠近训练样本集边缘的样本真实温度的识别精度偏低。说明GA-BP算法比BP神经网络可以更好地解决了目标真实温度的测量问题。     

Thermodynamics of rare events

Scaling laws of physics are derived from extreme value distributions. Small jump processes that comprise a compound Poisson distribution generate the asymptotic distributions of stable laws. These extreme value distributions, or their tails, can be expressed in terms of the entropy decrease. As an example, the scaling law for the radius of gyration of a polymer is derived which is comparable to Flory's formula. The entropy is identified by its property of concavity, which is shown to coincide with Boltzmann's probabilistic definition for first passage in a random walk. A more general definition is required for nonintegral dimensions. The relation to mean-field theory of the kinetic Weiss-Ising model is shown and this distribution of the order parameter is governed by an asymptotic distribution for the smallest value rather than a normal distribution. Finally, the logarithm of the sample size is shown to be the yardstick for the decrease in entropy.     

一种基于卷积神经网络的恒星光谱快速分类法

恒星光谱数据的分类是天体光谱自动识别的最基本任务之一,光谱分类的研究能够为恒星的演化提供线索。随着科技的发展,天文数据也向大数据时代迈进,需要处理的恒星光谱数量越来越多,如何对其进行自动而精准地分类成为了天文学家要解决的难题之一。当前恒星光谱自动分类问题的解决方法相对较少,为此本文使用了一种基于卷积神经网络的方法对恒星光谱MK系统进行分类。该网络由数据输入层、四个卷积层、四个池化层、全连接层、输出层构成,与传统网络相比具有局部感知、参数共享等优点实验。在Python3.5的环境下编程,利用Tensorflow构建了一个简单高效的具有四个卷积层的卷积神经网络,并将Dropout作用于全连接层之后以防止过度拟合。Dropout的基本思想：当网络模型进行训练时,把一些神经网络节点按一定的比例丢弃,使其暂时不发挥作用。Dropout可以理解成是一种十分高效的神经网络模型平均方法,由于它不依赖于某些局部特征所以能够让网络模型更加鲁棒。实验中使用的一维恒星光谱图是取自LAMOST DR3数据库,首先进行预处理截取光谱3 600~7 300 Å的部分,均匀采样后使用min-max标准化法对其进行初始化。实验包括两部分：第一部分为依据恒星光谱MK系统对光谱进行分类,每一类的训练样本包含1 000条光谱数据,测试样本为400条光谱数据,首先通过训练样本对CNN网络进行训练,进行3 000次的迭代,用训练后的网络将测试样本进行分类以验证网络的准确性;第二部分为相邻两类的恒星光谱的分类,其中O型星数据集样本为250条光谱,其余类别恒星样本数据集均为4 000条光谱,将数据5等分,每次选取当中的一份当作测试集,其余部分当作训练集,采用5折交叉验证法求得模型准确率,用BP神经网络进行对比实验。选择对网络模型进行评估的指标包括精确率P、召回率R、F-score、准确率A。实验结果显示CNN在对六类恒星光谱进行分类时其准确率都在95%以上,在对相邻类别的恒星进行分类时,由于O型星样本量较少,所以得到的分类结果不太理想,对其余类别的恒星分类准确率都高于98%,以上结果都证明了CNN算法能够很好地解决恒星光谱的分类问题。     

基于神经网络的大孔径厚针孔成像复原算法研究

为了更好地获取低强度辐射源空间分布图像,提出一种使用神经网络算法将大孔径厚针孔退化图像复原的方法。建立了孔径5 mm、10 mm、15 mm的厚针孔模型,获得了3600个汉字形状辐射源的厚针孔退化图像集。基于DnCNN神经网络模型,建立了大孔径厚针孔退化图像复原神经网络,并与维纳滤波、Lucy-Richardson这些传统算法进行了比较。在考虑噪声影响后,利用迁移学习理论,对原神经网络模型进行迁移训练,再对含噪大孔径厚针孔退化图像进行复原。神经网络算法复原的RMSE明显低于传统方法,迁移学习显著减小了噪声的影响。证明了神经网络算法在大孔径厚针孔退化图像复原领域的优越性,并验证了神经网络方法复原含噪大孔径厚针孔退化图像的可行性。