Generalized loopy 2U: A new algorithm for approximate inference in credal networks

Credal networks generalize Bayesian networks by relaxing the requirement of precision of probabilities. Credal networks are considerably more expressive than Bayesian networks, but this makes belief updating NP-hard even on polytrees. We develop a new efficient algorithm for approximate belief updating in credal networks. The algorithm is based on an important representation result we prove for general credal networks: that any credal network can be equivalently reformulated as a credal network with binary variables; moreover, the transformation, which is considerably more complex than in the Bayesian case, can be implemented in polynomial time. The equivalent binary credal network is then updated by L2U, a loopy approximate algorithm for binary credal networks. Overall, we generalize L2U to non-binary credal networks, obtaining a scalable algorithm for the general case, which is approximate only because of its loopy nature. The accuracy of the inferences with respect to other state-of-the-art algorithms is evaluated by extensive numerical tests.     

Constructive approximate interpolation by neural networks

We present a type of single-hidden layer feedforward neural networks with sigmoidal nondecreasing activation function. We call them ai-nets. They can approximately interpolate, with arbitrary precision, any set of distinct data in one or several dimensions. They can uniformly approximate any continuous function of one variable and can be used for constructing uniform approximants of continuous functions of several variables. All these capabilities are based on a closed expression of the networks.     

$L^p(K)$中RBF神经网络的系统识别问题

L^p approximation problems in system identification with RBF neural networks are investigated. It is proved that by superpositions of some functions of one variable in L^ploc（R）, one can approximate continuous functionals defined on a compact subset of L^P（K） and continuous operators from a compact subset of L^p1 （K1） to a compact subset of L^p2 （K2）. These results show that if its activation function is in L^ploc（R） and is not an even polynomial, then this RBF neural networks can approximate the above systems with any accuracy.     

Inference in hybrid Bayesian networks with mixtures of truncated exponentials

Mixtures of truncated exponentials (MTE) potentials are an alternative to discretization for solving hybrid Bayesian networks. Any probability density function (PDF) can be approximated with an MTE potential, which can always be marginalized in closed form. This allows propagation to be done exactly using the Shenoy–Shafer architecture for computing marginals, with no restrictions on the construction of a join tree. This paper presents MTE potentials that approximate an arbitrary normal PDF with any mean and a positive variance. The properties of these MTE potentials are presented, along with examples that demonstrate their use in solving hybrid Bayesian networks. Assuming that the joint density exists, MTE potentials can be used for inference in hybrid Bayesian networks that do not fit the restrictive assumptions of the conditional linear Gaussian (CLG) model, such as networks containing discrete nodes with continuous parents.     

单隐层神经网络与最佳多项式逼近

研究单隐层神经网络逼近问题．以最佳多项式逼近为度量,用构造性方法估计单隐层神经网络逼近连续函数的速度．所获结果表明:对定义在紧集上的任何连续函数,均可以构造一个单隐层神经网络逼近该函数,并且其逼近速度不超过该函数的最佳多项式逼近的二倍．     

����˿͹����Ŷ�ϵͳ����Ϣ����

The discriminatory processor sharing queues with multiple classes of customers (abbreviated as DPS queues) are an important but difficult research direction in queueing theory, and it has many important practical applications in the fields of, such as, computer networks, manufacturing systems, transportation networks, and so forth. Recently, researchers have carried out some key work for the DPS queues. They gave the generating function of the steady-state joint queue lengths, which leads to the first two moments of the steady-state joint queue lengths. However, using the generating function to provide explicit expressions for the steady-state joint queue lengths has been a difficult and challenging problem for many years. Based on this, this paper applies the maximum entropy principle in the information theory to providing an approximate expression with high precision, and this approximate expression can have the same first three moments as those of its exact expression. On the other hand, this paper gives efficiently numerical computation by means of this approximate expression, and analyzes how the key variables of this approximate expression depend on the original parameters of this queueing system in terms of some numerical experiments. Therefore, this approximate expression has important theoretical significance to promote practical applications of the DPS queues. At the same time, not only do the methodology and results given in this paper provide a new line in the study of DPS queues, but they also provide the theoretical basis and technical support for how to apply the information theory to the study of queueing systems, queueing networks and more generally, stochastic models.     

具有随机隐单元的渐增前馈神经网络的$L^2(R^d)$逼近能力

This paper studies approximation capability to L2(Rd) functions of incremental constructive feedforward neural networks(FNN) with random hidden units.Two kinds of therelayered feedforward neural networks are considered:radial basis function(RBF) neural networks and translation and dilation invariant(TDI) neural networks.In comparison with conventional methods that existence approach is mainly used in approximation theories for neural networks,we follow a constructive approach to prove that one may simply randomly choose parameters of hidden units and then adjust the weights between the hidden units and the output unit to make the neural network approximate any function in L2(Rd) to any accuracy.Our result shows given any non-zero activation function g :R+→R and g(x Rd) ∈ L2(Rd) for RBF hidden units,or any non-zero activation function g(x) ∈ L2(Rd) for TDI hidden units,the incremental network function fn with randomly generated hidden units converges to any target function in L2(Rd) with probability one as the number of hidden units n→∞,if one only properly adjusts the weights between the hidden units and output unit.     

Triply fuzzy function approximation for hierarchical Bayesian inference

We prove that three independent fuzzy systems can uniformly approximate Bayesian posterior probability density functions by approximating the prior and likelihood probability densities as well as the hyperprior probability densities that underly the priors. This triply fuzzy function approximation extends the recent theorem for uniformly approximating the posterior density by approximating just the prior and likelihood densities. This approximation allows users to state priors and hyper-priors in words or rules as well as to adapt them from sample data. A fuzzy system with just two rules can exactly represent common closed-form probability densities so long as they are bounded. The function approximators can also be neural networks or any other type of uniform function approximator. Iterative fuzzy Bayesian inference can lead to rule explosion. We prove that conjugacy in the if-part set functions for prior, hyperprior, and likelihood fuzzy approximators reduces rule explosion. We also prove that a type of semi-conjugacy of if-part set functions for those fuzzy approximators results in fewer parameters in the fuzzy posterior approximator.     

Neural networks for of continuous optimal approximation functions in R^d

Using some regular matrices we present a method to express any multivariate algebraic polynomial of total order n in a normal form. Consequently, we prove constructively that, to approximate continuous target functions defined on some compact set of ? ^d , neural networks are at least as good as algebraic polynomials.     

非线性时间序列的投影寻踪学习网络逼近

本文研究非线性自回归模型投影寻踪学习网络逼近的收敛性,证明了在L^k(k为正整数）空间上,投影寻踪学习网络可以以任意精度逼近非线性自回归模型,给出基于投影寻踪学习网络的非线性时间序列模型建模和预报的计算方法和应用实例,对太阳黑子数据,山猫数据及西安数据进行了拟合和预报,将其结果与改进BP网和门限自回归模型相应的结果进行比较,结果表明基于投影寻踪学习网络的非线性时间序列的建模预报方法是一类行之有效的方法。     

Approximation properties of a multilayered feedforward artificial neural network

We prove that an artificial neural network with multiple hidden layers and akth-order sigmoidal response function can be used to approximate any continuous function on any compact subset of a Euclidean space so as to achieve the Jackson rate of approximation. Moreover, if the function to be approximated has an analytic extension, then a nearly geometric rate of approximation can be achieved. We also discuss the problem of approximation of a compact subset of a Euclidean space with such networks with a classical sigmoidal response function.Dedicated to Dr. C.A. Micchelli on the occasion of his fiftieth birthday, December 1992Research supported in part by AFOSR Grant No. 226 113 and by the AvH Foundation.     

Verification of approximate opacity for switched systems: A compositional approach

The security in information-flow has become a major concern for cyber–physical systems (CPSs). In this work, we focus on the analysis of an information-flow security property, called opacity. Opacity characterizes the plausible deniability of a system’s secret in the presence of a malicious outside intruder. We propose a methodology of checking a notion of opacity, called approximate opacity, for networks of discrete-time switched systems. Our framework relies on compositional constructions of finite abstractions for networks of switched systems and their approximate opacity-preserving simulation functions. Those functions characterize how close concrete networks and their finite abstractions are in terms of the satisfaction of approximate opacity. We show that such simulation functions can be obtained compositionally by assuming some small-gain type conditions and composing local simulation functions constructed for each switched subsystem separately. Additionally, assuming certain stability property of switched systems, we also provide a technique on constructing their finite abstractions together with the corresponding local simulation functions. Finally, we illustrate the effectiveness of our results through an example.     

岩石裂隙网络的分形特征

分形特征与分形维数广泛应用于岩石裂隙网络的量化,及与工程参数的关系模型建立.然而,严格的分形维数的极限定义形式难以直接应用,工程应用中多用近似分形维数值代替,近似的结果在建立量化关系模型时会产生蝴蝶效应,在量化及预测过程中产生巨大偏差.本文回顾了分形研究一系列的发展过程,并基于最新的分形定义提出了一种新的分形维数计算方法.通过对于十个岩石裂隙网络分形维数的计算,证明该方法能够准确有效的计算出图形的复杂度,避免了以往计算分形维数所产生的问题.     

多层前向正则模糊神经网络的逼近能力

Abstract. Four-layer feedforward regular fuzzy neural networks are constructed. Universal ap-proximations to some continuous fuzzy functions defined on (R)“ by the four-layer fuzzyneural networks are shown. At first,multivariate Bernstein polynomials associated with fuzzyvalued functions are empolyed to approximate continuous fuzzy valued functions defined on eachcompact set of R“. Secondly,by introducing cut-preserving fuzzy mapping,the equivalent condi-tions for continuous fuzzy functions that can be arbitrarily closely approximated by regular fuzzyneural networks are shown. Finally a few of sufficient and necessary conditions for characteriz-ing approximation capabilities of regular fuzzy neural networks are obtained. And some concretefuzzy functions demonstrate our conclusions.     

三层前向神经网络的一致逼近性

近年来，前向神经网络泛逼近的一致性分析一直为众多学者所重视。本文系统分析三层前向网络对于拟差值保序函数族的一致逼近性，其中，转换函数σ是广义Sigmoidal函数。并将此一致性结果用于建立一类新的模糊神经网络(FNN)，即折线FNN．研究这类网络对于两个给定的模糊函数的逼近性，相关结论在分析折线FNN的泛逼近性时起关键作用。     

Approximating Networks and Extended Ritz Method for the Solution of Functional Optimization Problems

Functional optimization problems can be solved analytically only if special assumptions are verified; otherwise, approximations are needed. The approximate method that we propose is based on two steps. First, the decision functions are constrained to take on the structure of linear combinations of basis functions containing free parameters to be optimized (hence, this step can be considered as an extension to the Ritz method, for which fixed basis functions are used). Then, the functional optimization problem can be approximated by nonlinear programming problems. Linear combinations of basis functions are called approximating networks when they benefit from suitable density properties. We term such networks nonlinear (linear) approximating networks if their basis functions contain (do not contain) free parameters. For certain classes of d-variable functions to be approximated, nonlinear approximating networks may require a number of parameters increasing moderately with d, whereas linear approximating networks may be ruled out by the curse of dimensionality. Since the cost functions of the resulting nonlinear programming problems include complex averaging operations, we minimize such functions by stochastic approximation algorithms. As important special cases, we consider stochastic optimal control and estimation problems. Numerical examples show the effectiveness of the method in solving optimization problems stated in high-dimensional settings, involving for instance several tens of state variables.     

Diffusion approximations for open multiclass queueing networks: sufficient conditions involving state space collapse

Certain diffusion processes known as semimartingale reflecting Brownian motions (SRBMs) have been shown to approximate many single class and some multiclass open queueing networks under conditions of heavy traffic. While it is known that not all multiclass networks with feedback can be approximated in heavy traffic by SRBMs, one of the outstanding challenges in contemporary research on queueing networks is to identify broad categories of networks that can be so approximated and to prove a heavy traffic limit theorem justifying the approximation. In this paper, general sufficient conditions are given under which a heavy traffic limit theorem holds for open multiclass queueing networks with head-of-the-line (HL) service disciplines, which, in particular, require that service within each class is on a first-in-first-out (FIFO) basis. The two main conditions that need to be verified are that (a) the reflection matrix for the SRBM is well defined and completely- S, and (b) a form of state space collapse holds. A result of Dai and Harrison shows that condition (a) holds for FIFO networks of Kelly type and their proof is extended here to cover networks with the HLPPS (head-of-the-line proportional processor sharing) service discipline. In a companion work, Bramson shows that a multiplicative form of state space collapse holds for these two families of networks. These results, when combined with the main theorem of this paper, yield new heavy traffic limit theorems for FIFO networks of Kelly type and networks with the HLPPS service discipline. This revised version was published online in June 2006 with corrections to the Cover Date.     

Complexity of Neural Network Approximation with Limited Information: A Worst Case Approach

In neural network theory the complexity of constructing networks to approximate input-output functions is of interest. We study this in the more general context of approximating elements f of a normed space F using partial information about f. We assume information about f and the size of the network are limited, as is typical in radial basis function networks. We show complexity can be essentially split into two independent parts, information ε-complexity and neural ε-complexity. We use a worst case setting, and integrate elements of information-based complexity and nonlinear approximation. We consider deterministic and/or randomized approximations using information possibly corrupted by noise. The results are illustrated by examples including approximation by piecewise polynomial neural networks.     

A note on the expressive power of deep rectified linear unit networks in high‐dimensional spaces

We investigate the ability of deep deep rectified linear unit (ReLU) networks to approximate multivariate functions. Specially, we establish the approximation error estimate on a class of bandlimited functions; in this case, ReLU networks can overcome the “curse of dimensionality.”     

经济时间序列的连续参数小波网络预测模型

本文利用连续小波变换方法,给出一种连续参数小波网络。网络参数的学习采用一种类似神经网络的后向传播学习算法的随机梯度算法。另外,提出了一种借助小波分析理论指导网络参数赋初值的方法。进一步,通过对中国进出口贸易额时间序列预测建模的研究和仿真预测,提出了用连续参数小波网络建立经济时间序列预测模型的一般步骤和方法。预测结果表明,此模型具有较好的泛化、学习能力,可以有效地在数值上逼近时间序列难以定量描述的相互关系,所以利用连续参数小波网络建立的时间序列预测模型有较高的预测精度。