RELU DEEP NEURAL NETWORKS AND LINEAR FINITE ELEMENTS

In this paper, we investigate the relationship between deep neural networks (DNN) with rectified linear unit (ReLU) function as the activation function and continuous piecewise linear (CPWL) functions, especially CPWL functions from the simplicial linear finite element method (FEM). We first consider the special case of FEM. By exploring the DNN representation of its nodal basis functions, we present a ReLU DNN representation of CPWL in FEM. We theoretically establish that at least $2$ hidden layers are needed in a ReLU DNN to represent any linear finite element functions in $\Omega \subseteq \mathbb{R}^d$ when $d\ge2$. Consequently, for $d=2,3$ which are often encountered in scientific and engineering computing, the minimal number of two hidden layers are necessary and sufficient for any CPWL function to be represented by a ReLU DNN. Then we include a detailed account on how a general CPWL in $\mathbb R^d$ can be represented by a ReLU DNN with at most $\lceil\log_2(d+1)\rceil$ hidden layers and we also give an estimation of the number of neurons in DNN that are needed in such a representation. Furthermore, using the relationship between DNN and FEM, we theoretically argue that a special class of DNN models with low bit-width are still expected to have an adequate representation power in applications. Finally, as a proof of concept, we present some numerical results for using ReLU DNNs to solve a two-point boundary problem to demonstrate the potential of applying DNN for numerical solution of partial differential equations.     

基于动态神经网络的非线性系统鲁棒故障检测

针对一类具有不确定性扰动的非线性系统,将设计的系统线性观测器产生的误差信号作为残差,采用一种具有高斯型激励函数的动态神经网络(DNN)对残差信号进行分析处理,得到了系统的鲁棒故障检测方法.文中分析了该方法的稳定性和故障检测的鲁棒性,并通过算例验证了该方法的有效性.     

G-PNN: A genetically engineered probabilistic neural network

The probabilistic neural network (PNN) is a neural network architecture that approximates the functionality of the Bayesian classifier, the optimal classifier. Designing the optimal Bayesian classifier is infeasible in practice, since the distributions of data belonging to each class are unknown. PNN is an approximation of the Bayesian classifier by approximating these distributions using the Parzen window approach. One of the criticisms of the PNN classifier is that, at times, it uses a lot of training data for its design. Furthermore, the PNN classifier requires that the user specifies certain network parameters, called the smoothing (spread) parameters, in order to approximate the distributions of the class data, which is not an easy task. A number of approaches have been reported in the literature for addressing both of these issues (i.e., reducing the number of training data needed for the building of the PNN model and producing good values for the smoothing parameters). In this effort, genetic algorithms are used to achieve both goals at once, and some promising results are reported.     

Fast, Efficient Equipment Placement Heuristics for Broadband Switched or Internet Router Networks

Planning and designing the next generation of IP router or switched broadband networks seems a daunting challenge considering the many complex, interacting factors affecting the performance and cost of such networks. Generally, this complexity implies that it may not even be clear what constitutes a “good” network design for a particular specification. Different network owners or operators may view the same solution differently, depending on their unique needs and perspectives. Nevertheless, we have observed a core common issue arising in the early stages of network design efforts involving leading-edge broadband switched technologies such as ATM, Frame Relay, and SMDS; or even Internet IP router networks. This core issue can be stated as follows: Given a set of service demands for the various network nodes, where should switching or routing equipment be placed to minimize the Installed First Cost of the network? Note that the specified service demands are usually projections for a future scenario and generally entail significant uncertainty. Despite this uncertainty, we have found that network owners and operators generally feel it is worthwhile to obtain high-level advice on equipment placement with a goal of minimizing Installed First Cost. This paper reports on a heuristic approach we have implemented for this problem that has evolved out of real network design projects. A tool with both a Solution Engine and an intuitive Graphical User Interface has been developed. The approach is highly efficient; for example, the tool can often handle LATA-sized networks in seconds or less on a workstation processor. By using only nodal demands rather than the more complex point-to-point demands usually required in tools of this sort, we have created an approach that is not only highly efficient, but is also a better match to real design projects in which demand data is generally scant and highly uncertain.     

Material optimization of tri-directional functionally graded plates by using deep neural network and isogeometric multimesh design approach

The paper is aimed at enhancing computational performance for optimizing the material distribution of tri-directional functionally graded (FG) plates. We exploit advantages of using a non-uniform rational B-spline (NURBS) basis function for describing material distribution varying through all three directions of functionally graded (FG) plates. Two-dimensional free vibration and buckling behaviors of multi-directional (1D, 2D and 3D) FG plates analyzed by using a combination of generalized shear deformation theory (GSDT) and isogeometric analysis (IGA) is first proposed. This approach can help to save a significant amount of computational cost while still ensure the accuracy of the solutions. The effectiveness and reliability of the present method are demonstrated by comparing it to other methods in the literature. The obtained results are in excellent agreement with the reference ones. More importantly, data sets consisting of input-output pairs are randomly generated from the analysis process through iterations for the training process in deep neural networks (DNN). DNN is utilized as an analysis tool to supplant finite element analysis to reduce computational cost. By using DNN, behaviors of the multi-directional FG plates are directly predicted from those material distributions. Optimal material distributions of tri-directional FG plates under free vibration or compression in various volume fraction constraints are found by using modified symbiotic organisms search (mSOS) algorithm for the first time. Moreover, an isogeometric multimesh design technique is also used to diminish a large number of design variables in optimization. Optimal results obtained by DNN are compared with those of IGA to verify the effectiveness of the proposed method.     

基于灰色神经网络的企业风险特征指标动态预测方法研究

根据企业风险特征指标预测问题的特点,提出将灰色系统GM(1,1)模型与神经网络结合建立一阶灰色神经网络预测模型,以实现系统预测的动态性及提高系统的预测精度.但该模型具有一定的局限性,从模型参数的角度给出了该模型只适用于具有"单调"性数据的证明,进而提出了三阶灰色神经网络预测模型,以适应预测数据"非单调"或摆动的情况.但随着系统建模过程中阶数的增加,预测精度会有所下降,因此应根据数据特点选择预测模型.最后,通过实证分析验证了上述模型及证明结论.     

Multiresolution Network Models

Many existing statistical and machine learning tools for social network analysis focus on a single level of analysis. Methods designed for clustering optimize a global partition of the graph, whereas projection-based approaches (e.g., the latent space model in the statistics literature) represent in rich detail the roles of individuals. Many pertinent questions in sociology and economics, however, span multiple scales of analysis. Further, many questions involve comparisons across disconnected graphs that will, inevitably be of different sizes, either due to missing data or the inherent heterogeneity in real-world networks. We propose a class of network models that represent network structure on multiple scales and facilitate comparison across graphs with different numbers of individuals. These models differentially invest modeling effort within subgraphs of high density, often termed communities, while maintaining a parsimonious structure between said subgraphs. We show that our model class is projective, highlighting an ongoing discussion in the social network modeling literature on the dependence of inference paradigms on the size of the observed graph. We illustrate the utility of our method using data on household relations from Karnataka, India. Supplementary material for this article is available online.     

GasTurbnLab: a multidisciplinary problem solving environment for gas turbine engine design on a network of nonhomogeneous machines

Gas turbine engines are very complex (with 20–40,000 parts) and have extreme operating conditions. The important physical phenomena take place on scales from 10–100 microns to meters. A complete and accurate dynamic simulation of an entire engine is enormously demanding. Designing a complex system, like a gas turbine engine, will require fast, accurate simulations of computational models from multiple engineering disciplines along with sophisticated optimization techniques to help guide the design process. In this paper, we describe the architecture of an agent-based software framework for the simulation of various aspects of a gas turbine engine, utilizing a “network” of collaborating numerical objects through a set of interfaces among the engine parts. Moreover, we present its implementation using the Grasshopper agent middleware and provide simulation results that show the feasibility of the computational paradigm implemented.     

Gaussian kernel approximation algorithm for feedforward neural network design

A Gaussian kernel approximation algorithm for a feedforward neural network is presented. The approach used by the algorithm, which is based on a constructive learning algorithm, is to create the hidden units directly so that automatic design of the architecture of neural networks can be carried out. The algorithm is defined using the linear summation of input patterns and their randomized input weights. Hidden-layer nodes are defined so as to partition the input space into homogeneous regions, where each region contains patterns belonging to the same class. The largest region is used to define the center of the corresponding Gaussian hidden nodes. The algorithm is tested on three benchmark data sets of different dimensionality and sample sizes to compare the approach presented here with other algorithms. Real medical diagnoses and a biological classification of mushrooms are used to illustrate the performance of the algorithm. These results confirm the effectiveness of the proposed algorithm.     

The difference between doubly nonnegative and completely positive matrices

The convex cone of n×n completely positive (CP) matrices and its dual cone of copositive matrices arise in several areas of applied mathematics, including optimization. Every CP matrix is doubly nonnegative (DNN), i.e., positive semidefinite and component-wise nonnegative, and it is known that, for n4 only, every DNN matrix is CP. In this paper, we investigate the difference between 5×5 DNN and CP matrices. Defining a bad matrix to be one which is DNN but not CP, we: (i) design a finite procedure to decompose any n×n DNN matrix into the sum of a CP matrix and a bad matrix, which itself cannot be further decomposed; (ii) show that every bad 5×5 DNN matrix is the sum of a CP matrix and a single bad extreme matrix; and (iii) demonstrate how to separate bad extreme matrices from the cone of 5×5 CP matrices.     

Deep partial least squares for instrumental variable regression

In this paper, we propose deep partial least squares for the estimation of high-dimensional nonlinear instrumental variable regression. As a precursor to a flexible deep neural network architecture, our methodology uses partial least squares for dimension reduction and feature selection from the set of instruments and covariates. A central theoretical result, due to Brillinger (2012) Selected Works of Daving Brillinger. 589-606, shows that the feature selection provided by partial least squares is consistent and the weights are estimated up to a proportionality constant. We illustrate our methodology with synthetic datasets with a sparse and correlated network structure and draw applications to the effect of childbearing on the mother's labor supply based on classic data of Chernozhukov et al. Ann Rev Econ. (2015b):649–688. The results on synthetic data as well as applications show that the deep partial least squares method significantly outperforms other related methods. Finally, we conclude with directions for future research.     

Sharing Resources in a P2P Network with Artificial Ants

As networks are growing up, more and more information becomes available every day. Despite the presence of software enabling communications and content sharing, they are not always shared among people inside networks. We present here an architecture aimed at helping people to share information items and find collaborators inside an organization. It is part of our PIAF framework, an intelligent agent system used to develop recommender and personalization software. The main contribution of this paper is the introduction of principles of stigmergy and artificial ants to model data flows in a social network.      

Integrated supply chain planning under uncertainty using an improved stochastic approach

This paper proposes an integrated model and a modified solution method for solving supply chain network design problems under uncertainty. The stochastic supply chain network design model is provided as a two-stage stochastic program where the two stages in the decision-making process correspond to the strategic and tactical decisions. The uncertainties are mostly found in the tactical stage because most tactical parameters are not fully known when the strategic decisions have to be made. The main uncertain parameters are the operational costs, the customer demand and capacity of the facilities. In the improved solution method, the sample average approximation technique is integrated with the accelerated Benders’ decomposition approach to improvement of the mixed integer linear programming solution phase. The surrogate constraints method will be utilized to acceleration of the decomposition algorithm. A computational study on randomly generated data sets is presented to highlight the efficiency of the proposed solution method. The computational results show that the modified sample average approximation method effectively expedites the computational procedure in comparison with the original approach.     

Network structure,perception level,and participants' welfares

Based on a market consisting of one monopoly and several customers who are embedded in an economic network, we study how the different perception levels about the network structure affect the two kinds of participants' welfares, and then provide some good strategies for the monopoly to mine the information of the network structure. The above question is the embodiment of the “complex structure and its corresponding functions” question often mentioned in the field of complexity science. We apply a two‐stage game to solve for the optimal pricing and consumption at different perception levels of the monopoly and further utilize simulation analysis to explore the influence patterns. We also discuss how this theoretic model can be applied to a real world problem by introducing the statistical exponential random graph model and its estimation method. Further, the main findings have specific policy implications on uncovering network information and demonstrate that it is possible for the policy‐maker to design some win–win mechanisms for uplifting both the monopoly's profit and the whole customers' welfare at the same time. © 2014 Wiley Periodicals, Inc. Complexity 21: 349–362, 2015     

复杂疾病的分子网络模型研究

复杂疾病是危害人类健康的主要杀手.不同于单基因缺陷性遗传病,复杂疾病的发生发展与多个基因之间、基因与环境之间的相互作用有关,致病机理复杂,其早期诊断及治疗困难是21世纪生物医学研究的重大挑战之一.随着生物知识的不断积累和多层次"组学"数据的井喷式涌现,复杂疾病研究迎来了新的"组学革命",研究模式从以往的只关注某个分子扩展到对分子之间相互形成的生物分子网络的系统分析.作为系统生物学核心概念,生物分子网络系统整合大量生物知识和高通量生物数据,是研究复杂疾病的强有力工具.本文以分子网络为主线,以数学建模为工具来研究复杂疾病,针对复杂疾病关系和复杂疾病的发生发展机制等复杂疾病研究的关键热点问题,分析和集成高通量多层次组学数据,构建并求解生物分子网络的数学模型,在若干复杂疾病相关系统生物学问题中取得有生物学意义的结果.本文提出若干生物网络建模、分析及应用的方法并提供若干应用软件,为从系统层面理解复杂疾病提供重要参考;同时,网络模型在若干实例中的应用得到若干有生物学意义的结论,为揭示复杂疾病机理、推动疾病治疗与预防起到了一定的作用.     

Telecommunications Network Case Study: Selecting a Data Network Architecture

This paper documents a model that was pivotal in deciding which of two architectures should be selected for a frame relay data communications network. The choices are either to continue using the current architecture, or to make a large incremental investment in new equipment which reduces the number of high speed inter-office trunks required to interconnect the switches. The analysis requires optimizing the mix of two types of customer port cards to determine the maximum customer port capacity of a switch. Simple approximations are used to estimate the number of inter-office trunks and trunk cards required. Based in large part on the costs computed by this model, an executive level decision was made to move to the new architecture.     

基于神经网络的股票预测模型

针对股票市场的特征提取困难、预测精度较低等问题,本文基于深度学习算法,构建了一系列用于股票市场预测的神经网络模型,包括基于多层感知机(MLP)、卷积神经网络(CNN)、递归神经网络(RNN)、长短期记忆网络(LSTM)和门控神经单元(GRU)的模型。 针对RNN、LSTM和GRU无法充分利用所参考的时间维度的信息,引入注意力机制(Attention Mechanism) 给各时间维度的信息赋予不同权重,区分不同信息对预测的重要程度,从而提升递归网络模型的性能。上述模型均基于股票数据进行了优化,基于上证指数对各类模型进行了充分的对比实验,探索了模型中重要变量对性能的影响,旨在为基于神经网络的股票预测模型给出具体的优化方向。     

无标度网络演化过程中的结构熵

熵是度量复杂系统无序性的重要物理量,而且现实中的大多数网络都呈现出无标度网络的特性.在网络的节点熵和结构熵概念的基础上,给出了BA模型的网络结构熵演化的解析结论和数值模拟.从解析结论和数值模拟可以得到,网络结构熵随网络大小以对数的速度增长;但在同样规模下,无标度网络的结构熵小于随机网络的结构熵.     

基于ZigBee技术的黄河河道冰情多点监测系统设计

针对黄河河道冰情的多点监测问题,提出了一种基于ZigBee技术的河道冰层厚度多点监测系统的设计方案,简要介绍了ZigBee技术的主要特点,冰层厚度传感器结构与检测原理,提出了基于ZigBee的黄河河道冰情多点监测系统结构,讨论了监测系统硬件与软件的设计思路,利用系统可以实现黄河河道局部区域内多点冰情远程无人连续自动监测.     

An adaptive,multiple restarts neural network algorithm for graph coloring

The graph coloring problem is amongst the most difficult ones in combinatorial optimization, with a diverse set of significant applications in science and industry. Previous neural network attempts at coloring graphs have not worked well. In particular, they do not scale up to large graphs. Furthermore, experimental evaluations on real-world graphs have been lacking, and so have comparisons with state of the art conventional algorithms. In this paper we address all of these issues. We develop an improved neural network algorithm for graph coloring that scales well with graph size. The algorithm employs multiple restarts, and adaptively reduces the network's size from restart as it learns bettwe ways to color a given graph. Hence it gets faster and leaner as it evolves. We evaluate this algorithm on a structurally diverse set of graphs that arise in different applications. We compare its performance with that of a state of the art conventional algorithm on identical graphs. The conventional algorithm works better overall, though ours is not far behind. Ours works better on some graphs. The inherent parallel and distributed nature of our algorithm, especially within a neural network architecture, is a potential advantage for implementation and speed up.