Towards a unified recurrent neural network theory: The uniformly pseudo-projection-anti-monotone net |
| |
Authors: | Zong Ben Xu Chen Qiao |
| |
Institution: | 1.Institute for Information and System Science, Faculty of Science, MOE Key Lab for Intelligent Networks and Network Security,Xi’an Jiaotong University,Xi’an,P. R. China |
| |
Abstract: | In the past decades, various neural network models have been developed for modeling the behavior of human brain or performing
problem-solving through simulating the behavior of human brain. The recurrent neural networks are the type of neural networks
to model or simulate associative memory behavior of human being. A recurrent neural network (RNN) can be generally formalized
as a dynamic system associated with two fundamental operators: one is the nonlinear activation operator deduced from the input-output
properties of the involved neurons, and the other is the synaptic connections (a matrix) among the neurons. Through carefully
examining properties of various activation functions used, we introduce a novel type of monotone operators, the uniformly
pseudo-projectionanti-monotone (UPPAM) operators, to unify the various RNN models appeared in the literature. We develop a
unified encoding and stability theory for the UPPAM network model when the time is discrete. The established model and theory
not only unify but also jointly generalize the most known results of RNNs. The approach has lunched a visible step towards
establishment of a unified mathematical theory of recurrent neural networks. |
| |
Keywords: | Feedback neural networks essential characteristics uniformly pseudo-projection-anti- monotone net unified theory dynamics |
本文献已被 维普 SpringerLink 等数据库收录! |
|