The numerical solution of linear ordinary differential equations by feedforward neural networks

It is demonstrated, through theory and examples, how it is possible to construct directly and noniteratively a feedforward neural network to approximate arbitrary linear ordinary differential equations. The method, using the hard limit transfer function, is linear in storage and processing time, and the L₂ norm of the network approximation error decreases quadratically with the increasing number of hidden layer neurons. The construction requires imposing certain constraints on the values of the input, bias, and output weights, and the attribution of certain roles to each of these parameters.

All results presented used the hard limit transfer function. However, the noniterative approach should also be applicable to the use of hyperbolic tangents, sigmoids, and radial basis functions.     

Non-Parametric Regression Methods

The nature of the financial time series is complex, continuous interchange of stochastic and deterministic regimes. Therefore, it is difficult to forecast with parametric techniques. Instead of parametric models, we propose three techniques and compare with each other. Neural networks and support vector regression (SVR) are two universally approximators. They are data-driven non parametric models. ARCH/GARCH models are also investigated. Our assumption is that the future value of Istanbul Stock Exchange 100 index daily return depends on the financial indicators although there is no known parametric model to explain this relationship. This relationship comes from the technical analysis. Comparison shows that the multi layer perceptron networks overperform the SVR and time series model (GARCH).     

从支持向量机到非平行支持向量机

非平行支持向量机是支持向量机的延伸,受到了广泛的关注.非平行支持向量机构造允许非平行的支撑超平面,可以描述不同类别之间的数据分布差异,从而适用于更广泛的问题.然而,对非平行支持向量机模型与支持向量机模型之间的关系研究较少,且尚未有等价于标准支持向量机模型的非平行支持向量机模型.从支持向量机出发,构造出新的非平行支持向量机模型,该模型不仅可以退化为标准支持向量机,保留了支持向量机的稀疏性和核函数可扩展性.同时,可以描述不同类别之间的数据分布差异,适用于更广泛的非平行结构数据等.最后,通过实验初步验证了所提模型的有效性.     

Linearly constrained reconstruction of functions by kernels with applications to machine learning

This paper investigates the approximation of multivariate functions from data via linear combinations of translates of a positive definite kernel from a reproducing kernel Hilbert space. If standard interpolation conditions are relaxed by Chebyshev-type constraints, one can minimize the norm of the approximant in the Hilbert space under these constraints. By standard arguments of optimization theory, the solutions will take a simple form, based on the data related to the active constraints, called support vectors in the context of machine learning. The corresponding quadratic programming problems are investigated to some extent. Using monotonicity results concerning the Hilbert space norm, iterative techniques based on small quadratic subproblems on active sets are shown to be finite, even if they drop part of their previous information and even if they are used for infinite data, e.g., in the context of online learning. Numerical experiments confirm the theoretical results. Dedicated to C.A. Micchelli at the occasion of his 60th birthday Mathematics subject classifications (2000) 65D05, 65D10, 41A15, 41A17, 41A27, 41A30, 41A40, 41A63.     

Approximation by radial bases and neural networks

In this paper, we study approximation by radial basis functions including Gaussian, multiquadric, and thin plate spline functions, and derive order of approximation under certain conditions. Moreover, neural networks are also constructed by wavelet recovery formula and wavelet frames.     

State estimation of neural networks with both time-varying delays and norm-bounded parameter uncertainties via a delay decomposition approach

This paper is concerned with the state estimation problem for neural networks with both time-varying delays and norm-bounded parameter uncertainties. By employing a delay decomposition approach and a convex combination technique, we obtain less conservative delay-dependent stability criteria to guarantee the existence of desired state estimator for the delayed neural networks. Finally, numerical examples are presented to demonstrate the effectiveness of the proposed approach.     

State estimation with guaranteed performance for switching-type fuzzy neural networks in presence of sensor nonlinearities

This paper investigates the state estimation with guaranteed performance for a class of switching fuzzy neural networks. A switching-type fuzzy neural networks (STFNNs) model is proposed which captures external disturbances, sensor nonlinearities, and mode switching phenomenon of the fuzzy neural networks without the Markovian process assumption. For such a model, a state estimation problem is formulated to achieve the guaranteed performance: the estimation error system is exponentially stable with certain decay rate and a prescribed H_∞ disturbance attenuation level. A novel sufficient condition for this problem is established using the Lyapunov functional method and the average dwell time approach, and the estimator parameters are explicitly given. A numerical example is presented to show the effectiveness of the developed results.     

基于子空间多核学习的企业财务困境预测方法

财务指标的异构性是影响企业财务困境预测精度的重要因素,现有多核学习方法能够用于解决异构数据学习问题。本文首先介绍了子空间多核学习财务困境预测理论框架,在此基础上根据子空间学习的最大化方差准则、类别可分性最大化准则、非线性子空间映射原理,提出了三种子空间多核学习方法,分别为最大化方差投影子空间多核学习、类别可分性最大化子空间多核学习、非线性子空间多核学习。利用采集的我国上市公司数据进行实验,对比所提出的方法同现有代表性财务困境预测方法,并对实验结果进行分析。实验结果表明,本文提出的子空间多核学习财务困境预测框架行之有效,该框架下所构造的子空间多核学习预测方法能够有效地提升财务困境预测精度。     

Approximation by neural networks with sigmoidal functions

In this paper, we introduce a type of approximation operators of neural networks with sigmodal functions on compact intervals, and obtain the pointwise and uniform estimates of the ap- proximation. To improve the approximation rate, we further introduce a type of combinations of neurM networks. Moreover, we show that the derivatives of functions can also be simultaneously approximated by the derivatives of the combinations. We also apply our method to construct approximation operators of neural networks with sigmodal functions on infinite intervals.     

New results on input-to-state convergence for recurrent neural networks with variable inputs

In this paper, we study a class of recurrent neural networks (RNNs) arising from optimization problems. By constructing appropriate Lyapunov functions, we prove two new results on input-to-state convergence of RNNs with variable inputs. Numerical simulations are also given to demonstrate the convergence of the solutions.     

Surrogate based sensitivity analysis of process equipment

The computational cost associated with the use of high-fidelity computational fluid dynamics (CFD) models poses a serious impediment to the successful application of formal sensitivity analysis in engineering design. Even though advances in computing hardware and parallel processing have reduced costs by orders of magnitude over the last few decades, the fidelity with which engineers desire to model engineering systems has also increased considerably. Evaluation of such high-fidelity models may take significant computational time for complex geometries.     

State estimation of memristor‐based recurrent neural networks with time‐varying delays based on passivity theory

This article deals with the state estimation problem of memristor‐based recurrent neural networks (MRNNs) with time‐varying delay based on passivity theory. The main purpose is to estimate the neuron states, through available output measurements such that for all admissible time delay, the dynamics of the estimation error is passive from the control input to the output error. Based on the Lyapunov–Krasovskii functional (LKF) involving proper triple integral terms, convex combination technique, and reciprocal convex technique, a delay‐dependent state estimation of MRNNs with time‐varying delay is established in terms of linear matrix inequalities (LMIs). The information about the neuron activation functions and lower bound of the time‐varying delays is fully used in the LKF. Then, the desired estimator gain matrix is accomplished by solving LMIs. Finally, a numerical example is provided to demonstrate the effectiveness of the proposed theoretical results. © 2013 Wiley Periodicals, Inc. Complexity 19: 32–43, 2014     

Empirical Bayes estimation for queueing systems and networks

Empirical Bayes estimators are derived for standardM/M/1 queues,M/M/1 queues with state-dependent arrival and service rates, finite capacityM/M/1 queues with state-dependent rates and for open Jackson networks. The asymptotic properties of the empirical Bayes estimators are derived both with respect to the conditional distribution of the observations given the parameters, and with respect to the joint distribution of the observations and the parameters.     

Transit passenger origin-destination estimation in congested transit networks with elastic line frequencies

This paper deals with the transit passenger origin-destination (O-D) estimation problem by using updated passenger counts in congested transit networks and outdated prior O-D matrix. A bilevel programming approach is extended for the transit passenger O-D updating problem where the upper-level problem seeks to minimize the sum of error measurements in passenger counts and O-D matrices, while the lower level is the stochastic user equilibrium assignment problem for congested transit networks. The transit assignment framework is based on a frequency-adaptive transit network model in this paper, which can help determine transit line frequencies and the network flow pattern simultaneously in congested transit networks. A heuristic solution algorithm is adapted for solving the transit passenger O-D estimation problem. Finally, a numerical example is used to illustrate the applications of the proposed model and solution algorithm. The work described in this paper was mainly supported by two research grants from the Research Grants Council of the Hong Kong Special Administrative Region (Project No. PolyU 5143/03E and PolyU 5040/02E).     

Parameter estimation of delay differential equations: An integration-free LS-SVM approach

This paper introduces an estimation method based on Least Squares Support Vector Machines (LS-SVMs) for approximating time-varying as well as constant parameters in deterministic parameter-affine delay differential equations (DDEs). The proposed method reduces the parameter estimation problem to an algebraic optimization problem. Thus, as opposed to conventional approaches, it avoids iterative simulation of the given dynamical system and therefore a significant speedup can be achieved in the parameter estimation procedure. The solution obtained by the proposed approach can be further utilized for initialization of the conventional nonconvex optimization methods for parameter estimation of DDEs. Approximate LS-SVM based models for the state and its derivative are first estimated from the observed data. These estimates are then used for estimation of the unknown parameters of the model. Numerical results are presented and discussed for demonstrating the applicability of the proposed method.     

State estimation for Markovian jumping genetic regulatory networks with random delays

In this paper, the state estimation problem is investigated for stochastic genetic regulatory networks (GRNs) with random delays and Markovian jumping parameters. The delay considered is assumed to be satisfying a certain stochastic characteristic. Meantime, the delays of GRNs are described by a binary switching sequence satisfying a conditional probability distribution. The aim of this paper is to design a state estimator to estimate the true states of the considered GRNs through the available output measurements. By using Lyapunov functional and some stochastic analysis techniques, the stability criteria of the estimation error systems are obtained in the form of linear matrix inequalities under which the estimation error dynamics is globally asymptotically stable. Then, the explicit expression of the desired estimator is shown. Finally, a numerical example is presented to show the effectiveness of the proposed results.     

Robust state estimation for Markov jump genetic regulatory networks based on passivity theory

In this article, the robust state estimation problem for Markov jump genetic regulatory networks (GRNs) based on passivity theory is investigated. Moreover, the effect of time‐varying delays is taken into account. The focus is on designing a linear state estimator to estimate the concentrations of the mRNAs and the proteins of the GRNs, such that the dynamics of the state estimation error can be stochastically stable while achieving the prescribed passivity performance. By applying the Lyapunov–Krasovskii functional method, delay‐dependent criteria are established to ensure the existence of the mode‐dependent estimator in the form of linear matrix inequalities. Based on the obtained results, the parameters of the desired estimator gains can be further calculated. Finally, a numerical example is given to illustrate the effectiveness of our proposed methods. © 2015 Wiley Periodicals, Inc. Complexity 21: 214–223, 2016     

A methodology for developing Bayesian networks: An application to information technology (IT) implementation

Bayesian Networks (BNs) are probabilistic inference engines that support reasoning under uncertainty. This article presents a methodology for building an information technology (IT) implementation BN from client–server survey data. The article also demonstrates how to use the BN to predict the attainment of IT benefits, given specific implementation characteristics (e.g., application complexity) and activities (e.g., reengineering). The BN is an outcome of a machine learning process that finds the network’s structure and its associated parameters, which best fit the data. The article will be of interest to academicians who want to learn more about building BNs from real data and practitioners who are interested in IT implementation models that make probabilistic statements about certain implementation decisions.     

Enhanced artificial bee colony for training least squares support vector machines in commodity price forecasting

The importance of optimizing machine learning control parameters has motivated researchers to investigate for proficient optimization techniques. In this study, a Swarm Intelligence approach, namely artificial bee colony (ABC) is utilized to optimize parameters of least squares support vector machines. Considering critical issues such as enriching the searching strategy and preventing over fitting, two modifications to the original ABC are introduced. By using commodities prices time series as empirical data, the proposed technique is compared against two techniques, including Back Propagation Neural Network and by Genetic Algorithm. Empirical results show the capability of the proposed technique in producing higher prediction accuracy for the prices of interested time series data.