Parameter estimation of ordinary differential equations

This paper addresses the development of a new algorithm forparameter estimation of ordinary differential equations. Here,we show that (1) the simultaneous approach combined with orthogonalcyclic reduction can be used to reduce the estimation problemto an optimization problem subject to a fixed number of equalityconstraints without the need for structural information to devisea stable embedding in the case of non-trivial dichotomy and(2) the Newton approximation of the Hessian information of theLagrangian function of the estimation problem should be usedin cases where hypothesized models are incorrect or only a limitedamount of sample data is available. A new algorithm is proposedwhich includes the use of the sequential quadratic programming(SQP) Gauss–Newton approximation but also encompassesthe SQP Newton approximation along with tests of when to usethis approximation. This composite approach relaxes the restrictionson the SQP Gauss–Newton approximation that the hypothesizedmodel should be correct and the sample data set large enough.This new algorithm has been tested on two standard problems.     

Sequential Systems of Linear Equations Algorithm for Nonlinear Optimization Problems with General Constraints

In Ref. 1, a new superlinearly convergent algorithm of sequential systems of linear equations (SSLE) for nonlinear optimization problems with inequality constraints was proposed. At each iteration, this new algorithm only needs to solve four systems of linear equations having the same coefficient matrix, which is much less than the amount of computation required for existing SQP algorithms. Moreover, unlike the quadratic programming subproblems of the SQP algorithms (which may not have a solution), the subproblems of the SSLE algorithm are always solvable. In Ref. 2, it is shown that the new algorithm can also be used to deal with nonlinear optimization problems having both equality and inequality constraints, by solving an auxiliary problem. But the algorithm of Ref. 2 has to perform a pivoting operation to adjust the penalty parameter per iteration. In this paper, we improve the work of Ref. 2 and present a new algorithm of sequential systems of linear equations for general nonlinear optimization problems. This new algorithm preserves the advantages of the SSLE algorithms, while at the same time overcoming the aforementioned shortcomings. Some numerical results are also reported.     

SEQUENTIAL SYSTEMS OF LINEAR EQUATIONS ALGORITHM FOR NONLINEAR OPTIMIZATION PROBLEMS-INEQUALITY CONSTRAINED PROBLEMS

AbstractIn this paper, a new superlinearly convergent algorithm of sequential systems of linear equations (SSLE) for nonlinear optimization problems with inequality constraints is proposed. Since the new algorithm only needs to solve several systems of linear equations having a same coefficient matrix per iteration, the computation amount of the algorithm is much less than that of the existing SQP algorithms per iteration. Moreover, for the SQP type algorithms, there exist so-called inconsistent problems, i.e., quadratic programming subproblems of the SQP algorithms may not have a solution at some iterations, but this phenomenon will not occur with the SSLE algorithms because the related systems of linear equations always have solutions. Some numerical results are reported.     

A globally convergent trust region algorithm for optimization with general constraints and simple bounds

1.IntroductionInthispaper,weconsiderthefollowingnonlinearprogr~ngproblemwherec(x)=(c,(x),c2(2),',We(.))',i(x)andci(x)(i=1,2,',m)arerealfunctions*ThisworkissupPOrtedbytheNationalNaturalScienceFOundationofChinaandtheManagement,DecisionandinformationSystemLab,theChineseAcademyofSciences.definedinD={xEReIISx5u}.Weassumethath相似文献   

基于BP神经网络的动态稳健参数设计

传统的动态稳健参数设计方法(田口方法)虽然在工业生产实践中展现了极大的方便,但是其本身也存在较大的改进空间.当调节变量不存在时,传统的田口方法难以实现;此外,田口方法只能根据所选取的参数水平得到最优参数组合,而这种所谓的最优结果有时并不符合实际的需要.首先构建BP神经网络模型,利用训练后的BP神经网络获得参数设计中质量特性、噪声因子以及各参数间的动态关系;然后,利用超拉丁方抽样,计算信号与特性参数间的斜率,并由此将动态稳健参数设计的寻优问题转化为相应的非线性规划问题;最后,利用次序二次规划(SQP)算法解决并优化动态稳健参数的设计。此外,我们选取了一个简单的数据案例对本文提出的方法的有效性进行了说明.     

Numerical Methods for Parameter Estimation Using Krylov-Type Techniques

In the paper we consider constrained nonlinear parameter estimation problems. The method of choice to solve such problems is the generalized Gauss-Newton method. At each iteration of the Gauss-Newton we solve the linearized parameter estimation problem and compute covariance matrix, necessary for the error assessment of the estimates, using an iterative linear algebra technique, namely LSQR algorithm. (© 2011 Wiley-VCH Verlag GmbH & Co. KGaA, Weinheim)     

修正的SQP型并行变量分配算法

Ferris 和Mangasarian 提出求解最优化问题的PVD(并行变量分配)算法, 此算法是把变量分为主要变量和辅助变量, 分配到p个处理机上, 每个处理机除了负责更新本处理机的主要变量外, 同时还沿着给定的方向更新辅助变量, 使算法的鲁棒性和灵活性得到了很大的提高. 该文基于文献[6]提出一种修正的SQP型PVD算法, 构造其搜索方向是下降方向和可行方向的组合, 并对此方向给予一个高阶修正, 使此算法很好地防止 Maratos 效应发生, 而且能够克服在求解子问题时出现约束不相容的情况. 在合适的条件下, 推导出此算法具有全局收敛性.     

Parameter optimization and uncertainty analysis in a model of oceanic CO2 uptake using a hybrid algorithm and algorithmic differentiation

Methods and results for parameter optimization and uncertainty analysis for a one-dimensional marine biogeochemical model of NPZD type are presented. The model, developed by Schartau and Oschlies, simulates the distribution of nitrogen, phytoplankton, zooplankton and detritus in a water column and is driven by ocean circulation data. Our aim is to identify parameters and fit the model output to given observational data. For this model, it has been shown that a satisfactory fit could not be obtained, and that parameters with comparable fits can vary significantly. Since these results were obtained by evolutionary algorithms (EA), we used a wider range of optimization methods: A special type of EA (called quantum-EA) with coordinate line search and a quasi-Newton SQP method, where exact gradients were generated by Automatic/Algorithmic Differentiation. Both methods are parallelized and can be viewed as instances of a hybrid, mixed evolutionary and deterministic optimization algorithm that we present in detail. This algorithm provides a flexible and robust tool for parameter identification and model validation. We show how the obtained parameters depend on data sparsity and given data error. We present an uncertainty analysis of the optimized parameters w.r.t. Gaussian perturbed data. We show that the model is well suited for parameter identification if the data are attainable. On the other hand, the result that it cannot be fitted to the real observational data without extension or modification, is confirmed.     

Superlinear Convergence of a Smooth Approximation Method for Mathematical Programs with Nonlinear Complementarity Constraints

<正>Mathematical programs with complementarity constraints(MPCC) is an important subclass of MPEC.It is a natural way to solve MPCC by constructing a suitable approximation of the primal problem.In this paper,we propose a new smoothing method for MPCC by using the aggregation technique.A new SQP algorithm for solving the MPCC problem is presented.At each iteration,the master direction is computed by solving a quadratic program,and the revised direction for avoiding the Maratos effect is generated by an explicit formula.As the non-degeneracy condition holds and the smoothing parameter tends to zero,the proposed SQP algorithm converges globally to an S-stationary point of the MPEC problem,its convergence rate is superlinear.Some preliminary numerical results are reported.     

Approximate Gauss-Newton Method for Robust Parameter Estimation

We consider robust parameter estimation problems in which either the l1 norm or the Huber function of a measurement error vector used as cost functionals. In order to avoid high computational effort of computing exact derivatives needed for the solution of these problems with the Gauss-Newton method, we suggest to use approximations of the derivatives in the occurring linearized subproblems. We show how the error introduced by using only approximated derivatives can be compensated by adding a correction term to the objective function of the linearized problems. (© 2010 Wiley-VCH Verlag GmbH & Co. KGaA, Weinheim)     

Sequential Quadratic Programming Methods for Large-Scale Problems

Sequential quadratic (SQP) programming methodsare the method of choice when solving small or medium-sized problems. Sincethey are complex methods they are difficult (but not impossible) to adapt tosolve large-scale problems. We start by discussing the difficulties that needto be addressed and then describe some general ideas that may be used toresolve these difficulties. A number of SQP codes have been written to solve specific applications and there is a general purposed SQP code called SNOPT,which is intended for general applications of a particular type. These aredescribed briefly together with the ideas on which they are based. Finally wediscuss new work on developing SQP methods using explicit second derivatives.     

隐互补约束优化问题的磨光SQP算法

本文提出了一类隐互补约束优化问题的磨光SQP算法.首先,我们给出了这类优化问题的最优性和约束规范性条件.然后,在适当假设条件下,我们证明了算法具有全局收敛性.     

Biased Online Parameter Inference for State-Space Models

We consider Bayesian online static parameter estimation for state-space models. This is a very important problem, but is very computationally challenging as the state-of-the art methods that are exact, often have a computational cost that grows with the time parameter; perhaps the most successful algorithm is that of SM C² (Chopin et al., J R Stat Soc B 75: 397–426 2013). We present a version of the SM C² algorithm which has computational cost that does not grow with the time parameter. In addition, under assumptions, the algorithm is shown to provide consistent estimates of expectations w.r.t. the posterior. However, the cost to achieve this consistency can be exponential in the dimension of the parameter space; if this exponential cost is avoided, typically the algorithm is biased. The bias is investigated from a theoretical perspective and, under assumptions, we find that the bias does not accumulate as the time parameter grows. The algorithm is implemented on several Bayesian statistical models.     

Parameter estimation for Hammerstein CARARMA systems based on the Newton iteration

The Newton iteration is basic for solving nonlinear optimization problems and studying parameter estimation algorithms. In this letter, a maximum likelihood estimation algorithm is developed for estimating the parameters of Hammerstein nonlinear controlled autoregressive autoregressive moving average (CARARMA) systems by using the Newton iteration. A simulation example is provided to show the effectiveness of the proposed algorithm.     

Identification de la Non-Linearité D'Une équation Parabolique Quasilineaire

In this work, we study the problem of the determination of the non-linearity in a parabolic equation from measurements over its solution. This corresponds to an usual physical situation, in which the parameter depends on the state of the system (for instance, the heat conduction coefficient depends on the temperature). We first state the problem as a control problem, for which we show the existence of a solution, and we calculate the derivative of the criterion with respect to the nonlinearity without any assumption on the algebraic form of this latter. We finally give a numerical application of the algorithm, which shows that the situation which is here taken in account (estimation of a parameter function depending on the state) is much better, from the identification point of view, than the usually studied situation for distributed system (estimation of a parameter function depending on space and/or time variables).     

一般约束优化基于识别函数的模松弛算法

借助于半罚函数和产生工作集的识别函数以及模松弛SQP算法思想, 本文建立了求解带等式及不等式约束优化的一个新算法. 每次迭代中, 算法的搜索方向由一个简化的二次规划子问题及一个简化的线性方程组产生. 算法在不包含严格互补性的温和条件下具有全局收敛性和超线性收敛性. 最后给出了算法初步的数值试验报告.     

A comparative study of SQP-type algorithms for nonlinear and nonconvex mixed-integer optimization

We present numerical results of a comparative study of codes for nonlinear and nonconvex mixed-integer optimization. The underlying algorithms are based on sequential quadratic programming (SQP) with stabilization by trust-regions, linear outer approximations, and branch-and-bound techniques. The mixed-integer quadratic programming subproblems are solved by a branch-and-cut algorithm. Second order information is updated by a quasi-Newton update formula (BFGS) applied to the Lagrange function for continuous, but also for integer variables. We do not require that the model functions can be evaluated at fractional values of the integer variables. Thus, partial derivatives with respect to integer variables are replaced by descent directions obtained from function values at neighboring grid points, and the number of simulations or function evaluations, respectively, is our main performance criterion to measure the efficiency of a code. Numerical results are presented for a set of 100 academic mixed-integer test problems. Since not all of our test examples are convex, we reach the best-known solutions in about 90 % of the test runs, but at least feasible solutions in the other cases. The average number of function evaluations of the new mixed-integer SQP code is between 240 and 500 including those needed for one- or two-sided approximations of partial derivatives or descent directions, respectively. In addition, we present numerical results for a set of 55 test problems with some practical background in petroleum engineering.     

双边定时截尾样本下广义逆指数分布形状参数的估计

先给出了广义逆指数分布在双边定时截尾样本下形状参数的最大似然估计,并不能得到估计的显式表达式,但证明了参数在(0,+∞)上最大似然估计是唯一存在的.其次提出用EM算法求出形状参数的估计且该估计具有良好的收敛性,还给出了形状参数的EM估计的渐近方差和近似置信区间;最后通过数值模拟,对形状参数的最大似然估计和EM估计的效果进行了比较,说明了用EM算法求形状参数的估计是可行的,并且模拟效果相对比较好.     

A Superlinearly Convergent Implicit Smooth SQP Algorithm for Mathematical Programs with Nonlinear Complementarity Constraints

This paper discusses a special class of mathematical programs with nonlinear complementarity constraints, its goal is to present a globally and superlinearly convergent algorithm for the discussed problems. We first reformulate the complementarity constraints as a standard nonlinear equality and inequality constraints by making use of a class of generalized smoothing complementarity functions, then present a new SQP algorithm for the discussed problems. At each iteration, with the help of a pivoting operation, a master search direction is yielded by solving a quadratic program, and a correction search direction for avoiding the Maratos effect is generated by an explicit formula. Under suitable assumptions, without the strict complementarity on the upper-level inequality constraints, the proposed algorithm converges globally to a B-stationary point of the problems, and its convergence rate is superlinear.AMS Subject Classification: 90C, 49MThis work was supported by the National Natural Science Foundation (10261001) and the Guangxi Province Science Foundation (0236001, 0249003) of China.     

A type of efficient feasible SQP algorithms for inequality constrained optimization

In this paper, motivated by Zhu et al. methods [Z.B. Zhu, K.C. Zhang, J.B. Jian, An improved SQP algorithm for inequality constrained optimization, Math. Meth. Oper. Res. 58 (2003) 271-282; Zhibin Zhu, Jinbao Jian, An efficient feasible SQP algorithm for inequality constrained optimization, Nonlinear Anal. Real World Appl. 10(2) (2009) 1220-1228], we propose a type of efficient feasible SQP algorithms to solve nonlinear inequality constrained optimization problems. By solving only one QP subproblem with a subset of the constraints estimated as active, a class of revised feasible descent directions are generated per single iteration. These methods are implementable and globally convergent. We also prove that the algorithms have superlinear convergence rate under some mild conditions.