A gradient projection-multiplier method for nonlinear programming

This paper describes a gradient projection-multiplier method for solving the general nonlinear programming problem. The algorithm poses a sequence of unconstrained optimization problems which are solved using a new projection-like formula to define the search directions. The unconstrained minimization of the augmented objective function determines points where the gradient of the Lagrangian function is zero. Points satisfying the constraints are located by applying an unconstrained algorithm to a penalty function. New estimates of the Lagrange multipliers and basis constraints are made at points satisfying either a Lagrangian condition or a constraint satisfaction condition. The penalty weight is increased only to prevent cycling. The numerical effectiveness of the algorithm is demonstrated on a set of test problems.The author gratefully acknowledges the helpful suggestions of W. H. Ailor, J. L. Searcy, and D. A. Schermerhorn during the preparation of this paper. The author would also like to thank D. M. Himmelblau for supplying a number of interesting test problems.     

A quadratic programming algorithm using conjugate search directions

A quadratic programming algorithm is presented, resembling Beale's 1955 quadratic programming algorithm and Wolfe's Reduced Gradient method. It uses conjugate search directions. The algorithm is conceived as being particularly appropriate for problems with a large Hessian matrix. An experimental computer program has been written to validate the concepts, and has performed adequately, although it has not been used on very large problems. An outline of the solution to the quadratic capacity-constrained transportation problem using the above method is also presented.While engaged in this research the author had a part-time post with the Manpower Services Commission.     

A fast algorithm for image registration

An algorithm is presented here to estimate a smooth motion at a high frame rate. It is derived from the non-linear constant brightness assumption. A hierarchical approach reduces the dimension of the space of admissible displacements, hence the number of unknown parameters is small compared to the size of the data. The optimal displacement is estimated by a Gauss–Newton method, and the matrix required at each step is assembled rapidly using a finite-element method. To cite this article: J. Fehrenbach, M. Masmoudi, C. R. Acad. Sci. Paris, Ser. I 346 (2008).     

On variable-metric algorithms

In this paper, a generalized variable-metric algorithm is presented. It is shown that this algorithm attains the minimum of a positive-definite, quadratic function in a finite number of steps and that thevariable-metric matrix tends to the inverse of the Hessian matrix. Most known variable-metric algorithms can be derived from this generalized algorithm, and some new algorithms are also obtained.The author expresses his gratitude to Professor H. Tokumaru for guidance and encouragement.     

A curvilinear optimization method based upon iterative estimation of the eigensystem of the Hessian matrix

An algorithm was recently presented that minimizes a nonlinear function in several variables using a Newton-type curvilinear search path. In order to determine this curvilinear search path the eigenvalue problem of the Hessian matrix of the objective function has to be solved at each iteration of the algorithm. In this paper an iterative procedure requiring gradient information only is developed for the approximation of the eigensystem of the Hessian matrix. It is shown that for a quadratic function the approximated eigenvalues and eigenvectors tend rapidly to the actual eigenvalues and eigenvectors of its Hessian matrix. The numerical tests indicate that the resulting algorithm is very fast and stable. Moreover, the fact that some approximations to the eigenvectors of the Hessian matrix are available is used to get past saddle points and accelerate the rate of convergence on flat functions.     

边界约束二次规划问题的分解方法

１．引言本文考虑如下边界约束的二次规划问题：其中ＱＥ＊"""是对称的,Ｃ,人。Ｅ＊"是给定的常数向量,且Ｚ＜。这类问题经常出现在偏微分方程,离散化的连续时间最优控制问题、线性约束的最小二乘问题、工程设计、或作为非线性规划方法中的序列子问题．因此具有特殊的重要性．本文提出求解问题（１．１）的分解方法．它类似求解线性代数方程组的选代法,它是对Ｑ进行正则分裂【对即把Ｑ分裂为两个矩阵之和,Ｑ＝Ｎ十片而这两个矩阵之差（Ｎ一则是对称正定的．在每次迭代中用一个易于求解的矩阵Ｎ替代Ｑ进行计算一新的二次规划问题．在适…     

The Global Convergence of Self-Scaling BFGS Algorithm with Nonmonotone Line Search for Unconstrained Nonconvex Optimization Problems

The self-scaling quasi-Newton method solves an unconstrained optimization problem by scaling the Hessian approximation matrix before it is updated at each iteration to avoid the possible large eigenvalues in the Hessian approximation matrices of the objective function. It has been proved in the literature that this method has the global and superlinear convergence when the objective function is convex （or even uniformly convex）. We propose to solve unconstrained nonconvex optimization problems by a self-scaling BFGS algorithm with nonmonotone linear search. Nonmonotone line search has been recognized in numerical practices as a competitive approach for solving large-scale nonlinear problems. We consider two different nonmonotone line search forms and study the global convergence of these nonmonotone self-scale BFGS algorithms. We prove that, under some weaker condition than that in the literature, both forms of the self-scaling BFGS algorithm are globally convergent for unconstrained nonconvex optimization problems.     

COMPUTING TRACE OF FUNCTION OF MATRIX

1 IntroductionIn some applications such as computational phySics, one often computes det~inant Ofmatrix and trace Of function of matrix. For ~ fun~ such as f(x) ~1/x or f(x) In (x) computing tr(f(A) ),i. e. tr(A--' ) or In(det(A) ) respeCtively, may be highly sensitiveproblems. When the matrix she n is small, we can compute these problemS explicits by usaldense ~x computation methods L6J. General speaking, such methods require O(n3) floating point OPerations. However, when n atomes larg…     

Convex functions on Grassmannian manifolds and Lawson-Osserman problem

We derive estimates of the Hessian of two smooth functions defined on Grassmannian manifold. Based on it, we can derive curvature estimates for minimal submanifolds in Euclidean space via Gauss map as in [Y.L. Xin, Ling Yang, Curvature estimates for minimal submanifolds of higher codimension, arXiv: 0709.3686; 24]. In this way, the result for Bernstein type theorem done by Jost and the first author could be improved.     

A class of generalized variable penalty methods for nonlinear programming

A class of generalized variable penalty formulations for solving nonlinear programming problems is presented. The method poses a sequence of unconstrained optimization problems with mechanisms to control the quality of the approximation for the Hessian matrix, which is expressed in terms of the constraint functions and their first derivatives. The unconstrained problems are solved using a modified Newton's algorithm. The method is particularly applicable to solution techniques where an approximate analysis step has to be used (e.g., constraint approximations, etc.), which often results in the violation of the constraints. The generalized penalty formulation contains two floating parameters, which are used to meet the penalty requirements and to control the errors in the approximation of the Hessian matrix. A third parameter is used to vary the class of standard barrier or quasibarrier functions, forming a branch of the variable penalty formulation. Several possibilities for choosing such floating parameters are discussed. The numerical effectiveness of this algorithm is demonstrated on a relatively large set of test examples.The author is thankful for the constructive suggestions of the referees.     

求解一般约束优化问题的一个全局收敛的混合不精确SQP算法

对于一般约束优化问题,本文通过一种特殊的耦合策略,把一个局邵超线性收敛的不精确SQP算法与广义梯度投影法相结合,从而给出了一个混合算法.该算法无需计算拉格朗日函数的海色矩阵,并且在适当的假设下,算法具有全局和局部超线性收敛性.     

Stability of Huang's update for the conjugate gradient method

The conditions under which Huang's conjugate gradient method generates descent directions are given and discussed. Bounds for the condition number of the inverse Hessian matrix are estimated for the case of a symmetric update.The author is much indebted to Professor C. G. Broyden of the Essex University Computing Center, for valuable advice and criticism; he is also grateful to Drs. J. Greenstadt and A. R. Gourlay for having sent copies of their unpublished papers.     

基于非单调线搜索的对角二阶拟牛顿法

在二阶拟牛顿方程的基础上,结合Zhang H.C.提出的非单调线搜索构造了一种求解大规模无约束优化问题的对角二阶拟牛顿算法.算法在每次迭代中利用对角矩阵逼近Hessian矩阵的逆,使计算搜索方向的存储量和工作量明显减少,为大型无约束优化问题的求解提供了新的思路.在通常的假设条件下,证明了算法的全局收敛性和超线性收敛性.数值实验表明算法是有效可行的.     

An incomplete Hessian Newton minimization method and its application in a chemical database problem

To efficiently solve a large scale unconstrained minimization problem with a dense Hessian matrix, this paper proposes to use an incomplete Hessian matrix to define a new modified Newton method, called the incomplete Hessian Newton method (IHN). A theoretical analysis shows that IHN is convergent globally, and has a linear rate of convergence with a properly selected symmetric, positive definite incomplete Hessian matrix. It also shows that the Wolfe conditions hold in IHN with a line search step length of one. As an important application, an effective IHN and a modified IHN, called the truncated-IHN method (T-IHN), are constructed for solving a large scale chemical database optimal projection mapping problem. T-IHN is shown to work well even with indefinite incomplete Hessian matrices. Numerical results confirm the theoretical results of IHN, and demonstrate the promising potential of T-IHN as an efficient minimization algorithm.     

A parameterized hessian quadratic programming problem

We present a general active set algorithm for the solution of a convex quadratic programming problem having a parametrized Hessian matrix. The parametric Hessian matrix is a positive semidefinite Hessian matrix plus a real parameter multiplying a symmetric matrix of rank one or two. The algorithm solves the problem for all parameter values in the open interval upon which the parametric Hessian is positive semidefinite. The algorithm is general in that any of several existing quadratic programming algorithms can be extended in a straightforward manner for the solution of the parametric Hessian problem.This research was supported by the Natural Sciences and Engineering Research Council under Grant No. A8189 and under a Postgraduate Scholarship, by an Ontario Graduate Scholarship, and by the University of Windsor Research Board under Grant No. 9432.     

A parameterized hessian quadratic programming problem

We present a general active set algorithm for the solution of a convex quadratic programming problem having a parametrized Hessian matrix. The parametric Hessian matrix is a positive semidefinite Hessian matrix plus a real parameter multiplying a symmetric matrix of rank one or two. The algorithm solves the problem for all parameter values in the open interval upon which the parametric Hessian is positive semidefinite. The algorithm is general in that any of several existing quadratic programming algorithms can be extended in a straightforward manner for the solution of the parametric Hessian problem. This research was supported by the Natural Sciences and Engineering Research Council under Grant No. A8189 and under a Postgraduate Scholarship, by an Ontario Graduate Scholarship, and by the University of Windsor Research Board under Grant No. 9432.     

WEIERSTRASS REPRESENTATION FORSURFACES WITH PRESCRIBED NORMALGAUSS MAP AND GAUSS CURVATURE IN H~3

The author obtains a Weierstrass representation for surfaces with prescribed normal Gauss map and Gauss curvature in H3. A differential equation about the hyperbolic Gauss map is also obtained, which characterizes the relation among the hyperbolic Gauss map, the normal Gauss map and Gauss curvature. The author discusses the harmonicity of the normal Gauss map and the hyperbolic Gauss map from surface with constant Gauss curvature in H3 to S2 with certain altered conformal metric. Finally, the author considers the surface whose normal Gauss map is conformal and derives a completely nonlinear differential equation of second order which graph must satisfy.     

A pseudo Newton-Raphson method for function minimization

A pseudo Newton-Raphson algorithm for function minimization is presented. As in all such algorithms, an estimate of the inverse Hessian is calculated. In this case, the estimate is of the formXZX ^T , whereZ is a diagonal matrix; and this feature permits the use of the simple procedures to maintain the positive definiteness ofZ, and hence of the restriction ofXZX ^T to the range ofX. The algorithm is shown to have finite convergence for quadratic functions and asymptotic convergence for a fairly general class of functions. Some numerical results are presented, and the extension of the algorithm to deal with linear equality and inequality constraints is briefly discussed.R. Mamen acknowledges with gratitude the financial support afforded by an Athlone Fellowship and a National Research Council of Canada Post-Graduate Bursary. Dr. S. C. Chuang made useful comments on some of the proofs. Some of the results are closely related to those of Allwright (Ref. 1).     

Identification of a Discontinuous Parameter in Stochastic Parabolic Systems

The purpose of this paper is to study the identification problem for a spatially varying discontinuous parameter in stochastic diffusion equations. The consistency property of the maximum likelihood estimate (M.L.E.) and a generating algorithm for M.L.E. have been explored under the condition that the unknown parameter is in a sufficiently regular space with respect to spatial variables. In order to prove the consistency property of the M.L.E. for a discontinuous diffusion coefficient, we use the method of sieves, i.e., first the admissible class of unknown parameters is projected into a finite-dimensional space and next the convergence of the derived finite-dimensional M.L.E. to the infinite-dimensional M.L.E. is justified under some conditions. An iterative algorithm for generating the M.L.E. is also proposed with two numerical examples. Accepted 2 April 1996     

解一类Hessian矩阵亏秩的修正BFGs算法及其局部Q-超线性收敛性

本文对凸函数在极值点的Hessian矩阵是秩亏一的情况下,给出了一类求解无约束优化问题的修正BFGS算法.算法的思想是对凸函数加上一个修正项,得到一个等价的模型,然后简化此模型得到一个修正的BFGS算法.文中证明了该算法是一个具有超线性收敛的算法,并且把修正的BFGS算法同Tensor方法进行了数值比较,证明了该算法对求解秩亏一的无约束优化问题更有效.