Comparison of Two Sets of First-Order Conditions as Bases of Interior-Point Newton Methods for Optimization with Simple Bounds

In this paper, we compare the behavior of two Newton interior-point methods derived from two different first-order necessary conditions for the same nonlinear optimization problem with simple bounds. One set of conditions was proposed by Coleman and Li; the other is the standard KKT set of conditions. We discuss a perturbation of the CL conditions for problems with one-sided bounds and the difficulties involved in extending this to problems with general bounds. We study the numerical behavior of the Newton method applied to the systems of equations associated with the unperturbed and perturbed necessary conditions. Preliminary numerical results for convex quadratic objective functions indicate that, for this class of problems, the Newton method based on the perturbed KKT formulation appears to be the more robust.     

Lanczos-type Methods for Continuation Problems

We study the Lanczos type methods for continuation problems. First we indicate how the symmetric Lanczos method may be used to solve both positive definite and indefinite linear systems. Furthermore, it can be used to monitor the simple bifurcation points on the solution curve of the eigenvalue problems. This includes computing the minimum eigenvalue, the minimum singular value, and the condition number of the partial tridiagonalizations of the coefficient matrices. The Ritz vector thus obtained can be applied to compute the tangent vector at the bifurcation point for branch-switching. Next, we indicate that the block or band Lanczos method can be used to monitor the multiple bifurcations as well as to solve the multiple right hand sides. We also show that the unsymmetric Lanczos method can be exploited to compute the minimum eigenvalue of a nearly symmetric matrix, and therefore to detect the simple bifurcation point as well. Some preconditioning techniques are discussed. Sample numerical results are reported. Our test problems include second order semilinear elliptic eigenvalue problems. © 1997 by John Wiley & Sons, Ltd.     

Equivalent Optimization Formulations and Error Bounds for Variational Inequality Problems

We investigate whether some merit functions for variational inequality problems (VIP) provide error bounds for the underlying VIP. Under the condition that the involved mapping F is strongly monotone, but not necessarily Lipschitz continuous, we prove that the so-called regularized gap function provides an error bound for the underlying VIP. We give also an example showing that the so-called D-gap function might not provide error bounds for a strongly monotone VIP.This research was supported by United College and by a direct grant of the Chinese University of Hong Kong. The authors thank the referees for helpful comments and suggestions.     

A Projected Gradient Method for Vector Optimization Problems

Vector optimization problems are a significant extension of multiobjective optimization, which has a large number of real life applications. In vector optimization the preference order is related to an arbitrary closed and convex cone, rather than the nonnegative orthant. We consider extensions of the projected gradient gradient method to vector optimization, which work directly with vector-valued functions, without using scalar-valued objectives. We provide a direction which adequately substitutes for the projected gradient, and establish results which mirror those available for the scalar-valued case, namely stationarity of the cluster points (if any) without convexity assumptions, and convergence of the full sequence generated by the algorithm to a weakly efficient optimum in the convex case, under mild assumptions. We also prove that our results still hold when the search direction is only approximately computed.     

Steepest Descent Methods with Generalized Distances for Constrained Optimization

We consider the problem s.t. , where C is a closed and covex subset of with nonempty interior, and introduce a family of interior point methods for this problem, which can be seen as approximate versions of generalized proximal point methods. Each step consists of a one-dimensional search along either a curve or a segment in the interior of C. The information about the boundary of C is contained in a generalized distance which defines the segment of the curve, and whose gradient diverges at the boundary of C. The objective of the search is either f or f plus a regularizing term. When , the usual steepest descent method is a particular case of our general scheme, and we manage to extend known convergence results for the steepest descent method to our family: for nonregularized one-dimensional searches,under a level set boundedness assumption on f, the sequence is bounded, the difference between consecutive iterates converges to 0 and every cluster point of the sequence satisfies first-order optimality conditions for the problem, i.e. is a solution if f is convex. For the regularized search and convex f, no boundedness condition on f is needed and full and global convergence of the sequence to a solution of the problem is established.     

Steepest Descent, CG, and Iterative Regularization of Ill-Posed Problems

The state of the art iterative method for solving large linear systems is the conjugate gradient (CG) algorithm. Theoretical convergence analysis suggests that CG converges more rapidly than steepest descent. This paper argues that steepest descent may be an attractive alternative to CG when solving linear systems arising from the discretization of ill-posed problems. Specifically, it is shown that, for ill-posed problems, steepest descent has a more stable convergence behavior than CG, which may be explained by the fact that the filter factors for steepest descent behave much less erratically than those for CG. Moreover, it is shown that, with proper preconditioning, the convergence rate of steepest descent is competitive with that of CG.This revised version was published online in October 2005 with corrections to the Cover Date.     

无约束优化的一个充分下降共轭梯度算法

在修正PRP共轭梯度法的基础上,提出了求解无约束优化问题的一个充分下降共轭梯度算法,证明了算法在Wolfe线搜索下全局收敛,并用数值实验表明该算法具有较好的数值结果.     

Analysis and Optimization of Stability Robustness Bounds for Discretized Systems

In this paper, robustness bounds for the perturbations of continuous-time systems to ensure the stability of their discretized counterparts are developed. Both zero-order hold and P-step matrix integrators are considered. The effect of the sampling time on the robustness bounds is studied via examples. To determine how well a simulated system will retain the robustness properties of the continuous-time system being simulated, a new criterion for the selection of the simulation method and time step is introduced. Both implicit and explicit robustness measures for sampled-data systems are obtained.     

无约束最优化的Polak—Ribiere和Hestenes—Stiefel共轭梯度法的全局收敛性

本文在很弱的条件下得到了无约束最优化的Polak-Ribiere和Hestenes-Stiefel共轭梯度法的全局收敛性的新结果,这里PR方法和HS方法中的参数β^PRk和β^HSk可以在某个负的区域内取值,这一负的区域与k有关,这些新的收敛性结果改进了文献中已有的结果。数值检验的结果表明了本文中新的PR方法和HS方法是相当有效的。     

Fast Iterative Methods for Solving of Boundary Nonlinear Integral Equations with Singularity

A very efficient and fully discrete method for numerical solution of boundary nonlinear integral equation is described. There seems a lack of rigorous numerical analysis because of singular or hypersingular behavior. In this paper, we suggest variants of methods for solving numerical solutions. Moreover, our aim has been to show how the iterations can be effectively and efficiently regularized for solving ill-posed problems by using the preconditioner. We have compared these methods with CPU time and iterations. Finally, some numerical examples show the efficiency of the proposed methods.     

“一类新共轭下降算法的全局收敛性”一文注

本文首先指出文献 [1 ]给出的一类新共轭下降算法全局收敛条件是不恰当的 ,并给予了更正 ;然后将所得结果作了进一步的推广 .     

Optimization of Stability Robustness Bounds for Linear Discrete-Time Systems

In this paper, existing stability robustness measures for the perturbation of both continuous-time and discrete-time systems are reviewed. Optimized robustness bounds for discrete-time systems are derived. These optimized bounds are obtained reducing the conservatism of existing bounds by (a) using the structural information on the perturbation and (b) changing the system coordinates via a properly chosen similarity transformation matrix. Numerical examples are used to illustrate the proposed reduced conservatism bounds.     

Two Error Bounds for Constrained Optimization Problems and Their Applications

This paper presents a global error bound for the projected gradient and a local error bound for the distance from a feasible solution to the optimal solution set of a nonlinear programming problem by using some characteristic quantities such as value function, trust region radius etc., which are appeared in the trust region method. As applications of these error bounds, we obtain sufficient conditions under which a sequence of feasible solutions converges to a stationary point or to an optimal solution, respectively, and a necessary and sufficient condition under which a sequence of feasible solutions converges to a Kuhn–Tucker point. Other applications involve finite termination of a sequence of feasible solutions. For general optimization problems, when the optimal solution set is generalized non-degenerate or gives generalized weak sharp minima, we give a necessary and sufficient condition for a sequence of feasible solutions to terminate finitely at a Kuhn–Tucker point, and a  sufficient condition which guarantees that a sequence of feasible solutions terminates finitely at a stationary point. This research was supported by the National Natural Science Foundation of China (10571106) and CityU Strategic Research Grant.     

Computation of Error Bounds for P-matrix Linear Complementarity Problems

We give new error bounds for the linear complementarity problem where the involved matrix is a P-matrix. Computation of rigorous error bounds can be turned into a P-matrix linear interval system. Moreover, for the involved matrix being an H-matrix with positive diagonals, an error bound can be found by solving a linear system of equations, which is sharper than the Mathias-Pang error bound. Preliminary numerical results show that the proposed error bound is efficient for verifying accuracy of approximate solutions. This work is partly supported by a Grant-in-Aid from Japan Society for the Promotion of Science.     

求解界约束优化问题的有效集算法综述

主要介绍了求解界约束优化问题的有效集方法,包括投影共轭梯度法和有效集识别函数法,讨论了各自的优点和不足.最后,指出了有效集法的研究趋势及应用前景.     

A Globally Convergent Lagrangian Barrier Algorithm for Optimization with General Inequality Constraints and Simple Bounds

We consider the global and local convergence properties of a class of Lagrangian barrier methods for solving nonlinear programming problems. In such methods, simple bound constraints may be treated separately from more general constraints. The objective and general constraint functions are combined in a Lagrangian barrier function. A sequence of such functions are approximately minimized within the domain defined by the simple bounds. Global convergence of the sequence of generated iterates to a first-order stationary point for the original problem is established. Furthermore, possible numerical difficulties associated with barrier function methods are avoided as it is shown that a potentially troublesome penalty parameter is bounded away from zero. This paper is a companion to previous work of ours on augmented Lagrangian methods.

     

The computation of bounds for the norm of the error in the conjugate gradient algorithm

In this paper we consider computing estimates of the norm of the error in the conjugate gradient (CG) algorithm. Formulas were given in a paper by Golub and Meurant (1997). Here, we first prove that these expressions are indeed upper and lower bounds for the A-norm of the error. Moreover, starting from these formulas, we investigate the computation of the l ₂-norm of the error. Finally, we define an adaptive algorithm where the approximations of the extreme eigenvalues that are needed to obtain upper bounds are computed when running CG leading to an improvement of the upper bounds for the norm of the error. Numerical experiments show the effectiveness of this algorithm. This revised version was published online in August 2006 with corrections to the Cover Date.     

Optimization for Inconsistent Split Feasibility Problems

The split feasibility problem deals with finding a point in a closed convex subset of the domain space of a linear operator such that the image of the point under the linear operator is in a prescribed closed convex subset of the image space. The split feasibility problem and its variants and generalizations have been widely investigated as a means for resolving practical inverse problems in various disciplines. Many iterative algorithms have been proposed for solving the problem. This article discusses a split feasibility problem which does not have a solution, referred to as an inconsistent split feasibility problem. When the closed convex set of the domain space is the absolute set and the closed convex set of the image space is the subsidiary set, it would be reasonable to formulate a compromise solution of the inconsistent split feasibility problem by using a point in the absolute set such that its image of the linear operator is closest to the subsidiary set in terms of the norm. We show that the problem of finding the compromise solution can be expressed as a convex minimization problem over the fixed point set of a nonexpansive mapping and propose an iterative algorithm, with three-term conjugate gradient directions, for solving the minimization problem.     

Global Convergence of Conjugate Gradient Methods without Line Search

Global convergence results are derived for well-known conjugate gradient methods in which the line search step is replaced by a step whose length is determined by a formula. The results include the following cases: (1) The Fletcher–Reeves method, the Hestenes–Stiefel method, and the Dai–Yuan method applied to a strongly convex LC ¹ objective function; (2) The Polak–Ribière method and the Conjugate Descent method applied to a general, not necessarily convex, LC ¹ objective function.     

一类具有上下界的均衡问题

本文获得一类具有上、下界的抽象均衡问题解的存在性条件, 回答了Isac, Sehgal和Singh提出的一个公开问题,进一步研究了具上、下界及约束条件的抽象均 衡问题,获得了更为深刻的结果. 最后,把上述公开问题推广到两个空间的情形.