Convergence to a second-order point of a trust-region algorithm with a nonmonotonic penalty parameter for constrained optimization

In a recent paper (Ref. 1), the author proposed a trust-region algorithm for solving the problem of minimizing a nonlinear function subject to a set of equality constraints. The main feature of the algorithm is that the penalty parameter in the merit function can be decreased whenever it is warranted. He studied the behavior of the penalty parameter and proved several global and local convergence results. One of these results is that there exists a subsequence of the iterates generated by the algorithm that converges to a point that satisfies the first-order necessary conditions.In the current paper, we show that, for this algorithm, there exists a subsequence of iterates that converges to a point that satisfies both the first-order and the second-order necessary conditions.This research was supported by the Rice University Center for Research on Parallel Computation, Grant R31853, and the REDI Foundation.     

Convergence bounds for nonlinear programming algorithms

Bounds on convergence are given for a general class of nonlinear programming algorithms. Methods in this class generate at each interation both constraint multipliers and approximate solutions such that, under certain specified assumptions, accumulation points of the multiplier and solution sequences satisfy the Fritz John or the Kuhn—Tucker optimality conditions. Under stronger assumptions, convergence bounds are derived for the sequences of approximate solution, multiplier and objective function values. The theory is applied to an interior—exterior penalty function algorithm modified to allow for inexact subproblem solutions. An entirely new convergence bound in terms of the square root of the penalty controlling parameter is given for this algorithm.     

熵函数法的数学理论

１．引言考虑复合函数其中ｇ;：Ｒ"－Ｒ,ｉ＝１,２,...;。连续可微．因的ｘ）的不可微性,涉及的ｘ）的优化问题,如极大极小问题Ｉｒｌｍｌｎｏｔｘｊ．ｆｉｊＺＦＲ"通常属不可微优化范畴．文山借助最大嫡原理推导出一类一致逼近一（Ｘ）的可微函数（称之为妨函数）Ｏ。ｉｌｌ--一ｉｆｌｙｌｌＸＤＣＱ．ｌｏｌｌ．ＩＪｊ容易证明ｔｉｍ人一中且对任意ｘＥＲ",ＣＭ＋ＯＯｇｂｃｌ（ｌ＞ｏｆｃ。（ｌ,ＶＣＺ＞ＣＩ＞０,（４ｉｌｌｆｆｉ０＜ｕｌ。（）ｄｉ（］＜．（５基于该性质,文山一【４］通过一次取定较大有限值Ｃ。＞０,将…     

Convergence of a Penalty Method for Mathematical Programming with Complementarity Constraints

We adapt the convergence analysis of the smoothing (Ref. 1) and regularization (Ref. 2) methods to a penalty framework for mathematical programs with complementarity constraints (MPCC); we show that the penalty framework shares convergence properties similar to those of these methods. Moreover, we give sufficient conditions for a sequence generated by the penalty framework to be attracted to a B-stationary point of the MPCC.     

Decomposition Method with a Variable Parameter for a Class of Monotone Variational Inequality Problems

In this paper, we focus on a useful modification of the decomposition method by He et al. (Ref. 1). Experience on applications has shown that the number of iterations of the original method depends significantly on the penalty parameter. The main contribution of our method is that we allow the penalty parameter to vary automatically according to some self-adaptive rules. As our numerical simulations indicate, the modified method is more flexible and efficient in practice. A detailed convergence analysis of our method is also included.     

广义乘子交替方向法求解线性约束不可分非凸优化问题的收敛性

In this paper, we consider the convergence of the generalized alternating direction method of multipliers(GADMM) for solving linearly constrained nonconvex minimization model whose objective contains coupled functions. Under the assumption that the augmented Lagrangian function satisfies the Kurdyka-Lojasiewicz inequality, we prove that the sequence generated by the GADMM converges to a critical point of the augmented Lagrangian function when the penalty parameter in the augmented Lagrangian function is sufficiently large. Moreover, we also present some sufficient conditions guaranteeing the sublinear and linear rate of convergence of the algorithm.     

An exponential penalty method for nondifferentiable minimax problems with general constraints

A well-known approach to constrained minimization is via a sequence of unconstrained optimization computations applied to a penalty function. This paper shows how it is possible to generalize Murphy's penalty method for differentiable problems of mathematical programming (Ref. 1) to solve nondifferentiable problems of finding saddle points with constraints. As in mathematical programming, it is shown that the method has the advantages of both Fiacco and McCormick exterior and interior penalty methods (Ref. 2). Under mild assumptions, the method has the desirable property that all trial solutions become feasible after a finite number of iterations. The rate of convergence is also presented. It should be noted that the results presented here have been obtained without making any use of differentiability assumptions.     

On convergent numerical algorithms for unsymmetric collocation

In this paper, we are interested in some convergent formulations for the unsymmetric collocation method or the so-called Kansa’s method. We review some newly developed theories on solvability and convergence. The rates of convergence of these variations of Kansa’s method are examined and verified in arbitrary–precision computations. Numerical examples confirm with the theories that the modified Kansa’s method converges faster than the interpolant to the solution; that is, exponential convergence for the multiquadric and Gaussian radial basis functions (RBFs). Some numerical algorithms are proposed for efficiency and accuracy in practical applications of Kansa’s method. In double–precision, even for very large RBF shape parameters, we show that the modified Kansa’s method, through a subspace selection using a greedy algorithm, can produce acceptable approximate solutions. A benchmark algorithm is used to verify the optimality of the selection process.     

A Mixed Logarithmic Barrier-Augmented Lagrangian Method for Nonlinear Optimization

We present a primal–dual algorithm for solving a constrained optimization problem. This method is based on a Newtonian method applied to a sequence of perturbed KKT systems. These systems follow from a reformulation of the initial problem under the form of a sequence of penalized problems, by introducing an augmented Lagrangian for handling the equality constraints and a log-barrier penalty for the inequalities. We detail the updating rules for monitoring the different parameters (Lagrange multiplier estimate, quadratic penalty and log-barrier parameter), in order to get strong global convergence properties. We show that one advantage of this approach is that it introduces a natural regularization of the linear system to solve at each iteration, for the solution of a problem with a rank deficient Jacobian of constraints. The numerical experiments show the good practical performances of the proposed method especially for degenerate problems.     

Nonlinear Lagrangian Functions and Applications to Semi-Infinite Programs

In this paper a nonlinear penalty method via a nonlinear Lagrangian function is introduced for semi-infinite programs. A convergence result is established which shows that the sequence of optimal values of nonlinear penalty problems converges to that of semi-infinite programs. Moreover a conceptual convergence result of a discretization method with an adaptive scheme for solving semi-infinite programs is established. Preliminary numerical experiments show that better optimal values for some nonlinear semi-infinite programs can be obtained using the nonlinear penalty method.     

Dual Convergence for Penalty Algorithms in Convex Programming

Algorithms for convex programming, based on penalty methods, can be designed to have good primal convergence properties even without uniqueness of optimal solutions. Taking primal convergence for granted, in this paper we investigate the asymptotic behavior of an appropriate dual sequence obtained directly from primal iterates. First, under mild hypotheses, which include the standard Slater condition but neither strict complementarity nor second-order conditions, we show that this dual sequence is bounded and also, each cluster point belongs to the set of Karush–Kuhn–Tucker multipliers. Then we identify a general condition on the behavior of the generated primal objective values that ensures the full convergence of the dual sequence to a specific multiplier. This dual limit depends only on the particular penalty scheme used by the algorithm. Finally, we apply this approach to prove the first general dual convergence result of this kind for penalty-proximal algorithms in a nonlinear setting.     

二阶锥线性互补问题的低阶罚函数算法

本文研究了二阶锥线性互补问题的低阶罚函数算法.利用低阶罚函数算法将二阶锥线性互补问题转化为低阶罚函数方程组,获得了低阶罚函数方程组的解序列在特定条件下以指数速度收敛于二阶锥线性互补问题解的结果,推广了二阶锥线性互补问题的幂罚函数算法.数值实验结果验证了算法的有效性.     

Superlinear Convergence of a Newton-Type Algorithm for Monotone Equations

We consider the problem of finding solutions of systems of monotone equations. The Newton-type algorithm proposed in Ref. 1 has a very nice global convergence property in that the whole sequence of iterates generated by this algorithm converges to a solution, if it exists. Superlinear convergence of this algorithm is obtained under a standard nonsingularity assumption. The nonsingularity condition implies that the problem has a unique solution; thus, for a problem with more than one solution, such a nonsingularity condition cannot hold. In this paper, we show that the superlinear convergence of this algorithm still holds under a local error-bound assumption that is weaker than the standard nonsingularity condition. The local error-bound condition may hold even for problems with nonunique solutions. As an application, we obtain a Newton algorithm with very nice global and superlinear convergence for the minimum norm solution of linear programs.This research was supported by the Singapore-MIT Alliance and the Australian Research Council.     

Quadratic convergence of the Iri-Imai algorithm for degenerate linear programming problems

A local analysis of the Iri-Imai algorithm for linear programming is given to demonstrate quadratic convergence under degeneracy. Specifically, we show that the algorithm with an exact line search either terminates after a finite number of iterations yielding a point on the set of optimal solutions or converges quadratically to one of the relative analytic centers of the faces of the set of optimal solutions including vertices. Mostly, the sequence generated falls into one of the optimal vertices, and it is rare that the sequence converges to the relative analytic center of a face whose dimension is greater than or equal to one.This paper is based on Ref. 1.The author thanks Professor Kunio Tanabe of the Institute of Statistical Mathematics for valuable comments as well as stimulating discussions.     

A Nonlinear Lagrange Algorithm for Minimax Problems with General Constraints

This article presents a novel nonlinear Lagrange algorithm for solving minimax optimization problems with both inequality and equality constraints, which eliminates the nonsmoothness of the considered problems and the numerical difficulty of the penalty method. The convergence of the proposed algorithm is analyzed under some mild assumptions, in which the sequence of the generated solutions converges locally to a Karush-Kuhn-Tucker solution at a linear rate when the penalty parameter is less than a threshold and the error bound of the solutions is also obtained. Finally, the detailed numerical results for several typical testproblems are given in order to show the performance of the proposed algorithm.     

A power penalty approach to a Nonlinear Complementarity Problem

We propose a novel power penalty approach to a Nonlinear Complementarity Problem (NCP) in which the NCP is approximated by a nonlinear equation containing a power penalty term. We show that the solution to the penalty equation converges to that of the NCP at an exponential rate when the function involved is continuous and ξ-monotone. A higher convergence rate is also obtained when the function becomes Lipschitz continuous. Numerical results are presented to confirm the theoretical findings.     

A quadratically convergent primal-dual algorithm with global convergence properties for solving optimization problems with equality constraints

This paper presents a globally convergent multiplier method which utilizes an explicit formula for the multiplier. The algorithm solves finite dimensional optimization problems with equality constraints. A unique feature of the algorithm is that it automatically calculates a value for the penalty coefficient, which, under certain assumptions, leads to global convergence.Research sponsored by the Joint Services Electronics Program, Contract F44620-71-C-0087 and the National Science Foundation, Grant GK-37672.     

Enlarging the region of convergence of Newton's method for constrained optimization

In this paper, we consider Newton's method for solving the system of necessary optimality conditions of optimization problems with equality and inequality constraints. The principal drawbacks of the method are the need for a good starting point, the inability to distinguish between local maxima and local minima, and, when inequality constraints are present, the necessity to solve a quadratic programming problem at each iteration. We show that all these drawbacks can be overcome to a great extent without sacrificing the superlinear convergence rate by making use of exact differentiable penalty functions introduced by Di Pillo and Grippo (Ref. 1). We also show that there is a close relationship between the class of penalty functions of Di Pillo and Grippo and the class of Fletcher (Ref. 2), and that the region of convergence of a variation of Newton's method can be enlarged by making use of one of Fletcher's penalty functions.This work was supported by the National Science Foundation, Grant No. ENG-79-06332.     

非线性约束优化问题的一个修正 Lagrangian 算法

基于一个含有控制参数的修正Lagrangian函数,该文建立了一个求解非线性约束优化问题的修正Lagrangian算法.在一些适当的条件下,证明了控制参数存在一个阀值,当控制参数小于这一阀值时,由这一算法产生的序列解局部收敛于问题的Kuhn-Tucker点,并且建立了解的误差上界.最后给出一些约束优化问题的数值结果.     

一个修正的罚函数方法

本文通过给出的一个修正的罚函数,把约束非线性规划问题转化为无约束非线性规划问题.我们讨论了原问题与相应的罚问题局部最优解和全局最优解之间的关系,并给出了乘子参数和罚参数与迭代点之间的关系,最后给出了一个简单算法,数值试验表明算法是有效的.