Local convergence of the diagonalized method of multipliers

In this study, we consider a modification of the method of multipliers of Hestenes and Powell in which the iteration is diagonalized, that is, only a fixed finite number of iterations of Newton's method are taken in the primal minimization stage. Conditions are obtained for quadratic convergence of the standard method, and it is shown that a diagonalization where two Newton steps are taken preserves the quadratic convergence for all multipler update formulas satisfying these conditions.This work constitutes part of the author's doctoral dissertation in the Department of Mathematical Sciences, Rice University, under the direction of Professor R. A. Tapia and was supported in part by ERDA Contract No. E-(40-1)-5046.The author would like to thank Professor Richard Tapia for his comments, suggestions, and discussions on this material.     

GLOBAL COVERGENCE OF THE NON-QUASI-NEWTON METHOD FOR UNCONSTRAINED OPTIMIZATION PROBLEMS

In this paper, the non-quasi-Newton＇s family with inexact line search applied to unconstrained optimization problems is studied. A new update formula for non-quasi-Newton＇s family is proposed. It is proved that the constituted algorithm with either Wolfe-type or Armijotype line search converges globally and Q-superlinearly if the function to be minimized has Lipschitz continuous gradient.     

Strong convergence theorem by a new hybrid method for equilibrium problems and variational inequality problems

In this paper, we prove a strong convergence theorem for finding a common element of the set of solutions of an equilibrium problem and the set of the solutions of the variational inequality problem by using a new hybrid method. We obtain a new result for finding a solution of an equilibrium problem and the solutions of the variational inequality problem.     

On lagrange-kuhn-tucker multipliers for pareto optimization problems

We give a simple proof of the existence of Lagrange-Kuhn-Tucker multipliers for Pareto Multiobjective programming problems.     

A quasi-Newton strategy for the sSQP method for variational inequality and optimization problems

The quasi-Newton strategy presented in this paper preserves one of the most important features of the stabilized Sequential Quadratic Programming method, the local convergence without constraint qualifications assumptions. It is known that the primal-dual sequence converges quadratically assuming only the second-order sufficient condition. In this work, we show that if the matrices are updated by performing a minimization of a Bregman distance (which includes the classic updates), the quasi-Newton version of the method converges superlinearly without introducing further assumptions. Also, we show that even for an unbounded Lagrange multipliers set, the generated matrices satisfies a bounded deterioration property and the Dennis-Moré condition.     

New strong convergence theorem of the inertial projection and contraction method for variational inequality problems

Numerical Algorithms - In this paper, we introduce a new algorithm which combines the inertial projection and contraction method and the viscosity method for solving monotone variational inequality...     

Weak and strong convergence theorems for variational inequality problems

In this paper, we study the weak and strong convergence of two algorithms for solving Lipschitz continuous and monotone variational inequalities. The algorithms are inspired by Tseng’s extragradient method and the viscosity method with Armijo-like step size rule. The main advantages of our algorithms are that the construction of solution approximations and the proof of convergence of the algorithms are performed without the prior knowledge of the Lipschitz constant of cost operators. Finally, we provide numerical experiments to show the efficiency and advantage of the proposed algorithms.     

Strong convergence theorem by hybrid method for equilibrium problems,variational inequality problems and maximal monotone operators

We introduce an iterative scheme for finding a common element of the solution set of the equilibrium problem, the solution set of the variational inequality problem for an inverse-strongly-monotone operators and the solution set of a maximal monotone operator in a 2-uniformly convex and uniformly smooth Banach space, and then we present strong convergence theorems which generalize the results of many others.     

On a characterization of convergence for the Hestenes method of multipliers

The connection between the convergence of the Hestenes method of multipliers and the existence of augmented Lagrange multipliers for the constrained minimum problem (P): minimizef(x), subject tog(x)=0, is investigated under very general assumptions onX,f, andg.In the first part, we use the existence of augmented Lagrange multipliers as a sufficient condition for the convergence of the algorithm. In the second part, we prove that this is also a necessary condition for the convergence of the method and the boundedness of the sequence of the multiplier estimates.Further, we give very simple examples to show that the existence of augmented Lagrange multipliers is independent of smoothness condition onf andg. Finally, an application to the linear-convex problem is given.     

Local convergence of the method of pseudolinear equations for quasilinear elliptic boundary-value problems

A variant of the method of pseudolinear equations, an iterative method of solving quasilinear partial differential equations, is described for quasilinear elliptic boundary-value problems of the type -[p₁(u_x)]_x - [p₂(u_y)]_y = f on a bounded simply connected two-dimensional domain D. A theorem on local convergence in C^{2, λ}(D) of this variant, which has constant coefficients, is proved. Three other method of solving quasilinear elliptic boundary-value problems, namely. Newton's method, the Ka?anov method and a variant of the method of successive approximations that has constant coefficients, are briefly discussed. Results of a series of numerical experiments in a finite-difference setting of solving quasilinear Dirichlet problems of the above-mentioned type by the method of pseudolinear equations and these three methods are given. These results show that Newton's method converges for stronger nonlinearities than do the other methods, which, in order thereafter, are the Ka?anov method, the method of pseudolinear equations and, last, the method of successive approximations, which converges only for relatively weak nonlinearities. From fastest to slowest, the methods are: the method of successive approximations, the method of pseudolinear equations, Newton's method, the Ka?anov method.     

Strong convergence of a general iterative method for variational inequality problems and fixed point problems in Hilbert spaces

In this paper, we introduce a new iterative scheme to investigate the problem of finding a common element of the set of common fixed points of a finite family of nonexpansive mappings and the set of solutions of a variational inequality problem for a relaxed cocoercive mapping by viscosity approximate methods. Our results improve and extend the recent ones announced by Chen et al. [J.M. Chen, L.J. Zhang, T.G. Fan, Viscosity approximation methods for nonexpansive mappings and monotone mappings, doi:10.1016/j.jmaa.2006.12.088], Iiduka and Tahakshi [H. Iiduka, W. Takahashi, Strong convergence theorems for nonexpansive mappings and inverse-strongly monotone mappings, Nonlinear Anal. 61 (2005) 341–350], Yao and Yao [Y.H. Yao, J.C. Yao, On modified iterative method for nonexpansive mappings and monotone mappings, Appl. Math. Comput, doi:10.1016/j.amc.2006.08.062] and Many others.     

On the method of multipliers for mathematical programming problems

In this paper, the numerical solution of the basic problem of mathematical programming is considered. This is the problem of minimizing a functionf(x) subject to a constraint (x)=0. Here,f is a scalar,x is ann-vector, and is aq-vector, withq<n.The approach employed is based on the introduction of the augmented penalty functionW(x,,k)=f(x)+ ^T (x)+k ^T (x) (x). Here, theq-vector is an approximation to the Lagrange multiplier, and the scalark>0 is the penalty constant.Previously, the augmented penalty functionW(x, ,k) was used by Hestenes in his method of multipliers. In Hestenes' version, the method of multipliers involves cycles, in each of which the multiplier and the penalty constant are held constant. After the minimum of the augmented penalty function is achieved in any given cycle, the multiplier is updated, while the penalty constantk is held unchanged.In this paper, two modifications of the method of multipliers are presented in order to improve its convergence characteristics. The improved convergence is achieved by (i) increasing the updating frequency so that the number of iterations in a cycle is shortened to N=1 for the ordinary-gradient algorithm and the modified-quasilinearization algorithm and N=n for the conjugate-gradient algorithm, (ii) imbedding Hestenes' updating rule for the multiplier into a one-parameter family and determining the scalar parameter so that the error in the optimum condition is minimized, and (iii) updating the penalty constantk so as to cause some desirable effect in the ordinary-gradient algorithm, the conjugate-gradient algorithm, and the modified-quasilinearization algorithm. For the sake of identification, Hestenes' method of multipliers is called Method MM-1, the modification including (i) and (ii) is called Method MM-2, and the modification including (i), (ii), (iii) is called Method MM-3.Evaluation of the theory is accomplished with seven numerical examples. The first example pertains to a quadratic function subject to linear constraints. The remaining examples pertain to non-quadratic functions subject to nonlinear constraints. Each example is solved with the ordinary-gradient algorithm, the conjugate-gradient algorithm, and the modified-quasilinearization algorithm, which are employed in conjunction with Methods MM-1, MM-2, and MM-3.The numerical results show that (a) for given penalty constantk, Method MM-2 generally exhibits faster convergence than Method MM-1, (b) in both Methods MM-1 and MM-2, the number of iterations for convergence has a minimum with respect tok, and (c) the number of iterations for convergence of Method MM-3 is close to the minimum with respect tok of the number of iterations for convergence of Method MM-2. In this light, Method MM-3 has very desirable characteristics.This research was supported by the National Science Foundation, Grant No. GP-32453. The authors are indebted to Messieurs E. E. Cragg and A. Esterle for computational assistance.     

Local variational problems and conservation laws

We investigate globality properties of conserved currents associated with local variational problems admitting global Euler–Lagrange morphisms. We show that the obstruction to the existence of a global conserved current is the difference of two conceptually independent cohomology classes: one coming from using the symmetries of the Euler–Lagrange morphism and the other from the system of local Noether currents.     

Lagrange multipliers for set-valued optimization problems associated with coderivatives

In this paper we investigate a vector optimization problem (P) where objective and constraints are given by set-valued maps. We show that by mean of marginal functions and suitable scalarizing functions one can characterize certain solutions of (P) as solutions of a scalar optimization problem (SP) with single-valued objective and constraint functions. Then applying some classical or recent results in optimization theory to (SP) and using estimates of subdifferentials of marginal functions, we obtain optimality conditions for (P) expressed in terms of Lagrange or sequential Lagrange multipliers associated with various coderivatives of the set-valued data.     

Local convergence of a secant type method for solving least squares problems

Local convergence of a secant type iterative method for approximating a solution of nonlinear least squares problems is investigated in this paper. The radius of convergence is determined as well as usable error estimates. Numerical examples are also provided.     

Stabilized sequential quadratic programming for optimization and a stabilized Newton-type method for variational problems

The stabilized version of the sequential quadratic programming algorithm (sSQP) had been developed in order to achieve fast convergence despite possible degeneracy of constraints of optimization problems, when the Lagrange multipliers associated to a solution are not unique. Superlinear convergence of sSQP had been previously established under the strong second-order sufficient condition for optimality (without any constraint qualification assumptions). We prove a stronger superlinear convergence result than the above, assuming the usual second-order sufficient condition only. In addition, our analysis is carried out in the more general setting of variational problems, for which we introduce a natural extension of sSQP techniques. In the process, we also obtain a new error bound for Karush–Kuhn–Tucker systems for variational problems that holds under an appropriate second-order condition.     

On the convergence of a regularization method for variational inequalities

For variational inequalities in a finite-dimensional space, the convergence of a regularization method is examined in the case of a nonmonotone basic mapping. It is shown that a fairly general sufficient condition for the existence of solutions to the original problem also guarantees the convergence and existence of solutions to perturbed problems. Examples of applications to problems on order intervals are presented.     

Local minimizers of one-dimensional variational problems and obstacle problems

In this Note we suggest a direct approach to study local minimizers of one-dimensional variational problems. To cite this article: M.A. Sychev, C. R. Acad. Sci. Paris, Ser. I 346 (2008).     

Existence of augmented Lagrange multipliers for cone constrained optimization problems

In this paper, by using an augmented Lagrangian approach, we obtain several sufficient conditions for the existence of augmented Lagrange multipliers of a cone constrained optimization problem in Banach spaces, where the corresponding augmenting function is assumed to have a valley at zero. Furthermore, we deal with the relationship of saddle points, augmented Lagrange multipliers, and zero duality gap property between the cone constrained optimization problem and its augmented Lagrangian dual problem.     

On the convergence of the conditional gradient method in distributed optimization problems

A theorem is stated on sufficient conditions for the convergence of the conditional gradient method as applied to the optimization of a nonlinear controlled functional-operator equation in a Banach ideal space. The theory is illustrated by application to the controlled Goursat-Darboux problem.