Weak and strong convergence theorems for variational inequality problems

In this paper, we study the weak and strong convergence of two algorithms for solving Lipschitz continuous and monotone variational inequalities. The algorithms are inspired by Tseng’s extragradient method and the viscosity method with Armijo-like step size rule. The main advantages of our algorithms are that the construction of solution approximations and the proof of convergence of the algorithms are performed without the prior knowledge of the Lipschitz constant of cost operators. Finally, we provide numerical experiments to show the efficiency and advantage of the proposed algorithms.     

Global one-dimensional optimization using smooth auxiliary functions

In this paper new global optimization algorithms are proposed for solving problems where the objective function is univariate and has Lipschitzean first derivatives. To solve this problem, smooth auxiliary functions, which are adaptively improved during the course of the search, are constructed. Three new algorithms are introduced: the first used the exact a priori known Lipschitz constant for derivatives; the second, when this constant is unknown, estimates it during the course of the search and finally, the last method uses neither the exact global Lipschitz constant nor its estimate but instead adaptively estimates the local Lipschitz constants in different sectors of the search region during the course of optimization. Convergence conditions of the methods are investigated from a general viewpoint and some numerical results are also given. © 1998 The Mathematical Programming Society, Inc. Published by Elsevier Science B.V.     

New extragradient methods for solving variational inequality problems and fixed point problems

In this paper, we introduce two new iterative algorithms for finding a common element of the set of fixed points of a quasi-nonexpansive mapping and the set of solutions of the variational inequality problem with a monotone and Lipschitz continuous mapping in real Hilbert spaces, by combining a modified Tseng’s extragradient scheme with the Mann approximation method. We prove weak and strong convergence theorems for the sequences generated by these iterative algorithms. The main advantages of our algorithms are that the construction of solution approximations and the proof of convergence of the algorithms are performed without the prior knowledge of the Lipschitz constant of cost operators. Finally, we provide numerical experiments to show the efficiency and advantage of the proposed algorithms.     

Estimation of the Lipschitz constant of a function

A number of global optimisation algorithms rely on the value of the Lipschitz constant of the objective function. In this paper we present a stochastic method for estimating the Lipschitz constant. We show that the largest slope in a fixed size sample of slopes has an approximate Reverse Weibull distribution. Such a distribution is fitted to the largest slopes and the location parameter used as an estimator of the Lipschitz constant. Numerical results are presented.     

Halpern projection methods for solving pseudomonotone multivalued variational inequalities in Hilbert spaces

In this paper, we introduce new approximate projection and proximal algorithms for solving multivalued variational inequalities involving pseudomonotone and Lipschitz continuous multivalued cost mappings in a real Hilbert space. The first proposed algorithm combines the approximate projection method with the Halpern iteration technique. The second one is an extension of the Halpern projection method to variational inequalities by using proximal operators. The strongly convergent theorems are established under standard assumptions imposed on cost mappings. Finally we introduce a new and interesting example to the multivalued cost mapping, and show its pseudomontone and Lipschitz continuous properties. We also present some numerical experiments to illustrate the behavior of the proposed algorithms.

     

Optimal algorithms for global optimization in case of unknown Lipschitz constant

We consider the global optimization problem for d-variate Lipschitz functions which, in a certain sense, do not increase too slowly in a neighborhood of the global minimizer(s). On these functions, we apply optimization algorithms which use only function values. We propose two adaptive deterministic methods. The first one applies in a situation when the Lipschitz constant L is known. The second one applies if L is unknown. We show that for an optimal method, adaptiveness is necessary and that randomization (Monte Carlo) yields no further advantage. Both algorithms presented have the optimal rate of convergence.     

Acceleration Tools for Diagonal Information Global Optimization Algorithms

In this paper we face a classical global optimization problem—minimization of a multiextremal multidimensional Lipschitz function over a hyperinterval. We introduce two new diagonal global optimization algorithms unifying the power of the following three approaches: efficient univariate information global optimization methods, diagonal approach for generalizing univariate algorithms to the multidimensional case, and local tuning on the behaviour of the objective function (estimates of the local Lipschitz constants over different subregions) during the global search. Global convergence conditions of a new type are established for the diagonal information methods. The new algorithms demonstrate quite satisfactory performance in comparison with the diagonal methods using only global information about the Lipschitz constant.     

非可微分二层Lipschitz规划的广义微分和广义方向导数

本文对构成函数为Ｌｉｐｓｃｈｉｔｚ函数的二层规划问题,利用非光滑分析工具,讨论了下层极值函数和上层复合目标函数的Ｌｉｐｓｃｈｉｔｚ连续性,给出了这些函数的广义微分和广义方向导数的估计式。本文得到的结果为进一步研究非可微二层Ｌｉｐｓｃｈｉｔｚ规划的最优性条件和有效算法等理论和方法问题奠定了基础。     

Modified subgradient extragradient algorithms for solving monotone variational inequalities

ABSTRACT

In this paper, we introduce two new algorithms for solving classical variational inequalities problem with Lipschitz continuous and monotone mapping in real Hilbert space. We modify the subgradient extragradient methods with a new step size, the convergence of algorithms are established without the knowledge of the Lipschitz constant of the mapping. Finally, some numerical experiments are presented to show the efficiency and advantage of the proposed algorithms.     

The interior proximal extragradient method for solving equilibrium problems

In this article we present a new and efficient method for solving equilibrium problems on polyhedra. The method is based on an interior-quadratic proximal term which replaces the usual quadratic proximal term. This leads to an interior proximal type algorithm. Each iteration consists in a prediction step followed by a correction step as in the extragradient method. In a first algorithm each of these steps is obtained by solving an unconstrained minimization problem, while in a second algorithm the correction step is replaced by an Armijo-backtracking linesearch followed by an hyperplane projection step. We prove that our algorithms are convergent under mild assumptions: pseudomonotonicity for the two algorithms and a Lipschitz property for the first one. Finally we present some numerical experiments to illustrate the behavior of the proposed algorithms.     

Accelerations for a variety of global optimization methods

Optimization methods for a given class are easily modified to utilize additional information and work faster on a more restricted class. In particular algorithms that use only the Lipschitz constant (e.g. Mladineo, Piyavskii, Shubert and Wood) can be modified to use second derivative bounds or gradient calculations. The algorithm of Breiman & Cutler can be modified to use Lipschitz bounds. Test cases illustrating accelerations to various algorithms are provided.     

Golden ratio algorithms with new stepsize rules for variational inequalities

In this paper, we introduce two golden ratio algorithms with new stepsize rules for solving pseudomonotone and Lipschitz variational inequalities in finite dimensional Hilbert spaces. The presented stepsize rules allow the resulting algorithms to work without the prior knowledge of the Lipschitz constant of operator. The first algorithm uses a sequence of stepsizes that is previously chosen, diminishing, and nonsummable, while the stepsizes in the second one are updated at each iteration and by a simple computation. A special point is that the sequence of stepsizes generated by the second algorithm is separated from zero. The convergence and the convergence rate of the proposed algorithms are established under some standard conditions. Also, we give several numerical results to show the behavior of the algorithms in comparison with other algorithms.     

Two Interior-Point Methods for Nonlinear P *(τ)-Complementarity Problems

Two interior-point algorithms using a wide neighborhood of the central path are proposed to solve nonlinear P _*-complementarity problems. The proof of the polynomial complexity of the first method requires the problem to satisfy a scaled Lipschitz condition. When specialized to monotone complementarity problems, the results of the first method are similar to those in Ref. 1. The second method is quite different from the first in that the global convergence proof does not require the scaled Lipschitz assumption. However, at each step of this algorithm, one has to compute an approximate solution of a nonlinear system such that a certain accuracy requirement is satisfied.     

Optimization methods and stability of inclusions in Banach spaces

Our paper deals with the interrelation of optimization methods and Lipschitz stability of multifunctions in arbitrary Banach spaces. Roughly speaking, we show that linear convergence of several first order methods and Lipschitz stability mean the same. Particularly, we characterize calmness and the Aubin property by uniformly (with respect to certain starting points) linear convergence of descent methods and approximate projection methods. So we obtain, e.g., solution methods (for solving equations or variational problems) which require calmness only. The relations of these methods to several known basic algorithms are discussed, and errors in the subroutines as well as deformations of the given mappings are permitted. We also recall how such deformations are related to standard algorithms like barrier, penalty or regularization methods in optimization.     

Newton-Like Dynamics and Forward-Backward Methods for Structured Monotone Inclusions in Hilbert Spaces

In a Hilbert space setting we introduce dynamical systems, which are linked to Newton and Levenberg–Marquardt methods. They are intended to solve, by splitting methods, inclusions governed by structured monotone operators M=A+B, where A is a general maximal monotone operator, and B is monotone and locally Lipschitz continuous. Based on the Minty representation of A as a Lipschitz manifold, we show that these dynamics can be formulated as differential systems, which are relevant to the Cauchy–Lipschitz theorem, and involve separately B and the resolvents of A. In the convex subdifferential case, by using Lyapunov asymptotic analysis, we prove a descent minimizing property and weak convergence to equilibria of the trajectories. Time discretization of these dynamics gives algorithms combining Newton’s method and forward-backward methods for solving structured monotone inclusions.     

An Accelerated First-Order Method for Non-convex Optimization on Manifolds

We describe the first gradient methods on Riemannian manifolds to achieve accelerated rates in the non-convex case. Under Lipschitz assumptions on the Riemannian gradient and Hessian of the cost function, these methods find approximate first-order critical points faster than regular gradient descent. A randomized version also finds approximate second-order critical points. Both the algorithms and their analyses build extensively on existing work in the Euclidean case. The basic operation consists in running the Euclidean accelerated gradient descent method (appropriately safe-guarded against non-convexity) in the current tangent space, then moving back to the manifold and repeating. This requires lifting the cost function from the manifold to the tangent space, which can be done for example through the Riemannian exponential map. For this approach to succeed, the lifted cost function (called the pullback) must retain certain Lipschitz properties. As a contribution of independent interest, we prove precise claims to that effect, with explicit constants. Those claims are affected by the Riemannian curvature of the manifold, which in turn affects the worst-case complexity bounds for our optimization algorithms.

     

Determining zeroes of increasing Lipschitz functions

Summary We study the average case behavior of suitable algorithms to solve a nonlinear problem in numerical analysis: determining zeroes of increasing Lipschitz functions of one variable. The bisection method (which is optimal with respect to the maximal error over the whole class of functions) is far from being optimal in a more general sense: There are methods which behave like bisection in the worst case but which yield much better results on the average. We prove that the sequentially optimal algorithm found by Sukharev is also optimal in our average case setting.     

Efficient domain partitioning algorithms for global optimization of rational and Lipschitz continuous functions

A domain partitioning algorithm for minimizing or maximizing a Lipschitz continuous function is enhanced to yield two new, more efficient algorithms. The use of interval arithmetic in the case of rational functions and the estimates of Lipschitz constants valid in subsets of the domain in the case of others and the addition of local optimization have resulted in an algorithm which, in tests on standard functions, performs well.     

Parallel Asynchronous Global Search and the Nested Optimization Scheme

In this paper, a parallel asynchronous information algorithm for solving multidimensional Lipschitz global optimization problems, where times for evaluating the objective function can be different from point to point, is proposed. This method uses the nested optimization scheme and a new parallel asynchronous global optimization method for solving core univariate subproblems generated by the nested scheme. The properties of the scheme related to parallel computations are investigated. Global convergence conditions for the new method and theoretical conditions of speed up, which can be reached by using asynchronous parallelization in comparison with the pure sequential case, are established. Numerical experiments comparing sequential, synchronous, and asynchronous algorithms are also reported.     

Hybrid projection methods for equilibrium problems with non‐Lipschitz type bifunctions

Based on the extended extragradient‐like method and the linesearch technique, we propose three projection methods for finding a common solution of a finite family of equilibrium problems. The linesearch used in the proposed algorithms has allowed to reduce some conditions imposed on equilibrium bifunctions. The strongly convergent theorems are established without the Lipschitz‐type condition of bifunctions. The paper also helps in the design and analysis of practical algorithms and gives us a generalization of some previously known problems. Copyright © 2017 John Wiley & Sons, Ltd.