Iterative algorithms for solving fixed point problems and variational inequalities with uniformly continuous monotone operators

Using the double projection and Halpern methods, we prove two strong convergence results for finding a solution of a variational inequality problem involving uniformly continuous monotone operator which is also a fixed point of a quasi-nonexpansive mapping in a real Hilbert space. In our proposed methods, only two projections onto the feasible set in each iteration are performed, rather than one projection for each tentative step during the Armijo-type search, which represents a considerable saving especially when the projection is computationally expensive. We also give some numerical results which show that our proposed algorithms are efficient and implementable from the numerical point of view.     

不等式约束最优化超线性与二次收敛的强次可行SQP算法

利用ＳＱＰ方法、广义投影技术和强次可行方（向）法思想,建立不等式约束优化一个新的初始点任意的快速收敛算法． 算法每次迭代仅需解一个总存在可行解的二次子规划,或用广义投影计算“一阶”强次可行下降辅助搜索方向;采用曲线搜索与直线搜索相结合的方法产生步长． 在较温和的条件下,算法具有全局收敛性、强收敛性、超线性与二次收敛性． 给出了算法有效的数值试验．     

超线性与二次收敛的序列方程组可行方法

本文讨论不等式约束规划问题,给出一个线性方程组与辅助方向相结合的新可行算法,算法用一种新型的直线搜索产生步长.在一定条件下,当k充分大后,求方向dk每次只需解一个线性方程组.文中证明了算法的全局收敛性与超线性的收敛速度以及二次收敛性,并给出了方法初步的数值试验.     

有界约束半光滑方程组的非单调投影信赖域方法

投影信赖域策略结合非单调线搜索算法解有界约束非线性半光滑方程组.基于简单有界约束的非线性优化问题构建信赖域子问题,半光滑类牛顿步在可行域投影得到投影牛顿的试探步,获得新的搜索方向,结合非单调线搜索技术得到回代步,获得新的步长.在合理的条件下,证明算法不仅具有整体收敛性且保持超线性收敛速率.引入非单调技术能克服高度非线性的病态问题,加速收敛性进程,得到超线性收敛速率.     

An algorithm for linearly constrained nonlinear programming problems

In this paper an algorithm for solving a linearly constrained nonlinear programming problem is developed. Given a feasible point, a correction vector is computed by solving a least distance programming problem over a polyhedral cone defined in terms of the gradients of the “almost” binding constraints. Mukai's approximate scheme for computing the step size is generalized to handle the constraints. This scheme provides an estimate for the step size based on a quadratic approximation of the function. This estimate is used in conjunction with Armijo line search to calculate a new point. It is shown that each accumulation point is a Kuhn-Tucker point to a slight perturbation of the original problem. Furthermore, under suitable second order optimality conditions, it is shown that eventually only one trial is needed to compute the step size.     

Gradient projection and local region search for multiobjective optimisation

This paper presents a new method for multiobjective optimisation based on gradient projection and local region search. The gradient projection is conducted through the identification of normal vectors of an efficient frontier. The projection of the gradient of a nonlinear utility function onto the tangent plane of the efficient frontier at a given efficient solution leads to the definition of a feasible local region in a neighbourhood of the solution. Within this local region, a better efficient solution may be sought. To implement such a gradient-based local region search scheme, a new auxiliary problem is developed. If the utility function is given explicitly, this search scheme results in an iterative optimisation algorithm capable of general nonseparable multiobjective optimisation. Otherwise, an interactive decision making algorithm is developed where the decision maker (DM) is expected to provide local preference information in order to determine trade-off directions and step sizes. Optimality conditions for the algorithms are established and the convergence of the algorithms is proven. A multiobjective linear programming (MOLP) problem is taken for example to demonstrate this method both graphically and analytically. A nonlinear multiobjective water quality management problem is finally examined to show the potential application of the method to real world decision problems.     

关于两个非线性算子的一个分裂方法(英文)

<正>This paper generalizes a class of projection type methods for monotone variational inequalities to general monotone inclusion.It is shown that when the normal cone operator in projection is replaced by any maximal monotone operator,the resulting method inherits all attractive convergence properties of projection type methods,and allows an adjusting step size rule.Weaker convergence assumption entails an extra projection at each iteration.Moreover,this paper also addresses applications of the resulting method to convex programs and monotone variational inequalities.     

Boosting the feasibility pump

The feasibility pump (FP) has proved to be an effective method for finding feasible solutions to mixed integer programming problems. FP iterates between a rounding procedure and a projection procedure, which together provide a sequence of points alternating between LP feasible but fractional solutions, and integer but LP infeasible solutions. The process attempts to minimize the distance between consecutive iterates, producing an integer feasible solution when closing the distance between them. We investigate the benefits of enhancing the rounding procedure with a clever integer line search that efficiently explores a large set of integer points. An extensive computational study on benchmark instances demonstrates the efficacy of the proposed approach.     

Duality-based algorithms for total-variation-regularized image restoration

Image restoration models based on total variation (TV) have become popular since their introduction by Rudin, Osher, and Fatemi (ROF) in 1992. The dual formulation of this model has a quadratic objective with separable constraints, making projections onto the feasible set easy to compute. This paper proposes application of gradient projection (GP) algorithms to the dual formulation. We test variants of GP with different step length selection and line search strategies, including techniques based on the Barzilai-Borwein method. Global convergence can in some cases be proved by appealing to existing theory. We also propose a sequential quadratic programming (SQP) approach that takes account of the curvature of the boundary of the dual feasible set. Computational experiments show that the proposed approaches perform well in a wide range of applications and that some are significantly faster than previously proposed methods, particularly when only modest accuracy in the solution is required.     

A Successive Projection Algorithm for Solving the Multiple-Sets Split Feasibility Problem

The multiple-sets split feasibility problem (MSFP) arises in many areas and it can be unified as a model for many inverse problems where the constraints are required on the solutions in the domain of a linear operator as well as in the operator's range. Some existing algorithms, in order to get the suitable step size, need to compute the largest eigenvalue of the related matrix, estimate the Lipschitz constant, or use some step-size search scheme, which usually requires many inner iterations. In this article, we introduce a successive projection algorithm for solving the multiple-sets split feasibility problem. In each iteration of this algorithm, the step size is directly computed, which is not needed to compute the largest eigenvalue of the matrix or estimate the Lipschitz constant. It also does not need any step-size search scheme. Its theoretical convergence results are also given.     

A variant of forward-backward splitting method for the sum of two monotone operators with a new search strategy

In this paper, we propose variants of Forward-Backward splitting method for finding a zero of the sum of two operators. A classical modification of Forward-Backward method was proposed by Tseng, which is known to converge when the forward and the backward operators are monotone and with Lipschitz continuity of the forward operator. The conceptual algorithm proposed here improves Tseng’s method in some instances. The first and main part of our approach, contains an explicit Armijo-type search in the spirit of the extragradient-like methods for variational inequalities. During the iteration process, the search performs only one calculation of the forward-backward operator in each tentative of the step. This achieves a considerable computational saving when the forward-backward operator is computationally expensive. The second part of the scheme consists in special projection steps. The convergence analysis of the proposed scheme is given assuming monotonicity on both operators, without Lipschitz continuity assumption on the forward operator.     

A Projection Preconditioner for Solving the Implicit Immersed Boundary Equations

This paper presents a method for solving the linear semi-implicit immersed boundary equations which avoids the severe time step restriction presented by explicit-time methods. The Lagrangian variables are eliminated via a Schur complement to form a purely Eulerian saddle point system, which is preconditioned by a projection operator and then solved by a Krylov subspace method. From the viewpoint of projection methods, we derive an ideal preconditioner for the saddle point problem and compare the efficiency of a number of simpler preconditioners that approximate this perfect one. For low Reynolds number and high stiffness, one particular projection preconditioner yields an efficiency improvement of the explicit IB method by a factor around thirty. Substantial speed-ups over explicit-time method are achieved for Reynolds number below 100. This speedup increases as the Eulerian grid size and/or the Reynolds number are further reduced.     

Globally convergent DC trust-region methods

In this paper, we investigate the use of DC (Difference of Convex functions) models and algorithms in the application of trust-region methods to the solution of a class of nonlinear optimization problems where the constrained set is closed and convex (and, from a practical point of view, where projecting onto the feasible region is computationally affordable). We consider DC local models for the quadratic model of the objective function used to compute the trust-region step, and apply a primal-dual subgradient method to the solution of the corresponding trust-region subproblems. One is able to prove that the resulting scheme is globally convergent to first-order stationary points. The theory requires the use of exact second-order derivatives but, in turn, the computation of the trust-region step asks only for one projection onto the feasible region (in comparison to the calculation of the generalized Cauchy point which may require more). The numerical efficiency and robustness of the proposed new scheme when applied to bound-constrained problems is measured by comparing its performance against some of the current state-of-the-art nonlinear programming solvers on a vast collection of test problems.     

An interior multiobjective primal-dual linear programming algorithm using approximated gradients and sequential generation of anchor points

An algorithm for addressing multiple objective linear programming (MOLP) problems is presented. The algorithm modifies the path-following primal-dual algorithm to MOLP problems by using the single objective algorithm to generate interior search directions and later combine them to derive a single direction along which to step to the next iterate. Combining the different interior search directions is done by interacting with a Decision Maker (DM) to obtain locally-relevant preference information for the value vectors along these directions. This preference information is then used to derive an approximation to the gradient of an implicity-known utility function, and using a projection of this gradient provides a direction gradient of an implicitly-known utility function, and using a projection of this gradient provides a direction vector along which we step to the next iterate. At each iteration the algorithm also generates boundary points that aid in deriving the combined search direction. We refer to these boundary points, generated sequentially during the process, as anchor points that serve as candidate solutions at which to terminate the iterative process.     

Modified basic projection methods for a class of equilibrium problems

Projection methods are a popular class of methods for solving equilibrium problems. In this paper, we propose approximate one projection methods for solving a class of equilibrium problems, where the cost bifunctions are paramonotone, the feasible sets are defined by a continuous convex function inequality and not necessarily differentiable in the Euclidean space \(\mathcal R^{s}\). At each main iteration step in our algorithms, the usual projections onto the feasible set are replaced by computing inexact subgradients and one projection onto the intersection of two halfspaces containing the solution set of the equilibrium problems. Then, by choosing suitable parameters, we prove convergence of the whole generated sequence to a solution of the problems, under only the assumptions of continuity and paramonotonicity of the bifunctions. Finally, we present some computational examples to illustrate the assumptions of the proposed algorithms.     

Affine scaling inexact generalized Newton algorithm with interior backtracking technique for solving bound-constrained semismooth equations

We develop and analyze an affine scaling inexact generalized Newton algorithm in association with nonmonotone interior backtracking line technique for solving systems of semismooth equations subject to bounds on variables. By combining inexact affine scaling generalized Newton with interior backtracking line search technique, each iterate switches to inexact generalized Newton backtracking step to strict interior point feasibility. The global convergence results are developed in a very general setting of computing trial steps by the affine scaling generalized Newton-like method that is augmented by an interior backtracking line search technique projection onto the feasible set. Under some reasonable conditions we establish that close to a regular solution the inexact generalized Newton method is shown to converge locally p-order q-superlinearly. We characterize the order of local convergence based on convergence behavior of the quality of the approximate subdifferentials and indicate how to choose an inexact forcing sequence which preserves the rapid convergence of the proposed algorithm. A nonmonotonic criterion should bring about speeding up the convergence progress in some ill-conditioned cases.     

Minimizing a differentiable function over a differential manifold

To generalize the descent methods of unconstrained optimization to the constrained case, we define intrinsically the gradient field of the objective function on the constraint manifold and analyze descent methods along geodesics, including the gradient projection and reduced gradient methods for special choices of coordinate systems. In particular, we generalize the quasi-Newton methods and establish their superlinear convergence; we show that they only require the updating of a reduced size matrix. In practice, the geodesic search is approximated by a tangent step followed by a constraints restoration or by a simple arc search again followed by a restoration step.A first draft of this paper was first presented at the 10th International Symposium on Mathematical Programming, Montreal, Canada, 1979. The present version has been prepared while the author was visiting the Department of Engineering-Economic Systems at Stanford University, partially supported by AFOSR Grant No. 77-3141.     

A method to accelerate the rate of convergence of a class of optimization algorithms

Supposez ∈ E ⁿ is a solution to the optimization problem minimizeF(x) s.t.x ∈ E ⁿ and an algorithm is available which iteratively constructs a sequence of search directions {s _j } and points {x _j } with the property thatx _j →z. A method is presented to accelerate the rate of convergence of {x _j } toz provided that n consecutive search directions are linearly independent. The accelerating method uses n iterations of the underlying optimization algorithm. This is followed by a special step and then another n iterations of the underlying algorithm followed by a second special step. This pattern is then repeated. It is shown that a superlinear rate of convergence applies to the points determined by the special step. The special step which uses only first derivative information consists of the computation of a search direction and a step size. After a certain number of iterations a step size of one will always be used. The acceleration method is applied to the projection method of conjugate directions and the resulting algorithm is shown to have an (n + 1)-step cubic rate of convergence. The acceleration method is based on the work of Best and Ritter [2].     

一个改进的序列二次规划可行下降算法及其全局收敛性

利用广义投影校正技术对搜索方向进行某种修正,改进假设条件,采用一种新型的一阶修正方向并结合SQP技术,建立了求解非线性约束最优化问题(p)的一个新的SQP可行下降算法,在较温和的假设条件下证明了算法的全局收敛性.由于新算法仅需较小的存储,从而适合大规模最优化问题的计算.     

Increasing the Efficiency of Selected Grid Search Procedures Through the Introduction of a Best Step Mechanism

In this paper, several new developments are suggested in order to increase the efficiency of grid search procedures used within vertex enumeration solution algorithms based on both the classical cobweb technique (a two constraint approach) and a single constraint approach. The gains in efficiency (and robustness) are demonstrated by incorporating the developments into a grid search procedure (which is a part of a heuristic solution algorithm used to obtain solutions to a non-linear bi-level multi-period model of an aluminium smelter) and applying it to a test suite of problems. The classical cobweb technique is shown to have some serious problems when used in this application but with step size improvements suggested in the paper, is shown to be very competitive. The developments are also incorporated into a single constraint based approach which does not suffer from the cobweb specific difficulties. The improvements considered include the use of a step mechanism initially with a step size calculated {a priori} and then finally with a tailored (best) one. The latter development when applied to the cobweb approach, requires the approximation of the two constraints (and hence the intersection of their graphs) using second degree polynomials.