A Trust Region Algorithm with Conjugate Gradient Technique for Optimization Problems

By means of a conjugate gradient strategy, we propose a trust region method for unconstrained optimization problems. The search direction is an adequate combination of the conjugate gradient direction and the trust-region direction. The global convergence and the quadratic convergence of this method are established under suitable conditions. Numerical results show that the presented method is competitive to the trust region method and the conjugate gradient method.     

Symmetric Alternating Direction Method with Indefinite Proximal Regularization for Linearly Constrained Convex Optimization

The proximal alternating direction method of multipliers is a popular and useful method for linearly constrained, separable convex problems, especially for the linearized case. In the literature, convergence of the proximal alternating direction method has been established under the assumption that the proximal regularization matrix is positive semi-definite. Recently, it was shown that the regularizing proximal term in the proximal alternating direction method of multipliers does not necessarily have to be positive semi-definite, without any additional assumptions. However, it remains unknown as to whether the indefinite setting is valid for the proximal version of the symmetric alternating direction method of multipliers. In this paper, we confirm that the symmetric alternating direction method of multipliers can also be regularized with an indefinite proximal term. We theoretically prove the global convergence of the indefinite method and establish its worst-case convergence rate in an ergodic sense. In addition, the generalized alternating direction method of multipliers proposed by Eckstein and Bertsekas is a special case in our discussion. Finally, we demonstrate the performance improvements achieved when using the indefinite proximal term through experimental results.     

Pointwise and Ergodic Convergence Rates of a Variable Metric Proximal Alternating Direction Method of Multipliers

In this paper, we obtain global pointwise and ergodic convergence rates for a variable metric proximal alternating direction method of multipliers for solving linearly constrained convex optimization problems. We first propose and study nonasymptotic convergence rates of a variable metric hybrid proximal extragradient framework for solving monotone inclusions. Then, the convergence rates for the former method are obtained essentially by showing that it falls within the latter framework. To the best of our knowledge, this is the first time that global pointwise (resp. pointwise and ergodic) convergence rates are obtained for the variable metric proximal alternating direction method of multipliers (resp. variable metric hybrid proximal extragradient framework).     

A two-step extragradient method for variational inequalities

In this paper we consider an extragradient method for solving variational inequalities and related problems. On each iteration this method makes two trial steps along the gradient, and the value of the gradient at the second point is used at the first point as the iteration direction. We prove the convergence of this method in a general case. For problems with a bilinear functional we prove the geometric convergence rate.     

Gradient trust region algorithm with limited memory BFGS update for nonsmooth convex minimization

By means of a gradient strategy, the Moreau-Yosida regularization, limited memory BFGS update, and proximal method, we propose a trust-region method for nonsmooth convex minimization. The search direction is the combination of the gradient direction and the trust-region direction. The global convergence of this method is established under suitable conditions. Numerical results show that this method is competitive to other two methods.     

信赖域算法的修订算法

为了求得非线性优化问题的最优解,必须从收敛的可能性和收敛速度入手实现有效的计算方法.为此,通过改变作为搜索方向的下降方向,并适当修订信赖范围,在信赖域算法的基础上提出了一种修订的最优化问题的求解方法.计算方法的计算程序虽然有些复杂,但从整体收敛性和计算可行性方面来说是一个有效的方法.     

Convergence of quasi-Newton method with new inexact line search

Quasi-Newton method is a well-known effective method for solving optimization problems. Since it is a line search method, which needs a line search procedure after determining a search direction at each iteration, we must decide a line search rule to choose a step size along a search direction. In this paper, we propose a new inexact line search rule for quasi-Newton method and establish some global convergent results of this method. These results are useful in designing new quasi-Newton methods. Moreover, we analyze the convergence rate of quasi-Newton method with the new line search rule.     

A nonmonotone PSB algorithm for solving unconstrained optimization

PSB (Powell-Symmetric-Broyden) algorithm is a very important algorithm and has been extensively used in trust region methods. However, there are few studies on the line search type PSB algorithm. The primary reason is that the direction generated by this class of algorithms is not necessarily a descent direction of the objective function. In this paper, by combining a nonmonotone line search technique with the PSB method, we propose a nonmonotone PSB algorithm for solving unconstrained optimization. Under proper conditions, we establish global convergence and superlinear convergence of the proposed algorithm. At the same time we verify the efficiency of the proposed algorithm by some numerical experiments.     

Local Convergence Properties of Douglas–Rachford and Alternating Direction Method of Multipliers

The Douglas–Rachford and alternating direction method of multipliers are two proximal splitting algorithms designed to minimize the sum of two proper lower semi-continuous convex functions whose proximity operators are easy to compute. The goal of this work is to understand the local linear convergence behaviour of Douglas–Rachford (resp. alternating direction method of multipliers) when the involved functions (resp. their Legendre–Fenchel conjugates) are moreover partly smooth. More precisely, when the two functions (resp. their conjugates) are partly smooth relative to their respective smooth submanifolds, we show that Douglas–Rachford (resp. alternating direction method of multipliers) (i) identifies these manifolds in finite time; (ii) enters a local linear convergence regime. When both functions are locally polyhedral, we show that the optimal convergence radius is given in terms of the cosine of the Friedrichs angle between the tangent spaces of the identified submanifolds. Under polyhedrality of both functions, we also provide conditions sufficient for finite convergence. The obtained results are illustrated by several concrete examples and supported by numerical experiments.     

A modified alternating direction method for convex minimization problems

The alternating direction method is an attractive approach for large problems. The convergence proof of the method is based on the exact solutions of the subproblems. Computing the solution of the subproblems exactly can be expensive if the number of unknowns is large. In this paper, for convex quadratic minimization problems, we propose a modified alternating direction method which can overcome the above mentioned disadvantage.     

修正乘子交替方向法求解三个可分离算子的凸优化

指出直接推广的经典乘子交替方向法对三个算子的问题不能保证收敛的原因, 并且给出将其改造成收敛算法的相应策略. 同时, 在一个统一框架下, 证明了修正的乘子交替方向法的收敛性和遍历意义下具有O(1/t)~收敛速率.     

A SPARSE SUBSPACE TRUNCATED NEWTON METHOD FOR LARGE-SCALE BOUND CONSTRAINED NONLINEAR OPTIMIZATION

In this paper we report a sparse truncated Newton algorithm for handling large-scale simple bound nonlinear constrained minimixation problem. The truncated Newton method is used to update the variables with indices outside of the active set, while the projected gradient method is used to update the active variables. At each iterative level, the search direction consists of three parts, one of which is a subspace truncated Newton direction, the other two are subspace gradient and modified gradient directions. The subspace truncated Newton direction is obtained by solving a sparse system of linear equations. The global convergence and quadratic convergence rate of the algorithm are proved and some numerical tests are given.     

无约束优化问题的对角稀疏拟牛顿法

对无约束优化问题提出了对角稀疏拟牛顿法,该算法采用了Armijo非精确线性搜索,并在每次迭代中利用对角矩阵近似拟牛顿法中的校正矩阵,使计算搜索方向的存贮量和工作量明显减少,为大型无约束优化问题的求解提供了新的思路．在通常的假设条件下,证明了算法的全局收敛性,线性收敛速度并分析了超线性收敛特征。数值实验表明算法比共轭梯度法有效,适于求解大型无约束优化问题．     

An Alternating Direction Method for Nash Equilibrium of Two-Person Games with Alternating Offers

In this paper, we propose a method for finding a Nash equilibrium of two-person games with alternating offers. The proposed method is referred to as the inexact proximal alternating direction method. In this method, the idea of alternating direction method simulates alternating offers in the game, while the inexact solutions of subproblems can be matched to the assumptions of incomplete information and bounded individual rationality in practice. The convergence of the proposed method is proved under some suitable conditions. Numerical tests show that the proposed method is competitive to the state-of-the-art algorithms.     

GLOBAL CONVERGENCE AND IMPLEMENTATION OF NGTN METHOD FOR SOWING LARGE-SCALE SMRSE NONLINEAR PROGRAMMING PROBLEMS

1. IntroductionConsider the following NLP problemwhere the function f: Re --+ RI and gi: Re - R', j E J are twice continuously dtherentiable.In particular, we discuss the cajse, where the nUmber of variables and the nUmber of constraintsin (1.1) are large and second derivatives in (1.1) are sparse.There are some methods whiCh can solve largesscale problems, e.g. Lancelot in [2] andTDSQPLM in [9]. But they can not take adVantage of sparse structtire of the problem. A newefficient meth…     

A truncated Newton method in an augmented Lagrangian framework for nonlinear programming

In this paper we propose a primal-dual algorithm for the solution of general nonlinear programming problems. The core of the method is a local algorithm which relies on a truncated procedure for the computation of a search direction, and is thus suitable for large scale problems. The truncated direction produces a sequence of points which locally converges to a KKT pair with superlinear convergence rate.     

A simple feasible SQP method for inequality constrained optimization with global and superlinear convergence

In this paper, a simple feasible SQP method for nonlinear inequality constrained optimization is presented. At each iteration, we need to solve one QP subproblem only. After solving a system of linear equations, a new feasible descent direction is designed. The Maratos effect is avoided by using a high-order corrected direction. Under some suitable conditions the global and superlinear convergence can be induced. In the end, numerical experiments show that the method in this paper is effective.     

Analysis of tensor product multigrid

We consider anisotropic second order elliptic boundary value problems in two dimensions, for which the anisotropy is exactly aligned with the coordinate axes. This includes cases where the operator features a singular perturbation in one coordinate direction, whereas its restriction to the other direction remains neatly elliptic. Most prominently, such a situation arises when polar coordinates are introduced.The common multigrid approach to such problems relies on line relaxation in the direction of the singular perturbation combined with semi-coarsening in the other direction. Taking the idea from classical Fourier analysis of multigrid, we employ eigenspace techniques to separate the coordinate directions. Thus, convergence of the multigrid method can be examined by looking at one-dimensional operators only. In a tensor product Galerkin setting, this makes it possible to confirm that the convergence rates of the multigrid V-cycle are bounded independently of the number of grid levels involved. In addition, the estimates reveal that convergence is also robust with respect to a singular perturbation in one coordinate direction.Finally, we supply numerical evidence that the algorithm performs satisfactorily in settings more general than those covered by the proof.     

带非精确线搜索的调整搜索方向DFP算法

本文介绍一类新的带调整搜索方向的Broyden算法．我们着重讨论带调整搜索方向的DFP算法的收敛性，在某些非精确线搜索的情况下，我们证明对连续可微目标函数，这算法是整体收敛的，而对一致凸目标函数，收敛速度是一步超线收敛的．从这篇文章的证明过程中，可以得到对一致凸目标函数，DFP算法具有一步超线形收敛．     

Modified descent-projection method for solving variational inequalities

In this paper, we propose a modified descent-projection method for solving variational inequalities. The method makes use of a descent direction to produce the new iterate and can be viewed as an improvement of the descent-projection method by using a new step size. Under certain conditions, the global convergence of the proposed method is proved. In order to demonstrate the efficiency of the proposed method, we provide numerical results for a traffic equilibrium problems.